2024-03-30T02:05:48.735 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-30T02:05:48.735 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363 --verbose --owner scheduled_rishabh@teuthology --name rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} -- /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/orig.config.yaml 2024-03-30T02:05:48.810 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T02:05:48.922 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363 branch: wip-rishabh-testing-20240327.051042-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} email: null first_in_suite: false job_id: '7629363' kernel: client: branch: testing flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-testing-20240327.051042-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240327.051042-reef sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8052 sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f sleep_before_teardown: 0 subset: 497/512 suite: fs suite_branch: wip-rishabh-testing-20240327.051042-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBnV2lYsYPDWdzFnsqSHilP5gFUvEH6tYg2nrtts4HQdB1KD5f8DrYQZgF2G+KyE2hFQ0g4ZTS7B+RWAz4uajsI= smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAPsmWgWtA+qGMoofhF3cRgrQ8UIrqa4hvA1ltjlGnUPlNPmnDQ3qjKQEYuP1/Fq7A38uPb7hTepkjvXaoNCLpY= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH+u33hMnPJ9lV3T4yEUw1/EJlDzs4wChg+GzoMbYiZD04JbXQkOT/VZP0wc9iqvWfz1Xx5Ig68UionwRt9CbrQ= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - kernel_untar_build.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_18:05:24 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-30T02:05:48.923 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa; will attempt to use it 2024-03-30T02:05:48.923 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks 2024-03-30T02:05:48.923 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-30T02:05:48.925 INFO:teuthology.task.internal:Checking packages... 2024-03-30T02:05:48.955 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '470100a4ccbcf7769052bf09a5413881ccf1b97f' 2024-03-30T02:05:48.955 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-30T02:05:48.955 INFO:teuthology.packaging:ref: None 2024-03-30T02:05:48.955 INFO:teuthology.packaging:tag: None 2024-03-30T02:05:48.955 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240327.051042-reef 2024-03-30T02:05:48.956 INFO:teuthology.packaging:sha1: 470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:05:48.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-testing-20240327.051042-reef 2024-03-30T02:05:49.191 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1018.g470100a4 2024-03-30T02:05:49.193 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-30T02:05:49.199 INFO:teuthology.task.internal:no buildpackages task found 2024-03-30T02:05:49.199 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-30T02:05:49.242 INFO:teuthology.task.internal:Saving configuration 2024-03-30T02:05:49.265 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-30T02:05:49.271 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T02:05:49.293 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 02:00:40.399338', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBnV2lYsYPDWdzFnsqSHilP5gFUvEH6tYg2nrtts4HQdB1KD5f8DrYQZgF2G+KyE2hFQ0g4ZTS7B+RWAz4uajsI='} 2024-03-30T02:05:49.315 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 02:00:40.401104', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAPsmWgWtA+qGMoofhF3cRgrQ8UIrqa4hvA1ltjlGnUPlNPmnDQ3qjKQEYuP1/Fq7A38uPb7hTepkjvXaoNCLpY='} 2024-03-30T02:05:49.340 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 02:00:40.402510', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH+u33hMnPJ9lV3T4yEUw1/EJlDzs4wChg+GzoMbYiZD04JbXQkOT/VZP0wc9iqvWfz1Xx5Ig68UionwRt9CbrQ='} 2024-03-30T02:05:49.341 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-30T02:05:49.347 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-30T02:05:49.347 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-30T02:05:49.347 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-30T02:05:49.347 INFO:teuthology.run_tasks:Running task console_log... 2024-03-30T02:05:49.470 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fea2311d160>, signals=[15]) 2024-03-30T02:05:49.471 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-30T02:05:49.477 INFO:teuthology.task.internal:Opening connections... 2024-03-30T02:05:49.477 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:05:49.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T02:05:49.557 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-03-30T02:05:49.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T02:05:49.632 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-03-30T02:05:49.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T02:05:49.703 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-30T02:05:49.710 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-03-30T02:05:49.732 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-03-30T02:05:49.732 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-03-30T02:05:49.792 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="Red Hat Enterprise Linux" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="8.6 (Ootpa)" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:ID="rhel" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="fedora" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="8.6" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el8" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://www.redhat.com/" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-30T02:05:49.793 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T02:05:49.794 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:05:49.794 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-30T02:05:49.794 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-30T02:05:49.794 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-30T02:05:49.794 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-30T02:05:49.794 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-03-30T02:05:49.817 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-03-30T02:05:49.837 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-03-30T02:05:49.837 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-03-30T02:05:49.897 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Red Hat Enterprise Linux" 2024-03-30T02:05:49.897 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8.6 (Ootpa)" 2024-03-30T02:05:49.897 INFO:teuthology.orchestra.run.smithi082.stdout:ID="rhel" 2024-03-30T02:05:49.897 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="fedora" 2024-03-30T02:05:49.897 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8.6" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.redhat.com/" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-30T02:05:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-30T02:05:49.899 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-03-30T02:05:49.916 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-03-30T02:05:49.937 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-03-30T02:05:49.937 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Red Hat Enterprise Linux" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8.6 (Ootpa)" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:ID="rhel" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="fedora" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8.6" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.redhat.com/" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-30T02:05:49.996 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-30T02:05:49.997 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:05:49.997 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-30T02:05:49.997 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-30T02:05:49.997 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-30T02:05:49.997 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-30T02:05:49.997 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-03-30T02:05:50.017 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-30T02:05:50.028 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-30T02:05:50.055 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-30T02:05:50.056 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-03-30T02:05:50.058 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-03-30T02:05:50.060 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-03-30T02:05:50.079 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-30T02:05:50.085 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-30T02:05:50.085 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-03-30T02:05:50.118 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-03-30T02:05:50.123 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-03-30T02:05:50.154 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-30T02:05:50.226 INFO:teuthology.run_tasks:Running task kernel... 2024-03-30T02:05:50.241 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-30T02:05:50.242 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-30T02:05:50.242 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-30T02:05:50.242 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T02:05:50.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:05:50.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2024-03-30T02:05:50.565 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T02:05:50.565 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-30T02:05:50.565 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-03-30T02:05:50.586 INFO:teuthology.orchestra.run.smithi067.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-30T02:05:50.586 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T02:05:50.586 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-30T02:05:50.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum upgrade -y linux-firmware 2024-03-30T02:05:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2024-03-30T02:05:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:Unable to read consumer identity 2024-03-30T02:05:52.426 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:05:52.426 INFO:teuthology.orchestra.run.smithi067.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-30T02:05:52.427 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:05:54.266 INFO:teuthology.orchestra.run.smithi067.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2024-03-30T02:05:55.000 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2024-03-30T02:05:58.875 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 583 kB/s | 24 kB 00:00 2024-03-30T02:05:59.732 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-03-30T02:05:59.733 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-03-30T02:05:59.733 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-03-30T02:05:59.883 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-03-30T02:05:59.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-30T02:06:00.105 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ 2024-03-30T02:06:00.105 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-30T02:06:00.108 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi067.front.sepia.ceph.com to complete... 2024-03-30T02:06:51.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30 02:06:51 URL:https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [401601944/401601944] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-30T02:06:51.387 INFO:teuthology.orchestra.run.smithi067.stderr:FINISHED --2024-03-30 02:06:51-- 2024-03-30T02:06:51.387 INFO:teuthology.orchestra.run.smithi067.stderr:Total wall clock time: 51s 2024-03-30T02:06:51.387 INFO:teuthology.orchestra.run.smithi067.stderr:Downloaded: 1 files, 383M in 51s (7.51 MB/s) 2024-03-30T02:06:51.391 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-03-30T02:06:51.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-30T02:06:53.328 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying... ######################################## 2024-03-30T02:06:53.426 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing... ######################################## 2024-03-30T02:06:53.601 INFO:teuthology.orchestra.run.smithi067.stdout:Updating / installing... 2024-03-30T02:07:36.672 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-30T02:07:38.170 INFO:teuthology.orchestra.run.smithi067.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T02:08:00.047 INFO:teuthology.orchestra.run.smithi067.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-30T02:10:15.071 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-30T02:10:15.071 DEBUG:teuthology.orchestra.run.smithi067:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-30T02:10:15.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-30T02:10:15.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-30T02:10:15.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-30T02:10:15.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-30T02:10:15.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-30T02:10:15.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-30T02:10:15.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-30T02:10:15.132 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-30T02:10:15.133 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-30T02:10:15.134 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-30T02:10:15.135 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-30T02:10:15.136 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-30T02:10:15.137 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-30T02:10:15.138 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-30T02:10:15.139 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-30T02:10:15.140 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-30T02:10:15.141 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-30T02:10:15.142 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-30T02:10:15.143 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-30T02:10:15.144 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-30T02:10:15.145 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-30T02:10:15.146 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-30T02:10:15.147 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-30T02:10:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-30T02:10:15.149 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-30T02:10:15.150 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-30T02:10:15.151 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-30T02:10:15.152 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-30T02:10:15.153 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-30T02:10:15.154 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-30T02:10:15.155 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-30T02:10:15.156 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-30T02:10:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-30T02:10:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-30T02:10:15.159 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-30T02:10:15.160 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-30T02:10:15.161 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-30T02:10:15.162 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-30T02:10:15.163 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-30T02:10:15.164 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-30T02:10:15.165 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-30T02:10:15.166 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-30T02:10:15.167 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-30T02:10:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-30T02:10:15.169 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-30T02:10:15.170 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-30T02:10:15.171 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-30T02:10:15.172 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-30T02:10:15.173 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-30T02:10:15.174 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-30T02:10:15.175 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-30T02:10:15.176 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-30T02:10:15.177 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-30T02:10:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-30T02:10:15.179 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-30T02:10:15.180 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-30T02:10:15.181 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-30T02:10:15.182 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-30T02:10:15.183 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-30T02:10:15.184 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-30T02:10:15.185 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-30T02:10:15.186 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-30T02:10:15.187 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-30T02:10:15.188 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-30T02:10:15.189 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-30T02:10:15.190 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-30T02:10:15.191 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-30T02:10:15.192 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-30T02:10:15.193 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-30T02:10:15.194 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-30T02:10:15.195 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-30T02:10:15.196 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-30T02:10:15.197 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-30T02:10:15.198 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-30T02:10:15.199 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-30T02:10:15.200 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-30T02:10:15.201 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-30T02:10:15.202 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-30T02:10:15.203 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-30T02:10:15.204 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-30T02:10:15.205 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-30T02:10:15.206 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-30T02:10:15.207 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-30T02:10:15.208 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-30T02:10:15.209 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-30T02:10:15.210 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-30T02:10:15.211 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-30T02:10:15.212 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-30T02:10:15.213 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-30T02:10:15.214 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-30T02:10:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-30T02:10:15.216 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-30T02:10:15.217 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-30T02:10:15.218 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-30T02:10:15.219 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-30T02:10:15.220 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-30T02:10:15.221 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-30T02:10:15.222 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-30T02:10:15.223 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-30T02:10:15.224 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-30T02:10:15.225 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-30T02:10:15.226 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-30T02:10:15.227 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-30T02:10:15.228 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-30T02:10:15.229 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-30T02:10:15.230 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-30T02:10:15.231 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-30T02:10:15.232 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-30T02:10:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-30T02:10:15.234 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-30T02:10:15.235 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-30T02:10:15.236 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-30T02:10:15.237 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-30T02:10:15.238 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-30T02:10:15.239 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-30T02:10:15.240 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-30T02:10:15.241 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-30T02:10:15.242 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-30T02:10:15.243 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-30T02:10:15.244 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-30T02:10:15.245 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-30T02:10:15.246 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-30T02:10:15.247 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-30T02:10:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-30T02:10:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-30T02:10:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-30T02:10:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-30T02:10:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-30T02:10:15.253 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-30T02:10:15.254 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-30T02:10:15.255 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-30T02:10:15.256 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-30T02:10:15.257 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-30T02:10:15.258 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-30T02:10:15.259 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-30T02:10:15.259 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-30T02:10:15.259 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-30T02:10:15.259 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-30T02:10:15.259 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-30T02:10:15.259 INFO:teuthology.orchestra.run.smithi067.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-30T02:10:15.263 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-30T02:10:15.263 DEBUG:teuthology.orchestra.run.smithi067:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-30T02:10:15.286 INFO:teuthology.orchestra.run.smithi067.stdout:postinstall scriptlet (using /bin/sh): 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout:for file in vmlinuz System.map config; do 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout: fi 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout:done 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-30T02:10:15.287 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-30T02:10:15.288 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-03-30T02:10:15.288 INFO:teuthology.orchestra.run.smithi067.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-30T02:10:15.288 INFO:teuthology.orchestra.run.smithi067.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-30T02:10:15.288 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-30T02:10:15.288 INFO:teuthology.orchestra.run.smithi067.stdout:fi 2024-03-30T02:10:15.289 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.02 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 123.el8 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Group : System Environment/Base 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 9265450 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-03-30T02:10:15.364 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:Relocations : (not relocatable) 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : Red Hat, Inc. 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : Red Hat, Inc. 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:10:15.365 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-03-30T02:10:15.366 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-30T02:10:15.366 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 6.8.0-g1df0c7e8a235 2024-03-30T02:10:15.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-30T02:10:16.094 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-03-30T02:10:18.236 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-03-30T02:10:18.237 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-03-30T02:10:18.255 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.Qluw9NlBLm 2024-03-30T02:10:18.255 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Qluw9NlBLm 2024-03-30T02:10:18.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.Qluw9NlBLm 2024-03-30T02:10:18.504 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.Qluw9NlBLm is 6KB 2024-03-30T02:10:18.553 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.Qluw9NlBLm 2024-03-30T02:10:18.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-03-30T02:10:18.640 INFO:teuthology.orchestra.run.smithi067.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-03-30T02:10:18.640 INFO:teuthology.orchestra.run.smithi067.stdout:2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235.conf 2024-03-30T02:10:18.640 INFO:teuthology.orchestra.run.smithi067.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-03-30T02:10:18.640 INFO:teuthology.orchestra.run.smithi067.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-03-30T02:10:18.640 INFO:teuthology.orchestra.run.smithi067.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-03-30T02:10:18.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235 2024-03-30T02:10:18.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-03-30T02:10:48.810 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-30T02:10:48.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-30T02:10:48.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T02:11:23.298 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-03-30T02:11:32.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-30T02:11:32.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T02:11:50.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-03-30T02:12:02.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-30T02:12:02.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T02:12:05.421 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-03-30T02:12:05.840 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-30T02:12:05.840 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g1df0c7e8a235"... 2024-03-30T02:12:05.841 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-03-30T02:12:05.880 INFO:teuthology.orchestra.run.smithi067.stdout:6.8.0-g1df0c7e8a235 2024-03-30T02:12:05.880 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-30T02:12:05.880 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T02:12:05.880 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-03-30T02:12:06.881 DEBUG:teuthology.parallel:result is None 2024-03-30T02:12:06.882 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-30T02:12:06.889 INFO:teuthology.task.internal:Creating test directory... 2024-03-30T02:12:06.889 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T02:12:06.895 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T02:12:06.899 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T02:12:06.937 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-30T02:12:06.944 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-30T02:12:06.967 INFO:teuthology.task.internal:Creating archive directory... 2024-03-30T02:12:06.968 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T02:12:06.981 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T02:12:06.983 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T02:12:07.052 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-30T02:12:07.059 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-30T02:12:07.059 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T02:12:07.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:12:07.077 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T02:12:07.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:12:07.096 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T02:12:07.100 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T02:12:07.120 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T02:12:07.181 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T02:12:07.192 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T02:12:07.200 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T02:12:07.213 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T02:12:07.275 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T02:12:07.315 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T02:12:07.319 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-30T02:12:07.326 INFO:teuthology.task.internal:Configuring sudo... 2024-03-30T02:12:07.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T02:12:07.330 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T02:12:07.332 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T02:12:07.393 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-30T02:12:07.402 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-30T02:12:07.402 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T02:12:07.406 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T02:12:07.411 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T02:12:07.443 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T02:12:07.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T02:12:07.643 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T02:12:07.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T02:12:07.794 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:12:07.794 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T02:12:07.863 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T02:12:07.908 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T02:12:07.967 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T02:12:08.049 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T02:12:08.098 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:12:08.098 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T02:12:08.168 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T02:12:08.217 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T02:12:08.271 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T02:12:08.357 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T02:12:08.402 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:12:08.402 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T02:12:08.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-03-30T02:12:08.475 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-03-30T02:12:08.477 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-03-30T02:12:08.559 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T02:12:08.597 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T02:12:08.597 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T02:12:09.014 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-30T02:12:09.022 INFO:teuthology.task.internal:Starting timer... 2024-03-30T02:12:09.022 INFO:teuthology.run_tasks:Running task pcp... 2024-03-30T02:12:09.072 INFO:teuthology.run_tasks:Running task selinux... 2024-03-30T02:12:09.120 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-30T02:12:09.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-03-30T02:12:09.280 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-03-30T02:12:09.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-03-30T02:12:09.354 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-03-30T02:12:09.356 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-03-30T02:12:09.425 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-03-30T02:12:09.427 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-30T02:12:09.427 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-03-30T02:12:09.479 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-03-30T02:12:09.480 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-03-30T02:12:09.506 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-03-30T02:12:09.507 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-03-30T02:12:09.535 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-03-30T02:12:09.536 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2024-03-30T02:12:09.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T02:12:09.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:12:09.603 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T02:12:09.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:12:09.631 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T02:12:09.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:12:09.659 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-30T02:12:09.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-03-30T02:12:09.724 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-03-30T02:12:09.759 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-03-30T02:12:09.797 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-30T02:12:09.807 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-30T02:12:09.812 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-30T02:12:09.938 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-30T02:12:09.953 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-30T02:12:09.954 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi082.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-30T02:23:34.360 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-03-30T02:23:34.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-30T02:23:34.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T02:23:34.454 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-03-30T02:23:34.568 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-03-30T02:23:34.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-30T02:23:34.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T02:23:34.642 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-03-30T02:23:34.707 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-30T02:23:34.708 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-30T02:23:34.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T02:23:34.779 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-03-30T02:23:34.848 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-03-30T02:23:34.848 INFO:teuthology.run_tasks:Running task clock... 2024-03-30T02:23:34.861 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-30T02:23:34.862 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T02:23:34.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T02:23:34.866 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T02:23:34.866 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T02:23:34.868 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T02:23:34.868 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T02:23:34.897 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T02:23:34.912 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T02:23:34.922 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T02:23:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-30T02:23:34.937 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T02:23:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-30T02:23:34.981 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-03-30T02:23:34.992 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-03-30T02:23:35.008 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T02:23:35.012 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-03-30T02:23:35.022 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T02:23:35.023 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-03-30T02:23:35.038 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T02:23:35.052 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T02:23:35.059 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-03-30T02:23:35.088 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-03-30T02:23:35.092 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2024-03-30T02:23:35.096 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T02:23:35.096 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-03-30T02:23:35.096 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.096 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.096 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.097 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.122 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-30T02:23:35.127 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2024-03-30T02:23:35.131 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T02:23:35.131 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-03-30T02:23:35.131 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.131 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.131 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.131 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.155 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-30T02:23:35.263 INFO:teuthology.orchestra.run.smithi067.stderr:bash: ntpq: command not found 2024-03-30T02:23:35.269 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T02:23:35.270 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-03-30T02:23:35.270 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.270 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.270 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.270 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:23:35.271 INFO:teuthology.run_tasks:Running task pexec... 2024-03-30T02:23:35.281 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-30T02:23:35.282 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:23:35.282 DEBUG:teuthology.orchestra.run.smithi067:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-30T02:23:35.283 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi082.front.sepia.ceph.com 2024-03-30T02:23:35.283 DEBUG:teuthology.orchestra.run.smithi082:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-30T02:23:35.283 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi170.front.sepia.ceph.com 2024-03-30T02:23:35.284 DEBUG:teuthology.orchestra.run.smithi170:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-30T02:23:35.661 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-03-30T02:23:35.662 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2024-03-30T02:23:35.804 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2024-03-30T02:23:37.505 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 kB/s | 4.5 kB 00:00 2024-03-30T02:23:37.667 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 kB/s | 4.5 kB 00:00 2024-03-30T02:23:37.685 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 kB/s | 4.5 kB 00:00 2024-03-30T02:23:38.225 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-03-30T02:23:38.296 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2024-03-30T02:23:38.370 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 18 kB/s | 4.1 kB 00:00 2024-03-30T02:23:41.107 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-03-30T02:23:41.107 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-30T02:23:41.107 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2024-03-30T02:23:41.108 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-30T02:23:41.108 INFO:teuthology.orchestra.run.smithi082.stdout:Resetting modules: 2024-03-30T02:23:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools 2024-03-30T02:23:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:23:41.108 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-03-30T02:23:41.108 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-30T02:23:41.108 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:23:41.203 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-03-30T02:23:41.204 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-03-30T02:23:41.204 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2024-03-30T02:23:41.204 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-03-30T02:23:41.204 INFO:teuthology.orchestra.run.smithi170.stdout:Resetting modules: 2024-03-30T02:23:41.204 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools 2024-03-30T02:23:41.205 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:23:41.205 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-03-30T02:23:41.205 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-03-30T02:23:41.205 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:23:41.655 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-03-30T02:23:41.657 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-03-30T02:23:41.657 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-03-30T02:23:41.657 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-03-30T02:23:41.657 INFO:teuthology.orchestra.run.smithi067.stdout:Resetting modules: 2024-03-30T02:23:41.657 INFO:teuthology.orchestra.run.smithi067.stdout: container-tools 2024-03-30T02:23:41.657 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:23:41.657 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-03-30T02:23:41.657 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-03-30T02:23:41.657 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:23:50.828 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-03-30T02:23:50.874 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-03-30T02:23:51.329 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-03-30T02:23:51.360 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2024-03-30T02:23:53.206 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-03-30T02:23:53.289 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-03-30T02:23:53.780 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 36 kB/s | 4.1 kB 00:00 2024-03-30T02:23:53.886 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-03-30T02:23:54.706 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-03-30T02:23:55.347 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2024-03-30T02:23:56.559 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-03-30T02:23:56.692 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout:========================================================================================================== 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout:========================================================================================================== 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout:Installing group/module packages: 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-30T02:23:56.724 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout:Installing module profiles: 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools/common 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools rhel8 2024-03-30T02:23:56.725 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:23:56.726 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-03-30T02:23:56.726 INFO:teuthology.orchestra.run.smithi082.stdout:========================================================================================================== 2024-03-30T02:23:56.726 INFO:teuthology.orchestra.run.smithi082.stdout:Install 13 Packages 2024-03-30T02:23:56.726 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:23:56.726 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 26 M 2024-03-30T02:23:56.726 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 87 M 2024-03-30T02:23:56.726 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-03-30T02:23:56.858 INFO:teuthology.orchestra.run.smithi170.stdout:========================================================================================================== 2024-03-30T02:23:56.858 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-03-30T02:23:56.858 INFO:teuthology.orchestra.run.smithi170.stdout:========================================================================================================== 2024-03-30T02:23:56.858 INFO:teuthology.orchestra.run.smithi170.stdout:Installing group/module packages: 2024-03-30T02:23:56.858 INFO:teuthology.orchestra.run.smithi170.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-30T02:23:56.858 INFO:teuthology.orchestra.run.smithi170.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-30T02:23:56.858 INFO:teuthology.orchestra.run.smithi170.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout:Installing module profiles: 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools/common 2024-03-30T02:23:56.859 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:23:56.860 INFO:teuthology.orchestra.run.smithi170.stdout:Enabling module streams: 2024-03-30T02:23:56.860 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools rhel8 2024-03-30T02:23:56.860 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:23:56.860 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-03-30T02:23:56.860 INFO:teuthology.orchestra.run.smithi170.stdout:========================================================================================================== 2024-03-30T02:23:56.860 INFO:teuthology.orchestra.run.smithi170.stdout:Install 13 Packages 2024-03-30T02:23:56.860 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:23:56.860 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 26 M 2024-03-30T02:23:56.861 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 87 M 2024-03-30T02:23:56.861 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-03-30T02:23:56.951 INFO:teuthology.orchestra.run.smithi082.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 420 kB/s | 94 kB 00:00 2024-03-30T02:23:57.115 INFO:teuthology.orchestra.run.smithi170.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 97 kB/s | 25 kB 00:00 2024-03-30T02:23:57.140 INFO:teuthology.orchestra.run.smithi170.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 141 kB/s | 39 kB 00:00 2024-03-30T02:23:57.166 INFO:teuthology.orchestra.run.smithi170.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 308 kB/s | 94 kB 00:00 2024-03-30T02:23:57.176 INFO:teuthology.orchestra.run.smithi082.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 88 kB/s | 39 kB 00:00 2024-03-30T02:23:57.201 INFO:teuthology.orchestra.run.smithi082.stdout:(3/13): python3-pytoml-0.1.14-5.git7dea353.el8. 52 kB/s | 25 kB 00:00 2024-03-30T02:23:57.226 INFO:teuthology.orchestra.run.smithi082.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 149 kB/s | 41 kB 00:00 2024-03-30T02:23:57.258 INFO:teuthology.orchestra.run.smithi170.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 288 kB/s | 41 kB 00:00 2024-03-30T02:23:57.301 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-03-30T02:23:57.400 INFO:teuthology.orchestra.run.smithi170.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 3.9 MB/s | 1.0 MB 00:00 2024-03-30T02:23:57.427 INFO:teuthology.orchestra.run.smithi082.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 4.0 MB/s | 1.0 MB 00:00 2024-03-30T02:23:57.442 INFO:teuthology.orchestra.run.smithi170.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 3.9 MB/s | 739 kB 00:00 2024-03-30T02:23:57.510 INFO:teuthology.orchestra.run.smithi082.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 2.5 MB/s | 739 kB 00:00 2024-03-30T02:23:57.610 INFO:teuthology.orchestra.run.smithi170.stdout:(7/13): buildah-1.31.3-3.module+el8.9.0+21243+a 20 MB/s | 8.8 MB 00:00 2024-03-30T02:23:57.659 INFO:teuthology.orchestra.run.smithi170.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 924 kB/s | 239 kB 00:00 2024-03-30T02:23:57.660 INFO:teuthology.orchestra.run.smithi082.stdout:(7/13): buildah-1.31.3-3.module+el8.9.0+21243+a 19 MB/s | 8.8 MB 00:00 2024-03-30T02:23:57.686 INFO:teuthology.orchestra.run.smithi082.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 924 kB/s | 239 kB 00:00 2024-03-30T02:23:57.734 INFO:teuthology.orchestra.run.smithi170.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21243+a 13 MB/s | 3.7 MB 00:00 2024-03-30T02:23:57.769 INFO:teuthology.orchestra.run.smithi082.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21243+a 14 MB/s | 3.7 MB 00:00 2024-03-30T02:23:57.785 INFO:teuthology.orchestra.run.smithi170.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 882 kB/s | 154 kB 00:00 2024-03-30T02:23:57.820 INFO:teuthology.orchestra.run.smithi082.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 968 kB/s | 154 kB 00:00 2024-03-30T02:23:57.977 INFO:teuthology.orchestra.run.smithi170.stdout:(11/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 26 MB/s | 8.1 MB 00:00 2024-03-30T02:23:58.036 INFO:teuthology.orchestra.run.smithi082.stdout:(11/13): udica-0.2.6-20.module+el8.9.0+21243+a5 228 kB/s | 49 kB 00:00 2024-03-30T02:23:58.050 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-03-30T02:23:58.143 INFO:teuthology.orchestra.run.smithi170.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21243+a5 138 kB/s | 49 kB 00:00 2024-03-30T02:23:58.195 INFO:teuthology.orchestra.run.smithi082.stdout:(12/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 16 MB/s | 8.1 MB 00:00 2024-03-30T02:23:58.335 INFO:teuthology.orchestra.run.smithi170.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 4.2 MB/s | 2.5 MB 00:00 2024-03-30T02:23:58.336 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-03-30T02:23:58.336 INFO:teuthology.orchestra.run.smithi170.stdout:Total 17 MB/s | 26 MB 00:01 2024-03-30T02:23:58.537 INFO:teuthology.orchestra.run.smithi082.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 3.3 MB/s | 2.5 MB 00:00 2024-03-30T02:23:58.537 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-03-30T02:23:58.537 INFO:teuthology.orchestra.run.smithi082.stdout:Total 14 MB/s | 26 MB 00:01 2024-03-30T02:23:58.626 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-03-30T02:23:58.648 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-03-30T02:23:58.648 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-03-30T02:23:58.821 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-03-30T02:23:58.842 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-03-30T02:23:58.842 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-03-30T02:23:58.878 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-03-30T02:23:58.887 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-03-30T02:23:59.072 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-03-30T02:23:59.081 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-03-30T02:23:59.544 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-03-30T02:23:59.769 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-03-30T02:24:00.041 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-30T02:24:00.223 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-30T02:24:00.325 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-30T02:24:00.414 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-30T02:24:00.509 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-30T02:24:00.583 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-30T02:24:00.691 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-30T02:24:00.753 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-30T02:24:00.859 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-30T02:24:00.945 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-30T02:24:01.046 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-30T02:24:01.214 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-30T02:24:01.270 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-30T02:24:01.309 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-03-30T02:24:01.492 INFO:teuthology.orchestra.run.smithi067.stdout:========================================================================================================== 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout:========================================================================================================== 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout:Installing group/module packages: 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: buildah x86_64 1:1.31.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 739 k 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 239 k 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 154 k 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: udica noarch 0.2.6-20.module+el8.9.0+21243+a586538b rhel-8-for-x86_64-appstream-rpms 49 k 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-03-30T02:24:01.493 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout:Installing module profiles: 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: container-tools/common 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling module streams: 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: container-tools rhel8 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout:========================================================================================================== 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout:Install 13 Packages 2024-03-30T02:24:01.494 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:24:01.495 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 26 M 2024-03-30T02:24:01.495 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 87 M 2024-03-30T02:24:01.495 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-03-30T02:24:01.523 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-30T02:24:01.655 INFO:teuthology.orchestra.run.smithi067.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 157 kB/s | 25 kB 00:00 2024-03-30T02:24:01.706 INFO:teuthology.orchestra.run.smithi067.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 450 kB/s | 94 kB 00:00 2024-03-30T02:24:01.739 INFO:teuthology.orchestra.run.smithi067.stdout:(3/13): criu-libs-3.18-4.module+el8.9.0+20326+3 163 kB/s | 39 kB 00:00 2024-03-30T02:24:01.764 INFO:teuthology.orchestra.run.smithi067.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 375 kB/s | 41 kB 00:00 2024-03-30T02:24:01.864 INFO:teuthology.orchestra.run.smithi067.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+212 6.3 MB/s | 1.0 MB 00:00 2024-03-30T02:24:01.914 INFO:teuthology.orchestra.run.smithi067.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2124 4.8 MB/s | 739 kB 00:00 2024-03-30T02:24:01.998 INFO:teuthology.orchestra.run.smithi067.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21243+a5865 1.8 MB/s | 239 kB 00:00 2024-03-30T02:24:02.102 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T02:24:02.190 INFO:teuthology.orchestra.run.smithi067.stdout:(8/13): buildah-1.31.3-3.module+el8.9.0+21243+a 20 MB/s | 8.8 MB 00:00 2024-03-30T02:24:02.215 INFO:teuthology.orchestra.run.smithi067.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 708 kB/s | 154 kB 00:00 2024-03-30T02:24:02.337 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T02:24:02.357 INFO:teuthology.orchestra.run.smithi067.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21243+ 8.4 MB/s | 3.7 MB 00:00 2024-03-30T02:24:02.401 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T02:24:02.508 INFO:teuthology.orchestra.run.smithi067.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2124 8.7 MB/s | 2.5 MB 00:00 2024-03-30T02:24:02.549 INFO:teuthology.orchestra.run.smithi067.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21243+a5 258 kB/s | 49 kB 00:00 2024-03-30T02:24:02.647 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T02:24:02.658 INFO:teuthology.orchestra.run.smithi067.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21243+a 17 MB/s | 8.1 MB 00:00 2024-03-30T02:24:02.659 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-03-30T02:24:02.659 INFO:teuthology.orchestra.run.smithi067.stdout:Total 22 MB/s | 26 MB 00:01 2024-03-30T02:24:02.731 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-30T02:24:03.065 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-30T02:24:03.134 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-03-30T02:24:03.162 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-03-30T02:24:03.162 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-03-30T02:24:03.622 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-30T02:24:03.675 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-03-30T02:24:03.685 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-03-30T02:24:03.843 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-30T02:24:03.941 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-30T02:24:04.084 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T02:24:04.194 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-30T02:24:04.385 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T02:24:04.476 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-03-30T02:24:04.498 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T02:24:04.498 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-30T02:24:04.498 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-30T02:24:04.499 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-30T02:24:04.803 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T02:24:04.803 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-30T02:24:04.803 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-30T02:24:04.804 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-30T02:24:05.065 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 1/13 2024-03-30T02:24:05.280 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 2/13 2024-03-30T02:24:05.467 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-03-30T02:24:05.636 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-03-30T02:24:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-03-30T02:24:06.062 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-03-30T02:24:06.306 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 7/13 2024-03-30T02:24:07.277 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T02:24:07.526 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T02:24:07.922 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 10/13 2024-03-30T02:24:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 11/13 2024-03-30T02:24:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 12/13 2024-03-30T02:24:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 13/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a58653 5/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x 6/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21243+a586538b. 7/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 8/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x 9/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21243+a58653 10/13 2024-03-30T02:24:10.094 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x8 11/13 2024-03-30T02:24:10.095 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x 12/13 2024-03-30T02:24:17.152 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-30T02:24:17.152 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2024-03-30T02:24:17.246 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-30T02:24:17.247 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2024-03-30T02:24:17.695 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:24:17.695 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-30T02:24:17.696 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:24:17.697 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-03-30T02:24:17.801 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:24:17.801 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-03-30T02:24:17.801 INFO:teuthology.orchestra.run.smithi170.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.801 INFO:teuthology.orchestra.run.smithi170.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.801 INFO:teuthology.orchestra.run.smithi170.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-30T02:24:17.801 INFO:teuthology.orchestra.run.smithi170.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:24:17.802 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-03-30T02:24:17.842 DEBUG:teuthology.parallel:result is None 2024-03-30T02:24:17.971 DEBUG:teuthology.parallel:result is None 2024-03-30T02:24:21.417 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarc 13/13 2024-03-30T02:24:21.417 INFO:teuthology.orchestra.run.smithi067.stdout:Installed products updated. 2024-03-30T02:24:22.080 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: buildah-1:1.31.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: cockpit-podman-75-1.module+el8.9.0+21243+a586538b.noarch 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: crun-1.8.7-1.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: netavark-2:1.7.0-2.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: python3-podman-4.6.0-2.module+el8.9.0+21243+a586538b.noarch 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21243+a586538b.x86_64 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: udica-0.2.6-20.module+el8.9.0+21243+a586538b.noarch 2024-03-30T02:24:22.081 INFO:teuthology.orchestra.run.smithi067.stdout: yajl-2.1.0-12.el8.x86_64 2024-03-30T02:24:22.082 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:24:22.082 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-03-30T02:24:22.327 DEBUG:teuthology.parallel:result is None 2024-03-30T02:24:22.327 INFO:teuthology.run_tasks:Running task install... 2024-03-30T02:24:22.337 DEBUG:teuthology.task.install:project ceph 2024-03-30T02:24:22.338 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'}} 2024-03-30T02:24:22.338 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f'} 2024-03-30T02:24:22.338 INFO:teuthology.task.install:Using flavor: default 2024-03-30T02:24:22.344 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-30T02:24:22.344 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-30T02:24:22.344 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'tag': None, 'wait_for_package': False} 2024-03-30T02:24:22.345 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:24:22.347 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'tag': None, 'wait_for_package': False} 2024-03-30T02:24:22.347 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:24:22.348 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'tag': None, 'wait_for_package': False} 2024-03-30T02:24:22.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:24:22.520 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/ 2024-03-30T02:24:22.520 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1018.g470100a4 2024-03-30T02:24:22.608 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/ 2024-03-30T02:24:22.609 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1018.g470100a4 2024-03-30T02:24:22.635 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/ 2024-03-30T02:24:22.635 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1018.g470100a4 2024-03-30T02:24:22.707 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T02:24:22.708 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:24:22.708 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T02:24:22.741 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T02:24:22.741 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/470100a4ccbcf7769052bf09a5413881ccf1b97f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T02:24:22.755 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T02:24:22.755 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:24:22.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T02:24:22.770 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-30T02:24:22.770 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:24:22.770 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-30T02:24:22.803 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T02:24:22.803 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/470100a4ccbcf7769052bf09a5413881ccf1b97f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T02:24:22.812 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T02:24:22.822 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-30T02:24:22.823 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/470100a4ccbcf7769052bf09a5413881ccf1b97f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-30T02:24:22.873 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T02:24:22.893 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T02:24:22.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-30T02:24:22.953 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T02:24:22.961 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2024-03-30T02:24:22.962 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2024-03-30T02:24:23.022 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-30T02:24:23.024 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-03-30T02:24:23.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-03-30T02:24:23.088 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2024-03-30T02:24:23.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2024-03-30T02:24:23.338 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2024-03-30T02:24:23.400 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-03-30T02:24:23.568 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2024-03-30T02:24:24.570 INFO:teuthology.orchestra.run.smithi082.stdout:61 files removed 2024-03-30T02:24:24.626 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T02:24:24.789 INFO:teuthology.orchestra.run.smithi170.stdout:61 files removed 2024-03-30T02:24:24.842 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T02:24:24.961 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-03-30T02:24:25.049 INFO:teuthology.orchestra.run.smithi067.stdout:61 files removed 2024-03-30T02:24:25.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-30T02:24:25.182 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2024-03-30T02:24:25.586 INFO:teuthology.orchestra.run.smithi067.stdout:Updating Subscription Management repositories. 2024-03-30T02:24:26.872 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 222 kB/s | 80 kB 00:00 2024-03-30T02:24:27.190 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 194 kB/s | 80 kB 00:00 2024-03-30T02:24:27.197 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 34 kB/s | 9.9 kB 00:00 2024-03-30T02:24:27.507 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 35 kB/s | 9.9 kB 00:00 2024-03-30T02:24:27.573 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 4.3 kB/s | 1.5 kB 00:00 2024-03-30T02:24:27.639 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 221 kB/s | 80 kB 00:00 2024-03-30T02:24:27.731 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-03-30T02:24:27.849 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 4.7 kB/s | 1.5 kB 00:00 2024-03-30T02:24:27.981 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 33 kB/s | 9.9 kB 00:00 2024-03-30T02:24:28.000 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-03-30T02:24:28.365 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2024-03-30T02:24:28.516 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2024-03-30T02:24:28.549 INFO:teuthology.orchestra.run.smithi067.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-03-30T02:24:29.327 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2024-03-30T02:24:29.703 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 16 MB 00:01 2024-03-30T02:24:32.074 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 477 kB/s | 24 kB 00:00 2024-03-30T02:24:33.593 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 6.1 MB/s | 8.8 MB 00:01 2024-03-30T02:24:33.602 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 70 kB/s | 24 kB 00:00 2024-03-30T02:24:33.842 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 276 kB/s | 24 kB 00:00 2024-03-30T02:24:34.346 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-30T02:24:35.606 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 MB/s | 60 MB 00:01 2024-03-30T02:24:36.328 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 MB/s | 60 MB 00:01 2024-03-30T02:24:37.172 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 60 MB 00:01 2024-03-30T02:24:48.153 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 40 MB/s | 66 MB 00:01 2024-03-30T02:24:48.464 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 53 MB/s | 66 MB 00:01 2024-03-30T02:24:48.870 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 66 MB 00:01 2024-03-30T02:25:00.954 INFO:teuthology.orchestra.run.smithi067.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-30T02:25:02.881 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T02:25:02.882 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T02:25:02.900 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-30T02:25:02.902 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-30T02:25:02.903 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-30T02:25:02.905 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-30T02:25:02.907 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-30T02:25:02.911 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-30T02:25:02.912 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-30T02:25:03.127 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-03-30T02:25:03.141 INFO:teuthology.orchestra.run.smithi082.stdout:=================================================================================================================================== 2024-03-30T02:25:03.141 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-03-30T02:25:03.141 INFO:teuthology.orchestra.run.smithi082.stdout:=================================================================================================================================== 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.4 k 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:18.2.2-1018.g470100a4.el8 ceph 5.5 M 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 854 k 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1018.g470100a4.el8 ceph 166 k 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:18.2.2-1018.g470100a4.el8 ceph 1.6 M 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 138 k 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 1.8 M 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 7.4 M 2024-03-30T02:25:03.142 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 49 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 13 M 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:18.2.2-1018.g470100a4.el8 ceph 47 M 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 263 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 173 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 261 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 18 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 31 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 775 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 139 k 2024-03-30T02:25:03.143 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:18.2.2-1018.g470100a4.el8 ceph 196 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:18.2.2-1018.g470100a4.el8 ceph 371 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 366 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 107 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 95 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.5 M 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 181 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-03-30T02:25:03.144 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.7 M 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.6 M 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-30T02:25:03.145 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 22 M 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 23 k 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:18.2.2-1018.g470100a4.el8 ceph 2.3 M 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 245 k 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:18.2.2-1018.g470100a4.el8 ceph 4.8 M 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 18 M 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 15 k 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:18.2.2-1018.g470100a4.el8 ceph 24 k 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-30T02:25:03.146 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:18.2.2-1018.g470100a4.el8 ceph 142 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 475 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.5 M 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-30T02:25:03.147 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 45 k 2024-03-30T02:25:03.148 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 129 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-30T02:25:03.149 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-30T02:25:03.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-30T02:25:03.151 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql 10 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout:=================================================================================================================================== 2024-03-30T02:25:03.152 INFO:teuthology.orchestra.run.smithi082.stdout:Install 130 Packages 2024-03-30T02:25:03.153 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 16 Packages 2024-03-30T02:25:03.153 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:25:03.153 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 213 M 2024-03-30T02:25:03.153 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-03-30T02:25:03.245 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T02:25:03.246 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T02:25:03.265 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-30T02:25:03.267 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-30T02:25:03.268 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-30T02:25:03.270 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-30T02:25:03.273 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-30T02:25:03.278 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-30T02:25:03.279 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-30T02:25:03.334 INFO:teuthology.orchestra.run.smithi082.stdout:(1/146): ceph-18.2.2-1018.g470100a4.el8.x86_64. 53 kB/s | 6.4 kB 00:00 2024-03-30T02:25:03.500 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-03-30T02:25:03.514 INFO:teuthology.orchestra.run.smithi170.stdout:=================================================================================================================================== 2024-03-30T02:25:03.514 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-03-30T02:25:03.514 INFO:teuthology.orchestra.run.smithi170.stdout:=================================================================================================================================== 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.4 k 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:18.2.2-1018.g470100a4.el8 ceph 5.5 M 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 854 k 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1018.g470100a4.el8 ceph 166 k 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:18.2.2-1018.g470100a4.el8 ceph 1.6 M 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 138 k 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 1.8 M 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 7.4 M 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 49 k 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 13 M 2024-03-30T02:25:03.515 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:18.2.2-1018.g470100a4.el8 ceph 47 M 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 263 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 173 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 261 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 18 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 31 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 775 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 139 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-30T02:25:03.516 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:18.2.2-1018.g470100a4.el8 ceph 196 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:18.2.2-1018.g470100a4.el8 ceph 371 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 366 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 107 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 95 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.5 M 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 181 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-30T02:25:03.517 INFO:teuthology.orchestra.run.smithi170.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-30T02:25:03.518 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.7 M 2024-03-30T02:25:03.518 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.6 M 2024-03-30T02:25:03.518 INFO:teuthology.orchestra.run.smithi170.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 22 M 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 23 k 2024-03-30T02:25:03.519 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:18.2.2-1018.g470100a4.el8 ceph 2.3 M 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 245 k 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:18.2.2-1018.g470100a4.el8 ceph 4.8 M 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 18 M 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 15 k 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:18.2.2-1018.g470100a4.el8 ceph 24 k 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:18.2.2-1018.g470100a4.el8 ceph 142 k 2024-03-30T02:25:03.520 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 475 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.5 M 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-30T02:25:03.521 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-30T02:25:03.522 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-30T02:25:03.523 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 45 k 2024-03-30T02:25:03.523 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 129 k 2024-03-30T02:25:03.523 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-30T02:25:03.524 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-30T02:25:03.524 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-30T02:25:03.524 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-30T02:25:03.525 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-30T02:25:03.525 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-30T02:25:03.525 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T02:25:03.526 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-30T02:25:03.527 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-30T02:25:03.527 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-30T02:25:03.527 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-30T02:25:03.527 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-30T02:25:03.527 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-30T02:25:03.527 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-30T02:25:03.528 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-30T02:25:03.529 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout:Enabling module streams: 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql 10 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout:=================================================================================================================================== 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout:Install 130 Packages 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 16 Packages 2024-03-30T02:25:03.530 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 213 M 2024-03-30T02:25:03.531 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-03-30T02:25:03.684 INFO:teuthology.orchestra.run.smithi082.stdout:(2/146): ceph-fuse-18.2.2-1018.g470100a4.el8.x8 2.4 MB/s | 854 kB 00:00 2024-03-30T02:25:03.818 INFO:teuthology.orchestra.run.smithi082.stdout:(3/146): ceph-immutable-object-cache-18.2.2-101 1.2 MB/s | 166 kB 00:00 2024-03-30T02:25:03.893 INFO:teuthology.orchestra.run.smithi082.stdout:(4/146): ceph-base-18.2.2-1018.g470100a4.el8.x8 8.1 MB/s | 5.5 MB 00:00 2024-03-30T02:25:04.071 INFO:teuthology.orchestra.run.smithi170.stdout:(1/146): ceph-18.2.2-1018.g470100a4.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2024-03-30T02:25:04.110 INFO:teuthology.orchestra.run.smithi082.stdout:(5/146): ceph-mgr-18.2.2-1018.g470100a4.el8.x86 7.2 MB/s | 1.6 MB 00:00 2024-03-30T02:25:04.619 INFO:teuthology.orchestra.run.smithi082.stdout:(6/146): ceph-mds-18.2.2-1018.g470100a4.el8.x86 2.9 MB/s | 2.3 MB 00:00 2024-03-30T02:25:04.727 INFO:teuthology.orchestra.run.smithi082.stdout:(7/146): ceph-mon-18.2.2-1018.g470100a4.el8.x86 7.8 MB/s | 4.8 MB 00:00 2024-03-30T02:25:04.936 INFO:teuthology.orchestra.run.smithi082.stdout:(8/146): ceph-common-18.2.2-1018.g470100a4.el8. 13 MB/s | 22 MB 00:01 2024-03-30T02:25:05.003 INFO:teuthology.orchestra.run.smithi082.stdout:(9/146): ceph-selinux-18.2.2-1018.g470100a4.el8 366 kB/s | 24 kB 00:00 2024-03-30T02:25:05.214 INFO:teuthology.orchestra.run.smithi170.stdout:(2/146): ceph-fuse-18.2.2-1018.g470100a4.el8.x8 747 kB/s | 854 kB 00:01 2024-03-30T02:25:05.514 INFO:teuthology.orchestra.run.smithi170.stdout:(3/146): ceph-immutable-object-cache-18.2.2-101 553 kB/s | 166 kB 00:00 2024-03-30T02:25:05.740 INFO:teuthology.orchestra.run.smithi170.stdout:(4/146): ceph-base-18.2.2-1018.g470100a4.el8.x8 2.7 MB/s | 5.5 MB 00:02 2024-03-30T02:25:06.123 INFO:teuthology.orchestra.run.smithi170.stdout:(5/146): ceph-mgr-18.2.2-1018.g470100a4.el8.x86 4.1 MB/s | 1.6 MB 00:00 2024-03-30T02:25:06.270 INFO:teuthology.orchestra.run.smithi082.stdout:(10/146): ceph-radosgw-18.2.2-1018.g470100a4.el 8.5 MB/s | 13 MB 00:01 2024-03-30T02:25:06.361 INFO:teuthology.orchestra.run.smithi082.stdout:(11/146): cephfs-mirror-18.2.2-1018.g470100a4.e 2.8 MB/s | 261 kB 00:00 2024-03-30T02:25:06.445 INFO:teuthology.orchestra.run.smithi082.stdout:(12/146): libcephfs-devel-18.2.2-1018.g470100a4 372 kB/s | 31 kB 00:00 2024-03-30T02:25:06.620 INFO:teuthology.orchestra.run.smithi082.stdout:(13/146): libcephfs2-18.2.2-1018.g470100a4.el8. 4.3 MB/s | 775 kB 00:00 2024-03-30T02:25:06.687 INFO:teuthology.orchestra.run.smithi082.stdout:(14/146): libcephsqlite-18.2.2-1018.g470100a4.e 2.1 MB/s | 142 kB 00:00 2024-03-30T02:25:06.690 INFO:teuthology.orchestra.run.smithi170.stdout:(6/146): ceph-mon-18.2.2-1018.g470100a4.el8.x86 8.5 MB/s | 4.8 MB 00:00 2024-03-30T02:25:06.754 INFO:teuthology.orchestra.run.smithi082.stdout:(15/146): librados-devel-18.2.2-1018.g470100a4. 2.0 MB/s | 139 kB 00:00 2024-03-30T02:25:06.862 INFO:teuthology.orchestra.run.smithi082.stdout:(16/146): libradosstriper1-18.2.2-1018.g470100a 4.3 MB/s | 475 kB 00:00 2024-03-30T02:25:07.074 INFO:teuthology.orchestra.run.smithi170.stdout:(7/146): ceph-mds-18.2.2-1018.g470100a4.el8.x86 1.5 MB/s | 2.3 MB 00:01 2024-03-30T02:25:07.896 INFO:teuthology.orchestra.run.smithi082.stdout:(17/146): librgw2-18.2.2-1018.g470100a4.el8.x86 6.3 MB/s | 6.5 MB 00:01 2024-03-30T02:25:07.971 INFO:teuthology.orchestra.run.smithi082.stdout:(18/146): python3-ceph-argparse-18.2.2-1018.g47 597 kB/s | 45 kB 00:00 2024-03-30T02:25:08.047 INFO:teuthology.orchestra.run.smithi082.stdout:(19/146): python3-ceph-common-18.2.2-1018.g4701 1.7 MB/s | 129 kB 00:00 2024-03-30T02:25:08.130 INFO:teuthology.orchestra.run.smithi082.stdout:(20/146): python3-cephfs-18.2.2-1018.g470100a4. 2.3 MB/s | 196 kB 00:00 2024-03-30T02:25:08.239 INFO:teuthology.orchestra.run.smithi082.stdout:(21/146): python3-rados-18.2.2-1018.g470100a4.e 3.3 MB/s | 371 kB 00:00 2024-03-30T02:25:08.331 INFO:teuthology.orchestra.run.smithi082.stdout:(22/146): python3-rbd-18.2.2-1018.g470100a4.el8 3.9 MB/s | 366 kB 00:00 2024-03-30T02:25:08.389 INFO:teuthology.orchestra.run.smithi082.stdout:(23/146): python3-rgw-18.2.2-1018.g470100a4.el8 1.8 MB/s | 107 kB 00:00 2024-03-30T02:25:08.448 INFO:teuthology.orchestra.run.smithi082.stdout:(24/146): rbd-fuse-18.2.2-1018.g470100a4.el8.x8 1.6 MB/s | 95 kB 00:00 2024-03-30T02:25:08.565 INFO:teuthology.orchestra.run.smithi170.stdout:(8/146): ceph-osd-18.2.2-1018.g470100a4.el8.x86 9.8 MB/s | 18 MB 00:01 2024-03-30T02:25:08.624 INFO:teuthology.orchestra.run.smithi170.stdout:(9/146): ceph-selinux-18.2.2-1018.g470100a4.el8 419 kB/s | 24 kB 00:00 2024-03-30T02:25:08.865 INFO:teuthology.orchestra.run.smithi082.stdout:(25/146): rbd-mirror-18.2.2-1018.g470100a4.el8. 8.3 MB/s | 3.5 MB 00:00 2024-03-30T02:25:08.905 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T02:25:08.907 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-30T02:25:08.927 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-30T02:25:08.929 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-30T02:25:08.930 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-30T02:25:08.932 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-30T02:25:08.935 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-30T02:25:08.940 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-30T02:25:08.941 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-30T02:25:08.957 INFO:teuthology.orchestra.run.smithi082.stdout:(26/146): rbd-nbd-18.2.2-1018.g470100a4.el8.x86 1.9 MB/s | 181 kB 00:00 2024-03-30T02:25:09.040 INFO:teuthology.orchestra.run.smithi082.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1018.g 279 kB/s | 23 kB 00:00 2024-03-30T02:25:09.132 INFO:teuthology.orchestra.run.smithi082.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1018.g470100a 1.5 MB/s | 138 kB 00:00 2024-03-30T02:25:09.198 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-03-30T02:25:09.214 INFO:teuthology.orchestra.run.smithi067.stdout:=================================================================================================================================== 2024-03-30T02:25:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout:=================================================================================================================================== 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.4 k 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:18.2.2-1018.g470100a4.el8 ceph 5.5 M 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 854 k 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1018.g470100a4.el8 ceph 166 k 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:18.2.2-1018.g470100a4.el8 ceph 1.6 M 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 138 k 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 1.8 M 2024-03-30T02:25:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 7.4 M 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 49 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 13 M 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:18.2.2-1018.g470100a4.el8 ceph 47 M 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 263 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 173 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 261 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 18 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 31 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 775 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:18.2.2-1018.g470100a4.el8 ceph 139 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-30T02:25:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:18.2.2-1018.g470100a4.el8 ceph 196 k 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:18.2.2-1018.g470100a4.el8 ceph 371 k 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 366 k 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:18.2.2-1018.g470100a4.el8 ceph 107 k 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 95 k 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.5 M 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 181 k 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-03-30T02:25:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.7 M 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 3.6 M 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-30T02:25:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 22 M 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 23 k 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:18.2.2-1018.g470100a4.el8 ceph 2.3 M 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 245 k 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:18.2.2-1018.g470100a4.el8 ceph 4.8 M 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:18.2.2-1018.g470100a4.el8 ceph 18 M 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1018.g470100a4.el8 ceph-noarch 15 k 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:18.2.2-1018.g470100a4.el8 ceph 24 k 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-30T02:25:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:18.2.2-1018.g470100a4.el8 ceph 142 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 475 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-30T02:25:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:18.2.2-1018.g470100a4.el8 ceph 6.5 M 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-30T02:25:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:18.2.2-1018.g470100a4.el8 ceph 45 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:18.2.2-1018.g470100a4.el8 ceph 129 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-30T02:25:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-30T02:25:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-30T02:25:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling module streams: 2024-03-30T02:25:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql 10 2024-03-30T02:25:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:25:09.226 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-03-30T02:25:09.226 INFO:teuthology.orchestra.run.smithi067.stdout:=================================================================================================================================== 2024-03-30T02:25:09.226 INFO:teuthology.orchestra.run.smithi067.stdout:Install 130 Packages 2024-03-30T02:25:09.226 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 16 Packages 2024-03-30T02:25:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:25:09.226 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 213 M 2024-03-30T02:25:09.226 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-03-30T02:25:09.449 INFO:teuthology.orchestra.run.smithi082.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1018.g47010 5.6 MB/s | 1.8 MB 00:00 2024-03-30T02:25:09.713 INFO:teuthology.orchestra.run.smithi067.stdout:(1/146): ceph-18.2.2-1018.g470100a4.el8.x86_64. 15 kB/s | 6.4 kB 00:00 2024-03-30T02:25:10.564 INFO:teuthology.orchestra.run.smithi067.stdout:(2/146): ceph-fuse-18.2.2-1018.g470100a4.el8.x8 1.0 MB/s | 854 kB 00:00 2024-03-30T02:25:10.689 INFO:teuthology.orchestra.run.smithi067.stdout:(3/146): ceph-immutable-object-cache-18.2.2-101 1.3 MB/s | 166 kB 00:00 2024-03-30T02:25:11.116 INFO:teuthology.orchestra.run.smithi082.stdout:(30/146): ceph-test-18.2.2-1018.g470100a4.el8.x 7.7 MB/s | 47 MB 00:06 2024-03-30T02:25:11.209 INFO:teuthology.orchestra.run.smithi082.stdout:(31/146): ceph-mgr-diskprediction-local-18.2.2- 4.2 MB/s | 7.4 MB 00:01 2024-03-30T02:25:11.275 INFO:teuthology.orchestra.run.smithi082.stdout:(32/146): ceph-mgr-modules-core-18.2.2-1018.g47 1.5 MB/s | 245 kB 00:00 2024-03-30T02:25:11.300 INFO:teuthology.orchestra.run.smithi082.stdout:(33/146): ceph-mgr-rook-18.2.2-1018.g470100a4.e 536 kB/s | 49 kB 00:00 2024-03-30T02:25:11.367 INFO:teuthology.orchestra.run.smithi082.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1018.g4 161 kB/s | 15 kB 00:00 2024-03-30T02:25:11.397 INFO:teuthology.orchestra.run.smithi067.stdout:(4/146): ceph-base-18.2.2-1018.g470100a4.el8.x8 2.6 MB/s | 5.5 MB 00:02 2024-03-30T02:25:11.459 INFO:teuthology.orchestra.run.smithi082.stdout:(35/146): cephadm-18.2.2-1018.g470100a4.el8.noa 1.8 MB/s | 173 kB 00:00 2024-03-30T02:25:11.485 INFO:teuthology.orchestra.run.smithi082.stdout:(36/146): ceph-volume-18.2.2-1018.g470100a4.el8 1.4 MB/s | 263 kB 00:00 2024-03-30T02:25:11.539 INFO:teuthology.orchestra.run.smithi067.stdout:(5/146): ceph-mds-18.2.2-1018.g470100a4.el8.x86 2.7 MB/s | 2.3 MB 00:00 2024-03-30T02:25:11.543 INFO:teuthology.orchestra.run.smithi082.stdout:(37/146): cephfs-top-18.2.2-1018.g470100a4.el8. 217 kB/s | 18 kB 00:00 2024-03-30T02:25:11.740 INFO:teuthology.orchestra.run.smithi067.stdout:(6/146): ceph-mgr-18.2.2-1018.g470100a4.el8.x86 4.5 MB/s | 1.6 MB 00:00 2024-03-30T02:25:11.780 INFO:teuthology.orchestra.run.smithi082.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 249 kB/s | 59 kB 00:00 2024-03-30T02:25:11.852 INFO:teuthology.orchestra.run.smithi082.stdout:(39/146): gperftools-libs-2.7-9.el8.x86_64.rpm 834 kB/s | 306 kB 00:00 2024-03-30T02:25:11.886 INFO:teuthology.orchestra.run.smithi082.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 711 kB/s | 75 kB 00:00 2024-03-30T02:25:11.944 INFO:teuthology.orchestra.run.smithi082.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 748 kB/s | 44 kB 00:00 2024-03-30T02:25:11.978 INFO:teuthology.orchestra.run.smithi082.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 3.7 MB/s | 476 kB 00:00 2024-03-30T02:25:12.053 INFO:teuthology.orchestra.run.smithi082.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 440 kB/s | 33 kB 00:00 2024-03-30T02:25:12.095 INFO:teuthology.orchestra.run.smithi082.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2024-03-30T02:25:12.120 INFO:teuthology.orchestra.run.smithi082.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 244 kB/s | 16 kB 00:00 2024-03-30T02:25:12.179 INFO:teuthology.orchestra.run.smithi082.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2024-03-30T02:25:12.229 INFO:teuthology.orchestra.run.smithi082.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2024-03-30T02:25:12.255 INFO:teuthology.orchestra.run.smithi082.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2024-03-30T02:25:12.430 INFO:teuthology.orchestra.run.smithi082.stdout:(49/146): ceph-osd-18.2.2-1018.g470100a4.el8.x8 2.4 MB/s | 18 MB 00:07 2024-03-30T02:25:12.455 INFO:teuthology.orchestra.run.smithi082.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 49 kB/s | 11 kB 00:00 2024-03-30T02:25:12.506 INFO:teuthology.orchestra.run.smithi082.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 70 kB/s | 18 kB 00:00 2024-03-30T02:25:12.581 INFO:teuthology.orchestra.run.smithi082.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 389 kB/s | 49 kB 00:00 2024-03-30T02:25:12.639 INFO:teuthology.orchestra.run.smithi082.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2024-03-30T02:25:12.673 INFO:teuthology.orchestra.run.smithi082.stdout:(54/146): python3-more-itertools-7.2.0-3.el8.no 351 kB/s | 59 kB 00:00 2024-03-30T02:25:12.706 INFO:teuthology.orchestra.run.smithi067.stdout:(7/146): ceph-mon-18.2.2-1018.g470100a4.el8.x86 4.1 MB/s | 4.8 MB 00:01 2024-03-30T02:25:12.715 INFO:teuthology.orchestra.run.smithi082.stdout:(55/146): python3-kubernetes-11.0.0-6.el8.noarc 4.1 MB/s | 1.2 MB 00:00 2024-03-30T02:25:12.740 INFO:teuthology.orchestra.run.smithi082.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2024-03-30T02:25:12.782 INFO:teuthology.orchestra.run.smithi082.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 502 kB/s | 34 kB 00:00 2024-03-30T02:25:12.807 INFO:teuthology.orchestra.run.smithi082.stdout:(58/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2024-03-30T02:25:12.841 INFO:teuthology.orchestra.run.smithi082.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2024-03-30T02:25:12.866 INFO:teuthology.orchestra.run.smithi082.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 320 kB/s | 19 kB 00:00 2024-03-30T02:25:12.891 INFO:teuthology.orchestra.run.smithi082.stdout:(61/146): python3-rsa-4.9-2.el8.noarch.rpm 589 kB/s | 64 kB 00:00 2024-03-30T02:25:12.917 INFO:teuthology.orchestra.run.smithi082.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2024-03-30T02:25:12.942 INFO:teuthology.orchestra.run.smithi082.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2024-03-30T02:25:12.975 INFO:teuthology.orchestra.run.smithi082.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 4.3 MB/s | 255 kB 00:00 2024-03-30T02:25:13.001 INFO:teuthology.orchestra.run.smithi082.stdout:(65/146): python3-trustme-0.6.0-4.el8.noarch.rp 252 kB/s | 27 kB 00:00 2024-03-30T02:25:13.035 INFO:teuthology.orchestra.run.smithi082.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2024-03-30T02:25:13.068 INFO:teuthology.orchestra.run.smithi082.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2024-03-30T02:25:13.093 INFO:teuthology.orchestra.run.smithi082.stdout:(68/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.6 MB/s | 251 kB 00:00 2024-03-30T02:25:13.110 INFO:teuthology.orchestra.run.smithi082.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2024-03-30T02:25:13.135 INFO:teuthology.orchestra.run.smithi082.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-03-30T02:25:13.148 INFO:teuthology.orchestra.run.smithi170.stdout:(10/146): ceph-radosgw-18.2.2-1018.g470100a4.el 2.2 MB/s | 13 MB 00:06 2024-03-30T02:25:13.341 INFO:teuthology.orchestra.run.smithi170.stdout:(11/146): cephfs-mirror-18.2.2-1018.g470100a4.e 1.3 MB/s | 261 kB 00:00 2024-03-30T02:25:13.352 INFO:teuthology.orchestra.run.smithi082.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 6.1 MB/s | 1.7 MB 00:00 2024-03-30T02:25:13.369 INFO:teuthology.orchestra.run.smithi082.stdout:(72/146): python3-packaging-16.8-9.el8.noarch.r 186 kB/s | 48 kB 00:00 2024-03-30T02:25:13.386 INFO:teuthology.orchestra.run.smithi082.stdout:(73/146): python3-jsonpointer-1.10-11.el8.noarc 82 kB/s | 20 kB 00:00 2024-03-30T02:25:13.449 INFO:teuthology.orchestra.run.smithi170.stdout:(12/146): libcephfs-devel-18.2.2-1018.g470100a4 286 kB/s | 31 kB 00:00 2024-03-30T02:25:13.545 INFO:teuthology.orchestra.run.smithi082.stdout:(74/146): python3-pyasn1-modules-0.3.7-6.el8.no 629 kB/s | 110 kB 00:00 2024-03-30T02:25:13.753 INFO:teuthology.orchestra.run.smithi082.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 73 kB/s | 27 kB 00:00 2024-03-30T02:25:13.787 INFO:teuthology.orchestra.run.smithi082.stdout:(76/146): python3-prettytable-0.7.2-14.el8.noar 101 kB/s | 44 kB 00:00 2024-03-30T02:25:13.815 INFO:teuthology.orchestra.run.smithi067.stdout:(8/146): ceph-osd-18.2.2-1018.g470100a4.el8.x86 8.9 MB/s | 18 MB 00:02 2024-03-30T02:25:13.841 INFO:teuthology.orchestra.run.smithi170.stdout:(13/146): libcephfs2-18.2.2-1018.g470100a4.el8. 1.9 MB/s | 775 kB 00:00 2024-03-30T02:25:13.895 INFO:teuthology.orchestra.run.smithi067.stdout:(9/146): ceph-selinux-18.2.2-1018.g470100a4.el8 304 kB/s | 24 kB 00:00 2024-03-30T02:25:13.914 INFO:teuthology.orchestra.run.smithi082.stdout:(77/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 342 kB/s | 126 kB 00:00 2024-03-30T02:25:14.029 INFO:teuthology.orchestra.run.smithi082.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 1.6 MB/s | 457 kB 00:00 2024-03-30T02:25:14.033 INFO:teuthology.orchestra.run.smithi170.stdout:(14/146): libcephsqlite-18.2.2-1018.g470100a4.e 741 kB/s | 142 kB 00:00 2024-03-30T02:25:14.096 INFO:teuthology.orchestra.run.smithi082.stdout:(79/146): bison-3.0.4-10.el8.x86_64.rpm 2.2 MB/s | 688 kB 00:00 2024-03-30T02:25:14.133 INFO:teuthology.orchestra.run.smithi170.stdout:(15/146): librados-devel-18.2.2-1018.g470100a4. 1.4 MB/s | 139 kB 00:00 2024-03-30T02:25:14.188 INFO:teuthology.orchestra.run.smithi082.stdout:(80/146): flex-2.6.1-9.el8.x86_64.rpm 1.1 MB/s | 320 kB 00:00 2024-03-30T02:25:14.372 INFO:teuthology.orchestra.run.smithi082.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 2.0 MB/s | 709 kB 00:00 2024-03-30T02:25:14.405 INFO:teuthology.orchestra.run.smithi082.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2024-03-30T02:25:14.472 INFO:teuthology.orchestra.run.smithi082.stdout:(83/146): python3-markupsafe-0.23-19.el8.x86_64 103 kB/s | 39 kB 00:00 2024-03-30T02:25:14.559 INFO:teuthology.orchestra.run.smithi170.stdout:(16/146): ceph-test-18.2.2-1018.g470100a4.el8.x 8.0 MB/s | 47 MB 00:05 2024-03-30T02:25:14.592 INFO:teuthology.orchestra.run.smithi170.stdout:(17/146): libradosstriper1-18.2.2-1018.g470100a 1.0 MB/s | 475 kB 00:00 2024-03-30T02:25:14.692 INFO:teuthology.orchestra.run.smithi170.stdout:(18/146): python3-ceph-argparse-18.2.2-1018.g47 446 kB/s | 45 kB 00:00 2024-03-30T02:25:14.914 INFO:teuthology.orchestra.run.smithi082.stdout:(84/146): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2024-03-30T02:25:14.939 INFO:teuthology.orchestra.run.smithi082.stdout:(85/146): uuid-1.6.2-43.el8.x86_64.rpm 113 kB/s | 64 kB 00:00 2024-03-30T02:25:14.957 INFO:teuthology.orchestra.run.smithi067.stdout:(10/146): ceph-common-18.2.2-1018.g470100a4.el8 3.8 MB/s | 22 MB 00:05 2024-03-30T02:25:14.964 INFO:teuthology.orchestra.run.smithi082.stdout:(86/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 185 kB/s | 103 kB 00:00 2024-03-30T02:25:15.057 INFO:teuthology.orchestra.run.smithi067.stdout:(11/146): cephfs-mirror-18.2.2-1018.g470100a4.e 2.6 MB/s | 261 kB 00:00 2024-03-30T02:25:15.084 INFO:teuthology.orchestra.run.smithi170.stdout:(19/146): python3-ceph-common-18.2.2-1018.g4701 329 kB/s | 129 kB 00:00 2024-03-30T02:25:15.201 INFO:teuthology.orchestra.run.smithi170.stdout:(20/146): python3-cephfs-18.2.2-1018.g470100a4. 1.6 MB/s | 196 kB 00:00 2024-03-30T02:25:15.215 INFO:teuthology.orchestra.run.smithi082.stdout:(87/146): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2024-03-30T02:25:15.215 INFO:teuthology.orchestra.run.smithi067.stdout:(12/146): ceph-radosgw-18.2.2-1018.g470100a4.el 5.2 MB/s | 13 MB 00:02 2024-03-30T02:25:15.233 INFO:teuthology.orchestra.run.smithi067.stdout:(13/146): libcephfs-devel-18.2.2-1018.g470100a4 177 kB/s | 31 kB 00:00 2024-03-30T02:25:15.257 INFO:teuthology.orchestra.run.smithi082.stdout:(88/146): python3-jinja2-2.10.1-3.el8.noarch.rp 1.5 MB/s | 538 kB 00:00 2024-03-30T02:25:15.316 INFO:teuthology.orchestra.run.smithi067.stdout:(14/146): libcephsqlite-18.2.2-1018.g470100a4.e 1.7 MB/s | 142 kB 00:00 2024-03-30T02:25:15.332 INFO:teuthology.orchestra.run.smithi082.stdout:(89/146): socat-1.7.4.1-1.el8.x86_64.rpm 880 kB/s | 323 kB 00:00 2024-03-30T02:25:15.418 INFO:teuthology.orchestra.run.smithi170.stdout:(21/146): python3-rados-18.2.2-1018.g470100a4.e 1.7 MB/s | 371 kB 00:00 2024-03-30T02:25:15.433 INFO:teuthology.orchestra.run.smithi067.stdout:(15/146): librados-devel-18.2.2-1018.g470100a4. 1.2 MB/s | 139 kB 00:00 2024-03-30T02:25:15.440 INFO:teuthology.orchestra.run.smithi082.stdout:(90/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 999 kB/s | 225 kB 00:00 2024-03-30T02:25:15.467 INFO:teuthology.orchestra.run.smithi067.stdout:(16/146): libcephfs2-18.2.2-1018.g470100a4.el8. 3.0 MB/s | 775 kB 00:00 2024-03-30T02:25:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.5 MB/s | 354 kB 00:00 2024-03-30T02:25:15.600 INFO:teuthology.orchestra.run.smithi067.stdout:(17/146): libradosstriper1-18.2.2-1018.g470100a 2.8 MB/s | 475 kB 00:00 2024-03-30T02:25:15.627 INFO:teuthology.orchestra.run.smithi170.stdout:(22/146): python3-rbd-18.2.2-1018.g470100a4.el8 1.7 MB/s | 366 kB 00:00 2024-03-30T02:25:15.676 INFO:teuthology.orchestra.run.smithi067.stdout:(18/146): python3-ceph-argparse-18.2.2-1018.g47 596 kB/s | 45 kB 00:00 2024-03-30T02:25:15.691 INFO:teuthology.orchestra.run.smithi082.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 15 MB/s | 5.2 MB 00:00 2024-03-30T02:25:15.716 INFO:teuthology.orchestra.run.smithi082.stdout:(93/146): python3-mako-1.0.6-14.el8.noarch.rpm 570 kB/s | 157 kB 00:00 2024-03-30T02:25:15.727 INFO:teuthology.orchestra.run.smithi170.stdout:(23/146): python3-rgw-18.2.2-1018.g470100a4.el8 1.0 MB/s | 107 kB 00:00 2024-03-30T02:25:15.793 INFO:teuthology.orchestra.run.smithi067.stdout:(19/146): python3-ceph-common-18.2.2-1018.g4701 1.1 MB/s | 129 kB 00:00 2024-03-30T02:25:15.819 INFO:teuthology.orchestra.run.smithi170.stdout:(24/146): rbd-fuse-18.2.2-1018.g470100a4.el8.x8 1.0 MB/s | 95 kB 00:00 2024-03-30T02:25:15.825 INFO:teuthology.orchestra.run.smithi082.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-03-30T02:25:15.886 INFO:teuthology.orchestra.run.smithi067.stdout:(20/146): python3-cephfs-18.2.2-1018.g470100a4. 2.1 MB/s | 196 kB 00:00 2024-03-30T02:25:15.933 INFO:teuthology.orchestra.run.smithi082.stdout:(95/146): automake-1.16.1-8.el8.noarch.rpm 2.9 MB/s | 714 kB 00:00 2024-03-30T02:25:15.959 INFO:teuthology.orchestra.run.smithi082.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 817 kB/s | 198 kB 00:00 2024-03-30T02:25:15.961 INFO:teuthology.orchestra.run.smithi170.stdout:(25/146): librgw2-18.2.2-1018.g470100a4.el8.x86 4.7 MB/s | 6.5 MB 00:01 2024-03-30T02:25:16.012 INFO:teuthology.orchestra.run.smithi067.stdout:(21/146): python3-rados-18.2.2-1018.g470100a4.e 2.9 MB/s | 371 kB 00:00 2024-03-30T02:25:16.042 INFO:teuthology.orchestra.run.smithi082.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 932 kB/s | 202 kB 00:00 2024-03-30T02:25:16.044 INFO:teuthology.orchestra.run.smithi170.stdout:(26/146): rbd-nbd-18.2.2-1018.g470100a4.el8.x86 2.1 MB/s | 181 kB 00:00 2024-03-30T02:25:16.120 INFO:teuthology.orchestra.run.smithi170.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1018.g 310 kB/s | 23 kB 00:00 2024-03-30T02:25:16.195 INFO:teuthology.orchestra.run.smithi170.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1018.g470100a 1.8 MB/s | 138 kB 00:00 2024-03-30T02:25:16.209 INFO:teuthology.orchestra.run.smithi082.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 197 kB/s | 54 kB 00:00 2024-03-30T02:25:16.288 INFO:teuthology.orchestra.run.smithi067.stdout:(22/146): python3-rbd-18.2.2-1018.g470100a4.el8 1.3 MB/s | 366 kB 00:00 2024-03-30T02:25:16.294 INFO:teuthology.orchestra.run.smithi082.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 752 kB/s | 188 kB 00:00 2024-03-30T02:25:16.510 INFO:teuthology.orchestra.run.smithi082.stdout:(100/146): python3-scipy-1.0.0-21.module+el8.9. 25 MB/s | 14 MB 00:00 2024-03-30T02:25:16.513 INFO:teuthology.orchestra.run.smithi067.stdout:(23/146): python3-rgw-18.2.2-1018.g470100a4.el8 477 kB/s | 107 kB 00:00 2024-03-30T02:25:16.529 INFO:teuthology.orchestra.run.smithi170.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1018.g47010 5.4 MB/s | 1.8 MB 00:00 2024-03-30T02:25:16.552 INFO:teuthology.orchestra.run.smithi082.stdout:(101/146): postgresql-contrib-10.23-4.module+el 3.1 MB/s | 812 kB 00:00 2024-03-30T02:25:16.585 INFO:teuthology.orchestra.run.smithi082.stdout:(102/146): postgresql-10.23-4.module+el8.9.0+21 4.0 MB/s | 1.5 MB 00:00 2024-03-30T02:25:16.605 INFO:teuthology.orchestra.run.smithi067.stdout:(24/146): rbd-fuse-18.2.2-1018.g470100a4.el8.x8 1.0 MB/s | 95 kB 00:00 2024-03-30T02:25:16.810 INFO:teuthology.orchestra.run.smithi082.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 69 kB/s | 18 kB 00:00 2024-03-30T02:25:16.902 INFO:teuthology.orchestra.run.smithi082.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 134 kB/s | 43 kB 00:00 2024-03-30T02:25:16.905 INFO:teuthology.orchestra.run.smithi067.stdout:(25/146): librgw2-18.2.2-1018.g470100a4.el8.x86 4.6 MB/s | 6.5 MB 00:01 2024-03-30T02:25:16.994 INFO:teuthology.orchestra.run.smithi082.stdout:(105/146): postgresql-server-10.23-4.module+el8 10 MB/s | 5.1 MB 00:00 2024-03-30T02:25:17.022 INFO:teuthology.orchestra.run.smithi067.stdout:(26/146): rbd-nbd-18.2.2-1018.g470100a4.el8.x86 1.5 MB/s | 181 kB 00:00 2024-03-30T02:25:17.122 INFO:teuthology.orchestra.run.smithi067.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1018.g 233 kB/s | 23 kB 00:00 2024-03-30T02:25:17.253 INFO:teuthology.orchestra.run.smithi082.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 351 kB/s | 155 kB 00:00 2024-03-30T02:25:17.378 INFO:teuthology.orchestra.run.smithi082.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 230 kB/s | 109 kB 00:00 2024-03-30T02:25:17.387 INFO:teuthology.orchestra.run.smithi170.stdout:(30/146): rbd-mirror-18.2.2-1018.g470100a4.el8. 2.2 MB/s | 3.5 MB 00:01 2024-03-30T02:25:17.429 INFO:teuthology.orchestra.run.smithi082.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 98 kB/s | 43 kB 00:00 2024-03-30T02:25:17.454 INFO:teuthology.orchestra.run.smithi082.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 93 kB/s | 19 kB 00:00 2024-03-30T02:25:17.496 INFO:teuthology.orchestra.run.smithi170.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1018.g47 2.2 MB/s | 245 kB 00:00 2024-03-30T02:25:17.588 INFO:teuthology.orchestra.run.smithi170.stdout:(32/146): ceph-mgr-diskprediction-local-18.2.2- 7.0 MB/s | 7.4 MB 00:01 2024-03-30T02:25:17.597 INFO:teuthology.orchestra.run.smithi067.stdout:(28/146): ceph-test-18.2.2-1018.g470100a4.el8.x 13 MB/s | 47 MB 00:03 2024-03-30T02:25:17.604 INFO:teuthology.orchestra.run.smithi082.stdout:(110/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2024-03-30T02:25:17.613 INFO:teuthology.orchestra.run.smithi170.stdout:(33/146): ceph-mgr-rook-18.2.2-1018.g470100a4.e 419 kB/s | 49 kB 00:00 2024-03-30T02:25:17.622 INFO:teuthology.orchestra.run.smithi067.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1018.g470100a 276 kB/s | 138 kB 00:00 2024-03-30T02:25:17.637 INFO:teuthology.orchestra.run.smithi082.stdout:(111/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 169 kB/s | 44 kB 00:00 2024-03-30T02:25:17.638 INFO:teuthology.orchestra.run.smithi170.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1018.g4 295 kB/s | 15 kB 00:00 2024-03-30T02:25:17.671 INFO:teuthology.orchestra.run.smithi082.stdout:(112/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.4 MB/s | 605 kB 00:00 2024-03-30T02:25:17.681 INFO:teuthology.orchestra.run.smithi067.stdout:(30/146): rbd-mirror-18.2.2-1018.g470100a4.el8. 3.2 MB/s | 3.5 MB 00:01 2024-03-30T02:25:17.689 INFO:teuthology.orchestra.run.smithi170.stdout:(35/146): cephadm-18.2.2-1018.g470100a4.el8.noa 3.4 MB/s | 173 kB 00:00 2024-03-30T02:25:17.723 INFO:teuthology.orchestra.run.smithi170.stdout:(36/146): ceph-volume-18.2.2-1018.g470100a4.el8 2.4 MB/s | 263 kB 00:00 2024-03-30T02:25:17.765 INFO:teuthology.orchestra.run.smithi170.stdout:(37/146): cephfs-top-18.2.2-1018.g470100a4.el8. 241 kB/s | 18 kB 00:00 2024-03-30T02:25:17.821 INFO:teuthology.orchestra.run.smithi082.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.0 MB/s | 220 kB 00:00 2024-03-30T02:25:17.823 INFO:teuthology.orchestra.run.smithi067.stdout:(31/146): ceph-mgr-dashboard-18.2.2-1018.g47010 7.9 MB/s | 1.8 MB 00:00 2024-03-30T02:25:17.848 INFO:teuthology.orchestra.run.smithi067.stdout:(32/146): ceph-mgr-modules-core-18.2.2-1018.g47 1.4 MB/s | 245 kB 00:00 2024-03-30T02:25:17.865 INFO:teuthology.orchestra.run.smithi067.stdout:(33/146): ceph-mgr-rook-18.2.2-1018.g470100a4.e 1.2 MB/s | 49 kB 00:00 2024-03-30T02:25:17.882 INFO:teuthology.orchestra.run.smithi067.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1018.g4 442 kB/s | 15 kB 00:00 2024-03-30T02:25:17.923 INFO:teuthology.orchestra.run.smithi170.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 372 kB/s | 59 kB 00:00 2024-03-30T02:25:17.924 INFO:teuthology.orchestra.run.smithi067.stdout:(35/146): ceph-volume-18.2.2-1018.g470100a4.el8 4.4 MB/s | 263 kB 00:00 2024-03-30T02:25:17.950 INFO:teuthology.orchestra.run.smithi067.stdout:(36/146): cephadm-18.2.2-1018.g470100a4.el8.noa 2.5 MB/s | 173 kB 00:00 2024-03-30T02:25:17.963 INFO:teuthology.orchestra.run.smithi082.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 165 kB/s | 48 kB 00:00 2024-03-30T02:25:17.966 INFO:teuthology.orchestra.run.smithi067.stdout:(37/146): cephfs-top-18.2.2-1018.g470100a4.el8. 433 kB/s | 18 kB 00:00 2024-03-30T02:25:17.989 INFO:teuthology.orchestra.run.smithi082.stdout:(115/146): libselinux-devel-2.9-5.el8.x86_64.rp 570 kB/s | 200 kB 00:00 2024-03-30T02:25:17.990 INFO:teuthology.orchestra.run.smithi170.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2024-03-30T02:25:18.015 INFO:teuthology.orchestra.run.smithi170.stdout:(40/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2024-03-30T02:25:18.030 INFO:teuthology.orchestra.run.smithi082.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 417 kB/s | 87 kB 00:00 2024-03-30T02:25:18.057 INFO:teuthology.orchestra.run.smithi170.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2024-03-30T02:25:18.141 INFO:teuthology.orchestra.run.smithi170.stdout:(42/146): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2024-03-30T02:25:18.164 INFO:teuthology.orchestra.run.smithi082.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 220 kB/s | 39 kB 00:00 2024-03-30T02:25:18.166 INFO:teuthology.orchestra.run.smithi170.stdout:(43/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.6 MB/s | 476 kB 00:00 2024-03-30T02:25:18.191 INFO:teuthology.orchestra.run.smithi170.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 657 kB/s | 33 kB 00:00 2024-03-30T02:25:18.192 INFO:teuthology.orchestra.run.smithi067.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 261 kB/s | 59 kB 00:00 2024-03-30T02:25:18.206 INFO:teuthology.orchestra.run.smithi082.stdout:(118/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 238 kB/s | 58 kB 00:00 2024-03-30T02:25:18.217 INFO:teuthology.orchestra.run.smithi170.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2024-03-30T02:25:18.231 INFO:teuthology.orchestra.run.smithi082.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 414 kB/s | 83 kB 00:00 2024-03-30T02:25:18.265 INFO:teuthology.orchestra.run.smithi082.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 6.3 MB/s | 645 kB 00:00 2024-03-30T02:25:18.317 INFO:teuthology.orchestra.run.smithi170.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2024-03-30T02:25:18.323 INFO:teuthology.orchestra.run.smithi082.stdout:(121/146): libuuid-devel-2.32.1-43.el8.x86_64.r 1.1 MB/s | 100 kB 00:00 2024-03-30T02:25:18.350 INFO:teuthology.orchestra.run.smithi067.stdout:(39/146): libunwind-1.3.1-3.el8.x86_64.rpm 475 kB/s | 75 kB 00:00 2024-03-30T02:25:18.390 INFO:teuthology.orchestra.run.smithi082.stdout:(122/146): libquadmath-8.5.0-20.el8.x86_64.rpm 937 kB/s | 172 kB 00:00 2024-03-30T02:25:18.401 INFO:teuthology.orchestra.run.smithi170.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 976 kB/s | 81 kB 00:00 2024-03-30T02:25:18.442 INFO:teuthology.orchestra.run.smithi067.stdout:(40/146): gperftools-libs-2.7-9.el8.x86_64.rpm 622 kB/s | 306 kB 00:00 2024-03-30T02:25:18.457 INFO:teuthology.orchestra.run.smithi082.stdout:(123/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2024-03-30T02:25:18.468 INFO:teuthology.orchestra.run.smithi170.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2024-03-30T02:25:18.491 INFO:teuthology.orchestra.run.smithi082.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 435 kB/s | 73 kB 00:00 2024-03-30T02:25:18.493 INFO:teuthology.orchestra.run.smithi170.stdout:(49/146): python3-cherrypy-18.4.0-1.el8.noarch. 1.4 MB/s | 384 kB 00:00 2024-03-30T02:25:18.516 INFO:teuthology.orchestra.run.smithi082.stdout:(125/146): NetworkManager-initscripts-updown-1. 1.1 MB/s | 144 kB 00:00 2024-03-30T02:25:18.535 INFO:teuthology.orchestra.run.smithi170.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2024-03-30T02:25:18.558 INFO:teuthology.orchestra.run.smithi082.stdout:(126/146): elfutils-libelf-devel-0.189-3.el8.x8 616 kB/s | 62 kB 00:00 2024-03-30T02:25:18.576 INFO:teuthology.orchestra.run.smithi067.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 328 kB/s | 44 kB 00:00 2024-03-30T02:25:18.594 INFO:teuthology.orchestra.run.smithi170.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 836 kB/s | 49 kB 00:00 2024-03-30T02:25:18.666 INFO:teuthology.orchestra.run.smithi082.stdout:(127/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.1 MB/s | 562 kB 00:00 2024-03-30T02:25:18.692 INFO:teuthology.orchestra.run.smithi082.stdout:(128/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 352 kB/s | 47 kB 00:00 2024-03-30T02:25:18.718 INFO:teuthology.orchestra.run.smithi067.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 1.3 MB/s | 476 kB 00:00 2024-03-30T02:25:18.803 INFO:teuthology.orchestra.run.smithi170.stdout:(52/146): ceph-common-18.2.2-1018.g470100a4.el8 1.4 MB/s | 22 MB 00:15 2024-03-30T02:25:18.828 INFO:teuthology.orchestra.run.smithi170.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 251 kB/s | 59 kB 00:00 2024-03-30T02:25:18.829 INFO:teuthology.orchestra.run.smithi082.stdout:(129/146): openssl-devel-1.1.1k-12.el8_9.x86_64 7.4 MB/s | 2.3 MB 00:00 2024-03-30T02:25:18.843 INFO:teuthology.orchestra.run.smithi067.stdout:(43/146): ceph-mgr-diskprediction-local-18.2.2- 6.1 MB/s | 7.4 MB 00:01 2024-03-30T02:25:18.868 INFO:teuthology.orchestra.run.smithi082.stdout:(130/146): python3-cryptography-3.2.1-7.el8_9.x 2.7 MB/s | 559 kB 00:00 2024-03-30T02:25:18.869 INFO:teuthology.orchestra.run.smithi067.stdout:(44/146): python3-beautifulsoup4-4.6.3-2.el8.1. 634 kB/s | 185 kB 00:00 2024-03-30T02:25:18.894 INFO:teuthology.orchestra.run.smithi067.stdout:(45/146): python3-cachetools-3.1.1-4.el8.noarch 188 kB/s | 33 kB 00:00 2024-03-30T02:25:18.970 INFO:teuthology.orchestra.run.smithi170.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 362 kB/s | 60 kB 00:00 2024-03-30T02:25:18.994 INFO:teuthology.orchestra.run.smithi067.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2024-03-30T02:25:19.020 INFO:teuthology.orchestra.run.smithi170.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 314 kB/s | 16 kB 00:00 2024-03-30T02:25:19.028 INFO:teuthology.orchestra.run.smithi067.stdout:(47/146): python3-certifi-2018.10.15-7.el8.noar 89 kB/s | 16 kB 00:00 2024-03-30T02:25:19.070 INFO:teuthology.orchestra.run.smithi067.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2024-03-30T02:25:19.070 INFO:teuthology.orchestra.run.smithi170.stdout:(56/146): python3-kubernetes-11.0.0-6.el8.noarc 2.0 MB/s | 1.2 MB 00:00 2024-03-30T02:25:19.093 INFO:teuthology.orchestra.run.smithi082.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.0 MB/s | 232 kB 00:00 2024-03-30T02:25:19.095 INFO:teuthology.orchestra.run.smithi067.stdout:(49/146): python3-cherrypy-18.4.0-1.el8.noarch. 1.9 MB/s | 384 kB 00:00 2024-03-30T02:25:19.095 INFO:teuthology.orchestra.run.smithi170.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 448 kB/s | 34 kB 00:00 2024-03-30T02:25:19.120 INFO:teuthology.orchestra.run.smithi067.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 120 kB/s | 11 kB 00:00 2024-03-30T02:25:19.137 INFO:teuthology.orchestra.run.smithi170.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2024-03-30T02:25:19.145 INFO:teuthology.orchestra.run.smithi067.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 232 kB/s | 18 kB 00:00 2024-03-30T02:25:19.188 INFO:teuthology.orchestra.run.smithi170.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2024-03-30T02:25:19.205 INFO:teuthology.orchestra.run.smithi170.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 279 kB/s | 19 kB 00:00 2024-03-30T02:25:19.221 INFO:teuthology.orchestra.run.smithi067.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 784 kB/s | 59 kB 00:00 2024-03-30T02:25:19.243 INFO:teuthology.orchestra.run.smithi082.stdout:(132/146): librbd1-18.2.2-1018.g470100a4.el8.x8 8.6 MB/s | 3.6 MB 00:00 2024-03-30T02:25:19.246 INFO:teuthology.orchestra.run.smithi067.stdout:(53/146): python3-logutils-0.3.5-11.el8.noarch. 388 kB/s | 49 kB 00:00 2024-03-30T02:25:19.270 INFO:teuthology.orchestra.run.smithi170.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 369 kB/s | 24 kB 00:00 2024-03-30T02:25:19.289 INFO:teuthology.orchestra.run.smithi170.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 351 kB/s | 29 kB 00:00 2024-03-30T02:25:19.321 INFO:teuthology.orchestra.run.smithi067.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 603 kB/s | 60 kB 00:00 2024-03-30T02:25:19.335 INFO:teuthology.orchestra.run.smithi082.stdout:(133/146): librados2-18.2.2-1018.g470100a4.el8. 5.7 MB/s | 3.7 MB 00:00 2024-03-30T02:25:19.339 INFO:teuthology.orchestra.run.smithi170.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 402 kB/s | 27 kB 00:00 2024-03-30T02:25:19.362 INFO:teuthology.orchestra.run.smithi082.stdout:(134/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.1 MB/s | 303 kB 00:00 2024-03-30T02:25:19.364 INFO:teuthology.orchestra.run.smithi170.stdout:(64/146): python3-pecan-1.3.2-9.el8.noarch.rpm 532 kB/s | 283 kB 00:00 2024-03-30T02:25:19.388 INFO:teuthology.orchestra.run.smithi067.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 235 kB/s | 16 kB 00:00 2024-03-30T02:25:19.448 INFO:teuthology.orchestra.run.smithi170.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2024-03-30T02:25:19.463 INFO:teuthology.orchestra.run.smithi067.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 448 kB/s | 34 kB 00:00 2024-03-30T02:25:19.481 INFO:teuthology.orchestra.run.smithi170.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 524 kB/s | 61 kB 00:00 2024-03-30T02:25:19.489 INFO:teuthology.orchestra.run.smithi067.stdout:(57/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2024-03-30T02:25:19.523 INFO:teuthology.orchestra.run.smithi170.stdout:(67/146): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2024-03-30T02:25:19.539 INFO:teuthology.orchestra.run.smithi067.stdout:(58/146): python3-kubernetes-11.0.0-6.el8.noarc 2.6 MB/s | 1.2 MB 00:00 2024-03-30T02:25:19.540 INFO:teuthology.orchestra.run.smithi170.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2024-03-30T02:25:19.543 INFO:teuthology.orchestra.run.smithi082.stdout:(135/146): libmount-2.32.1-43.el8.x86_64.rpm 788 kB/s | 237 kB 00:00 2024-03-30T02:25:19.557 INFO:teuthology.orchestra.run.smithi170.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2024-03-30T02:25:19.566 INFO:teuthology.orchestra.run.smithi067.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 853 kB/s | 64 kB 00:00 2024-03-30T02:25:19.577 INFO:teuthology.orchestra.run.smithi082.stdout:(136/146): libsmartcols-2.32.1-43.el8.x86_64.rp 742 kB/s | 179 kB 00:00 2024-03-30T02:25:19.606 INFO:teuthology.orchestra.run.smithi067.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 281 kB/s | 19 kB 00:00 2024-03-30T02:25:19.627 INFO:teuthology.orchestra.run.smithi082.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 370 kB/s | 99 kB 00:00 2024-03-30T02:25:19.631 INFO:teuthology.orchestra.run.smithi067.stdout:(61/146): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2024-03-30T02:25:19.648 INFO:teuthology.orchestra.run.smithi067.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2024-03-30T02:25:19.682 INFO:teuthology.orchestra.run.smithi170.stdout:(70/146): python3-packaging-16.8-9.el8.noarch.r 386 kB/s | 48 kB 00:00 2024-03-30T02:25:19.707 INFO:teuthology.orchestra.run.smithi067.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 292 kB/s | 29 kB 00:00 2024-03-30T02:25:19.732 INFO:teuthology.orchestra.run.smithi067.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 275 kB/s | 27 kB 00:00 2024-03-30T02:25:19.782 INFO:teuthology.orchestra.run.smithi067.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2024-03-30T02:25:19.791 INFO:teuthology.orchestra.run.smithi170.stdout:(71/146): python3-waitress-1.4.3-1.el8.noarch.r 508 kB/s | 255 kB 00:00 2024-03-30T02:25:19.844 INFO:teuthology.orchestra.run.smithi082.stdout:(138/146): util-linux-2.32.1-43.el8.x86_64.rpm 8.2 MB/s | 2.5 MB 00:00 2024-03-30T02:25:19.849 INFO:teuthology.orchestra.run.smithi067.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 524 kB/s | 61 kB 00:00 2024-03-30T02:25:19.869 INFO:teuthology.orchestra.run.smithi082.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 351 kB/s | 103 kB 00:00 2024-03-30T02:25:19.883 INFO:teuthology.orchestra.run.smithi067.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.4 MB/s | 251 kB 00:00 2024-03-30T02:25:19.891 INFO:teuthology.orchestra.run.smithi170.stdout:(72/146): python3-jsonpointer-1.10-11.el8.noarc 96 kB/s | 20 kB 00:00 2024-03-30T02:25:19.903 INFO:teuthology.orchestra.run.smithi082.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.0 MB/s | 854 kB 00:00 2024-03-30T02:25:19.908 INFO:teuthology.orchestra.run.smithi067.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 680 kB/s | 85 kB 00:00 2024-03-30T02:25:19.916 INFO:teuthology.orchestra.run.smithi170.stdout:(73/146): python3-prettytable-0.7.2-14.el8.noar 351 kB/s | 44 kB 00:00 2024-03-30T02:25:19.933 INFO:teuthology.orchestra.run.smithi067.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 274 kB/s | 23 kB 00:00 2024-03-30T02:25:19.959 INFO:teuthology.orchestra.run.smithi067.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-03-30T02:25:19.992 INFO:teuthology.orchestra.run.smithi170.stdout:(74/146): thrift-0.13.0-2.el8.x86_64.rpm 3.7 MB/s | 1.7 MB 00:00 2024-03-30T02:25:20.128 INFO:teuthology.orchestra.run.smithi082.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 778 kB/s | 220 kB 00:00 2024-03-30T02:25:20.133 INFO:teuthology.orchestra.run.smithi170.stdout:(75/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 893 kB/s | 126 kB 00:00 2024-03-30T02:25:20.178 INFO:teuthology.orchestra.run.smithi170.stdout:(76/146): python3-pyasn1-modules-0.3.7-6.el8.no 388 kB/s | 110 kB 00:00 2024-03-30T02:25:20.209 INFO:teuthology.orchestra.run.smithi170.stdout:(77/146): python3-jsonpatch-1.21-2.el8.noarch.r 91 kB/s | 27 kB 00:00 2024-03-30T02:25:20.292 INFO:teuthology.orchestra.run.smithi067.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 56 kB/s | 20 kB 00:00 2024-03-30T02:25:20.293 INFO:teuthology.orchestra.run.smithi082.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 600 kB/s | 254 kB 00:00 2024-03-30T02:25:20.309 INFO:teuthology.orchestra.run.smithi170.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.5 MB/s | 457 kB 00:00 2024-03-30T02:25:20.343 INFO:teuthology.orchestra.run.smithi067.stdout:(72/146): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2024-03-30T02:25:20.360 INFO:teuthology.orchestra.run.smithi170.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2024-03-30T02:25:20.393 INFO:teuthology.orchestra.run.smithi067.stdout:(73/146): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2024-03-30T02:25:20.429 INFO:teuthology.orchestra.run.smithi082.stdout:(143/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 4.9 MB/s | 1.5 MB 00:00 2024-03-30T02:25:20.468 INFO:teuthology.orchestra.run.smithi067.stdout:(74/146): python3-prettytable-0.7.2-14.el8.noar 88 kB/s | 44 kB 00:00 2024-03-30T02:25:20.471 INFO:teuthology.orchestra.run.smithi082.stdout:(144/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 1.2 MB/s | 711 kB 00:00 2024-03-30T02:25:20.486 INFO:teuthology.orchestra.run.smithi170.stdout:(80/146): bison-3.0.4-10.el8.x86_64.rpm 2.2 MB/s | 688 kB 00:00 2024-03-30T02:25:20.493 INFO:teuthology.orchestra.run.smithi067.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 176 kB/s | 27 kB 00:00 2024-03-30T02:25:20.518 INFO:teuthology.orchestra.run.smithi067.stdout:(76/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2024-03-30T02:25:20.552 INFO:teuthology.orchestra.run.smithi170.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 202 kB/s | 39 kB 00:00 2024-03-30T02:25:20.571 INFO:teuthology.orchestra.run.smithi082.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.2 MB/s | 333 kB 00:00 2024-03-30T02:25:20.586 INFO:teuthology.orchestra.run.smithi170.stdout:(82/146): libtool-2.4.6-25.el8.x86_64.rpm 2.5 MB/s | 709 kB 00:00 2024-03-30T02:25:20.611 INFO:teuthology.orchestra.run.smithi170.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-03-30T02:25:20.635 INFO:teuthology.orchestra.run.smithi067.stdout:(77/146): bison-3.0.4-10.el8.x86_64.rpm 4.7 MB/s | 688 kB 00:00 2024-03-30T02:25:20.669 INFO:teuthology.orchestra.run.smithi067.stdout:(78/146): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2024-03-30T02:25:20.703 INFO:teuthology.orchestra.run.smithi170.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 425 kB/s | 64 kB 00:00 2024-03-30T02:25:20.728 INFO:teuthology.orchestra.run.smithi170.stdout:(85/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 728 kB/s | 103 kB 00:00 2024-03-30T02:25:20.736 INFO:teuthology.orchestra.run.smithi067.stdout:(79/146): python3-werkzeug-0.12.2-4.el8.noarch. 1.7 MB/s | 457 kB 00:00 2024-03-30T02:25:20.772 INFO:teuthology.orchestra.run.smithi082.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 3.1 MB/s | 1.1 MB 00:00 2024-03-30T02:25:20.774 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-03-30T02:25:20.775 INFO:teuthology.orchestra.run.smithi082.stdout:Total 12 MB/s | 213 MB 00:17 2024-03-30T02:25:20.778 INFO:teuthology.orchestra.run.smithi067.stdout:(80/146): python3-markupsafe-0.23-19.el8.x86_64 357 kB/s | 39 kB 00:00 2024-03-30T02:25:20.803 INFO:teuthology.orchestra.run.smithi170.stdout:(86/146): python3-jinja2-2.10.1-3.el8.noarch.rp 5.2 MB/s | 538 kB 00:00 2024-03-30T02:25:20.845 INFO:teuthology.orchestra.run.smithi067.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 3.3 MB/s | 709 kB 00:00 2024-03-30T02:25:20.887 INFO:teuthology.orchestra.run.smithi170.stdout:(87/146): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2024-03-30T02:25:20.904 INFO:teuthology.orchestra.run.smithi067.stdout:(82/146): uuid-1.6.2-43.el8.x86_64.rpm 512 kB/s | 64 kB 00:00 2024-03-30T02:25:20.938 INFO:teuthology.orchestra.run.smithi067.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2024-03-30T02:25:20.979 INFO:teuthology.orchestra.run.smithi170.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2024-03-30T02:25:21.060 INFO:teuthology.orchestra.run.smithi170.stdout:(89/146): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2024-03-30T02:25:21.079 INFO:teuthology.orchestra.run.smithi067.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 442 kB/s | 103 kB 00:00 2024-03-30T02:25:21.138 INFO:teuthology.orchestra.run.smithi170.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2024-03-30T02:25:21.164 INFO:teuthology.orchestra.run.smithi170.stdout:(91/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 815 kB/s | 225 kB 00:00 2024-03-30T02:25:21.246 INFO:teuthology.orchestra.run.smithi067.stdout:(85/146): python3-jinja2-2.10.1-3.el8.noarch.rp 1.7 MB/s | 538 kB 00:00 2024-03-30T02:25:21.272 INFO:teuthology.orchestra.run.smithi170.stdout:(92/146): openblas-threads-0.3.15-6.el8.x86_64. 25 MB/s | 5.2 MB 00:00 2024-03-30T02:25:21.297 INFO:teuthology.orchestra.run.smithi170.stdout:(93/146): python3-mako-1.0.6-14.el8.noarch.rpm 990 kB/s | 157 kB 00:00 2024-03-30T02:25:21.355 INFO:teuthology.orchestra.run.smithi067.stdout:(86/146): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2024-03-30T02:25:21.397 INFO:teuthology.orchestra.run.smithi067.stdout:(87/146): socat-1.7.4.1-1.el8.x86_64.rpm 2.1 MB/s | 323 kB 00:00 2024-03-30T02:25:21.397 INFO:teuthology.orchestra.run.smithi170.stdout:(94/146): openblas-0.3.15-6.el8.x86_64.rpm 22 MB/s | 5.0 MB 00:00 2024-03-30T02:25:21.423 INFO:teuthology.orchestra.run.smithi170.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 1.5 MB/s | 198 kB 00:00 2024-03-30T02:25:21.456 INFO:teuthology.orchestra.run.smithi170.stdout:(96/146): automake-1.16.1-8.el8.noarch.rpm 3.8 MB/s | 714 kB 00:00 2024-03-30T02:25:21.480 INFO:teuthology.orchestra.run.smithi067.stdout:(88/146): python3-babel-2.5.1-7.el8.noarch.rpm 8.3 MB/s | 4.8 MB 00:00 2024-03-30T02:25:21.506 INFO:teuthology.orchestra.run.smithi067.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.5 MB/s | 225 kB 00:00 2024-03-30T02:25:21.557 INFO:teuthology.orchestra.run.smithi170.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 406 kB/s | 54 kB 00:00 2024-03-30T02:25:21.598 INFO:teuthology.orchestra.run.smithi067.stdout:(90/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.7 MB/s | 354 kB 00:00 2024-03-30T02:25:21.598 INFO:teuthology.orchestra.run.smithi170.stdout:(98/146): jq-1.6-7.el8.x86_64.rpm 1.0 MB/s | 202 kB 00:00 2024-03-30T02:25:21.674 INFO:teuthology.orchestra.run.smithi170.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-03-30T02:25:21.791 INFO:teuthology.orchestra.run.smithi170.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 7.8 MB/s | 1.5 MB 00:00 2024-03-30T02:25:21.806 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-03-30T02:25:21.833 INFO:teuthology.orchestra.run.smithi170.stdout:(101/146): postgresql-contrib-10.23-4.module+el 5.0 MB/s | 812 kB 00:00 2024-03-30T02:25:21.875 INFO:teuthology.orchestra.run.smithi067.stdout:(91/146): python3-mako-1.0.6-14.el8.noarch.rpm 426 kB/s | 157 kB 00:00 2024-03-30T02:25:22.008 INFO:teuthology.orchestra.run.smithi170.stdout:(102/146): postgresql-server-10.23-4.module+el8 23 MB/s | 5.1 MB 00:00 2024-03-30T02:25:22.015 INFO:teuthology.orchestra.run.smithi067.stdout:(92/146): openblas-0.3.15-6.el8.x86_64.rpm 12 MB/s | 5.0 MB 00:00 2024-03-30T02:25:22.033 INFO:teuthology.orchestra.run.smithi170.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 89 kB/s | 18 kB 00:00 2024-03-30T02:25:22.090 INFO:teuthology.orchestra.run.smithi067.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 8.5 MB/s | 5.2 MB 00:00 2024-03-30T02:25:22.124 INFO:teuthology.orchestra.run.smithi067.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 2.8 MB/s | 714 kB 00:00 2024-03-30T02:25:22.149 INFO:teuthology.orchestra.run.smithi067.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2024-03-30T02:25:22.183 INFO:teuthology.orchestra.run.smithi170.stdout:(104/146): python3-scipy-1.0.0-21.module+el8.9. 19 MB/s | 14 MB 00:00 2024-03-30T02:25:22.198 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-03-30T02:25:22.198 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-03-30T02:25:22.207 INFO:teuthology.orchestra.run.smithi067.stdout:(96/146): jq-1.6-7.el8.x86_64.rpm 1.7 MB/s | 202 kB 00:00 2024-03-30T02:25:22.209 INFO:teuthology.orchestra.run.smithi170.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 213 kB/s | 43 kB 00:00 2024-03-30T02:25:22.234 INFO:teuthology.orchestra.run.smithi170.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 773 kB/s | 155 kB 00:00 2024-03-30T02:25:22.241 INFO:teuthology.orchestra.run.smithi067.stdout:(97/146): python3-pytz-2017.2-11.el8.noarch.rpm 463 kB/s | 54 kB 00:00 2024-03-30T02:25:22.284 INFO:teuthology.orchestra.run.smithi170.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 1.1 MB/s | 109 kB 00:00 2024-03-30T02:25:22.383 INFO:teuthology.orchestra.run.smithi067.stdout:(98/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-03-30T02:25:22.385 INFO:teuthology.orchestra.run.smithi170.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 243 kB/s | 43 kB 00:00 2024-03-30T02:25:22.410 INFO:teuthology.orchestra.run.smithi170.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 350 kB/s | 44 kB 00:00 2024-03-30T02:25:22.433 INFO:teuthology.orchestra.run.smithi067.stdout:(99/146): postgresql-10.23-4.module+el8.9.0+213 7.8 MB/s | 1.5 MB 00:00 2024-03-30T02:25:22.435 INFO:teuthology.orchestra.run.smithi170.stdout:(110/146): libaio-devel-0.3.112-1.el8.x86_64.rp 93 kB/s | 19 kB 00:00 2024-03-30T02:25:22.586 INFO:teuthology.orchestra.run.smithi170.stdout:(111/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-30T02:25:22.642 INFO:teuthology.orchestra.run.smithi067.stdout:(100/146): postgresql-contrib-10.23-4.module+el 3.7 MB/s | 812 kB 00:00 2024-03-30T02:25:22.642 INFO:teuthology.orchestra.run.smithi170.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.1 MB/s | 229 kB 00:00 2024-03-30T02:25:22.704 INFO:teuthology.orchestra.run.smithi170.stdout:(113/146): pcre2-devel-10.32-2.el8.x86_64.rpm 1.9 MB/s | 605 kB 00:00 2024-03-30T02:25:22.729 INFO:teuthology.orchestra.run.smithi170.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.4 MB/s | 200 kB 00:00 2024-03-30T02:25:22.753 INFO:teuthology.orchestra.run.smithi170.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 338 kB/s | 48 kB 00:00 2024-03-30T02:25:22.818 INFO:teuthology.orchestra.run.smithi067.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 21 MB/s | 14 MB 00:00 2024-03-30T02:25:22.839 INFO:teuthology.orchestra.run.smithi067.stdout:(102/146): libverto-devel-0.3.0-5.el8.x86_64.rp 77 kB/s | 18 kB 00:00 2024-03-30T02:25:22.879 INFO:teuthology.orchestra.run.smithi170.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 496 kB/s | 87 kB 00:00 2024-03-30T02:25:22.904 INFO:teuthology.orchestra.run.smithi170.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 257 kB/s | 39 kB 00:00 2024-03-30T02:25:22.926 INFO:teuthology.orchestra.run.smithi067.stdout:(103/146): python3-jwt-1.6.1-2.el8.noarch.rpm 394 kB/s | 43 kB 00:00 2024-03-30T02:25:22.929 INFO:teuthology.orchestra.run.smithi170.stdout:(118/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 287 kB/s | 58 kB 00:00 2024-03-30T02:25:22.961 INFO:teuthology.orchestra.run.smithi067.stdout:(104/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.2 MB/s | 155 kB 00:00 2024-03-30T02:25:23.014 INFO:teuthology.orchestra.run.smithi170.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 619 kB/s | 83 kB 00:00 2024-03-30T02:25:23.039 INFO:teuthology.orchestra.run.smithi067.stdout:(105/146): postgresql-server-10.23-4.module+el8 8.4 MB/s | 5.1 MB 00:00 2024-03-30T02:25:23.046 INFO:teuthology.orchestra.run.smithi170.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.4 MB/s | 645 kB 00:00 2024-03-30T02:25:23.072 INFO:teuthology.orchestra.run.smithi170.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2024-03-30T02:25:23.094 INFO:teuthology.orchestra.run.smithi067.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 652 kB/s | 109 kB 00:00 2024-03-30T02:25:23.111 INFO:teuthology.orchestra.run.smithi067.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 283 kB/s | 43 kB 00:00 2024-03-30T02:25:23.127 INFO:teuthology.orchestra.run.smithi067.stdout:(108/146): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2024-03-30T02:25:23.189 INFO:teuthology.orchestra.run.smithi170.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2024-03-30T02:25:23.236 INFO:teuthology.orchestra.run.smithi067.stdout:(109/146): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2024-03-30T02:25:23.242 INFO:teuthology.orchestra.run.smithi170.stdout:(123/146): libuuid-devel-2.32.1-43.el8.x86_64.r 442 kB/s | 100 kB 00:00 2024-03-30T02:25:23.270 INFO:teuthology.orchestra.run.smithi067.stdout:(110/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2024-03-30T02:25:23.306 INFO:teuthology.orchestra.run.smithi170.stdout:(124/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 311 kB/s | 73 kB 00:00 2024-03-30T02:25:23.354 INFO:teuthology.orchestra.run.smithi067.stdout:(111/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 169 kB/s | 44 kB 00:00 2024-03-30T02:25:23.356 INFO:teuthology.orchestra.run.smithi170.stdout:(125/146): NetworkManager-initscripts-updown-1. 863 kB/s | 144 kB 00:00 2024-03-30T02:25:23.379 INFO:teuthology.orchestra.run.smithi067.stdout:(112/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2024-03-30T02:25:23.390 INFO:teuthology.orchestra.run.smithi170.stdout:(126/146): elfutils-libelf-devel-0.189-3.el8.x8 410 kB/s | 62 kB 00:00 2024-03-30T02:25:23.396 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-03-30T02:25:23.407 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-03-30T02:25:23.450 INFO:teuthology.orchestra.run.smithi170.stdout:(127/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.9 MB/s | 562 kB 00:00 2024-03-30T02:25:23.451 INFO:teuthology.orchestra.run.smithi067.stdout:(113/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.1 MB/s | 200 kB 00:00 2024-03-30T02:25:23.479 INFO:teuthology.orchestra.run.smithi067.stdout:(114/146): keyutils-libs-devel-1.5.10-9.el8.x86 384 kB/s | 48 kB 00:00 2024-03-30T02:25:23.505 INFO:teuthology.orchestra.run.smithi067.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 693 kB/s | 87 kB 00:00 2024-03-30T02:25:23.532 INFO:teuthology.orchestra.run.smithi170.stdout:(128/146): openssl-devel-1.1.1k-12.el8_9.x86_64 13 MB/s | 2.3 MB 00:00 2024-03-30T02:25:23.566 INFO:teuthology.orchestra.run.smithi170.stdout:(129/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 268 kB/s | 47 kB 00:00 2024-03-30T02:25:23.613 INFO:teuthology.orchestra.run.smithi067.stdout:(116/146): fuse-2.9.7-17.el8.x86_64.rpm 768 kB/s | 83 kB 00:00 2024-03-30T02:25:23.616 INFO:teuthology.orchestra.run.smithi170.stdout:(130/146): python3-cryptography-3.2.1-7.el8_9.x 3.3 MB/s | 559 kB 00:00 2024-03-30T02:25:23.638 INFO:teuthology.orchestra.run.smithi067.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 243 kB/s | 39 kB 00:00 2024-03-30T02:25:23.663 INFO:teuthology.orchestra.run.smithi067.stdout:(118/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 271 kB/s | 58 kB 00:00 2024-03-30T02:25:23.748 INFO:teuthology.orchestra.run.smithi067.stdout:(119/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.7 MB/s | 645 kB 00:00 2024-03-30T02:25:23.766 INFO:teuthology.orchestra.run.smithi170.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.5 MB/s | 232 kB 00:00 2024-03-30T02:25:23.774 INFO:teuthology.orchestra.run.smithi067.stdout:(120/146): libuuid-devel-2.32.1-43.el8.x86_64.r 916 kB/s | 100 kB 00:00 2024-03-30T02:25:23.798 INFO:teuthology.orchestra.run.smithi067.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2024-03-30T02:25:23.873 INFO:teuthology.orchestra.run.smithi067.stdout:(122/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 726 kB/s | 73 kB 00:00 2024-03-30T02:25:23.907 INFO:teuthology.orchestra.run.smithi067.stdout:(123/146): NetworkManager-initscripts-updown-1. 1.3 MB/s | 144 kB 00:00 2024-03-30T02:25:23.932 INFO:teuthology.orchestra.run.smithi067.stdout:(124/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-03-30T02:25:23.933 INFO:teuthology.orchestra.run.smithi170.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2024-03-30T02:25:24.009 INFO:teuthology.orchestra.run.smithi170.stdout:(133/146): librbd1-18.2.2-1018.g470100a4.el8.x8 8.1 MB/s | 3.6 MB 00:00 2024-03-30T02:25:24.074 INFO:teuthology.orchestra.run.smithi067.stdout:(125/146): elfutils-libelf-devel-0.189-3.el8.x8 308 kB/s | 62 kB 00:00 2024-03-30T02:25:24.109 INFO:teuthology.orchestra.run.smithi170.stdout:(134/146): libmount-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 237 kB 00:00 2024-03-30T02:25:24.125 INFO:teuthology.orchestra.run.smithi067.stdout:(126/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 2.5 MB/s | 562 kB 00:00 2024-03-30T02:25:24.151 INFO:teuthology.orchestra.run.smithi170.stdout:(135/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 179 kB 00:00 2024-03-30T02:25:24.243 INFO:teuthology.orchestra.run.smithi170.stdout:(136/146): libuuid-2.32.1-43.el8.x86_64.rpm 740 kB/s | 99 kB 00:00 2024-03-30T02:25:24.250 INFO:teuthology.orchestra.run.smithi067.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 7.3 MB/s | 2.3 MB 00:00 2024-03-30T02:25:24.275 INFO:teuthology.orchestra.run.smithi067.stdout:(128/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 235 kB/s | 47 kB 00:00 2024-03-30T02:25:24.318 INFO:teuthology.orchestra.run.smithi170.stdout:(137/146): util-linux-2.32.1-43.el8.x86_64.rpm 15 MB/s | 2.5 MB 00:00 2024-03-30T02:25:24.358 INFO:teuthology.orchestra.run.smithi067.stdout:(129/146): python3-cryptography-3.2.1-7.el8_9.x 2.3 MB/s | 559 kB 00:00 2024-03-30T02:25:24.368 INFO:teuthology.orchestra.run.smithi170.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 821 kB/s | 103 kB 00:00 2024-03-30T02:25:24.433 INFO:teuthology.orchestra.run.smithi067.stdout:(130/146): python3-packaging-16.8-9.el8.noarch. 262 kB/s | 48 kB 00:00 2024-03-30T02:25:24.460 INFO:teuthology.orchestra.run.smithi170.stdout:(139/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2024-03-30T02:25:24.485 INFO:teuthology.orchestra.run.smithi170.stdout:(140/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-03-30T02:25:24.500 INFO:teuthology.orchestra.run.smithi067.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 3.4 MB/s | 232 kB 00:00 2024-03-30T02:25:24.619 INFO:teuthology.orchestra.run.smithi170.stdout:(141/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 5.2 MB/s | 711 kB 00:00 2024-03-30T02:25:24.659 INFO:teuthology.orchestra.run.smithi067.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.9 MB/s | 303 kB 00:00 2024-03-30T02:25:24.669 INFO:teuthology.orchestra.run.smithi170.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 254 kB 00:00 2024-03-30T02:25:24.778 INFO:teuthology.orchestra.run.smithi170.stdout:(143/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.3 MB/s | 1.5 MB 00:00 2024-03-30T02:25:24.818 INFO:teuthology.orchestra.run.smithi067.stdout:(133/146): libmount-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 237 kB 00:00 2024-03-30T02:25:24.845 INFO:teuthology.orchestra.run.smithi170.stdout:(144/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2024-03-30T02:25:24.920 INFO:teuthology.orchestra.run.smithi170.stdout:(145/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.5 MB/s | 1.1 MB 00:00 2024-03-30T02:25:24.993 INFO:teuthology.orchestra.run.smithi067.stdout:(134/146): libsmartcols-2.32.1-43.el8.x86_64.rp 1.0 MB/s | 179 kB 00:00 2024-03-30T02:25:25.152 INFO:teuthology.orchestra.run.smithi067.stdout:(135/146): libuuid-2.32.1-43.el8.x86_64.rpm 626 kB/s | 99 kB 00:00 2024-03-30T02:25:25.153 INFO:teuthology.orchestra.run.smithi170.stdout:(146/146): librados2-18.2.2-1018.g470100a4.el8. 2.3 MB/s | 3.7 MB 00:01 2024-03-30T02:25:25.158 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-03-30T02:25:25.159 INFO:teuthology.orchestra.run.smithi170.stdout:Total 9.9 MB/s | 213 MB 00:21 2024-03-30T02:25:25.227 INFO:teuthology.orchestra.run.smithi067.stdout:(136/146): librbd1-18.2.2-1018.g470100a4.el8.x8 4.1 MB/s | 3.6 MB 00:00 2024-03-30T02:25:25.369 INFO:teuthology.orchestra.run.smithi067.stdout:(137/146): util-linux-user-2.32.1-43.el8.x86_64 726 kB/s | 103 kB 00:00 2024-03-30T02:25:25.427 INFO:teuthology.orchestra.run.smithi067.stdout:(138/146): util-linux-2.32.1-43.el8.x86_64.rpm 9.0 MB/s | 2.5 MB 00:00 2024-03-30T02:25:25.553 INFO:teuthology.orchestra.run.smithi067.stdout:(139/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2024-03-30T02:25:25.603 INFO:teuthology.orchestra.run.smithi067.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 3.6 MB/s | 854 kB 00:00 2024-03-30T02:25:25.662 INFO:teuthology.orchestra.run.smithi067.stdout:(141/146): libfdisk-2.32.1-43.el8.x86_64.rpm 2.3 MB/s | 254 kB 00:00 2024-03-30T02:25:25.734 INFO:teuthology.orchestra.run.smithi067.stdout:(142/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 5.3 MB/s | 711 kB 00:00 2024-03-30T02:25:25.846 INFO:teuthology.orchestra.run.smithi067.stdout:(143/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.0 MB/s | 1.5 MB 00:00 2024-03-30T02:25:25.896 INFO:teuthology.orchestra.run.smithi067.stdout:(144/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.0 MB/s | 333 kB 00:00 2024-03-30T02:25:26.021 INFO:teuthology.orchestra.run.smithi067.stdout:(145/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.1 MB/s | 1.1 MB 00:00 2024-03-30T02:25:26.080 INFO:teuthology.orchestra.run.smithi067.stdout:(146/146): librados2-18.2.2-1018.g470100a4.el8. 2.0 MB/s | 3.7 MB 00:01 2024-03-30T02:25:26.082 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-03-30T02:25:26.083 INFO:teuthology.orchestra.run.smithi067.stdout:Total 13 MB/s | 213 MB 00:16 2024-03-30T02:25:26.256 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-03-30T02:25:26.652 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-03-30T02:25:26.652 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-03-30T02:25:27.851 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-03-30T02:25:27.862 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-03-30T02:25:28.258 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-03-30T02:25:28.736 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-03-30T02:25:28.737 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-03-30T02:25:28.947 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-03-30T02:25:29.081 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-30T02:25:29.324 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T02:25:29.560 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T02:25:29.756 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-30T02:25:29.925 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T02:25:30.005 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T02:25:30.217 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T02:25:30.283 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T02:25:30.393 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-03-30T02:25:30.406 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-03-30T02:25:30.724 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-30T02:25:30.934 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T02:25:31.000 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T02:25:31.196 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-30T02:25:31.367 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T02:25:31.772 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T02:25:31.975 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T02:25:32.369 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T02:25:32.542 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T02:25:32.658 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T02:25:32.850 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T02:25:32.978 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T02:25:33.294 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T02:25:33.550 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T02:25:33.565 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-03-30T02:25:33.677 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T02:25:33.700 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-30T02:25:33.872 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-30T02:25:33.943 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T02:25:34.121 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-30T02:25:34.161 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T02:25:34.295 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-30T02:25:34.325 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-30T02:25:34.545 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T02:25:34.604 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T02:25:34.695 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-30T02:25:34.771 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T02:25:34.842 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T02:25:34.945 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-30T02:25:35.069 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-30T02:25:35.141 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T02:25:35.245 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T02:25:35.271 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T02:25:35.333 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T02:25:35.391 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T02:25:35.467 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T02:25:35.547 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-30T02:25:35.616 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T02:25:35.729 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T02:25:35.737 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T02:25:35.903 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T02:25:36.138 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T02:25:36.387 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T02:25:36.525 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T02:25:36.691 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T02:25:36.777 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-03-30T02:25:36.779 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T02:25:36.962 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T02:25:36.984 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-30T02:25:37.090 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T02:25:37.216 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T02:25:37.295 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T02:25:37.298 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T02:25:37.395 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T02:25:37.466 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T02:25:37.612 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-30T02:25:37.697 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T02:25:37.892 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-30T02:25:37.970 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T02:25:38.060 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T02:25:38.063 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T02:25:38.160 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T02:25:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-30T02:25:38.301 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-30T02:25:38.379 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T02:25:38.405 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-30T02:25:38.473 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-30T02:25:38.557 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-30T02:25:38.579 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-30T02:25:38.716 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-30T02:25:38.723 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-30T02:25:38.735 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-30T02:25:38.928 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T02:25:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-30T02:25:39.021 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-30T02:25:39.100 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-30T02:25:39.237 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-30T02:25:39.313 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-30T02:25:39.346 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-30T02:25:39.454 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T02:25:39.503 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T02:25:39.576 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-30T02:25:39.596 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T02:25:39.746 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 31/162 2024-03-30T02:25:39.753 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T02:25:39.818 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T02:25:39.884 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 32/162 2024-03-30T02:25:39.941 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-30T02:25:40.011 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T02:25:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T02:25:40.189 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 33/162 2024-03-30T02:25:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T02:25:40.296 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T02:25:40.736 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T02:25:40.920 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T02:25:40.921 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T02:25:41.078 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T02:25:41.094 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T02:25:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-30T02:25:41.287 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T02:25:41.420 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T02:25:41.513 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-30T02:25:41.701 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T02:25:41.769 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-30T02:25:41.793 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T02:25:41.878 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T02:25:41.956 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T02:25:42.054 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T02:25:42.163 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-30T02:25:42.383 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-30T02:25:42.567 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T02:25:42.664 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-30T02:25:42.800 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-30T02:25:42.949 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-30T02:25:42.994 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-30T02:25:43.114 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-30T02:25:43.377 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-30T02:25:43.428 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-30T02:25:43.677 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-30T02:25:43.683 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-30T02:25:43.702 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T02:25:43.804 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T02:25:43.939 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T02:25:43.945 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-30T02:25:44.040 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-30T02:25:44.108 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 31/162 2024-03-30T02:25:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T02:25:44.215 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 20/162 2024-03-30T02:25:44.255 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 32/162 2024-03-30T02:25:44.310 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-30T02:25:44.412 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T02:25:44.541 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-30T02:25:44.567 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 33/162 2024-03-30T02:25:44.584 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-30T02:25:44.712 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T02:25:44.744 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 41/162 2024-03-30T02:25:44.953 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 42/162 2024-03-30T02:25:45.181 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T02:25:45.241 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T02:25:45.430 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T02:25:45.476 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-30T02:25:45.500 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T02:25:45.514 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-30T02:25:45.654 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T02:25:45.745 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-30T02:25:45.850 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-30T02:25:45.941 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-30T02:25:46.097 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-30T02:25:46.139 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T02:25:46.304 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T02:25:46.349 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-30T02:25:46.529 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T02:25:46.584 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T02:25:46.931 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T02:25:46.948 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T02:25:47.184 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-30T02:25:47.375 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-30T02:25:47.385 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-30T02:25:47.567 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-30T02:25:47.628 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-30T02:25:47.771 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-30T02:25:47.792 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-30T02:25:47.961 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-30T02:25:47.984 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-30T02:25:48.038 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T02:25:48.243 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-30T02:25:48.277 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-30T02:25:48.352 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-30T02:25:48.516 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-30T02:25:48.571 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-30T02:25:48.587 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-30T02:25:48.768 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-30T02:25:48.810 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-30T02:25:48.812 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-30T02:25:48.936 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 31/162 2024-03-30T02:25:48.992 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-30T02:25:49.041 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 41/162 2024-03-30T02:25:49.101 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 32/162 2024-03-30T02:25:49.216 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 42/162 2024-03-30T02:25:49.257 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-30T02:25:49.354 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 33/162 2024-03-30T02:25:49.435 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T02:25:49.512 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T02:25:49.613 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-30T02:25:49.751 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T02:25:49.820 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T02:25:49.940 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-30T02:25:50.046 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-30T02:25:50.253 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-30T02:25:50.271 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-30T02:25:50.452 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T02:25:50.461 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-30T02:25:50.479 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-30T02:25:50.698 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-30T02:25:50.798 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-30T02:25:50.896 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T02:25:50.955 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-30T02:25:51.110 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-30T02:25:51.169 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T02:25:51.189 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T02:25:51.244 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-30T02:25:51.320 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T02:25:51.389 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-30T02:25:51.599 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-30T02:25:51.613 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-30T02:25:52.082 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-30T02:25:52.130 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-30T02:25:52.289 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-30T02:25:52.313 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-30T02:25:52.532 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-30T02:25:52.546 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-30T02:25:52.754 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-30T02:25:52.831 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-30T02:25:52.840 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-30T02:25:53.103 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-30T02:25:53.130 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-03-30T02:25:53.381 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-30T02:25:53.382 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-03-30T02:25:53.488 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-30T02:25:53.537 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-03-30T02:25:53.563 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-30T02:25:53.609 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-03-30T02:25:53.786 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-30T02:25:53.811 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-30T02:25:53.820 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-30T02:25:54.062 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-03-30T02:25:54.066 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-30T02:25:54.131 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-30T02:25:54.282 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-30T02:25:54.286 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-30T02:25:54.447 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-30T02:25:54.494 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 41/162 2024-03-30T02:25:54.505 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-30T02:25:54.639 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 42/162 2024-03-30T02:25:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-30T02:25:54.741 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-30T02:25:54.861 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T02:25:54.901 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-30T02:25:54.925 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-30T02:25:54.964 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-30T02:25:55.120 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-30T02:25:55.210 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-30T02:25:55.210 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T02:25:55.301 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-30T02:25:55.345 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-30T02:25:55.465 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-30T02:25:55.469 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-30T02:25:55.579 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-30T02:25:55.650 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-30T02:25:55.666 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-30T02:25:55.801 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-30T02:25:55.828 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T02:25:55.832 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-30T02:25:56.036 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-30T02:25:56.038 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-30T02:25:56.238 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-30T02:25:56.241 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-30T02:25:56.441 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-30T02:25:56.481 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-30T02:25:56.495 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-30T02:25:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-30T02:25:56.690 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-30T02:25:56.745 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T02:25:56.745 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-30T02:25:56.789 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-30T02:25:56.892 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-30T02:25:56.956 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-packaging-16.8-9.el8.noarch 49/162 2024-03-30T02:25:57.044 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-03-30T02:25:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-30T02:25:57.149 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.7-17.el8.x86_64 50/162 2024-03-30T02:25:57.295 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-03-30T02:25:57.340 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 51/162 2024-03-30T02:25:57.348 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-30T02:25:57.484 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-03-30T02:25:57.525 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-30T02:25:57.556 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-03-30T02:25:57.561 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 52/162 2024-03-30T02:25:57.725 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 53/162 2024-03-30T02:25:57.732 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-30T02:25:57.833 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-30T02:25:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 54/162 2024-03-30T02:25:57.965 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-03-30T02:25:58.060 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-30T02:25:58.168 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-30T02:25:58.222 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 55/162 2024-03-30T02:25:58.277 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-30T02:25:58.376 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-30T02:25:58.485 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/162 2024-03-30T02:25:58.536 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-30T02:25:58.550 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-30T02:25:58.700 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/162 2024-03-30T02:25:58.771 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-30T02:25:58.873 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 58/162 2024-03-30T02:25:59.055 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-30T02:25:59.123 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/162 2024-03-30T02:25:59.299 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-30T02:25:59.471 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-30T02:25:59.583 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/162 2024-03-30T02:25:59.613 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-30T02:25:59.653 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-30T02:25:59.851 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-30T02:25:59.885 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 61/162 2024-03-30T02:25:59.965 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-30T02:26:00.071 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-30T02:26:00.193 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-30T02:26:00.218 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/162 2024-03-30T02:26:00.294 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-30T02:26:00.421 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 63/162 2024-03-30T02:26:00.519 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-30T02:26:00.520 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-30T02:26:00.651 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-30T02:26:00.697 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-30T02:26:00.769 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-30T02:26:00.876 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 65/162 2024-03-30T02:26:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-30T02:26:00.976 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-30T02:26:01.121 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 66/162 2024-03-30T02:26:01.145 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-30T02:26:01.306 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-30T02:26:01.347 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 67/162 2024-03-30T02:26:01.521 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 68/162 2024-03-30T02:26:01.586 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-30T02:26:01.748 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 69/162 2024-03-30T02:26:01.748 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-30T02:26:01.840 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-30T02:26:01.943 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T02:26:01.975 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 70/162 2024-03-30T02:26:02.018 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T02:26:02.023 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-30T02:26:02.127 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-03-30T02:26:02.218 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 71/162 2024-03-30T02:26:02.265 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-30T02:26:02.307 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 104/162 2024-03-30T02:26:02.545 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : jq-1.6-7.el8.x86_64 72/162 2024-03-30T02:26:02.819 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.1-8.el8.noarch 73/162 2024-03-30T02:26:02.968 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : uuid-1.6.2-43.el8.x86_64 74/162 2024-03-30T02:26:03.052 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 74/162 2024-03-30T02:26:03.302 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 75/162 2024-03-30T02:26:03.371 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-30T02:26:03.520 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 76/162 2024-03-30T02:26:03.670 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-30T02:26:03.730 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 77/162 2024-03-30T02:26:03.871 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-30T02:26:03.898 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-30T02:26:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-30T02:26:04.206 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-30T02:26:04.360 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-30T02:26:04.491 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-30T02:26:04.577 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-30T02:26:04.715 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-30T02:26:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-30T02:26:04.908 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-30T02:26:05.172 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-30T02:26:05.337 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-30T02:26:05.497 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-30T02:26:05.561 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-30T02:26:05.665 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T02:26:05.722 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-30T02:26:05.737 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T02:26:05.901 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-30T02:26:05.941 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 104/162 2024-03-30T02:26:06.095 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-30T02:26:06.256 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T02:26:06.316 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-30T02:26:06.458 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-30T02:26:06.484 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T02:26:06.740 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-30T02:26:06.961 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-30T02:26:07.075 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T02:26:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-30T02:26:07.365 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T02:26:07.380 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-30T02:26:07.446 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T02:26:07.634 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T02:26:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-30T02:26:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-30T02:26:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-30T02:26:09.613 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-30T02:26:09.895 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-30T02:26:10.122 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-30T02:26:10.142 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T02:26:10.374 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T02:26:10.962 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T02:26:10.979 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-30T02:26:11.159 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T02:26:11.220 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T02:26:11.288 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 103/162 2024-03-30T02:26:11.305 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T02:26:11.443 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 104/162 2024-03-30T02:26:11.490 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T02:26:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T02:26:17.184 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T02:26:17.914 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 105/162 2024-03-30T02:26:18.170 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T02:26:18.328 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 106/162 2024-03-30T02:26:18.443 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /run 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-03-30T02:26:27.140 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2024-03-30T02:26:27.141 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:26:27.555 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T02:26:29.183 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T02:26:29.424 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T02:26:29.643 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T02:26:29.804 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T02:26:30.278 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T02:26:30.462 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T02:26:30.480 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T02:26:30.627 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /run 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2024-03-30T02:26:30.628 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:26:30.979 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T02:26:32.249 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T02:26:32.396 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T02:26:32.526 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 113/162 2024-03-30T02:26:32.608 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T02:26:32.734 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 114/162 2024-03-30T02:26:32.809 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T02:26:32.999 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T02:26:33.029 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T02:26:33.186 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T02:26:33.227 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T02:26:33.501 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T02:26:33.662 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T02:26:33.828 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T02:26:33.846 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T02:26:34.462 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T02:26:34.619 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T02:26:34.713 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T02:26:35.062 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 118/162 2024-03-30T02:26:35.226 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T02:26:35.264 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T02:26:35.419 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T02:26:35.431 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T02:26:35.664 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T02:26:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T02:26:35.975 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 113/162 2024-03-30T02:26:36.169 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 114/162 2024-03-30T02:26:36.394 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T02:26:36.623 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T02:26:36.694 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 121/162 2024-03-30T02:26:36.884 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T02:26:36.918 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T02:26:36.941 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T02:26:37.094 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T02:26:37.137 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T02:26:37.344 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T02:26:37.690 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T02:26:37.903 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T02:26:37.919 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T02:26:38.085 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T02:26:38.179 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T02:26:38.569 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 118/162 2024-03-30T02:26:38.759 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T02:26:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T02:26:38.968 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T02:26:38.981 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T02:26:40.242 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 121/162 2024-03-30T02:26:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 107/162 2024-03-30T02:26:40.256 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2024-03-30T02:26:40.256 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2024-03-30T02:26:40.256 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /dev 2024-03-30T02:26:40.256 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /run 2024-03-30T02:26:40.256 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2024-03-30T02:26:40.256 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2024-03-30T02:26:40.257 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2024-03-30T02:26:40.257 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2024-03-30T02:26:40.257 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /dev 2024-03-30T02:26:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:26:40.432 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T02:26:40.468 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T02:26:40.631 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T02:26:40.660 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T02:26:40.704 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T02:26:40.877 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T02:26:41.224 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T02:26:41.402 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T02:26:42.383 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T02:26:42.508 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 108/162 2024-03-30T02:26:42.733 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T02:26:42.862 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 126/162 2024-03-30T02:26:43.015 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 109/162 2024-03-30T02:26:43.156 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T02:26:43.170 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-30T02:26:43.400 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T02:26:43.745 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 110/162 2024-03-30T02:26:43.889 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T02:26:43.933 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 111/162 2024-03-30T02:26:44.084 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T02:26:44.397 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-30T02:26:44.674 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-30T02:26:44.882 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T02:26:45.120 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-30T02:26:45.297 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-30T02:26:45.476 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 134/162 2024-03-30T02:26:45.872 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T02:26:46.305 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 126/162 2024-03-30T02:26:46.627 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-30T02:26:46.817 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T02:26:46.838 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T02:26:47.003 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 112/162 2024-03-30T02:26:47.154 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 113/162 2024-03-30T02:26:47.322 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 114/162 2024-03-30T02:26:47.495 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T02:26:47.594 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T02:26:47.741 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T02:26:47.788 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-30T02:26:47.876 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 115/162 2024-03-30T02:26:48.050 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T02:26:48.057 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-30T02:26:48.088 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T02:26:48.090 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T02:26:48.282 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T02:26:48.485 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-30T02:26:48.511 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-30T02:26:48.721 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-30T02:26:48.806 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-30T02:26:48.885 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 134/162 2024-03-30T02:26:49.045 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-30T02:26:49.244 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 116/162 2024-03-30T02:26:49.249 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 139/162 2024-03-30T02:26:49.437 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T02:26:49.465 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 140/162 2024-03-30T02:26:49.575 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 117/162 2024-03-30T02:26:49.739 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 141/162 2024-03-30T02:26:49.983 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 118/162 2024-03-30T02:26:49.993 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-30T02:26:50.124 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T02:26:50.157 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T02:26:50.166 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T02:26:50.195 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 119/162 2024-03-30T02:26:50.344 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-30T02:26:50.354 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T02:26:50.384 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 120/162 2024-03-30T02:26:50.597 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T02:26:50.675 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T02:26:50.882 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T02:26:50.895 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T02:26:51.076 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-30T02:26:51.156 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T02:26:51.267 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-30T02:26:51.425 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T02:26:51.427 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-30T02:26:51.427 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T02:26:51.463 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T02:26:51.710 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T02:26:51.710 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T02:26:51.810 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 121/162 2024-03-30T02:26:51.851 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-30T02:26:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T02:26:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T02:26:51.999 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T02:26:52.064 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 122/162 2024-03-30T02:26:52.215 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-30T02:26:52.230 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T02:26:52.237 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T02:26:52.311 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1018.g47010 123/162 2024-03-30T02:26:52.434 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-30T02:26:52.446 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-30T02:26:52.521 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T02:26:52.601 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-30T02:26:52.633 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 139/162 2024-03-30T02:26:52.867 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-30T02:26:52.867 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T02:26:52.875 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 140/162 2024-03-30T02:26:52.948 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 124/162 2024-03-30T02:26:53.093 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T02:26:53.093 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T02:26:53.136 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T02:26:53.145 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 141/162 2024-03-30T02:26:53.318 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T02:26:53.318 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T02:26:53.401 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-30T02:26:53.566 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T02:26:53.577 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T02:26:53.593 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T02:26:53.593 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T02:26:53.796 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-30T02:26:53.801 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T02:26:53.801 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T02:26:54.034 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T02:26:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T02:26:54.123 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T02:26:54.234 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-30T02:26:54.234 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T02:26:54.358 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T02:26:54.373 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T02:26:54.585 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-30T02:26:54.776 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-30T02:26:54.937 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-30T02:26:54.937 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T02:26:55.079 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T02:26:55.079 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 1/162 2024-03-30T02:26:55.079 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2/162 2024-03-30T02:26:55.079 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 3/162 2024-03-30T02:26:55.079 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 4/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1018.g47010 5/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 6/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 7/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 8/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 13/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 14/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 15/162 2024-03-30T02:26:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 16/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 17/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 18/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 20/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 21/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 22/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 23/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 24/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 25/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 26/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 27/162 2024-03-30T02:26:55.081 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 28/162 2024-03-30T02:26:55.082 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 29/162 2024-03-30T02:26:55.082 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 30/162 2024-03-30T02:26:55.096 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 31/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 32/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 33/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 34/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 35/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 36/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 37/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 38/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-30T02:26:55.097 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-30T02:26:55.098 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-30T02:26:55.099 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-03-30T02:26:55.100 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-03-30T02:26:55.101 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-30T02:26:55.102 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-30T02:26:55.103 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 133/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-30T02:26:55.104 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-30T02:26:55.105 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-30T02:26:55.106 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-30T02:26:55.106 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-30T02:26:55.106 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-30T02:26:55.106 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-30T02:26:55.106 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-30T02:26:55.106 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-30T02:26:55.106 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-30T02:26:55.106 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-30T02:26:55.106 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-30T02:26:55.219 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T02:26:55.219 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T02:26:55.495 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T02:26:55.495 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T02:26:55.747 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T02:26:55.968 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-30T02:26:56.142 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-30T02:26:56.417 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-30T02:26:56.418 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T02:26:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T02:26:56.677 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T02:26:56.910 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T02:26:56.910 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T02:26:57.160 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T02:26:57.160 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T02:26:57.426 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T02:26:57.427 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T02:26:57.643 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T02:26:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-30T02:26:57.827 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T02:26:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 125/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 1/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 3/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 4/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1018.g47010 5/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 6/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 7/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 8/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T02:26:58.732 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 13/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 14/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 15/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 16/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 17/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 18/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 20/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 21/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 22/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 23/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 24/162 2024-03-30T02:26:58.733 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 25/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 26/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 27/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 28/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 29/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 30/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 31/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 32/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 33/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 34/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 35/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 36/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 37/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 38/162 2024-03-30T02:26:58.734 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-30T02:26:58.735 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-30T02:26:58.736 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-03-30T02:26:58.737 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-03-30T02:26:58.738 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-30T02:26:58.739 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-30T02:26:58.740 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 133/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-30T02:26:58.741 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-30T02:26:58.742 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-30T02:26:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-30T02:26:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-30T02:26:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-30T02:26:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-30T02:26:58.743 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-30T02:26:58.867 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 126/162 2024-03-30T02:26:59.186 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-30T02:26:59.436 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T02:27:00.539 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-30T02:27:00.851 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-30T02:27:01.072 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-30T02:27:01.424 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T02:27:01.669 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-30T02:27:01.851 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-30T02:27:02.071 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 134/162 2024-03-30T02:27:05.706 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T02:27:06.003 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T02:27:06.121 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-30T02:27:06.620 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-30T02:27:07.028 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-30T02:27:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-30T02:27:07.446 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 139/162 2024-03-30T02:27:07.664 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 140/162 2024-03-30T02:27:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 141/162 2024-03-30T02:27:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-30T02:27:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T02:27:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-30T02:27:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-30T02:27:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T02:27:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-30T02:27:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T02:27:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-30T02:27:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-30T02:27:09.787 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-30T02:27:09.950 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-30T02:27:09.950 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T02:27:10.240 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-30T02:27:10.241 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T02:27:10.485 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-30T02:27:10.485 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2024-03-30T02:27:10.501 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-30T02:27:10.501 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T02:27:10.799 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-30T02:27:10.996 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-30T02:27:11.215 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-30T02:27:11.262 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:27:11.262 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-03-30T02:27:11.262 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-30T02:27:11.262 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-30T02:27:11.262 INFO:teuthology.orchestra.run.smithi082.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-30T02:27:11.262 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-30T02:27:11.262 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-03-30T02:27:11.263 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.1-8.el8.noarch 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.0.4-10.el8.x86_64 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.264 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-30T02:27:11.265 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.1-9.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.6-7.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-30T02:27:11.266 INFO:teuthology.orchestra.run.smithi082.stdout: libpq-13.11-1.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-30T02:27:11.267 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-30T02:27:11.268 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-30T02:27:11.269 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-2.6-1.el8.noarch 2024-03-30T02:27:11.270 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-30T02:27:11.271 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-30T02:27:11.272 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-30T02:27:11.273 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-30T02:27:11.273 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-30T02:27:11.273 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-30T02:27:11.273 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-30T02:27:11.273 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-30T02:27:11.273 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-03-30T02:27:11.404 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-30T02:27:11.404 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T02:27:11.456 DEBUG:teuthology.parallel:result is None 2024-03-30T02:27:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-30T02:27:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T02:27:11.980 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-30T02:27:11.980 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T02:27:12.214 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-30T02:27:12.214 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T02:27:12.480 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-30T02:27:12.480 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T02:27:12.722 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-30T02:27:12.847 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-30T02:27:12.847 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T02:27:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-30T02:27:13.443 INFO:teuthology.orchestra.run.smithi067.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-30T02:27:13.443 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:27:13.904 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-30T02:27:13.904 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2024-03-30T02:27:14.184 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:18.2.2-1018.g470100a4.el8.x86_64 1/162 2024-03-30T02:27:14.184 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2/162 2024-03-30T02:27:14.184 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 3/162 2024-03-30T02:27:14.184 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 4/162 2024-03-30T02:27:14.184 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1018.g47010 5/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 6/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 7/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 8/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 9/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 10/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 11/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 12/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 13/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_ 14/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 15/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 16/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:18.2.2-1018.g470100a4.el8.x86_6 17/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86 18/162 2024-03-30T02:27:14.185 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 19/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1018.g470100a4.el 20/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:18.2.2-1018.g470100a4.el8. 21/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_6 22/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 23/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 24/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 25/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 26/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 27/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 28/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1018.g470100a4. 29/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noa 30/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.n 31/162 2024-03-30T02:27:14.186 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1018.g470 32/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el 33/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 34/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.e 35/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 36/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:18.2.2-1018.g470100a4.el8.noarch 37/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 38/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-30T02:27:14.187 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-30T02:27:14.188 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/162 2024-03-30T02:27:14.189 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 83/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 84/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 85/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 86/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 90/162 2024-03-30T02:27:14.190 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 91/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 92/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 93/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.1-8.el8.noarch 94/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpq-13.11-1.el8.x86_64 95/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : jq-1.6-7.el8.x86_64 96/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 97/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 98/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 99/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 100/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 101/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 102/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 103/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 104/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 105/162 2024-03-30T02:27:14.191 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 106/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 107/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 108/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 109/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 110/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 112/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 113/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 114/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 115/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 117/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 118/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 119/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 120/162 2024-03-30T02:27:14.192 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 121/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 124/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 125/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 126/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 128/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 129/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 130/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:18.2.2-1018.g470100a4.el8.x86_64 131/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 133/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-30T02:27:14.193 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-30T02:27:14.194 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-30T02:27:14.195 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-30T02:27:14.195 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-30T02:27:14.195 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-30T02:27:14.195 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-30T02:27:14.195 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-30T02:27:14.195 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-30T02:27:14.195 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-30T02:27:14.195 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-30T02:27:14.511 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-30T02:27:14.512 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.1-8.el8.noarch 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.0.4-10.el8.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.513 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.514 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.1-9.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: jq-1.6-7.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.515 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libpq-13.11-1.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-30T02:27:14.516 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-30T02:27:14.517 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-30T02:27:14.518 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-30T02:27:14.519 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-2.6-1.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-30T02:27:14.520 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-30T02:27:14.521 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-30T02:27:14.522 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-30T02:27:14.522 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-30T02:27:14.522 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-30T02:27:14.522 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:27:14.522 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-03-30T02:27:14.724 DEBUG:teuthology.parallel:result is None 2024-03-30T02:27:26.106 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-30T02:27:26.106 INFO:teuthology.orchestra.run.smithi067.stdout:Installed products updated. 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-30T02:27:28.167 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.1-8.el8.noarch 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.0.4-10.el8.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.168 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.169 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:18.2.2-1018.g470100a4.el8.noarch 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.1-9.el8.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: jq-1.6-7.el8.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-30T02:27:28.170 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: libpq-13.11-1.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.171 INFO:teuthology.orchestra.run.smithi067.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-30T02:27:28.172 INFO:teuthology.orchestra.run.smithi067.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-30T02:27:28.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-30T02:27:28.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-2.6-1.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-30T02:27:28.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-30T02:27:28.176 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:18.2.2-1018.g470100a4.el8.x86_64 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-30T02:27:28.177 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-30T02:27:28.178 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:27:28.178 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-03-30T02:27:28.546 DEBUG:teuthology.parallel:result is None 2024-03-30T02:27:28.547 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:27:28.690 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T02:27:28.742 INFO:teuthology.orchestra.run.smithi067.stdout:18.2.2-1018.g470100a4.el8 2024-03-30T02:27:28.742 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018.g470100a4.el8 2024-03-30T02:27:28.743 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018.g470100a4 is installed. 2024-03-30T02:27:28.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:27:28.895 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T02:27:28.924 INFO:teuthology.orchestra.run.smithi082.stdout:18.2.2-1018.g470100a4.el8 2024-03-30T02:27:28.924 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018.g470100a4.el8 2024-03-30T02:27:28.924 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018.g470100a4 is installed. 2024-03-30T02:27:28.926 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:27:29.147 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-30T02:27:29.176 INFO:teuthology.orchestra.run.smithi170.stdout:18.2.2-1018.g470100a4.el8 2024-03-30T02:27:29.176 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1018.g470100a4.el8 2024-03-30T02:27:29.177 INFO:teuthology.task.install:The correct ceph version 18.2.2-1018.g470100a4 is installed. 2024-03-30T02:27:29.179 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-30T02:27:29.179 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:27:29.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T02:27:29.267 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:27:29.267 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T02:27:29.316 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:27:29.316 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T02:27:29.366 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-30T02:27:29.366 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:27:29.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T02:27:29.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T02:27:29.536 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:27:29.536 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T02:27:29.569 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T02:27:29.641 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:27:29.641 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T02:27:29.674 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T02:27:29.747 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-30T02:27:29.747 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:27:29.747 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T02:27:29.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T02:27:29.916 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:27:29.916 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T02:27:29.948 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T02:27:30.019 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:27:30.020 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T02:27:30.052 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T02:27:30.123 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-30T02:27:30.123 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:27:30.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T02:27:30.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T02:27:30.292 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:27:30.292 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T02:27:30.324 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T02:27:30.395 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:27:30.395 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T02:27:30.428 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T02:27:30.498 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-30T02:27:30.611 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-30T02:27:30.611 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:27:30.612 INFO:tasks.cephadm:Cluster fsid is 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:27:30.612 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-30T02:27:30.612 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.67', 'mon.b': '172.21.15.82', 'mon.c': '172.21.15.170'} 2024-03-30T02:27:30.612 INFO:tasks.cephadm:First mon is mon.a on smithi067 2024-03-30T02:27:30.612 INFO:tasks.cephadm:First mgr is x 2024-03-30T02:27:30.612 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-30T02:27:30.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname $(hostname -s) 2024-03-30T02:27:30.685 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2024-03-30T02:27:30.721 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2024-03-30T02:27:30.757 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-30T02:27:30.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:27:30.916 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '470100a4ccbcf7769052bf09a5413881ccf1b97f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/78078/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1018-g470100a4', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1018.g470100a4'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-03-29 13:05:59.448245', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240327.051042-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-30T02:27:31.071 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-rishabh-testing-20240327.051042-reef, sha1 470100a4ccbcf7769052bf09a5413881ccf1b97f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:27:31.073 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/x86_64/flavors/default/cephadm 2024-03-30T02:27:31.074 DEBUG:teuthology.orchestra.run.smithi067:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-30T02:27:31.700 INFO:teuthology.orchestra.run.smithi067.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Mar 30 02:27 /home/ubuntu/cephtest/cephadm 2024-03-30T02:27:31.700 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-30T02:27:32.142 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Mar 30 02:27 /home/ubuntu/cephtest/cephadm 2024-03-30T02:27:32.143 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240327.051042-reef/470100a4ccbcf7769052bf09a5413881ccf1b97f/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-30T02:27:32.573 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Mar 30 02:27 /home/ubuntu/cephtest/cephadm 2024-03-30T02:27:32.573 DEBUG:teuthology.orchestra.run.smithi067:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-30T02:27:32.623 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-30T02:27:32.644 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-30T02:27:32.689 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f on all hosts... 2024-03-30T02:27:32.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f pull 2024-03-30T02:27:32.693 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f pull 2024-03-30T02:27:32.696 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f pull 2024-03-30T02:27:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T02:27:33.535 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T02:27:34.502 INFO:teuthology.orchestra.run.smithi067.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T02:28:01.975 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:28:01.976 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T02:28:01.976 INFO:teuthology.orchestra.run.smithi067.stdout: "image_id": "e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d", 2024-03-30T02:28:01.976 INFO:teuthology.orchestra.run.smithi067.stdout: "repo_digests": [ 2024-03-30T02:28:01.976 INFO:teuthology.orchestra.run.smithi067.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c123be959ad18653301727c9f149d0b2813c48e5bad48157ed6a183a20cecd35" 2024-03-30T02:28:01.976 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:28:01.976 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:28:11.863 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-30T02:28:11.863 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T02:28:11.864 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d", 2024-03-30T02:28:11.864 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2024-03-30T02:28:11.864 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c123be959ad18653301727c9f149d0b2813c48e5bad48157ed6a183a20cecd35" 2024-03-30T02:28:11.864 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-03-30T02:28:11.864 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-30T02:28:15.541 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-30T02:28:15.541 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T02:28:15.541 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d", 2024-03-30T02:28:15.542 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2024-03-30T02:28:15.542 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c123be959ad18653301727c9f149d0b2813c48e5bad48157ed6a183a20cecd35" 2024-03-30T02:28:15.542 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-30T02:28:15.542 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-30T02:28:15.560 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph 2024-03-30T02:28:15.639 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2024-03-30T02:28:15.669 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2024-03-30T02:28:15.706 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /etc/ceph 2024-03-30T02:28:15.774 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2024-03-30T02:28:15.802 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2024-03-30T02:28:15.835 INFO:tasks.cephadm:Writing seed config... 2024-03-30T02:28:15.836 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-30T02:28:15.836 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-30T02:28:15.836 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-30T02:28:15.836 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-30T02:28:15.836 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-30T02:28:15.836 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-30T02:28:15.837 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-30T02:28:15.838 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-30T02:28:15.839 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-30T02:28:15.840 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-30T02:28:15.840 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-30T02:28:15.840 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:28:15.840 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-30T02:28:15.883 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-30T02:28:15.884 DEBUG:teuthology.orchestra.run.smithi067:mon.a> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.a.service 2024-03-30T02:28:15.928 DEBUG:teuthology.orchestra.run.smithi067:mgr.x> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.x.service 2024-03-30T02:28:15.972 INFO:tasks.cephadm:Bootstrapping... 2024-03-30T02:28:15.972 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f -v bootstrap --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.67 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-30T02:28:15.992 INFO:journalctl@ceph.mon.a.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:28:16.038 INFO:journalctl@ceph.mgr.x.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:28:16.196 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-03-30T02:28:16.196 INFO:teuthology.orchestra.run.smithi067.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f', '-v', 'bootstrap', '--fsid', '0ef0748e-ee3d-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.67', '--skip-admin-label'] 2024-03-30T02:28:16.245 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 4.6.1 2024-03-30T02:28:16.245 INFO:teuthology.orchestra.run.smithi067.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-30T02:28:16.246 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying podman|docker is present... 2024-03-30T02:28:16.289 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 4.6.1 2024-03-30T02:28:16.289 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying lvm2 is present... 2024-03-30T02:28:16.289 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying time synchronization is in place... 2024-03-30T02:28:16.303 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-30T02:28:16.303 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-30T02:28:16.315 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-30T02:28:16.315 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-03-30T02:28:16.329 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-03-30T02:28:16.341 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-03-30T02:28:16.341 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-03-30T02:28:16.341 INFO:teuthology.orchestra.run.smithi067.stdout:Repeating the final host check... 2024-03-30T02:28:16.388 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 4.6.1 2024-03-30T02:28:16.388 INFO:teuthology.orchestra.run.smithi067.stdout:podman (/bin/podman) version 4.6.1 is present 2024-03-30T02:28:16.389 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl is present 2024-03-30T02:28:16.389 INFO:teuthology.orchestra.run.smithi067.stdout:lvcreate is present 2024-03-30T02:28:16.402 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-30T02:28:16.402 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-30T02:28:16.414 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-30T02:28:16.414 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-03-30T02:28:16.428 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-03-30T02:28:16.443 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-03-30T02:28:16.444 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-03-30T02:28:16.444 INFO:teuthology.orchestra.run.smithi067.stdout:Host looks OK 2024-03-30T02:28:16.444 INFO:teuthology.orchestra.run.smithi067.stdout:Cluster fsid: 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:28:16.444 INFO:teuthology.orchestra.run.smithi067.stdout:Acquiring lock 140088709773800 on /run/cephadm/0ef0748e-ee3d-11ee-b647-cb9ed24678a4.lock 2024-03-30T02:28:16.444 INFO:teuthology.orchestra.run.smithi067.stdout:Lock 140088709773800 acquired on /run/cephadm/0ef0748e-ee3d-11ee-b647-cb9ed24678a4.lock 2024-03-30T02:28:16.444 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 3300 ... 2024-03-30T02:28:16.444 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 6789 ... 2024-03-30T02:28:16.444 INFO:teuthology.orchestra.run.smithi067.stdout:Base mon IP(s) is [172.21.15.67:3300, 172.21.15.67:6789], mon addrv is [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-03-30T02:28:16.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2024-03-30T02:28:16.450 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.67 metric 100 2024-03-30T02:28:16.457 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-30T02:28:16.457 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-30T02:28:16.457 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1787sec hoplimit 64 pref medium 2024-03-30T02:28:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-30T02:28:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-30T02:28:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-30T02:28:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-30T02:28:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-03-30T02:28:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-30T02:28:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-03-30T02:28:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-03-30T02:28:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-30T02:28:16.466 INFO:teuthology.orchestra.run.smithi067.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-30T02:28:16.467 INFO:teuthology.orchestra.run.smithi067.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T02:28:17.540 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d 2024-03-30T02:28:17.540 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f... 2024-03-30T02:28:17.541 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Getting image source signatures 2024-03-30T02:28:17.541 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:86ca209f044bd2e3705adc9da4fc1145309402ec2e51b9b9b051617a85f31417 2024-03-30T02:28:17.541 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-03-30T02:28:17.541 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying config sha256:e7c444a08cf749270ac19a49d6d49787223f88c26222d2dfc1d77b830f40791d 2024-03-30T02:28:17.541 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-30T02:28:18.323 INFO:teuthology.orchestra.run.smithi067.stdout:ceph: stdout ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable) 2024-03-30T02:28:18.323 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph version: ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable) 2024-03-30T02:28:18.323 INFO:teuthology.orchestra.run.smithi067.stdout:Extracting ceph user uid/gid from container image... 2024-03-30T02:28:18.918 INFO:teuthology.orchestra.run.smithi067.stdout:stat: stdout 167 167 2024-03-30T02:28:18.918 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial keys... 2024-03-30T02:28:19.531 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQDDeAdmcX/kChAAjvRXuKdFptRO0PN/1yIbng== 2024-03-30T02:28:20.159 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQDDeAdmZ4KhLxAAKgh/ToDjEsEGwHOU+wtg5g== 2024-03-30T02:28:20.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQDEeAdmBIzBHBAA68k5nyebm4NNX9+xsq0Y5w== 2024-03-30T02:28:20.822 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial monmap... 2024-03-30T02:28:21.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-30T02:28:21.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-30T02:28:21.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:28:21.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-30T02:28:21.463 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool for a [v2:172.21.15.67:3300,v1:172.21.15.67:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-30T02:28:21.463 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = pacific 2024-03-30T02:28:21.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: set fsid to 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:28:21.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-30T02:28:21.463 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:28:21.464 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mon... 2024-03-30T02:28:22.088 INFO:teuthology.orchestra.run.smithi067.stdout:create mon.a on 2024-03-30T02:28:22.368 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-03-30T02:28:22.561 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-30T02:28:22.811 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4.target. 2024-03-30T02:28:22.811 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4.target. 2024-03-30T02:28:23.149 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.a 2024-03-30T02:28:23.149 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.a.service: Unit ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.a.service not loaded. 2024-03-30T02:28:23.351 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4.target.wants/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.a.service → /etc/systemd/system/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@.service. 2024-03-30T02:28:23.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:23 smithi067 systemd[1]: Starting Ceph mon.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:28:23.920 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-30T02:28:23.920 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-03-30T02:28:23.934 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:23 smithi067 bash[37277]: c34c5c48f1d9e1adf0b1e3c95f6f4118babcda578380040c15942dc4f40a83d6 2024-03-30T02:28:23.934 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:23 smithi067 systemd[1]: Started Ceph mon.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:28:23.934 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:23 smithi067 ceph-mon[37315]: mkfs 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:28:23.937 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-30T02:28:23.937 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-03-30T02:28:23.937 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-03-30T02:28:23.937 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-03-30T02:28:23.938 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon to start... 2024-03-30T02:28:23.938 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon... 2024-03-30T02:28:24.159 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:23 smithi067 ceph-mon[37315]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-30T02:28:24.861 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout cluster: 2024-03-30T02:28:24.867 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout id: 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:28:24.867 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout services: 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.585524s) 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout data: 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pgs: 2024-03-30T02:28:24.868 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.423+0000 7f666ac00700 1 Processor -- start 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.424+0000 7f666ac00700 1 -- start start 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.424+0000 7f666ac00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.424+0000 7f666ac00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666407a8c0 con 0x7f666407be90 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.425+0000 7f6663e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.425+0000 7f6663e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42264/0 (socket says 172.21.15.67:42264) 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.425+0000 7f6663e00700 1 -- 172.21.15.67:0/136709173 learned_addr learned my addr 172.21.15.67:0/136709173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.425+0000 7f6663e00700 1 -- 172.21.15.67:0/136709173 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666407aa00 con 0x7f666407be90 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.426+0000 7f6663e00700 1 --2- 172.21.15.67:0/136709173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666407a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66540098a0 tx=0x7f6654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6898f3ee810b41fa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.426+0000 7f6662a00700 1 -- 172.21.15.67:0/136709173 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6654004160 con 0x7f666407be90 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.426+0000 7f6662a00700 1 -- 172.21.15.67:0/136709173 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66540042c0 con 0x7f666407be90 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.426+0000 7f6662a00700 1 -- 172.21.15.67:0/136709173 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66540045a0 con 0x7f666407be90 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.426+0000 7f666ac00700 1 -- 172.21.15.67:0/136709173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 msgr2=0x7f666407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:24.869 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.426+0000 7f666ac00700 1 --2- 172.21.15.67:0/136709173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666407a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66540098a0 tx=0x7f6654009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.427+0000 7f666ac00700 1 -- 172.21.15.67:0/136709173 shutdown_connections 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.427+0000 7f666ac00700 1 --2- 172.21.15.67:0/136709173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666407a2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.427+0000 7f666ac00700 1 -- 172.21.15.67:0/136709173 >> 172.21.15.67:0/136709173 conn(0x7f6664101ae0 msgr2=0x7f6664103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.427+0000 7f666ac00700 1 -- 172.21.15.67:0/136709173 shutdown_connections 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.427+0000 7f666ac00700 1 -- 172.21.15.67:0/136709173 wait complete. 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.428+0000 7f666ac00700 1 Processor -- start 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.428+0000 7f666ac00700 1 -- start start 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.428+0000 7f666ac00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.428+0000 7f666ac00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666419c150 con 0x7f666407be90 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.428+0000 7f6663e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.429+0000 7f6663e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42270/0 (socket says 172.21.15.67:42270) 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.429+0000 7f6663e00700 1 -- 172.21.15.67:0/2481692911 learned_addr learned my addr 172.21.15.67:0/2481692911 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.429+0000 7f6663e00700 1 -- 172.21.15.67:0/2481692911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654009550 con 0x7f666407be90 2024-03-30T02:28:24.870 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.429+0000 7f6663e00700 1 --2- 172.21.15.67:0/2481692911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666419bc10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6654004000 tx=0x7f6654004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.429+0000 7f6660c00700 1 -- 172.21.15.67:0/2481692911 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6654011630 con 0x7f666407be90 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.429+0000 7f6660c00700 1 -- 172.21.15.67:0/2481692911 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654011790 con 0x7f666407be90 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.429+0000 7f666ac00700 1 -- 172.21.15.67:0/2481692911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666419c350 con 0x7f666407be90 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.430+0000 7f6660c00700 1 -- 172.21.15.67:0/2481692911 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6654011a30 con 0x7f666407be90 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.430+0000 7f666ac00700 1 -- 172.21.15.67:0/2481692911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666419c7f0 con 0x7f666407be90 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.430+0000 7f6660c00700 1 -- 172.21.15.67:0/2481692911 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6654022070 con 0x7f666407be90 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.430+0000 7f6660c00700 1 -- 172.21.15.67:0/2481692911 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f665401c450 con 0x7f666407be90 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.431+0000 7f666ac00700 1 -- 172.21.15.67:0/2481692911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6664195a30 con 0x7f666407be90 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.434+0000 7f6660c00700 1 -- 172.21.15.67:0/2481692911 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f665401b450 con 0x7f666407be90 2024-03-30T02:28:24.871 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.510+0000 7f666ac00700 1 -- 172.21.15.67:0/2481692911 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f666407af10 con 0x7f666407be90 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.510+0000 7f6660c00700 1 -- 172.21.15.67:0/2481692911 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f665401b5f0 con 0x7f666407be90 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.511+0000 7f666ac00700 1 -- 172.21.15.67:0/2481692911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 msgr2=0x7f666419bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.511+0000 7f666ac00700 1 --2- 172.21.15.67:0/2481692911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666419bc10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6654004000 tx=0x7f6654004750 comp rx=0 tx=0).stop 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.511+0000 7f666ac00700 1 -- 172.21.15.67:0/2481692911 shutdown_connections 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.511+0000 7f666ac00700 1 --2- 172.21.15.67:0/2481692911 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f666407be90 0x7f666419bc10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.511+0000 7f666ac00700 1 -- 172.21.15.67:0/2481692911 >> 172.21.15.67:0/2481692911 conn(0x7f6664101ae0 msgr2=0x7f6664103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.511+0000 7f666ac00700 1 -- 172.21.15.67:0/2481692911 shutdown_connections 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:24.511+0000 7f666ac00700 1 -- 172.21.15.67:0/2481692911 wait complete. 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:mon is available 2024-03-30T02:28:24.872 INFO:teuthology.orchestra.run.smithi067.stdout:Assimilating anything we can from ceph.conf... 2024-03-30T02:28:25.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:24 smithi067 ceph-mon[37315]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-30T02:28:25.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:24 smithi067 ceph-mon[37315]: monmap e1: 1 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:28:25.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:24 smithi067 ceph-mon[37315]: fsmap 2024-03-30T02:28:25.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:24 smithi067 ceph-mon[37315]: osdmap e1: 0 total, 0 up, 0 in 2024-03-30T02:28:25.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:24 smithi067 ceph-mon[37315]: mgrmap e1: no daemons active 2024-03-30T02:28:25.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:24 smithi067 ceph-mon[37315]: from='client.? 172.21.15.67:0/2481692911' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-30T02:28:25.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:25.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-03-30T02:28:25.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-30T02:28:25.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.287+0000 7fc517800700 1 Processor -- start 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.288+0000 7fc517800700 1 -- start start 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.288+0000 7fc517800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.288+0000 7fc517800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51007a8a0 con 0x7fc51007be70 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.289+0000 7fc514e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.289+0000 7fc514e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42280/0 (socket says 172.21.15.67:42280) 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.289+0000 7fc514e00700 1 -- 172.21.15.67:0/3079399439 learned_addr learned my addr 172.21.15.67:0/3079399439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.289+0000 7fc514e00700 1 -- 172.21.15.67:0/3079399439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51007a9e0 con 0x7fc51007be70 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.289+0000 7fc514e00700 1 --2- 172.21.15.67:0/3079399439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51007a2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc5040098a0 tx=0x7fc504009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cdf11ee3ee4fdf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc50f400700 1 -- 172.21.15.67:0/3079399439 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc504004160 con 0x7fc51007be70 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc50f400700 1 -- 172.21.15.67:0/3079399439 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5040042c0 con 0x7fc51007be70 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc50f400700 1 -- 172.21.15.67:0/3079399439 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5040045a0 con 0x7fc51007be70 2024-03-30T02:28:25.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc517800700 1 -- 172.21.15.67:0/3079399439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 msgr2=0x7fc51007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc517800700 1 --2- 172.21.15.67:0/3079399439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51007a2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc5040098a0 tx=0x7fc504009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc517800700 1 -- 172.21.15.67:0/3079399439 shutdown_connections 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc517800700 1 --2- 172.21.15.67:0/3079399439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51007a2d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc517800700 1 -- 172.21.15.67:0/3079399439 >> 172.21.15.67:0/3079399439 conn(0x7fc510101b40 msgr2=0x7fc510103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc517800700 1 -- 172.21.15.67:0/3079399439 shutdown_connections 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.290+0000 7fc517800700 1 -- 172.21.15.67:0/3079399439 wait complete. 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.291+0000 7fc517800700 1 Processor -- start 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.292+0000 7fc517800700 1 -- start start 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.292+0000 7fc517800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.292+0000 7fc517800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51019c190 con 0x7fc51007be70 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.292+0000 7fc514e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.292+0000 7fc514e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42290/0 (socket says 172.21.15.67:42290) 2024-03-30T02:28:25.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.292+0000 7fc514e00700 1 -- 172.21.15.67:0/4259667645 learned_addr learned my addr 172.21.15.67:0/4259667645 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.293+0000 7fc514e00700 1 -- 172.21.15.67:0/4259667645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc504009550 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.293+0000 7fc514e00700 1 --2- 172.21.15.67:0/4259667645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51019bc50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc51007b340 tx=0x7fc504004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.293+0000 7fc50d600700 1 -- 172.21.15.67:0/4259667645 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc504011630 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.293+0000 7fc50d600700 1 -- 172.21.15.67:0/4259667645 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc504011790 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.293+0000 7fc50d600700 1 -- 172.21.15.67:0/4259667645 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc504011a30 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.293+0000 7fc517800700 1 -- 172.21.15.67:0/4259667645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51019c390 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.293+0000 7fc517800700 1 -- 172.21.15.67:0/4259667645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51019c830 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.293+0000 7fc50d600700 1 -- 172.21.15.67:0/4259667645 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc504021070 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.294+0000 7fc50d600700 1 -- 172.21.15.67:0/4259667645 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc50401b440 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.294+0000 7fc517800700 1 -- 172.21.15.67:0/4259667645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc510195b60 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.297+0000 7fc50d600700 1 -- 172.21.15.67:0/4259667645 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fc50401a460 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.369+0000 7fc517800700 1 -- 172.21.15.67:0/4259667645 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc51007af80 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.374+0000 7fc50d600700 1 -- 172.21.15.67:0/4259667645 <== mon.0 v2:172.21.15.67:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc50401a600 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.374+0000 7fc50d600700 1 -- 172.21.15.67:0/4259667645 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fc504035590 con 0x7fc51007be70 2024-03-30T02:28:25.779 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.375+0000 7fc517800700 1 -- 172.21.15.67:0/4259667645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 msgr2=0x7fc51019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:25.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.375+0000 7fc517800700 1 --2- 172.21.15.67:0/4259667645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51019bc50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc51007b340 tx=0x7fc504004750 comp rx=0 tx=0).stop 2024-03-30T02:28:25.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.375+0000 7fc517800700 1 -- 172.21.15.67:0/4259667645 shutdown_connections 2024-03-30T02:28:25.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.375+0000 7fc517800700 1 --2- 172.21.15.67:0/4259667645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc51007be70 0x7fc51019bc50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:25.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.375+0000 7fc517800700 1 -- 172.21.15.67:0/4259667645 >> 172.21.15.67:0/4259667645 conn(0x7fc510101b40 msgr2=0x7fc510103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:25.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.375+0000 7fc517800700 1 -- 172.21.15.67:0/4259667645 shutdown_connections 2024-03-30T02:28:25.780 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:25.375+0000 7fc517800700 1 -- 172.21.15.67:0/4259667645 wait complete. 2024-03-30T02:28:25.780 INFO:teuthology.orchestra.run.smithi067.stdout:Generating new minimal ceph.conf... 2024-03-30T02:28:26.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:25 smithi067 ceph-mon[37315]: from='client.? 172.21.15.67:0/4259667645' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-30T02:28:26.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:25 smithi067 ceph-mon[37315]: from='client.? 172.21.15.67:0/4259667645' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.187+0000 7fd0d1800700 1 Processor -- start 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.187+0000 7fd0d1800700 1 -- start start 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.188+0000 7fd0d1800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.188+0000 7fd0d1800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc07a8a0 con 0x7fd0cc07be70 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.188+0000 7fd0caa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.188+0000 7fd0caa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56548/0 (socket says 172.21.15.67:56548) 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.188+0000 7fd0caa00700 1 -- 172.21.15.67:0/2563973879 learned_addr learned my addr 172.21.15.67:0/2563973879 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.188+0000 7fd0caa00700 1 -- 172.21.15.67:0/2563973879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc07a9e0 con 0x7fd0cc07be70 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.189+0000 7fd0caa00700 1 --2- 172.21.15.67:0/2563973879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc07a2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0098a0 tx=0x7fd0bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab65d7e6a911be5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.189+0000 7fd0ca000700 1 -- 172.21.15.67:0/2563973879 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0bc004160 con 0x7fd0cc07be70 2024-03-30T02:28:26.638 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.189+0000 7fd0ca000700 1 -- 172.21.15.67:0/2563973879 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0bc0042c0 con 0x7fd0cc07be70 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.189+0000 7fd0ca000700 1 -- 172.21.15.67:0/2563973879 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0bc011a00 con 0x7fd0cc07be70 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.189+0000 7fd0d1800700 1 -- 172.21.15.67:0/2563973879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 msgr2=0x7fd0cc07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.189+0000 7fd0d1800700 1 --2- 172.21.15.67:0/2563973879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc07a2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0098a0 tx=0x7fd0bc009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.190+0000 7fd0d1800700 1 -- 172.21.15.67:0/2563973879 shutdown_connections 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.190+0000 7fd0d1800700 1 --2- 172.21.15.67:0/2563973879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc07a2d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.190+0000 7fd0d1800700 1 -- 172.21.15.67:0/2563973879 >> 172.21.15.67:0/2563973879 conn(0x7fd0cc101b40 msgr2=0x7fd0cc103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.190+0000 7fd0d1800700 1 -- 172.21.15.67:0/2563973879 shutdown_connections 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.190+0000 7fd0d1800700 1 -- 172.21.15.67:0/2563973879 wait complete. 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.191+0000 7fd0d1800700 1 Processor -- start 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.191+0000 7fd0d1800700 1 -- start start 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.191+0000 7fd0d1800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.191+0000 7fd0d1800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc19ca00 con 0x7fd0cc07be70 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.191+0000 7fd0caa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.191+0000 7fd0caa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56560/0 (socket says 172.21.15.67:56560) 2024-03-30T02:28:26.639 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.191+0000 7fd0caa00700 1 -- 172.21.15.67:0/3451788567 learned_addr learned my addr 172.21.15.67:0/3451788567 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.192+0000 7fd0caa00700 1 -- 172.21.15.67:0/3451788567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc009550 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.192+0000 7fd0caa00700 1 --2- 172.21.15.67:0/3451788567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc19c4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009130 tx=0x7fd0bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.192+0000 7fd0c3400700 1 -- 172.21.15.67:0/3451788567 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0bc0038b0 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.192+0000 7fd0c3400700 1 -- 172.21.15.67:0/3451788567 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0bc003a10 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.192+0000 7fd0d1800700 1 -- 172.21.15.67:0/3451788567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc19cc00 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.192+0000 7fd0c3400700 1 -- 172.21.15.67:0/3451788567 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0bc019550 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.193+0000 7fd0d1800700 1 -- 172.21.15.67:0/3451788567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc19d0a0 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.193+0000 7fd0c3400700 1 -- 172.21.15.67:0/3451788567 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd0bc019b10 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.193+0000 7fd0c3400700 1 -- 172.21.15.67:0/3451788567 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd0bc02de30 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.194+0000 7fd0d1800700 1 -- 172.21.15.67:0/3451788567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc10c970 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.196+0000 7fd0c3400700 1 -- 172.21.15.67:0/3451788567 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fd0bc003b80 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.270+0000 7fd0d1800700 1 -- 172.21.15.67:0/3451788567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd0cc10cbd0 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.270+0000 7fd0c3400700 1 -- 172.21.15.67:0/3451788567 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fd0bc01a960 con 0x7fd0cc07be70 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.271+0000 7fd0d1800700 1 -- 172.21.15.67:0/3451788567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 msgr2=0x7fd0cc19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:26.640 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.271+0000 7fd0d1800700 1 --2- 172.21.15.67:0/3451788567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc19c4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009130 tx=0x7fd0bc011ad0 comp rx=0 tx=0).stop 2024-03-30T02:28:26.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.271+0000 7fd0d1800700 1 -- 172.21.15.67:0/3451788567 shutdown_connections 2024-03-30T02:28:26.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.271+0000 7fd0d1800700 1 --2- 172.21.15.67:0/3451788567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0cc07be70 0x7fd0cc19c4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:26.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.271+0000 7fd0d1800700 1 -- 172.21.15.67:0/3451788567 >> 172.21.15.67:0/3451788567 conn(0x7fd0cc101b40 msgr2=0x7fd0cc1026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:26.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.271+0000 7fd0d1800700 1 -- 172.21.15.67:0/3451788567 shutdown_connections 2024-03-30T02:28:26.641 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:26.271+0000 7fd0d1800700 1 -- 172.21.15.67:0/3451788567 wait complete. 2024-03-30T02:28:26.641 INFO:teuthology.orchestra.run.smithi067.stdout:Restarting the monitor... 2024-03-30T02:28:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:26 smithi067 systemd[1]: Stopping Ceph mon.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:28:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:26 smithi067 ceph-mon[37315]: from='client.? 172.21.15.67:0/3451788567' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:28:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-a[37292]: 2024-03-30T02:28:26.939+0000 7f5d27800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:28:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-a[37292]: 2024-03-30T02:28:26.939+0000 7f5d27800700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-30T02:28:31.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:31 smithi067 bash[37646]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-a 2024-03-30T02:28:34.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:34 smithi067 systemd[1]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.a.service: Succeeded. 2024-03-30T02:28:34.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:34 smithi067 systemd[1]: Stopped Ceph mon.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:28:34.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:34 smithi067 systemd[1]: Starting Ceph mon.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:28:35.863 INFO:teuthology.orchestra.run.smithi067.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-30T02:28:36.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 bash[37770]: e3ad484b6d89ccd5085acc38bdaee51f7769b150b87ed8c82651ce4051762bfe 2024-03-30T02:28:36.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: set uid:gid to 167:167 (ceph:ceph) 2024-03-30T02:28:36.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable), process ceph-mon, pid 7 2024-03-30T02:28:36.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: pidfile_write: ignore empty --pid-file 2024-03-30T02:28:36.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 systemd[1]: Started Ceph mon.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:28:36.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: load: jerasure load: lrc 2024-03-30T02:28:36.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: RocksDB version: 7.9.2 2024-03-30T02:28:36.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Git sha 0 2024-03-30T02:28:36.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Compile date 2024-03-29 11:49:16 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: DB SUMMARY 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: DB Session ID: SF37A9RHUEZB4A6FRLS7 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: CURRENT file: CURRENT 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: IDENTITY file: IDENTITY 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99425 ; 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.error_if_exists: 0 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.create_if_missing: 0 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.paranoid_checks: 1 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-30T02:28:36.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.env: 0x5621099c0720 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.fs: PosixFileSystem 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.info_log: 0x56210c36f520 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_file_opening_threads: 16 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.statistics: (nil) 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.use_fsync: 0 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_log_file_size: 0 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.keep_log_file_num: 1000 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.recycle_log_file_num: 0 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.allow_fallocate: 1 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.allow_mmap_reads: 0 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.allow_mmap_writes: 0 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.use_direct_reads: 0 2024-03-30T02:28:36.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-30T02:28:36.168 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.create_missing_column_families: 0 2024-03-30T02:28:36.168 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.db_log_dir: 2024-03-30T02:28:36.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.wal_dir: 2024-03-30T02:28:36.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-30T02:28:36.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-30T02:28:36.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-30T02:28:36.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-30T02:28:36.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-30T02:28:36.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-30T02:28:36.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.advise_random_on_open: 1 2024-03-30T02:28:36.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.db_write_buffer_size: 0 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.write_buffer_manager: 0x56210b606320 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.rate_limiter: (nil) 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.wal_recovery_mode: 2 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.enable_thread_tracking: 0 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.enable_pipelined_write: 0 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.unordered_write: 0 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-30T02:28:36.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.row_cache: None 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.wal_filter: None 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.allow_ingest_behind: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.two_write_queues: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.manual_wal_flush: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.wal_compression: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.atomic_flush: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.log_readahead_size: 0 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-30T02:28:36.171 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.best_efforts_recovery: 0 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.allow_data_in_errors: 0 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.db_host_id: __hostname__ 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_background_jobs: 2 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_background_compactions: -1 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_subcompactions: 1 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_total_wal_size: 0 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-30T02:28:36.172 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_open_files: -1 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bytes_per_sync: 0 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_readahead_size: 0 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_background_flushes: -1 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Compression algorithms supported: 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: kZSTD supported: 0 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: kXpressCompression supported: 0 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: kLZ4HCCompression supported: 1 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: kZlibCompression supported: 1 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: kSnappyCompression supported: 1 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: kLZ4Compression supported: 1 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: kBZip2Compression supported: 0 2024-03-30T02:28:36.173 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.merge_operator: 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_filter: None 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_filter_factory: None 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.sst_partitioner_factory: None 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56210c36f620) 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks: 1 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-30T02:28:36.174 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_top_level_index_and_filter: 1 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_type: 0 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_index_type: 0 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_shortening: 1 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: checksum: 4 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: no_block_cache: 0 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache: 0x56210b687350 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_name: BinnedLRUCache 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_options: 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: capacity : 536870912 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_shard_bits : 4 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: strict_capacity_limit : 0 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: high_pri_pool_ratio: 0.000 2024-03-30T02:28:36.175 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_compressed: (nil) 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: persistent_cache: (nil) 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size: 4096 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size_deviation: 10 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_restart_interval: 16 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_block_restart_interval: 1 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: metadata_block_size: 4096 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: partition_filters: 0 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: use_delta_encoding: 1 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: filter_policy: bloomfilter 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: whole_key_filtering: 1 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: verify_compression: 0 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: read_amp_bytes_per_bit: 0 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: format_version: 5 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_index_compression: 1 2024-03-30T02:28:36.176 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_align: 0 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout: max_auto_readahead_size: 262144 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout: prepopulate_block_cache: 0 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout: initial_auto_readahead_size: 8192 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_file_reads_for_auto_readahead: 2 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.write_buffer_size: 33554432 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_write_buffer_number: 2 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression: NoCompression 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression: Disabled 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.prefix_extractor: nullptr 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.num_levels: 7 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-30T02:28:36.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression_opts.level: 32767 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression_opts.strategy: 0 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression_opts.enabled: false 2024-03-30T02:28:36.178 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.target_file_size_base: 67108864 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-30T02:28:36.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.arena_block_size: 1048576 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.disable_auto_compactions: 0 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-30T02:28:36.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.table_properties_collectors: 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.inplace_update_support: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.bloom_locality: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.max_successive_merges: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.paranoid_file_checks: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.force_consistency_checks: 1 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.report_bg_io_stats: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.ttl: 2592000 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.enable_blob_files: false 2024-03-30T02:28:36.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.min_blob_size: 0 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.blob_file_size: 268435456 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.blob_file_starting_level: 0 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e2701cf8-ff79-4011-a5b1-3839f14f35cc 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711765715869782, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-30T02:28:36.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711765715872089, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94627, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92632, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13781, "raw_average_key_size": 50, "raw_value_size": 85105, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711765715, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e2701cf8-ff79-4011-a5b1-3839f14f35cc", "db_session_id": "SF37A9RHUEZB4A6FRLS7", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711765715872248, "job": 1, "event": "recovery_finished"} 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56210b724000 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: DB pointer 0x56210b712000 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** DB Stats ** 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-30T02:28:36.183 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: L0 2/0 94.24 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: Sum 2/0 94.24 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 74.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-03-30T02:28:36.184 INFO:journalctl@ceph.mon.a.smithi067.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative compaction: 0.00 GB write, 11.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval compaction: 0.00 GB write, 11.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache BinnedLRUCache@0x56210b687350#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-30T02:28:36.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: starting mon.a rank 0 at public addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] at bind addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mon.a@-1(???) e1 preinit fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mon.a@-1(???).mds e1 new map 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mon.a@-1(???).mds e1 print_map 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout: e1 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout: legacy client fscid: -1 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout: No filesystems configured 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T02:28:36.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-30T02:28:36.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-30T02:28:36.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: monmap e1: 1 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:28:36.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: fsmap 2024-03-30T02:28:36.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: osdmap e1: 0 total, 0 up, 0 in 2024-03-30T02:28:36.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:35 smithi067 ceph-mon[37808]: mgrmap e1: no daemons active 2024-03-30T02:28:36.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.297+0000 7f2274200700 1 Processor -- start 2024-03-30T02:28:36.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.297+0000 7f2274200700 1 -- start start 2024-03-30T02:28:36.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.297+0000 7f2274200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:36.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.297+0000 7f2274200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c106ef0 con 0x7f226c106500 2024-03-30T02:28:36.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.298+0000 7f2271800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:36.775 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.298+0000 7f2271800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34216/0 (socket says 172.21.15.67:34216) 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.298+0000 7f2271800700 1 -- 172.21.15.67:0/789547206 learned_addr learned my addr 172.21.15.67:0/789547206 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.298+0000 7f2271800700 1 -- 172.21.15.67:0/789547206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c107700 con 0x7f226c106500 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.299+0000 7f2271800700 1 --2- 172.21.15.67:0/789547206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c106920 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22600098a0 tx=0x7f2260009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c4eb405d50c4a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.299+0000 7f226be00700 1 -- 172.21.15.67:0/789547206 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2260004160 con 0x7f226c106500 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.299+0000 7f226be00700 1 -- 172.21.15.67:0/789547206 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22600042c0 con 0x7f226c106500 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.299+0000 7f226be00700 1 -- 172.21.15.67:0/789547206 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2260011a00 con 0x7f226c106500 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.299+0000 7f2274200700 1 -- 172.21.15.67:0/789547206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 msgr2=0x7f226c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.299+0000 7f2274200700 1 --2- 172.21.15.67:0/789547206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c106920 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22600098a0 tx=0x7f2260009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.300+0000 7f2274200700 1 -- 172.21.15.67:0/789547206 shutdown_connections 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.300+0000 7f2274200700 1 --2- 172.21.15.67:0/789547206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c106920 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.300+0000 7f2274200700 1 -- 172.21.15.67:0/789547206 >> 172.21.15.67:0/789547206 conn(0x7f226c101ae0 msgr2=0x7f226c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.300+0000 7f2274200700 1 -- 172.21.15.67:0/789547206 shutdown_connections 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.300+0000 7f2274200700 1 -- 172.21.15.67:0/789547206 wait complete. 2024-03-30T02:28:36.776 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.301+0000 7f2274200700 1 Processor -- start 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.301+0000 7f2274200700 1 -- start start 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.302+0000 7f2274200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.302+0000 7f2274200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c19a840 con 0x7f226c106500 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.302+0000 7f2271800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.302+0000 7f2271800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34218/0 (socket says 172.21.15.67:34218) 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.302+0000 7f2271800700 1 -- 172.21.15.67:0/3149515903 learned_addr learned my addr 172.21.15.67:0/3149515903 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.302+0000 7f2271800700 1 -- 172.21.15.67:0/3149515903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2260009550 con 0x7f226c106500 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.303+0000 7f2271800700 1 --2- 172.21.15.67:0/3149515903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c19a300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2260009130 tx=0x7f2260011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.303+0000 7f226a000700 1 -- 172.21.15.67:0/3149515903 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22600038b0 con 0x7f226c106500 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.303+0000 7f226a000700 1 -- 172.21.15.67:0/3149515903 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2260003a10 con 0x7f226c106500 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.303+0000 7f226a000700 1 -- 172.21.15.67:0/3149515903 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2260019550 con 0x7f226c106500 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.303+0000 7f2274200700 1 -- 172.21.15.67:0/3149515903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226c19aa40 con 0x7f226c106500 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.303+0000 7f2274200700 1 -- 172.21.15.67:0/3149515903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c19aee0 con 0x7f226c106500 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.303+0000 7f226a000700 1 -- 172.21.15.67:0/3149515903 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2260019b10 con 0x7f226c106500 2024-03-30T02:28:36.777 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.304+0000 7f226a000700 1 -- 172.21.15.67:0/3149515903 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f226002de30 con 0x7f226c106500 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.304+0000 7f2274200700 1 -- 172.21.15.67:0/3149515903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c0623e0 con 0x7f226c106500 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.307+0000 7f226a000700 1 -- 172.21.15.67:0/3149515903 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f2260003b80 con 0x7f226c106500 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.380+0000 7f2274200700 1 -- 172.21.15.67:0/3149515903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f226c19d770 con 0x7f226c106500 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.386+0000 7f226a000700 1 -- 172.21.15.67:0/3149515903 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f226001a960 con 0x7f226c106500 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.386+0000 7f2274200700 1 -- 172.21.15.67:0/3149515903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 msgr2=0x7f226c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.386+0000 7f2274200700 1 --2- 172.21.15.67:0/3149515903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c19a300 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2260009130 tx=0x7f2260011ad0 comp rx=0 tx=0).stop 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.387+0000 7f2274200700 1 -- 172.21.15.67:0/3149515903 shutdown_connections 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.387+0000 7f2274200700 1 --2- 172.21.15.67:0/3149515903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f226c106500 0x7f226c19a300 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.387+0000 7f2274200700 1 -- 172.21.15.67:0/3149515903 >> 172.21.15.67:0/3149515903 conn(0x7f226c101ae0 msgr2=0x7f226c102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.387+0000 7f2274200700 1 -- 172.21.15.67:0/3149515903 shutdown_connections 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:36.387+0000 7f2274200700 1 -- 172.21.15.67:0/3149515903 wait complete. 2024-03-30T02:28:36.778 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-30T02:28:36.801 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-30T02:28:36.801 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mgr... 2024-03-30T02:28:36.801 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-30T02:28:36.802 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-30T02:28:37.171 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.x 2024-03-30T02:28:37.171 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.x.service: Unit ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.x.service not loaded. 2024-03-30T02:28:37.371 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4.target.wants/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.x.service → /etc/systemd/system/ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@.service. 2024-03-30T02:28:37.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:37 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3149515903' entity='client.admin' 2024-03-30T02:28:37.664 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:37 smithi067 systemd[1]: Starting Ceph mgr.x for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:28:37.935 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-30T02:28:37.935 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-03-30T02:28:37.952 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-30T02:28:37.952 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-03-30T02:28:37.953 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-03-30T02:28:37.953 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-03-30T02:28:37.970 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-30T02:28:37.970 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-03-30T02:28:37.986 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-30T02:28:37.986 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-03-30T02:28:37.986 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-03-30T02:28:37.986 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-30T02:28:37.986 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr to start... 2024-03-30T02:28:37.986 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr... 2024-03-30T02:28:38.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:37 smithi067 bash[38042]: e4f3bcb9efcded80f102ea2a02fb09297cad364bf9e5761d895c013d9d95b1fd 2024-03-30T02:28:38.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:37 smithi067 systemd[1]: Started Ceph mgr.x for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:28:38.723 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:38 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/922373463' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-30T02:28:38.723 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:38 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:38.467+0000 7f0d9f164200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T02:28:38.723 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:38 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:38.599+0000 7f0d9f164200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T02:28:38.723 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:38 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:38.718+0000 7f0d9f164200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T02:28:38.896 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:38.896 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-03-30T02:28:38.896 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:28:38.896 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-03-30T02:28:38.896 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-30T02:28:38.896 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:38.897 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-30T02:28:38.906 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-30T02:28:38.907 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-03-30T02:28:23.927930+0000", 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.417+0000 7f0e37e00700 1 Processor -- start 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.417+0000 7f0e37e00700 1 -- start start 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.417+0000 7f0e37e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3007a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.417+0000 7f0e37e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3007a880 con 0x7f0e3007be50 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.418+0000 7f0e35400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3007a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:38.908 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.418+0000 7f0e35400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3007a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34244/0 (socket says 172.21.15.67:34244) 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.418+0000 7f0e35400700 1 -- 172.21.15.67:0/536535840 learned_addr learned my addr 172.21.15.67:0/536535840 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.418+0000 7f0e35400700 1 -- 172.21.15.67:0/536535840 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3007a9c0 con 0x7f0e3007be50 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.419+0000 7f0e35400700 1 --2- 172.21.15.67:0/536535840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3007a2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e240098a0 tx=0x7f0e24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49d60239db2aace server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.419+0000 7f0e2fe00700 1 -- 172.21.15.67:0/536535840 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e24004160 con 0x7f0e3007be50 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.419+0000 7f0e2fe00700 1 -- 172.21.15.67:0/536535840 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e240042c0 con 0x7f0e3007be50 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.419+0000 7f0e2fe00700 1 -- 172.21.15.67:0/536535840 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e24011a00 con 0x7f0e3007be50 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.420+0000 7f0e37e00700 1 -- 172.21.15.67:0/536535840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 msgr2=0x7f0e3007a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.420+0000 7f0e37e00700 1 --2- 172.21.15.67:0/536535840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3007a2b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e240098a0 tx=0x7f0e24009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.420+0000 7f0e37e00700 1 -- 172.21.15.67:0/536535840 shutdown_connections 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.420+0000 7f0e37e00700 1 --2- 172.21.15.67:0/536535840 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3007a2b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.420+0000 7f0e37e00700 1 -- 172.21.15.67:0/536535840 >> 172.21.15.67:0/536535840 conn(0x7f0e30101b40 msgr2=0x7f0e30103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.420+0000 7f0e37e00700 1 -- 172.21.15.67:0/536535840 shutdown_connections 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.420+0000 7f0e37e00700 1 -- 172.21.15.67:0/536535840 wait complete. 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.421+0000 7f0e37e00700 1 Processor -- start 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.421+0000 7f0e37e00700 1 -- start start 2024-03-30T02:28:38.909 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.421+0000 7f0e37e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.421+0000 7f0e37e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3019ca00 con 0x7f0e3007be50 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.422+0000 7f0e35400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.422+0000 7f0e35400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34254/0 (socket says 172.21.15.67:34254) 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.422+0000 7f0e35400700 1 -- 172.21.15.67:0/922373463 learned_addr learned my addr 172.21.15.67:0/922373463 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.422+0000 7f0e35400700 1 -- 172.21.15.67:0/922373463 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e24009550 con 0x7f0e3007be50 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.422+0000 7f0e35400700 1 --2- 172.21.15.67:0/922373463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3019c4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e24011a20 tx=0x7f0e24011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.423+0000 7f0e2e000700 1 -- 172.21.15.67:0/922373463 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e24003af0 con 0x7f0e3007be50 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.423+0000 7f0e2e000700 1 -- 172.21.15.67:0/922373463 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e24003c50 con 0x7f0e3007be50 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.423+0000 7f0e2e000700 1 -- 172.21.15.67:0/922373463 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e24019760 con 0x7f0e3007be50 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.423+0000 7f0e37e00700 1 -- 172.21.15.67:0/922373463 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3019cc00 con 0x7f0e3007be50 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.423+0000 7f0e37e00700 1 -- 172.21.15.67:0/922373463 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3019d0a0 con 0x7f0e3007be50 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.423+0000 7f0e2e000700 1 -- 172.21.15.67:0/922373463 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0e24019950 con 0x7f0e3007be50 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.423+0000 7f0e2e000700 1 -- 172.21.15.67:0/922373463 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0e2400b3e0 con 0x7f0e3007be50 2024-03-30T02:28:38.910 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.424+0000 7f0e37e00700 1 -- 172.21.15.67:0/922373463 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3010c930 con 0x7f0e3007be50 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.427+0000 7f0e2e000700 1 -- 172.21.15.67:0/922373463 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f0e2402a060 con 0x7f0e3007be50 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.504+0000 7f0e37e00700 1 -- 172.21.15.67:0/922373463 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0e3010cb60 con 0x7f0e3007be50 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.504+0000 7f0e2e000700 1 -- 172.21.15.67:0/922373463 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f0e24019d90 con 0x7f0e3007be50 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.505+0000 7f0e37e00700 1 -- 172.21.15.67:0/922373463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 msgr2=0x7f0e3019c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.505+0000 7f0e37e00700 1 --2- 172.21.15.67:0/922373463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3019c4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0e24011a20 tx=0x7f0e24011b00 comp rx=0 tx=0).stop 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.505+0000 7f0e37e00700 1 -- 172.21.15.67:0/922373463 shutdown_connections 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.505+0000 7f0e37e00700 1 --2- 172.21.15.67:0/922373463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e3007be50 0x7f0e3019c4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.505+0000 7f0e37e00700 1 -- 172.21.15.67:0/922373463 >> 172.21.15.67:0/922373463 conn(0x7f0e30101b40 msgr2=0x7f0e301026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.505+0000 7f0e37e00700 1 -- 172.21.15.67:0/922373463 shutdown_connections 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:38.505+0000 7f0e37e00700 1 -- 172.21.15.67:0/922373463 wait complete. 2024-03-30T02:28:38.911 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (1/15)... 2024-03-30T02:28:39.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:38 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:38.836+0000 7f0d9f164200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T02:28:40.058 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:39 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:39.732+0000 7f0d9f164200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:28:40.319 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:40.056+0000 7f0d9f164200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T02:28:40.319 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:40.172+0000 7f0d9f164200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T02:28:40.319 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:40.317+0000 7f0d9f164200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T02:28:40.664 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:40.519+0000 7f0d9f164200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T02:28:41.053 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:40.695+0000 7f0d9f164200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T02:28:41.053 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:40.810+0000 7f0d9f164200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T02:28:41.648 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:41.331+0000 7f0d9f164200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T02:28:41.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-30T02:28:41.807 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-30T02:28:41.808 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-30T02:28:41.823 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-30T02:28:41.823 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-30T02:28:41.823 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-30T02:28:41.823 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-30T02:28:41.823 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:41.824 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-30T02:28:41.824 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-30T02:28:41.824 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-30T02:28:41.824 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-30T02:28:41.824 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-30T02:28:41.824 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-30T02:28:41.824 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-30T02:28:41.824 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-30T02:28:41.825 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-30T02:28:41.826 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-03-30T02:28:41.826 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:41.826 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:41.826 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:41.826 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-30T02:28:41.826 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:41.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-03-30T02:28:23.927930+0000", 2024-03-30T02:28:41.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:41.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:41.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-30T02:28:41.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-03-30T02:28:41.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.335+0000 7f4c1c600700 1 Processor -- start 2024-03-30T02:28:41.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.335+0000 7f4c1c600700 1 -- start start 2024-03-30T02:28:41.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.335+0000 7f4c1c600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c141069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:41.827 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.335+0000 7f4c1c600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14106f70 con 0x7f4c14106580 2024-03-30T02:28:41.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.336+0000 7f4c19c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c141069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:41.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.336+0000 7f4c19c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c141069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34258/0 (socket says 172.21.15.67:34258) 2024-03-30T02:28:41.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.336+0000 7f4c19c00700 1 -- 172.21.15.67:0/1792336457 learned_addr learned my addr 172.21.15.67:0/1792336457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:41.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.336+0000 7f4c19c00700 1 -- 172.21.15.67:0/1792336457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14107780 con 0x7f4c14106580 2024-03-30T02:28:41.828 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.337+0000 7f4c19c00700 1 --2- 172.21.15.67:0/1792336457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c141069a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c080098a0 tx=0x7f4c08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c1299820228ed2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:41.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.337+0000 7f4c13e00700 1 -- 172.21.15.67:0/1792336457 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c08004160 con 0x7f4c14106580 2024-03-30T02:28:41.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.337+0000 7f4c13e00700 1 -- 172.21.15.67:0/1792336457 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c080042c0 con 0x7f4c14106580 2024-03-30T02:28:41.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.337+0000 7f4c13e00700 1 -- 172.21.15.67:0/1792336457 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c08011a00 con 0x7f4c14106580 2024-03-30T02:28:41.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.337+0000 7f4c1c600700 1 -- 172.21.15.67:0/1792336457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 msgr2=0x7f4c141069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:41.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.337+0000 7f4c1c600700 1 --2- 172.21.15.67:0/1792336457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c141069a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c080098a0 tx=0x7f4c08009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:41.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.338+0000 7f4c1c600700 1 -- 172.21.15.67:0/1792336457 shutdown_connections 2024-03-30T02:28:41.829 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.338+0000 7f4c1c600700 1 --2- 172.21.15.67:0/1792336457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c141069a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.338+0000 7f4c1c600700 1 -- 172.21.15.67:0/1792336457 >> 172.21.15.67:0/1792336457 conn(0x7f4c14101b40 msgr2=0x7f4c14103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.338+0000 7f4c1c600700 1 -- 172.21.15.67:0/1792336457 shutdown_connections 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.338+0000 7f4c1c600700 1 -- 172.21.15.67:0/1792336457 wait complete. 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.339+0000 7f4c1c600700 1 Processor -- start 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.339+0000 7f4c1c600700 1 -- start start 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.339+0000 7f4c1c600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c1419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.339+0000 7f4c1c600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1419ca30 con 0x7f4c14106580 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.339+0000 7f4c19c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c1419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.340+0000 7f4c19c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c1419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34272/0 (socket says 172.21.15.67:34272) 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.340+0000 7f4c19c00700 1 -- 172.21.15.67:0/3258454459 learned_addr learned my addr 172.21.15.67:0/3258454459 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.340+0000 7f4c19c00700 1 -- 172.21.15.67:0/3258454459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c08009550 con 0x7f4c14106580 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.340+0000 7f4c19c00700 1 --2- 172.21.15.67:0/3258454459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c1419c4f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4c08011710 tx=0x7f4c08011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:41.830 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.340+0000 7f4c12000700 1 -- 172.21.15.67:0/3258454459 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c08003aa0 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.340+0000 7f4c12000700 1 -- 172.21.15.67:0/3258454459 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c08003c00 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.340+0000 7f4c12000700 1 -- 172.21.15.67:0/3258454459 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c08024800 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.340+0000 7f4c1c600700 1 -- 172.21.15.67:0/3258454459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1419cc30 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.341+0000 7f4c1c600700 1 -- 172.21.15.67:0/3258454459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1419d0d0 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.341+0000 7f4c12000700 1 -- 172.21.15.67:0/3258454459 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4c08024c60 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.341+0000 7f4c12000700 1 -- 172.21.15.67:0/3258454459 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4c080193c0 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.342+0000 7f4c1c600700 1 -- 172.21.15.67:0/3258454459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c140623e0 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.346+0000 7f4c12000700 1 -- 172.21.15.67:0/3258454459 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f4c0802de40 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.422+0000 7f4c1c600700 1 -- 172.21.15.67:0/3258454459 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4c1410a980 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.422+0000 7f4c12000700 1 -- 172.21.15.67:0/3258454459 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f4c0801ac80 con 0x7f4c14106580 2024-03-30T02:28:41.831 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.423+0000 7f4c1c600700 1 -- 172.21.15.67:0/3258454459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 msgr2=0x7f4c1419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:41.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.423+0000 7f4c1c600700 1 --2- 172.21.15.67:0/3258454459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c1419c4f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4c08011710 tx=0x7f4c08011ad0 comp rx=0 tx=0).stop 2024-03-30T02:28:41.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.423+0000 7f4c1c600700 1 -- 172.21.15.67:0/3258454459 shutdown_connections 2024-03-30T02:28:41.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.423+0000 7f4c1c600700 1 --2- 172.21.15.67:0/3258454459 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c14106580 0x7f4c1419c4f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:41.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.423+0000 7f4c1c600700 1 -- 172.21.15.67:0/3258454459 >> 172.21.15.67:0/3258454459 conn(0x7f4c14101b40 msgr2=0x7f4c14102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:41.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.423+0000 7f4c1c600700 1 -- 172.21.15.67:0/3258454459 shutdown_connections 2024-03-30T02:28:41.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:41.424+0000 7f4c1c600700 1 -- 172.21.15.67:0/3258454459 wait complete. 2024-03-30T02:28:41.832 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (2/15)... 2024-03-30T02:28:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:41 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3258454459' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-30T02:28:41.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:41.858+0000 7f0d9f164200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T02:28:42.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:42.117+0000 7f0d9f164200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T02:28:42.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:42.244+0000 7f0d9f164200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T02:28:43.582 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:43 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:43.250+0000 7f0d9f164200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T02:28:43.583 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:43 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:43.380+0000 7f0d9f164200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T02:28:43.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:43 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:43.581+0000 7f0d9f164200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T02:28:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:44 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2097174964' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-30T02:28:44.664 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:44 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:44.443+0000 7f0d9f164200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T02:28:44.678 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:44.678 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-03-30T02:28:44.678 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:28:44.678 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-03-30T02:28:44.678 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-30T02:28:44.678 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-30T02:28:44.678 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-30T02:28:44.678 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:44.678 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:44.679 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-30T02:28:44.680 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-30T02:28:44.693 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-30T02:28:44.693 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-30T02:28:44.694 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-30T02:28:44.694 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-30T02:28:44.694 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:44.694 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-30T02:28:44.694 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-30T02:28:44.694 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-30T02:28:44.694 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-30T02:28:44.694 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-30T02:28:44.695 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-30T02:28:44.695 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-30T02:28:44.695 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-30T02:28:44.695 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-30T02:28:44.695 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:44.695 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-30T02:28:44.695 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:44.695 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-30T02:28:44.696 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-30T02:28:44.696 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:44.696 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-30T02:28:44.696 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-30T02:28:44.696 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-30T02:28:44.696 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-30T02:28:44.696 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-30T02:28:44.696 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-30T02:28:44.696 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-03-30T02:28:44.697 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:44.697 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:44.697 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:44.697 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-30T02:28:44.697 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:44.697 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-03-30T02:28:23.927930+0000", 2024-03-30T02:28:44.697 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:44.697 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:44.697 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-30T02:28:44.698 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-03-30T02:28:44.698 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.220+0000 7ff2f6c00700 1 Processor -- start 2024-03-30T02:28:44.698 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.220+0000 7ff2f6c00700 1 -- start start 2024-03-30T02:28:44.698 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.220+0000 7ff2f6c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f01069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:44.698 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.220+0000 7ff2f6c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0106f70 con 0x7ff2f0106580 2024-03-30T02:28:44.698 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.221+0000 7ff2efe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f01069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:44.698 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.221+0000 7ff2efe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f01069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34280/0 (socket says 172.21.15.67:34280) 2024-03-30T02:28:44.698 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.221+0000 7ff2efe00700 1 -- 172.21.15.67:0/3622462778 learned_addr learned my addr 172.21.15.67:0/3622462778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.221+0000 7ff2efe00700 1 -- 172.21.15.67:0/3622462778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f0107780 con 0x7ff2f0106580 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.221+0000 7ff2efe00700 1 --2- 172.21.15.67:0/3622462778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f01069a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098a0 tx=0x7ff2e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2506096f4a93305c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.222+0000 7ff2eea00700 1 -- 172.21.15.67:0/3622462778 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0004160 con 0x7ff2f0106580 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.222+0000 7ff2eea00700 1 -- 172.21.15.67:0/3622462778 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2e00042c0 con 0x7ff2f0106580 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.222+0000 7ff2eea00700 1 -- 172.21.15.67:0/3622462778 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0011a00 con 0x7ff2f0106580 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.222+0000 7ff2f6c00700 1 -- 172.21.15.67:0/3622462778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 msgr2=0x7ff2f01069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.222+0000 7ff2f6c00700 1 --2- 172.21.15.67:0/3622462778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f01069a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00098a0 tx=0x7ff2e0009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.222+0000 7ff2f6c00700 1 -- 172.21.15.67:0/3622462778 shutdown_connections 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.222+0000 7ff2f6c00700 1 --2- 172.21.15.67:0/3622462778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f01069a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:44.699 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.222+0000 7ff2f6c00700 1 -- 172.21.15.67:0/3622462778 >> 172.21.15.67:0/3622462778 conn(0x7ff2f0101b40 msgr2=0x7ff2f0103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.223+0000 7ff2f6c00700 1 -- 172.21.15.67:0/3622462778 shutdown_connections 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.223+0000 7ff2f6c00700 1 -- 172.21.15.67:0/3622462778 wait complete. 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.224+0000 7ff2f6c00700 1 Processor -- start 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.224+0000 7ff2f6c00700 1 -- start start 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.224+0000 7ff2f6c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.224+0000 7ff2f6c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f019ca30 con 0x7ff2f0106580 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.224+0000 7ff2efe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.224+0000 7ff2efe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34296/0 (socket says 172.21.15.67:34296) 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.224+0000 7ff2efe00700 1 -- 172.21.15.67:0/2097174964 learned_addr learned my addr 172.21.15.67:0/2097174964 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.225+0000 7ff2efe00700 1 -- 172.21.15.67:0/2097174964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e0009550 con 0x7ff2f0106580 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.225+0000 7ff2efe00700 1 --2- 172.21.15.67:0/2097174964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f019c4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0011710 tx=0x7ff2e0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.225+0000 7ff2ecc00700 1 -- 172.21.15.67:0/2097174964 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0003aa0 con 0x7ff2f0106580 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.225+0000 7ff2ecc00700 1 -- 172.21.15.67:0/2097174964 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff2e0003c00 con 0x7ff2f0106580 2024-03-30T02:28:44.700 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.225+0000 7ff2f6c00700 1 -- 172.21.15.67:0/2097174964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f019cc30 con 0x7ff2f0106580 2024-03-30T02:28:44.701 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.225+0000 7ff2ecc00700 1 -- 172.21.15.67:0/2097174964 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2e0024800 con 0x7ff2f0106580 2024-03-30T02:28:44.701 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.225+0000 7ff2f6c00700 1 -- 172.21.15.67:0/2097174964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f019d0d0 con 0x7ff2f0106580 2024-03-30T02:28:44.701 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.226+0000 7ff2ecc00700 1 -- 172.21.15.67:0/2097174964 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff2e0024c60 con 0x7ff2f0106580 2024-03-30T02:28:44.701 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.226+0000 7ff2ecc00700 1 -- 172.21.15.67:0/2097174964 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff2e00193c0 con 0x7ff2f0106580 2024-03-30T02:28:44.701 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.227+0000 7ff2f6c00700 1 -- 172.21.15.67:0/2097174964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f00623e0 con 0x7ff2f0106580 2024-03-30T02:28:44.701 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.229+0000 7ff2ecc00700 1 -- 172.21.15.67:0/2097174964 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7ff2e002de40 con 0x7ff2f0106580 2024-03-30T02:28:44.701 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.306+0000 7ff2f6c00700 1 -- 172.21.15.67:0/2097174964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff2f010a970 con 0x7ff2f0106580 2024-03-30T02:28:44.701 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.307+0000 7ff2ecc00700 1 -- 172.21.15.67:0/2097174964 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7ff2e001abd0 con 0x7ff2f0106580 2024-03-30T02:28:44.701 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.308+0000 7ff2f6c00700 1 -- 172.21.15.67:0/2097174964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 msgr2=0x7ff2f019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:44.702 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.308+0000 7ff2f6c00700 1 --2- 172.21.15.67:0/2097174964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f019c4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0011710 tx=0x7ff2e0011ad0 comp rx=0 tx=0).stop 2024-03-30T02:28:44.702 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.308+0000 7ff2f6c00700 1 -- 172.21.15.67:0/2097174964 shutdown_connections 2024-03-30T02:28:44.702 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.308+0000 7ff2f6c00700 1 --2- 172.21.15.67:0/2097174964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2f0106580 0x7ff2f019c4f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:44.702 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.308+0000 7ff2f6c00700 1 -- 172.21.15.67:0/2097174964 >> 172.21.15.67:0/2097174964 conn(0x7ff2f0101b40 msgr2=0x7ff2f0102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:44.702 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.308+0000 7ff2f6c00700 1 -- 172.21.15.67:0/2097174964 shutdown_connections 2024-03-30T02:28:44.702 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:44.308+0000 7ff2f6c00700 1 -- 172.21.15.67:0/2097174964 wait complete. 2024-03-30T02:28:44.702 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (3/15)... 2024-03-30T02:28:44.959 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:44 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:44.694+0000 7f0d9f164200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T02:28:45.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:44 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:44.958+0000 7f0d9f164200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T02:28:46.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:46.152+0000 7f0d9f164200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T02:28:46.875 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:46.539+0000 7f0d9f164200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:28:46.875 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:46.660+0000 7f0d9f164200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T02:28:47.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:46.929+0000 7f0d9f164200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T02:28:47.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: Activating manager daemon x 2024-03-30T02:28:47.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: mgrmap e2: x(active, starting, since 0.00541953s) 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: Manager daemon x is now available 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' 2024-03-30T02:28:47.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:46 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-30T02:28:47.595 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-30T02:28:47.596 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-30T02:28:47.597 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-03-30T02:28:47.598 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-03-30T02:28:23.927930+0000", 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.134+0000 7f577e800700 1 Processor -- start 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.134+0000 7f577e800700 1 -- start start 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.134+0000 7f577e800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577807a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.134+0000 7f577e800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577807a8a0 con 0x7f577807be70 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.135+0000 7f5777400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577807a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:47.599 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.135+0000 7f5777400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577807a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59538/0 (socket says 172.21.15.67:59538) 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.135+0000 7f5777400700 1 -- 172.21.15.67:0/1648732970 learned_addr learned my addr 172.21.15.67:0/1648732970 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.135+0000 7f5777400700 1 -- 172.21.15.67:0/1648732970 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577807a9e0 con 0x7f577807be70 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.136+0000 7f5777400700 1 --2- 172.21.15.67:0/1648732970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577807a2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f57680098a0 tx=0x7f5768009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be278ffd3645508 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.136+0000 7f5776000700 1 -- 172.21.15.67:0/1648732970 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5768004160 con 0x7f577807be70 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.136+0000 7f5776000700 1 -- 172.21.15.67:0/1648732970 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57680042c0 con 0x7f577807be70 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.136+0000 7f5776000700 1 -- 172.21.15.67:0/1648732970 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5768011a00 con 0x7f577807be70 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.136+0000 7f577e800700 1 -- 172.21.15.67:0/1648732970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 msgr2=0x7f577807a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.136+0000 7f577e800700 1 --2- 172.21.15.67:0/1648732970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577807a2d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f57680098a0 tx=0x7f5768009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.137+0000 7f577e800700 1 -- 172.21.15.67:0/1648732970 shutdown_connections 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.137+0000 7f577e800700 1 --2- 172.21.15.67:0/1648732970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577807a2d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.137+0000 7f577e800700 1 -- 172.21.15.67:0/1648732970 >> 172.21.15.67:0/1648732970 conn(0x7f5778101b40 msgr2=0x7f5778103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.137+0000 7f577e800700 1 -- 172.21.15.67:0/1648732970 shutdown_connections 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.137+0000 7f577e800700 1 -- 172.21.15.67:0/1648732970 wait complete. 2024-03-30T02:28:47.600 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.138+0000 7f577e800700 1 Processor -- start 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.138+0000 7f577e800700 1 -- start start 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.138+0000 7f577e800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.138+0000 7f577e800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577819ca00 con 0x7f577807be70 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.138+0000 7f5777400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.138+0000 7f5777400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59542/0 (socket says 172.21.15.67:59542) 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.138+0000 7f5777400700 1 -- 172.21.15.67:0/1500587514 learned_addr learned my addr 172.21.15.67:0/1500587514 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.139+0000 7f5777400700 1 -- 172.21.15.67:0/1500587514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768009550 con 0x7f577807be70 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.139+0000 7f5777400700 1 --2- 172.21.15.67:0/1500587514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577819c4c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5768011a20 tx=0x7f5768011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.139+0000 7f576fe00700 1 -- 172.21.15.67:0/1500587514 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5768003af0 con 0x7f577807be70 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.139+0000 7f576fe00700 1 -- 172.21.15.67:0/1500587514 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5768003c50 con 0x7f577807be70 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.139+0000 7f576fe00700 1 -- 172.21.15.67:0/1500587514 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5768019760 con 0x7f577807be70 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.139+0000 7f577e800700 1 -- 172.21.15.67:0/1500587514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577819cc00 con 0x7f577807be70 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.139+0000 7f577e800700 1 -- 172.21.15.67:0/1500587514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577819d0a0 con 0x7f577807be70 2024-03-30T02:28:47.601 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.140+0000 7f576fe00700 1 -- 172.21.15.67:0/1500587514 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 2) v1 ==== 45289+0+0 (secure 0 0 0) 0x7f576801ab20 con 0x7f577807be70 2024-03-30T02:28:47.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.140+0000 7f577e800700 1 -- 172.21.15.67:0/1500587514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577810c970 con 0x7f577807be70 2024-03-30T02:28:47.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.141+0000 7f576fe00700 1 -- 172.21.15.67:0/1500587514 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f576804c490 con 0x7f577807be70 2024-03-30T02:28:47.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.143+0000 7f576fe00700 1 -- 172.21.15.67:0/1500587514 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f576802d9a0 con 0x7f577807be70 2024-03-30T02:28:47.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.218+0000 7f577e800700 1 -- 172.21.15.67:0/1500587514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f577810cbb0 con 0x7f577807be70 2024-03-30T02:28:47.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.219+0000 7f576fe00700 1 -- 172.21.15.67:0/1500587514 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f5768024e70 con 0x7f577807be70 2024-03-30T02:28:47.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.219+0000 7f577e800700 1 -- 172.21.15.67:0/1500587514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 msgr2=0x7f577819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:47.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.219+0000 7f577e800700 1 --2- 172.21.15.67:0/1500587514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577819c4c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5768011a20 tx=0x7f5768011b00 comp rx=0 tx=0).stop 2024-03-30T02:28:47.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.219+0000 7f577e800700 1 -- 172.21.15.67:0/1500587514 shutdown_connections 2024-03-30T02:28:47.602 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.219+0000 7f577e800700 1 --2- 172.21.15.67:0/1500587514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f577807be70 0x7f577819c4c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:47.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.219+0000 7f577e800700 1 -- 172.21.15.67:0/1500587514 >> 172.21.15.67:0/1500587514 conn(0x7f5778101b40 msgr2=0x7f57781026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:47.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.220+0000 7f577e800700 1 -- 172.21.15.67:0/1500587514 shutdown_connections 2024-03-30T02:28:47.603 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:47.220+0000 7f577e800700 1 -- 172.21.15.67:0/1500587514 wait complete. 2024-03-30T02:28:47.603 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (4/15)... 2024-03-30T02:28:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:47 smithi067 ceph-mon[37808]: from='mgr.14100 172.21.15.67:0/1957377764' entity='mgr.x' 2024-03-30T02:28:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:47 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1500587514' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-30T02:28:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:47 smithi067 ceph-mon[37808]: mgrmap e3: x(active, since 1.00852s) 2024-03-30T02:28:50.329 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:49 smithi067 ceph-mon[37808]: mgrmap e4: x(active, since 2s) 2024-03-30T02:28:50.745 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:50.745 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:50.746 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-30T02:28:50.747 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-30T02:28:50.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-03-30T02:28:23.927930+0000", 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-30T02:28:50.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.992+0000 7fece6200700 1 Processor -- start 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.992+0000 7fece6200700 1 -- start start 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.992+0000 7fece6200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece0106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.992+0000 7fece6200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece0106d20 con 0x7fece0106330 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.993+0000 7fece4e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece0106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.993+0000 7fece4e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece0106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59558/0 (socket says 172.21.15.67:59558) 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.993+0000 7fece4e00700 1 -- 172.21.15.67:0/1446781483 learned_addr learned my addr 172.21.15.67:0/1446781483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.993+0000 7fece4e00700 1 -- 172.21.15.67:0/1446781483 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0107530 con 0x7fece0106330 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.993+0000 7fece4e00700 1 --2- 172.21.15.67:0/1446781483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece0106750 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fecd40098a0 tx=0x7fecd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99b4972bd7c31e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.995+0000 7fecdf400700 1 -- 172.21.15.67:0/1446781483 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecd4004160 con 0x7fece0106330 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.995+0000 7fecdf400700 1 -- 172.21.15.67:0/1446781483 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fecd40042c0 con 0x7fece0106330 2024-03-30T02:28:50.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.995+0000 7fecdf400700 1 -- 172.21.15.67:0/1446781483 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecd4011a00 con 0x7fece0106330 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.996+0000 7fece6200700 1 -- 172.21.15.67:0/1446781483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 msgr2=0x7fece0106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.996+0000 7fece6200700 1 --2- 172.21.15.67:0/1446781483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece0106750 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fecd40098a0 tx=0x7fecd4009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.996+0000 7fece6200700 1 -- 172.21.15.67:0/1446781483 shutdown_connections 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.996+0000 7fece6200700 1 --2- 172.21.15.67:0/1446781483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece0106750 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.996+0000 7fece6200700 1 -- 172.21.15.67:0/1446781483 >> 172.21.15.67:0/1446781483 conn(0x7fece01018b0 msgr2=0x7fece0103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.996+0000 7fece6200700 1 -- 172.21.15.67:0/1446781483 shutdown_connections 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.996+0000 7fece6200700 1 -- 172.21.15.67:0/1446781483 wait complete. 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.997+0000 7fece6200700 1 Processor -- start 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.997+0000 7fece6200700 1 -- start start 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.998+0000 7fece6200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.998+0000 7fece6200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece019a5c0 con 0x7fece0106330 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.998+0000 7fece4e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.998+0000 7fece4e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59560/0 (socket says 172.21.15.67:59560) 2024-03-30T02:28:50.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.998+0000 7fece4e00700 1 -- 172.21.15.67:0/2210938528 learned_addr learned my addr 172.21.15.67:0/2210938528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.998+0000 7fece4e00700 1 -- 172.21.15.67:0/2210938528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd4009550 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.998+0000 7fece4e00700 1 --2- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece019a080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009130 tx=0x7fecd4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.999+0000 7fecdd600700 1 -- 172.21.15.67:0/2210938528 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecd4003ab0 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.999+0000 7fecdd600700 1 -- 172.21.15.67:0/2210938528 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fecd4003c10 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.999+0000 7fecdd600700 1 -- 172.21.15.67:0/2210938528 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecd40247d0 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.999+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece019a7c0 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:49.999+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece019ac60 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.000+0000 7fecdd600700 1 -- 172.21.15.67:0/2210938528 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7fecd401a430 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.000+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece010a760 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.000+0000 7fecdd600700 1 --2- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fecc80386b0 0x7fecc803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.000+0000 7fecdd600700 1 -- 172.21.15.67:0/2210938528 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fecd404c470 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.001+0000 7fecdfe00700 1 --2- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fecc80386b0 0x7fecc803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.001+0000 7fecdfe00700 1 --2- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fecc80386b0 0x7fecc803ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feccc006fd0 tx=0x7feccc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.006+0000 7fecdd600700 1 -- 172.21.15.67:0/2210938528 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fecd4018350 con 0x7fece0106330 2024-03-30T02:28:50.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.304+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fece019d4f0 con 0x7fece0106330 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.305+0000 7fecdd600700 1 -- 172.21.15.67:0/2210938528 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fecd4018350 con 0x7fece0106330 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fecc80386b0 msgr2=0x7fecc803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 --2- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fecc80386b0 0x7fecc803ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feccc006fd0 tx=0x7feccc006e40 comp rx=0 tx=0).stop 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 msgr2=0x7fece019a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 --2- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece019a080 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009130 tx=0x7fecd4011ad0 comp rx=0 tx=0).stop 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 shutdown_connections 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 --2- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fecc80386b0 0x7fecc803ab70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 --2- 172.21.15.67:0/2210938528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fece0106330 0x7fece019a080 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 >> 172.21.15.67:0/2210938528 conn(0x7fece01018b0 msgr2=0x7fece0102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 shutdown_connections 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:50.307+0000 7fece6200700 1 -- 172.21.15.67:0/2210938528 wait complete. 2024-03-30T02:28:50.753 INFO:teuthology.orchestra.run.smithi067.stdout:mgr is available 2024-03-30T02:28:51.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:50 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2210938528' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-30T02:28:51.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.159+0000 7f9413800700 1 Processor -- start 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.159+0000 7f9413800700 1 -- start start 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.160+0000 7f9413800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.160+0000 7f9413800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c106ef0 con 0x7f940c106500 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.160+0000 7f9410e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.160+0000 7f9410e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59572/0 (socket says 172.21.15.67:59572) 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.160+0000 7f9410e00700 1 -- 172.21.15.67:0/3466024547 learned_addr learned my addr 172.21.15.67:0/3466024547 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.160+0000 7f9410e00700 1 -- 172.21.15.67:0/3466024547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c107700 con 0x7f940c106500 2024-03-30T02:28:51.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.161+0000 7f9410e00700 1 --2- 172.21.15.67:0/3466024547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c106920 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94000098a0 tx=0x7f9400009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457543e515a51d0a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.161+0000 7f940b400700 1 -- 172.21.15.67:0/3466024547 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9400004160 con 0x7f940c106500 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.161+0000 7f940b400700 1 -- 172.21.15.67:0/3466024547 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f94000042c0 con 0x7f940c106500 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.161+0000 7f940b400700 1 -- 172.21.15.67:0/3466024547 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9400011a00 con 0x7f940c106500 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.162+0000 7f9413800700 1 -- 172.21.15.67:0/3466024547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 msgr2=0x7f940c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.162+0000 7f9413800700 1 --2- 172.21.15.67:0/3466024547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c106920 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94000098a0 tx=0x7f9400009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.162+0000 7f9413800700 1 -- 172.21.15.67:0/3466024547 shutdown_connections 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.162+0000 7f9413800700 1 --2- 172.21.15.67:0/3466024547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c106920 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.162+0000 7f9413800700 1 -- 172.21.15.67:0/3466024547 >> 172.21.15.67:0/3466024547 conn(0x7f940c101ae0 msgr2=0x7f940c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.162+0000 7f9413800700 1 -- 172.21.15.67:0/3466024547 shutdown_connections 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.162+0000 7f9413800700 1 -- 172.21.15.67:0/3466024547 wait complete. 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.163+0000 7f9413800700 1 Processor -- start 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.163+0000 7f9413800700 1 -- start start 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.163+0000 7f9413800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:51.750 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.163+0000 7f9413800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c19c9b0 con 0x7f940c106500 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.164+0000 7f9410e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.164+0000 7f9410e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59588/0 (socket says 172.21.15.67:59588) 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.164+0000 7f9410e00700 1 -- 172.21.15.67:0/3182879859 learned_addr learned my addr 172.21.15.67:0/3182879859 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.164+0000 7f9410e00700 1 -- 172.21.15.67:0/3182879859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9400009550 con 0x7f940c106500 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.164+0000 7f9410e00700 1 --2- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c19c470 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9400009130 tx=0x7f9400011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.164+0000 7f9409600700 1 -- 172.21.15.67:0/3182879859 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9400003ab0 con 0x7f940c106500 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.164+0000 7f9409600700 1 -- 172.21.15.67:0/3182879859 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9400003c10 con 0x7f940c106500 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.164+0000 7f9409600700 1 -- 172.21.15.67:0/3182879859 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94000247d0 con 0x7f940c106500 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.164+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c19cbb0 con 0x7f940c106500 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.165+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c19d050 con 0x7f940c106500 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.166+0000 7f9409600700 1 -- 172.21.15.67:0/3182879859 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7f940001abb0 con 0x7f940c106500 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.166+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c10a930 con 0x7f940c106500 2024-03-30T02:28:51.751 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.166+0000 7f9409600700 1 --2- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f93f4038700 0x7f93f403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.166+0000 7f9409600700 1 -- 172.21.15.67:0/3182879859 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f940004c3e0 con 0x7f940c106500 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.166+0000 7f940be00700 1 --2- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f93f4038700 0x7f93f403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.167+0000 7f940be00700 1 --2- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f93f4038700 0x7f93f403abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f93f8006fd0 tx=0x7f93f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.171+0000 7f9409600700 1 -- 172.21.15.67:0/3182879859 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9400018710 con 0x7f940c106500 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.381+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f940c19f8e0 con 0x7f940c106500 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.382+0000 7f9409600700 1 -- 172.21.15.67:0/3182879859 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f9400018350 con 0x7f940c106500 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.384+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f93f4038700 msgr2=0x7f93f403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.384+0000 7f9413800700 1 --2- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f93f4038700 0x7f93f403abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f93f8006fd0 tx=0x7f93f8006e40 comp rx=0 tx=0).stop 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.384+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 msgr2=0x7f940c19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.384+0000 7f9413800700 1 --2- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c19c470 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9400009130 tx=0x7f9400011ad0 comp rx=0 tx=0).stop 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.384+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 shutdown_connections 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.384+0000 7f9413800700 1 --2- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f93f4038700 0x7f93f403abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.384+0000 7f9413800700 1 --2- 172.21.15.67:0/3182879859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f940c106500 0x7f940c19c470 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.384+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 >> 172.21.15.67:0/3182879859 conn(0x7f940c101ae0 msgr2=0x7f940c102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:51.752 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.385+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 shutdown_connections 2024-03-30T02:28:51.753 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:51.385+0000 7f9413800700 1 -- 172.21.15.67:0/3182879859 wait complete. 2024-03-30T02:28:51.753 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling cephadm module... 2024-03-30T02:28:52.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:51 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3182879859' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.147+0000 7fe258a00700 1 Processor -- start 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.147+0000 7fe258a00700 1 -- start start 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.147+0000 7fe258a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe254106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.147+0000 7fe258a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe254106d20 con 0x7fe254106330 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.147+0000 7fe253400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe254106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.148+0000 7fe253400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe254106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59596/0 (socket says 172.21.15.67:59596) 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.148+0000 7fe253400700 1 -- 172.21.15.67:0/525205023 learned_addr learned my addr 172.21.15.67:0/525205023 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.148+0000 7fe253400700 1 -- 172.21.15.67:0/525205023 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe254107530 con 0x7fe254106330 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.148+0000 7fe253400700 1 --2- 172.21.15.67:0/525205023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe254106750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2440098a0 tx=0x7fe244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08a384882c4c4db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.148+0000 7fe252000700 1 -- 172.21.15.67:0/525205023 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe244004160 con 0x7fe254106330 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.148+0000 7fe252000700 1 -- 172.21.15.67:0/525205023 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2440042c0 con 0x7fe254106330 2024-03-30T02:28:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.148+0000 7fe252000700 1 -- 172.21.15.67:0/525205023 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe244011a00 con 0x7fe254106330 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.149+0000 7fe258a00700 1 -- 172.21.15.67:0/525205023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 msgr2=0x7fe254106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.149+0000 7fe258a00700 1 --2- 172.21.15.67:0/525205023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe254106750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe2440098a0 tx=0x7fe244009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.149+0000 7fe258a00700 1 -- 172.21.15.67:0/525205023 shutdown_connections 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.149+0000 7fe258a00700 1 --2- 172.21.15.67:0/525205023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe254106750 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.149+0000 7fe258a00700 1 -- 172.21.15.67:0/525205023 >> 172.21.15.67:0/525205023 conn(0x7fe2541018b0 msgr2=0x7fe254103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.149+0000 7fe258a00700 1 -- 172.21.15.67:0/525205023 shutdown_connections 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.149+0000 7fe258a00700 1 -- 172.21.15.67:0/525205023 wait complete. 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.150+0000 7fe258a00700 1 Processor -- start 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.150+0000 7fe258a00700 1 -- start start 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.151+0000 7fe258a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe25419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.151+0000 7fe258a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25419c7d0 con 0x7fe254106330 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.151+0000 7fe253400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe25419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.151+0000 7fe253400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe25419c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59606/0 (socket says 172.21.15.67:59606) 2024-03-30T02:28:53.356 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.151+0000 7fe253400700 1 -- 172.21.15.67:0/1813202537 learned_addr learned my addr 172.21.15.67:0/1813202537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.151+0000 7fe253400700 1 -- 172.21.15.67:0/1813202537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe244009550 con 0x7fe254106330 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.151+0000 7fe253400700 1 --2- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe25419c290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe244009130 tx=0x7fe244011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.152+0000 7fe24be00700 1 -- 172.21.15.67:0/1813202537 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe244003ab0 con 0x7fe254106330 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.152+0000 7fe24be00700 1 -- 172.21.15.67:0/1813202537 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe244003c10 con 0x7fe254106330 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.152+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25419c9d0 con 0x7fe254106330 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.152+0000 7fe24be00700 1 -- 172.21.15.67:0/1813202537 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2440247d0 con 0x7fe254106330 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.152+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25419ce70 con 0x7fe254106330 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.153+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25410a760 con 0x7fe254106330 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.153+0000 7fe24be00700 1 -- 172.21.15.67:0/1813202537 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 45522+0+0 (secure 0 0 0) 0x7fe24401abb0 con 0x7fe254106330 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.153+0000 7fe24be00700 1 --2- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fe238038700 0x7fe23803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.154+0000 7fe24be00700 1 -- 172.21.15.67:0/1813202537 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe24404c410 con 0x7fe254106330 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.154+0000 7fe252a00700 1 --2- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fe238038700 0x7fe23803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.154+0000 7fe252a00700 1 --2- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fe238038700 0x7fe23803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe240006fd0 tx=0x7fe240006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:53.357 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.158+0000 7fe24be00700 1 -- 172.21.15.67:0/1813202537 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe2440188f0 con 0x7fe254106330 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:52.429+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fe25419f700 con 0x7fe254106330 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.003+0000 7fe24be00700 1 -- 172.21.15.67:0/1813202537 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fe244021d60 con 0x7fe254106330 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.004+0000 7fe24be00700 1 -- 172.21.15.67:0/1813202537 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fe24402d770 con 0x7fe254106330 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.007+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fe238038700 msgr2=0x7fe23803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.007+0000 7fe258a00700 1 --2- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fe238038700 0x7fe23803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe240006fd0 tx=0x7fe240006e40 comp rx=0 tx=0).stop 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.007+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 msgr2=0x7fe25419c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.007+0000 7fe258a00700 1 --2- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe25419c290 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe244009130 tx=0x7fe244011ad0 comp rx=0 tx=0).stop 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.007+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 shutdown_connections 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.007+0000 7fe258a00700 1 --2- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fe238038700 0x7fe23803abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.007+0000 7fe258a00700 1 --2- 172.21.15.67:0/1813202537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe254106330 0x7fe25419c290 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.007+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 >> 172.21.15.67:0/1813202537 conn(0x7fe2541018b0 msgr2=0x7fe254102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.008+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 shutdown_connections 2024-03-30T02:28:53.358 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.008+0000 7fe258a00700 1 -- 172.21.15.67:0/1813202537 wait complete. 2024-03-30T02:28:53.359 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: ignoring --setuser ceph since I am not root 2024-03-30T02:28:53.359 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: ignoring --setgroup ceph since I am not root 2024-03-30T02:28:53.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:52 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1813202537' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-30T02:28:53.635 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:53.634+0000 7f402985a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T02:28:53.886 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:53.760+0000 7f402985a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T02:28:54.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:54 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1813202537' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-30T02:28:54.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:54 smithi067 ceph-mon[37808]: mgrmap e5: x(active, since 6s) 2024-03-30T02:28:54.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:53.885+0000 7f402985a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T02:28:54.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:53.998+0000 7f402985a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T02:28:54.449 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-03-30T02:28:54.455 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.787+0000 7fa095800700 1 Processor -- start 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.787+0000 7fa095800700 1 -- start start 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.787+0000 7fa095800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa09007a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.787+0000 7fa095800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09007a900 con 0x7fa09007bed0 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.788+0000 7fa08ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa09007a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.788+0000 7fa08ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa09007a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59628/0 (socket says 172.21.15.67:59628) 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.788+0000 7fa08ea00700 1 -- 172.21.15.67:0/267694598 learned_addr learned my addr 172.21.15.67:0/267694598 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.788+0000 7fa08ea00700 1 -- 172.21.15.67:0/267694598 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09007aa40 con 0x7fa09007bed0 2024-03-30T02:28:54.456 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.788+0000 7fa08ea00700 1 --2- 172.21.15.67:0/267694598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa09007a330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0800098a0 tx=0x7fa080009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd6beba37b017ea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.789+0000 7fa08d600700 1 -- 172.21.15.67:0/267694598 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa080004160 con 0x7fa09007bed0 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.789+0000 7fa08d600700 1 -- 172.21.15.67:0/267694598 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa0800042c0 con 0x7fa09007bed0 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.789+0000 7fa08d600700 1 -- 172.21.15.67:0/267694598 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa080011a00 con 0x7fa09007bed0 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.789+0000 7fa095800700 1 -- 172.21.15.67:0/267694598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 msgr2=0x7fa09007a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.789+0000 7fa095800700 1 --2- 172.21.15.67:0/267694598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa09007a330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0800098a0 tx=0x7fa080009bb0 comp rx=0 tx=0).stop 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.789+0000 7fa095800700 1 -- 172.21.15.67:0/267694598 shutdown_connections 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.789+0000 7fa095800700 1 --2- 172.21.15.67:0/267694598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa09007a330 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.789+0000 7fa095800700 1 -- 172.21.15.67:0/267694598 >> 172.21.15.67:0/267694598 conn(0x7fa090101af0 msgr2=0x7fa090103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.790+0000 7fa095800700 1 -- 172.21.15.67:0/267694598 shutdown_connections 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.790+0000 7fa095800700 1 -- 172.21.15.67:0/267694598 wait complete. 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.791+0000 7fa095800700 1 Processor -- start 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.791+0000 7fa095800700 1 -- start start 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.792+0000 7fa095800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa0901a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.792+0000 7fa095800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0901a0d80 con 0x7fa09007bed0 2024-03-30T02:28:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.792+0000 7fa08ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa0901a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.792+0000 7fa08ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa0901a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59632/0 (socket says 172.21.15.67:59632) 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.792+0000 7fa08ea00700 1 -- 172.21.15.67:0/3088141548 learned_addr learned my addr 172.21.15.67:0/3088141548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.792+0000 7fa08ea00700 1 -- 172.21.15.67:0/3088141548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa080009550 con 0x7fa09007bed0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.793+0000 7fa08ea00700 1 --2- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa0901a0840 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa080009130 tx=0x7fa080011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.793+0000 7fa087400700 1 -- 172.21.15.67:0/3088141548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa080003ab0 con 0x7fa09007bed0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.793+0000 7fa087400700 1 -- 172.21.15.67:0/3088141548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa080003c10 con 0x7fa09007bed0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.793+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0901a0f80 con 0x7fa09007bed0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.793+0000 7fa087400700 1 -- 172.21.15.67:0/3088141548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0800247d0 con 0x7fa09007bed0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.793+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0901a1420 con 0x7fa09007bed0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.794+0000 7fa087400700 1 -- 172.21.15.67:0/3088141548 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fa08001a430 con 0x7fa09007bed0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.794+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa090199fb0 con 0x7fa09007bed0 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.795+0000 7fa087400700 1 --2- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fa074038750 0x7fa07403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:28:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.795+0000 7fa087400700 1 -- 172.21.15.67:0/3088141548 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa08004c490 con 0x7fa09007bed0 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.795+0000 7fa08e000700 1 -- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fa074038750 msgr2=0x7fa07403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4186481961 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.795+0000 7fa08e000700 1 --2- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fa074038750 0x7fa07403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.804+0000 7fa087400700 1 -- 172.21.15.67:0/3088141548 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa080018b40 con 0x7fa09007bed0 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.996+0000 7fa08e000700 1 -- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fa074038750 msgr2=0x7fa07403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4186481961 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:53.996+0000 7fa08e000700 1 --2- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fa074038750 0x7fa07403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.074+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fa09002cf00 con 0x7fa09007bed0 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.075+0000 7fa087400700 1 -- 172.21.15.67:0/3088141548 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fa080018b40 con 0x7fa09007bed0 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.077+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fa074038750 msgr2=0x7fa07403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.077+0000 7fa095800700 1 --2- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fa074038750 0x7fa07403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.077+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 msgr2=0x7fa0901a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.077+0000 7fa095800700 1 --2- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa0901a0840 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa080009130 tx=0x7fa080011ad0 comp rx=0 tx=0).stop 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.077+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 shutdown_connections 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.077+0000 7fa095800700 1 --2- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7fa074038750 0x7fa07403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.077+0000 7fa095800700 1 --2- 172.21.15.67:0/3088141548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa09007bed0 0x7fa0901a0840 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:28:54.460 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.077+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 >> 172.21.15.67:0/3088141548 conn(0x7fa090101af0 msgr2=0x7fa090103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:28:54.460 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.078+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 shutdown_connections 2024-03-30T02:28:54.460 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.078+0000 7fa095800700 1 -- 172.21.15.67:0/3088141548 wait complete. 2024-03-30T02:28:54.460 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-03-30T02:28:54.460 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 5... 2024-03-30T02:28:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:28:55 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3088141548' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-30T02:28:55.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:54 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:54.885+0000 7f402985a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:28:55.468 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:55.210+0000 7f402985a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T02:28:55.469 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:55.324+0000 7f402985a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T02:28:55.841 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:55.468+0000 7f402985a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T02:28:55.841 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:55.668+0000 7f402985a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T02:28:56.165 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:55.840+0000 7f402985a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T02:28:56.165 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:55.954+0000 7f402985a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T02:28:56.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:56 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:56.433+0000 7f402985a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T02:28:57.189 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:56 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:56.932+0000 7f402985a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T02:28:57.664 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:57.188+0000 7f402985a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T02:28:57.664 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:57.315+0000 7f402985a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T02:28:58.630 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:58.299+0000 7f402985a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T02:28:58.630 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:58.428+0000 7f402985a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T02:28:58.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:58.629+0000 7f402985a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T02:28:59.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:59 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:59.460+0000 7f402985a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T02:28:59.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:59 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:59.705+0000 7f402985a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T02:29:00.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:28:59 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:28:59.967+0000 7f402985a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T02:29:01.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:01.143+0000 7f402985a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T02:29:01.907 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:01.527+0000 7f402985a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:29:01.908 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:01.646+0000 7f402985a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T02:29:02.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:01.905+0000 7f402985a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T02:29:02.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: Active manager daemon x restarted 2024-03-30T02:29:02.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: Activating manager daemon x 2024-03-30T02:29:02.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: osdmap e2: 0 total, 0 up, 0 in 2024-03-30T02:29:02.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: mgrmap e6: x(active, starting, since 0.00413851s) 2024-03-30T02:29:02.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:29:02.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-30T02:29:02.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-30T02:29:02.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-30T02:29:02.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-30T02:29:02.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: Manager daemon x is now available 2024-03-30T02:29:02.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:02.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:02.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:01 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:03.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-03-30T02:29:03.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-30T02:29:03.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-30T02:29:03.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-03-30T02:29:03.262 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:02 smithi067 ceph-mon[37808]: Found migration_current of "None". Setting to last migration. 2024-03-30T02:29:03.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:02 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:03.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:02 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-30T02:29:03.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:02 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-30T02:29:03.263 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:02 smithi067 ceph-mon[37808]: mgrmap e7: x(active, since 1.00681s) 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.897+0000 7f25d6600700 1 Processor -- start 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.897+0000 7f25d6600700 1 -- start start 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.898+0000 7f25d6600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d0106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.898+0000 7f25d6600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0106ef0 con 0x7f25d0106500 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.898+0000 7f25cf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d0106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.898+0000 7f25cf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d0106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59638/0 (socket says 172.21.15.67:59638) 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.898+0000 7f25cf400700 1 -- 172.21.15.67:0/443664528 learned_addr learned my addr 172.21.15.67:0/443664528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.899+0000 7f25cf400700 1 -- 172.21.15.67:0/443664528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0107700 con 0x7f25d0106500 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.899+0000 7f25cf400700 1 --2- 172.21.15.67:0/443664528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d0106920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f25c00098a0 tx=0x7f25c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124ea7c70a97a8f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.899+0000 7f25ce000700 1 -- 172.21.15.67:0/443664528 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25c0004160 con 0x7f25d0106500 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.899+0000 7f25ce000700 1 -- 172.21.15.67:0/443664528 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25c00042c0 con 0x7f25d0106500 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.899+0000 7f25ce000700 1 -- 172.21.15.67:0/443664528 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25c0011a00 con 0x7f25d0106500 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.900+0000 7f25d6600700 1 -- 172.21.15.67:0/443664528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 msgr2=0x7f25d0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.900+0000 7f25d6600700 1 --2- 172.21.15.67:0/443664528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d0106920 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f25c00098a0 tx=0x7f25c0009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.900+0000 7f25d6600700 1 -- 172.21.15.67:0/443664528 shutdown_connections 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.900+0000 7f25d6600700 1 --2- 172.21.15.67:0/443664528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d0106920 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.900+0000 7f25d6600700 1 -- 172.21.15.67:0/443664528 >> 172.21.15.67:0/443664528 conn(0x7f25d0101ae0 msgr2=0x7f25d0103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.900+0000 7f25d6600700 1 -- 172.21.15.67:0/443664528 shutdown_connections 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.900+0000 7f25d6600700 1 -- 172.21.15.67:0/443664528 wait complete. 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.901+0000 7f25d6600700 1 Processor -- start 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.901+0000 7f25d6600700 1 -- start start 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.902+0000 7f25d6600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d019c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.902+0000 7f25d6600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d019c9e0 con 0x7f25d0106500 2024-03-30T02:29:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.902+0000 7f25cf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d019c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.902+0000 7f25cf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d019c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59642/0 (socket says 172.21.15.67:59642) 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.902+0000 7f25cf400700 1 -- 172.21.15.67:0/246395185 learned_addr learned my addr 172.21.15.67:0/246395185 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.903+0000 7f25cf400700 1 -- 172.21.15.67:0/246395185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0009550 con 0x7f25d0106500 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.903+0000 7f25cf400700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d019c4a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009130 tx=0x7f25c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.903+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25c0003ab0 con 0x7f25d0106500 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.903+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25c0003c10 con 0x7f25d0106500 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.903+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25c00247d0 con 0x7f25d0106500 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.903+0000 7f25d6600700 1 -- 172.21.15.67:0/246395185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d019cbe0 con 0x7f25d0106500 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.903+0000 7f25d6600700 1 -- 172.21.15.67:0/246395185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d019d080 con 0x7f25d0106500 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.904+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f25c002d3f0 con 0x7f25d0106500 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.905+0000 7f25c7e00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 0x7f25b403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.905+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 --> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25b403b320 con 0x7f25b4038750 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.905+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f25c004c600 con 0x7f25d0106500 2024-03-30T02:29:03.273 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.905+0000 7f25cea00700 1 -- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 msgr2=0x7f25b403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4186481961 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:54.905+0000 7f25cea00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 0x7f25b403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:55.106+0000 7f25cea00700 1 -- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 msgr2=0x7f25b403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4186481961 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:55.106+0000 7f25cea00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 0x7f25b403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:55.507+0000 7f25cea00700 1 -- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 msgr2=0x7f25b403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4186481961 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:55.507+0000 7f25cea00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 0x7f25b403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:56.308+0000 7f25cea00700 1 -- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 msgr2=0x7f25b403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4186481961 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:56.308+0000 7f25cea00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 0x7f25b403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:57.910+0000 7f25cea00700 1 -- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 msgr2=0x7f25b403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4186481961 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:28:57.910+0000 7f25cea00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 0x7f25b403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:01.112+0000 7f25cea00700 1 -- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 msgr2=0x7f25b403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4186481961 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:01.112+0000 7f25cea00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 0x7f25b403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:01.910+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mgrmap(e 6) v1 ==== 45300+0+0 (secure 0 0 0) 0x7f25c002d8d0 con 0x7f25d0106500 2024-03-30T02:29:03.274 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:01.910+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 msgr2=0x7f25b403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:01.910+0000 7f25c7e00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 0x7f25b403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.912+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7f25c0017b20 con 0x7f25d0106500 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.912+0000 7f25c7e00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f25b403bfe0 0x7f25b403e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.912+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25b403b320 con 0x7f25b403bfe0 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.913+0000 7f25cea00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f25b403bfe0 0x7f25b403e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.914+0000 7f25cea00700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f25b403bfe0 0x7f25b403e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25bc003a10 tx=0x7f25bc0092b0 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.915+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 <== mgr.14122 v2:172.21.15.67:6800/2682757440 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f25b403b320 con 0x7f25b403bfe0 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.921+0000 7f25d6600700 1 -- 172.21.15.67:0/246395185 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f25d00623e0 con 0x7f25b403bfe0 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25c7e00700 1 -- 172.21.15.67:0/246395185 <== mgr.14122 v2:172.21.15.67:6800/2682757440 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f25d00623e0 con 0x7f25b403bfe0 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25d6600700 1 -- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f25b403bfe0 msgr2=0x7f25b403e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25d6600700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f25b403bfe0 0x7f25b403e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25bc003a10 tx=0x7f25bc0092b0 comp rx=0 tx=0).stop 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25d6600700 1 -- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 msgr2=0x7f25d019c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25d6600700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d019c4a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009130 tx=0x7f25c0011ad0 comp rx=0 tx=0).stop 2024-03-30T02:29:03.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25d6600700 1 -- 172.21.15.67:0/246395185 shutdown_connections 2024-03-30T02:29:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25d6600700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f25b403bfe0 0x7f25b403e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25d6600700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25d0106500 0x7f25d019c4a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25d6600700 1 --2- 172.21.15.67:0/246395185 >> [v2:172.21.15.67:6800/4186481961,v1:172.21.15.67:6801/4186481961] conn(0x7f25b4038750 0x7f25b403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.922+0000 7f25d6600700 1 -- 172.21.15.67:0/246395185 >> 172.21.15.67:0/246395185 conn(0x7f25d0101ae0 msgr2=0x7f25d0102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.923+0000 7f25d6600700 1 -- 172.21.15.67:0/246395185 shutdown_connections 2024-03-30T02:29:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:02.923+0000 7f25d6600700 1 -- 172.21.15.67:0/246395185 wait complete. 2024-03-30T02:29:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 5 is available 2024-03-30T02:29:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:Setting orchestrator backend to cephadm... 2024-03-30T02:29:04.304 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.667+0000 7f11a6400700 1 Processor -- start 2024-03-30T02:29:04.304 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.668+0000 7f11a6400700 1 -- start start 2024-03-30T02:29:04.304 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.668+0000 7f11a6400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a01093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:04.304 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.668+0000 7f11a6400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a0079e20 con 0x7f11a0107000 2024-03-30T02:29:04.304 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.669+0000 7f119f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a01093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.669+0000 7f119f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a01093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53864/0 (socket says 172.21.15.67:53864) 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.669+0000 7f119f400700 1 -- 172.21.15.67:0/3238139595 learned_addr learned my addr 172.21.15.67:0/3238139595 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.669+0000 7f119f400700 1 -- 172.21.15.67:0/3238139595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0109930 con 0x7f11a0107000 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.669+0000 7f119f400700 1 --2- 172.21.15.67:0/3238139595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a01093f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f11900098a0 tx=0x7f1190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c15c4010197ec38 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.669+0000 7f119e000700 1 -- 172.21.15.67:0/3238139595 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1190004160 con 0x7f11a0107000 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.670+0000 7f119e000700 1 -- 172.21.15.67:0/3238139595 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f11900042c0 con 0x7f11a0107000 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.670+0000 7f119e000700 1 -- 172.21.15.67:0/3238139595 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1190011a00 con 0x7f11a0107000 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.670+0000 7f11a6400700 1 -- 172.21.15.67:0/3238139595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 msgr2=0x7f11a01093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.670+0000 7f11a6400700 1 --2- 172.21.15.67:0/3238139595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a01093f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f11900098a0 tx=0x7f1190009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.670+0000 7f11a6400700 1 -- 172.21.15.67:0/3238139595 shutdown_connections 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.670+0000 7f11a6400700 1 --2- 172.21.15.67:0/3238139595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a01093f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.670+0000 7f11a6400700 1 -- 172.21.15.67:0/3238139595 >> 172.21.15.67:0/3238139595 conn(0x7f11a01009d0 msgr2=0x7f11a0102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.670+0000 7f11a6400700 1 -- 172.21.15.67:0/3238139595 shutdown_connections 2024-03-30T02:29:04.305 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.670+0000 7f11a6400700 1 -- 172.21.15.67:0/3238139595 wait complete. 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.671+0000 7f11a6400700 1 Processor -- start 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.672+0000 7f11a6400700 1 -- start start 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.672+0000 7f11a6400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.672+0000 7f11a6400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a019c9c0 con 0x7f11a0107000 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.672+0000 7f119f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.672+0000 7f119f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53872/0 (socket says 172.21.15.67:53872) 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.672+0000 7f119f400700 1 -- 172.21.15.67:0/1938714428 learned_addr learned my addr 172.21.15.67:0/1938714428 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.672+0000 7f119f400700 1 -- 172.21.15.67:0/1938714428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190009550 con 0x7f11a0107000 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.673+0000 7f119f400700 1 --2- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a019c480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1190009130 tx=0x7f1190011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.673+0000 7f1197e00700 1 -- 172.21.15.67:0/1938714428 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1190003b70 con 0x7f11a0107000 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.673+0000 7f1197e00700 1 -- 172.21.15.67:0/1938714428 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1190003cd0 con 0x7f11a0107000 2024-03-30T02:29:04.306 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.673+0000 7f1197e00700 1 -- 172.21.15.67:0/1938714428 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11900197d0 con 0x7f11a0107000 2024-03-30T02:29:04.307 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.673+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a019cbc0 con 0x7f11a0107000 2024-03-30T02:29:04.307 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.673+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a019d060 con 0x7f11a0107000 2024-03-30T02:29:04.307 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.674+0000 7f1197e00700 1 -- 172.21.15.67:0/1938714428 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 7) v1 ==== 45427+0+0 (secure 0 0 0) 0x7f119001abb0 con 0x7f11a0107000 2024-03-30T02:29:04.307 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.674+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a010a680 con 0x7f11a0107000 2024-03-30T02:29:04.307 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.675+0000 7f1197e00700 1 --2- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f11840385d0 0x7f118403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:04.307 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.675+0000 7f1197e00700 1 -- 172.21.15.67:0/1938714428 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f119004c770 con 0x7f11a0107000 2024-03-30T02:29:04.307 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.675+0000 7f119ea00700 1 --2- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f11840385d0 0x7f118403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:04.307 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.676+0000 7f119ea00700 1 --2- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f11840385d0 0x7f118403aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f118c006fd0 tx=0x7f118c006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:04.307 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.682+0000 7f1197e00700 1 -- 172.21.15.67:0/1938714428 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f119001a430 con 0x7f11a0107000 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.915+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f11a0101c00 con 0x7f11840385d0 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.925+0000 7f1197e00700 1 -- 172.21.15.67:0/1938714428 <== mgr.14122 v2:172.21.15.67:6800/2682757440 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f11a0101c00 con 0x7f11840385d0 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f11840385d0 msgr2=0x7f118403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 --2- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f11840385d0 0x7f118403aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f118c006fd0 tx=0x7f118c006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 msgr2=0x7f11a019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 --2- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a019c480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1190009130 tx=0x7f1190011ad0 comp rx=0 tx=0).stop 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 shutdown_connections 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 --2- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f11840385d0 0x7f118403aa90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 --2- 172.21.15.67:0/1938714428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a0107000 0x7f11a019c480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 >> 172.21.15.67:0/1938714428 conn(0x7f11a01009d0 msgr2=0x7f11a01014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 shutdown_connections 2024-03-30T02:29:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:03.927+0000 7f11a6400700 1 -- 172.21.15.67:0/1938714428 wait complete. 2024-03-30T02:29:04.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:03 smithi067 ceph-mon[37808]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-30T02:29:04.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:03 smithi067 ceph-mon[37808]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-30T02:29:04.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:03 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:04.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:03 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:04.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:03 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:04.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:03 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:04.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:03 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:05.303 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-30T02:29:05.303 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.724+0000 7f5032200700 1 Processor -- start 2024-03-30T02:29:05.303 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.725+0000 7f5032200700 1 -- start start 2024-03-30T02:29:05.303 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.725+0000 7f5032200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:04 smithi067 ceph-mon[37808]: [30/Mar/2024:02:29:03] ENGINE Bus STARTING 2024-03-30T02:29:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:04 smithi067 ceph-mon[37808]: [30/Mar/2024:02:29:03] ENGINE Serving on https://172.21.15.67:7150 2024-03-30T02:29:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:04 smithi067 ceph-mon[37808]: [30/Mar/2024:02:29:03] ENGINE Serving on http://172.21.15.67:8765 2024-03-30T02:29:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:04 smithi067 ceph-mon[37808]: [30/Mar/2024:02:29:03] ENGINE Bus STARTED 2024-03-30T02:29:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:04 smithi067 ceph-mon[37808]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:05.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:04 smithi067 ceph-mon[37808]: mgrmap e8: x(active, since 2s) 2024-03-30T02:29:05.309 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.725+0000 7f5032200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c07a8a0 con 0x7f502c07be70 2024-03-30T02:29:05.309 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.725+0000 7f502b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:05.309 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.725+0000 7f502b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53886/0 (socket says 172.21.15.67:53886) 2024-03-30T02:29:05.309 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.725+0000 7f502b400700 1 -- 172.21.15.67:0/4019807635 learned_addr learned my addr 172.21.15.67:0/4019807635 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.726+0000 7f502b400700 1 -- 172.21.15.67:0/4019807635 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c07a9e0 con 0x7f502c07be70 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.726+0000 7f502b400700 1 --2- 172.21.15.67:0/4019807635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c07a2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f501c0098a0 tx=0x7f501c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f838f780ea86ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.726+0000 7f502a000700 1 -- 172.21.15.67:0/4019807635 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f501c004160 con 0x7f502c07be70 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.726+0000 7f502a000700 1 -- 172.21.15.67:0/4019807635 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f501c0042c0 con 0x7f502c07be70 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.726+0000 7f502a000700 1 -- 172.21.15.67:0/4019807635 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f501c011a00 con 0x7f502c07be70 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.727+0000 7f5032200700 1 -- 172.21.15.67:0/4019807635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 msgr2=0x7f502c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.727+0000 7f5032200700 1 --2- 172.21.15.67:0/4019807635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c07a2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f501c0098a0 tx=0x7f501c009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.727+0000 7f5032200700 1 -- 172.21.15.67:0/4019807635 shutdown_connections 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.727+0000 7f5032200700 1 --2- 172.21.15.67:0/4019807635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c07a2d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.727+0000 7f5032200700 1 -- 172.21.15.67:0/4019807635 >> 172.21.15.67:0/4019807635 conn(0x7f502c101b40 msgr2=0x7f502c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.727+0000 7f5032200700 1 -- 172.21.15.67:0/4019807635 shutdown_connections 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.727+0000 7f5032200700 1 -- 172.21.15.67:0/4019807635 wait complete. 2024-03-30T02:29:05.310 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.728+0000 7f5032200700 1 Processor -- start 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.728+0000 7f5032200700 1 -- start start 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.728+0000 7f5032200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.728+0000 7f5032200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c19ca00 con 0x7f502c07be70 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.729+0000 7f502b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.729+0000 7f502b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53888/0 (socket says 172.21.15.67:53888) 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.729+0000 7f502b400700 1 -- 172.21.15.67:0/1921076557 learned_addr learned my addr 172.21.15.67:0/1921076557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.729+0000 7f502b400700 1 -- 172.21.15.67:0/1921076557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c009550 con 0x7f502c07be70 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.729+0000 7f502b400700 1 --2- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c19c4c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f501c011a20 tx=0x7f501c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.729+0000 7f5023e00700 1 -- 172.21.15.67:0/1921076557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f501c003af0 con 0x7f502c07be70 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.729+0000 7f5023e00700 1 -- 172.21.15.67:0/1921076557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f501c003c50 con 0x7f502c07be70 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.729+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c19cc00 con 0x7f502c07be70 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.729+0000 7f5023e00700 1 -- 172.21.15.67:0/1921076557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f501c019760 con 0x7f502c07be70 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.730+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c19d0a0 con 0x7f502c07be70 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.731+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c10c970 con 0x7f502c07be70 2024-03-30T02:29:05.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.731+0000 7f5023e00700 1 -- 172.21.15.67:0/1921076557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f501c01ab20 con 0x7f502c07be70 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.731+0000 7f5023e00700 1 --2- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f5010038750 0x7f501003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.731+0000 7f5023e00700 1 -- 172.21.15.67:0/1921076557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f501c04c9a0 con 0x7f502c07be70 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.732+0000 7f502aa00700 1 --2- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f5010038750 0x7f501003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.732+0000 7f502aa00700 1 --2- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f5010038750 0x7f501003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5018006fd0 tx=0x7f5018006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.737+0000 7f5023e00700 1 -- 172.21.15.67:0/1921076557 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f501c01a7b0 con 0x7f502c07be70 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.952+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f502c102e00 con 0x7f5010038750 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.952+0000 7f5023e00700 1 -- 172.21.15.67:0/1921076557 <== mgr.14122 v2:172.21.15.67:6800/2682757440 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f502c102e00 con 0x7f5010038750 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f5010038750 msgr2=0x7f501003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 --2- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f5010038750 0x7f501003ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5018006fd0 tx=0x7f5018006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 msgr2=0x7f502c19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 --2- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c19c4c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f501c011a20 tx=0x7f501c011b00 comp rx=0 tx=0).stop 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 shutdown_connections 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 --2- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f5010038750 0x7f501003ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:05.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 --2- 172.21.15.67:0/1921076557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f502c07be70 0x7f502c19c4c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:05.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 >> 172.21.15.67:0/1921076557 conn(0x7f502c101b40 msgr2=0x7f502c1026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:05.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 shutdown_connections 2024-03-30T02:29:05.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:04.955+0000 7f5032200700 1 -- 172.21.15.67:0/1921076557 wait complete. 2024-03-30T02:29:05.313 INFO:teuthology.orchestra.run.smithi067.stdout:Generating ssh key... 2024-03-30T02:29:06.320 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:05 smithi067 ceph-mon[37808]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: Generating public/private rsa key pair. 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: Your identification has been saved in /tmp/tmplnrhc57n/key. 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: Your public key has been saved in /tmp/tmplnrhc57n/key.pub. 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: The key fingerprint is: 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: SHA256:xl+KLNxWRW6zNJ3heuPaGUtg5Qykno2ULOTraxe96RQ ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: The key's randomart image is: 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: +---[RSA 3072]----+ 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: | . o . | 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: | o . * o o | 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: | o = O = | 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: | . = B X | 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: | S =.E = | 2024-03-30T02:29:06.584 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: | . = +.+.= . | 2024-03-30T02:29:06.585 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: | o * o..o+ | 2024-03-30T02:29:06.585 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: | o....o+ + | 2024-03-30T02:29:06.585 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: | ... .o + | 2024-03-30T02:29:06.585 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: +----[SHA256]-----+ 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.722+0000 7fdc51600700 1 Processor -- start 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.723+0000 7fdc51600700 1 -- start start 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.723+0000 7fdc51600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c07a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.723+0000 7fdc51600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c07a880 con 0x7fdc4c07be50 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.723+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c07a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.723+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c07a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53904/0 (socket says 172.21.15.67:53904) 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.723+0000 7fdc4aa00700 1 -- 172.21.15.67:0/2141982332 learned_addr learned my addr 172.21.15.67:0/2141982332 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.724+0000 7fdc4aa00700 1 -- 172.21.15.67:0/2141982332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c07a9c0 con 0x7fdc4c07be50 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.724+0000 7fdc4aa00700 1 --2- 172.21.15.67:0/2141982332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c07a2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0098a0 tx=0x7fdc3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f145647840c553 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.724+0000 7fdc49600700 1 -- 172.21.15.67:0/2141982332 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc3c004160 con 0x7fdc4c07be50 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.724+0000 7fdc49600700 1 -- 172.21.15.67:0/2141982332 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdc3c0042c0 con 0x7fdc4c07be50 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.724+0000 7fdc49600700 1 -- 172.21.15.67:0/2141982332 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc3c011a00 con 0x7fdc4c07be50 2024-03-30T02:29:06.727 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.725+0000 7fdc51600700 1 -- 172.21.15.67:0/2141982332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 msgr2=0x7fdc4c07a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.725+0000 7fdc51600700 1 --2- 172.21.15.67:0/2141982332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c07a2b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0098a0 tx=0x7fdc3c009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.725+0000 7fdc51600700 1 -- 172.21.15.67:0/2141982332 shutdown_connections 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.725+0000 7fdc51600700 1 --2- 172.21.15.67:0/2141982332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c07a2b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.725+0000 7fdc51600700 1 -- 172.21.15.67:0/2141982332 >> 172.21.15.67:0/2141982332 conn(0x7fdc4c101b40 msgr2=0x7fdc4c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.725+0000 7fdc51600700 1 -- 172.21.15.67:0/2141982332 shutdown_connections 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.725+0000 7fdc51600700 1 -- 172.21.15.67:0/2141982332 wait complete. 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.726+0000 7fdc51600700 1 Processor -- start 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.726+0000 7fdc51600700 1 -- start start 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.726+0000 7fdc51600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.726+0000 7fdc51600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c19ca00 con 0x7fdc4c07be50 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.727+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.727+0000 7fdc4aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53914/0 (socket says 172.21.15.67:53914) 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.727+0000 7fdc4aa00700 1 -- 172.21.15.67:0/1659680169 learned_addr learned my addr 172.21.15.67:0/1659680169 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.727+0000 7fdc4aa00700 1 -- 172.21.15.67:0/1659680169 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009550 con 0x7fdc4c07be50 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.727+0000 7fdc4aa00700 1 --2- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c19c4c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c011a20 tx=0x7fdc3c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:06.728 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.728+0000 7fdc43400700 1 -- 172.21.15.67:0/1659680169 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc3c003af0 con 0x7fdc4c07be50 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.728+0000 7fdc43400700 1 -- 172.21.15.67:0/1659680169 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdc3c003c50 con 0x7fdc4c07be50 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.728+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c19cc00 con 0x7fdc4c07be50 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.728+0000 7fdc43400700 1 -- 172.21.15.67:0/1659680169 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc3c019760 con 0x7fdc4c07be50 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.728+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c19d0a0 con 0x7fdc4c07be50 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.729+0000 7fdc43400700 1 -- 172.21.15.67:0/1659680169 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fdc3c01ab20 con 0x7fdc4c07be50 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.729+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c10c930 con 0x7fdc4c07be50 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.729+0000 7fdc43400700 1 --2- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fdc30038750 0x7fdc3003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.729+0000 7fdc43400700 1 -- 172.21.15.67:0/1659680169 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdc3c04c9a0 con 0x7fdc4c07be50 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.729+0000 7fdc4a000700 1 --2- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fdc30038750 0x7fdc3003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.730+0000 7fdc4a000700 1 --2- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fdc30038750 0x7fdc3003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006fd0 tx=0x7fdc38006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.734+0000 7fdc43400700 1 -- 172.21.15.67:0/1659680169 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdc3c01a7b0 con 0x7fdc4c07be50 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:05.947+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc4c102e00 con 0x7fdc30038750 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.326+0000 7fdc43400700 1 -- 172.21.15.67:0/1659680169 <== mgr.14122 v2:172.21.15.67:6800/2682757440 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdc4c102e00 con 0x7fdc30038750 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.328+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fdc30038750 msgr2=0x7fdc3003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:06.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.328+0000 7fdc51600700 1 --2- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fdc30038750 0x7fdc3003ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdc38006fd0 tx=0x7fdc38006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:06.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.328+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 msgr2=0x7fdc4c19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:06.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.328+0000 7fdc51600700 1 --2- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c19c4c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c011a20 tx=0x7fdc3c011b00 comp rx=0 tx=0).stop 2024-03-30T02:29:06.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.329+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 shutdown_connections 2024-03-30T02:29:06.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.329+0000 7fdc51600700 1 --2- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fdc30038750 0x7fdc3003ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:06.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.329+0000 7fdc51600700 1 --2- 172.21.15.67:0/1659680169 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc4c07be50 0x7fdc4c19c4c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:06.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.329+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 >> 172.21.15.67:0/1659680169 conn(0x7fdc4c101b40 msgr2=0x7fdc4c1026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:06.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.329+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 shutdown_connections 2024-03-30T02:29:06.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:06.329+0000 7fdc51600700 1 -- 172.21.15.67:0/1659680169 wait complete. 2024-03-30T02:29:07.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-mon[37808]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:07.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-mon[37808]: Generating ssh key... 2024-03-30T02:29:07.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:07.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:06 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:07.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKqO/4UK2rfMETcEn/x/hJclnuebIuywr2BaMr4/YZ1n8BGxLmB0wh0KE9o7+Fio9je/dNqkp3Rfh8wxSWWUFsobKBRWeY6j49S2HHv6CVuC299xzW80bazJNuYbUWVaxs+oPel13ZyL1ugw5Cl3a2GD7Tr1WO+xILUlLTt1crsDnAStMqNNEyPtvGeNIjzpRS0GUXxvVB/JXJVaInnHVQ8CoRCrjmevyRYCYIWBMWGUaDhVar846QgiyRjTU/y7SnsHLyt2r1Fo1eg7i2ab9Lcf+GWDEjAkQlamW5CCVKKne0wQEUPWm1dEMdvv3VwowBAbnD4SXLU6/g5Nq/7ncTAcjJvZ3x57oyAqKQKrCmauUKK9Vw3EpO8GtfJOGPvidKGuVdTZDE/w6i0V+RRXqWxj0h/pkMSPj6dkSLq+U3BGmjlntiW6aEdru/+dvtEsfjYILrvWfPA0ROyDRqomX7G2YfcSseq9ji0EsRMnuHeH0D7v9wKITApMrttoot37M= ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:29:07.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.123+0000 7f4450400700 1 Processor -- start 2024-03-30T02:29:07.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.123+0000 7f4450400700 1 -- start start 2024-03-30T02:29:07.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.124+0000 7f4450400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444807a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:07.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.124+0000 7f4450400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444807a880 con 0x7f444807be50 2024-03-30T02:29:07.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.124+0000 7f444da00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444807a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:07.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.124+0000 7f444da00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444807a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39584/0 (socket says 172.21.15.67:39584) 2024-03-30T02:29:07.729 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.124+0000 7f444da00700 1 -- 172.21.15.67:0/2403566437 learned_addr learned my addr 172.21.15.67:0/2403566437 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.124+0000 7f444da00700 1 -- 172.21.15.67:0/2403566437 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444807a9c0 con 0x7f444807be50 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.124+0000 7f444da00700 1 --2- 172.21.15.67:0/2403566437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444807a2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f443c0098a0 tx=0x7f443c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26427b8a146ffe6f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.125+0000 7f4447e00700 1 -- 172.21.15.67:0/2403566437 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f443c004160 con 0x7f444807be50 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.125+0000 7f4447e00700 1 -- 172.21.15.67:0/2403566437 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f443c0042c0 con 0x7f444807be50 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.125+0000 7f4447e00700 1 -- 172.21.15.67:0/2403566437 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f443c011a00 con 0x7f444807be50 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.125+0000 7f4450400700 1 -- 172.21.15.67:0/2403566437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 msgr2=0x7f444807a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.125+0000 7f4450400700 1 --2- 172.21.15.67:0/2403566437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444807a2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f443c0098a0 tx=0x7f443c009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.125+0000 7f4450400700 1 -- 172.21.15.67:0/2403566437 shutdown_connections 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.125+0000 7f4450400700 1 --2- 172.21.15.67:0/2403566437 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444807a2b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.125+0000 7f4450400700 1 -- 172.21.15.67:0/2403566437 >> 172.21.15.67:0/2403566437 conn(0x7f4448101ae0 msgr2=0x7f4448103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.126+0000 7f4450400700 1 -- 172.21.15.67:0/2403566437 shutdown_connections 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.126+0000 7f4450400700 1 -- 172.21.15.67:0/2403566437 wait complete. 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.127+0000 7f4450400700 1 Processor -- start 2024-03-30T02:29:07.730 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.127+0000 7f4450400700 1 -- start start 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.127+0000 7f4450400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.127+0000 7f4450400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444819c970 con 0x7f444807be50 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.127+0000 7f444da00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.127+0000 7f444da00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39588/0 (socket says 172.21.15.67:39588) 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.127+0000 7f444da00700 1 -- 172.21.15.67:0/3725657550 learned_addr learned my addr 172.21.15.67:0/3725657550 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.128+0000 7f444da00700 1 -- 172.21.15.67:0/3725657550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c009550 con 0x7f444807be50 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.128+0000 7f444da00700 1 --2- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444819c430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f443c009130 tx=0x7f443c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.128+0000 7f4446000700 1 -- 172.21.15.67:0/3725657550 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f443c003b70 con 0x7f444807be50 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.128+0000 7f4446000700 1 -- 172.21.15.67:0/3725657550 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f443c003cd0 con 0x7f444807be50 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.128+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444819cb70 con 0x7f444807be50 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.128+0000 7f4446000700 1 -- 172.21.15.67:0/3725657550 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f443c0197d0 con 0x7f444807be50 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.128+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444819d010 con 0x7f444807be50 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.129+0000 7f4446000700 1 -- 172.21.15.67:0/3725657550 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f443c01a430 con 0x7f444807be50 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.129+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444810c8b0 con 0x7f444807be50 2024-03-30T02:29:07.731 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.130+0000 7f4446000700 1 --2- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f4430038750 0x7f443003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.130+0000 7f4446000700 1 -- 172.21.15.67:0/3725657550 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f443c04c950 con 0x7f444807be50 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.130+0000 7f444d000700 1 --2- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f4430038750 0x7f443003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.131+0000 7f444d000700 1 --2- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f4430038750 0x7f443003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4434006fd0 tx=0x7f4434006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.136+0000 7f4446000700 1 -- 172.21.15.67:0/3725657550 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f443c02d960 con 0x7f444807be50 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.349+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4448102d80 con 0x7f4430038750 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.350+0000 7f4446000700 1 -- 172.21.15.67:0/3725657550 <== mgr.14122 v2:172.21.15.67:6800/2682757440 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f4448102d80 con 0x7f4430038750 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f4430038750 msgr2=0x7f443003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 --2- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f4430038750 0x7f443003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4434006fd0 tx=0x7f4434006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 msgr2=0x7f444819c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 --2- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444819c430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f443c009130 tx=0x7f443c011ad0 comp rx=0 tx=0).stop 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 shutdown_connections 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 --2- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f4430038750 0x7f443003ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 --2- 172.21.15.67:0/3725657550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f444807be50 0x7f444819c430 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 >> 172.21.15.67:0/3725657550 conn(0x7f4448101ae0 msgr2=0x7f4448102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:07.732 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 shutdown_connections 2024-03-30T02:29:07.733 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:07.353+0000 7f4450400700 1 -- 172.21.15.67:0/3725657550 wait complete. 2024-03-30T02:29:07.733 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-30T02:29:07.733 INFO:teuthology.orchestra.run.smithi067.stdout:Adding key to root@localhost authorized_keys... 2024-03-30T02:29:07.733 INFO:teuthology.orchestra.run.smithi067.stdout:Adding host smithi067... 2024-03-30T02:29:08.474 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:08 smithi067 ceph-mon[37808]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:09 smithi067 ceph-mon[37808]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:10 smithi067 ceph-mon[37808]: Deploying cephadm binary to smithi067 2024-03-30T02:29:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Added host 'smithi067' with addr '172.21.15.67' 2024-03-30T02:29:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.159+0000 7f9db4200700 1 Processor -- start 2024-03-30T02:29:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.160+0000 7f9db4200700 1 -- start start 2024-03-30T02:29:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.160+0000 7f9db4200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.160+0000 7f9db4200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac07a8e0 con 0x7f9dac07beb0 2024-03-30T02:29:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.160+0000 7f9db1800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.160+0000 7f9db1800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39602/0 (socket says 172.21.15.67:39602) 2024-03-30T02:29:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.160+0000 7f9db1800700 1 -- 172.21.15.67:0/3340197476 learned_addr learned my addr 172.21.15.67:0/3340197476 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.161+0000 7f9db1800700 1 -- 172.21.15.67:0/3340197476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac07aa20 con 0x7f9dac07beb0 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.161+0000 7f9db1800700 1 --2- 172.21.15.67:0/3340197476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac07a310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9da00098a0 tx=0x7f9da0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3364f31d84dd07 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.161+0000 7f9dabe00700 1 -- 172.21.15.67:0/3340197476 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9da0004160 con 0x7f9dac07beb0 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.161+0000 7f9dabe00700 1 -- 172.21.15.67:0/3340197476 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9da00042c0 con 0x7f9dac07beb0 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.161+0000 7f9dabe00700 1 -- 172.21.15.67:0/3340197476 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9da0011a00 con 0x7f9dac07beb0 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.162+0000 7f9db4200700 1 -- 172.21.15.67:0/3340197476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 msgr2=0x7f9dac07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.162+0000 7f9db4200700 1 --2- 172.21.15.67:0/3340197476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac07a310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9da00098a0 tx=0x7f9da0009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.162+0000 7f9db4200700 1 -- 172.21.15.67:0/3340197476 shutdown_connections 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.162+0000 7f9db4200700 1 --2- 172.21.15.67:0/3340197476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac07a310 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.162+0000 7f9db4200700 1 -- 172.21.15.67:0/3340197476 >> 172.21.15.67:0/3340197476 conn(0x7f9dac101ae0 msgr2=0x7f9dac103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.162+0000 7f9db4200700 1 -- 172.21.15.67:0/3340197476 shutdown_connections 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.162+0000 7f9db4200700 1 -- 172.21.15.67:0/3340197476 wait complete. 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.163+0000 7f9db4200700 1 Processor -- start 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.163+0000 7f9db4200700 1 -- start start 2024-03-30T02:29:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.163+0000 7f9db4200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.163+0000 7f9db4200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac19ca00 con 0x7f9dac07beb0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.164+0000 7f9db1800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.164+0000 7f9db1800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39616/0 (socket says 172.21.15.67:39616) 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.164+0000 7f9db1800700 1 -- 172.21.15.67:0/2365063880 learned_addr learned my addr 172.21.15.67:0/2365063880 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.164+0000 7f9db1800700 1 -- 172.21.15.67:0/2365063880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da0009550 con 0x7f9dac07beb0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.164+0000 7f9db1800700 1 --2- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac19c4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9da0011710 tx=0x7f9da0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.164+0000 7f9daa000700 1 -- 172.21.15.67:0/2365063880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9da0003aa0 con 0x7f9dac07beb0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.164+0000 7f9daa000700 1 -- 172.21.15.67:0/2365063880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9da0003c00 con 0x7f9dac07beb0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.164+0000 7f9daa000700 1 -- 172.21.15.67:0/2365063880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9da0024800 con 0x7f9dac07beb0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.164+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dac19cc00 con 0x7f9dac07beb0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.165+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac19d0a0 con 0x7f9dac07beb0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.166+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dac10c8b0 con 0x7f9dac07beb0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.166+0000 7f9daa000700 1 -- 172.21.15.67:0/2365063880 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f9da001abd0 con 0x7f9dac07beb0 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.166+0000 7f9daa000700 1 --2- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f9d94038750 0x7f9d9403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.166+0000 7f9daa000700 1 -- 172.21.15.67:0/2365063880 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9da004c6e0 con 0x7f9dac07beb0 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.167+0000 7f9db0e00700 1 --2- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f9d94038750 0x7f9d9403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.167+0000 7f9db0e00700 1 --2- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f9d94038750 0x7f9d9403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d98006fd0 tx=0x7f9d98006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.171+0000 7f9daa000700 1 -- 172.21.15.67:0/2365063880 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9da0018760 con 0x7f9dac07beb0 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:08.391+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}) v1 -- 0x7f9dac102dc0 con 0x7f9d94038750 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.440+0000 7f9daa000700 1 -- 172.21.15.67:0/2365063880 <== mgr.14122 v2:172.21.15.67:6800/2682757440 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f9dac102dc0 con 0x7f9d94038750 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f9d94038750 msgr2=0x7f9d9403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 --2- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f9d94038750 0x7f9d9403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9d98006fd0 tx=0x7f9d98006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 msgr2=0x7f9dac19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 --2- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac19c4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9da0011710 tx=0x7f9da0011ad0 comp rx=0 tx=0).stop 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 shutdown_connections 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 --2- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f9d94038750 0x7f9d9403ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 --2- 172.21.15.67:0/2365063880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dac07beb0 0x7f9dac19c4c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 >> 172.21.15.67:0/2365063880 conn(0x7f9dac101ae0 msgr2=0x7f9dac102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 shutdown_connections 2024-03-30T02:29:11.823 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:11.443+0000 7f9db4200700 1 -- 172.21.15.67:0/2365063880 wait complete. 2024-03-30T02:29:11.823 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mon service... 2024-03-30T02:29:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:12 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:12 smithi067 ceph-mon[37808]: Added host smithi067 2024-03-30T02:29:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:12 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:13.058 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-30T02:29:13.064 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.313+0000 7fd7e6600700 1 Processor -- start 2024-03-30T02:29:13.064 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.313+0000 7fd7e6600700 1 -- start start 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.313+0000 7fd7e6600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e0106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.313+0000 7fd7e6600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0106d20 con 0x7fd7e0106330 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.314+0000 7fd7e5200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e0106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.314+0000 7fd7e5200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e0106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39626/0 (socket says 172.21.15.67:39626) 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.314+0000 7fd7e5200700 1 -- 172.21.15.67:0/4285685185 learned_addr learned my addr 172.21.15.67:0/4285685185 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.314+0000 7fd7e5200700 1 -- 172.21.15.67:0/4285685185 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e0107530 con 0x7fd7e0106330 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.314+0000 7fd7e5200700 1 --2- 172.21.15.67:0/4285685185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e0106750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40098a0 tx=0x7fd7d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8ec675eed4849a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.315+0000 7fd7df400700 1 -- 172.21.15.67:0/4285685185 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7d4004160 con 0x7fd7e0106330 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.315+0000 7fd7df400700 1 -- 172.21.15.67:0/4285685185 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7d40042c0 con 0x7fd7e0106330 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.315+0000 7fd7df400700 1 -- 172.21.15.67:0/4285685185 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7d4011a00 con 0x7fd7e0106330 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.315+0000 7fd7e6600700 1 -- 172.21.15.67:0/4285685185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 msgr2=0x7fd7e0106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.315+0000 7fd7e6600700 1 --2- 172.21.15.67:0/4285685185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e0106750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40098a0 tx=0x7fd7d4009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.315+0000 7fd7e6600700 1 -- 172.21.15.67:0/4285685185 shutdown_connections 2024-03-30T02:29:13.065 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.315+0000 7fd7e6600700 1 --2- 172.21.15.67:0/4285685185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e0106750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.315+0000 7fd7e6600700 1 -- 172.21.15.67:0/4285685185 >> 172.21.15.67:0/4285685185 conn(0x7fd7e01018b0 msgr2=0x7fd7e0103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.316+0000 7fd7e6600700 1 -- 172.21.15.67:0/4285685185 shutdown_connections 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.316+0000 7fd7e6600700 1 -- 172.21.15.67:0/4285685185 wait complete. 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.317+0000 7fd7e6600700 1 Processor -- start 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.317+0000 7fd7e6600700 1 -- start start 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.318+0000 7fd7e6600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.318+0000 7fd7e6600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e019c7d0 con 0x7fd7e0106330 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.318+0000 7fd7e5200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.318+0000 7fd7e5200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39636/0 (socket says 172.21.15.67:39636) 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.318+0000 7fd7e5200700 1 -- 172.21.15.67:0/4281143964 learned_addr learned my addr 172.21.15.67:0/4281143964 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.319+0000 7fd7e5200700 1 -- 172.21.15.67:0/4281143964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d4009550 con 0x7fd7e0106330 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.319+0000 7fd7e5200700 1 --2- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e019c290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009130 tx=0x7fd7d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.319+0000 7fd7dd600700 1 -- 172.21.15.67:0/4281143964 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7d4003ab0 con 0x7fd7e0106330 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.319+0000 7fd7dd600700 1 -- 172.21.15.67:0/4281143964 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7d4003c10 con 0x7fd7e0106330 2024-03-30T02:29:13.066 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.319+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e019c9d0 con 0x7fd7e0106330 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.319+0000 7fd7dd600700 1 -- 172.21.15.67:0/4281143964 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7d40247d0 con 0x7fd7e0106330 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.319+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e019ce70 con 0x7fd7e0106330 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.320+0000 7fd7dd600700 1 -- 172.21.15.67:0/4281143964 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fd7d402d3f0 con 0x7fd7e0106330 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.320+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e010a760 con 0x7fd7e0106330 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.321+0000 7fd7dd600700 1 --2- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fd7c8038750 0x7fd7c803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.321+0000 7fd7dd600700 1 -- 172.21.15.67:0/4281143964 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd7d404c890 con 0x7fd7e0106330 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.321+0000 7fd7dfe00700 1 --2- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fd7c8038750 0x7fd7c803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.322+0000 7fd7dfe00700 1 --2- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fd7c8038750 0x7fd7c803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc006fd0 tx=0x7fd7cc006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.326+0000 7fd7dd600700 1 -- 172.21.15.67:0/4281143964 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd7d4010970 con 0x7fd7e0106330 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.554+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd7e0102b80 con 0x7fd7c8038750 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.560+0000 7fd7dd600700 1 -- 172.21.15.67:0/4281143964 <== mgr.14122 v2:172.21.15.67:6800/2682757440 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd7e0102b80 con 0x7fd7c8038750 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.562+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fd7c8038750 msgr2=0x7fd7c803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.562+0000 7fd7e6600700 1 --2- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fd7c8038750 0x7fd7c803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc006fd0 tx=0x7fd7cc006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:13.067 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.562+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 msgr2=0x7fd7e019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.562+0000 7fd7e6600700 1 --2- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e019c290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009130 tx=0x7fd7d4011ad0 comp rx=0 tx=0).stop 2024-03-30T02:29:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.562+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 shutdown_connections 2024-03-30T02:29:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.562+0000 7fd7e6600700 1 --2- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fd7c8038750 0x7fd7c803ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.562+0000 7fd7e6600700 1 --2- 172.21.15.67:0/4281143964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd7e0106330 0x7fd7e019c290 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.562+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 >> 172.21.15.67:0/4281143964 conn(0x7fd7e01018b0 msgr2=0x7fd7e0102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.563+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 shutdown_connections 2024-03-30T02:29:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:12.563+0000 7fd7e6600700 1 -- 172.21.15.67:0/4281143964 wait complete. 2024-03-30T02:29:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mgr service... 2024-03-30T02:29:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:13 smithi067 ceph-mon[37808]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:13 smithi067 ceph-mon[37808]: Saving service mon spec with placement count:5 2024-03-30T02:29:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:13 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:13 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:13 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:14.146 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-30T02:29:14.146 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.487+0000 7f648c600700 1 Processor -- start 2024-03-30T02:29:14.146 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.487+0000 7f648c600700 1 -- start start 2024-03-30T02:29:14.146 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.488+0000 7f648c600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f6484106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:14.146 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.488+0000 7f648c600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6484106f10 con 0x7f6484106520 2024-03-30T02:29:14.146 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.488+0000 7f6489c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f6484106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:14.146 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.488+0000 7f6489c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f6484106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39640/0 (socket says 172.21.15.67:39640) 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.488+0000 7f6489c00700 1 -- 172.21.15.67:0/883400634 learned_addr learned my addr 172.21.15.67:0/883400634 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.489+0000 7f6489c00700 1 -- 172.21.15.67:0/883400634 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6484107720 con 0x7f6484106520 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.489+0000 7f6489c00700 1 --2- 172.21.15.67:0/883400634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f6484106940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64780098a0 tx=0x7f6478009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763d6c351c2227d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.489+0000 7f6483e00700 1 -- 172.21.15.67:0/883400634 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6478004160 con 0x7f6484106520 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.489+0000 7f6483e00700 1 -- 172.21.15.67:0/883400634 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64780042c0 con 0x7f6484106520 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.489+0000 7f6483e00700 1 -- 172.21.15.67:0/883400634 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64780119b0 con 0x7f6484106520 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.490+0000 7f648c600700 1 -- 172.21.15.67:0/883400634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 msgr2=0x7f6484106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.490+0000 7f648c600700 1 --2- 172.21.15.67:0/883400634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f6484106940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f64780098a0 tx=0x7f6478009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.490+0000 7f648c600700 1 -- 172.21.15.67:0/883400634 shutdown_connections 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.490+0000 7f648c600700 1 --2- 172.21.15.67:0/883400634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f6484106940 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.490+0000 7f648c600700 1 -- 172.21.15.67:0/883400634 >> 172.21.15.67:0/883400634 conn(0x7f6484101ae0 msgr2=0x7f6484103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.490+0000 7f648c600700 1 -- 172.21.15.67:0/883400634 shutdown_connections 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.490+0000 7f648c600700 1 -- 172.21.15.67:0/883400634 wait complete. 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.492+0000 7f648c600700 1 Processor -- start 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.492+0000 7f648c600700 1 -- start start 2024-03-30T02:29:14.147 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.492+0000 7f648c600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f648419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.492+0000 7f648c600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648419a840 con 0x7f6484106520 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.492+0000 7f6489c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f648419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.493+0000 7f6489c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f648419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39642/0 (socket says 172.21.15.67:39642) 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.493+0000 7f6489c00700 1 -- 172.21.15.67:0/2367369040 learned_addr learned my addr 172.21.15.67:0/2367369040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.493+0000 7f6489c00700 1 -- 172.21.15.67:0/2367369040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6478009550 con 0x7f6484106520 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.493+0000 7f6489c00700 1 --2- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f648419a300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6478011670 tx=0x7f6478011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.493+0000 7f6482000700 1 -- 172.21.15.67:0/2367369040 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6478003af0 con 0x7f6484106520 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.493+0000 7f6482000700 1 -- 172.21.15.67:0/2367369040 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6478003c50 con 0x7f6484106520 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.493+0000 7f6482000700 1 -- 172.21.15.67:0/2367369040 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6478024820 con 0x7f6484106520 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.493+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648419aa40 con 0x7f6484106520 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.494+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648419aee0 con 0x7f6484106520 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.495+0000 7f6482000700 1 -- 172.21.15.67:0/2367369040 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f647801ab60 con 0x7f6484106520 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.495+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648410a950 con 0x7f6484106520 2024-03-30T02:29:14.148 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.495+0000 7f6482000700 1 --2- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f646c038750 0x7f646c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.495+0000 7f6482000700 1 -- 172.21.15.67:0/2367369040 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f647804c7b0 con 0x7f6484106520 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.495+0000 7f6489200700 1 --2- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f646c038750 0x7f646c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.496+0000 7f6489200700 1 --2- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f646c038750 0x7f646c03ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6470006fd0 tx=0x7f6470006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.500+0000 7f6482000700 1 -- 172.21.15.67:0/2367369040 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6478018760 con 0x7f6484106520 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.750+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f648419d770 con 0x7f646c038750 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.758+0000 7f6482000700 1 -- 172.21.15.67:0/2367369040 <== mgr.14122 v2:172.21.15.67:6800/2682757440 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f648419d770 con 0x7f646c038750 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.760+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f646c038750 msgr2=0x7f646c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.760+0000 7f648c600700 1 --2- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f646c038750 0x7f646c03ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6470006fd0 tx=0x7f6470006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.760+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 msgr2=0x7f648419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.760+0000 7f648c600700 1 --2- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f648419a300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6478011670 tx=0x7f6478011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.761+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 shutdown_connections 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.761+0000 7f648c600700 1 --2- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f646c038750 0x7f646c03ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.761+0000 7f648c600700 1 --2- 172.21.15.67:0/2367369040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6484106520 0x7f648419a300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.761+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 >> 172.21.15.67:0/2367369040 conn(0x7f6484101ae0 msgr2=0x7f6484102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.761+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 shutdown_connections 2024-03-30T02:29:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:13.761+0000 7f648c600700 1 -- 172.21.15.67:0/2367369040 wait complete. 2024-03-30T02:29:15.061 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:14 smithi067 ceph-mon[37808]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:15.061 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:14 smithi067 ceph-mon[37808]: Saving service mgr spec with placement count:2 2024-03-30T02:29:15.061 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:14 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.638+0000 7fedb3e00700 1 Processor -- start 2024-03-30T02:29:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.638+0000 7fedb3e00700 1 -- start start 2024-03-30T02:29:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.638+0000 7fedb3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac105820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:15.248 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.638+0000 7fedb3e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac105df0 con 0x7fedac105400 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.638+0000 7fedb1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.639+0000 7fedb1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac105820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39646/0 (socket says 172.21.15.67:39646) 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.639+0000 7fedb1400700 1 -- 172.21.15.67:0/682932725 learned_addr learned my addr 172.21.15.67:0/682932725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.639+0000 7fedb1400700 1 -- 172.21.15.67:0/682932725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac106600 con 0x7fedac105400 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.639+0000 7fedb1400700 1 --2- 172.21.15.67:0/682932725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac105820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feda00098a0 tx=0x7feda0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f915f4a83c8ec3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.639+0000 7fedabe00700 1 -- 172.21.15.67:0/682932725 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feda0004160 con 0x7fedac105400 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.639+0000 7fedabe00700 1 -- 172.21.15.67:0/682932725 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda00042c0 con 0x7fedac105400 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.639+0000 7fedabe00700 1 -- 172.21.15.67:0/682932725 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feda00119b0 con 0x7fedac105400 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.640+0000 7fedb3e00700 1 -- 172.21.15.67:0/682932725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 msgr2=0x7fedac105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.640+0000 7fedb3e00700 1 --2- 172.21.15.67:0/682932725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac105820 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feda00098a0 tx=0x7feda0009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.640+0000 7fedb3e00700 1 -- 172.21.15.67:0/682932725 shutdown_connections 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.640+0000 7fedb3e00700 1 --2- 172.21.15.67:0/682932725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac105820 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.640+0000 7fedb3e00700 1 -- 172.21.15.67:0/682932725 >> 172.21.15.67:0/682932725 conn(0x7fedac100a00 msgr2=0x7fedac102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.640+0000 7fedb3e00700 1 -- 172.21.15.67:0/682932725 shutdown_connections 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.640+0000 7fedb3e00700 1 -- 172.21.15.67:0/682932725 wait complete. 2024-03-30T02:29:15.249 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.641+0000 7fedb3e00700 1 Processor -- start 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.642+0000 7fedb3e00700 1 -- start start 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.642+0000 7fedb3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.642+0000 7fedb3e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac19c9c0 con 0x7fedac105400 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.642+0000 7fedb1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.642+0000 7fedb1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39660/0 (socket says 172.21.15.67:39660) 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.642+0000 7fedb1400700 1 -- 172.21.15.67:0/809767102 learned_addr learned my addr 172.21.15.67:0/809767102 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.642+0000 7fedb1400700 1 -- 172.21.15.67:0/809767102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda0009550 con 0x7fedac105400 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.643+0000 7fedb1400700 1 --2- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac19c480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feda0003e60 tx=0x7feda0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.643+0000 7fedaa000700 1 -- 172.21.15.67:0/809767102 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feda0003a60 con 0x7fedac105400 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.643+0000 7fedaa000700 1 -- 172.21.15.67:0/809767102 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda0003bc0 con 0x7fedac105400 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.643+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac19cbc0 con 0x7fedac105400 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.643+0000 7fedaa000700 1 -- 172.21.15.67:0/809767102 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feda00196c0 con 0x7fedac105400 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.643+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac19d060 con 0x7fedac105400 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.644+0000 7fedaa000700 1 -- 172.21.15.67:0/809767102 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7feda001a430 con 0x7fedac105400 2024-03-30T02:29:15.250 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.644+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac1960a0 con 0x7fedac105400 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.645+0000 7fedaa000700 1 --2- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fed94038730 0x7fed9403abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.645+0000 7fedaa000700 1 -- 172.21.15.67:0/809767102 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7feda004c9a0 con 0x7fedac105400 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.645+0000 7fedb0a00700 1 --2- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fed94038730 0x7fed9403abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.646+0000 7fedb0a00700 1 --2- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fed94038730 0x7fed9403abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed98006fd0 tx=0x7fed98006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.650+0000 7fedaa000700 1 -- 172.21.15.67:0/809767102 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7feda0010380 con 0x7fedac105400 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.866+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fedac078fc0 con 0x7fedac105400 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.872+0000 7fedaa000700 1 -- 172.21.15.67:0/809767102 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7feda0010380 con 0x7fedac105400 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.874+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fed94038730 msgr2=0x7fed9403abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.874+0000 7fedb3e00700 1 --2- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fed94038730 0x7fed9403abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed98006fd0 tx=0x7fed98006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.874+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 msgr2=0x7fedac19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.874+0000 7fedb3e00700 1 --2- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac19c480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feda0003e60 tx=0x7feda0011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.875+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 shutdown_connections 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.875+0000 7fedb3e00700 1 --2- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fed94038730 0x7fed9403abf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.875+0000 7fedb3e00700 1 --2- 172.21.15.67:0/809767102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fedac105400 0x7fedac19c480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.875+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 >> 172.21.15.67:0/809767102 conn(0x7fedac100a00 msgr2=0x7fedac1931e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:15.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.875+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 shutdown_connections 2024-03-30T02:29:15.252 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:14.875+0000 7fedb3e00700 1 -- 172.21.15.67:0/809767102 wait complete. 2024-03-30T02:29:16.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:15 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:16.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:15 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/809767102' entity='client.admin' 2024-03-30T02:29:16.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.665+0000 7f7a48a00700 1 Processor -- start 2024-03-30T02:29:16.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.665+0000 7f7a48a00700 1 -- start start 2024-03-30T02:29:16.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.665+0000 7f7a48a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:16.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.665+0000 7f7a48a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4407a8c0 con 0x7f7a4407be90 2024-03-30T02:29:16.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.666+0000 7f7a41e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:16.462 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.666+0000 7f7a41e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39666/0 (socket says 172.21.15.67:39666) 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.666+0000 7f7a41e00700 1 -- 172.21.15.67:0/3333493115 learned_addr learned my addr 172.21.15.67:0/3333493115 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.666+0000 7f7a41e00700 1 -- 172.21.15.67:0/3333493115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4407aa00 con 0x7f7a4407be90 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.667+0000 7f7a41e00700 1 --2- 172.21.15.67:0/3333493115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4407a2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7a340098a0 tx=0x7f7a34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1db7f555bd3fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.667+0000 7f7a40a00700 1 -- 172.21.15.67:0/3333493115 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a34004160 con 0x7f7a4407be90 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.667+0000 7f7a40a00700 1 -- 172.21.15.67:0/3333493115 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a340042c0 con 0x7f7a4407be90 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.667+0000 7f7a40a00700 1 -- 172.21.15.67:0/3333493115 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a340119b0 con 0x7f7a4407be90 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.668+0000 7f7a48a00700 1 -- 172.21.15.67:0/3333493115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 msgr2=0x7f7a4407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.668+0000 7f7a48a00700 1 --2- 172.21.15.67:0/3333493115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4407a2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7a340098a0 tx=0x7f7a34009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.668+0000 7f7a48a00700 1 -- 172.21.15.67:0/3333493115 shutdown_connections 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.668+0000 7f7a48a00700 1 --2- 172.21.15.67:0/3333493115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4407a2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.668+0000 7f7a48a00700 1 -- 172.21.15.67:0/3333493115 >> 172.21.15.67:0/3333493115 conn(0x7f7a44101ae0 msgr2=0x7f7a44103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.668+0000 7f7a48a00700 1 -- 172.21.15.67:0/3333493115 shutdown_connections 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.668+0000 7f7a48a00700 1 -- 172.21.15.67:0/3333493115 wait complete. 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.670+0000 7f7a48a00700 1 Processor -- start 2024-03-30T02:29:16.463 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.670+0000 7f7a48a00700 1 -- start start 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.670+0000 7f7a48a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.670+0000 7f7a48a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4419ca40 con 0x7f7a4407be90 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.670+0000 7f7a41e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.670+0000 7f7a41e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39682/0 (socket says 172.21.15.67:39682) 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.670+0000 7f7a41e00700 1 -- 172.21.15.67:0/3185500241 learned_addr learned my addr 172.21.15.67:0/3185500241 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.671+0000 7f7a41e00700 1 -- 172.21.15.67:0/3185500241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a34009550 con 0x7f7a4407be90 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.671+0000 7f7a41e00700 1 --2- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4419c500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a34003e60 tx=0x7f7a34011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.671+0000 7f7a3aa00700 1 -- 172.21.15.67:0/3185500241 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a340038b0 con 0x7f7a4407be90 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.672+0000 7f7a3aa00700 1 -- 172.21.15.67:0/3185500241 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a34003a10 con 0x7f7a4407be90 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.672+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4419cc40 con 0x7f7a4407be90 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.672+0000 7f7a3aa00700 1 -- 172.21.15.67:0/3185500241 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a34019530 con 0x7f7a4407be90 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.672+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4419d0e0 con 0x7f7a4407be90 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.673+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4410c920 con 0x7f7a4407be90 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.673+0000 7f7a3aa00700 1 -- 172.21.15.67:0/3185500241 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7f7a340197f0 con 0x7f7a4407be90 2024-03-30T02:29:16.464 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.674+0000 7f7a3aa00700 1 --2- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7a280386b0 0x7f7a2803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.674+0000 7f7a3aa00700 1 -- 172.21.15.67:0/3185500241 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7a3404ca20 con 0x7f7a4407be90 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.674+0000 7f7a41400700 1 --2- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7a280386b0 0x7f7a2803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.675+0000 7f7a41400700 1 --2- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7a280386b0 0x7f7a2803ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006fd0 tx=0x7f7a30006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.683+0000 7f7a3aa00700 1 -- 172.21.15.67:0/3185500241 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7a340187c0 con 0x7f7a4407be90 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.900+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f7a4410cb90 con 0x7f7a4407be90 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.906+0000 7f7a3aa00700 1 -- 172.21.15.67:0/3185500241 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f7a340187c0 con 0x7f7a4407be90 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.908+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7a280386b0 msgr2=0x7f7a2803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.908+0000 7f7a48a00700 1 --2- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7a280386b0 0x7f7a2803ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006fd0 tx=0x7f7a30006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.908+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 msgr2=0x7f7a4419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.908+0000 7f7a48a00700 1 --2- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4419c500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a34003e60 tx=0x7f7a34011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.909+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 shutdown_connections 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.909+0000 7f7a48a00700 1 --2- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7a280386b0 0x7f7a2803ab70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.909+0000 7f7a48a00700 1 --2- 172.21.15.67:0/3185500241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a4407be90 0x7f7a4419c500 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:16.465 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.909+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 >> 172.21.15.67:0/3185500241 conn(0x7f7a44101ae0 msgr2=0x7f7a44102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:16.466 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.909+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 shutdown_connections 2024-03-30T02:29:16.466 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:15.909+0000 7f7a48a00700 1 -- 172.21.15.67:0/3185500241 wait complete. 2024-03-30T02:29:16.466 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling the dashboard module... 2024-03-30T02:29:17.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:16 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3185500241' entity='client.admin' 2024-03-30T02:29:17.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:16 smithi067 ceph-mon[37808]: from='mgr.14122 172.21.15.67:0/920908976' entity='mgr.x' 2024-03-30T02:29:18.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:17 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4201898133' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-30T02:29:18.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: ignoring --setuser ceph since I am not root 2024-03-30T02:29:18.164 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: ignoring --setgroup ceph since I am not root 2024-03-30T02:29:18.311 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.913+0000 7fb506c00700 1 Processor -- start 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.914+0000 7fb506c00700 1 -- start start 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.914+0000 7fb506c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.914+0000 7fb506c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50007a8c0 con 0x7fb50007be90 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.914+0000 7fb4ffe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.915+0000 7fb4ffe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36948/0 (socket says 172.21.15.67:36948) 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.915+0000 7fb4ffe00700 1 -- 172.21.15.67:0/3534857063 learned_addr learned my addr 172.21.15.67:0/3534857063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.915+0000 7fb4ffe00700 1 -- 172.21.15.67:0/3534857063 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50007aa00 con 0x7fb50007be90 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.915+0000 7fb4ffe00700 1 --2- 172.21.15.67:0/3534857063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50007a2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00098a0 tx=0x7fb4f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1886ffca2c157c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.915+0000 7fb4fea00700 1 -- 172.21.15.67:0/3534857063 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0004160 con 0x7fb50007be90 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.916+0000 7fb4fea00700 1 -- 172.21.15.67:0/3534857063 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f00042c0 con 0x7fb50007be90 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.916+0000 7fb4fea00700 1 -- 172.21.15.67:0/3534857063 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4f00119b0 con 0x7fb50007be90 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.916+0000 7fb506c00700 1 -- 172.21.15.67:0/3534857063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 msgr2=0x7fb50007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.916+0000 7fb506c00700 1 --2- 172.21.15.67:0/3534857063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50007a2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00098a0 tx=0x7fb4f0009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:18.312 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.916+0000 7fb506c00700 1 -- 172.21.15.67:0/3534857063 shutdown_connections 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.916+0000 7fb506c00700 1 --2- 172.21.15.67:0/3534857063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50007a2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.916+0000 7fb506c00700 1 -- 172.21.15.67:0/3534857063 >> 172.21.15.67:0/3534857063 conn(0x7fb500101ae0 msgr2=0x7fb500103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.916+0000 7fb506c00700 1 -- 172.21.15.67:0/3534857063 shutdown_connections 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.916+0000 7fb506c00700 1 -- 172.21.15.67:0/3534857063 wait complete. 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.918+0000 7fb506c00700 1 Processor -- start 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.918+0000 7fb506c00700 1 -- start start 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.919+0000 7fb506c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.919+0000 7fb506c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50019ca40 con 0x7fb50007be90 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.919+0000 7fb4ffe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.919+0000 7fb4ffe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36952/0 (socket says 172.21.15.67:36952) 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.919+0000 7fb4ffe00700 1 -- 172.21.15.67:0/4201898133 learned_addr learned my addr 172.21.15.67:0/4201898133 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.919+0000 7fb4ffe00700 1 -- 172.21.15.67:0/4201898133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0009550 con 0x7fb50007be90 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.920+0000 7fb4ffe00700 1 --2- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50019c500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0011670 tx=0x7fb4f0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.920+0000 7fb4fcc00700 1 -- 172.21.15.67:0/4201898133 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0003af0 con 0x7fb50007be90 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.920+0000 7fb4fcc00700 1 -- 172.21.15.67:0/4201898133 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f0003c50 con 0x7fb50007be90 2024-03-30T02:29:18.313 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.920+0000 7fb4fcc00700 1 -- 172.21.15.67:0/4201898133 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0024820 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.920+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50019cc40 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.920+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50019d0e0 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.922+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50010c920 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.922+0000 7fb4fcc00700 1 -- 172.21.15.67:0/4201898133 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 45533+0+0 (secure 0 0 0) 0x7fb4f001ab60 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.922+0000 7fb4fcc00700 1 --2- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fb4e4038750 0x7fb4e403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.922+0000 7fb4fcc00700 1 -- 172.21.15.67:0/4201898133 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb4f004c840 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.923+0000 7fb4ff400700 1 --2- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fb4e4038750 0x7fb4e403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.924+0000 7fb4ff400700 1 --2- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fb4e4038750 0x7fb4e403ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec006fd0 tx=0x7fb4ec006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:16.928+0000 7fb4fcc00700 1 -- 172.21.15.67:0/4201898133 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb4f0018760 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.213+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fb50010cb60 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.910+0000 7fb4fcc00700 1 -- 172.21.15.67:0/4201898133 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fb4f001a430 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.912+0000 7fb4fcc00700 1 -- 172.21.15.67:0/4201898133 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fb4f002c560 con 0x7fb50007be90 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.914+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fb4e4038750 msgr2=0x7fb4e403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:18.314 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.914+0000 7fb506c00700 1 --2- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fb4e4038750 0x7fb4e403ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec006fd0 tx=0x7fb4ec006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:18.315 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.914+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 msgr2=0x7fb50019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:18.315 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.914+0000 7fb506c00700 1 --2- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50019c500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0011670 tx=0x7fb4f0011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:18.315 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.915+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 shutdown_connections 2024-03-30T02:29:18.315 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.915+0000 7fb506c00700 1 --2- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fb4e4038750 0x7fb4e403ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:18.315 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.915+0000 7fb506c00700 1 --2- 172.21.15.67:0/4201898133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb50007be90 0x7fb50019c500 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:18.315 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.915+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 >> 172.21.15.67:0/4201898133 conn(0x7fb500101ae0 msgr2=0x7fb500102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:18.315 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.915+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 shutdown_connections 2024-03-30T02:29:18.315 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:17.915+0000 7fb506c00700 1 -- 172.21.15.67:0/4201898133 wait complete. 2024-03-30T02:29:18.830 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:18.570+0000 7febe2f2f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T02:29:18.830 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:18.696+0000 7febe2f2f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T02:29:18.830 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:18.828+0000 7febe2f2f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T02:29:19.111 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:18.944+0000 7febe2f2f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T02:29:19.112 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:18 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4201898133' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-30T02:29:19.112 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:18 smithi067 ceph-mon[37808]: mgrmap e9: x(active, since 16s) 2024-03-30T02:29:19.469 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-03-30T02:29:19.475 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-30T02:29:19.475 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-30T02:29:19.475 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-30T02:29:19.475 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-30T02:29:19.475 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-03-30T02:29:19.475 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.789+0000 7fba31400700 1 Processor -- start 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.789+0000 7fba31400700 1 -- start start 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.789+0000 7fba31400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.789+0000 7fba31400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c07a8a0 con 0x7fba2c07be70 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.790+0000 7fba2a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.790+0000 7fba2a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36972/0 (socket says 172.21.15.67:36972) 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.790+0000 7fba2a800700 1 -- 172.21.15.67:0/1534969049 learned_addr learned my addr 172.21.15.67:0/1534969049 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.790+0000 7fba2a800700 1 -- 172.21.15.67:0/1534969049 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c07a9e0 con 0x7fba2c07be70 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.790+0000 7fba2a800700 1 --2- 172.21.15.67:0/1534969049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c07a2d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0098a0 tx=0x7fba1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a94f231e700c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.790+0000 7fba29400700 1 -- 172.21.15.67:0/1534969049 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba1c004160 con 0x7fba2c07be70 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.791+0000 7fba29400700 1 -- 172.21.15.67:0/1534969049 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba1c0042c0 con 0x7fba2c07be70 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.791+0000 7fba29400700 1 -- 172.21.15.67:0/1534969049 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba1c0119b0 con 0x7fba2c07be70 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.791+0000 7fba31400700 1 -- 172.21.15.67:0/1534969049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 msgr2=0x7fba2c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.791+0000 7fba31400700 1 --2- 172.21.15.67:0/1534969049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c07a2d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0098a0 tx=0x7fba1c009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:19.476 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.791+0000 7fba31400700 1 -- 172.21.15.67:0/1534969049 shutdown_connections 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.791+0000 7fba31400700 1 --2- 172.21.15.67:0/1534969049 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c07a2d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.791+0000 7fba31400700 1 -- 172.21.15.67:0/1534969049 >> 172.21.15.67:0/1534969049 conn(0x7fba2c101b40 msgr2=0x7fba2c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.791+0000 7fba31400700 1 -- 172.21.15.67:0/1534969049 shutdown_connections 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.791+0000 7fba31400700 1 -- 172.21.15.67:0/1534969049 wait complete. 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.792+0000 7fba31400700 1 Processor -- start 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.793+0000 7fba31400700 1 -- start start 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.793+0000 7fba31400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.793+0000 7fba31400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c19c9d0 con 0x7fba2c07be70 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.793+0000 7fba2a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.793+0000 7fba2a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36980/0 (socket says 172.21.15.67:36980) 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.793+0000 7fba2a800700 1 -- 172.21.15.67:0/634402779 learned_addr learned my addr 172.21.15.67:0/634402779 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.794+0000 7fba2a800700 1 -- 172.21.15.67:0/634402779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c009550 con 0x7fba2c07be70 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.794+0000 7fba2a800700 1 --2- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c19c490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba1c003e60 tx=0x7fba1c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.794+0000 7fba23400700 1 -- 172.21.15.67:0/634402779 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba1c0038b0 con 0x7fba2c07be70 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.794+0000 7fba23400700 1 -- 172.21.15.67:0/634402779 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba1c003a10 con 0x7fba2c07be70 2024-03-30T02:29:19.477 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.794+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c19cbd0 con 0x7fba2c07be70 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.794+0000 7fba23400700 1 -- 172.21.15.67:0/634402779 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba1c019530 con 0x7fba2c07be70 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.794+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c19d070 con 0x7fba2c07be70 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.795+0000 7fba23400700 1 -- 172.21.15.67:0/634402779 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7fba1c01a980 con 0x7fba2c07be70 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.796+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c10c9c0 con 0x7fba2c07be70 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.796+0000 7fba23400700 1 --2- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fba10038750 0x7fba1003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.796+0000 7fba29e00700 1 -- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fba10038750 msgr2=0x7fba1003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2682757440 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.796+0000 7fba23400700 1 -- 172.21.15.67:0/634402779 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fba1c04c880 con 0x7fba2c07be70 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.796+0000 7fba29e00700 1 --2- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fba10038750 0x7fba1003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.801+0000 7fba23400700 1 -- 172.21.15.67:0/634402779 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fba1c003b80 con 0x7fba2c07be70 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.997+0000 7fba29e00700 1 -- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fba10038750 msgr2=0x7fba1003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2682757440 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:18.997+0000 7fba29e00700 1 --2- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fba10038750 0x7fba1003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.086+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fba2c19f900 con 0x7fba2c07be70 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.086+0000 7fba23400700 1 -- 172.21.15.67:0/634402779 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fba1c02de00 con 0x7fba2c07be70 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fba10038750 msgr2=0x7fba1003ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 --2- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fba10038750 0x7fba1003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:19.478 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 msgr2=0x7fba2c19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:19.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 --2- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c19c490 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba1c003e60 tx=0x7fba1c011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:19.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 shutdown_connections 2024-03-30T02:29:19.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 --2- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7fba10038750 0x7fba1003ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:19.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 --2- 172.21.15.67:0/634402779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba2c07be70 0x7fba2c19c490 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:19.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 >> 172.21.15.67:0/634402779 conn(0x7fba2c101b40 msgr2=0x7fba2c102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:19.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 shutdown_connections 2024-03-30T02:29:19.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.089+0000 7fba31400700 1 -- 172.21.15.67:0/634402779 wait complete. 2024-03-30T02:29:19.479 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-03-30T02:29:19.479 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 9... 2024-03-30T02:29:20.161 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:19 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/634402779' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-30T02:29:20.162 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:19 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:19.829+0000 7febe2f2f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:29:20.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:20 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:20.160+0000 7febe2f2f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T02:29:20.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:20 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:20.275+0000 7febe2f2f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T02:29:20.796 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:20 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:20.419+0000 7febe2f2f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T02:29:20.797 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:20 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:20.621+0000 7febe2f2f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T02:29:20.797 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:20 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:20.795+0000 7febe2f2f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T02:29:21.163 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:20 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:20.910+0000 7febe2f2f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T02:29:21.663 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:21 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:21.394+0000 7febe2f2f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T02:29:22.160 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:21 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:21.899+0000 7febe2f2f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T02:29:22.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:22 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:22.159+0000 7febe2f2f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T02:29:22.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:22 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:22.286+0000 7febe2f2f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T02:29:23.610 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:23 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:23.276+0000 7febe2f2f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T02:29:23.610 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:23 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:23.406+0000 7febe2f2f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T02:29:24.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:23 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:23.608+0000 7febe2f2f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T02:29:24.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:24 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:24.447+0000 7febe2f2f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T02:29:24.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:24 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:24.695+0000 7febe2f2f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T02:29:25.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:24 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:24.960+0000 7febe2f2f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T02:29:26.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:26.150+0000 7febe2f2f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T02:29:26.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:26.540+0000 7febe2f2f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:29:26.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:26.662+0000 7febe2f2f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T02:29:27.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:29:26.926+0000 7febe2f2f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T02:29:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: Active manager daemon x restarted 2024-03-30T02:29:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: Activating manager daemon x 2024-03-30T02:29:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: osdmap e3: 0 total, 0 up, 0 in 2024-03-30T02:29:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: mgrmap e10: x(active, starting, since 0.00626277s) 2024-03-30T02:29:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:29:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-30T02:29:27.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-30T02:29:27.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-30T02:29:27.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-30T02:29:27.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: Manager daemon x is now available 2024-03-30T02:29:27.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:28.242 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-30T02:29:28.242 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-30T02:29:28.243 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:27 smithi067 ceph-mon[37808]: mgrmap e11: x(active, since 1.00891s) 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.953+0000 7f7e88200700 1 Processor -- start 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.953+0000 7f7e88200700 1 -- start start 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.953+0000 7f7e88200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e801069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.953+0000 7f7e88200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e80106f70 con 0x7f7e80106580 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.954+0000 7f7e85800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e801069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:28.379 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.954+0000 7f7e85800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e801069a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36982/0 (socket says 172.21.15.67:36982) 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.954+0000 7f7e85800700 1 -- 172.21.15.67:0/967562334 learned_addr learned my addr 172.21.15.67:0/967562334 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.954+0000 7f7e85800700 1 -- 172.21.15.67:0/967562334 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e80107780 con 0x7f7e80106580 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.955+0000 7f7e85800700 1 --2- 172.21.15.67:0/967562334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e801069a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7e740098a0 tx=0x7f7e74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edfebbaa25b46ca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.955+0000 7f7e7fe00700 1 -- 172.21.15.67:0/967562334 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e74004160 con 0x7f7e80106580 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.955+0000 7f7e7fe00700 1 -- 172.21.15.67:0/967562334 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e740042c0 con 0x7f7e80106580 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.955+0000 7f7e7fe00700 1 -- 172.21.15.67:0/967562334 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e740119b0 con 0x7f7e80106580 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.956+0000 7f7e88200700 1 -- 172.21.15.67:0/967562334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 msgr2=0x7f7e801069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.956+0000 7f7e88200700 1 --2- 172.21.15.67:0/967562334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e801069a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7e740098a0 tx=0x7f7e74009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.956+0000 7f7e88200700 1 -- 172.21.15.67:0/967562334 shutdown_connections 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.956+0000 7f7e88200700 1 --2- 172.21.15.67:0/967562334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e801069a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.956+0000 7f7e88200700 1 -- 172.21.15.67:0/967562334 >> 172.21.15.67:0/967562334 conn(0x7f7e80101b40 msgr2=0x7f7e80103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.956+0000 7f7e88200700 1 -- 172.21.15.67:0/967562334 shutdown_connections 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.956+0000 7f7e88200700 1 -- 172.21.15.67:0/967562334 wait complete. 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.957+0000 7f7e88200700 1 Processor -- start 2024-03-30T02:29:28.380 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.958+0000 7f7e88200700 1 -- start start 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.958+0000 7f7e88200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e8019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.958+0000 7f7e88200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8019a820 con 0x7f7e80106580 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.958+0000 7f7e85800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e8019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.958+0000 7f7e85800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e8019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36996/0 (socket says 172.21.15.67:36996) 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.958+0000 7f7e85800700 1 -- 172.21.15.67:0/3248385014 learned_addr learned my addr 172.21.15.67:0/3248385014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.959+0000 7f7e85800700 1 -- 172.21.15.67:0/3248385014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e74009550 con 0x7f7e80106580 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.959+0000 7f7e85800700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e8019a2e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7e74011670 tx=0x7f7e74011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.959+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e74003af0 con 0x7f7e80106580 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.959+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e74003c50 con 0x7f7e80106580 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.959+0000 7f7e88200700 1 -- 172.21.15.67:0/3248385014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8019aa20 con 0x7f7e80106580 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.959+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e74024820 con 0x7f7e80106580 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.959+0000 7f7e88200700 1 -- 172.21.15.67:0/3248385014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8019aec0 con 0x7f7e80106580 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.960+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f7e7401ab60 con 0x7f7e80106580 2024-03-30T02:29:28.381 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.961+0000 7f7e7e000700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 0x7f7e6803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.961+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 --> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e6803b370 con 0x7f7e680387a0 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.961+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7e7404c7f0 con 0x7f7e80106580 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.961+0000 7f7e84e00700 1 -- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 msgr2=0x7f7e6803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2682757440 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:19.961+0000 7f7e84e00700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 0x7f7e6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:20.162+0000 7f7e84e00700 1 -- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 msgr2=0x7f7e6803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2682757440 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:20.162+0000 7f7e84e00700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 0x7f7e6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:20.563+0000 7f7e84e00700 1 -- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 msgr2=0x7f7e6803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2682757440 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:20.563+0000 7f7e84e00700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 0x7f7e6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:21.364+0000 7f7e84e00700 1 -- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 msgr2=0x7f7e6803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2682757440 2024-03-30T02:29:28.382 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:21.364+0000 7f7e84e00700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 0x7f7e6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:22.965+0000 7f7e84e00700 1 -- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 msgr2=0x7f7e6803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2682757440 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:22.965+0000 7f7e84e00700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 0x7f7e6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:26.168+0000 7f7e84e00700 1 -- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 msgr2=0x7f7e6803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2682757440 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:26.168+0000 7f7e84e00700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 0x7f7e6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:26.933+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mgrmap(e 10) v1 ==== 45313+0+0 (secure 0 0 0) 0x7f7e7401a430 con 0x7f7e80106580 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:26.934+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 msgr2=0x7f7e6803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:26.934+0000 7f7e7e000700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 0x7f7e6803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.935+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 11) v1 ==== 45440+0+0 (secure 0 0 0) 0x7f7e74021be0 con 0x7f7e80106580 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.936+0000 7f7e7e000700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7e6803c270 0x7f7e6803e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.936+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e6803b370 con 0x7f7e6803c270 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.936+0000 7f7e84e00700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7e6803c270 0x7f7e6803e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.937+0000 7f7e84e00700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7e6803c270 0x7f7e6803e660 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c003a10 tx=0x7f7e6c009270 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.938+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f7e6803b370 con 0x7f7e6803c270 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.946+0000 7f7e88200700 1 -- 172.21.15.67:0/3248385014 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7e8010a9b0 con 0x7f7e6803c270 2024-03-30T02:29:28.383 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.946+0000 7f7e7e000700 1 -- 172.21.15.67:0/3248385014 <== mgr.14154 v2:172.21.15.67:6800/3801648476 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f7e8010a9b0 con 0x7f7e6803c270 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.946+0000 7f7e88200700 1 -- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7e6803c270 msgr2=0x7f7e6803e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.946+0000 7f7e88200700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7e6803c270 0x7f7e6803e660 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c003a10 tx=0x7f7e6c009270 comp rx=0 tx=0).stop 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.946+0000 7f7e88200700 1 -- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 msgr2=0x7f7e8019a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.946+0000 7f7e88200700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e8019a2e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7e74011670 tx=0x7f7e74011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.947+0000 7f7e88200700 1 -- 172.21.15.67:0/3248385014 shutdown_connections 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.947+0000 7f7e88200700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7e80106580 0x7f7e8019a2e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.947+0000 7f7e88200700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7e6803c270 0x7f7e6803e660 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.947+0000 7f7e88200700 1 --2- 172.21.15.67:0/3248385014 >> [v2:172.21.15.67:6800/2682757440,v1:172.21.15.67:6801/2682757440] conn(0x7f7e680387a0 0x7f7e6803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.947+0000 7f7e88200700 1 -- 172.21.15.67:0/3248385014 >> 172.21.15.67:0/3248385014 conn(0x7f7e80101b40 msgr2=0x7f7e801026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.947+0000 7f7e88200700 1 -- 172.21.15.67:0/3248385014 shutdown_connections 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:27.947+0000 7f7e88200700 1 -- 172.21.15.67:0/3248385014 wait complete. 2024-03-30T02:29:28.384 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 9 is available 2024-03-30T02:29:28.385 INFO:teuthology.orchestra.run.smithi067.stdout:Generating a dashboard self-signed certificate... 2024-03-30T02:29:29.367 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:28 smithi067 ceph-mon[37808]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-30T02:29:29.367 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:28 smithi067 ceph-mon[37808]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-30T02:29:29.367 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:29.536 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-30T02:29:29.536 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.783+0000 7fa966e00700 1 Processor -- start 2024-03-30T02:29:29.536 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.783+0000 7fa966e00700 1 -- start start 2024-03-30T02:29:29.536 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.783+0000 7fa966e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa960106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:29.536 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.783+0000 7fa966e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa960106d20 con 0x7fa960106330 2024-03-30T02:29:29.536 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.784+0000 7fa965a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa960106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.784+0000 7fa965a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa960106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51656/0 (socket says 172.21.15.67:51656) 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.784+0000 7fa965a00700 1 -- 172.21.15.67:0/2419665309 learned_addr learned my addr 172.21.15.67:0/2419665309 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.784+0000 7fa965a00700 1 -- 172.21.15.67:0/2419665309 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa960107530 con 0x7fa960106330 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.785+0000 7fa965a00700 1 --2- 172.21.15.67:0/2419665309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa960106750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe16f5f6b284552b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.785+0000 7fa95fe00700 1 -- 172.21.15.67:0/2419665309 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa954004160 con 0x7fa960106330 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.785+0000 7fa95fe00700 1 -- 172.21.15.67:0/2419665309 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9540042c0 con 0x7fa960106330 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.785+0000 7fa95fe00700 1 -- 172.21.15.67:0/2419665309 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9540119b0 con 0x7fa960106330 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.785+0000 7fa966e00700 1 -- 172.21.15.67:0/2419665309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 msgr2=0x7fa960106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.785+0000 7fa966e00700 1 --2- 172.21.15.67:0/2419665309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa960106750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa954009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.786+0000 7fa966e00700 1 -- 172.21.15.67:0/2419665309 shutdown_connections 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.786+0000 7fa966e00700 1 --2- 172.21.15.67:0/2419665309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa960106750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.786+0000 7fa966e00700 1 -- 172.21.15.67:0/2419665309 >> 172.21.15.67:0/2419665309 conn(0x7fa9601018b0 msgr2=0x7fa960103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:29.537 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.786+0000 7fa966e00700 1 -- 172.21.15.67:0/2419665309 shutdown_connections 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.786+0000 7fa966e00700 1 -- 172.21.15.67:0/2419665309 wait complete. 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.787+0000 7fa966e00700 1 Processor -- start 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.787+0000 7fa966e00700 1 -- start start 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.787+0000 7fa966e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa96019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.787+0000 7fa966e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96019c800 con 0x7fa960106330 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.788+0000 7fa965a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa96019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.788+0000 7fa965a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa96019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51672/0 (socket says 172.21.15.67:51672) 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.788+0000 7fa965a00700 1 -- 172.21.15.67:0/4031809709 learned_addr learned my addr 172.21.15.67:0/4031809709 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.788+0000 7fa965a00700 1 -- 172.21.15.67:0/4031809709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa954009550 con 0x7fa960106330 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.788+0000 7fa965a00700 1 --2- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa96019c2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa954003e60 tx=0x7fa954011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.788+0000 7fa95e000700 1 -- 172.21.15.67:0/4031809709 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa954003a60 con 0x7fa960106330 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.788+0000 7fa95e000700 1 -- 172.21.15.67:0/4031809709 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa954003bc0 con 0x7fa960106330 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.788+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96019ca00 con 0x7fa960106330 2024-03-30T02:29:29.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.788+0000 7fa95e000700 1 -- 172.21.15.67:0/4031809709 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9540196c0 con 0x7fa960106330 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.789+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96019cea0 con 0x7fa960106330 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.790+0000 7fa95e000700 1 -- 172.21.15.67:0/4031809709 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 11) v1 ==== 45440+0+0 (secure 0 0 0) 0x7fa95401ab60 con 0x7fa960106330 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.790+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96010a760 con 0x7fa960106330 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.790+0000 7fa95e000700 1 --2- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa948038620 0x7fa94803aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.790+0000 7fa95e000700 1 -- 172.21.15.67:0/4031809709 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa95404c8f0 con 0x7fa960106330 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.790+0000 7fa965000700 1 --2- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa948038620 0x7fa94803aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.791+0000 7fa965000700 1 --2- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa948038620 0x7fa94803aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa94c006fd0 tx=0x7fa94c006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:28.800+0000 7fa95e000700 1 -- 172.21.15.67:0/4031809709 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa954010770 con 0x7fa960106330 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.022+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fa960102c10 con 0x7fa948038620 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.162+0000 7fa95e000700 1 -- 172.21.15.67:0/4031809709 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fa960102c10 con 0x7fa948038620 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa948038620 msgr2=0x7fa94803aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 --2- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa948038620 0x7fa94803aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa94c006fd0 tx=0x7fa94c006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:29.539 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 msgr2=0x7fa96019c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:29.540 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 --2- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa96019c2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa954003e60 tx=0x7fa954011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:29.540 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 shutdown_connections 2024-03-30T02:29:29.540 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 --2- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa948038620 0x7fa94803aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:29.540 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 --2- 172.21.15.67:0/4031809709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa960106330 0x7fa96019c2c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:29.540 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 >> 172.21.15.67:0/4031809709 conn(0x7fa9601018b0 msgr2=0x7fa960102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:29.540 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 shutdown_connections 2024-03-30T02:29:29.540 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.165+0000 7fa966e00700 1 -- 172.21.15.67:0/4031809709 wait complete. 2024-03-30T02:29:29.540 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial admin user... 2024-03-30T02:29:30.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:29 smithi067 ceph-mon[37808]: [30/Mar/2024:02:29:28] ENGINE Bus STARTING 2024-03-30T02:29:30.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:29 smithi067 ceph-mon[37808]: [30/Mar/2024:02:29:28] ENGINE Serving on https://172.21.15.67:7150 2024-03-30T02:29:30.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:29 smithi067 ceph-mon[37808]: [30/Mar/2024:02:29:28] ENGINE Serving on http://172.21.15.67:8765 2024-03-30T02:29:30.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:29 smithi067 ceph-mon[37808]: [30/Mar/2024:02:29:28] ENGINE Bus STARTED 2024-03-30T02:29:30.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:29 smithi067 ceph-mon[37808]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:30.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:30.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:30.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:29 smithi067 ceph-mon[37808]: mgrmap e12: x(active, since 2s) 2024-03-30T02:29:30.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$OCEEZziv6z49YgcfVX4BiuQWdDL5kbQVYRi6JvwlclJ6ZwNwAMKZW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711765770, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.966+0000 7f4682600700 1 Processor -- start 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.966+0000 7f4682600700 1 -- start start 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.967+0000 7f4682600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.967+0000 7f4682600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c07a8a0 con 0x7f467c07be70 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.967+0000 7f467b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.967+0000 7f467b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51678/0 (socket says 172.21.15.67:51678) 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.967+0000 7f467b400700 1 -- 172.21.15.67:0/1164972956 learned_addr learned my addr 172.21.15.67:0/1164972956 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.967+0000 7f467b400700 1 -- 172.21.15.67:0/1164972956 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c07a9e0 con 0x7f467c07be70 2024-03-30T02:29:30.989 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.968+0000 7f467b400700 1 --2- 172.21.15.67:0/1164972956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c07a2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f466c0098a0 tx=0x7f466c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15902dfd0373e73 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.968+0000 7f467a000700 1 -- 172.21.15.67:0/1164972956 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f466c004160 con 0x7f467c07be70 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.968+0000 7f467a000700 1 -- 172.21.15.67:0/1164972956 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f466c0042c0 con 0x7f467c07be70 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.968+0000 7f467a000700 1 -- 172.21.15.67:0/1164972956 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f466c0119b0 con 0x7f467c07be70 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.969+0000 7f4682600700 1 -- 172.21.15.67:0/1164972956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 msgr2=0x7f467c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.969+0000 7f4682600700 1 --2- 172.21.15.67:0/1164972956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c07a2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f466c0098a0 tx=0x7f466c009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.969+0000 7f4682600700 1 -- 172.21.15.67:0/1164972956 shutdown_connections 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.969+0000 7f4682600700 1 --2- 172.21.15.67:0/1164972956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c07a2d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.969+0000 7f4682600700 1 -- 172.21.15.67:0/1164972956 >> 172.21.15.67:0/1164972956 conn(0x7f467c101b40 msgr2=0x7f467c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.969+0000 7f4682600700 1 -- 172.21.15.67:0/1164972956 shutdown_connections 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.969+0000 7f4682600700 1 -- 172.21.15.67:0/1164972956 wait complete. 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.970+0000 7f4682600700 1 Processor -- start 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.970+0000 7f4682600700 1 -- start start 2024-03-30T02:29:30.990 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.970+0000 7f4682600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.970+0000 7f4682600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c19ca00 con 0x7f467c07be70 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.971+0000 7f467b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.971+0000 7f467b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51690/0 (socket says 172.21.15.67:51690) 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.971+0000 7f467b400700 1 -- 172.21.15.67:0/3173214216 learned_addr learned my addr 172.21.15.67:0/3173214216 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.971+0000 7f467b400700 1 -- 172.21.15.67:0/3173214216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c009550 con 0x7f467c07be70 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.971+0000 7f467b400700 1 --2- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c19c4c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f466c011980 tx=0x7f466c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.972+0000 7f4673e00700 1 -- 172.21.15.67:0/3173214216 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f466c003af0 con 0x7f467c07be70 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.972+0000 7f4673e00700 1 -- 172.21.15.67:0/3173214216 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f466c003c50 con 0x7f467c07be70 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.972+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c19cc00 con 0x7f467c07be70 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.972+0000 7f4673e00700 1 -- 172.21.15.67:0/3173214216 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f466c019760 con 0x7f467c07be70 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.972+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c19d0a0 con 0x7f467c07be70 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.973+0000 7f4673e00700 1 -- 172.21.15.67:0/3173214216 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f466c019950 con 0x7f467c07be70 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.973+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c10c9c0 con 0x7f467c07be70 2024-03-30T02:29:30.991 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.973+0000 7f4673e00700 1 --2- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46600387a0 0x7f466003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.974+0000 7f4673e00700 1 -- 172.21.15.67:0/3173214216 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f466c04cb50 con 0x7f467c07be70 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.974+0000 7f467aa00700 1 --2- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46600387a0 0x7f466003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.974+0000 7f467aa00700 1 --2- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46600387a0 0x7f466003ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4668006fd0 tx=0x7f4668006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:29.982+0000 7f4673e00700 1 -- 172.21.15.67:0/3173214216 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f466c018db0 con 0x7f467c07be70 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.248+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f467c062200 con 0x7f46600387a0 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.500+0000 7f4673e00700 1 -- 172.21.15.67:0/3173214216 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f467c062200 con 0x7f46600387a0 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.502+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46600387a0 msgr2=0x7f466003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.502+0000 7f4682600700 1 --2- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46600387a0 0x7f466003ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4668006fd0 tx=0x7f4668006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.502+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 msgr2=0x7f467c19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.502+0000 7f4682600700 1 --2- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c19c4c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f466c011980 tx=0x7f466c011a60 comp rx=0 tx=0).stop 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.502+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 shutdown_connections 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.502+0000 7f4682600700 1 --2- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46600387a0 0x7f466003ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.502+0000 7f4682600700 1 --2- 172.21.15.67:0/3173214216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f467c07be70 0x7f467c19c4c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:30.992 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.502+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 >> 172.21.15.67:0/3173214216 conn(0x7f467c101b40 msgr2=0x7f467c102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:30.993 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.502+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 shutdown_connections 2024-03-30T02:29:30.993 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:30.503+0000 7f4682600700 1 -- 172.21.15.67:0/3173214216 wait complete. 2024-03-30T02:29:30.993 INFO:teuthology.orchestra.run.smithi067.stdout:Fetching dashboard port number... 2024-03-30T02:29:31.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:31 smithi067 ceph-mon[37808]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:31.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:31.971 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 8443 2024-03-30T02:29:31.971 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.400+0000 7f3f41a00700 1 Processor -- start 2024-03-30T02:29:31.971 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.400+0000 7f3f41a00700 1 -- start start 2024-03-30T02:29:31.971 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.401+0000 7f3f41a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c105210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:31.971 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.401+0000 7f3f41a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c079e20 con 0x7f3f3c104df0 2024-03-30T02:29:31.971 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.401+0000 7f3f3aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.401+0000 7f3f3aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c105210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51706/0 (socket says 172.21.15.67:51706) 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.401+0000 7f3f3aa00700 1 -- 172.21.15.67:0/3977979346 learned_addr learned my addr 172.21.15.67:0/3977979346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.402+0000 7f3f3aa00700 1 -- 172.21.15.67:0/3977979346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c105750 con 0x7f3f3c104df0 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.402+0000 7f3f3aa00700 1 --2- 172.21.15.67:0/3977979346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c105210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0098a0 tx=0x7f3f2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960932cb93a936c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.402+0000 7f3f3a000700 1 -- 172.21.15.67:0/3977979346 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f2c004160 con 0x7f3f3c104df0 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.402+0000 7f3f3a000700 1 -- 172.21.15.67:0/3977979346 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f2c0042c0 con 0x7f3f3c104df0 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.402+0000 7f3f3a000700 1 -- 172.21.15.67:0/3977979346 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f2c0119b0 con 0x7f3f3c104df0 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.403+0000 7f3f41a00700 1 -- 172.21.15.67:0/3977979346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 msgr2=0x7f3f3c105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.403+0000 7f3f41a00700 1 --2- 172.21.15.67:0/3977979346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c105210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0098a0 tx=0x7f3f2c009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.403+0000 7f3f41a00700 1 -- 172.21.15.67:0/3977979346 shutdown_connections 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.403+0000 7f3f41a00700 1 --2- 172.21.15.67:0/3977979346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c105210 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.403+0000 7f3f41a00700 1 -- 172.21.15.67:0/3977979346 >> 172.21.15.67:0/3977979346 conn(0x7f3f3c1009d0 msgr2=0x7f3f3c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:31.972 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.403+0000 7f3f41a00700 1 -- 172.21.15.67:0/3977979346 shutdown_connections 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.403+0000 7f3f41a00700 1 -- 172.21.15.67:0/3977979346 wait complete. 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.404+0000 7f3f41a00700 1 Processor -- start 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.405+0000 7f3f41a00700 1 -- start start 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.405+0000 7f3f41a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c19c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.405+0000 7f3f41a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c19c9c0 con 0x7f3f3c104df0 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.405+0000 7f3f3aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c19c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.405+0000 7f3f3aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c19c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51716/0 (socket says 172.21.15.67:51716) 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.405+0000 7f3f3aa00700 1 -- 172.21.15.67:0/2475407835 learned_addr learned my addr 172.21.15.67:0/2475407835 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.406+0000 7f3f3aa00700 1 -- 172.21.15.67:0/2475407835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c009550 con 0x7f3f3c104df0 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.406+0000 7f3f3aa00700 1 --2- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c19c480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c011670 tx=0x7f3f2c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.406+0000 7f3f33400700 1 -- 172.21.15.67:0/2475407835 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f2c003b20 con 0x7f3f3c104df0 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.406+0000 7f3f33400700 1 -- 172.21.15.67:0/2475407835 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f2c003c80 con 0x7f3f3c104df0 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.406+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c19cbc0 con 0x7f3f3c104df0 2024-03-30T02:29:31.973 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.406+0000 7f3f33400700 1 -- 172.21.15.67:0/2475407835 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f2c019770 con 0x7f3f3c104df0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.406+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c19d060 con 0x7f3f3c104df0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.407+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c196210 con 0x7f3f3c104df0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.407+0000 7f3f33400700 1 -- 172.21.15.67:0/2475407835 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f3f2c01ab60 con 0x7f3f3c104df0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.408+0000 7f3f33400700 1 --2- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3f200387a0 0x7f3f2003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.408+0000 7f3f33400700 1 -- 172.21.15.67:0/2475407835 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3f2c04cb60 con 0x7f3f3c104df0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.408+0000 7f3f33e00700 1 --2- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3f200387a0 0x7f3f2003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.409+0000 7f3f33e00700 1 --2- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3f200387a0 0x7f3f2003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f24006fd0 tx=0x7f3f24006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.413+0000 7f3f33400700 1 -- 172.21.15.67:0/2475407835 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3f2c01a430 con 0x7f3f3c104df0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.625+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f3f3c106600 con 0x7f3f3c104df0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.626+0000 7f3f33400700 1 -- 172.21.15.67:0/2475407835 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f3f2c02db40 con 0x7f3f3c104df0 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.628+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3f200387a0 msgr2=0x7f3f2003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.628+0000 7f3f41a00700 1 --2- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3f200387a0 0x7f3f2003ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f24006fd0 tx=0x7f3f24006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.628+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 msgr2=0x7f3f3c19c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:31.974 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.628+0000 7f3f41a00700 1 --2- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c19c480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c011670 tx=0x7f3f2c011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:31.975 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.629+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 shutdown_connections 2024-03-30T02:29:31.975 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.629+0000 7f3f41a00700 1 --2- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3f200387a0 0x7f3f2003ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:31.975 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.629+0000 7f3f41a00700 1 --2- 172.21.15.67:0/2475407835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f3c104df0 0x7f3f3c19c480 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:31.975 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.629+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 >> 172.21.15.67:0/2475407835 conn(0x7f3f3c1009d0 msgr2=0x7f3f3c193350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:31.975 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.629+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 shutdown_connections 2024-03-30T02:29:31.975 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:31.629+0000 7f3f41a00700 1 -- 172.21.15.67:0/2475407835 wait complete. 2024-03-30T02:29:31.989 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-30T02:29:31.989 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-03-30T02:29:32.005 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-30T02:29:32.005 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-03-30T02:29:32.005 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-03-30T02:29:32.005 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-30T02:29:32.008 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph Dashboard is now available at: 2024-03-30T02:29:32.008 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:32.008 INFO:teuthology.orchestra.run.smithi067.stdout: URL: https://smithi067.front.sepia.ceph.com:8443/ 2024-03-30T02:29:32.008 INFO:teuthology.orchestra.run.smithi067.stdout: User: admin 2024-03-30T02:29:32.009 INFO:teuthology.orchestra.run.smithi067.stdout: Password: mexcpd8ysm 2024-03-30T02:29:32.009 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:32.009 INFO:teuthology.orchestra.run.smithi067.stdout:Saving cluster configuration to /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config directory 2024-03-30T02:29:32.010 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling autotune for osd_memory_target 2024-03-30T02:29:32.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:32 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2475407835' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-30T02:29:33.044 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.435+0000 7f2e2c800700 1 Processor -- start 2024-03-30T02:29:33.044 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.435+0000 7f2e2c800700 1 -- start start 2024-03-30T02:29:33.045 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.436+0000 7f2e2c800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:33.045 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.436+0000 7f2e2c800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2407a8e0 con 0x7f2e2407beb0 2024-03-30T02:29:33.045 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.436+0000 7f2e29e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:33.045 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.436+0000 7f2e29e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51720/0 (socket says 172.21.15.67:51720) 2024-03-30T02:29:33.045 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.436+0000 7f2e29e00700 1 -- 172.21.15.67:0/3060508317 learned_addr learned my addr 172.21.15.67:0/3060508317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:33.045 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.436+0000 7f2e29e00700 1 -- 172.21.15.67:0/3060508317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2407aa20 con 0x7f2e2407beb0 2024-03-30T02:29:33.045 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.437+0000 7f2e29e00700 1 --2- 172.21.15.67:0/3060508317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2407a310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2e180098a0 tx=0x7f2e18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae040bcb5f374b03 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:33.045 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.437+0000 7f2e28a00700 1 -- 172.21.15.67:0/3060508317 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e18004160 con 0x7f2e2407beb0 2024-03-30T02:29:33.045 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.437+0000 7f2e28a00700 1 -- 172.21.15.67:0/3060508317 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e180042c0 con 0x7f2e2407beb0 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.437+0000 7f2e28a00700 1 -- 172.21.15.67:0/3060508317 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e180119b0 con 0x7f2e2407beb0 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.437+0000 7f2e2c800700 1 -- 172.21.15.67:0/3060508317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 msgr2=0x7f2e2407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.437+0000 7f2e2c800700 1 --2- 172.21.15.67:0/3060508317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2407a310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2e180098a0 tx=0x7f2e18009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.438+0000 7f2e2c800700 1 -- 172.21.15.67:0/3060508317 shutdown_connections 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.438+0000 7f2e2c800700 1 --2- 172.21.15.67:0/3060508317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2407a310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.438+0000 7f2e2c800700 1 -- 172.21.15.67:0/3060508317 >> 172.21.15.67:0/3060508317 conn(0x7f2e24101ae0 msgr2=0x7f2e24103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.438+0000 7f2e2c800700 1 -- 172.21.15.67:0/3060508317 shutdown_connections 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.438+0000 7f2e2c800700 1 -- 172.21.15.67:0/3060508317 wait complete. 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.439+0000 7f2e2c800700 1 Processor -- start 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.439+0000 7f2e2c800700 1 -- start start 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.439+0000 7f2e2c800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:33.046 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.439+0000 7f2e2c800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2419ca20 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.439+0000 7f2e29e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.439+0000 7f2e29e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2419c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51734/0 (socket says 172.21.15.67:51734) 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.439+0000 7f2e29e00700 1 -- 172.21.15.67:0/3464501137 learned_addr learned my addr 172.21.15.67:0/3464501137 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.440+0000 7f2e29e00700 1 -- 172.21.15.67:0/3464501137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18009550 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.440+0000 7f2e29e00700 1 --2- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2419c4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e18011670 tx=0x7f2e18011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.440+0000 7f2e1ea00700 1 -- 172.21.15.67:0/3464501137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e18003af0 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.440+0000 7f2e1ea00700 1 -- 172.21.15.67:0/3464501137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e18003c50 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.440+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2419cc20 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.440+0000 7f2e1ea00700 1 -- 172.21.15.67:0/3464501137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e18024820 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.440+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2419d0c0 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.441+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2410c900 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.441+0000 7f2e1ea00700 1 -- 172.21.15.67:0/3464501137 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f2e1801ab60 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.442+0000 7f2e1ea00700 1 --2- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2e0c0387a0 0x7f2e0c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.442+0000 7f2e1ea00700 1 -- 172.21.15.67:0/3464501137 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2e1804c8d0 con 0x7f2e2407beb0 2024-03-30T02:29:33.047 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.442+0000 7f2e29400700 1 --2- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2e0c0387a0 0x7f2e0c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.443+0000 7f2e29400700 1 --2- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2e0c0387a0 0x7f2e0c03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e14006fd0 tx=0x7f2e14006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.447+0000 7f2e1ea00700 1 -- 172.21.15.67:0/3464501137 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2e18018760 con 0x7f2e2407beb0 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.661+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f2e2410cb40 con 0x7f2e2407beb0 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.661+0000 7f2e1ea00700 1 -- 172.21.15.67:0/3464501137 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2e1800b3e0 con 0x7f2e2407beb0 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2e0c0387a0 msgr2=0x7f2e0c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 --2- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2e0c0387a0 0x7f2e0c03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e14006fd0 tx=0x7f2e14006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 msgr2=0x7f2e2419c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 --2- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2419c4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e18011670 tx=0x7f2e18011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 shutdown_connections 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 --2- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2e0c0387a0 0x7f2e0c03ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 --2- 172.21.15.67:0/3464501137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2e2407beb0 0x7f2e2419c4e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 >> 172.21.15.67:0/3464501137 conn(0x7f2e24101ae0 msgr2=0x7f2e24102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 shutdown_connections 2024-03-30T02:29:33.048 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:32.664+0000 7f2e2c800700 1 -- 172.21.15.67:0/3464501137 wait complete. 2024-03-30T02:29:34.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.466+0000 7f3502400700 1 Processor -- start 2024-03-30T02:29:34.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.466+0000 7f3502400700 1 -- start start 2024-03-30T02:29:34.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.466+0000 7f3502400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:34.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.466+0000 7f3502400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc106ef0 con 0x7f34fc106500 2024-03-30T02:29:34.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.467+0000 7f34fb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:34.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.467+0000 7f34fb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51746/0 (socket says 172.21.15.67:51746) 2024-03-30T02:29:34.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.467+0000 7f34fb400700 1 -- 172.21.15.67:0/536718226 learned_addr learned my addr 172.21.15.67:0/536718226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:34.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.467+0000 7f34fb400700 1 -- 172.21.15.67:0/536718226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc107700 con 0x7f34fc106500 2024-03-30T02:29:34.192 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.467+0000 7f34fb400700 1 --2- 172.21.15.67:0/536718226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc106920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0098a0 tx=0x7f34ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989b1dee7d2a1a85 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.467+0000 7f34faa00700 1 -- 172.21.15.67:0/536718226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ec004160 con 0x7f34fc106500 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.467+0000 7f34faa00700 1 -- 172.21.15.67:0/536718226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ec0042c0 con 0x7f34fc106500 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.467+0000 7f34faa00700 1 -- 172.21.15.67:0/536718226 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ec0119b0 con 0x7f34fc106500 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.468+0000 7f3502400700 1 -- 172.21.15.67:0/536718226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 msgr2=0x7f34fc106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.468+0000 7f3502400700 1 --2- 172.21.15.67:0/536718226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc106920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0098a0 tx=0x7f34ec009bb0 comp rx=0 tx=0).stop 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.468+0000 7f3502400700 1 -- 172.21.15.67:0/536718226 shutdown_connections 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.468+0000 7f3502400700 1 --2- 172.21.15.67:0/536718226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc106920 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.468+0000 7f3502400700 1 -- 172.21.15.67:0/536718226 >> 172.21.15.67:0/536718226 conn(0x7f34fc101ae0 msgr2=0x7f34fc103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.468+0000 7f3502400700 1 -- 172.21.15.67:0/536718226 shutdown_connections 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.468+0000 7f3502400700 1 -- 172.21.15.67:0/536718226 wait complete. 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.469+0000 7f3502400700 1 Processor -- start 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.469+0000 7f3502400700 1 -- start start 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.470+0000 7f3502400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc07bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:34.193 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.470+0000 7f3502400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc07c050 con 0x7f34fc106500 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.470+0000 7f34fb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc07bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.470+0000 7f34fb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc07bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51760/0 (socket says 172.21.15.67:51760) 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.470+0000 7f34fb400700 1 -- 172.21.15.67:0/1135210202 learned_addr learned my addr 172.21.15.67:0/1135210202 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.470+0000 7f34fb400700 1 -- 172.21.15.67:0/1135210202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec009550 con 0x7f34fc106500 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.471+0000 7f34fb400700 1 --2- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc07bb10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34ec011670 tx=0x7f34ec011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.471+0000 7f34f8c00700 1 -- 172.21.15.67:0/1135210202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ec003af0 con 0x7f34fc106500 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.471+0000 7f34f8c00700 1 -- 172.21.15.67:0/1135210202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ec003c50 con 0x7f34fc106500 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.471+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc07a290 con 0x7f34fc106500 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.471+0000 7f34f8c00700 1 -- 172.21.15.67:0/1135210202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ec024820 con 0x7f34fc106500 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.471+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc07a730 con 0x7f34fc106500 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.472+0000 7f34f8c00700 1 -- 172.21.15.67:0/1135210202 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) v1 ==== 45546+0+0 (secure 0 0 0) 0x7f34ec01ab60 con 0x7f34fc106500 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.472+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc10a930 con 0x7f34fc106500 2024-03-30T02:29:34.194 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.472+0000 7f34f8c00700 1 --2- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34e00387a0 0x7f34e003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.472+0000 7f34f8c00700 1 -- 172.21.15.67:0/1135210202 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f34ec04c8d0 con 0x7f34fc106500 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.473+0000 7f34f3e00700 1 --2- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34e00387a0 0x7f34e003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.473+0000 7f34f3e00700 1 --2- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34e00387a0 0x7f34e003ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34e4006fd0 tx=0x7f34e4006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.478+0000 7f34f8c00700 1 -- 172.21.15.67:0/1135210202 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f34ec018760 con 0x7f34fc106500 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.502+0000 7f34f8c00700 1 -- 172.21.15.67:0/1135210202 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f34ec04ecd0 con 0x7f34fc106500 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.803+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f34fc10ab70 con 0x7f34fc106500 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.805+0000 7f34f8c00700 1 -- 172.21.15.67:0/1135210202 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f34ec01a950 con 0x7f34fc106500 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34e00387a0 msgr2=0x7f34e003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 --2- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34e00387a0 0x7f34e003ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f34e4006fd0 tx=0x7f34e4006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 msgr2=0x7f34fc07bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 --2- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc07bb10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f34ec011670 tx=0x7f34ec011a30 comp rx=0 tx=0).stop 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 shutdown_connections 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 --2- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34e00387a0 0x7f34e003ac60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:34.195 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 --2- 172.21.15.67:0/1135210202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34fc106500 0x7f34fc07bb10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 >> 172.21.15.67:0/1135210202 conn(0x7f34fc101ae0 msgr2=0x7f34fc102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 shutdown_connections 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr 2024-03-30T02:29:33.808+0000 7f3502400700 1 -- 172.21.15.67:0/1135210202 wait complete. 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout:Or, if you are only running a single cluster on this host: 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:34.196 INFO:teuthology.orchestra.run.smithi067.stdout: ceph telemetry on 2024-03-30T02:29:34.197 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:34.197 INFO:teuthology.orchestra.run.smithi067.stdout:For more information see: 2024-03-30T02:29:34.197 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:34.197 INFO:teuthology.orchestra.run.smithi067.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-30T02:29:34.197 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:34.197 INFO:teuthology.orchestra.run.smithi067.stdout:Bootstrap complete. 2024-03-30T02:29:34.254 INFO:tasks.cephadm:Fetching config... 2024-03-30T02:29:34.254 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:29:34.255 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-30T02:29:34.338 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-30T02:29:34.338 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:29:34.338 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-30T02:29:34.421 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-30T02:29:34.421 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:29:34.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/keyring of=/dev/stdout 2024-03-30T02:29:34.535 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-30T02:29:34.536 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:29:34.536 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-30T02:29:34.618 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-30T02:29:34.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKqO/4UK2rfMETcEn/x/hJclnuebIuywr2BaMr4/YZ1n8BGxLmB0wh0KE9o7+Fio9je/dNqkp3Rfh8wxSWWUFsobKBRWeY6j49S2HHv6CVuC299xzW80bazJNuYbUWVaxs+oPel13ZyL1ugw5Cl3a2GD7Tr1WO+xILUlLTt1crsDnAStMqNNEyPtvGeNIjzpRS0GUXxvVB/JXJVaInnHVQ8CoRCrjmevyRYCYIWBMWGUaDhVar846QgiyRjTU/y7SnsHLyt2r1Fo1eg7i2ab9Lcf+GWDEjAkQlamW5CCVKKne0wQEUPWm1dEMdvv3VwowBAbnD4SXLU6/g5Nq/7ncTAcjJvZ3x57oyAqKQKrCmauUKK9Vw3EpO8GtfJOGPvidKGuVdTZDE/w6i0V+RRXqWxj0h/pkMSPj6dkSLq+U3BGmjlntiW6aEdru/+dvtEsfjYILrvWfPA0ROyDRqomX7G2YfcSseq9ji0EsRMnuHeH0D7v9wKITApMrttoot37M= ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-30T02:29:34.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:34 smithi067 ceph-mon[37808]: mgrmap e13: x(active, since 6s) 2024-03-30T02:29:34.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:34 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1135210202' entity='client.admin' 2024-03-30T02:29:34.753 INFO:teuthology.orchestra.run.smithi067.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKqO/4UK2rfMETcEn/x/hJclnuebIuywr2BaMr4/YZ1n8BGxLmB0wh0KE9o7+Fio9je/dNqkp3Rfh8wxSWWUFsobKBRWeY6j49S2HHv6CVuC299xzW80bazJNuYbUWVaxs+oPel13ZyL1ugw5Cl3a2GD7Tr1WO+xILUlLTt1crsDnAStMqNNEyPtvGeNIjzpRS0GUXxvVB/JXJVaInnHVQ8CoRCrjmevyRYCYIWBMWGUaDhVar846QgiyRjTU/y7SnsHLyt2r1Fo1eg7i2ab9Lcf+GWDEjAkQlamW5CCVKKne0wQEUPWm1dEMdvv3VwowBAbnD4SXLU6/g5Nq/7ncTAcjJvZ3x57oyAqKQKrCmauUKK9Vw3EpO8GtfJOGPvidKGuVdTZDE/w6i0V+RRXqWxj0h/pkMSPj6dkSLq+U3BGmjlntiW6aEdru/+dvtEsfjYILrvWfPA0ROyDRqomX7G2YfcSseq9ji0EsRMnuHeH0D7v9wKITApMrttoot37M= ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:29:34.782 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKqO/4UK2rfMETcEn/x/hJclnuebIuywr2BaMr4/YZ1n8BGxLmB0wh0KE9o7+Fio9je/dNqkp3Rfh8wxSWWUFsobKBRWeY6j49S2HHv6CVuC299xzW80bazJNuYbUWVaxs+oPel13ZyL1ugw5Cl3a2GD7Tr1WO+xILUlLTt1crsDnAStMqNNEyPtvGeNIjzpRS0GUXxvVB/JXJVaInnHVQ8CoRCrjmevyRYCYIWBMWGUaDhVar846QgiyRjTU/y7SnsHLyt2r1Fo1eg7i2ab9Lcf+GWDEjAkQlamW5CCVKKne0wQEUPWm1dEMdvv3VwowBAbnD4SXLU6/g5Nq/7ncTAcjJvZ3x57oyAqKQKrCmauUKK9Vw3EpO8GtfJOGPvidKGuVdTZDE/w6i0V+RRXqWxj0h/pkMSPj6dkSLq+U3BGmjlntiW6aEdru/+dvtEsfjYILrvWfPA0ROyDRqomX7G2YfcSseq9ji0EsRMnuHeH0D7v9wKITApMrttoot37M= ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-30T02:29:34.829 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKqO/4UK2rfMETcEn/x/hJclnuebIuywr2BaMr4/YZ1n8BGxLmB0wh0KE9o7+Fio9je/dNqkp3Rfh8wxSWWUFsobKBRWeY6j49S2HHv6CVuC299xzW80bazJNuYbUWVaxs+oPel13ZyL1ugw5Cl3a2GD7Tr1WO+xILUlLTt1crsDnAStMqNNEyPtvGeNIjzpRS0GUXxvVB/JXJVaInnHVQ8CoRCrjmevyRYCYIWBMWGUaDhVar846QgiyRjTU/y7SnsHLyt2r1Fo1eg7i2ab9Lcf+GWDEjAkQlamW5CCVKKne0wQEUPWm1dEMdvv3VwowBAbnD4SXLU6/g5Nq/7ncTAcjJvZ3x57oyAqKQKrCmauUKK9Vw3EpO8GtfJOGPvidKGuVdTZDE/w6i0V+RRXqWxj0h/pkMSPj6dkSLq+U3BGmjlntiW6aEdru/+dvtEsfjYILrvWfPA0ROyDRqomX7G2YfcSseq9ji0EsRMnuHeH0D7v9wKITApMrttoot37M= ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:29:34.842 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKqO/4UK2rfMETcEn/x/hJclnuebIuywr2BaMr4/YZ1n8BGxLmB0wh0KE9o7+Fio9je/dNqkp3Rfh8wxSWWUFsobKBRWeY6j49S2HHv6CVuC299xzW80bazJNuYbUWVaxs+oPel13ZyL1ugw5Cl3a2GD7Tr1WO+xILUlLTt1crsDnAStMqNNEyPtvGeNIjzpRS0GUXxvVB/JXJVaInnHVQ8CoRCrjmevyRYCYIWBMWGUaDhVar846QgiyRjTU/y7SnsHLyt2r1Fo1eg7i2ab9Lcf+GWDEjAkQlamW5CCVKKne0wQEUPWm1dEMdvv3VwowBAbnD4SXLU6/g5Nq/7ncTAcjJvZ3x57oyAqKQKrCmauUKK9Vw3EpO8GtfJOGPvidKGuVdTZDE/w6i0V+RRXqWxj0h/pkMSPj6dkSLq+U3BGmjlntiW6aEdru/+dvtEsfjYILrvWfPA0ROyDRqomX7G2YfcSseq9ji0EsRMnuHeH0D7v9wKITApMrttoot37M= ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-30T02:29:34.889 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKqO/4UK2rfMETcEn/x/hJclnuebIuywr2BaMr4/YZ1n8BGxLmB0wh0KE9o7+Fio9je/dNqkp3Rfh8wxSWWUFsobKBRWeY6j49S2HHv6CVuC299xzW80bazJNuYbUWVaxs+oPel13ZyL1ugw5Cl3a2GD7Tr1WO+xILUlLTt1crsDnAStMqNNEyPtvGeNIjzpRS0GUXxvVB/JXJVaInnHVQ8CoRCrjmevyRYCYIWBMWGUaDhVar846QgiyRjTU/y7SnsHLyt2r1Fo1eg7i2ab9Lcf+GWDEjAkQlamW5CCVKKne0wQEUPWm1dEMdvv3VwowBAbnD4SXLU6/g5Nq/7ncTAcjJvZ3x57oyAqKQKrCmauUKK9Vw3EpO8GtfJOGPvidKGuVdTZDE/w6i0V+RRXqWxj0h/pkMSPj6dkSLq+U3BGmjlntiW6aEdru/+dvtEsfjYILrvWfPA0ROyDRqomX7G2YfcSseq9ji0EsRMnuHeH0D7v9wKITApMrttoot37M= ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:29:34.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-30T02:29:35.274 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:29:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.686+0000 7fc76c400700 1 -- 172.21.15.67:0/691636498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 msgr2=0x7fc764073200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.686+0000 7fc76c400700 1 --2- 172.21.15.67:0/691636498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 0x7fc764073200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc758009910 tx=0x7fc758009c20 comp rx=0 tx=0).stop 2024-03-30T02:29:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.686+0000 7fc76c400700 1 -- 172.21.15.67:0/691636498 shutdown_connections 2024-03-30T02:29:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.686+0000 7fc76c400700 1 --2- 172.21.15.67:0/691636498 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 0x7fc764073200 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.686+0000 7fc76c400700 1 -- 172.21.15.67:0/691636498 >> 172.21.15.67:0/691636498 conn(0x7fc7640fbcd0 msgr2=0x7fc7640fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.686+0000 7fc76c400700 1 -- 172.21.15.67:0/691636498 shutdown_connections 2024-03-30T02:29:36.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.686+0000 7fc76c400700 1 -- 172.21.15.67:0/691636498 wait complete. 2024-03-30T02:29:36.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.687+0000 7fc76c400700 1 Processor -- start 2024-03-30T02:29:36.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.687+0000 7fc76c400700 1 -- start start 2024-03-30T02:29:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.687+0000 7fc76c400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 0x7fc764197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.687+0000 7fc76c400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc764198320 con 0x7fc764074da0 2024-03-30T02:29:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.688+0000 7fc769a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 0x7fc764197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.688+0000 7fc769a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 0x7fc764197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42774/0 (socket says 172.21.15.67:42774) 2024-03-30T02:29:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.688+0000 7fc769a00700 1 -- 172.21.15.67:0/3865728406 learned_addr learned my addr 172.21.15.67:0/3865728406 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.688+0000 7fc769a00700 1 -- 172.21.15.67:0/3865728406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7580095f0 con 0x7fc764074da0 2024-03-30T02:29:36.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.688+0000 7fc769a00700 1 --2- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 0x7fc764197de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc758005950 tx=0x7fc7580057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:36.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.689+0000 7fc762000700 1 -- 172.21.15.67:0/3865728406 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc75801d070 con 0x7fc764074da0 2024-03-30T02:29:36.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.689+0000 7fc762000700 1 -- 172.21.15.67:0/3865728406 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc75800bc10 con 0x7fc764074da0 2024-03-30T02:29:36.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.689+0000 7fc762000700 1 -- 172.21.15.67:0/3865728406 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc758003a10 con 0x7fc764074da0 2024-03-30T02:29:36.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.689+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc764198520 con 0x7fc764074da0 2024-03-30T02:29:36.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.689+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7641989c0 con 0x7fc764074da0 2024-03-30T02:29:36.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.690+0000 7fc762000700 1 -- 172.21.15.67:0/3865728406 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fc758004390 con 0x7fc764074da0 2024-03-30T02:29:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.690+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc764191660 con 0x7fc764074da0 2024-03-30T02:29:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.691+0000 7fc762000700 1 --2- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc74c0387f0 0x7fc74c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.691+0000 7fc762000700 1 -- 172.21.15.67:0/3865728406 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc75804c370 con 0x7fc764074da0 2024-03-30T02:29:36.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.691+0000 7fc769000700 1 --2- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc74c0387f0 0x7fc74c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:36.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.691+0000 7fc769000700 1 --2- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc74c0387f0 0x7fc74c03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc750006fd0 tx=0x7fc750006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:36.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.696+0000 7fc762000700 1 -- 172.21.15.67:0/3865728406 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc75802a3b0 con 0x7fc764074da0 2024-03-30T02:29:36.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.915+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fc764108940 con 0x7fc764074da0 2024-03-30T02:29:36.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.919+0000 7fc762000700 1 -- 172.21.15.67:0/3865728406 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc758026030 con 0x7fc764074da0 2024-03-30T02:29:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc74c0387f0 msgr2=0x7fc74c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 --2- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc74c0387f0 0x7fc74c03acb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc750006fd0 tx=0x7fc750006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:36.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 msgr2=0x7fc764197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:36.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 --2- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 0x7fc764197de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc758005950 tx=0x7fc7580057c0 comp rx=0 tx=0).stop 2024-03-30T02:29:36.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 shutdown_connections 2024-03-30T02:29:36.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 --2- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc74c0387f0 0x7fc74c03acb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:36.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 --2- 172.21.15.67:0/3865728406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc764074da0 0x7fc764197de0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:36.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 >> 172.21.15.67:0/3865728406 conn(0x7fc7640fbcd0 msgr2=0x7fc7640fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:36.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 shutdown_connections 2024-03-30T02:29:36.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:36.922+0000 7fc76c400700 1 -- 172.21.15.67:0/3865728406 wait complete. 2024-03-30T02:29:37.297 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-30T02:29:37.298 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-30T02:29:37.662 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:29:37.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:37.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:37.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:29:37.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:37.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:37.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:37.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:37 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3865728406' entity='client.admin' 2024-03-30T02:29:37.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:37.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:38.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.947+0000 7fcec7e00700 1 -- 172.21.15.67:0/2038458543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 msgr2=0x7fcec81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.947+0000 7fcec7e00700 1 --2- 172.21.15.67:0/2038458543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 0x7fcec81009a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009910 tx=0x7fceb8009c20 comp rx=0 tx=0).stop 2024-03-30T02:29:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.947+0000 7fcec7e00700 1 -- 172.21.15.67:0/2038458543 shutdown_connections 2024-03-30T02:29:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.947+0000 7fcec7e00700 1 --2- 172.21.15.67:0/2038458543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 0x7fcec81009a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.947+0000 7fcec7e00700 1 -- 172.21.15.67:0/2038458543 >> 172.21.15.67:0/2038458543 conn(0x7fcec80fbb20 msgr2=0x7fcec80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.947+0000 7fcec7e00700 1 -- 172.21.15.67:0/2038458543 shutdown_connections 2024-03-30T02:29:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.947+0000 7fcec7e00700 1 -- 172.21.15.67:0/2038458543 wait complete. 2024-03-30T02:29:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.948+0000 7fcec7e00700 1 Processor -- start 2024-03-30T02:29:38.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.948+0000 7fcec7e00700 1 -- start start 2024-03-30T02:29:38.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.948+0000 7fcec7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 0x7fcec8074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:38.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.948+0000 7fcec7e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec8074f60 con 0x7fcec8100580 2024-03-30T02:29:38.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.949+0000 7fcec6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 0x7fcec8074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.949+0000 7fcec6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 0x7fcec8074a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42806/0 (socket says 172.21.15.67:42806) 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.949+0000 7fcec6a00700 1 -- 172.21.15.67:0/2870631890 learned_addr learned my addr 172.21.15.67:0/2870631890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.949+0000 7fcec6a00700 1 -- 172.21.15.67:0/2870631890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb80095f0 con 0x7fcec8100580 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.949+0000 7fcec6a00700 1 --2- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 0x7fcec8074a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fceb8005950 tx=0x7fceb80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.950+0000 7fcecca00700 1 -- 172.21.15.67:0/2870631890 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fceb801d070 con 0x7fcec8100580 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.950+0000 7fcecca00700 1 -- 172.21.15.67:0/2870631890 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fceb800bc10 con 0x7fcec8100580 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.950+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec80731a0 con 0x7fcec8100580 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.950+0000 7fcecca00700 1 -- 172.21.15.67:0/2870631890 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fceb8003a10 con 0x7fcec8100580 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.950+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec8073640 con 0x7fcec8100580 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.951+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec8106bd0 con 0x7fcec8100580 2024-03-30T02:29:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.951+0000 7fcecca00700 1 -- 172.21.15.67:0/2870631890 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fceb8004390 con 0x7fcec8100580 2024-03-30T02:29:38.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.952+0000 7fcecca00700 1 --2- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fceac0387a0 0x7fceac03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:38.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.952+0000 7fcecca00700 1 -- 172.21.15.67:0/2870631890 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fceb804c4b0 con 0x7fcec8100580 2024-03-30T02:29:38.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.952+0000 7fcebfe00700 1 --2- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fceac0387a0 0x7fceac03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:38.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.952+0000 7fcebfe00700 1 --2- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fceac0387a0 0x7fceac03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fceb0006fd0 tx=0x7fceb0006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:38.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:38.957+0000 7fcecca00700 1 -- 172.21.15.67:0/2870631890 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fceb800ba70 con 0x7fcec8100580 2024-03-30T02:29:39.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.173+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fcec8073940 con 0x7fceac0387a0 2024-03-30T02:29:39.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.177+0000 7fcecca00700 1 -- 172.21.15.67:0/2870631890 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcec8073940 con 0x7fceac0387a0 2024-03-30T02:29:39.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fceac0387a0 msgr2=0x7fceac03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 --2- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fceac0387a0 0x7fceac03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fceb0006fd0 tx=0x7fceb0006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 msgr2=0x7fcec8074a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 --2- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 0x7fcec8074a20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fceb8005950 tx=0x7fceb80057c0 comp rx=0 tx=0).stop 2024-03-30T02:29:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 shutdown_connections 2024-03-30T02:29:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 --2- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fceac0387a0 0x7fceac03ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 --2- 172.21.15.67:0/2870631890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcec8100580 0x7fcec8074a20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 >> 172.21.15.67:0/2870631890 conn(0x7fcec80fbb20 msgr2=0x7fcec80fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 shutdown_connections 2024-03-30T02:29:39.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:39.180+0000 7fcec7e00700 1 -- 172.21.15.67:0/2870631890 wait complete. 2024-03-30T02:29:39.604 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi082 2024-03-30T02:29:39.604 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:29:39.604 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.conf 2024-03-30T02:29:39.624 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:29:39.625 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:29:39.684 INFO:tasks.cephadm:Adding host smithi082 to orchestrator... 2024-03-30T02:29:39.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi082 2024-03-30T02:29:40.045 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:29:40.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:40 smithi067 ceph-mon[37808]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:40.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:40.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:40.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:29:40.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:29:40.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:40 smithi067 ceph-mon[37808]: Updating smithi067:/etc/ceph/ceph.conf 2024-03-30T02:29:40.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:40 smithi067 ceph-mon[37808]: Updating smithi067:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:29:41.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.297+0000 7f6a98c00700 1 -- 172.21.15.67:0/1885333436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 msgr2=0x7f6a94108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:41.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.297+0000 7f6a98c00700 1 --2- 172.21.15.67:0/1885333436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 0x7f6a94108ca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009910 tx=0x7f6a84009c20 comp rx=0 tx=0).stop 2024-03-30T02:29:41.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.297+0000 7f6a98c00700 1 -- 172.21.15.67:0/1885333436 shutdown_connections 2024-03-30T02:29:41.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.297+0000 7f6a98c00700 1 --2- 172.21.15.67:0/1885333436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 0x7f6a94108ca0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:41.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.297+0000 7f6a98c00700 1 -- 172.21.15.67:0/1885333436 >> 172.21.15.67:0/1885333436 conn(0x7f6a940fe390 msgr2=0x7f6a941007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:41.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.297+0000 7f6a98c00700 1 -- 172.21.15.67:0/1885333436 shutdown_connections 2024-03-30T02:29:41.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.297+0000 7f6a98c00700 1 -- 172.21.15.67:0/1885333436 wait complete. 2024-03-30T02:29:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.298+0000 7f6a98c00700 1 Processor -- start 2024-03-30T02:29:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.298+0000 7f6a98c00700 1 -- start start 2024-03-30T02:29:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.299+0000 7f6a98c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 0x7f6a94198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.299+0000 7f6a98c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941986d0 con 0x7f6a941088c0 2024-03-30T02:29:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.299+0000 7f6a91e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 0x7f6a94198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.299+0000 7f6a91e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 0x7f6a94198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42830/0 (socket says 172.21.15.67:42830) 2024-03-30T02:29:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.299+0000 7f6a91e00700 1 -- 172.21.15.67:0/500317030 learned_addr learned my addr 172.21.15.67:0/500317030 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:41.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.299+0000 7f6a91e00700 1 -- 172.21.15.67:0/500317030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a840095f0 con 0x7f6a941088c0 2024-03-30T02:29:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.300+0000 7f6a91e00700 1 --2- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 0x7f6a94198190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6a84006010 tx=0x7f6a84004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.300+0000 7f6a8aa00700 1 -- 172.21.15.67:0/500317030 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a8401d070 con 0x7f6a941088c0 2024-03-30T02:29:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.300+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a94198930 con 0x7f6a941088c0 2024-03-30T02:29:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.300+0000 7f6a8aa00700 1 -- 172.21.15.67:0/500317030 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a8400bbb0 con 0x7f6a941088c0 2024-03-30T02:29:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.300+0000 7f6a8aa00700 1 -- 172.21.15.67:0/500317030 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a84003810 con 0x7f6a941088c0 2024-03-30T02:29:41.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.300+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a94198db0 con 0x7f6a941088c0 2024-03-30T02:29:41.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.302+0000 7f6a8aa00700 1 -- 172.21.15.67:0/500317030 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f6a84003a10 con 0x7f6a941088c0 2024-03-30T02:29:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.302+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a94104000 con 0x7f6a941088c0 2024-03-30T02:29:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.302+0000 7f6a8aa00700 1 --2- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780387a0 0x7f6a7803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.302+0000 7f6a8aa00700 1 -- 172.21.15.67:0/500317030 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6a8404c460 con 0x7f6a941088c0 2024-03-30T02:29:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.302+0000 7f6a91400700 1 --2- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780387a0 0x7f6a7803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:41.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.303+0000 7f6a91400700 1 --2- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780387a0 0x7f6a7803ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6a80006fd0 tx=0x7f6a80006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:41.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.308+0000 7f6a8aa00700 1 -- 172.21.15.67:0/500317030 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6a84029770 con 0x7f6a941088c0 2024-03-30T02:29:41.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:41.526+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a9410cbc0 con 0x7f6a780387a0 2024-03-30T02:29:42.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:41 smithi067 ceph-mon[37808]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:29:42.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:41 smithi067 ceph-mon[37808]: Updating smithi067:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:29:42.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:41 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:42.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:41 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:42.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:41 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:42 smithi067 ceph-mon[37808]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:42 smithi067 ceph-mon[37808]: Deploying cephadm binary to smithi082 2024-03-30T02:29:43.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.746+0000 7f6a8aa00700 1 -- 172.21.15.67:0/500317030 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f6a9410cbc0 con 0x7f6a780387a0 2024-03-30T02:29:43.747 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi082' with addr '172.21.15.82' 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780387a0 msgr2=0x7f6a7803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 --2- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780387a0 0x7f6a7803ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6a80006fd0 tx=0x7f6a80006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 msgr2=0x7f6a94198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 --2- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 0x7f6a94198190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6a84006010 tx=0x7f6a84004ff0 comp rx=0 tx=0).stop 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 shutdown_connections 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 --2- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780387a0 0x7f6a7803ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 --2- 172.21.15.67:0/500317030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a941088c0 0x7f6a94198190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 >> 172.21.15.67:0/500317030 conn(0x7f6a940fe390 msgr2=0x7f6a9410a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 shutdown_connections 2024-03-30T02:29:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:43.749+0000 7f6a98c00700 1 -- 172.21.15.67:0/500317030 wait complete. 2024-03-30T02:29:44.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-03-30T02:29:44.503 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:29:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:44 smithi067 ceph-mon[37808]: Added host smithi082 2024-03-30T02:29:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:45.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.615+0000 7efe6a400700 1 -- 172.21.15.67:0/3164571116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 msgr2=0x7efe64106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.615+0000 7efe6a400700 1 --2- 172.21.15.67:0/3164571116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 0x7efe64106aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efe54009910 tx=0x7efe54009c20 comp rx=0 tx=0).stop 2024-03-30T02:29:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.615+0000 7efe6a400700 1 -- 172.21.15.67:0/3164571116 shutdown_connections 2024-03-30T02:29:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.615+0000 7efe6a400700 1 --2- 172.21.15.67:0/3164571116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 0x7efe64106aa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.615+0000 7efe6a400700 1 -- 172.21.15.67:0/3164571116 >> 172.21.15.67:0/3164571116 conn(0x7efe640fc1b0 msgr2=0x7efe640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.615+0000 7efe6a400700 1 -- 172.21.15.67:0/3164571116 shutdown_connections 2024-03-30T02:29:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.615+0000 7efe6a400700 1 -- 172.21.15.67:0/3164571116 wait complete. 2024-03-30T02:29:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.616+0000 7efe6a400700 1 Processor -- start 2024-03-30T02:29:45.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.616+0000 7efe6a400700 1 -- start start 2024-03-30T02:29:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.617+0000 7efe6a400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 0x7efe64198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.617+0000 7efe6a400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe641986a0 con 0x7efe641066c0 2024-03-30T02:29:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.617+0000 7efe63400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 0x7efe64198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:45.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.617+0000 7efe63400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 0x7efe64198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42844/0 (socket says 172.21.15.67:42844) 2024-03-30T02:29:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.617+0000 7efe63400700 1 -- 172.21.15.67:0/2487264211 learned_addr learned my addr 172.21.15.67:0/2487264211 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.617+0000 7efe63400700 1 -- 172.21.15.67:0/2487264211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe540095f0 con 0x7efe641066c0 2024-03-30T02:29:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.617+0000 7efe63400700 1 --2- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 0x7efe64198160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe54006010 tx=0x7efe54004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.618+0000 7efe5be00700 1 -- 172.21.15.67:0/2487264211 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe5401d070 con 0x7efe641066c0 2024-03-30T02:29:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.618+0000 7efe5be00700 1 -- 172.21.15.67:0/2487264211 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe5400bbb0 con 0x7efe641066c0 2024-03-30T02:29:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.618+0000 7efe5be00700 1 -- 172.21.15.67:0/2487264211 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe54003810 con 0x7efe641066c0 2024-03-30T02:29:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.618+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe64198900 con 0x7efe641066c0 2024-03-30T02:29:45.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.618+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe64198d80 con 0x7efe641066c0 2024-03-30T02:29:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.619+0000 7efe5be00700 1 -- 172.21.15.67:0/2487264211 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7efe5400bd20 con 0x7efe641066c0 2024-03-30T02:29:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.619+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe64101e00 con 0x7efe641066c0 2024-03-30T02:29:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.619+0000 7efe5be00700 1 --2- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe480387f0 0x7efe4803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.619+0000 7efe5be00700 1 -- 172.21.15.67:0/2487264211 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efe5404c410 con 0x7efe641066c0 2024-03-30T02:29:45.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.620+0000 7efe62a00700 1 --2- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe480387f0 0x7efe4803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:45.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.620+0000 7efe62a00700 1 --2- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe480387f0 0x7efe4803acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efe50006fd0 tx=0x7efe50006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:45.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.624+0000 7efe5be00700 1 -- 172.21.15.67:0/2487264211 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efe54029360 con 0x7efe641066c0 2024-03-30T02:29:45.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.841+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe6410a9c0 con 0x7efe480387f0 2024-03-30T02:29:45.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.842+0000 7efe5be00700 1 -- 172.21.15.67:0/2487264211 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7efe6410a9c0 con 0x7efe480387f0 2024-03-30T02:29:45.844 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:45.844 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}] 2024-03-30T02:29:45.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.844+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe480387f0 msgr2=0x7efe4803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.844+0000 7efe6a400700 1 --2- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe480387f0 0x7efe4803acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efe50006fd0 tx=0x7efe50006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.844+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 msgr2=0x7efe64198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.844+0000 7efe6a400700 1 --2- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 0x7efe64198160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe54006010 tx=0x7efe54004ff0 comp rx=0 tx=0).stop 2024-03-30T02:29:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.845+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 shutdown_connections 2024-03-30T02:29:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.845+0000 7efe6a400700 1 --2- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe480387f0 0x7efe4803acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.845+0000 7efe6a400700 1 --2- 172.21.15.67:0/2487264211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe641066c0 0x7efe64198160 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.845+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 >> 172.21.15.67:0/2487264211 conn(0x7efe640fc1b0 msgr2=0x7efe64108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.845+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 shutdown_connections 2024-03-30T02:29:45.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:45.845+0000 7efe6a400700 1 -- 172.21.15.67:0/2487264211 wait complete. 2024-03-30T02:29:46.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:46 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:46.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:46 smithi067 ceph-mon[37808]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:29:46.236 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2024-03-30T02:29:46.236 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:29:46.236 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2024-03-30T02:29:46.257 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:29:46.257 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:29:46.317 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2024-03-30T02:29:46.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi170 2024-03-30T02:29:46.671 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:29:47.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.782+0000 7f471f000700 1 -- 172.21.15.67:0/1723348974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 msgr2=0x7f4718106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:47.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.782+0000 7f471f000700 1 --2- 172.21.15.67:0/1723348974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 0x7f4718106aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4708009910 tx=0x7f4708009c20 comp rx=0 tx=0).stop 2024-03-30T02:29:47.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.782+0000 7f471f000700 1 -- 172.21.15.67:0/1723348974 shutdown_connections 2024-03-30T02:29:47.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.782+0000 7f471f000700 1 --2- 172.21.15.67:0/1723348974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 0x7f4718106aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:47.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.782+0000 7f471f000700 1 -- 172.21.15.67:0/1723348974 >> 172.21.15.67:0/1723348974 conn(0x7f47180fc1b0 msgr2=0x7f47180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:47.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.782+0000 7f471f000700 1 -- 172.21.15.67:0/1723348974 shutdown_connections 2024-03-30T02:29:47.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.782+0000 7f471f000700 1 -- 172.21.15.67:0/1723348974 wait complete. 2024-03-30T02:29:47.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.783+0000 7f471f000700 1 Processor -- start 2024-03-30T02:29:47.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.783+0000 7f471f000700 1 -- start start 2024-03-30T02:29:47.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.783+0000 7f471f000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 0x7f4718198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:47.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.783+0000 7f471f000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47181986a0 con 0x7f47181066c0 2024-03-30T02:29:47.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.784+0000 7f4717e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 0x7f4718198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:47.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.784+0000 7f4717e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 0x7f4718198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47038/0 (socket says 172.21.15.67:47038) 2024-03-30T02:29:47.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.784+0000 7f4717e00700 1 -- 172.21.15.67:0/3537772880 learned_addr learned my addr 172.21.15.67:0/3537772880 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:47.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.784+0000 7f4717e00700 1 -- 172.21.15.67:0/3537772880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47080095f0 con 0x7f47181066c0 2024-03-30T02:29:47.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.785+0000 7f4717e00700 1 --2- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 0x7f4718198160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4708006010 tx=0x7f4708004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.785+0000 7f4714c00700 1 -- 172.21.15.67:0/3537772880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f470801d070 con 0x7f47181066c0 2024-03-30T02:29:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.785+0000 7f4714c00700 1 -- 172.21.15.67:0/3537772880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f470800bbb0 con 0x7f47181066c0 2024-03-30T02:29:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.785+0000 7f4714c00700 1 -- 172.21.15.67:0/3537772880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4708003810 con 0x7f47181066c0 2024-03-30T02:29:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.785+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4718198900 con 0x7f47181066c0 2024-03-30T02:29:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.785+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4718198d80 con 0x7f47181066c0 2024-03-30T02:29:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.786+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4718101e00 con 0x7f47181066c0 2024-03-30T02:29:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.786+0000 7f4714c00700 1 -- 172.21.15.67:0/3537772880 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f470800bd20 con 0x7f47181066c0 2024-03-30T02:29:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.787+0000 7f4714c00700 1 --2- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46fc0387f0 0x7f46fc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:47.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.787+0000 7f4714c00700 1 -- 172.21.15.67:0/3537772880 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f470804c430 con 0x7f47181066c0 2024-03-30T02:29:47.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.787+0000 7f4717400700 1 --2- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46fc0387f0 0x7f46fc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:47.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.788+0000 7f4717400700 1 --2- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46fc0387f0 0x7f46fc03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4704006fd0 tx=0x7f4704006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:47.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:47.792+0000 7f4714c00700 1 -- 172.21.15.67:0/3537772880 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f470802aba0 con 0x7f47181066c0 2024-03-30T02:29:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:48.009+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}) v1 -- 0x7f471810a9c0 con 0x7f46fc0387f0 2024-03-30T02:29:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:47 smithi067 ceph-mon[37808]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:29:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:29:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:29:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:29:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:48 smithi067 ceph-mon[37808]: Updating smithi082:/etc/ceph/ceph.conf 2024-03-30T02:29:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:48 smithi067 ceph-mon[37808]: Updating smithi082:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:29:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:48 smithi067 ceph-mon[37808]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:29:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:48 smithi067 ceph-mon[37808]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:48 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:48 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:48 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:50.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.263+0000 7f4714c00700 1 -- 172.21.15.67:0/3537772880 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f471810a9c0 con 0x7f46fc0387f0 2024-03-30T02:29:50.265 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi170' with addr '172.21.15.170' 2024-03-30T02:29:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.265+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46fc0387f0 msgr2=0x7f46fc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.265+0000 7f471f000700 1 --2- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46fc0387f0 0x7f46fc03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4704006fd0 tx=0x7f4704006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.265+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 msgr2=0x7f4718198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.265+0000 7f471f000700 1 --2- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 0x7f4718198160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4708006010 tx=0x7f4708004ff0 comp rx=0 tx=0).stop 2024-03-30T02:29:50.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.265+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 shutdown_connections 2024-03-30T02:29:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.265+0000 7f471f000700 1 --2- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46fc0387f0 0x7f46fc03acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.265+0000 7f471f000700 1 --2- 172.21.15.67:0/3537772880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f47181066c0 0x7f4718198160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.265+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 >> 172.21.15.67:0/3537772880 conn(0x7f47180fc1b0 msgr2=0x7f4718108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.265+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 shutdown_connections 2024-03-30T02:29:50.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:50.266+0000 7f471f000700 1 -- 172.21.15.67:0/3537772880 wait complete. 2024-03-30T02:29:50.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:49 smithi067 ceph-mon[37808]: Updating smithi082:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:29:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:49 smithi067 ceph-mon[37808]: Deploying cephadm binary to smithi170 2024-03-30T02:29:50.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:49 smithi067 ceph-mon[37808]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:29:50.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-03-30T02:29:51.041 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:29:51.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:51.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:51 smithi067 ceph-mon[37808]: Added host smithi170 2024-03-30T02:29:51.506 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:51.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:51.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:51.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:51 smithi067 ceph-mon[37808]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:29:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.174+0000 7facadc00700 1 -- 172.21.15.67:0/1540955865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 msgr2=0x7faca8106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.174+0000 7facadc00700 1 --2- 172.21.15.67:0/1540955865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 0x7faca8106aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fac98009910 tx=0x7fac98009c20 comp rx=0 tx=0).stop 2024-03-30T02:29:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.174+0000 7facadc00700 1 -- 172.21.15.67:0/1540955865 shutdown_connections 2024-03-30T02:29:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.174+0000 7facadc00700 1 --2- 172.21.15.67:0/1540955865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 0x7faca8106aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.174+0000 7facadc00700 1 -- 172.21.15.67:0/1540955865 >> 172.21.15.67:0/1540955865 conn(0x7faca80fc1b0 msgr2=0x7faca80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.174+0000 7facadc00700 1 -- 172.21.15.67:0/1540955865 shutdown_connections 2024-03-30T02:29:52.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.174+0000 7facadc00700 1 -- 172.21.15.67:0/1540955865 wait complete. 2024-03-30T02:29:52.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.175+0000 7facadc00700 1 Processor -- start 2024-03-30T02:29:52.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.175+0000 7facadc00700 1 -- start start 2024-03-30T02:29:52.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.175+0000 7facadc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 0x7faca80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:52.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.175+0000 7facadc00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca8074f70 con 0x7faca81066c0 2024-03-30T02:29:52.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.176+0000 7faca6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 0x7faca80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:52.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.176+0000 7faca6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 0x7faca80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47052/0 (socket says 172.21.15.67:47052) 2024-03-30T02:29:52.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.176+0000 7faca6a00700 1 -- 172.21.15.67:0/2755600799 learned_addr learned my addr 172.21.15.67:0/2755600799 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:52.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.176+0000 7faca6a00700 1 -- 172.21.15.67:0/2755600799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980095f0 con 0x7faca81066c0 2024-03-30T02:29:52.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.176+0000 7faca6a00700 1 --2- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 0x7faca80730b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac98004f40 tx=0x7fac98005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:52.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.177+0000 7fac9f400700 1 -- 172.21.15.67:0/2755600799 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac9801d070 con 0x7faca81066c0 2024-03-30T02:29:52.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.177+0000 7fac9f400700 1 -- 172.21.15.67:0/2755600799 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac9800bbb0 con 0x7faca81066c0 2024-03-30T02:29:52.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.177+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca80735f0 con 0x7faca81066c0 2024-03-30T02:29:52.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.177+0000 7fac9f400700 1 -- 172.21.15.67:0/2755600799 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac98003810 con 0x7faca81066c0 2024-03-30T02:29:52.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.177+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca8073a70 con 0x7faca81066c0 2024-03-30T02:29:52.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.178+0000 7fac9f400700 1 -- 172.21.15.67:0/2755600799 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fac9800bd20 con 0x7faca81066c0 2024-03-30T02:29:52.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.178+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca8101e00 con 0x7faca81066c0 2024-03-30T02:29:52.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.178+0000 7fac9f400700 1 --2- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fac8c0387a0 0x7fac8c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:52.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.178+0000 7fac9f400700 1 -- 172.21.15.67:0/2755600799 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac9804c480 con 0x7faca81066c0 2024-03-30T02:29:52.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.179+0000 7fac9fe00700 1 --2- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fac8c0387a0 0x7fac8c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:52.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.179+0000 7fac9fe00700 1 --2- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fac8c0387a0 0x7fac8c03ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac90006fd0 tx=0x7fac90006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:52.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.183+0000 7fac9f400700 1 -- 172.21.15.67:0/2755600799 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fac9802aba0 con 0x7faca81066c0 2024-03-30T02:29:52.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.401+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faca8073cb0 con 0x7fac8c0387a0 2024-03-30T02:29:52.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.402+0000 7fac9f400700 1 -- 172.21.15.67:0/2755600799 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7faca8073cb0 con 0x7fac8c0387a0 2024-03-30T02:29:52.404 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:29:52.404 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fac8c0387a0 msgr2=0x7fac8c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 --2- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fac8c0387a0 0x7fac8c03ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac90006fd0 tx=0x7fac90006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 msgr2=0x7faca80730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 --2- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 0x7faca80730b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac98004f40 tx=0x7fac98005020 comp rx=0 tx=0).stop 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 shutdown_connections 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 --2- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fac8c0387a0 0x7fac8c03ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 --2- 172.21.15.67:0/2755600799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faca81066c0 0x7faca80730b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 >> 172.21.15.67:0/2755600799 conn(0x7faca80fc1b0 msgr2=0x7faca81082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 shutdown_connections 2024-03-30T02:29:52.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:52.405+0000 7facadc00700 1 -- 172.21.15.67:0/2755600799 wait complete. 2024-03-30T02:29:52.782 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:52.806 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-30T02:29:52.806 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-03-30T02:29:53.161 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:29:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:53 smithi067 ceph-mon[37808]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:29:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:53 smithi067 ceph-mon[37808]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:29:54.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.308+0000 7f34ce400700 1 -- 172.21.15.67:0/1444995893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 msgr2=0x7f34c80682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:54.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.308+0000 7f34ce400700 1 --2- 172.21.15.67:0/1444995893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 0x7f34c80682d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009910 tx=0x7f34b8009c20 comp rx=0 tx=0).stop 2024-03-30T02:29:54.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.308+0000 7f34ce400700 1 -- 172.21.15.67:0/1444995893 shutdown_connections 2024-03-30T02:29:54.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.308+0000 7f34ce400700 1 --2- 172.21.15.67:0/1444995893 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 0x7f34c80682d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:54.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.308+0000 7f34ce400700 1 -- 172.21.15.67:0/1444995893 >> 172.21.15.67:0/1444995893 conn(0x7f34c8075110 msgr2=0x7f34c8075520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:54.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.308+0000 7f34ce400700 1 -- 172.21.15.67:0/1444995893 shutdown_connections 2024-03-30T02:29:54.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.308+0000 7f34ce400700 1 -- 172.21.15.67:0/1444995893 wait complete. 2024-03-30T02:29:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.309+0000 7f34ce400700 1 Processor -- start 2024-03-30T02:29:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.309+0000 7f34ce400700 1 -- start start 2024-03-30T02:29:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.310+0000 7f34ce400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 0x7f34c8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.310+0000 7f34ce400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c81986d0 con 0x7f34c8067ef0 2024-03-30T02:29:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.310+0000 7f34c7400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 0x7f34c8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.310+0000 7f34c7400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 0x7f34c8198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47060/0 (socket says 172.21.15.67:47060) 2024-03-30T02:29:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.310+0000 7f34c7400700 1 -- 172.21.15.67:0/2075940075 learned_addr learned my addr 172.21.15.67:0/2075940075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:29:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.310+0000 7f34c7400700 1 -- 172.21.15.67:0/2075940075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80095f0 con 0x7f34c8067ef0 2024-03-30T02:29:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.310+0000 7f34c7400700 1 --2- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 0x7f34c8198190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006010 tx=0x7f34b8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:54.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.311+0000 7f34bfe00700 1 -- 172.21.15.67:0/2075940075 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34b801d070 con 0x7f34c8067ef0 2024-03-30T02:29:54.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.311+0000 7f34bfe00700 1 -- 172.21.15.67:0/2075940075 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34b800bc10 con 0x7f34c8067ef0 2024-03-30T02:29:54.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.311+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c81988d0 con 0x7f34c8067ef0 2024-03-30T02:29:54.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.311+0000 7f34bfe00700 1 -- 172.21.15.67:0/2075940075 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34b80038a0 con 0x7f34c8067ef0 2024-03-30T02:29:54.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.311+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c8198cf0 con 0x7f34c8067ef0 2024-03-30T02:29:54.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.312+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c80ffbf0 con 0x7f34c8067ef0 2024-03-30T02:29:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.312+0000 7f34bfe00700 1 -- 172.21.15.67:0/2075940075 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f34b800b6f0 con 0x7f34c8067ef0 2024-03-30T02:29:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.312+0000 7f34bfe00700 1 --2- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34ac0387f0 0x7f34ac03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.313+0000 7f34bfe00700 1 -- 172.21.15.67:0/2075940075 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f34b804c470 con 0x7f34c8067ef0 2024-03-30T02:29:54.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.313+0000 7f34c6a00700 1 --2- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34ac0387f0 0x7f34ac03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.313+0000 7f34c6a00700 1 --2- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34ac0387f0 0x7f34ac03acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f34b4006fd0 tx=0x7f34b4006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.317+0000 7f34bfe00700 1 -- 172.21.15.67:0/2075940075 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f34b8029540 con 0x7f34c8067ef0 2024-03-30T02:29:54.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.535+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f34c80623e0 con 0x7f34c8067ef0 2024-03-30T02:29:54.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.906+0000 7f34bfe00700 1 -- 172.21.15.67:0/2075940075 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f34b8026020 con 0x7f34c8067ef0 2024-03-30T02:29:54.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.908+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34ac0387f0 msgr2=0x7f34ac03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:54.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.908+0000 7f34ce400700 1 --2- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34ac0387f0 0x7f34ac03acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f34b4006fd0 tx=0x7f34b4006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:54.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.908+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 msgr2=0x7f34c8198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:54.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.908+0000 7f34ce400700 1 --2- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 0x7f34c8198190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006010 tx=0x7f34b8004ff0 comp rx=0 tx=0).stop 2024-03-30T02:29:54.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.908+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 shutdown_connections 2024-03-30T02:29:54.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.908+0000 7f34ce400700 1 --2- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f34ac0387f0 0x7f34ac03acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:54.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.908+0000 7f34ce400700 1 --2- 172.21.15.67:0/2075940075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34c8067ef0 0x7f34c8198190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:54.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.908+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 >> 172.21.15.67:0/2075940075 conn(0x7f34c8075110 msgr2=0x7f34c80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:54.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.909+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 shutdown_connections 2024-03-30T02:29:54.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:29:54.909+0000 7f34ce400700 1 -- 172.21.15.67:0/2075940075 wait complete. 2024-03-30T02:29:54.911 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-03-30T02:29:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:29:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:29:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:29:55.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-30T02:29:55.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2075940075' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-30T02:29:55.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:55.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:55.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:55.283 INFO:tasks.cephadm:Adding mon.a on smithi067 2024-03-30T02:29:55.283 INFO:tasks.cephadm:Adding mon.b on smithi082 2024-03-30T02:29:55.283 INFO:tasks.cephadm:Adding mon.c on smithi170 2024-03-30T02:29:55.283 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '3;smithi067:172.21.15.67=a;smithi082:172.21.15.82=b;smithi170:172.21.15.170=c' 2024-03-30T02:29:55.526 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:29:56.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:55 smithi067 ceph-mon[37808]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:29:56.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:55 smithi067 ceph-mon[37808]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:29:56.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:55 smithi067 ceph-mon[37808]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:29:56.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:55 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2075940075' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-30T02:29:56.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:55 smithi067 ceph-mon[37808]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:29:56.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:55 smithi067 ceph-mon[37808]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.432+0000 7f99f2d43700 1 -- 172.21.15.170:0/977837182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 msgr2=0x7f99ec105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.432+0000 7f99f2d43700 1 --2- 172.21.15.170:0/977837182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 0x7f99ec105a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009910 tx=0x7f99e0009c20 comp rx=0 tx=0).stop 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.432+0000 7f99f2d43700 1 -- 172.21.15.170:0/977837182 shutdown_connections 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.432+0000 7f99f2d43700 1 --2- 172.21.15.170:0/977837182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 0x7f99ec105a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.432+0000 7f99f2d43700 1 -- 172.21.15.170:0/977837182 >> 172.21.15.170:0/977837182 conn(0x7f99ec075110 msgr2=0x7f99ec075520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.432+0000 7f99f2d43700 1 -- 172.21.15.170:0/977837182 shutdown_connections 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.432+0000 7f99f2d43700 1 -- 172.21.15.170:0/977837182 wait complete. 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.432+0000 7f99f2d43700 1 Processor -- start 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.432+0000 7f99f2d43700 1 -- start start 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.433+0000 7f99f2d43700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 0x7f99ec0fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.433+0000 7f99f2d43700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec0fead0 con 0x7f99ec105630 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.433+0000 7f99f0adf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 0x7f99ec0fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.433+0000 7f99f0adf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 0x7f99ec0fcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.170:41958/0 (socket says 172.21.15.170:41958) 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.433+0000 7f99f0adf700 1 -- 172.21.15.170:0/3586632899 learned_addr learned my addr 172.21.15.170:0/3586632899 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.433+0000 7f99f0adf700 1 -- 172.21.15.170:0/3586632899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00095f0 con 0x7f99ec105630 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.433+0000 7f99f0adf700 1 --2- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 0x7f99ec0fcc10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f99e0004f40 tx=0x7f99e0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.434+0000 7f99e9ffb700 1 -- 172.21.15.170:0/3586632899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e001d070 con 0x7f99ec105630 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.434+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ec0fd150 con 0x7f99ec105630 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.434+0000 7f99e9ffb700 1 -- 172.21.15.170:0/3586632899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e000bc10 con 0x7f99ec105630 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.434+0000 7f99e9ffb700 1 -- 172.21.15.170:0/3586632899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99e00037d0 con 0x7f99ec105630 2024-03-30T02:29:56.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.434+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ec0fd570 con 0x7f99ec105630 2024-03-30T02:29:56.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.435+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ec107ee0 con 0x7f99ec105630 2024-03-30T02:29:56.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.435+0000 7f99e9ffb700 1 -- 172.21.15.170:0/3586632899 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f99e000b6f0 con 0x7f99ec105630 2024-03-30T02:29:56.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.435+0000 7f99e9ffb700 1 --2- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f99dc0387a0 0x7f99dc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:56.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.435+0000 7f99e9ffb700 1 -- 172.21.15.170:0/3586632899 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f99e004c400 con 0x7f99ec105630 2024-03-30T02:29:56.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.435+0000 7f99ebfff700 1 --2- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f99dc0387a0 0x7f99dc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:56.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.436+0000 7f99ebfff700 1 --2- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f99dc0387a0 0x7f99dc03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f99d8006fd0 tx=0x7f99d8006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:56.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.441+0000 7f99e9ffb700 1 -- 172.21.15.170:0/3586632899 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f99e0003a90 con 0x7f99ec105630 2024-03-30T02:29:56.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.644+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi082:172.21.15.82=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f99ec0fd810 con 0x7f99dc0387a0 2024-03-30T02:29:56.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.651+0000 7f99e9ffb700 1 -- 172.21.15.170:0/3586632899 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f99ec0fd810 con 0x7f99dc0387a0 2024-03-30T02:29:56.652 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2024-03-30T02:29:56.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f99dc0387a0 msgr2=0x7f99dc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:56.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 --2- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f99dc0387a0 0x7f99dc03ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f99d8006fd0 tx=0x7f99d8006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:56.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 msgr2=0x7f99ec0fcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:56.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 --2- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 0x7f99ec0fcc10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f99e0004f40 tx=0x7f99e0005020 comp rx=0 tx=0).stop 2024-03-30T02:29:56.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 shutdown_connections 2024-03-30T02:29:56.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 --2- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f99dc0387a0 0x7f99dc03ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:56.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 --2- 172.21.15.170:0/3586632899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99ec105630 0x7f99ec0fcc10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:56.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 >> 172.21.15.170:0/3586632899 conn(0x7f99ec075110 msgr2=0x7f99ec18ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:56.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 shutdown_connections 2024-03-30T02:29:56.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:56.653+0000 7f99f2d43700 1 -- 172.21.15.170:0/3586632899 wait complete. 2024-03-30T02:29:57.051 DEBUG:teuthology.orchestra.run.smithi082:mon.b> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.b.service 2024-03-30T02:29:57.053 DEBUG:teuthology.orchestra.run.smithi170:mon.c> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.c.service 2024-03-30T02:29:57.055 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-30T02:29:57.056 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-30T02:29:57.087 INFO:journalctl@ceph.mon.c.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:29:57.089 INFO:journalctl@ceph.mon.b.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:29:57.427 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:29:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi082:172.21.15.82=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:29:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi082:172.21.15.82=b;smithi170:172.21.15.170=c;count:3 2024-03-30T02:29:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:29:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:29:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:29:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:29:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:29:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:29:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: Deploying daemon mon.c on smithi170 2024-03-30T02:29:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:29:57 smithi067 ceph-mon[37808]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.949+0000 7fe46716a700 1 -- 172.21.15.170:0/3718035160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 msgr2=0x7fe460105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.949+0000 7fe46716a700 1 --2- 172.21.15.170:0/3718035160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 0x7fe460105a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe45800b0a0 tx=0x7fe45800b3b0 comp rx=0 tx=0).stop 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.949+0000 7fe46716a700 1 -- 172.21.15.170:0/3718035160 shutdown_connections 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.949+0000 7fe46716a700 1 --2- 172.21.15.170:0/3718035160 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 0x7fe460105a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.949+0000 7fe46716a700 1 -- 172.21.15.170:0/3718035160 >> 172.21.15.170:0/3718035160 conn(0x7fe460075110 msgr2=0x7fe460075520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.949+0000 7fe46716a700 1 -- 172.21.15.170:0/3718035160 shutdown_connections 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.949+0000 7fe46716a700 1 -- 172.21.15.170:0/3718035160 wait complete. 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.950+0000 7fe46716a700 1 Processor -- start 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.950+0000 7fe46716a700 1 -- start start 2024-03-30T02:29:58.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.950+0000 7fe46716a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 0x7fe460194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:58.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.950+0000 7fe46716a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe460195360 con 0x7fe460105630 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.950+0000 7fe464f06700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 0x7fe460194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.950+0000 7fe464f06700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 0x7fe460194e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.170:41974/0 (socket says 172.21.15.170:41974) 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.950+0000 7fe464f06700 1 -- 172.21.15.170:0/2108486702 learned_addr learned my addr 172.21.15.170:0/2108486702 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.951+0000 7fe464f06700 1 -- 172.21.15.170:0/2108486702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe458009d00 con 0x7fe460105630 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.951+0000 7fe464f06700 1 --2- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 0x7fe460194e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe458015040 tx=0x7fe45800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.951+0000 7fe45dffb700 1 -- 172.21.15.170:0/2108486702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe45800e040 con 0x7fe460105630 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.951+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe460195560 con 0x7fe460105630 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.951+0000 7fe45dffb700 1 -- 172.21.15.170:0/2108486702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe45801d070 con 0x7fe460105630 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.952+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe460195a60 con 0x7fe460105630 2024-03-30T02:29:58.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.952+0000 7fe45dffb700 1 -- 172.21.15.170:0/2108486702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe458004a40 con 0x7fe460105630 2024-03-30T02:29:58.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.953+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe460100d00 con 0x7fe460105630 2024-03-30T02:29:58.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.953+0000 7fe45dffb700 1 -- 172.21.15.170:0/2108486702 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7fe458003c10 con 0x7fe460105630 2024-03-30T02:29:58.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.953+0000 7fe45dffb700 1 --2- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe4480387a0 0x7fe44803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:29:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.953+0000 7fe45dffb700 1 -- 172.21.15.170:0/2108486702 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe45801f070 con 0x7fe460105630 2024-03-30T02:29:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.953+0000 7fe45ffff700 1 --2- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe4480387a0 0x7fe44803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:29:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.954+0000 7fe45ffff700 1 --2- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe4480387a0 0x7fe44803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe454006fd0 tx=0x7fe454006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:29:58.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:58.958+0000 7fe45dffb700 1 -- 172.21.15.170:0/2108486702 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe458022b70 con 0x7fe460105630 2024-03-30T02:29:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.260+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe4601099a0 con 0x7fe460105630 2024-03-30T02:29:59.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.261+0000 7fe45dffb700 1 -- 172.21.15.170:0/2108486702 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fe458017020 con 0x7fe460105630 2024-03-30T02:29:59.262 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:29:59.262 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","modified":"2024-03-30T02:28:21.112353Z","created":"2024-03-30T02:28:21.112353Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-30T02:29:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe4480387a0 msgr2=0x7fe44803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 --2- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe4480387a0 0x7fe44803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe454006fd0 tx=0x7fe454006e40 comp rx=0 tx=0).stop 2024-03-30T02:29:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 msgr2=0x7fe460194e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:29:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 --2- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 0x7fe460194e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe458015040 tx=0x7fe45800bd40 comp rx=0 tx=0).stop 2024-03-30T02:29:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 shutdown_connections 2024-03-30T02:29:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 --2- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe4480387a0 0x7fe44803ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 --2- 172.21.15.170:0/2108486702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe460105630 0x7fe460194e20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:29:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 >> 172.21.15.170:0/2108486702 conn(0x7fe460075110 msgr2=0x7fe4600fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:29:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 shutdown_connections 2024-03-30T02:29:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:29:59.264+0000 7fe46716a700 1 -- 172.21.15.170:0/2108486702 wait complete. 2024-03-30T02:29:59.269 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2024-03-30T02:29:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:29:59 smithi170 ceph-mon[78798]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-30T02:30:00.760 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-30T02:30:00.761 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-30T02:30:00.992 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:30:04.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.905+0000 7f60817c5700 1 -- 172.21.15.170:0/415755572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640035c0 msgr2=0x7f6064005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:04.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.905+0000 7f60817c5700 1 --2- 172.21.15.170:0/415755572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640035c0 0x7f6064005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f607c0ff620 tx=0x7f607000feb0 comp rx=0 tx=0).stop 2024-03-30T02:30:04.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.905+0000 7f60817c5700 1 -- 172.21.15.170:0/415755572 shutdown_connections 2024-03-30T02:30:04.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.905+0000 7f60817c5700 1 --2- 172.21.15.170:0/415755572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60640035c0 0x7f6064005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:04.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.905+0000 7f60817c5700 1 -- 172.21.15.170:0/415755572 >> 172.21.15.170:0/415755572 conn(0x7f607c076390 msgr2=0x7f607c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:04.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.905+0000 7f60817c5700 1 -- 172.21.15.170:0/415755572 shutdown_connections 2024-03-30T02:30:04.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.905+0000 7f60817c5700 1 -- 172.21.15.170:0/415755572 wait complete. 2024-03-30T02:30:04.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.906+0000 7f60817c5700 1 Processor -- start 2024-03-30T02:30:04.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f60817c5700 1 -- start start 2024-03-30T02:30:04.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f60817c5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 0x7f607c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:04.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f60817c5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f607c1987d0 0x7f607c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:04.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f60817c5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c198d60 con 0x7f607c1987d0 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f60817c5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c198ed0 con 0x7f60640035c0 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f607affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 0x7f607c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f607affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 0x7f607c198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54508/0 (socket says 172.21.15.170:54508) 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f607affd700 1 -- 172.21.15.170:0/4014334204 learned_addr learned my addr 172.21.15.170:0/4014334204 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f607affd700 1 -- 172.21.15.170:0/4014334204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 msgr2=0x7f607c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f607affd700 1 -- 172.21.15.170:0/4014334204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 msgr2=0x7f607c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f607affd700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 0x7f607c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f607affd700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 0x7f607c198290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.907+0000 7f607a7fc700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f607c1987d0 0x7f607c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.908+0000 7f607a7fc700 1 -- 172.21.15.170:0/4014334204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 msgr2=0x7f607c198290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.908+0000 7f607a7fc700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 0x7f607c198290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.908+0000 7f607a7fc700 1 -- 172.21.15.170:0/4014334204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607000f900 con 0x7f607c1987d0 2024-03-30T02:30:04.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.908+0000 7f607a7fc700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f607c1987d0 0x7f607c19cb80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b4d0 tx=0x7f606c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:04.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.908+0000 7f6063fff700 1 -- 172.21.15.170:0/4014334204 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f606c011630 con 0x7f607c1987d0 2024-03-30T02:30:04.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.908+0000 7f6063fff700 1 -- 172.21.15.170:0/4014334204 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606c011c50 con 0x7f607c1987d0 2024-03-30T02:30:04.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.908+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c19d120 con 0x7f607c1987d0 2024-03-30T02:30:04.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.908+0000 7f6063fff700 1 -- 172.21.15.170:0/4014334204 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f606c010d90 con 0x7f607c1987d0 2024-03-30T02:30:04.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.908+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c19d670 con 0x7f607c1987d0 2024-03-30T02:30:04.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.909+0000 7f6063fff700 1 -- 172.21.15.170:0/4014334204 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f606c01a400 con 0x7f607c1987d0 2024-03-30T02:30:04.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.909+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c19d9a0 con 0x7f607c1987d0 2024-03-30T02:30:04.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.910+0000 7f6063fff700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f60680385d0 0x7f606803aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:04.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.910+0000 7f6063fff700 1 -- 172.21.15.170:0/4014334204 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f606c04ae40 con 0x7f607c1987d0 2024-03-30T02:30:04.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.910+0000 7f607affd700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f60680385d0 0x7f606803aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:04.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.911+0000 7f607affd700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f60680385d0 0x7f606803aa90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6070011510 tx=0x7f6070005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:04.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:04.916+0000 7f6063fff700 1 -- 172.21.15.170:0/4014334204 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f606c017680 con 0x7f607c1987d0 2024-03-30T02:30:05.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:30:05.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: mon.a calling monitor election 2024-03-30T02:30:05.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:05.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: mon.c calling monitor election 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: fsmap 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: mgrmap e13: x(active, since 37s) 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: overall HEALTH_OK 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:05.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:05.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:05.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.183+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f607c199640 con 0x7f607c1987d0 2024-03-30T02:30:05.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.183+0000 7f6063fff700 1 -- 172.21.15.170:0/4014334204 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f606c0172c0 con 0x7f607c1987d0 2024-03-30T02:30:05.185 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:30:05.185 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","modified":"2024-03-30T02:29:59.615318Z","created":"2024-03-30T02:28:21.112353Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f60680385d0 msgr2=0x7f606803aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f60680385d0 0x7f606803aa90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6070011510 tx=0x7f6070005960 comp rx=0 tx=0).stop 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f607c1987d0 msgr2=0x7f607c19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f607c1987d0 0x7f607c19cb80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b4d0 tx=0x7f606c00b890 comp rx=0 tx=0).stop 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 shutdown_connections 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f60680385d0 0x7f606803aa90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60640035c0 0x7f607c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 --2- 172.21.15.170:0/4014334204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f607c1987d0 0x7f607c19cb80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 >> 172.21.15.170:0/4014334204 conn(0x7f607c076390 msgr2=0x7f607c068cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 shutdown_connections 2024-03-30T02:30:05.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:05.186+0000 7f60817c5700 1 -- 172.21.15.170:0/4014334204 wait complete. 2024-03-30T02:30:05.188 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2024-03-30T02:30:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:30:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: mon.a calling monitor election 2024-03-30T02:30:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: mon.c calling monitor election 2024-03-30T02:30:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: fsmap 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: mgrmap e13: x(active, since 37s) 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: overall HEALTH_OK 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:05.192 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:05 smithi170 ceph-mon[78798]: Deploying daemon mon.b on smithi082 2024-03-30T02:30:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:05 smithi170 ceph-mon[78798]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:05 smithi170 ceph-mon[78798]: from='client.? 172.21.15.170:0/4014334204' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T02:30:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:05 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:06.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:05 smithi067 ceph-mon[37808]: Deploying daemon mon.b on smithi082 2024-03-30T02:30:06.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:05 smithi067 ceph-mon[37808]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:06.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:05 smithi067 ceph-mon[37808]: from='client.? 172.21.15.170:0/4014334204' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T02:30:06.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:05 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:06.543 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-30T02:30:06.543 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-03-30T02:30:06.770 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:30:06.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:30:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:30:06.614+0000 7feb9f000700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-30T02:30:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 systemd[1]: Started Ceph mon.b for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:30:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711765807271179, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711765807, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e99ae78f-4097-4674-85a6-99a883ab5ea8", "db_session_id": "HZRG6QSQ92V6S9TPKK3X", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-30T02:30:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711765807271261, "job": 1, "event": "recovery_finished"} 2024-03-30T02:30:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556b1f7f8000 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: rocksdb: DB pointer 0x556b1f7e6000 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout: ** DB Stats ** 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-30T02:30:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: ** Compaction Stats [default] ** 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: ** Compaction Stats [default] ** 2024-03-30T02:30:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-30T02:30:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout: Block cache BinnedLRUCache@0x556b1f75b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: using public_addr v2:172.21.15.82:0/0 -> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: starting mon.b rank -1 at public addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] at bind addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(???) e0 preinit fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).mds e1 new map 2024-03-30T02:30:07.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout: e1 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout: legacy client fscid: -1 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout: 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout: No filesystems configured 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-30T02:30:07.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: Added host smithi170 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2075940075' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-30T02:30:07.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2075940075' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi082:172.21.15.82=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi082:172.21.15.82=b;smithi170:172.21.15.170=c;count:3 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: Deploying daemon mon.c on smithi170 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.a calling monitor election 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:07.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.c calling monitor election 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: fsmap 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mgrmap e13: x(active, since 37s) 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: overall HEALTH_OK 2024-03-30T02:30:07.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:07.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:07.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:07.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: Deploying daemon mon.b on smithi082 2024-03-30T02:30:07.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:07.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='client.? 172.21.15.170:0/4014334204' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-30T02:30:07.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:07.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:07 smithi082 ceph-mon[79176]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-30T02:30:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: mon.c calling monitor election 2024-03-30T02:30:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:30:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: mon.a calling monitor election 2024-03-30T02:30:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: fsmap 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: mgrmap e13: x(active, since 45s) 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-30T02:30:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-30T02:30:12.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-30T02:30:12.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: mon.b (rank 2) addr [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] is down (out of quorum) 2024-03-30T02:30:12.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:12.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: mon.c calling monitor election 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: mon.a calling monitor election 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: fsmap 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: mgrmap e13: x(active, since 45s) 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: mon.b (rank 2) addr [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] is down (out of quorum) 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.653+0000 7f4ad669d700 1 -- 172.21.15.170:0/3107837668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4abc006370 msgr2=0x7f4abc0087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.653+0000 7f4ad669d700 1 --2- 172.21.15.170:0/3107837668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4abc006370 0x7f4abc0087c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00685b0 tx=0x7f4ab800fe90 comp rx=0 tx=0).stop 2024-03-30T02:30:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.653+0000 7f4ad669d700 1 -- 172.21.15.170:0/3107837668 shutdown_connections 2024-03-30T02:30:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.653+0000 7f4ad669d700 1 --2- 172.21.15.170:0/3107837668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4abc006370 0x7f4abc0087c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.653+0000 7f4ad669d700 1 -- 172.21.15.170:0/3107837668 >> 172.21.15.170:0/3107837668 conn(0x7f4ad0076390 msgr2=0x7f4ad00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.653+0000 7f4ad669d700 1 -- 172.21.15.170:0/3107837668 shutdown_connections 2024-03-30T02:30:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.653+0000 7f4ad669d700 1 -- 172.21.15.170:0/3107837668 wait complete. 2024-03-30T02:30:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.654+0000 7f4ad669d700 1 Processor -- start 2024-03-30T02:30:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.654+0000 7f4ad669d700 1 -- start start 2024-03-30T02:30:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.654+0000 7f4ad669d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4abc006370 0x7f4ad00ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.654+0000 7f4ad669d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ad00ffbf0 0x7f4ad0100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.654+0000 7f4ad669d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 0x7f4ad01a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.654+0000 7f4ad669d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad01a4d60 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.654+0000 7f4ad669d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad01a4ea0 con 0x7f4abc006370 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.654+0000 7f4ad669d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad01a5060 con 0x7f4ad01a23a0 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.654+0000 7f4ad4c3a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 0x7f4ad01a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4ad4c3a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 0x7f4ad01a4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53270/0 (socket says 172.21.15.170:53270) 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4ad4c3a700 1 -- 172.21.15.170:0/3681495183 learned_addr learned my addr 172.21.15.170:0/3681495183 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acffff700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4abc006370 0x7f4ad00ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acf7fe700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ad00ffbf0 0x7f4ad0100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4ad4c3a700 1 -- 172.21.15.170:0/3681495183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 msgr2=0x7f4ad01a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4ad4c3a700 1 -- 172.21.15.170:0/3681495183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 msgr2=0x7f4ad01a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4ad4c3a700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 0x7f4ad01a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acffff700 1 -- 172.21.15.170:0/3681495183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4abc006370 msgr2=0x7f4ad00ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acffff700 1 -- 172.21.15.170:0/3681495183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4abc006370 msgr2=0x7f4ad00ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-30T02:30:13.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acffff700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4abc006370 0x7f4ad00ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4ad4c3a700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 0x7f4ad01a4790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acffff700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4abc006370 0x7f4ad00ff6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acf7fe700 1 -- 172.21.15.170:0/3681495183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 msgr2=0x7f4ad01a4790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acf7fe700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 0x7f4ad01a4790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acf7fe700 1 -- 172.21.15.170:0/3681495183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4abc006370 msgr2=0x7f4ad00ff6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acf7fe700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4abc006370 0x7f4ad00ff6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acf7fe700 1 -- 172.21.15.170:0/3681495183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab800f8e0 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acf7fe700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ad00ffbf0 0x7f4ad0100070 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000b550 tx=0x7f4ac000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acd7fa700 1 -- 172.21.15.170:0/3681495183 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0004230 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acd7fa700 1 -- 172.21.15.170:0/3681495183 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ac0004390 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad01a5340 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4acd7fa700 1 -- 172.21.15.170:0/3681495183 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0011af0 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.655+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad01a5840 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.656+0000 7f4acd7fa700 1 -- 172.21.15.170:0/3681495183 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f4ac0011c50 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.659 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.656+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad0068c30 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.656+0000 7f4acd7fa700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4ab0038c20 0x7f4ab003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.656+0000 7f4acd7fa700 1 -- 172.21.15.170:0/3681495183 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4ac004ad20 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.657+0000 7f4acffff700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4ab0038c20 0x7f4ab003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.657+0000 7f4acffff700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4ab0038c20 0x7f4ab003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8001eb0 tx=0x7f4ab800f4a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:13.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.662+0000 7f4acd7fa700 1 -- 172.21.15.170:0/3681495183 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4ac001a400 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:13 smithi067 ceph-mon[37808]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:13.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:13.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:13.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:13 smithi170 ceph-mon[78798]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.952+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ad00623e0 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.953+0000 7f4acd7fa700 1 -- 172.21.15.170:0/3681495183 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1270 (secure 0 0 0) 0x7f4ac000f2e0 con 0x7f4ad00ffbf0 2024-03-30T02:30:13.954 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-03-30T02:30:13.954 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","modified":"2024-03-30T02:30:07.303327Z","created":"2024-03-30T02:28:21.112353Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-30T02:30:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4ab0038c20 msgr2=0x7f4ab003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4ab0038c20 0x7f4ab003b0e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8001eb0 tx=0x7f4ab800f4a0 comp rx=0 tx=0).stop 2024-03-30T02:30:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ad00ffbf0 msgr2=0x7f4ad0100070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ad00ffbf0 0x7f4ad0100070 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000b550 tx=0x7f4ac000b910 comp rx=0 tx=0).stop 2024-03-30T02:30:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 shutdown_connections 2024-03-30T02:30:13.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4ab0038c20 0x7f4ab003b0e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ad01a23a0 0x7f4ad01a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4abc006370 0x7f4ad00ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 --2- 172.21.15.170:0/3681495183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ad00ffbf0 0x7f4ad0100070 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 >> 172.21.15.170:0/3681495183 conn(0x7f4ad0076390 msgr2=0x7f4ad0102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 shutdown_connections 2024-03-30T02:30:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:13.955+0000 7f4ad669d700 1 -- 172.21.15.170:0/3681495183 wait complete. 2024-03-30T02:30:13.957 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2024-03-30T02:30:14.323 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-30T02:30:14.324 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-03-30T02:30:14.697 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:30:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mon.c calling monitor election 2024-03-30T02:30:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mon.a calling monitor election 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: fsmap 2024-03-30T02:30:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mgrmap e13: x(active, since 45s) 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mon.b (rank 2) addr [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] is down (out of quorum) 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: mon.b calling monitor election 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: mon.b calling monitor election 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: mon.c calling monitor election 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: mon.a calling monitor election 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: fsmap 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: mgrmap e13: x(active, since 47s) 2024-03-30T02:30:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: Cluster is now healthy 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: overall HEALTH_OK 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T02:30:15.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mon.b calling monitor election 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mon.b calling monitor election 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mon.c calling monitor election 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mon.a calling monitor election 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: fsmap 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: mgrmap e13: x(active, since 47s) 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: Cluster is now healthy 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: overall HEALTH_OK 2024-03-30T02:30:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:15.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T02:30:15.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: mon.b calling monitor election 2024-03-30T02:30:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: mon.b calling monitor election 2024-03-30T02:30:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: mon.c calling monitor election 2024-03-30T02:30:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: mon.a calling monitor election 2024-03-30T02:30:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-30T02:30:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-30T02:30:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: fsmap 2024-03-30T02:30:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: osdmap e4: 0 total, 0 up, 0 in 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: mgrmap e13: x(active, since 47s) 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: Cluster is now healthy 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: overall HEALTH_OK 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T02:30:15.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.989+0000 7f5fd8600700 1 -- 172.21.15.67:0/1717491672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd0073150 msgr2=0x7f5fd0073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:15.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.989+0000 7f5fd8600700 1 --2- 172.21.15.67:0/1717491672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd0073150 0x7f5fd0073530 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009910 tx=0x7f5fc4009c20 comp rx=0 tx=0).stop 2024-03-30T02:30:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.989+0000 7f5fd8600700 1 -- 172.21.15.67:0/1717491672 shutdown_connections 2024-03-30T02:30:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.989+0000 7f5fd8600700 1 --2- 172.21.15.67:0/1717491672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd0073150 0x7f5fd0073530 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.989+0000 7f5fd8600700 1 -- 172.21.15.67:0/1717491672 >> 172.21.15.67:0/1717491672 conn(0x7f5fd00fc1b0 msgr2=0x7f5fd00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.989+0000 7f5fd8600700 1 -- 172.21.15.67:0/1717491672 shutdown_connections 2024-03-30T02:30:15.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.989+0000 7f5fd8600700 1 -- 172.21.15.67:0/1717491672 wait complete. 2024-03-30T02:30:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.990+0000 7f5fd8600700 1 Processor -- start 2024-03-30T02:30:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.990+0000 7f5fd8600700 1 -- start start 2024-03-30T02:30:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd8600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd0073150 0x7f5fd019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd8600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd019cbd0 0x7f5fd01a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd8600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd01a14c0 0x7f5fd01a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd8600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd019d2c0 con 0x7f5fd01a14c0 2024-03-30T02:30:15.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd8600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01a9230 con 0x7f5fd019cbd0 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd8600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01a9370 con 0x7f5fd0073150 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd6600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd01a14c0 0x7f5fd01a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd6600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd01a14c0 0x7f5fd01a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45308/0 (socket says 172.21.15.67:45308) 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd6600700 1 -- 172.21.15.67:0/4084925065 learned_addr learned my addr 172.21.15.67:0/4084925065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd5200700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd019cbd0 0x7f5fd01a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.991+0000 7f5fd5c00700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd0073150 0x7f5fd019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fd6600700 1 -- 172.21.15.67:0/4084925065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd0073150 msgr2=0x7f5fd019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fd6600700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd0073150 0x7f5fd019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fd6600700 1 -- 172.21.15.67:0/4084925065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd019cbd0 msgr2=0x7f5fd01a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fd6600700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd019cbd0 0x7f5fd01a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fd6600700 1 -- 172.21.15.67:0/4084925065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40095f0 con 0x7f5fd01a14c0 2024-03-30T02:30:15.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fd6600700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd01a14c0 0x7f5fd01a1970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ba50 tx=0x7f5fc800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fce000700 1 -- 172.21.15.67:0/4084925065 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800c930 con 0x7f5fd01a14c0 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fce000700 1 -- 172.21.15.67:0/4084925065 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fc8014440 con 0x7f5fd01a14c0 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd01a9650 con 0x7f5fd01a14c0 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fce000700 1 -- 172.21.15.67:0/4084925065 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc80075b0 con 0x7f5fd01a14c0 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.992+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd01a9b00 con 0x7f5fd01a14c0 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.993+0000 7f5fce000700 1 -- 172.21.15.67:0/4084925065 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f5fc800ca90 con 0x7f5fd01a14c0 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.993+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd0074890 con 0x7f5fd01a14c0 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.994+0000 7f5fce000700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5fb8038c70 0x7f5fb803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.994+0000 7f5fce000700 1 -- 172.21.15.67:0/4084925065 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5fc804a960 con 0x7f5fd01a14c0 2024-03-30T02:30:15.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.994+0000 7f5fd5c00700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5fb8038c70 0x7f5fb803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:15.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.995+0000 7f5fd5c00700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5fb8038c70 0x7f5fb803b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4004690 tx=0x7f5fc4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:16.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:15.999+0000 7f5fce000700 1 -- 172.21.15.67:0/4084925065 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5fc80148b0 con 0x7f5fd01a14c0 2024-03-30T02:30:16.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.217+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5fd0110fb0 con 0x7f5fd01a14c0 2024-03-30T02:30:16.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.217+0000 7f5fce000700 1 -- 172.21.15.67:0/4084925065 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f5fc8012340 con 0x7f5fd01a14c0 2024-03-30T02:30:16.218 INFO:teuthology.orchestra.run.smithi067.stdout:# minimal ceph.conf for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:30:16.218 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-03-30T02:30:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:30:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: mon_host = [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-03-30T02:30:16.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5fb8038c70 msgr2=0x7f5fb803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:16.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5fb8038c70 0x7f5fb803b130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4004690 tx=0x7f5fc4005fb0 comp rx=0 tx=0).stop 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd01a14c0 msgr2=0x7f5fd01a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd01a14c0 0x7f5fd01a1970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800ba50 tx=0x7f5fc800bd60 comp rx=0 tx=0).stop 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 shutdown_connections 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5fb8038c70 0x7f5fb803b130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5fd01a14c0 0x7f5fd01a1970 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fd0073150 0x7f5fd019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 --2- 172.21.15.67:0/4084925065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fd019cbd0 0x7f5fd01a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 >> 172.21.15.67:0/4084925065 conn(0x7f5fd00fc1b0 msgr2=0x7f5fd00fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 shutdown_connections 2024-03-30T02:30:16.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:16.220+0000 7f5fd8600700 1 -- 172.21.15.67:0/4084925065 wait complete. 2024-03-30T02:30:16.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:15 smithi067 ceph-mon[37808]: Reconfiguring mon.a (unknown last config time)... 2024-03-30T02:30:16.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:15 smithi067 ceph-mon[37808]: Reconfiguring daemon mon.a on smithi067 2024-03-30T02:30:16.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:15 smithi067 ceph-mon[37808]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:16.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:15 smithi082 ceph-mon[79176]: Reconfiguring mon.a (unknown last config time)... 2024-03-30T02:30:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:15 smithi082 ceph-mon[79176]: Reconfiguring daemon mon.a on smithi067 2024-03-30T02:30:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:15 smithi082 ceph-mon[79176]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:16.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:15 smithi170 ceph-mon[78798]: Reconfiguring mon.a (unknown last config time)... 2024-03-30T02:30:16.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:15 smithi170 ceph-mon[78798]: Reconfiguring daemon mon.a on smithi067 2024-03-30T02:30:16.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:15 smithi170 ceph-mon[78798]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:16.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:30:16.632 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-30T02:30:16.632 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:30:16.632 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.conf 2024-03-30T02:30:16.700 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:30:16.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:30:16.805 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:30:16.805 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2024-03-30T02:30:16.836 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:30:16.836 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:30:16.903 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:30:16.903 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2024-03-30T02:30:16.940 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:30:16.940 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:30:17.012 INFO:tasks.cephadm:Adding mgr.x on smithi067 2024-03-30T02:30:17.012 INFO:tasks.cephadm:Adding mgr.y on smithi082 2024-03-30T02:30:17.012 INFO:tasks.cephadm:Adding mgr.z on smithi170 2024-03-30T02:30:17.013 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch apply mgr '3;smithi067=x;smithi082=y;smithi170=z' 2024-03-30T02:30:17.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: Reconfiguring mon.b (monmap changed)... 2024-03-30T02:30:17.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:17.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T02:30:17.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:17.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: Reconfiguring daemon mon.b on smithi082 2024-03-30T02:30:17.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4084925065' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:17.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:17.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:17.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T02:30:17.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:17.276 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:30:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: Reconfiguring mon.b (monmap changed)... 2024-03-30T02:30:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T02:30:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: Reconfiguring daemon mon.b on smithi082 2024-03-30T02:30:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4084925065' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T02:30:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: Reconfiguring mon.b (monmap changed)... 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: Reconfiguring daemon mon.b on smithi082 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4084925065' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-30T02:30:17.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:18.363 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:18 smithi170 ceph-mon[78798]: Reconfiguring mon.c (monmap changed)... 2024-03-30T02:30:18.363 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:18 smithi170 ceph-mon[78798]: Reconfiguring daemon mon.c on smithi170 2024-03-30T02:30:18.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:18 smithi082 ceph-mon[79176]: Reconfiguring mon.c (monmap changed)... 2024-03-30T02:30:18.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:18 smithi082 ceph-mon[79176]: Reconfiguring daemon mon.c on smithi170 2024-03-30T02:30:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 -- 172.21.15.170:0/448428543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0106ac0 msgr2=0x7f2ff0108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/448428543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0106ac0 0x7f2ff0108f30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800b0a0 tx=0x7f2fe800b3b0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 -- 172.21.15.170:0/448428543 shutdown_connections 2024-03-30T02:30:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/448428543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff0101860 0x7f2ff0103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/448428543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0106ac0 0x7f2ff0108f30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/448428543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff0104190 0x7f2ff0106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 -- 172.21.15.170:0/448428543 >> 172.21.15.170:0/448428543 conn(0x7f2ff00fb2b0 msgr2=0x7f2ff00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:18.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 -- 172.21.15.170:0/448428543 shutdown_connections 2024-03-30T02:30:18.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 -- 172.21.15.170:0/448428543 wait complete. 2024-03-30T02:30:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.563+0000 7f2ff5a86700 1 Processor -- start 2024-03-30T02:30:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2ff5a86700 1 -- start start 2024-03-30T02:30:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2ff5a86700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0101860 0x7f2ff0196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2ff5a86700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff0104190 0x7f2ff01973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2ff5a86700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff0106ac0 0x7f2ff019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2ff5a86700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff019e9d0 con 0x7f2ff0104190 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2ff5a86700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff019eb10 con 0x7f2ff0106ac0 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2ff5a86700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff019eca0 con 0x7f2ff0101860 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2fef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0101860 0x7f2ff0196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2fef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0101860 0x7f2ff0196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53298/0 (socket says 172.21.15.170:53298) 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2fef7fe700 1 -- 172.21.15.170:0/2440593269 learned_addr learned my addr 172.21.15.170:0/2440593269 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2feeffd700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff0104190 0x7f2ff01973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2fef7fe700 1 -- 172.21.15.170:0/2440593269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff0106ac0 msgr2=0x7f2ff019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2fef7fe700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff0106ac0 0x7f2ff019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2fef7fe700 1 -- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff0104190 msgr2=0x7f2ff01973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2fef7fe700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff0104190 0x7f2ff01973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2fef7fe700 1 -- 172.21.15.170:0/2440593269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe8009d00 con 0x7f2ff0101860 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2feeffd700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff0104190 0x7f2ff01973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:30:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.564+0000 7f2fef7fe700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0101860 0x7f2ff0196e90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000b560 tx=0x7f2fe000b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.565+0000 7f2fecff9700 1 -- 172.21.15.170:0/2440593269 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0011830 con 0x7f2ff0101860 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.565+0000 7f2fecff9700 1 -- 172.21.15.170:0/2440593269 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fe0011e50 con 0x7f2ff0101860 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.565+0000 7f2fecff9700 1 -- 172.21.15.170:0/2440593269 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000f2e0 con 0x7f2ff0101860 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.565+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff00ff6b0 con 0x7f2ff0101860 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.565+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff00ffbb0 con 0x7f2ff0101860 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.566+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff00fce30 con 0x7f2ff0101860 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.566+0000 7f2fecff9700 1 -- 172.21.15.170:0/2440593269 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f2fe000f440 con 0x7f2ff0101860 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.566+0000 7f2fecff9700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80389d0 0x7f2fd803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.566+0000 7f2fecff9700 1 -- 172.21.15.170:0/2440593269 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2fe004ad90 con 0x7f2ff0101860 2024-03-30T02:30:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.566+0000 7f2feeffd700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80389d0 0x7f2fd803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:18.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.567+0000 7f2feeffd700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80389d0 0x7f2fd803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4005fd0 tx=0x7f2fe4005f60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:18.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.571+0000 7f2fecff9700 1 -- 172.21.15.170:0/2440593269 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2fe00178b0 con 0x7f2ff0101860 2024-03-30T02:30:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:18 smithi067 ceph-mon[37808]: Reconfiguring mon.c (monmap changed)... 2024-03-30T02:30:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:18 smithi067 ceph-mon[37808]: Reconfiguring daemon mon.c on smithi170 2024-03-30T02:30:18.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.778+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi082=y;smithi170=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ff002cd20 con 0x7f2fd80389d0 2024-03-30T02:30:18.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.786+0000 7f2fecff9700 1 -- 172.21.15.170:0/2440593269 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2ff002cd20 con 0x7f2fd80389d0 2024-03-30T02:30:18.788 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2024-03-30T02:30:18.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80389d0 msgr2=0x7f2fd803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:18.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80389d0 0x7f2fd803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4005fd0 tx=0x7f2fe4005f60 comp rx=0 tx=0).stop 2024-03-30T02:30:18.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0101860 msgr2=0x7f2ff0196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:18.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0101860 0x7f2ff0196e90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000b560 tx=0x7f2fe000b920 comp rx=0 tx=0).stop 2024-03-30T02:30:18.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 shutdown_connections 2024-03-30T02:30:18.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80389d0 0x7f2fd803ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.790 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff0106ac0 0x7f2ff019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff0101860 0x7f2ff0196e90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 --2- 172.21.15.170:0/2440593269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff0104190 0x7f2ff01973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:18.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 >> 172.21.15.170:0/2440593269 conn(0x7f2ff00fb2b0 msgr2=0x7f2ff0102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:18.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 shutdown_connections 2024-03-30T02:30:18.791 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:30:18.789+0000 7f2ff5a86700 1 -- 172.21.15.170:0/2440593269 wait complete. 2024-03-30T02:30:19.156 DEBUG:teuthology.orchestra.run.smithi082:mgr.y> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.y.service 2024-03-30T02:30:19.159 DEBUG:teuthology.orchestra.run.smithi170:mgr.z> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.z.service 2024-03-30T02:30:19.161 INFO:tasks.cephadm:Deploying OSDs... 2024-03-30T02:30:19.161 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:30:19.162 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T02:30:19.193 INFO:journalctl@ceph.mgr.z.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:30:19.193 INFO:journalctl@ceph.mgr.y.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:30:19.208 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T02:30:19.208 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-03-30T02:30:19.294 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T02:30:19.294 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:19.294 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 448 Links: 1 2024-03-30T02:30:19.295 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:19.295 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:19.295 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-03-30 02:29:37.749171190 +0000 2024-03-30T02:30:19.295 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-30 02:22:44.029991547 +0000 2024-03-30T02:30:19.295 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-30 02:22:44.029991547 +0000 2024-03-30T02:30:19.295 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-30 02:22:44.005992078 +0000 2024-03-30T02:30:19.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi082=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: Saving service mgr spec with placement smithi067=x;smithi082=y;smithi170=z;count:3 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:19.372 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:19.373 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:19.373 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.373 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T02:30:19.373 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T02:30:19.373 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T02:30:19.373 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:19.373 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: Deploying daemon mgr.z on smithi170 2024-03-30T02:30:19.373 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:19 smithi170 ceph-mon[78798]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-03-30T02:30:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-03-30T02:30:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000513635 s, 997 kB/s 2024-03-30T02:30:19.405 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T02:30:19.449 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-03-30T02:30:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi082=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: Saving service mgr spec with placement smithi067=x;smithi082=y;smithi170=z;count:3 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T02:30:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:19.502 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: Deploying daemon mgr.z on smithi170 2024-03-30T02:30:19.502 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:19 smithi082 ceph-mon[79176]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:19.532 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T02:30:19.532 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:19.532 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 454 Links: 1 2024-03-30T02:30:19.532 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:19.532 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:19.532 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-03-30 02:29:37.846169027 +0000 2024-03-30T02:30:19.532 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-30 02:22:44.516980763 +0000 2024-03-30T02:30:19.532 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-30 02:22:44.516980763 +0000 2024-03-30T02:30:19.532 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-30 02:22:44.488981383 +0000 2024-03-30T02:30:19.533 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T02:30:19.629 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.629 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.629 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi082=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: Saving service mgr spec with placement smithi067=x;smithi082=y;smithi170=z;count:3 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:19.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:19.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T02:30:19.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T02:30:19.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T02:30:19.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:19.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: Deploying daemon mgr.z on smithi170 2024-03-30T02:30:19.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:19 smithi067 ceph-mon[37808]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-03-30T02:30:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-03-30T02:30:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000280559 s, 1.8 MB/s 2024-03-30T02:30:19.638 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T02:30:19.681 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-03-30T02:30:19.763 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T02:30:19.764 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:19.764 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 460 Links: 1 2024-03-30T02:30:19.764 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:19.764 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:19.764 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-03-30 02:29:37.928167199 +0000 2024-03-30T02:30:19.764 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-30 02:22:44.998970090 +0000 2024-03-30T02:30:19.764 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-30 02:22:44.998970090 +0000 2024-03-30T02:30:19.764 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-30 02:22:44.973970644 +0000 2024-03-30T02:30:19.764 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T02:30:19.866 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-03-30T02:30:19.866 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-03-30T02:30:19.866 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000279751 s, 1.8 MB/s 2024-03-30T02:30:19.869 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T02:30:19.912 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-03-30T02:30:19.995 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T02:30:19.995 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:19.995 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 466 Links: 1 2024-03-30T02:30:19.995 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:19.995 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:19.995 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-03-30 02:29:38.013165303 +0000 2024-03-30T02:30:19.995 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-30 02:22:45.482959373 +0000 2024-03-30T02:30:19.995 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-30 02:22:45.482959373 +0000 2024-03-30T02:30:19.995 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-30 02:22:45.456959949 +0000 2024-03-30T02:30:19.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T02:30:20.097 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-03-30T02:30:20.097 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-03-30T02:30:20.097 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000296099 s, 1.7 MB/s 2024-03-30T02:30:20.099 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T02:30:20.143 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-30T02:30:20.143 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T02:30:20.164 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T02:30:20.164 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-03-30T02:30:20.225 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T02:30:20.225 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:20.225 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 213136 Links: 1 2024-03-30T02:30:20.225 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:20.225 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:20.225 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-30 02:30:06.901922622 +0000 2024-03-30T02:30:20.225 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-30 02:21:02.946744216 +0000 2024-03-30T02:30:20.226 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-30 02:21:02.946744216 +0000 2024-03-30T02:30:20.226 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-03-30T02:30:20.226 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T02:30:20.295 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-30T02:30:20.295 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-30T02:30:20.295 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000278021 s, 1.8 MB/s 2024-03-30T02:30:20.296 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T02:30:20.356 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-03-30T02:30:20.415 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T02:30:20.415 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:20.415 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 213210 Links: 1 2024-03-30T02:30:20.416 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:20.416 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:20.416 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-30 02:30:06.901922622 +0000 2024-03-30T02:30:20.416 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-30 02:21:03.294737436 +0000 2024-03-30T02:30:20.416 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-30 02:21:03.294737436 +0000 2024-03-30T02:30:20.416 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-03-30T02:30:20.416 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T02:30:20.440 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:20 smithi170 systemd[1]: Starting Ceph mgr.z for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:30:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-30T02:30:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-30T02:30:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000251155 s, 2.0 MB/s 2024-03-30T02:30:20.485 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T02:30:20.545 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-03-30T02:30:20.604 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T02:30:20.604 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:20.604 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 209346 Links: 1 2024-03-30T02:30:20.604 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:20.604 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:20.604 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-30 02:30:06.901922622 +0000 2024-03-30T02:30:20.604 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-30 02:21:03.645730598 +0000 2024-03-30T02:30:20.604 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-30 02:21:03.645730598 +0000 2024-03-30T02:30:20.604 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-03-30T02:30:20.604 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T02:30:20.672 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-30T02:30:20.673 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-30T02:30:20.673 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000188317 s, 2.7 MB/s 2024-03-30T02:30:20.674 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T02:30:20.733 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-03-30T02:30:20.792 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T02:30:20.792 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:20.792 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 209386 Links: 1 2024-03-30T02:30:20.792 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:20.792 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:20.792 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-30 02:30:06.901922622 +0000 2024-03-30T02:30:20.793 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-30 02:21:03.977724130 +0000 2024-03-30T02:30:20.793 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-30 02:21:03.977724130 +0000 2024-03-30T02:30:20.793 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-03-30T02:30:20.793 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T02:30:20.859 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-30T02:30:20.859 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-30T02:30:20.859 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000212416 s, 2.4 MB/s 2024-03-30T02:30:20.860 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T02:30:20.919 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-03-30T02:30:20.920 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T02:30:20.938 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T02:30:20.938 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-03-30T02:30:20.939 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:20 smithi170 ceph-mgr[80689]: -- 172.21.15.170:0/2225699251 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x565001a261a0 con 0x5650019fd000 2024-03-30T02:30:20.940 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:20 smithi170 bash[80651]: 51a960dd8f634a5065623890048d199f36ce532164c4e7f61458e5ae0de8ec06 2024-03-30T02:30:20.940 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:20 smithi170 systemd[1]: Started Ceph mgr.z for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:30:20.956 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T02:30:20.957 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:20.957 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 207523 Links: 1 2024-03-30T02:30:20.957 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:20.957 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:20.957 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-30 02:30:20.165289442 +0000 2024-03-30T02:30:20.957 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-30 02:21:09.694380488 +0000 2024-03-30T02:30:20.957 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-30 02:21:09.694380488 +0000 2024-03-30T02:30:20.957 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-03-30T02:30:20.957 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T02:30:21.024 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:20.982+0000 7f860745b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T02:30:21.026 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-30T02:30:21.027 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-30T02:30:21.027 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.2906e-05 s, 5.5 MB/s 2024-03-30T02:30:21.028 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T02:30:21.085 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-03-30T02:30:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T02:30:21.143 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:21.143 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 207616 Links: 1 2024-03-30T02:30:21.143 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:21.143 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:21.143 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-30 02:30:20.165289442 +0000 2024-03-30T02:30:21.143 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-30 02:21:10.049372281 +0000 2024-03-30T02:30:21.144 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-30 02:21:10.049372281 +0000 2024-03-30T02:30:21.144 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-03-30T02:30:21.144 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T02:30:21.209 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-30T02:30:21.209 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-30T02:30:21.209 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.3888e-05 s, 5.5 MB/s 2024-03-30T02:30:21.211 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T02:30:21.258 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:21.079+0000 7f860745b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T02:30:21.258 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:21.169+0000 7f860745b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T02:30:21.258 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:21.257+0000 7f860745b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T02:30:21.269 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-03-30T02:30:21.327 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T02:30:21.327 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:21.327 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 206836 Links: 1 2024-03-30T02:30:21.327 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:21.327 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:21.327 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-30 02:30:20.165289442 +0000 2024-03-30T02:30:21.327 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-30 02:21:10.398364212 +0000 2024-03-30T02:30:21.328 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-30 02:21:10.398364212 +0000 2024-03-30T02:30:21.328 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-03-30T02:30:21.328 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T02:30:21.393 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-30T02:30:21.393 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-30T02:30:21.393 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000106296 s, 4.8 MB/s 2024-03-30T02:30:21.394 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T02:30:21.452 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-03-30T02:30:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T02:30:21.510 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T02:30:21.510 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 213065 Links: 1 2024-03-30T02:30:21.510 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:30:21.510 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-03-30T02:30:21.510 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-03-30 02:30:20.165289442 +0000 2024-03-30T02:30:21.510 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-03-30 02:21:10.764355750 +0000 2024-03-30T02:30:21.510 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-03-30 02:21:10.764355750 +0000 2024-03-30T02:30:21.511 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2024-03-30T02:30:21.511 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T02:30:21.578 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-03-30T02:30:21.578 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-03-30T02:30:21.578 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000133988 s, 3.8 MB/s 2024-03-30T02:30:21.579 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T02:30:21.636 INFO:tasks.cephadm:Deploying osd.0 on smithi067 with /dev/vg_nvme/lv_4... 2024-03-30T02:30:21.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-30T02:30:21.913 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:21.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:21.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:21.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T02:30:21.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T02:30:21.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T02:30:21.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:21.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-mon[78798]: Deploying daemon mgr.y on smithi082 2024-03-30T02:30:21.914 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-mon[78798]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:21.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:21.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:21.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:21.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T02:30:21.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T02:30:21.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T02:30:21.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:21.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:21 smithi067 ceph-mon[37808]: Deploying daemon mgr.y on smithi082 2024-03-30T02:30:21.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:21 smithi067 ceph-mon[37808]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:22.000 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:30:22.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:22.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:22.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:22.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T02:30:22.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-30T02:30:22.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T02:30:22.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:22.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:21 smithi082 ceph-mon[79176]: Deploying daemon mgr.y on smithi082 2024-03-30T02:30:22.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:21 smithi082 ceph-mon[79176]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:22.002 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:21 smithi082 systemd[1]: Starting Ceph mgr.y for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:30:22.190 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:21.912+0000 7f860745b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:30:22.190 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:22.162+0000 7f860745b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T02:30:22.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:22 smithi082 ceph-mgr[80680]: -- 172.21.15.82:0/1716624624 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55641199c1a0 con 0x556411973000 2024-03-30T02:30:22.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:22 smithi082 bash[80643]: 31212d440ea050873f7f0085fab3f683e99630f3f2eb9e2fae18231472059ef0 2024-03-30T02:30:22.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:22 smithi082 systemd[1]: Started Ceph mgr.y for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:30:22.514 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:22.247+0000 7f860745b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T02:30:22.515 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:22.355+0000 7f860745b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T02:30:22.515 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:22.514+0000 7f860745b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T02:30:22.713 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:22 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:22.613+0000 7fdbefc82200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T02:30:22.904 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:22 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:22.711+0000 7fdbefc82200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T02:30:22.904 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:22 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:22.807+0000 7fdbefc82200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T02:30:22.904 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:22 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:22.902+0000 7fdbefc82200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T02:30:22.940 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:22.658+0000 7f860745b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T02:30:22.940 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:22.747+0000 7f860745b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T02:30:23.440 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:23.166+0000 7f860745b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T02:30:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-mon[78798]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:23.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:23 smithi067 ceph-mon[37808]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.750 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:23.571+0000 7fdbefc82200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:30:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-mon[79176]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:23.770 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-30T02:30:23.770 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-30T02:30:23.770 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-03-30T02:30:23.770 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-03-30T02:30:23.771 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.050878 s, 206 MB/s 2024-03-30T02:30:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-03-30T02:30:23.859 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:23.569+0000 7f860745b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T02:30:23.859 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:23.764+0000 7f860745b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T02:30:24.163 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:23.819+0000 7fdbefc82200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T02:30:24.164 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:23 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:23.902+0000 7fdbefc82200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T02:30:24.164 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:24.008+0000 7fdbefc82200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T02:30:24.164 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:24.163+0000 7fdbefc82200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T02:30:24.183 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi067:vg_nvme/lv_4 2024-03-30T02:30:24.189 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:23.858+0000 7f860745b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T02:30:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: Reconfiguring mgr.x (unknown last config time)... 2024-03-30T02:30:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T02:30:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T02:30:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:24.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:24 smithi067 ceph-mon[37808]: Reconfiguring daemon mgr.x on smithi067 2024-03-30T02:30:24.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:24.288+0000 7fdbefc82200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T02:30:24.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:24.371+0000 7fdbefc82200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T02:30:24.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:24.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:24.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:24.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:24.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:24.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: Reconfiguring mgr.x (unknown last config time)... 2024-03-30T02:30:24.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T02:30:24.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T02:30:24.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:24.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-mon[79176]: Reconfiguring daemon mgr.x on smithi067 2024-03-30T02:30:24.690 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:24.625+0000 7f860745b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T02:30:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: Reconfiguring mgr.x (unknown last config time)... 2024-03-30T02:30:24.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-30T02:30:24.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-30T02:30:24.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:24.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-mon[78798]: Reconfiguring daemon mgr.x on smithi067 2024-03-30T02:30:24.882 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:30:25.000 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:24 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:24.736+0000 7fdbefc82200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T02:30:25.190 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:24.721+0000 7f860745b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T02:30:25.190 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:24 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:24.868+0000 7f860745b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T02:30:25.407 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:25 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:25.118+0000 7fdbefc82200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T02:30:25.407 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:25 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:25.315+0000 7fdbefc82200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T02:30:25.407 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:25 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:25.406+0000 7fdbefc82200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T02:30:25.870 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:25 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:25.498+0000 7f860745b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T02:30:25.870 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:25 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:25.679+0000 7f860745b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T02:30:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.031+0000 7f196be00700 1 -- 172.21.15.67:0/2140164203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c1027c0 msgr2=0x7f196c109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.031+0000 7f196be00700 1 --2- 172.21.15.67:0/2140164203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c1027c0 0x7f196c109070 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f196400b0d0 tx=0x7f196400b3e0 comp rx=0 tx=0).stop 2024-03-30T02:30:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.031+0000 7f196be00700 1 -- 172.21.15.67:0/2140164203 shutdown_connections 2024-03-30T02:30:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.031+0000 7f196be00700 1 --2- 172.21.15.67:0/2140164203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c1027c0 0x7f196c109070 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.031+0000 7f196be00700 1 --2- 172.21.15.67:0/2140164203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196c100c20 0x7f196c101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.031+0000 7f196be00700 1 --2- 172.21.15.67:0/2140164203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f196c101e20 0x7f196c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.031+0000 7f196be00700 1 -- 172.21.15.67:0/2140164203 >> 172.21.15.67:0/2140164203 conn(0x7f196c0fc1c0 msgr2=0x7f196c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.031+0000 7f196be00700 1 -- 172.21.15.67:0/2140164203 shutdown_connections 2024-03-30T02:30:26.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.031+0000 7f196be00700 1 -- 172.21.15.67:0/2140164203 wait complete. 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.032+0000 7f196be00700 1 Processor -- start 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.032+0000 7f196be00700 1 -- start start 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.032+0000 7f196be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c100c20 0x7f196c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f196c101e20 0x7f196c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196c1027c0 0x7f196c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196be00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c19e760 con 0x7f196c101e20 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196be00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c19e8a0 con 0x7f196c100c20 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196be00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c19ea30 con 0x7f196c1027c0 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f196c101e20 0x7f196c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c100c20 0x7f196c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f196c101e20 0x7f196c197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45266/0 (socket says 172.21.15.67:45266) 2024-03-30T02:30:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196a000700 1 -- 172.21.15.67:0/1913999641 learned_addr learned my addr 172.21.15.67:0/1913999641 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c100c20 0x7f196c196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43398/0 (socket says 172.21.15.67:43398) 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196b400700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196c1027c0 0x7f196c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196aa00700 1 -- 172.21.15.67:0/1913999641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196c1027c0 msgr2=0x7f196c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196aa00700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196c1027c0 0x7f196c19e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196aa00700 1 -- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f196c101e20 msgr2=0x7f196c197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196aa00700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f196c101e20 0x7f196c197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196aa00700 1 -- 172.21.15.67:0/1913999641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958009520 con 0x7f196c100c20 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196b400700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196c1027c0 0x7f196c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196a000700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f196c101e20 0x7f196c197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.033+0000 7f196aa00700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c100c20 0x7f196c196c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f195c00e9c0 tx=0x7f195c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.034+0000 7f1963400700 1 -- 172.21.15.67:0/1913999641 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c004220 con 0x7f196c100c20 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.034+0000 7f1963400700 1 -- 172.21.15.67:0/1913999641 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f195c004380 con 0x7f196c100c20 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.034+0000 7f1963400700 1 -- 172.21.15.67:0/1913999641 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c003c50 con 0x7f196c100c20 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.034+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1964009d00 con 0x7f196c100c20 2024-03-30T02:30:26.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.034+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c19f030 con 0x7f196c100c20 2024-03-30T02:30:26.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.035+0000 7f1963400700 1 -- 172.21.15.67:0/1913999641 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45589+0+0 (secure 0 0 0) 0x7f195c00c7c0 con 0x7f196c100c20 2024-03-30T02:30:26.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.035+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c108060 con 0x7f196c100c20 2024-03-30T02:30:26.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.035+0000 7f1963400700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f19500389d0 0x7f195003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:26.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.035+0000 7f1963400700 1 -- 172.21.15.67:0/1913999641 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f195c014070 con 0x7f196c100c20 2024-03-30T02:30:26.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.036+0000 7f196a000700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f19500389d0 0x7f195003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:26.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.036+0000 7f196a000700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f19500389d0 0x7f195003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1958009fd0 tx=0x7f1958009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.041+0000 7f1963400700 1 -- 172.21.15.67:0/1913999641 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f195c00f5a0 con 0x7f196c100c20 2024-03-30T02:30:26.190 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:25 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:25.869+0000 7f860745b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T02:30:26.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:26 smithi170 ceph-mon[78798]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:26.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:26 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:26.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:26 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:26.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:26 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:26.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:26 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:26.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:26 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:26.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:26 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:26.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:26.263+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f196c02ce20 con 0x7f19500389d0 2024-03-30T02:30:26.397 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:26.156+0000 7fdbefc82200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T02:30:26.397 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:26.251+0000 7fdbefc82200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T02:30:26.397 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:26.396+0000 7fdbefc82200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T02:30:26.398 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-mon[79176]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:26.398 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:26.398 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:26.398 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:26.398 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:26.398 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:26.398 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:26 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:26.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:26 smithi067 ceph-mon[37808]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:26.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:26.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:26.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:26.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:26.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:26.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:27.050 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:26 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:26.795+0000 7f860745b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T02:30:27.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:30:27.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:30:27.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:27.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:27.357 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:27 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:27.077+0000 7f860745b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:30:27.357 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:27 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:27.165+0000 7f860745b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T02:30:27.358 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:30:27.358 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:30:27.358 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:27.358 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:27.378 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:27 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:27.009+0000 7fdbefc82200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T02:30:27.378 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:27 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:27.188+0000 7fdbefc82200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T02:30:27.379 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:30:27.379 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:30:27.379 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:27.379 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:27.690 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:30:27 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:30:27.356+0000 7f860745b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T02:30:27.750 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:27 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:27.377+0000 7fdbefc82200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T02:30:28.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:28.055+0000 7f1963400700 1 -- 172.21.15.67:0/1913999641 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 14) v1 ==== 90699+0+0 (secure 0 0 0) 0x7f195c017cd0 con 0x7f196c100c20 2024-03-30T02:30:28.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:28 smithi067 ceph-mon[37808]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:28.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:28 smithi067 ceph-mon[37808]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:28.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:28 smithi067 ceph-mon[37808]: Standby manager daemon z started 2024-03-30T02:30:28.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:28 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-30T02:30:28.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:28 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:30:28.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:28 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-30T02:30:28.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:28 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:30:28.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:28 smithi170 ceph-mon[78798]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:28.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:28 smithi170 ceph-mon[78798]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:28.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:28 smithi170 ceph-mon[78798]: Standby manager daemon z started 2024-03-30T02:30:28.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:28 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-30T02:30:28.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:28 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:30:28.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:28 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-30T02:30:28.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:28 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:30:28.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:28.296+0000 7fdbefc82200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T02:30:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-mon[79176]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:28.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-mon[79176]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:28.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-mon[79176]: Standby manager daemon z started 2024-03-30T02:30:28.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-30T02:30:28.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:30:28.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-30T02:30:28.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.170:0/2897901600' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:30:28.858 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:28.581+0000 7fdbefc82200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:30:28.858 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:28.668+0000 7fdbefc82200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T02:30:29.250 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:30:28 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:30:28.857+0000 7fdbefc82200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: mgrmap e14: x(active, since 61s), standbys: z 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3191180784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5709d485-60d6-4959-b2dd-a5aeb11363af"}]: dispatch 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5709d485-60d6-4959-b2dd-a5aeb11363af"}]: dispatch 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5709d485-60d6-4959-b2dd-a5aeb11363af"}]': finished 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: osdmap e5: 1 total, 0 up, 1 in 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: Standby manager daemon y started 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-30T02:30:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:29 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: mgrmap e14: x(active, since 61s), standbys: z 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3191180784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5709d485-60d6-4959-b2dd-a5aeb11363af"}]: dispatch 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5709d485-60d6-4959-b2dd-a5aeb11363af"}]: dispatch 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5709d485-60d6-4959-b2dd-a5aeb11363af"}]': finished 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: osdmap e5: 1 total, 0 up, 1 in 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: Standby manager daemon y started 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-30T02:30:29.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:30:29.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-30T02:30:29.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:29 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:30:29.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: mgrmap e14: x(active, since 61s), standbys: z 2024-03-30T02:30:29.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-30T02:30:29.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3191180784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5709d485-60d6-4959-b2dd-a5aeb11363af"}]: dispatch 2024-03-30T02:30:29.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5709d485-60d6-4959-b2dd-a5aeb11363af"}]: dispatch 2024-03-30T02:30:29.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5709d485-60d6-4959-b2dd-a5aeb11363af"}]': finished 2024-03-30T02:30:29.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: osdmap e5: 1 total, 0 up, 1 in 2024-03-30T02:30:29.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:29.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: Standby manager daemon y started 2024-03-30T02:30:29.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-30T02:30:29.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:30:29.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-30T02:30:29.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:29 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.82:0/1227965162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:30:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:29.690+0000 7f1963400700 1 -- 172.21.15.67:0/1913999641 <== mon.2 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7f195c018430 con 0x7f196c100c20 2024-03-30T02:30:30.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:30 smithi067 ceph-mon[37808]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:30.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:30 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1475664681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:30:30.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:30 smithi067 ceph-mon[37808]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-03-30T02:30:30.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:30 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-30T02:30:30.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:30 smithi170 ceph-mon[78798]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:30.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:30 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1475664681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:30:30.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:30 smithi170 ceph-mon[78798]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-03-30T02:30:30.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:30 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-30T02:30:30.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:30 smithi082 ceph-mon[79176]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:30.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:30 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1475664681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:30:30.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:30 smithi082 ceph-mon[79176]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-03-30T02:30:30.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:30 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-30T02:30:32.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:31 smithi067 ceph-mon[37808]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:32.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:31 smithi170 ceph-mon[78798]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:32.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:31 smithi082 ceph-mon[79176]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:34.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:34 smithi067 ceph-mon[37808]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:34.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:34 smithi170 ceph-mon[78798]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:34 smithi082 ceph-mon[79176]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:36.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:36 smithi067 ceph-mon[37808]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:36.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:36 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-30T02:30:36.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:36 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:36.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:36 smithi170 ceph-mon[78798]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:36.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:36 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-30T02:30:36.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:36 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:36 smithi082 ceph-mon[79176]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:36 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-30T02:30:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:36 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:37.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:37 smithi067 ceph-mon[37808]: Deploying daemon osd.0 on smithi067 2024-03-30T02:30:37.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:37 smithi170 ceph-mon[78798]: Deploying daemon osd.0 on smithi067 2024-03-30T02:30:37.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:37 smithi082 ceph-mon[79176]: Deploying daemon osd.0 on smithi067 2024-03-30T02:30:38.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:38 smithi067 ceph-mon[37808]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:38 smithi170 ceph-mon[78798]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:38 smithi082 ceph-mon[79176]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:40.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:40 smithi067 ceph-mon[37808]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:40 smithi170 ceph-mon[78798]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:40.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:40 smithi082 ceph-mon[79176]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:41.037 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:41 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:41.037 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:41 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:41.037 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:41 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:41.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:41 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:41.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:41 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:41.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:41 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:41.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:41 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:41.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:41 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:41.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:41 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:42.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:42 smithi067 ceph-mon[37808]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:42.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:42.397 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:42.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:42 smithi170 ceph-mon[78798]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:42.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:42.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:42 smithi082 ceph-mon[79176]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:42.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.762+0000 7f1963400700 1 -- 172.21.15.67:0/1913999641 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f196c02ce20 con 0x7f19500389d0 2024-03-30T02:30:42.764 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 0 on host 'smithi067' 2024-03-30T02:30:42.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f19500389d0 msgr2=0x7f195003ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:42.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f19500389d0 0x7f195003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1958009fd0 tx=0x7f1958009260 comp rx=0 tx=0).stop 2024-03-30T02:30:42.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c100c20 msgr2=0x7f196c196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:42.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c100c20 0x7f196c196c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f195c00e9c0 tx=0x7f195c00ecd0 comp rx=0 tx=0).stop 2024-03-30T02:30:42.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 shutdown_connections 2024-03-30T02:30:42.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f19500389d0 0x7f195003ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:42.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f196c1027c0 0x7f196c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:42.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f196c100c20 0x7f196c196c20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:42.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 --2- 172.21.15.67:0/1913999641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f196c101e20 0x7f196c197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:42.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.764+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 >> 172.21.15.67:0/1913999641 conn(0x7f196c0fc1c0 msgr2=0x7f196c1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:42.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.765+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 shutdown_connections 2024-03-30T02:30:42.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:30:42.765+0000 7f196be00700 1 -- 172.21.15.67:0/1913999641 wait complete. 2024-03-30T02:30:43.183 DEBUG:teuthology.orchestra.run.smithi067:osd.0> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.0.service 2024-03-30T02:30:43.187 INFO:tasks.cephadm:Deploying osd.1 on smithi082 with /dev/vg_nvme/lv_4... 2024-03-30T02:30:43.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-30T02:30:43.261 INFO:journalctl@ceph.osd.0.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:30:43.413 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:30:43.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:43.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:43.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:43.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:43.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:43.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:43.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:43.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:43.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:43.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:43.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:44.138 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:44 smithi082 ceph-mon[79176]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:44.338 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:44 smithi067 ceph-mon[37808]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:44.339 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:30:44 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-0[56175]: 2024-03-30T02:30:44.121+0000 7ff333638640 -1 osd.0 0 log_to_monitors true 2024-03-30T02:30:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:44 smithi170 ceph-mon[78798]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-30T02:30:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-30T02:30:44.656 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-03-30T02:30:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-03-30T02:30:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111237 s, 943 MB/s 2024-03-30T02:30:44.656 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 2024-03-30T02:30:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-03-30T02:30:45.054 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2024-03-30T02:30:45.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:45 smithi082 ceph-mon[79176]: from='osd.0 [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T02:30:45.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:45 smithi082 ceph-mon[79176]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T02:30:45.268 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:30:45.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:45 smithi067 ceph-mon[37808]: from='osd.0 [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T02:30:45.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:45 smithi067 ceph-mon[37808]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T02:30:45.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:45 smithi170 ceph-mon[78798]: from='osd.0 [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T02:30:45.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:45 smithi170 ceph-mon[78798]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-30T02:30:46.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.171+0000 7ffa585d0700 1 -- 172.21.15.82:0/422428835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa500ffcf0 msgr2=0x7ffa50100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:46.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.171+0000 7ffa585d0700 1 --2- 172.21.15.82:0/422428835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa500ffcf0 0x7ffa50100110 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffa400099c0 tx=0x7ffa40009cd0 comp rx=0 tx=0).stop 2024-03-30T02:30:46.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.171+0000 7ffa585d0700 1 -- 172.21.15.82:0/422428835 shutdown_connections 2024-03-30T02:30:46.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.171+0000 7ffa585d0700 1 --2- 172.21.15.82:0/422428835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa50101890 0x7ffa50108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:46.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.171+0000 7ffa585d0700 1 --2- 172.21.15.82:0/422428835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa500ffcf0 0x7ffa50100110 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:46.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.171+0000 7ffa585d0700 1 --2- 172.21.15.82:0/422428835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa50100ef0 0x7ffa50101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:46.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.171+0000 7ffa585d0700 1 -- 172.21.15.82:0/422428835 >> 172.21.15.82:0/422428835 conn(0x7ffa500fb2b0 msgr2=0x7ffa500fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:46.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.171+0000 7ffa585d0700 1 -- 172.21.15.82:0/422428835 shutdown_connections 2024-03-30T02:30:46.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.171+0000 7ffa585d0700 1 -- 172.21.15.82:0/422428835 wait complete. 2024-03-30T02:30:46.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa585d0700 1 Processor -- start 2024-03-30T02:30:46.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa585d0700 1 -- start start 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa585d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa500ffcf0 0x7ffa50069260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa585d0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa50100ef0 0x7ffa5019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa585d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa50101890 0x7ffa5019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa585d0700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5019e820 con 0x7ffa50100ef0 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa585d0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5019e990 con 0x7ffa50101890 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa585d0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5019eb50 con 0x7ffa500ffcf0 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa56b6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa50101890 0x7ffa5019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa56b6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa50101890 0x7ffa5019e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54844/0 (socket says 172.21.15.82:54844) 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa56b6d700 1 -- 172.21.15.82:0/898924431 learned_addr learned my addr 172.21.15.82:0/898924431 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa5636c700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa500ffcf0 0x7ffa50069260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa56b6d700 1 -- 172.21.15.82:0/898924431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa500ffcf0 msgr2=0x7ffa50069260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa56b6d700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa500ffcf0 0x7ffa50069260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:46.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa56b6d700 1 -- 172.21.15.82:0/898924431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa50100ef0 msgr2=0x7ffa5019bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:46.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa56b6d700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa50100ef0 0x7ffa5019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:46.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa56b6d700 1 -- 172.21.15.82:0/898924431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400095f0 con 0x7ffa50101890 2024-03-30T02:30:46.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.172+0000 7ffa56b6d700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa50101890 0x7ffa5019e080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800b970 tx=0x7ffa4800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:46.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.173+0000 7ffa477fe700 1 -- 172.21.15.82:0/898924431 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800c930 con 0x7ffa50101890 2024-03-30T02:30:46.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.173+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5019ee30 con 0x7ffa50101890 2024-03-30T02:30:46.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.173+0000 7ffa477fe700 1 -- 172.21.15.82:0/898924431 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa48014440 con 0x7ffa50101890 2024-03-30T02:30:46.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.173+0000 7ffa477fe700 1 -- 172.21.15.82:0/898924431 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480075d0 con 0x7ffa50101890 2024-03-30T02:30:46.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.173+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5019f2a0 con 0x7ffa50101890 2024-03-30T02:30:46.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.174+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa50107130 con 0x7ffa50101890 2024-03-30T02:30:46.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.175+0000 7ffa477fe700 1 -- 172.21.15.82:0/898924431 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7ffa480145b0 con 0x7ffa50101890 2024-03-30T02:30:46.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.175+0000 7ffa477fe700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:30:46.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.175+0000 7ffa477fe700 1 -- 172.21.15.82:0/898924431 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7ffa48006e40 con 0x7ffa50101890 2024-03-30T02:30:46.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.176+0000 7ffa5636c700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:30:46.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.176+0000 7ffa5636c700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa40006010 tx=0x7ffa4000b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:30:46.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.179+0000 7ffa477fe700 1 -- 172.21.15.82:0/898924431 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffa48095b20 con 0x7ffa50101890 2024-03-30T02:30:46.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:46.381+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa5002cdc0 con 0x7ffa3c0a1090 2024-03-30T02:30:46.413 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:30:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-0[56175]: 2024-03-30T02:30:46.048+0000 7ff327c00700 -1 osd.0 0 waiting for initial osdmap 2024-03-30T02:30:46.414 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:30:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-0[56175]: 2024-03-30T02:30:46.064+0000 7ff322a00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:30:46.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:46 smithi067 ceph-mon[37808]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:46.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:46 smithi067 ceph-mon[37808]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-30T02:30:46.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:46 smithi067 ceph-mon[37808]: osdmap e6: 1 total, 0 up, 1 in 2024-03-30T02:30:46.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:46 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:46.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:46 smithi067 ceph-mon[37808]: from='osd.0 [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:30:46.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:46 smithi067 ceph-mon[37808]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:30:46.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:46 smithi170 ceph-mon[78798]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:46.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:46 smithi170 ceph-mon[78798]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-30T02:30:46.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:46 smithi170 ceph-mon[78798]: osdmap e6: 1 total, 0 up, 1 in 2024-03-30T02:30:46.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:46 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:46.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:46 smithi170 ceph-mon[78798]: from='osd.0 [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:30:46.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:46 smithi170 ceph-mon[78798]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:30:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:46 smithi082 ceph-mon[79176]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:46 smithi082 ceph-mon[79176]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-30T02:30:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:46 smithi082 ceph-mon[79176]: osdmap e6: 1 total, 0 up, 1 in 2024-03-30T02:30:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:46 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:46 smithi082 ceph-mon[79176]: from='osd.0 [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:30:46.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:46 smithi082 ceph-mon[79176]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:30:47.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: osdmap e7: 1 total, 0 up, 1 in 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi067 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi067 to 17270M 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:30:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:30:47.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:47.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:30:47.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: osdmap e7: 1 total, 0 up, 1 in 2024-03-30T02:30:47.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:47.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi067 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi067 to 17270M 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:30:47.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: osdmap e7: 1 total, 0 up, 1 in 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi067 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi067 to 17270M 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.461 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:47.462 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:30:47.462 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:47.462 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:30:47.462 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:30:47.462 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:48 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:30:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:48 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:30:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:48 smithi067 ceph-mon[37808]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:48 smithi067 ceph-mon[37808]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:48 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:48 smithi067 ceph-mon[37808]: osd.0 [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] boot 2024-03-30T02:30:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:48 smithi067 ceph-mon[37808]: osdmap e8: 1 total, 1 up, 1 in 2024-03-30T02:30:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:48 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:48.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:48 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:30:48.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:48 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:30:48.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:48 smithi170 ceph-mon[78798]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:48.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:48 smithi170 ceph-mon[78798]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:48.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:48 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:48.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:48 smithi170 ceph-mon[78798]: osd.0 [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] boot 2024-03-30T02:30:48.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:48 smithi170 ceph-mon[78798]: osdmap e8: 1 total, 1 up, 1 in 2024-03-30T02:30:48.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:48 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:48 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:30:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:48 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:30:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:48 smithi082 ceph-mon[79176]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:30:48.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:48 smithi082 ceph-mon[79176]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-30T02:30:48.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:48 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:48.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:48 smithi082 ceph-mon[79176]: osd.0 [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] boot 2024-03-30T02:30:48.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:48 smithi082 ceph-mon[79176]: osdmap e8: 1 total, 1 up, 1 in 2024-03-30T02:30:48.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:48 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:30:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:49 smithi067 ceph-mon[37808]: osdmap e9: 1 total, 1 up, 1 in 2024-03-30T02:30:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:49 smithi067 ceph-mon[37808]: from='client.? 172.21.15.82:0/2542248873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1deb576f-c68d-421e-b78a-cfd6e72aafc1"}]: dispatch 2024-03-30T02:30:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:49 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1deb576f-c68d-421e-b78a-cfd6e72aafc1"}]: dispatch 2024-03-30T02:30:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:49 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1deb576f-c68d-421e-b78a-cfd6e72aafc1"}]': finished 2024-03-30T02:30:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:49 smithi067 ceph-mon[37808]: osdmap e10: 2 total, 1 up, 2 in 2024-03-30T02:30:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:49 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:30:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:49 smithi067 ceph-mon[37808]: from='client.? 172.21.15.82:0/3861069959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:30:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:49 smithi067 ceph-mon[37808]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:49.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:49 smithi170 ceph-mon[78798]: osdmap e9: 1 total, 1 up, 1 in 2024-03-30T02:30:49.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:49 smithi170 ceph-mon[78798]: from='client.? 172.21.15.82:0/2542248873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1deb576f-c68d-421e-b78a-cfd6e72aafc1"}]: dispatch 2024-03-30T02:30:49.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:49 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1deb576f-c68d-421e-b78a-cfd6e72aafc1"}]: dispatch 2024-03-30T02:30:49.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:49 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1deb576f-c68d-421e-b78a-cfd6e72aafc1"}]': finished 2024-03-30T02:30:49.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:49 smithi170 ceph-mon[78798]: osdmap e10: 2 total, 1 up, 2 in 2024-03-30T02:30:49.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:49 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:30:49.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:49 smithi170 ceph-mon[78798]: from='client.? 172.21.15.82:0/3861069959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:30:49.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:49 smithi170 ceph-mon[78798]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:49.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:49 smithi082 ceph-mon[79176]: osdmap e9: 1 total, 1 up, 1 in 2024-03-30T02:30:49.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:49 smithi082 ceph-mon[79176]: from='client.? 172.21.15.82:0/2542248873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1deb576f-c68d-421e-b78a-cfd6e72aafc1"}]: dispatch 2024-03-30T02:30:49.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:49 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1deb576f-c68d-421e-b78a-cfd6e72aafc1"}]: dispatch 2024-03-30T02:30:49.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:49 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1deb576f-c68d-421e-b78a-cfd6e72aafc1"}]': finished 2024-03-30T02:30:49.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:49 smithi082 ceph-mon[79176]: osdmap e10: 2 total, 1 up, 2 in 2024-03-30T02:30:49.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:49 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:30:49.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:49 smithi082 ceph-mon[79176]: from='client.? 172.21.15.82:0/3861069959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:30:49.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:49 smithi082 ceph-mon[79176]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:52.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:51 smithi170 ceph-mon[78798]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:52.205 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:51 smithi082 ceph-mon[79176]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:52.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:51 smithi067 ceph-mon[37808]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:54.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:54 smithi082 ceph-mon[79176]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:54.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:54 smithi067 ceph-mon[37808]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:54.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:54 smithi170 ceph-mon[78798]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:55.159 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-30T02:30:55.159 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:55.159 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:55 smithi082 ceph-mon[79176]: Deploying daemon osd.1 on smithi082 2024-03-30T02:30:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-30T02:30:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:55 smithi067 ceph-mon[37808]: Deploying daemon osd.1 on smithi082 2024-03-30T02:30:55.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-30T02:30:55.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:30:55.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:55 smithi170 ceph-mon[78798]: Deploying daemon osd.1 on smithi082 2024-03-30T02:30:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:56 smithi082 ceph-mon[79176]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:56.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:56 smithi067 ceph-mon[37808]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:56.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:56 smithi170 ceph-mon[78798]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:58.358 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:58 smithi082 ceph-mon[79176]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:58.359 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:58 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:58.359 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:58 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:58.359 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:58 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:58.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:58 smithi067 ceph-mon[37808]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:58.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:58 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:58.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:58 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:58.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:58 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:58 smithi170 ceph-mon[78798]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:30:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:58 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:58 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:30:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:58 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:30:59.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.983+0000 7ffa477fe700 1 -- 172.21.15.82:0/898924431 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffa5002cdc0 con 0x7ffa3c0a1090 2024-03-30T02:30:59.986 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 1 on host 'smithi082' 2024-03-30T02:30:59.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ffa3c0a1090 msgr2=0x7ffa3c0a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:59.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa40006010 tx=0x7ffa4000b540 comp rx=0 tx=0).stop 2024-03-30T02:30:59.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa50101890 msgr2=0x7ffa5019e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:30:59.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa50101890 0x7ffa5019e080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800b970 tx=0x7ffa4800bc80 comp rx=0 tx=0).stop 2024-03-30T02:30:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 shutdown_connections 2024-03-30T02:30:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ffa3c0a1090 0x7ffa3c0a3550 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa50101890 0x7ffa5019e080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa500ffcf0 0x7ffa50069260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 --2- 172.21.15.82:0/898924431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa50100ef0 0x7ffa5019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:30:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 >> 172.21.15.82:0/898924431 conn(0x7ffa500fb2b0 msgr2=0x7ffa50104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:30:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 shutdown_connections 2024-03-30T02:30:59.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:30:59.986+0000 7ffa585d0700 1 -- 172.21.15.82:0/898924431 wait complete. 2024-03-30T02:31:00.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:59 smithi082 ceph-mon[79176]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:31:00.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:00.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:00.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:30:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:00.374 DEBUG:teuthology.orchestra.run.smithi082:osd.1> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.1.service 2024-03-30T02:31:00.376 INFO:tasks.cephadm:Deploying osd.2 on smithi170 with /dev/vg_nvme/lv_4... 2024-03-30T02:31:00.377 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-30T02:31:00.408 INFO:journalctl@ceph.osd.1.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:31:00.408 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:59 smithi170 ceph-mon[78798]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:31:00.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:00.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:00.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:30:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:00.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:59 smithi067 ceph-mon[37808]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:31:00.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:30:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:31:01.862 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-30T02:31:01.862 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-30T02:31:01.862 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-03-30T02:31:01.862 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-03-30T02:31:01.862 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00832768 s, 1.3 GB/s 2024-03-30T02:31:01.862 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-03-30T02:31:01.923 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:01 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-1[84173]: 2024-03-30T02:31:01.597+0000 7f27e6516640 -1 osd.1 0 log_to_monitors true 2024-03-30T02:31:02.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:01 smithi170 ceph-mon[78798]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:31:02.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:01 smithi170 ceph-mon[78798]: from='osd.1 [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T02:31:02.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:01 smithi170 ceph-mon[78798]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T02:31:02.199 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:01 smithi082 ceph-mon[79176]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:31:02.199 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:01 smithi082 ceph-mon[79176]: from='osd.1 [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T02:31:02.199 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:01 smithi082 ceph-mon[79176]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T02:31:02.249 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2024-03-30T02:31:02.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:01 smithi067 ceph-mon[37808]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:31:02.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:01 smithi067 ceph-mon[37808]: from='osd.1 [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T02:31:02.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:01 smithi067 ceph-mon[37808]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-30T02:31:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: osdmap e11: 2 total, 1 up, 2 in 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='osd.1 [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:03.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:03.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:02 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.250 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-1[84173]: 2024-03-30T02:31:02.929+0000 7f27dbb7a700 -1 osd.1 0 waiting for initial osdmap 2024-03-30T02:31:03.250 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-1[84173]: 2024-03-30T02:31:02.932+0000 7f27d796f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: osdmap e11: 2 total, 1 up, 2 in 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='osd.1 [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:02 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: osdmap e11: 2 total, 1 up, 2 in 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='osd.1 [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:02 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 -- 172.21.15.170:0/1627775540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50100ef0 msgr2=0x7f8d50101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/1627775540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50100ef0 0x7f8d50101350 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009990 tx=0x7f8d4c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 -- 172.21.15.170:0/1627775540 shutdown_connections 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/1627775540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d500ffcf0 0x7f8d50100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/1627775540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d50101890 0x7f8d50108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/1627775540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50100ef0 0x7f8d50101350 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 -- 172.21.15.170:0/1627775540 >> 172.21.15.170:0/1627775540 conn(0x7f8d500fb2b0 msgr2=0x7f8d500fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 -- 172.21.15.170:0/1627775540 shutdown_connections 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 -- 172.21.15.170:0/1627775540 wait complete. 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 Processor -- start 2024-03-30T02:31:03.458 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.455+0000 7f8d57bc5700 1 -- start start 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d57bc5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d500ffcf0 0x7f8d50196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d57bc5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d50100ef0 0x7f8d501972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d57bc5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50101890 0x7f8d5019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d57bc5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5019e860 con 0x7f8d50100ef0 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d57bc5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5019e9a0 con 0x7f8d500ffcf0 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d57bc5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5019eb60 con 0x7f8d50101890 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d56162700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50101890 0x7f8d5019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d56162700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50101890 0x7f8d5019e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53092/0 (socket says 172.21.15.170:53092) 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d56162700 1 -- 172.21.15.170:0/110027739 learned_addr learned my addr 172.21.15.170:0/110027739 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d55961700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d500ffcf0 0x7f8d50196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d56162700 1 -- 172.21.15.170:0/110027739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d500ffcf0 msgr2=0x7f8d50196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d56162700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d500ffcf0 0x7f8d50196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d56162700 1 -- 172.21.15.170:0/110027739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d50100ef0 msgr2=0x7f8d501972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:03.459 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d56162700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d50100ef0 0x7f8d501972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:03.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d56162700 1 -- 172.21.15.170:0/110027739 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c0095f0 con 0x7f8d50101890 2024-03-30T02:31:03.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d56162700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50101890 0x7f8d5019e290 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d48007ee0 tx=0x7f8d4800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:31:03.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d42ffd700 1 -- 172.21.15.170:0/110027739 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48017070 con 0x7f8d50101890 2024-03-30T02:31:03.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d42ffd700 1 -- 172.21.15.170:0/110027739 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d4800f040 con 0x7f8d50101890 2024-03-30T02:31:03.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d42ffd700 1 -- 172.21.15.170:0/110027739 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48005740 con 0x7f8d50101890 2024-03-30T02:31:03.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5019ee40 con 0x7f8d50101890 2024-03-30T02:31:03.460 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.456+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5019f230 con 0x7f8d50101890 2024-03-30T02:31:03.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.457+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5004ea80 con 0x7f8d50101890 2024-03-30T02:31:03.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.458+0000 7f8d42ffd700 1 -- 172.21.15.170:0/110027739 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7f8d4800dc60 con 0x7f8d50101890 2024-03-30T02:31:03.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.459+0000 7f8d42ffd700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d3c0a1110 0x7f8d3c0a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:03.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.459+0000 7f8d42ffd700 1 -- 172.21.15.170:0/110027739 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f8d480cadc0 con 0x7f8d50101890 2024-03-30T02:31:03.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.460+0000 7f8d55961700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d3c0a1110 0x7f8d3c0a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:03.461 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.460+0000 7f8d55961700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d3c0a1110 0x7f8d3c0a35d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d501981f0 tx=0x7f8d4400a380 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:31:03.464 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.463+0000 7f8d42ffd700 1 -- 172.21.15.170:0/110027739 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d48095e20 con 0x7f8d50101890 2024-03-30T02:31:03.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:03.663+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d50068e60 con 0x7f8d3c0a1110 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi082 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi082 to 17113M 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: osdmap e12: 2 total, 1 up, 2 in 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:31:04.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:03 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:04.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi082 2024-03-30T02:31:04.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi082 to 17113M 2024-03-30T02:31:04.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:31:04.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: osdmap e12: 2 total, 1 up, 2 in 2024-03-30T02:31:04.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:04.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:04.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:31:04.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:31:04.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:31:04.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:03 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi082 2024-03-30T02:31:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi082 to 17113M 2024-03-30T02:31:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:31:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: osdmap e12: 2 total, 1 up, 2 in 2024-03-30T02:31:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-30T02:31:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:31:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:31:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:03 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:04 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:31:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:04 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:31:05.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:04 smithi170 ceph-mon[78798]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:31:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:04 smithi170 ceph-mon[78798]: osd.1 [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] boot 2024-03-30T02:31:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:04 smithi170 ceph-mon[78798]: osdmap e13: 2 total, 2 up, 2 in 2024-03-30T02:31:05.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:05.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:04 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:31:05.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:04 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:31:05.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:04 smithi082 ceph-mon[79176]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:31:05.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:04 smithi082 ceph-mon[79176]: osd.1 [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] boot 2024-03-30T02:31:05.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:04 smithi082 ceph-mon[79176]: osdmap e13: 2 total, 2 up, 2 in 2024-03-30T02:31:05.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:04 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:05.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:04 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:31:05.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:04 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:31:05.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:04 smithi067 ceph-mon[37808]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:31:05.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:04 smithi067 ceph-mon[37808]: osd.1 [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] boot 2024-03-30T02:31:05.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:04 smithi067 ceph-mon[37808]: osdmap e13: 2 total, 2 up, 2 in 2024-03-30T02:31:05.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:31:06.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:05 smithi082 ceph-mon[79176]: osdmap e14: 2 total, 2 up, 2 in 2024-03-30T02:31:06.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:05 smithi082 ceph-mon[79176]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:06.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:05 smithi082 ceph-mon[79176]: from='client.? 172.21.15.170:0/2809521878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bcf7865-4fd5-4a95-9cb1-294899b88e2d"}]: dispatch 2024-03-30T02:31:06.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:05 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bcf7865-4fd5-4a95-9cb1-294899b88e2d"}]: dispatch 2024-03-30T02:31:06.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:05 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bcf7865-4fd5-4a95-9cb1-294899b88e2d"}]': finished 2024-03-30T02:31:06.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:05 smithi082 ceph-mon[79176]: osdmap e15: 3 total, 2 up, 3 in 2024-03-30T02:31:06.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:05 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:05 smithi067 ceph-mon[37808]: osdmap e14: 2 total, 2 up, 2 in 2024-03-30T02:31:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:05 smithi067 ceph-mon[37808]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:05 smithi067 ceph-mon[37808]: from='client.? 172.21.15.170:0/2809521878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bcf7865-4fd5-4a95-9cb1-294899b88e2d"}]: dispatch 2024-03-30T02:31:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:05 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bcf7865-4fd5-4a95-9cb1-294899b88e2d"}]: dispatch 2024-03-30T02:31:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:05 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bcf7865-4fd5-4a95-9cb1-294899b88e2d"}]': finished 2024-03-30T02:31:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:05 smithi067 ceph-mon[37808]: osdmap e15: 3 total, 2 up, 3 in 2024-03-30T02:31:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:05 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:06.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:05 smithi170 ceph-mon[78798]: osdmap e14: 2 total, 2 up, 2 in 2024-03-30T02:31:06.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:05 smithi170 ceph-mon[78798]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:06.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:05 smithi170 ceph-mon[78798]: from='client.? 172.21.15.170:0/2809521878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bcf7865-4fd5-4a95-9cb1-294899b88e2d"}]: dispatch 2024-03-30T02:31:06.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:05 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bcf7865-4fd5-4a95-9cb1-294899b88e2d"}]: dispatch 2024-03-30T02:31:06.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:05 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bcf7865-4fd5-4a95-9cb1-294899b88e2d"}]': finished 2024-03-30T02:31:06.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:05 smithi170 ceph-mon[78798]: osdmap e15: 3 total, 2 up, 3 in 2024-03-30T02:31:06.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:05 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:07.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:06 smithi082 ceph-mon[79176]: from='client.? 172.21.15.170:0/1031642920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:31:07.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:06 smithi067 ceph-mon[37808]: from='client.? 172.21.15.170:0/1031642920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:31:07.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:06 smithi170 ceph-mon[78798]: from='client.? 172.21.15.170:0/1031642920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:31:08.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:07 smithi082 ceph-mon[79176]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:08.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:07 smithi067 ceph-mon[37808]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:08.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:07 smithi170 ceph-mon[78798]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:10.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:09 smithi082 ceph-mon[79176]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:10.293 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:09 smithi170 ceph-mon[78798]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:10.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:09 smithi067 ceph-mon[37808]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:12.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:11 smithi170 ceph-mon[78798]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:11 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-30T02:31:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:11 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:11 smithi082 ceph-mon[79176]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:11 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-30T02:31:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:11 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:12.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:11 smithi067 ceph-mon[37808]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:12.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:11 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-30T02:31:12.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:11 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:13.088 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:12 smithi170 ceph-mon[78798]: Deploying daemon osd.2 on smithi170 2024-03-30T02:31:13.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:12 smithi082 ceph-mon[79176]: Deploying daemon osd.2 on smithi170 2024-03-30T02:31:13.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:12 smithi067 ceph-mon[37808]: Deploying daemon osd.2 on smithi170 2024-03-30T02:31:14.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:13 smithi170 ceph-mon[78798]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:14.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:13 smithi082 ceph-mon[79176]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:14.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:13 smithi067 ceph-mon[37808]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:16.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:15 smithi170 ceph-mon[78798]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:16.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:16.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:16.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:15 smithi082 ceph-mon[79176]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:16.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:15 smithi067 ceph-mon[37808]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:17.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.120+0000 7f8d42ffd700 1 -- 172.21.15.170:0/110027739 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8d50068e60 con 0x7f8d3c0a1110 2024-03-30T02:31:17.123 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 2 on host 'smithi170' 2024-03-30T02:31:17.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d3c0a1110 msgr2=0x7f8d3c0a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:17.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d3c0a1110 0x7f8d3c0a35d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d501981f0 tx=0x7f8d4400a380 comp rx=0 tx=0).stop 2024-03-30T02:31:17.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50101890 msgr2=0x7f8d5019e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:17.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50101890 0x7f8d5019e290 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d48007ee0 tx=0x7f8d4800d3b0 comp rx=0 tx=0).stop 2024-03-30T02:31:17.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 shutdown_connections 2024-03-30T02:31:17.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d3c0a1110 0x7f8d3c0a35d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:17.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d50101890 0x7f8d5019e290 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:17.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d500ffcf0 0x7f8d50196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:17.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 --2- 172.21.15.170:0/110027739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d50100ef0 0x7f8d501972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:17.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 >> 172.21.15.170:0/110027739 conn(0x7f8d500fb2b0 msgr2=0x7f8d50068750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:31:17.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 shutdown_connections 2024-03-30T02:31:17.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:17.123+0000 7f8d57bc5700 1 -- 172.21.15.170:0/110027739 wait complete. 2024-03-30T02:31:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:17 smithi067 ceph-mon[37808]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:17.428 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:17.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:17.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:17 smithi170 ceph-mon[78798]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:17.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:17.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:17.766 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:17.766 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.766 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:17 smithi082 ceph-mon[79176]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:17.766 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.766 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:17.766 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:17.767 DEBUG:teuthology.orchestra.run.smithi170:osd.2> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.2.service 2024-03-30T02:31:17.769 INFO:tasks.cephadm:Deploying osd.3 on smithi067 with /dev/vg_nvme/lv_3... 2024-03-30T02:31:17.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-30T02:31:17.801 INFO:journalctl@ceph.osd.2.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:31:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:31:19.233 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:19 smithi170 ceph-mon[78798]: from='osd.2 [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T02:31:19.233 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:19 smithi170 ceph-mon[78798]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T02:31:19.233 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:18 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-2[84585]: 2024-03-30T02:31:18.963+0000 7fc386b7c640 -1 osd.2 0 log_to_monitors true 2024-03-30T02:31:19.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:19 smithi067 ceph-mon[37808]: from='osd.2 [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T02:31:19.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:19 smithi067 ceph-mon[37808]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T02:31:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:19 smithi082 ceph-mon[79176]: from='osd.2 [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T02:31:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:19 smithi082 ceph-mon[79176]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-30T02:31:19.936 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-30T02:31:19.937 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-30T02:31:19.937 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-03-30T02:31:19.937 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-03-30T02:31:19.937 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0296264 s, 354 MB/s 2024-03-30T02:31:19.937 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-03-30T02:31:20.279 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: osdmap e16: 3 total, 2 up, 3 in 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='osd.2 [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:20.280 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi067:vg_nvme/lv_3 2024-03-30T02:31:20.440 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-2[84585]: 2024-03-30T02:31:20.035+0000 7fc37c1e0700 -1 osd.2 0 waiting for initial osdmap 2024-03-30T02:31:20.440 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-2[84585]: 2024-03-30T02:31:20.039+0000 7fc377fd5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:31:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-30T02:31:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: osdmap e16: 3 total, 2 up, 3 in 2024-03-30T02:31:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='osd.2 [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:31:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:31:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:20.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:20.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:20.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: osdmap e16: 3 total, 2 up, 3 in 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='osd.2 [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:20.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:20.666 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:31:21.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:21 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi170 2024-03-30T02:31:21.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:21 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi170 to 17114M 2024-03-30T02:31:21.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:21 smithi067 ceph-mon[37808]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:31:21.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:21 smithi067 ceph-mon[37808]: osdmap e17: 3 total, 2 up, 3 in 2024-03-30T02:31:21.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:21.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:21 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi170 2024-03-30T02:31:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:21 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi170 to 17114M 2024-03-30T02:31:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:21 smithi170 ceph-mon[78798]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:31:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:21 smithi170 ceph-mon[78798]: osdmap e17: 3 total, 2 up, 3 in 2024-03-30T02:31:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:21 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi170 2024-03-30T02:31:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:21 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi170 to 17114M 2024-03-30T02:31:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:21 smithi082 ceph-mon[79176]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:31:21.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:21 smithi082 ceph-mon[79176]: osdmap e17: 3 total, 2 up, 3 in 2024-03-30T02:31:21.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:21.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:21.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.814+0000 7fea82400700 1 -- 172.21.15.67:0/1565599251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea7c100e00 msgr2=0x7fea7c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:21.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.814+0000 7fea82400700 1 --2- 172.21.15.67:0/1565599251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea7c100e00 0x7fea7c101220 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009990 tx=0x7fea6c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:31:21.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.814+0000 7fea82400700 1 -- 172.21.15.67:0/1565599251 shutdown_connections 2024-03-30T02:31:21.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.814+0000 7fea82400700 1 --2- 172.21.15.67:0/1565599251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea7c100e00 0x7fea7c101220 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:21.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.814+0000 7fea82400700 1 --2- 172.21.15.67:0/1565599251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7c102920 0x7fea7c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:21.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.814+0000 7fea82400700 1 --2- 172.21.15.67:0/1565599251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c101f60 0x7fea7c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:21.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.814+0000 7fea82400700 1 -- 172.21.15.67:0/1565599251 >> 172.21.15.67:0/1565599251 conn(0x7fea7c0fc3c0 msgr2=0x7fea7c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:31:21.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.814+0000 7fea82400700 1 -- 172.21.15.67:0/1565599251 shutdown_connections 2024-03-30T02:31:21.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.814+0000 7fea82400700 1 -- 172.21.15.67:0/1565599251 wait complete. 2024-03-30T02:31:21.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.815+0000 7fea82400700 1 Processor -- start 2024-03-30T02:31:21.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea82400700 1 -- start start 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea82400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c100e00 0x7fea7c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea82400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea7c101f60 0x7fea7c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea82400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7c102920 0x7fea7c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea82400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c073e80 con 0x7fea7c101f60 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea82400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c073ff0 con 0x7fea7c100e00 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea82400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c1a0db0 con 0x7fea7c102920 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea7b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c100e00 0x7fea7c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea7aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea7c101f60 0x7fea7c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea7b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c100e00 0x7fea7c074b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40164/0 (socket says 172.21.15.67:40164) 2024-03-30T02:31:21.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea7b400700 1 -- 172.21.15.67:0/313699828 learned_addr learned my addr 172.21.15.67:0/313699828 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.816+0000 7fea7be00700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7c102920 0x7fea7c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea7b400700 1 -- 172.21.15.67:0/313699828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7c102920 msgr2=0x7fea7c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea7b400700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7c102920 0x7fea7c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea7b400700 1 -- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea7c101f60 msgr2=0x7fea7c0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea7b400700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea7c101f60 0x7fea7c0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea7b400700 1 -- 172.21.15.67:0/313699828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea68009520 con 0x7fea7c100e00 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea7aa00700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea7c101f60 0x7fea7c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea7b400700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c100e00 0x7fea7c074b00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00b620 tx=0x7fea6c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea73e00700 1 -- 172.21.15.67:0/313699828 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c01d070 con 0x7fea7c100e00 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea73e00700 1 -- 172.21.15.67:0/313699828 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea6c022470 con 0x7fea7c100e00 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c0095f0 con 0x7fea7c100e00 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea73e00700 1 -- 172.21.15.67:0/313699828 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c017650 con 0x7fea7c100e00 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.817+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c1a11b0 con 0x7fea7c100e00 2024-03-30T02:31:21.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.819+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7c1082d0 con 0x7fea7c100e00 2024-03-30T02:31:21.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.820+0000 7fea73e00700 1 -- 172.21.15.67:0/313699828 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) v1 ==== 135809+0+0 (secure 0 0 0) 0x7fea6c0225e0 con 0x7fea7c100e00 2024-03-30T02:31:21.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.821+0000 7fea73e00700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fea600a1090 0x7fea600a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:21.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.822+0000 7fea73e00700 1 -- 172.21.15.67:0/313699828 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fea6c0cc590 con 0x7fea7c100e00 2024-03-30T02:31:21.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.822+0000 7fea7aa00700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fea600a1090 0x7fea600a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:21.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.822+0000 7fea7aa00700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fea600a1090 0x7fea600a3550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fea68009fd0 tx=0x7fea68009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:31:21.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:21.826+0000 7fea73e00700 1 -- 172.21.15.67:0/313699828 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fea6c097340 con 0x7fea7c100e00 2024-03-30T02:31:22.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:22.052+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fea7c02ce10 con 0x7fea600a1090 2024-03-30T02:31:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:22 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:31:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:22 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:31:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:22 smithi067 ceph-mon[37808]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:22 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:22 smithi067 ceph-mon[37808]: osd.2 [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] boot 2024-03-30T02:31:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:22 smithi067 ceph-mon[37808]: osdmap e18: 3 total, 3 up, 3 in 2024-03-30T02:31:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:22 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:22 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:31:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:22 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:31:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:22 smithi170 ceph-mon[78798]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:22 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:22 smithi170 ceph-mon[78798]: osd.2 [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] boot 2024-03-30T02:31:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:22 smithi170 ceph-mon[78798]: osdmap e18: 3 total, 3 up, 3 in 2024-03-30T02:31:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:22 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:22 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:31:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:22 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:31:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:22 smithi082 ceph-mon[79176]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-30T02:31:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:22 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:22 smithi082 ceph-mon[79176]: osd.2 [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] boot 2024-03-30T02:31:22.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:22 smithi082 ceph-mon[79176]: osdmap e18: 3 total, 3 up, 3 in 2024-03-30T02:31:22.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:22 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:31:23.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:23 smithi067 ceph-mon[37808]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:31:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:31:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:31:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:23.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:23 smithi067 ceph-mon[37808]: osdmap e19: 3 total, 3 up, 3 in 2024-03-30T02:31:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:23 smithi170 ceph-mon[78798]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:31:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:31:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:31:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:23 smithi170 ceph-mon[78798]: osdmap e19: 3 total, 3 up, 3 in 2024-03-30T02:31:24.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:23 smithi082 ceph-mon[79176]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:31:24.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:31:24.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:31:24.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:24.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:23 smithi082 ceph-mon[79176]: osdmap e19: 3 total, 3 up, 3 in 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: pgmap v67: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: osdmap e20: 3 total, 3 up, 3 in 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3141891960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eea2b812-7830-4563-aa7e-b340ae8a35bb"}]: dispatch 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eea2b812-7830-4563-aa7e-b340ae8a35bb"}]: dispatch 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eea2b812-7830-4563-aa7e-b340ae8a35bb"}]': finished 2024-03-30T02:31:24.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: osdmap e21: 4 total, 3 up, 4 in 2024-03-30T02:31:24.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:24.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: pgmap v67: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: osdmap e20: 3 total, 3 up, 3 in 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3141891960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eea2b812-7830-4563-aa7e-b340ae8a35bb"}]: dispatch 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eea2b812-7830-4563-aa7e-b340ae8a35bb"}]: dispatch 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eea2b812-7830-4563-aa7e-b340ae8a35bb"}]': finished 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: osdmap e21: 4 total, 3 up, 4 in 2024-03-30T02:31:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:25.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: pgmap v67: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:25.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:31:25.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-30T02:31:25.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: osdmap e20: 3 total, 3 up, 3 in 2024-03-30T02:31:25.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:31:25.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3141891960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eea2b812-7830-4563-aa7e-b340ae8a35bb"}]: dispatch 2024-03-30T02:31:25.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eea2b812-7830-4563-aa7e-b340ae8a35bb"}]: dispatch 2024-03-30T02:31:25.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-30T02:31:25.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eea2b812-7830-4563-aa7e-b340ae8a35bb"}]': finished 2024-03-30T02:31:25.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: osdmap e21: 4 total, 3 up, 4 in 2024-03-30T02:31:25.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:25.414 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:31:25 smithi067 sudo[61969]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T02:31:25.414 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:31:25 smithi067 sudo[61969]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:25.414 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:31:25 smithi067 sudo[61969]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:25.414 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:31:25 smithi067 sudo[61969]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:25.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:25.561+0000 7fea73e00700 1 -- 172.21.15.67:0/313699828 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fea6c097710 con 0x7fea7c100e00 2024-03-30T02:31:25.910 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:25 smithi082 ceph-mon[79176]: pgmap v70: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:25.910 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:25 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/284364253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:31:25.910 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:25 smithi082 ceph-mon[79176]: osdmap e22: 4 total, 3 up, 4 in 2024-03-30T02:31:25.910 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:25 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:25 smithi067 ceph-mon[37808]: pgmap v70: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:25 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/284364253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:31:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:25 smithi067 ceph-mon[37808]: osdmap e22: 4 total, 3 up, 4 in 2024-03-30T02:31:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:25 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:25.914 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:31:25 smithi067 sudo[62022]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T02:31:25.914 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:31:25 smithi067 sudo[62022]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:25.914 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:31:25 smithi067 sudo[62022]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:25.914 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:31:25 smithi067 sudo[62022]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:25 smithi170 ceph-mon[78798]: pgmap v70: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:25 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/284364253' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:31:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:25 smithi170 ceph-mon[78798]: osdmap e22: 4 total, 3 up, 4 in 2024-03-30T02:31:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:25 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:26.250 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:25 smithi082 sudo[89108]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T02:31:26.250 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:25 smithi082 sudo[89108]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:26.250 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:25 smithi082 sudo[89108]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:26.250 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:25 smithi082 sudo[89108]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:26.250 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:26 smithi082 sudo[89111]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T02:31:26.251 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:26 smithi082 sudo[89111]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:26.251 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:26 smithi082 sudo[89111]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:26.251 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:31:26 smithi082 sudo[89111]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:26.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:26 smithi170 ceph-mon[78798]: mgrmap e16: x(active, since 118s), standbys: z, y 2024-03-30T02:31:26.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:26 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:26.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:26 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:26.690 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:26 smithi170 sudo[90024]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T02:31:26.690 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:26 smithi170 sudo[90024]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:26.690 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:26 smithi170 sudo[90024]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:26.690 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:26 smithi170 sudo[90024]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:26.690 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:26 smithi170 sudo[90027]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T02:31:26.691 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:26 smithi170 sudo[90027]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:26.691 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:26 smithi170 sudo[90027]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:26.691 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:31:26 smithi170 sudo[90027]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:26 smithi067 ceph-mon[37808]: mgrmap e16: x(active, since 118s), standbys: z, y 2024-03-30T02:31:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:26 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:27.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:26 smithi082 ceph-mon[79176]: mgrmap e16: x(active, since 118s), standbys: z, y 2024-03-30T02:31:27.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:26 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:27.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:26 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:27.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 sudo[62776]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T02:31:27.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 sudo[62776]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 sudo[62776]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 sudo[62776]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:28.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 sudo[63156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T02:31:28.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 sudo[63156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:28.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 sudo[63156]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:28.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 sudo[63156]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:28.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:28.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:27 smithi067 ceph-mon[37808]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T02:31:28.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:28.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:27 smithi170 ceph-mon[78798]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T02:31:28.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 sudo[90030]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T02:31:28.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 sudo[90030]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:28.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 sudo[90030]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:28.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 sudo[90030]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:28.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:28.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:27 smithi082 ceph-mon[79176]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T02:31:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 sudo[90033]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T02:31:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 sudo[90033]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 sudo[90033]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 sudo[90033]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:28.818 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 sudo[89114]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-30T02:31:28.819 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 sudo[89114]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:28.819 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 sudo[89114]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:28.819 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 sudo[89114]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:28.819 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 sudo[89117]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-30T02:31:28.819 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 sudo[89117]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-30T02:31:28.819 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 sudo[89117]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-30T02:31:28.819 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 sudo[89117]: pam_unix(sudo:session): session closed for user root 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: pgmap v72: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:31:29.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:31:29.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:31:29.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:28 smithi067 ceph-mon[37808]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: pgmap v72: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T02:31:29.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:31:29.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:31:29.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:31:29.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:28 smithi170 ceph-mon[78798]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T02:31:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: pgmap v72: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T02:31:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:31:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:31:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:31:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T02:31:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:31:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:31:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:31:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T02:31:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-30T02:31:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:31:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:31:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:31:29.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:28 smithi082 ceph-mon[79176]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-30T02:31:30.111 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:29 smithi067 ceph-mon[37808]: pgmap v73: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:30.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:29 smithi170 ceph-mon[78798]: pgmap v73: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:30.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:29 smithi082 ceph-mon[79176]: pgmap v73: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:32.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:31 smithi067 ceph-mon[37808]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-30T02:31:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-30T02:31:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:31 smithi067 ceph-mon[37808]: Deploying daemon osd.3 on smithi067 2024-03-30T02:31:32.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:31 smithi170 ceph-mon[78798]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-30T02:31:32.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-30T02:31:32.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:32.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:31 smithi170 ceph-mon[78798]: Deploying daemon osd.3 on smithi067 2024-03-30T02:31:32.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:31 smithi082 ceph-mon[79176]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-30T02:31:32.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-30T02:31:32.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:32.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:31 smithi082 ceph-mon[79176]: Deploying daemon osd.3 on smithi067 2024-03-30T02:31:34.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:34 smithi067 ceph-mon[37808]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-30T02:31:34.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:34 smithi170 ceph-mon[78798]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-30T02:31:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:34 smithi082 ceph-mon[79176]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-30T02:31:36.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:36 smithi067 ceph-mon[37808]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-30T02:31:36.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:36 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:36.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:36 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:36.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:36 smithi170 ceph-mon[78798]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-30T02:31:36.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:36 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:36.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:36 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:36 smithi082 ceph-mon[79176]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-30T02:31:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:36 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:36 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:37.052 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:37.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:37 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:37.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:37 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:38.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:38 smithi067 ceph-mon[37808]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 159 KiB/s wr, 10 op/s 2024-03-30T02:31:38.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:38.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:38.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:38.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:38.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:38 smithi170 ceph-mon[78798]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 159 KiB/s wr, 10 op/s 2024-03-30T02:31:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:38 smithi082 ceph-mon[79176]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 159 KiB/s wr, 10 op/s 2024-03-30T02:31:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:38.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.698+0000 7fea73e00700 1 -- 172.21.15.67:0/313699828 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fea7c02ce10 con 0x7fea600a1090 2024-03-30T02:31:38.700 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 3 on host 'smithi067' 2024-03-30T02:31:38.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fea600a1090 msgr2=0x7fea600a3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:38.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fea600a1090 0x7fea600a3550 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fea68009fd0 tx=0x7fea68009260 comp rx=0 tx=0).stop 2024-03-30T02:31:38.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c100e00 msgr2=0x7fea7c074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c100e00 0x7fea7c074b00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00b620 tx=0x7fea6c004730 comp rx=0 tx=0).stop 2024-03-30T02:31:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 shutdown_connections 2024-03-30T02:31:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fea600a1090 0x7fea600a3550 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea7c102920 0x7fea7c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c100e00 0x7fea7c074b00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 --2- 172.21.15.67:0/313699828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea7c101f60 0x7fea7c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 >> 172.21.15.67:0/313699828 conn(0x7fea7c0fc3c0 msgr2=0x7fea7c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:31:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 shutdown_connections 2024-03-30T02:31:38.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:31:38.701+0000 7fea82400700 1 -- 172.21.15.67:0/313699828 wait complete. 2024-03-30T02:31:39.116 DEBUG:teuthology.orchestra.run.smithi067:osd.3> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.3.service 2024-03-30T02:31:39.121 INFO:tasks.cephadm:Deploying osd.4 on smithi082 with /dev/vg_nvme/lv_3... 2024-03-30T02:31:39.121 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-30T02:31:39.196 INFO:journalctl@ceph.osd.3.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:31:39.354 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:31:39.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:39 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:39.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:39 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:39.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:39 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:39.735 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:39 smithi067 ceph-mon[37808]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-30T02:31:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:39 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:39 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:39 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:39 smithi082 ceph-mon[79176]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-30T02:31:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:39 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:39 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:39 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:39 smithi170 ceph-mon[78798]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-30T02:31:40.414 INFO:journalctl@ceph.osd.3.smithi067.stdout:Mar 30 02:31:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-3[64175]: 2024-03-30T02:31:40.005+0000 7f88ee08a640 -1 osd.3 0 log_to_monitors true 2024-03-30T02:31:40.642 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-30T02:31:40.642 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-30T02:31:40.642 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-03-30T02:31:40.642 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-03-30T02:31:40.643 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127436 s, 823 MB/s 2024-03-30T02:31:40.643 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-03-30T02:31:40.919 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:40 smithi082 ceph-mon[79176]: from='osd.3 [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-30T02:31:40.919 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:40 smithi082 ceph-mon[79176]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-30T02:31:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:40 smithi170 ceph-mon[78798]: from='osd.3 [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-30T02:31:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:40 smithi170 ceph-mon[78798]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-30T02:31:41.019 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2024-03-30T02:31:41.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:40 smithi067 ceph-mon[37808]: from='osd.3 [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-30T02:31:41.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:40 smithi067 ceph-mon[37808]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-30T02:31:41.235 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:31:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:41 smithi067 ceph-mon[37808]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-30T02:31:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:41 smithi067 ceph-mon[37808]: osdmap e23: 4 total, 3 up, 4 in 2024-03-30T02:31:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:41 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:41 smithi067 ceph-mon[37808]: from='osd.3 [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:31:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:41 smithi067 ceph-mon[37808]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:31:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:41 smithi067 ceph-mon[37808]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:41 smithi170 ceph-mon[78798]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-30T02:31:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:41 smithi170 ceph-mon[78798]: osdmap e23: 4 total, 3 up, 4 in 2024-03-30T02:31:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:41 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:41 smithi170 ceph-mon[78798]: from='osd.3 [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:31:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:41 smithi170 ceph-mon[78798]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:31:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:41 smithi170 ceph-mon[78798]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:41.990 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:41 smithi082 ceph-mon[79176]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-30T02:31:41.990 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:41 smithi082 ceph-mon[79176]: osdmap e23: 4 total, 3 up, 4 in 2024-03-30T02:31:41.990 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:41 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:41.991 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:41 smithi082 ceph-mon[79176]: from='osd.3 [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:31:41.991 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:41 smithi082 ceph-mon[79176]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:31:41.991 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:41 smithi082 ceph-mon[79176]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:42.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.178+0000 7fbc11bb7700 1 -- 172.21.15.82:0/1776290835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c1018a0 msgr2=0x7fbc0c108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:42.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.178+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/1776290835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c1018a0 0x7fbc0c108150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400b0d0 tx=0x7fbc0400b3e0 comp rx=0 tx=0).stop 2024-03-30T02:31:42.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.178+0000 7fbc11bb7700 1 -- 172.21.15.82:0/1776290835 shutdown_connections 2024-03-30T02:31:42.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.178+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/1776290835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c1018a0 0x7fbc0c108150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:42.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.178+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/1776290835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc0c0ffd00 0x7fbc0c100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:42.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.178+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/1776290835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc0c100f00 0x7fbc0c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:42.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.178+0000 7fbc11bb7700 1 -- 172.21.15.82:0/1776290835 >> 172.21.15.82:0/1776290835 conn(0x7fbc0c0fb2c0 msgr2=0x7fbc0c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:31:42.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 -- 172.21.15.82:0/1776290835 shutdown_connections 2024-03-30T02:31:42.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 -- 172.21.15.82:0/1776290835 wait complete. 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 Processor -- start 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 -- start start 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c0ffd00 0x7fbc0c196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc0c100f00 0x7fbc0c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc0c1018a0 0x7fbc0c19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c19e940 con 0x7fbc0c1018a0 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c19ea80 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.179+0000 7fbc11bb7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c19ec10 con 0x7fbc0c100f00 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c0ffd00 0x7fbc0c196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c0ffd00 0x7fbc0c196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45258/0 (socket says 172.21.15.82:45258) 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0b7fe700 1 -- 172.21.15.82:0/817334558 learned_addr learned my addr 172.21.15.82:0/817334558 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-30T02:31:42.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0bfff700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc0c1018a0 0x7fbc0c19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0affd700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc0c100f00 0x7fbc0c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0b7fe700 1 -- 172.21.15.82:0/817334558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc0c100f00 msgr2=0x7fbc0c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0b7fe700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc0c100f00 0x7fbc0c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0b7fe700 1 -- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc0c1018a0 msgr2=0x7fbc0c19e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0b7fe700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc0c1018a0 0x7fbc0c19e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0b7fe700 1 -- 172.21.15.82:0/817334558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc04009d00 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc0b7fe700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c0ffd00 0x7fbc0c196e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00d980 tx=0x7fbbfc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc08ff9700 1 -- 172.21.15.82:0/817334558 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc004020 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc08ff9700 1 -- 172.21.15.82:0/817334558 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbfc009730 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0c19eef0 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc08ff9700 1 -- 172.21.15.82:0/817334558 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc003b20 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.180+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0c19f310 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.181+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0c04ea80 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.183+0000 7fbc08ff9700 1 -- 172.21.15.82:0/817334558 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbbfc0098a0 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.183+0000 7fbc08ff9700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbbf40a11a0 0x7fbbf40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:42.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.183+0000 7fbc08ff9700 1 -- 172.21.15.82:0/817334558 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3389+0+0 (secure 0 0 0) 0x7fbbfc0cb510 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.183+0000 7fbc0affd700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbbf40a11a0 0x7fbbf40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:42.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.184+0000 7fbc0affd700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbbf40a11a0 0x7fbbf40a3660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005950 tx=0x7fbc000058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:31:42.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.187+0000 7fbc08ff9700 1 -- 172.21.15.82:0/817334558 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbbfc096060 con 0x7fbc0c0ffd00 2024-03-30T02:31:42.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:42.389+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc0c0fd4d0 con 0x7fbbf40a11a0 2024-03-30T02:31:42.663 INFO:journalctl@ceph.osd.3.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-3[64175]: 2024-03-30T02:31:42.220+0000 7f88e2600700 -1 osd.3 0 waiting for initial osdmap 2024-03-30T02:31:42.664 INFO:journalctl@ceph.osd.3.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-3[64175]: 2024-03-30T02:31:42.237+0000 7f88d8400700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: osdmap e24: 4 total, 3 up, 4 in 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi067 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi067 to 8635M 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='osd.3 ' entity='osd.3' 2024-03-30T02:31:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:31:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:31:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:43.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:31:43.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: osdmap e24: 4 total, 3 up, 4 in 2024-03-30T02:31:43.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:43.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:43.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi067 2024-03-30T02:31:43.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:43.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi067 to 8635M 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='osd.3 ' entity='osd.3' 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:31:43.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: osdmap e24: 4 total, 3 up, 4 in 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi067 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi067 to 8635M 2024-03-30T02:31:43.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:43.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:43.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:43.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:43.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='osd.3 ' entity='osd.3' 2024-03-30T02:31:43.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:31:43.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:31:43.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:43 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:31:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:43 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:31:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:43 smithi170 ceph-mon[78798]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:31:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:43 smithi170 ceph-mon[78798]: osd.3 [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] boot 2024-03-30T02:31:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:43 smithi170 ceph-mon[78798]: osdmap e25: 4 total, 4 up, 4 in 2024-03-30T02:31:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:43 smithi170 ceph-mon[78798]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:44.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:43 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:31:44.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:43 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:31:44.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:43 smithi082 ceph-mon[79176]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:31:44.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:43 smithi082 ceph-mon[79176]: osd.3 [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] boot 2024-03-30T02:31:44.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:43 smithi082 ceph-mon[79176]: osdmap e25: 4 total, 4 up, 4 in 2024-03-30T02:31:44.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:44.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:43 smithi082 ceph-mon[79176]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:44.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:43 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:31:44.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:43 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:31:44.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:43 smithi067 ceph-mon[37808]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:31:44.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:43 smithi067 ceph-mon[37808]: osd.3 [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] boot 2024-03-30T02:31:44.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:43 smithi067 ceph-mon[37808]: osdmap e25: 4 total, 4 up, 4 in 2024-03-30T02:31:44.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:31:44.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:43 smithi067 ceph-mon[37808]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-30T02:31:45.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:44 smithi082 ceph-mon[79176]: osdmap e26: 4 total, 4 up, 4 in 2024-03-30T02:31:45.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:44 smithi082 ceph-mon[79176]: from='client.? 172.21.15.82:0/802139525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a35b5bb-587d-489e-941d-5b9ec9c1c2e4"}]: dispatch 2024-03-30T02:31:45.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:44 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a35b5bb-587d-489e-941d-5b9ec9c1c2e4"}]: dispatch 2024-03-30T02:31:45.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:44 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a35b5bb-587d-489e-941d-5b9ec9c1c2e4"}]': finished 2024-03-30T02:31:45.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:44 smithi082 ceph-mon[79176]: osdmap e27: 5 total, 4 up, 5 in 2024-03-30T02:31:45.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:44 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:45.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:44 smithi067 ceph-mon[37808]: osdmap e26: 4 total, 4 up, 4 in 2024-03-30T02:31:45.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:44 smithi067 ceph-mon[37808]: from='client.? 172.21.15.82:0/802139525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a35b5bb-587d-489e-941d-5b9ec9c1c2e4"}]: dispatch 2024-03-30T02:31:45.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:44 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a35b5bb-587d-489e-941d-5b9ec9c1c2e4"}]: dispatch 2024-03-30T02:31:45.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:44 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a35b5bb-587d-489e-941d-5b9ec9c1c2e4"}]': finished 2024-03-30T02:31:45.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:44 smithi067 ceph-mon[37808]: osdmap e27: 5 total, 4 up, 5 in 2024-03-30T02:31:45.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:45.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:44 smithi170 ceph-mon[78798]: osdmap e26: 4 total, 4 up, 4 in 2024-03-30T02:31:45.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:44 smithi170 ceph-mon[78798]: from='client.? 172.21.15.82:0/802139525' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a35b5bb-587d-489e-941d-5b9ec9c1c2e4"}]: dispatch 2024-03-30T02:31:45.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:44 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a35b5bb-587d-489e-941d-5b9ec9c1c2e4"}]: dispatch 2024-03-30T02:31:45.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:44 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a35b5bb-587d-489e-941d-5b9ec9c1c2e4"}]': finished 2024-03-30T02:31:45.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:44 smithi170 ceph-mon[78798]: osdmap e27: 5 total, 4 up, 5 in 2024-03-30T02:31:45.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:44 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:46.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:45 smithi082 ceph-mon[79176]: from='client.? 172.21.15.82:0/4147786611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:31:46.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:45 smithi082 ceph-mon[79176]: pgmap v86: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:31:46.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:45 smithi082 ceph-mon[79176]: osdmap e28: 5 total, 4 up, 5 in 2024-03-30T02:31:46.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:45 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:46.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:45 smithi067 ceph-mon[37808]: from='client.? 172.21.15.82:0/4147786611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:31:46.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:45 smithi067 ceph-mon[37808]: pgmap v86: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:31:46.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:45 smithi067 ceph-mon[37808]: osdmap e28: 5 total, 4 up, 5 in 2024-03-30T02:31:46.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:45 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:46.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:45 smithi170 ceph-mon[78798]: from='client.? 172.21.15.82:0/4147786611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:31:46.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:45 smithi170 ceph-mon[78798]: pgmap v86: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:31:46.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:45 smithi170 ceph-mon[78798]: osdmap e28: 5 total, 4 up, 5 in 2024-03-30T02:31:46.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:45 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:47 smithi067 ceph-mon[37808]: osdmap e29: 5 total, 4 up, 5 in 2024-03-30T02:31:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:47.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:47 smithi067 ceph-mon[37808]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:31:47.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:47 smithi170 ceph-mon[78798]: osdmap e29: 5 total, 4 up, 5 in 2024-03-30T02:31:47.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:47.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:47 smithi170 ceph-mon[78798]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:31:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:47 smithi082 ceph-mon[79176]: osdmap e29: 5 total, 4 up, 5 in 2024-03-30T02:31:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:47 smithi082 ceph-mon[79176]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:31:50.345 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:50 smithi082 ceph-mon[79176]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-30T02:31:50.345 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:50 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-30T02:31:50.346 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:50 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:50.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:50 smithi067 ceph-mon[37808]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-30T02:31:50.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:50 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-30T02:31:50.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:50 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:50.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:50 smithi170 ceph-mon[78798]: pgmap v90: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-30T02:31:50.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:50 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-30T02:31:50.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:50 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:51.355 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:51 smithi082 ceph-mon[79176]: Deploying daemon osd.4 on smithi082 2024-03-30T02:31:51.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:51 smithi067 ceph-mon[37808]: Deploying daemon osd.4 on smithi082 2024-03-30T02:31:51.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:51 smithi170 ceph-mon[78798]: Deploying daemon osd.4 on smithi082 2024-03-30T02:31:52.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:52 smithi082 ceph-mon[79176]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-30T02:31:52.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:52 smithi067 ceph-mon[37808]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-30T02:31:52.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:52 smithi170 ceph-mon[78798]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-30T02:31:54.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:54 smithi082 ceph-mon[79176]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:54.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:54 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:54.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:54 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:54.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:54 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:54.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:54 smithi067 ceph-mon[37808]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:54.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:54 smithi170 ceph-mon[78798]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:54.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:54 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:54.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:54 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:54.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:54 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.743+0000 7fbc08ff9700 1 -- 172.21.15.82:0/817334558 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbc0c0fd4d0 con 0x7fbbf40a11a0 2024-03-30T02:31:55.744 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 4 on host 'smithi082' 2024-03-30T02:31:55.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbbf40a11a0 msgr2=0x7fbbf40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:55.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbbf40a11a0 0x7fbbf40a3660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005950 tx=0x7fbc000058e0 comp rx=0 tx=0).stop 2024-03-30T02:31:55.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c0ffd00 msgr2=0x7fbc0c196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c0ffd00 0x7fbc0c196e00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00d980 tx=0x7fbbfc00dc90 comp rx=0 tx=0).stop 2024-03-30T02:31:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 shutdown_connections 2024-03-30T02:31:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbbf40a11a0 0x7fbbf40a3660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc0c1018a0 0x7fbc0c19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc0c0ffd00 0x7fbc0c196e00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 --2- 172.21.15.82:0/817334558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc0c100f00 0x7fbc0c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 >> 172.21.15.82:0/817334558 conn(0x7fbc0c0fb2c0 msgr2=0x7fbc0c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:31:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 shutdown_connections 2024-03-30T02:31:55.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:31:55.745+0000 7fbc11bb7700 1 -- 172.21.15.82:0/817334558 wait complete. 2024-03-30T02:31:56.145 DEBUG:teuthology.orchestra.run.smithi082:osd.4> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.4.service 2024-03-30T02:31:56.148 INFO:tasks.cephadm:Deploying osd.5 on smithi170 with /dev/vg_nvme/lv_3... 2024-03-30T02:31:56.148 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-30T02:31:56.177 INFO:journalctl@ceph.osd.4.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:31:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:56 smithi082 ceph-mon[79176]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.377 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:56.382 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:31:56.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:56 smithi067 ceph-mon[37808]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:56.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:56.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:56 smithi170 ceph-mon[78798]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:56.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:56.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:56.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:56.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:31:57.718 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-30T02:31:57.718 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-30T02:31:57.718 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-03-30T02:31:57.718 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-03-30T02:31:57.718 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134368 s, 780 MB/s 2024-03-30T02:31:57.718 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-03-30T02:31:57.750 INFO:journalctl@ceph.osd.4.smithi082.stdout:Mar 30 02:31:57 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-4[92189]: 2024-03-30T02:31:57.446+0000 7f68b94f9640 -1 osd.4 0 log_to_monitors true 2024-03-30T02:31:58.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2024-03-30T02:31:58.156 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:58 smithi082 ceph-mon[79176]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:58.156 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:58 smithi082 ceph-mon[79176]: from='osd.4 [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T02:31:58.156 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:58 smithi082 ceph-mon[79176]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T02:31:58.332 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:31:58.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:58 smithi067 ceph-mon[37808]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:58.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:58 smithi067 ceph-mon[37808]: from='osd.4 [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T02:31:58.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:58 smithi067 ceph-mon[37808]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T02:31:58.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:58 smithi170 ceph-mon[78798]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:58 smithi170 ceph-mon[78798]: from='osd.4 [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T02:31:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:58 smithi170 ceph-mon[78798]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 -- 172.21.15.170:0/632926421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c0ffcf0 msgr2=0x7f180c100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 --2- 172.21.15.170:0/632926421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c0ffcf0 0x7f180c100110 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009990 tx=0x7f17fc009ca0 comp rx=0 tx=0).stop 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 -- 172.21.15.170:0/632926421 shutdown_connections 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 --2- 172.21.15.170:0/632926421 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180c101890 0x7f180c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 --2- 172.21.15.170:0/632926421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c0ffcf0 0x7f180c100110 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 --2- 172.21.15.170:0/632926421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f180c100ef0 0x7f180c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 -- 172.21.15.170:0/632926421 >> 172.21.15.170:0/632926421 conn(0x7f180c0fb2b0 msgr2=0x7f180c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 -- 172.21.15.170:0/632926421 shutdown_connections 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 -- 172.21.15.170:0/632926421 wait complete. 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.261+0000 7f1810a6b700 1 Processor -- start 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f1810a6b700 1 -- start start 2024-03-30T02:31:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f1810a6b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180c0ffcf0 0x7f180c069210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f1810a6b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f180c100ef0 0x7f180c19dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f1810a6b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c101890 0x7f180c1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f1810a6b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c1a0a20 con 0x7f180c0ffcf0 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f1810a6b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c1a0b60 con 0x7f180c100ef0 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f1810a6b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c1a0d20 con 0x7f180c101890 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c101890 0x7f180c1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c101890 0x7f180c1a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46184/0 (socket says 172.21.15.170:46184) 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180ad9d700 1 -- 172.21.15.170:0/3257089823 learned_addr learned my addr 172.21.15.170:0/3257089823 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f1809d9b700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f180c100ef0 0x7f180c19dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180a59c700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180c0ffcf0 0x7f180c069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180ad9d700 1 -- 172.21.15.170:0/3257089823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f180c100ef0 msgr2=0x7f180c19dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180ad9d700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f180c100ef0 0x7f180c19dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:59.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180ad9d700 1 -- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180c0ffcf0 msgr2=0x7f180c069210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:31:59.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180ad9d700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180c0ffcf0 0x7f180c069210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:31:59.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180ad9d700 1 -- 172.21.15.170:0/3257089823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0095f0 con 0x7f180c101890 2024-03-30T02:31:59.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f180ad9d700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c101890 0x7f180c1a0410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1804007ee0 tx=0x7f180400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:31:59.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f17fb7fe700 1 -- 172.21.15.170:0/3257089823 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804017070 con 0x7f180c101890 2024-03-30T02:31:59.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f17fb7fe700 1 -- 172.21.15.170:0/3257089823 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f180400f040 con 0x7f180c101890 2024-03-30T02:31:59.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.262+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c1a0fa0 con 0x7f180c101890 2024-03-30T02:31:59.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.263+0000 7f17fb7fe700 1 -- 172.21.15.170:0/3257089823 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804005740 con 0x7f180c101890 2024-03-30T02:31:59.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.263+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c1a1420 con 0x7f180c101890 2024-03-30T02:31:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.264+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c1071d0 con 0x7f180c101890 2024-03-30T02:31:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.265+0000 7f17fb7fe700 1 -- 172.21.15.170:0/3257089823 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f180400dc60 con 0x7f180c101890 2024-03-30T02:31:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.265+0000 7f17fb7fe700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17f40a1120 0x7f17f40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:31:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.266+0000 7f17fb7fe700 1 -- 172.21.15.170:0/3257089823 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3821+0+0 (secure 0 0 0) 0x7f18040cb540 con 0x7f180c101890 2024-03-30T02:31:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.266+0000 7f180a59c700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17f40a1120 0x7f17f40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:31:59.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.266+0000 7f180a59c700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17f40a1120 0x7f17f40a35e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b7b0 tx=0x7f17fc005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:31:59.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.269+0000 7f17fb7fe700 1 -- 172.21.15.170:0/3257089823 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1804095e60 con 0x7f180c101890 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: osdmap e30: 5 total, 4 up, 5 in 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='osd.4 [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi082 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:59.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:59.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi082 to 8556M 2024-03-30T02:31:59.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:59.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:59.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:31:59 smithi067 ceph-mon[37808]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: osdmap e30: 5 total, 4 up, 5 in 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='osd.4 [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi082 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi082 to 8556M 2024-03-30T02:31:59.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:59.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:59.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:31:59 smithi170 ceph-mon[78798]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:31:59.471 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:31:59.469+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f180c02cd20 con 0x7f17f40a1120 2024-03-30T02:31:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-30T02:31:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: osdmap e30: 5 total, 4 up, 5 in 2024-03-30T02:31:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:31:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='osd.4 [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:31:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi082 2024-03-30T02:31:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:59.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:31:59.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi082 to 8556M 2024-03-30T02:31:59.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:31:59.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:31:59.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:31:59.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-mon[79176]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:32:00.000 INFO:journalctl@ceph.osd.4.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-4[92189]: 2024-03-30T02:31:59.512+0000 7f68af35e700 -1 osd.4 0 waiting for initial osdmap 2024-03-30T02:32:00.000 INFO:journalctl@ceph.osd.4.smithi082.stdout:Mar 30 02:31:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-4[92189]: 2024-03-30T02:31:59.518+0000 7f68a694a700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:32:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:00 smithi067 ceph-mon[37808]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:32:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:00 smithi067 ceph-mon[37808]: osdmap e31: 5 total, 4 up, 5 in 2024-03-30T02:32:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:00 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:00 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:00 smithi067 ceph-mon[37808]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:00 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:00 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:00 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:00 smithi067 ceph-mon[37808]: from='osd.4 ' entity='osd.4' 2024-03-30T02:32:00.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:00 smithi170 ceph-mon[78798]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:32:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:00 smithi170 ceph-mon[78798]: osdmap e31: 5 total, 4 up, 5 in 2024-03-30T02:32:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:00 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:00 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:00 smithi170 ceph-mon[78798]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:00 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:00 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:00 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:00 smithi170 ceph-mon[78798]: from='osd.4 ' entity='osd.4' 2024-03-30T02:32:00.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:00 smithi082 ceph-mon[79176]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:32:00.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:00 smithi082 ceph-mon[79176]: osdmap e31: 5 total, 4 up, 5 in 2024-03-30T02:32:00.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:00 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:00.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:00 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:00.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:00 smithi082 ceph-mon[79176]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:00.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:00 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:00.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:00 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:00.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:00 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:00.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:00 smithi082 ceph-mon[79176]: from='osd.4 ' entity='osd.4' 2024-03-30T02:32:01.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:01 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:32:01.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:01 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:32:01.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:01 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:01.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:01 smithi067 ceph-mon[37808]: osd.4 [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] boot 2024-03-30T02:32:01.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:01 smithi067 ceph-mon[37808]: osdmap e32: 5 total, 5 up, 5 in 2024-03-30T02:32:01.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:01 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:01.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:01 smithi067 ceph-mon[37808]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:32:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:01 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:32:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:01 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:32:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:01 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:01 smithi170 ceph-mon[78798]: osd.4 [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] boot 2024-03-30T02:32:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:01 smithi170 ceph-mon[78798]: osdmap e32: 5 total, 5 up, 5 in 2024-03-30T02:32:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:01 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:01 smithi170 ceph-mon[78798]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:32:01.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:01 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:32:01.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:01 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:32:01.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:01 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:01.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:01 smithi082 ceph-mon[79176]: osd.4 [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] boot 2024-03-30T02:32:01.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:01 smithi082 ceph-mon[79176]: osdmap e32: 5 total, 5 up, 5 in 2024-03-30T02:32:01.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:01 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:32:01.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:01 smithi082 ceph-mon[79176]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-30T02:32:02.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:02 smithi067 ceph-mon[37808]: from='client.? 172.21.15.170:0/3448743021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "787b667a-b309-4292-8a66-d7716b659d95"}]: dispatch 2024-03-30T02:32:02.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:02 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "787b667a-b309-4292-8a66-d7716b659d95"}]: dispatch 2024-03-30T02:32:02.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:02 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "787b667a-b309-4292-8a66-d7716b659d95"}]': finished 2024-03-30T02:32:02.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:02 smithi067 ceph-mon[37808]: osdmap e33: 6 total, 5 up, 6 in 2024-03-30T02:32:02.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:02 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:02.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:02 smithi067 ceph-mon[37808]: from='client.? 172.21.15.170:0/503771837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:02.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:02 smithi170 ceph-mon[78798]: from='client.? 172.21.15.170:0/3448743021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "787b667a-b309-4292-8a66-d7716b659d95"}]: dispatch 2024-03-30T02:32:02.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:02 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "787b667a-b309-4292-8a66-d7716b659d95"}]: dispatch 2024-03-30T02:32:02.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:02 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "787b667a-b309-4292-8a66-d7716b659d95"}]': finished 2024-03-30T02:32:02.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:02 smithi170 ceph-mon[78798]: osdmap e33: 6 total, 5 up, 6 in 2024-03-30T02:32:02.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:02 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:02.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:02 smithi170 ceph-mon[78798]: from='client.? 172.21.15.170:0/503771837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:02.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:02 smithi082 ceph-mon[79176]: from='client.? 172.21.15.170:0/3448743021' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "787b667a-b309-4292-8a66-d7716b659d95"}]: dispatch 2024-03-30T02:32:02.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:02 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "787b667a-b309-4292-8a66-d7716b659d95"}]: dispatch 2024-03-30T02:32:02.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:02 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "787b667a-b309-4292-8a66-d7716b659d95"}]': finished 2024-03-30T02:32:02.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:02 smithi082 ceph-mon[79176]: osdmap e33: 6 total, 5 up, 6 in 2024-03-30T02:32:02.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:02 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:02.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:02 smithi082 ceph-mon[79176]: from='client.? 172.21.15.170:0/503771837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:03.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:03 smithi067 ceph-mon[37808]: osdmap e34: 6 total, 5 up, 6 in 2024-03-30T02:32:03.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:03 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:03.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:03 smithi067 ceph-mon[37808]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:03.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:03 smithi170 ceph-mon[78798]: osdmap e34: 6 total, 5 up, 6 in 2024-03-30T02:32:03.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:03 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:03.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:03 smithi170 ceph-mon[78798]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:03.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:03 smithi082 ceph-mon[79176]: osdmap e34: 6 total, 5 up, 6 in 2024-03-30T02:32:03.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:03 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:03.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:03 smithi082 ceph-mon[79176]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:06.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:06 smithi170 ceph-mon[78798]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:06.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:06 smithi067 ceph-mon[37808]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:06 smithi082 ceph-mon[79176]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:08.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:08 smithi170 ceph-mon[78798]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:08.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:08 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-30T02:32:08.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:08 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:08.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:08 smithi067 ceph-mon[37808]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:08.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:08 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-30T02:32:08.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:08 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:08 smithi082 ceph-mon[79176]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:08 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-30T02:32:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:08 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:09.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:09 smithi170 ceph-mon[78798]: Deploying daemon osd.5 on smithi170 2024-03-30T02:32:09.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:09 smithi067 ceph-mon[37808]: Deploying daemon osd.5 on smithi170 2024-03-30T02:32:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:09 smithi082 ceph-mon[79176]: Deploying daemon osd.5 on smithi170 2024-03-30T02:32:10.325 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:10 smithi170 ceph-mon[78798]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:10.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:10 smithi067 ceph-mon[37808]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:10.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:10 smithi082 ceph-mon[79176]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:12.134 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:11 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:12.134 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:11 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:12.134 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:11 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:12.134 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:11 smithi170 ceph-mon[78798]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:12.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:11 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:12.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:11 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:12.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:11 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:12.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:11 smithi067 ceph-mon[37808]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:11 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:11 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:11 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:11 smithi082 ceph-mon[79176]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:13.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.132+0000 7f17fb7fe700 1 -- 172.21.15.170:0/3257089823 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f180c02cd20 con 0x7f17f40a1120 2024-03-30T02:32:13.134 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 5 on host 'smithi170' 2024-03-30T02:32:13.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17f40a1120 msgr2=0x7f17f40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:13.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17f40a1120 0x7f17f40a35e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00b7b0 tx=0x7f17fc005fb0 comp rx=0 tx=0).stop 2024-03-30T02:32:13.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c101890 msgr2=0x7f180c1a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:13.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c101890 0x7f180c1a0410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1804007ee0 tx=0x7f180400d3b0 comp rx=0 tx=0).stop 2024-03-30T02:32:13.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 shutdown_connections 2024-03-30T02:32:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17f40a1120 0x7f17f40a35e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f180c0ffcf0 0x7f180c069210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f180c101890 0x7f180c1a0410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 --2- 172.21.15.170:0/3257089823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f180c100ef0 0x7f180c19dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.134+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 >> 172.21.15.170:0/3257089823 conn(0x7f180c0fb2b0 msgr2=0x7f180c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:32:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.135+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 shutdown_connections 2024-03-30T02:32:13.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:13.135+0000 7f1810a6b700 1 -- 172.21.15.170:0/3257089823 wait complete. 2024-03-30T02:32:13.524 DEBUG:teuthology.orchestra.run.smithi170:osd.5> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.5.service 2024-03-30T02:32:13.526 INFO:tasks.cephadm:Deploying osd.6 on smithi067 with /dev/vg_nvme/lv_2... 2024-03-30T02:32:13.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-30T02:32:13.558 INFO:journalctl@ceph.osd.5.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:32:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:13 smithi170 ceph-mon[78798]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:13 smithi082 ceph-mon[79176]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:13.899 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:32:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:13 smithi067 ceph-mon[37808]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:14 smithi067 ceph-mon[37808]: from='osd.5 [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T02:32:15.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:14 smithi067 ceph-mon[37808]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T02:32:15.190 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:32:14 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:32:14.907+0000 7f20fc4fd640 -1 osd.5 0 log_to_monitors true 2024-03-30T02:32:15.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:14 smithi170 ceph-mon[78798]: from='osd.5 [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T02:32:15.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:14 smithi170 ceph-mon[78798]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T02:32:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:14 smithi082 ceph-mon[79176]: from='osd.5 [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T02:32:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:14 smithi082 ceph-mon[79176]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-30T02:32:15.626 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-30T02:32:15.627 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-30T02:32:15.627 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-03-30T02:32:15.627 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-03-30T02:32:15.627 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0331039 s, 317 MB/s 2024-03-30T02:32:15.627 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-03-30T02:32:15.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi067:vg_nvme/lv_2 2024-03-30T02:32:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-30T02:32:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: osdmap e35: 6 total, 5 up, 6 in 2024-03-30T02:32:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='osd.5 [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: osdmap e36: 6 total, 5 up, 6 in 2024-03-30T02:32:16.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:15 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:16.345 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:32:16.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: osdmap e35: 6 total, 5 up, 6 in 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='osd.5 [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:16.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:16.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:32:16.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: osdmap e36: 6 total, 5 up, 6 in 2024-03-30T02:32:16.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:15 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: osdmap e35: 6 total, 5 up, 6 in 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='osd.5 [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:16.434 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:16.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:16.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:16.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:16.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:32:16.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: osdmap e36: 6 total, 5 up, 6 in 2024-03-30T02:32:16.435 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:15 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:16.689 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:32:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:32:16.433+0000 7f20f2362700 -1 osd.5 0 waiting for initial osdmap 2024-03-30T02:32:16.690 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:32:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:32:16.439+0000 7f20e994e700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:32:17.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:16 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi170 2024-03-30T02:32:17.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:16 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi170 to 8557M 2024-03-30T02:32:17.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:16 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:17.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:16 smithi067 ceph-mon[37808]: from='osd.5 ' entity='osd.5' 2024-03-30T02:32:17.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:16 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:17.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:16 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi170 2024-03-30T02:32:17.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:16 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi170 to 8557M 2024-03-30T02:32:17.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:16 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:17.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:16 smithi082 ceph-mon[79176]: from='osd.5 ' entity='osd.5' 2024-03-30T02:32:17.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:16 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:17.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:16 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi170 2024-03-30T02:32:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:16 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi170 to 8557M 2024-03-30T02:32:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:16 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:16 smithi170 ceph-mon[78798]: from='osd.5 ' entity='osd.5' 2024-03-30T02:32:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:16 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.510+0000 7efe49800700 1 -- 172.21.15.67:0/761502265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44074d60 msgr2=0x7efe440731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.510+0000 7efe49800700 1 --2- 172.21.15.67:0/761502265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44074d60 0x7efe440731c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe34009990 tx=0x7efe34009ca0 comp rx=0 tx=0).stop 2024-03-30T02:32:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.510+0000 7efe49800700 1 -- 172.21.15.67:0/761502265 shutdown_connections 2024-03-30T02:32:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.510+0000 7efe49800700 1 --2- 172.21.15.67:0/761502265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44074d60 0x7efe440731c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.510+0000 7efe49800700 1 --2- 172.21.15.67:0/761502265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44108ce0 0x7efe4410b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.510+0000 7efe49800700 1 --2- 172.21.15.67:0/761502265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44073790 0x7efe44073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.510+0000 7efe49800700 1 -- 172.21.15.67:0/761502265 >> 172.21.15.67:0/761502265 conn(0x7efe440fc380 msgr2=0x7efe440fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:32:17.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.510+0000 7efe49800700 1 -- 172.21.15.67:0/761502265 shutdown_connections 2024-03-30T02:32:17.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.510+0000 7efe49800700 1 -- 172.21.15.67:0/761502265 wait complete. 2024-03-30T02:32:17.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.511+0000 7efe49800700 1 Processor -- start 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.511+0000 7efe49800700 1 -- start start 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe49800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44073790 0x7efe44198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe49800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44074d60 0x7efe44199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe49800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44108ce0 0x7efe441a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe49800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe441a0a40 con 0x7efe44073790 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe49800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe441a0b80 con 0x7efe44108ce0 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe49800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe441a0d10 con 0x7efe44074d60 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe42a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44073790 0x7efe44198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe42a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44073790 0x7efe44198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49488/0 (socket says 172.21.15.67:49488) 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe42a00700 1 -- 172.21.15.67:0/1018749847 learned_addr learned my addr 172.21.15.67:0/1018749847 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe42000700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44074d60 0x7efe44199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.512+0000 7efe43400700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44108ce0 0x7efe441a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:17.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe42a00700 1 -- 172.21.15.67:0/1018749847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44074d60 msgr2=0x7efe44199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe42a00700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44074d60 0x7efe44199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe42a00700 1 -- 172.21.15.67:0/1018749847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44108ce0 msgr2=0x7efe441a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe42a00700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44108ce0 0x7efe441a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe42a00700 1 -- 172.21.15.67:0/1018749847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe340095f0 con 0x7efe44073790 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe42a00700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44073790 0x7efe44198f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efe34005910 tx=0x7efe34009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe3b400700 1 -- 172.21.15.67:0/1018749847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3401d070 con 0x7efe44073790 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe3b400700 1 -- 172.21.15.67:0/1018749847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe34022470 con 0x7efe44073790 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe441a0f90 con 0x7efe44073790 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe3b400700 1 -- 172.21.15.67:0/1018749847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34017650 con 0x7efe44073790 2024-03-30T02:32:17.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.513+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe441a12f0 con 0x7efe44073790 2024-03-30T02:32:17.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.514+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe44107af0 con 0x7efe44073790 2024-03-30T02:32:17.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.516+0000 7efe3b400700 1 -- 172.21.15.67:0/1018749847 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7efe3400fad0 con 0x7efe44073790 2024-03-30T02:32:17.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.518+0000 7efe3b400700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe280a1120 0x7efe280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:17.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.518+0000 7efe3b400700 1 -- 172.21.15.67:0/1018749847 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4566+0+0 (secure 0 0 0) 0x7efe340cd2d0 con 0x7efe44073790 2024-03-30T02:32:17.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.518+0000 7efe42000700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe280a1120 0x7efe280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:17.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.519+0000 7efe42000700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe280a1120 0x7efe280a35e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efe30009c60 tx=0x7efe30009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:32:17.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.521+0000 7efe3b400700 1 -- 172.21.15.67:0/1018749847 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efe340978f0 con 0x7efe44073790 2024-03-30T02:32:17.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:17.742+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7efe4402cdc0 con 0x7efe280a1120 2024-03-30T02:32:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:17 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:32:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:17 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:32:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:17 smithi082 ceph-mon[79176]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:17 smithi082 ceph-mon[79176]: osd.5 [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] boot 2024-03-30T02:32:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:17 smithi082 ceph-mon[79176]: osdmap e37: 6 total, 6 up, 6 in 2024-03-30T02:32:18.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:18.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:18.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:18.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:18.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:17 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:32:18.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:17 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:32:18.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:17 smithi067 ceph-mon[37808]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:18.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:17 smithi067 ceph-mon[37808]: osd.5 [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] boot 2024-03-30T02:32:18.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:17 smithi067 ceph-mon[37808]: osdmap e37: 6 total, 6 up, 6 in 2024-03-30T02:32:18.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:18.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:18.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:18.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:18.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:17 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:32:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:17 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:32:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:17 smithi170 ceph-mon[78798]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-30T02:32:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:17 smithi170 ceph-mon[78798]: osd.5 [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] boot 2024-03-30T02:32:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:17 smithi170 ceph-mon[78798]: osdmap e37: 6 total, 6 up, 6 in 2024-03-30T02:32:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:32:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:19 smithi067 ceph-mon[37808]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:19.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:19 smithi067 ceph-mon[37808]: osdmap e38: 6 total, 6 up, 6 in 2024-03-30T02:32:19.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:18 smithi170 ceph-mon[78798]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:19.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:18 smithi170 ceph-mon[78798]: osdmap e38: 6 total, 6 up, 6 in 2024-03-30T02:32:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:19 smithi082 ceph-mon[79176]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:19 smithi082 ceph-mon[79176]: osdmap e38: 6 total, 6 up, 6 in 2024-03-30T02:32:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:20 smithi067 ceph-mon[37808]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:20 smithi067 ceph-mon[37808]: osdmap e39: 6 total, 6 up, 6 in 2024-03-30T02:32:20.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:20 smithi170 ceph-mon[78798]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:20 smithi170 ceph-mon[78798]: osdmap e39: 6 total, 6 up, 6 in 2024-03-30T02:32:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:20 smithi082 ceph-mon[79176]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:20 smithi082 ceph-mon[79176]: osdmap e39: 6 total, 6 up, 6 in 2024-03-30T02:32:21.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:21 smithi067 ceph-mon[37808]: osdmap e40: 6 total, 6 up, 6 in 2024-03-30T02:32:21.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:21 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2784675079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "986d28e3-52e9-462b-b728-d76c2457f15a"}]: dispatch 2024-03-30T02:32:21.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:21 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2784675079' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "986d28e3-52e9-462b-b728-d76c2457f15a"}]': finished 2024-03-30T02:32:21.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:21 smithi067 ceph-mon[37808]: osdmap e41: 7 total, 6 up, 7 in 2024-03-30T02:32:21.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:21 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:21.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:21 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/625679436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:21.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:21 smithi170 ceph-mon[78798]: osdmap e40: 6 total, 6 up, 6 in 2024-03-30T02:32:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:21 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2784675079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "986d28e3-52e9-462b-b728-d76c2457f15a"}]: dispatch 2024-03-30T02:32:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:21 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2784675079' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "986d28e3-52e9-462b-b728-d76c2457f15a"}]': finished 2024-03-30T02:32:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:21 smithi170 ceph-mon[78798]: osdmap e41: 7 total, 6 up, 7 in 2024-03-30T02:32:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:21 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:21.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:21 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/625679436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:21 smithi082 ceph-mon[79176]: osdmap e40: 6 total, 6 up, 6 in 2024-03-30T02:32:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:21 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2784675079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "986d28e3-52e9-462b-b728-d76c2457f15a"}]: dispatch 2024-03-30T02:32:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:21 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2784675079' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "986d28e3-52e9-462b-b728-d76c2457f15a"}]': finished 2024-03-30T02:32:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:21 smithi082 ceph-mon[79176]: osdmap e41: 7 total, 6 up, 7 in 2024-03-30T02:32:21.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:21 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:21.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:21 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/625679436' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:22 smithi067 ceph-mon[37808]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:32:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:22 smithi170 ceph-mon[78798]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:32:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:22 smithi082 ceph-mon[79176]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:32:24.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:24 smithi067 ceph-mon[37808]: pgmap v119: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-30T02:32:24.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:24 smithi170 ceph-mon[78798]: pgmap v119: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-30T02:32:24.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:24 smithi082 ceph-mon[79176]: pgmap v119: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-30T02:32:26.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:26 smithi067 ceph-mon[37808]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T02:32:26.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:26 smithi170 ceph-mon[78798]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T02:32:26.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:26 smithi082 ceph-mon[79176]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-30T02:32:27.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-30T02:32:27.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:27.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-30T02:32:27.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:27.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-30T02:32:27.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:28.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:28 smithi067 ceph-mon[37808]: Deploying daemon osd.6 on smithi067 2024-03-30T02:32:28.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:28 smithi067 ceph-mon[37808]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T02:32:28.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:28 smithi170 ceph-mon[78798]: Deploying daemon osd.6 on smithi067 2024-03-30T02:32:28.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:28 smithi170 ceph-mon[78798]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T02:32:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:28 smithi082 ceph-mon[79176]: Deploying daemon osd.6 on smithi067 2024-03-30T02:32:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:28 smithi082 ceph-mon[79176]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T02:32:30.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:30 smithi067 ceph-mon[37808]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:30.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:30 smithi170 ceph-mon[78798]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:30.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:30 smithi082 ceph-mon[79176]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:32.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:31 smithi067 ceph-mon[37808]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:32.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:31 smithi082 ceph-mon[79176]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:32.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:32.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:32.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:32.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:31 smithi170 ceph-mon[78798]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:32.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:32.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:32.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:34.406 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:34 smithi067 ceph-mon[37808]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:34.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:34.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:34.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:34.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:34.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:34.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:34 smithi170 ceph-mon[78798]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:34.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:34.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:34.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:34.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:34.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:34 smithi082 ceph-mon[79176]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:34.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:34.797 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 6 on host 'smithi067' 2024-03-30T02:32:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.796+0000 7efe3b400700 1 -- 172.21.15.67:0/1018749847 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efe4402cdc0 con 0x7efe280a1120 2024-03-30T02:32:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe280a1120 msgr2=0x7efe280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe280a1120 0x7efe280a35e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efe30009c60 tx=0x7efe30009260 comp rx=0 tx=0).stop 2024-03-30T02:32:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44073790 msgr2=0x7efe44198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44073790 0x7efe44198f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efe34005910 tx=0x7efe34009580 comp rx=0 tx=0).stop 2024-03-30T02:32:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 shutdown_connections 2024-03-30T02:32:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efe280a1120 0x7efe280a35e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe44073790 0x7efe44198f00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe44108ce0 0x7efe441a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 --2- 172.21.15.67:0/1018749847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe44074d60 0x7efe44199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.799+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 >> 172.21.15.67:0/1018749847 conn(0x7efe440fc380 msgr2=0x7efe44104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:32:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.800+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 shutdown_connections 2024-03-30T02:32:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:32:34.800+0000 7efe49800700 1 -- 172.21.15.67:0/1018749847 wait complete. 2024-03-30T02:32:35.185 DEBUG:teuthology.orchestra.run.smithi067:osd.6> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.6.service 2024-03-30T02:32:35.189 INFO:tasks.cephadm:Deploying osd.7 on smithi082 with /dev/vg_nvme/lv_2... 2024-03-30T02:32:35.189 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-30T02:32:35.299 INFO:journalctl@ceph.osd.6.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:32:35.431 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:32:35.828 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:35.828 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:35.828 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:35.828 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:35 smithi082 ceph-mon[79176]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:35.878 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:35.878 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:35.879 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:35.879 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:35 smithi067 ceph-mon[37808]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:36.163 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:32:36 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:32:36.121+0000 7f6fd3735640 -1 osd.6 0 log_to_monitors true 2024-03-30T02:32:36.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:36.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:36.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:36.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:35 smithi170 ceph-mon[78798]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:36.709 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-30T02:32:36.709 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-30T02:32:36.710 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-03-30T02:32:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-03-30T02:32:36.710 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130688 s, 802 MB/s 2024-03-30T02:32:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-03-30T02:32:36.997 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:36 smithi082 ceph-mon[79176]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-30T02:32:37.098 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2024-03-30T02:32:37.316 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:32:37.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:36 smithi067 ceph-mon[37808]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-30T02:32:37.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:36 smithi170 ceph-mon[78798]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-30T02:32:38.120 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:37 smithi082 ceph-mon[79176]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-30T02:32:38.121 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:37 smithi082 ceph-mon[79176]: osdmap e42: 7 total, 6 up, 7 in 2024-03-30T02:32:38.121 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:37 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:38.121 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:37 smithi082 ceph-mon[79176]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:32:38.121 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:37 smithi082 ceph-mon[79176]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:38.121 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:37 smithi082 ceph-mon[79176]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:32:38.121 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:37 smithi082 ceph-mon[79176]: osdmap e43: 7 total, 6 up, 7 in 2024-03-30T02:32:38.121 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:37 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.298+0000 7f965543d700 1 -- 172.21.15.82:0/1008610483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 msgr2=0x7f9650101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.298+0000 7f965543d700 1 --2- 172.21.15.82:0/1008610483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 0x7f9650101370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f96440099c0 tx=0x7f9644009cd0 comp rx=0 tx=0).stop 2024-03-30T02:32:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.298+0000 7f965543d700 1 -- 172.21.15.82:0/1008610483 shutdown_connections 2024-03-30T02:32:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.298+0000 7f965543d700 1 --2- 172.21.15.82:0/1008610483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96500ffd10 0x7f9650100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.298+0000 7f965543d700 1 --2- 172.21.15.82:0/1008610483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96501018b0 0x7f9650108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.298+0000 7f965543d700 1 --2- 172.21.15.82:0/1008610483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 0x7f9650101370 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.298+0000 7f965543d700 1 -- 172.21.15.82:0/1008610483 >> 172.21.15.82:0/1008610483 conn(0x7f96500fb2d0 msgr2=0x7f96500fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:32:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.298+0000 7f965543d700 1 -- 172.21.15.82:0/1008610483 shutdown_connections 2024-03-30T02:32:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.298+0000 7f965543d700 1 -- 172.21.15.82:0/1008610483 wait complete. 2024-03-30T02:32:38.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f965543d700 1 Processor -- start 2024-03-30T02:32:38.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f965543d700 1 -- start start 2024-03-30T02:32:38.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f965543d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96500ffd10 0x7f9650195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f965543d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 0x7f9650196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f965543d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96501018b0 0x7f965019d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f965543d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965019d730 con 0x7f96500ffd10 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f965543d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965019d870 con 0x7f9650100f10 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f965543d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965019da00 con 0x7f96501018b0 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 0x7f9650196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 0x7f9650196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40708/0 (socket says 172.21.15.82:40708) 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964e7fc700 1 -- 172.21.15.82:0/3146691558 learned_addr learned my addr 172.21.15.82:0/3146691558 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964effd700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96500ffd10 0x7f9650195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964f7fe700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96501018b0 0x7f965019d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964e7fc700 1 -- 172.21.15.82:0/3146691558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96501018b0 msgr2=0x7f965019d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964e7fc700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96501018b0 0x7f965019d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964e7fc700 1 -- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96500ffd10 msgr2=0x7f9650195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:38.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964e7fc700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96500ffd10 0x7f9650195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:38.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.299+0000 7f964e7fc700 1 -- 172.21.15.82:0/3146691558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440095f0 con 0x7f9650100f10 2024-03-30T02:32:38.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.300+0000 7f964e7fc700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 0x7f9650196130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9644009fd0 tx=0x7f9644005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:32:38.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.300+0000 7f9637fff700 1 -- 172.21.15.82:0/3146691558 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964401d070 con 0x7f9650100f10 2024-03-30T02:32:38.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.300+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965019dc80 con 0x7f9650100f10 2024-03-30T02:32:38.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.300+0000 7f9637fff700 1 -- 172.21.15.82:0/3146691558 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96440059a0 con 0x7f9650100f10 2024-03-30T02:32:38.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.300+0000 7f9637fff700 1 -- 172.21.15.82:0/3146691558 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96440173f0 con 0x7f9650100f10 2024-03-30T02:32:38.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.300+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965019e170 con 0x7f9650100f10 2024-03-30T02:32:38.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.301+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96501071e0 con 0x7f9650100f10 2024-03-30T02:32:38.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.302+0000 7f9637fff700 1 -- 172.21.15.82:0/3146691558 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f96440226e0 con 0x7f9650100f10 2024-03-30T02:32:38.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.302+0000 7f9637fff700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f96380a1120 0x7f96380a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:38.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.303+0000 7f9637fff700 1 -- 172.21.15.82:0/3146691558 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f96440ccaa0 con 0x7f9650100f10 2024-03-30T02:32:38.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.303+0000 7f964effd700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f96380a1120 0x7f96380a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:38.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.303+0000 7f964effd700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f96380a1120 0x7f96380a35e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9650100d70 tx=0x7f964000b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:32:38.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.307+0000 7f9637fff700 1 -- 172.21.15.82:0/3146691558 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f964405d9f0 con 0x7f9650100f10 2024-03-30T02:32:38.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:37 smithi067 ceph-mon[37808]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-30T02:32:38.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:37 smithi067 ceph-mon[37808]: osdmap e42: 7 total, 6 up, 7 in 2024-03-30T02:32:38.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:38.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:37 smithi067 ceph-mon[37808]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:32:38.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:37 smithi067 ceph-mon[37808]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:38.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:37 smithi067 ceph-mon[37808]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:32:38.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:37 smithi067 ceph-mon[37808]: osdmap e43: 7 total, 6 up, 7 in 2024-03-30T02:32:38.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:37 smithi170 ceph-mon[78798]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-30T02:32:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:37 smithi170 ceph-mon[78798]: osdmap e42: 7 total, 6 up, 7 in 2024-03-30T02:32:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:37 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:37 smithi170 ceph-mon[78798]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:32:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:37 smithi170 ceph-mon[78798]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:37 smithi170 ceph-mon[78798]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:32:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:37 smithi170 ceph-mon[78798]: osdmap e43: 7 total, 6 up, 7 in 2024-03-30T02:32:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:37 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:38.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:38.504+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f96500611f0 con 0x7f96380a1120 2024-03-30T02:32:38.913 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:32:38.511+0000 7f6fc8600700 -1 osd.6 0 waiting for initial osdmap 2024-03-30T02:32:38.913 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:32:38.536+0000 7f6fbda00700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:32:39.227 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:39.227 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi067 2024-03-30T02:32:39.227 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.227 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.227 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:39.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:39.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi067 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:39.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:39.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' 2024-03-30T02:32:39.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:39.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:39.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi067 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:39.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543]' entity='osd.6' 2024-03-30T02:32:39.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:39.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:40.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:39 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:32:40.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:39 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:32:40.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:39 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi067 to 5756M 2024-03-30T02:32:40.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:39 smithi082 ceph-mon[79176]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:40.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:39 smithi082 ceph-mon[79176]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:40.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:39 smithi082 ceph-mon[79176]: osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] boot 2024-03-30T02:32:40.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:39 smithi082 ceph-mon[79176]: osdmap e44: 7 total, 7 up, 7 in 2024-03-30T02:32:40.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:39 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:40.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:39 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:32:40.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:39 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:32:40.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:39 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi067 to 5756M 2024-03-30T02:32:40.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:39 smithi067 ceph-mon[37808]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:40.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:39 smithi067 ceph-mon[37808]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:40.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:39 smithi067 ceph-mon[37808]: osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] boot 2024-03-30T02:32:40.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:39 smithi067 ceph-mon[37808]: osdmap e44: 7 total, 7 up, 7 in 2024-03-30T02:32:40.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:39 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:40.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:39 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:32:40.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:39 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:32:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:39 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi067 to 5756M 2024-03-30T02:32:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:39 smithi170 ceph-mon[78798]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:39 smithi170 ceph-mon[78798]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-30T02:32:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:39 smithi170 ceph-mon[78798]: osd.6 [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] boot 2024-03-30T02:32:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:39 smithi170 ceph-mon[78798]: osdmap e44: 7 total, 7 up, 7 in 2024-03-30T02:32:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:39 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:32:41.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:40 smithi082 ceph-mon[79176]: from='client.? 172.21.15.82:0/2509942326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa6e309c-75b6-4011-a6d1-bb0b1c438ef4"}]: dispatch 2024-03-30T02:32:41.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:40 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa6e309c-75b6-4011-a6d1-bb0b1c438ef4"}]: dispatch 2024-03-30T02:32:41.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:40 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa6e309c-75b6-4011-a6d1-bb0b1c438ef4"}]': finished 2024-03-30T02:32:41.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:40 smithi082 ceph-mon[79176]: osdmap e45: 8 total, 7 up, 8 in 2024-03-30T02:32:41.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:40 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:41.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:40 smithi067 ceph-mon[37808]: from='client.? 172.21.15.82:0/2509942326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa6e309c-75b6-4011-a6d1-bb0b1c438ef4"}]: dispatch 2024-03-30T02:32:41.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:40 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa6e309c-75b6-4011-a6d1-bb0b1c438ef4"}]: dispatch 2024-03-30T02:32:41.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:40 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa6e309c-75b6-4011-a6d1-bb0b1c438ef4"}]': finished 2024-03-30T02:32:41.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:40 smithi067 ceph-mon[37808]: osdmap e45: 8 total, 7 up, 8 in 2024-03-30T02:32:41.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:41.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:40 smithi170 ceph-mon[78798]: from='client.? 172.21.15.82:0/2509942326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa6e309c-75b6-4011-a6d1-bb0b1c438ef4"}]: dispatch 2024-03-30T02:32:41.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:40 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa6e309c-75b6-4011-a6d1-bb0b1c438ef4"}]: dispatch 2024-03-30T02:32:41.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:40 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa6e309c-75b6-4011-a6d1-bb0b1c438ef4"}]': finished 2024-03-30T02:32:41.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:40 smithi170 ceph-mon[78798]: osdmap e45: 8 total, 7 up, 8 in 2024-03-30T02:32:41.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:40 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:42.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:41 smithi082 ceph-mon[79176]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:42.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:41 smithi082 ceph-mon[79176]: from='client.? 172.21.15.82:0/1230645492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:42.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:41 smithi067 ceph-mon[37808]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:42.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:41 smithi067 ceph-mon[37808]: from='client.? 172.21.15.82:0/1230645492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:42.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:41 smithi170 ceph-mon[78798]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:42.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:41 smithi170 ceph-mon[78798]: from='client.? 172.21.15.82:0/1230645492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:44.321 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:44 smithi082 ceph-mon[79176]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:44 smithi067 ceph-mon[37808]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:44 smithi170 ceph-mon[78798]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:46.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:46 smithi082 ceph-mon[79176]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:46.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:46 smithi067 ceph-mon[37808]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:46.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:46 smithi170 ceph-mon[78798]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:47.242 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-30T02:32:47.242 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:47.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-30T02:32:47.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:47.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-30T02:32:47.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:48.314 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:48 smithi082 ceph-mon[79176]: Deploying daemon osd.7 on smithi082 2024-03-30T02:32:48.314 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:48 smithi082 ceph-mon[79176]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:48 smithi067 ceph-mon[37808]: Deploying daemon osd.7 on smithi082 2024-03-30T02:32:48.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:48 smithi067 ceph-mon[37808]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:48.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:48 smithi170 ceph-mon[78798]: Deploying daemon osd.7 on smithi082 2024-03-30T02:32:48.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:48 smithi170 ceph-mon[78798]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:50.303 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:50 smithi082 ceph-mon[79176]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:50.303 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:50 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:50.303 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:50 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:50.303 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:50 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:50.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:50 smithi067 ceph-mon[37808]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:50.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:50 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:50.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:50 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:50.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:50 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:50.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:50 smithi170 ceph-mon[78798]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:50.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:50 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:50.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:50 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:50.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:50 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.110+0000 7f9637fff700 1 -- 172.21.15.82:0/3146691558 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f96500611f0 con 0x7f96380a1120 2024-03-30T02:32:52.112 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 7 on host 'smithi082' 2024-03-30T02:32:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f96380a1120 msgr2=0x7f96380a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f96380a1120 0x7f96380a35e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9650100d70 tx=0x7f964000b410 comp rx=0 tx=0).stop 2024-03-30T02:32:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 msgr2=0x7f9650196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:52.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 0x7f9650196130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9644009fd0 tx=0x7f9644005f10 comp rx=0 tx=0).stop 2024-03-30T02:32:52.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 shutdown_connections 2024-03-30T02:32:52.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f96380a1120 0x7f96380a35e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:52.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96500ffd10 0x7f9650195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:52.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96501018b0 0x7f965019d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:52.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 --2- 172.21.15.82:0/3146691558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9650100f10 0x7f9650196130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:52.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 >> 172.21.15.82:0/3146691558 conn(0x7f96500fb2d0 msgr2=0x7f9650104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:32:52.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 shutdown_connections 2024-03-30T02:32:52.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:32:52.112+0000 7f965543d700 1 -- 172.21.15.82:0/3146691558 wait complete. 2024-03-30T02:32:52.382 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:52 smithi082 ceph-mon[79176]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:52.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:52.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:52.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:52 smithi067 ceph-mon[37808]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:52.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:52.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:52.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:52 smithi170 ceph-mon[78798]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:52.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:52.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:52.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:52.499 DEBUG:teuthology.orchestra.run.smithi082:osd.7> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.7.service 2024-03-30T02:32:52.502 INFO:tasks.cephadm:Deploying osd.8 on smithi170 with /dev/vg_nvme/lv_2... 2024-03-30T02:32:52.502 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-30T02:32:52.531 INFO:journalctl@ceph.osd.7.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:32:52.748 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:32:53.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:53 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:53.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:53 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:53.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:53 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:53.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:53 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:53.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:53 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:53.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:53 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:53.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:53 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:53.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:53 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:53.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:53 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:32:54.091 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:32:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:32:53.769+0000 7fa6b7512640 -1 osd.7 0 log_to_monitors true 2024-03-30T02:32:54.127 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-30T02:32:54.127 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-30T02:32:54.127 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-03-30T02:32:54.127 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-03-30T02:32:54.127 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132591 s, 791 MB/s 2024-03-30T02:32:54.127 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-03-30T02:32:54.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:54 smithi170 ceph-mon[78798]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:54.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:54 smithi170 ceph-mon[78798]: from='osd.7 [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T02:32:54.413 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:54 smithi170 ceph-mon[78798]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T02:32:54.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:54 smithi067 ceph-mon[37808]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:54.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:54 smithi067 ceph-mon[37808]: from='osd.7 [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T02:32:54.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:54 smithi067 ceph-mon[37808]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T02:32:54.443 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:54 smithi082 ceph-mon[79176]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:54.443 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:54 smithi082 ceph-mon[79176]: from='osd.7 [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T02:32:54.443 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:54 smithi082 ceph-mon[79176]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-30T02:32:54.522 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2024-03-30T02:32:54.743 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:32:55.184 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: osdmap e46: 8 total, 7 up, 8 in 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='osd.7 [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi082 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi082 to 5704M 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.185 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:55.186 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:55.186 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.186 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:55 smithi170 ceph-mon[78798]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:55.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: osdmap e46: 8 total, 7 up, 8 in 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='osd.7 [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi082 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi082 to 5704M 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:55.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:55.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:55 smithi067 ceph-mon[37808]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:55.500 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:32:55.106+0000 7fa6acb76700 -1 osd.7 0 waiting for initial osdmap 2024-03-30T02:32:55.500 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:32:55.117+0000 7fa6a896b700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:32:55.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-30T02:32:55.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: osdmap e46: 8 total, 7 up, 8 in 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='osd.7 [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi082 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi082 to 5704M 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:32:55.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:55 smithi082 ceph-mon[79176]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-30T02:32:55.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.753+0000 7f4663ffa700 1 -- 172.21.15.170:0/1729055416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 msgr2=0x7f465c100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:55.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.753+0000 7f4663ffa700 1 --2- 172.21.15.170:0/1729055416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 0x7f465c100110 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4650009990 tx=0x7f4650009ca0 comp rx=0 tx=0).stop 2024-03-30T02:32:55.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.753+0000 7f4663ffa700 1 -- 172.21.15.170:0/1729055416 shutdown_connections 2024-03-30T02:32:55.756 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.753+0000 7f4663ffa700 1 --2- 172.21.15.170:0/1729055416 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c101890 0x7f465c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.753+0000 7f4663ffa700 1 --2- 172.21.15.170:0/1729055416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 0x7f465c100110 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.753+0000 7f4663ffa700 1 --2- 172.21.15.170:0/1729055416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f465c100ef0 0x7f465c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.753+0000 7f4663ffa700 1 -- 172.21.15.170:0/1729055416 >> 172.21.15.170:0/1729055416 conn(0x7f465c0fb2b0 msgr2=0x7f465c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.753+0000 7f4663ffa700 1 -- 172.21.15.170:0/1729055416 shutdown_connections 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.753+0000 7f4663ffa700 1 -- 172.21.15.170:0/1729055416 wait complete. 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4663ffa700 1 Processor -- start 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4663ffa700 1 -- start start 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4663ffa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 0x7f465c069af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4663ffa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c100ef0 0x7f465c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4663ffa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f465c101890 0x7f465c19e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4663ffa700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c19e8e0 con 0x7f465c100ef0 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4663ffa700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c19ea50 con 0x7f465c101890 2024-03-30T02:32:55.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4663ffa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c19ec10 con 0x7f465c0ffcf0 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4661d96700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 0x7f465c069af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4661d96700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 0x7f465c069af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37262/0 (socket says 172.21.15.170:37262) 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.754+0000 7f4661d96700 1 -- 172.21.15.170:0/835145505 learned_addr learned my addr 172.21.15.170:0/835145505 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f4661d96700 1 -- 172.21.15.170:0/835145505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f465c101890 msgr2=0x7f465c19e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f4661d96700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f465c101890 0x7f465c19e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f4661d96700 1 -- 172.21.15.170:0/835145505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c100ef0 msgr2=0x7f465c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f4661d96700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c100ef0 0x7f465c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f4661d96700 1 -- 172.21.15.170:0/835145505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500095f0 con 0x7f465c0ffcf0 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f4661d96700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 0x7f465c069af0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f46500038c0 tx=0x7f4650003cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f464effd700 1 -- 172.21.15.170:0/835145505 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465001d070 con 0x7f465c0ffcf0 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f464effd700 1 -- 172.21.15.170:0/835145505 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f465000f460 con 0x7f465c0ffcf0 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c19ee90 con 0x7f465c0ffcf0 2024-03-30T02:32:55.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f464effd700 1 -- 172.21.15.170:0/835145505 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650017650 con 0x7f465c0ffcf0 2024-03-30T02:32:55.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.755+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c19f2c0 con 0x7f465c0ffcf0 2024-03-30T02:32:55.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.756+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c04eaf0 con 0x7f465c0ffcf0 2024-03-30T02:32:55.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.757+0000 7f464effd700 1 -- 172.21.15.170:0/835145505 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f465000f5d0 con 0x7f465c0ffcf0 2024-03-30T02:32:55.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.758+0000 7f464effd700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46480a1120 0x7f46480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:32:55.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.758+0000 7f464effd700 1 -- 172.21.15.170:0/835145505 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f46500cc930 con 0x7f465c0ffcf0 2024-03-30T02:32:55.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.758+0000 7f4661595700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46480a1120 0x7f46480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:32:55.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.759+0000 7f4661595700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46480a1120 0x7f46480a35e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f46580097b0 tx=0x7f4658006d20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:32:55.762 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.761+0000 7f464effd700 1 -- 172.21.15.170:0/835145505 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4650096e60 con 0x7f465c0ffcf0 2024-03-30T02:32:55.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:32:55.967+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f465c04ea80 con 0x7f46480a1120 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: osdmap e47: 8 total, 7 up, 8 in 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: osd.7 [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] boot 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: osdmap e48: 8 total, 8 up, 8 in 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:56.387 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:56.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:32:56.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: osdmap e47: 8 total, 7 up, 8 in 2024-03-30T02:32:56.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: osd.7 [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] boot 2024-03-30T02:32:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: osdmap e48: 8 total, 8 up, 8 in 2024-03-30T02:32:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:56.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:56.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:32:56.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: osdmap e47: 8 total, 7 up, 8 in 2024-03-30T02:32:56.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:56.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:56.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: osd.7 [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] boot 2024-03-30T02:32:56.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: osdmap e48: 8 total, 8 up, 8 in 2024-03-30T02:32:56.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:32:56.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:32:56.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:32:56.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:32:56.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:32:57.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:57 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:32:57.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:57 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:32:57.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:57 smithi067 ceph-mon[37808]: osdmap e49: 8 total, 8 up, 8 in 2024-03-30T02:32:57.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:57 smithi067 ceph-mon[37808]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:32:57.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:57 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:32:57.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:57 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:32:57.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:57 smithi170 ceph-mon[78798]: osdmap e49: 8 total, 8 up, 8 in 2024-03-30T02:32:57.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:57 smithi170 ceph-mon[78798]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:32:57.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:57 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:32:57.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:57 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:32:57.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:57 smithi082 ceph-mon[79176]: osdmap e49: 8 total, 8 up, 8 in 2024-03-30T02:32:57.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:57 smithi082 ceph-mon[79176]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:32:58.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:58 smithi067 ceph-mon[37808]: from='client.? 172.21.15.170:0/3582412989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90ecfccd-eb50-4be0-927c-ae2c8a415c83"}]: dispatch 2024-03-30T02:32:58.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:58 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90ecfccd-eb50-4be0-927c-ae2c8a415c83"}]: dispatch 2024-03-30T02:32:58.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:58 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90ecfccd-eb50-4be0-927c-ae2c8a415c83"}]': finished 2024-03-30T02:32:58.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:58 smithi067 ceph-mon[37808]: osdmap e50: 9 total, 8 up, 9 in 2024-03-30T02:32:58.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:58 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:32:58.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:58 smithi170 ceph-mon[78798]: from='client.? 172.21.15.170:0/3582412989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90ecfccd-eb50-4be0-927c-ae2c8a415c83"}]: dispatch 2024-03-30T02:32:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:58 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90ecfccd-eb50-4be0-927c-ae2c8a415c83"}]: dispatch 2024-03-30T02:32:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:58 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90ecfccd-eb50-4be0-927c-ae2c8a415c83"}]': finished 2024-03-30T02:32:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:58 smithi170 ceph-mon[78798]: osdmap e50: 9 total, 8 up, 9 in 2024-03-30T02:32:58.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:58 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:32:58.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:58 smithi082 ceph-mon[79176]: from='client.? 172.21.15.170:0/3582412989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90ecfccd-eb50-4be0-927c-ae2c8a415c83"}]: dispatch 2024-03-30T02:32:58.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:58 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90ecfccd-eb50-4be0-927c-ae2c8a415c83"}]: dispatch 2024-03-30T02:32:58.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:58 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90ecfccd-eb50-4be0-927c-ae2c8a415c83"}]': finished 2024-03-30T02:32:58.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:58 smithi082 ceph-mon[79176]: osdmap e50: 9 total, 8 up, 9 in 2024-03-30T02:32:58.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:58 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:32:59.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:59 smithi067 ceph-mon[37808]: from='client.? 172.21.15.170:0/2753135325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:59.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:59 smithi067 ceph-mon[37808]: osdmap e51: 9 total, 8 up, 9 in 2024-03-30T02:32:59.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:59 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:32:59.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:32:59 smithi067 ceph-mon[37808]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:32:59.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:59 smithi170 ceph-mon[78798]: from='client.? 172.21.15.170:0/2753135325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:59 smithi170 ceph-mon[78798]: osdmap e51: 9 total, 8 up, 9 in 2024-03-30T02:32:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:59 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:32:59.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:32:59 smithi170 ceph-mon[78798]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:32:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:59 smithi082 ceph-mon[79176]: from='client.? 172.21.15.170:0/2753135325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:32:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:59 smithi082 ceph-mon[79176]: osdmap e51: 9 total, 8 up, 9 in 2024-03-30T02:32:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:59 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:32:59.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:32:59 smithi082 ceph-mon[79176]: pgmap v147: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:01.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:00 smithi067 ceph-mon[37808]: osdmap e52: 9 total, 8 up, 9 in 2024-03-30T02:33:01.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:00 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:01.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:00 smithi170 ceph-mon[78798]: osdmap e52: 9 total, 8 up, 9 in 2024-03-30T02:33:01.191 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:00 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:01.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:00 smithi082 ceph-mon[79176]: osdmap e52: 9 total, 8 up, 9 in 2024-03-30T02:33:01.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:00 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:02.231 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:01 smithi170 ceph-mon[78798]: pgmap v149: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:02.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:01 smithi082 ceph-mon[79176]: pgmap v149: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:02.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:01 smithi067 ceph-mon[37808]: pgmap v149: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:04.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:04 smithi170 ceph-mon[78798]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:04.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-30T02:33:04.374 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:04 smithi067 ceph-mon[37808]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-30T02:33:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:04 smithi082 ceph-mon[79176]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:04 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-30T02:33:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:04 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:05.308 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:05 smithi170 ceph-mon[78798]: Deploying daemon osd.8 on smithi170 2024-03-30T02:33:05.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:05 smithi067 ceph-mon[37808]: Deploying daemon osd.8 on smithi170 2024-03-30T02:33:05.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:05 smithi082 ceph-mon[79176]: Deploying daemon osd.8 on smithi170 2024-03-30T02:33:06.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:06 smithi067 ceph-mon[37808]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:06.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:06 smithi170 ceph-mon[78798]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:06 smithi082 ceph-mon[79176]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:08.130 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:08 smithi170 ceph-mon[78798]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:08.130 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:08 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:08.130 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:08 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:08.130 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:08 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:08.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:08 smithi067 ceph-mon[37808]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:08.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:08 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:08.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:08 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:08.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:08 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:08 smithi082 ceph-mon[79176]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:08 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:08 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:08 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:09.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.725+0000 7f464effd700 1 -- 172.21.15.170:0/835145505 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f465c04ea80 con 0x7f46480a1120 2024-03-30T02:33:09.726 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 8 on host 'smithi170' 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46480a1120 msgr2=0x7f46480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46480a1120 0x7f46480a35e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f46580097b0 tx=0x7f4658006d20 comp rx=0 tx=0).stop 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 msgr2=0x7f465c069af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 0x7f465c069af0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f46500038c0 tx=0x7f4650003cb0 comp rx=0 tx=0).stop 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 shutdown_connections 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f46480a1120 0x7f46480a35e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f465c101890 0x7f465c19e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f465c0ffcf0 0x7f465c069af0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 --2- 172.21.15.170:0/835145505 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f465c100ef0 0x7f465c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 >> 172.21.15.170:0/835145505 conn(0x7f465c0fb2b0 msgr2=0x7f465c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:33:09.728 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 shutdown_connections 2024-03-30T02:33:09.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:09.727+0000 7f4663ffa700 1 -- 172.21.15.170:0/835145505 wait complete. 2024-03-30T02:33:09.873 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:09.873 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:09.874 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:09.874 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:09.874 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:09.874 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:09 smithi170 ceph-mon[78798]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:09.874 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:09.874 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:09.874 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:10.162 DEBUG:teuthology.orchestra.run.smithi170:osd.8> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.8.service 2024-03-30T02:33:10.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:10.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:10.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:09 smithi067 ceph-mon[37808]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:10.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:10.165 INFO:tasks.cephadm:Deploying osd.9 on smithi067 with /dev/vg_nvme/lv_1... 2024-03-30T02:33:10.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-30T02:33:10.195 INFO:journalctl@ceph.osd.8.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:33:10.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:10.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:10.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:09 smithi082 ceph-mon[79176]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:10.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:10.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:10.536 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:33:11.846 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:33:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:33:11.450+0000 7f4c02158640 -1 osd.8 0 log_to_monitors true 2024-03-30T02:33:12.168 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:11 smithi170 ceph-mon[78798]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:12.168 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:11 smithi170 ceph-mon[78798]: from='osd.8 [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T02:33:12.168 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:11 smithi170 ceph-mon[78798]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T02:33:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:11 smithi082 ceph-mon[79176]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:11 smithi082 ceph-mon[79176]: from='osd.8 [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T02:33:12.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:11 smithi082 ceph-mon[79176]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T02:33:12.373 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-30T02:33:12.373 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-30T02:33:12.373 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-03-30T02:33:12.373 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-03-30T02:33:12.373 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0307998 s, 340 MB/s 2024-03-30T02:33:12.373 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-03-30T02:33:12.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:11 smithi067 ceph-mon[37808]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:12.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:11 smithi067 ceph-mon[37808]: from='osd.8 [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T02:33:12.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:11 smithi067 ceph-mon[37808]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-30T02:33:12.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi067:vg_nvme/lv_1 2024-03-30T02:33:13.144 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:33:13.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-30T02:33:13.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: osdmap e53: 9 total, 8 up, 9 in 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='osd.8 [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:13.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:13.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:12 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-30T02:33:13.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: osdmap e53: 9 total, 8 up, 9 in 2024-03-30T02:33:13.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:13.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='osd.8 [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:33:13.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:33:13.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:13.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:13.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:13.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:13.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:13.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:12 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.440 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:33:12.968+0000 7f4bf77bc700 -1 osd.8 0 waiting for initial osdmap 2024-03-30T02:33:13.440 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:33:12.981+0000 7f4bf35b1700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:33:13.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-30T02:33:13.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: osdmap e53: 9 total, 8 up, 9 in 2024-03-30T02:33:13.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:13.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='osd.8 [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:33:13.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:33:13.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:13.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:13.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:13.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:13.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:13.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:13.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:12 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:14.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:13 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi170 2024-03-30T02:33:14.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:13 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi170 to 5704M 2024-03-30T02:33:14.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:13 smithi082 ceph-mon[79176]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:33:14.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:13 smithi082 ceph-mon[79176]: osdmap e54: 9 total, 8 up, 9 in 2024-03-30T02:33:14.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:14.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:13 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:14.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:13 smithi082 ceph-mon[79176]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:14.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.296+0000 7fed86400700 1 -- 172.21.15.67:0/1097819790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed80100e00 msgr2=0x7fed80101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:14.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.296+0000 7fed86400700 1 --2- 172.21.15.67:0/1097819790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed80100e00 0x7fed80101220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fed70009990 tx=0x7fed70009ca0 comp rx=0 tx=0).stop 2024-03-30T02:33:14.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.296+0000 7fed86400700 1 -- 172.21.15.67:0/1097819790 shutdown_connections 2024-03-30T02:33:14.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.296+0000 7fed86400700 1 --2- 172.21.15.67:0/1097819790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed80100e00 0x7fed80101220 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:14.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.296+0000 7fed86400700 1 --2- 172.21.15.67:0/1097819790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed80102920 0x7fed80109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:14.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.296+0000 7fed86400700 1 --2- 172.21.15.67:0/1097819790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed80101f60 0x7fed801023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:14.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.296+0000 7fed86400700 1 -- 172.21.15.67:0/1097819790 >> 172.21.15.67:0/1097819790 conn(0x7fed800fc3c0 msgr2=0x7fed800fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:33:14.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.296+0000 7fed86400700 1 -- 172.21.15.67:0/1097819790 shutdown_connections 2024-03-30T02:33:14.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.296+0000 7fed86400700 1 -- 172.21.15.67:0/1097819790 wait complete. 2024-03-30T02:33:14.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.297+0000 7fed86400700 1 Processor -- start 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.297+0000 7fed86400700 1 -- start start 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.297+0000 7fed86400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed80100e00 0x7fed80199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.297+0000 7fed86400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed80101f60 0x7fed80199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.297+0000 7fed86400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed80102920 0x7fed801a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed86400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed801a0af0 con 0x7fed80102920 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed86400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed801a0c30 con 0x7fed80101f60 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed86400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed801a0dc0 con 0x7fed80100e00 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed80102920 0x7fed801a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed80101f60 0x7fed80199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:14.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed80102920 0x7fed801a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57286/0 (socket says 172.21.15.67:57286) 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed80101f60 0x7fed80199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35992/0 (socket says 172.21.15.67:35992) 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7fe00700 1 -- 172.21.15.67:0/3434102355 learned_addr learned my addr 172.21.15.67:0/3434102355 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7f400700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed80100e00 0x7fed80199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7ea00700 1 -- 172.21.15.67:0/3434102355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed80100e00 msgr2=0x7fed80199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7ea00700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed80100e00 0x7fed80199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7ea00700 1 -- 172.21.15.67:0/3434102355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed80102920 msgr2=0x7fed801a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7ea00700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed80102920 0x7fed801a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.298+0000 7fed7ea00700 1 -- 172.21.15.67:0/3434102355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed78009c40 con 0x7fed80101f60 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.299+0000 7fed7ea00700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed80101f60 0x7fed80199540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00e9f0 tx=0x7fed6c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.299+0000 7fed77e00700 1 -- 172.21.15.67:0/3434102355 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00c9c0 con 0x7fed80101f60 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.299+0000 7fed77e00700 1 -- 172.21.15.67:0/3434102355 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6c010430 con 0x7fed80101f60 2024-03-30T02:33:14.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.299+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed700095f0 con 0x7fed80101f60 2024-03-30T02:33:14.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.299+0000 7fed77e00700 1 -- 172.21.15.67:0/3434102355 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c018680 con 0x7fed80101f60 2024-03-30T02:33:14.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.299+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed801a1460 con 0x7fed80101f60 2024-03-30T02:33:14.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.300+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8004ea80 con 0x7fed80101f60 2024-03-30T02:33:14.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.302+0000 7fed77e00700 1 -- 172.21.15.67:0/3434102355 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fed6c010860 con 0x7fed80101f60 2024-03-30T02:33:14.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.303+0000 7fed77e00700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fed640a11a0 0x7fed640a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:14.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.303+0000 7fed77e00700 1 -- 172.21.15.67:0/3434102355 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fed6c014070 con 0x7fed80101f60 2024-03-30T02:33:14.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.303+0000 7fed7f400700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fed640a11a0 0x7fed640a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:14.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.304+0000 7fed7f400700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fed640a11a0 0x7fed640a3660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fed7000b2e0 tx=0x7fed70005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:33:14.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.308+0000 7fed77e00700 1 -- 172.21.15.67:0/3434102355 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fed6c0529f0 con 0x7fed80101f60 2024-03-30T02:33:14.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:13 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi170 2024-03-30T02:33:14.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:13 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi170 to 5704M 2024-03-30T02:33:14.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:13 smithi067 ceph-mon[37808]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:33:14.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:13 smithi067 ceph-mon[37808]: osdmap e54: 9 total, 8 up, 9 in 2024-03-30T02:33:14.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:14.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:13 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:14.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:13 smithi067 ceph-mon[37808]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:14.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:13 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi170 2024-03-30T02:33:14.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:13 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi170 to 5704M 2024-03-30T02:33:14.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:13 smithi170 ceph-mon[78798]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:33:14.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:13 smithi170 ceph-mon[78798]: osdmap e54: 9 total, 8 up, 9 in 2024-03-30T02:33:14.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:14.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:13 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:14.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:13 smithi170 ceph-mon[78798]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-30T02:33:14.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:14.533+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fed80108eb0 con 0x7fed640a11a0 2024-03-30T02:33:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:14 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:33:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:14 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:33:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:14 smithi082 ceph-mon[79176]: osd.8 [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] boot 2024-03-30T02:33:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:14 smithi082 ceph-mon[79176]: osdmap e55: 9 total, 9 up, 9 in 2024-03-30T02:33:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:33:15.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:33:15.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:14 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:15.251 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:14 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:33:15.251 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:14 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:33:15.251 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:14 smithi067 ceph-mon[37808]: osd.8 [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] boot 2024-03-30T02:33:15.251 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:14 smithi067 ceph-mon[37808]: osdmap e55: 9 total, 9 up, 9 in 2024-03-30T02:33:15.251 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:15.251 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:33:15.251 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:33:15.251 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:14 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:14 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:33:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:14 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:33:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:14 smithi170 ceph-mon[78798]: osd.8 [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] boot 2024-03-30T02:33:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:14 smithi170 ceph-mon[78798]: osdmap e55: 9 total, 9 up, 9 in 2024-03-30T02:33:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:33:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:33:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:33:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:14 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:16.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:15 smithi067 ceph-mon[37808]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:33:16.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:15 smithi067 ceph-mon[37808]: osdmap e56: 9 total, 9 up, 9 in 2024-03-30T02:33:16.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:15 smithi067 ceph-mon[37808]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-03-30T02:33:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:15 smithi082 ceph-mon[79176]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:33:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:15 smithi082 ceph-mon[79176]: osdmap e56: 9 total, 9 up, 9 in 2024-03-30T02:33:16.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:15 smithi082 ceph-mon[79176]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-03-30T02:33:16.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:15 smithi170 ceph-mon[78798]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:33:16.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:15 smithi170 ceph-mon[78798]: osdmap e56: 9 total, 9 up, 9 in 2024-03-30T02:33:16.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:15 smithi170 ceph-mon[78798]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-03-30T02:33:17.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:16 smithi082 ceph-mon[79176]: osdmap e57: 9 total, 9 up, 9 in 2024-03-30T02:33:17.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:16 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/200353246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ec7fee7-63f1-4193-aa2c-769a1f5e54d5"}]: dispatch 2024-03-30T02:33:17.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:16 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/200353246' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ec7fee7-63f1-4193-aa2c-769a1f5e54d5"}]': finished 2024-03-30T02:33:17.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:16 smithi082 ceph-mon[79176]: osdmap e58: 10 total, 9 up, 10 in 2024-03-30T02:33:17.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:16 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:16 smithi067 ceph-mon[37808]: osdmap e57: 9 total, 9 up, 9 in 2024-03-30T02:33:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:16 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/200353246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ec7fee7-63f1-4193-aa2c-769a1f5e54d5"}]: dispatch 2024-03-30T02:33:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:16 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/200353246' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ec7fee7-63f1-4193-aa2c-769a1f5e54d5"}]': finished 2024-03-30T02:33:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:16 smithi067 ceph-mon[37808]: osdmap e58: 10 total, 9 up, 10 in 2024-03-30T02:33:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:16 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:17.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:16 smithi170 ceph-mon[78798]: osdmap e57: 9 total, 9 up, 9 in 2024-03-30T02:33:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:16 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/200353246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ec7fee7-63f1-4193-aa2c-769a1f5e54d5"}]: dispatch 2024-03-30T02:33:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:16 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/200353246' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ec7fee7-63f1-4193-aa2c-769a1f5e54d5"}]': finished 2024-03-30T02:33:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:16 smithi170 ceph-mon[78798]: osdmap e58: 10 total, 9 up, 10 in 2024-03-30T02:33:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:16 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:17 smithi082 ceph-mon[79176]: pgmap v163: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-30T02:33:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:17 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/446426314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:33:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:17 smithi082 ceph-mon[79176]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-30T02:33:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:17 smithi082 ceph-mon[79176]: osdmap e59: 10 total, 9 up, 10 in 2024-03-30T02:33:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:17 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:18.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:17 smithi067 ceph-mon[37808]: pgmap v163: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-30T02:33:18.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:17 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/446426314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:33:18.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:17 smithi067 ceph-mon[37808]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-30T02:33:18.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:17 smithi067 ceph-mon[37808]: osdmap e59: 10 total, 9 up, 10 in 2024-03-30T02:33:18.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:17 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:18.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:17 smithi170 ceph-mon[78798]: pgmap v163: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-30T02:33:18.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:17 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/446426314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:33:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:17 smithi170 ceph-mon[78798]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-30T02:33:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:17 smithi170 ceph-mon[78798]: osdmap e59: 10 total, 9 up, 10 in 2024-03-30T02:33:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:17 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:20.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:19 smithi082 ceph-mon[79176]: pgmap v165: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-30T02:33:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:19 smithi067 ceph-mon[37808]: pgmap v165: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-30T02:33:20.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:19 smithi170 ceph-mon[78798]: pgmap v165: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-30T02:33:22.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:21 smithi067 ceph-mon[37808]: pgmap v166: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-30T02:33:22.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:21 smithi082 ceph-mon[79176]: pgmap v166: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-30T02:33:22.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:21 smithi170 ceph-mon[78798]: pgmap v166: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-30T02:33:23.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:23 smithi067 ceph-mon[37808]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-30T02:33:23.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:23 smithi067 ceph-mon[37808]: Cluster is now healthy 2024-03-30T02:33:23.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:23 smithi170 ceph-mon[78798]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-30T02:33:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:23 smithi170 ceph-mon[78798]: Cluster is now healthy 2024-03-30T02:33:23.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:23 smithi082 ceph-mon[79176]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-30T02:33:23.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:23 smithi082 ceph-mon[79176]: Cluster is now healthy 2024-03-30T02:33:24.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:24 smithi067 ceph-mon[37808]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-30T02:33:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-30T02:33:24.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:24.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:24 smithi170 ceph-mon[78798]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-30T02:33:24.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-30T02:33:24.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:24.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:24 smithi082 ceph-mon[79176]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-30T02:33:24.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-30T02:33:24.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:25.405 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:25 smithi067 ceph-mon[37808]: Deploying daemon osd.9 on smithi067 2024-03-30T02:33:25.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:25 smithi170 ceph-mon[78798]: Deploying daemon osd.9 on smithi067 2024-03-30T02:33:25.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:25 smithi082 ceph-mon[79176]: Deploying daemon osd.9 on smithi067 2024-03-30T02:33:26.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:26 smithi067 ceph-mon[37808]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T02:33:26.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:26 smithi170 ceph-mon[78798]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T02:33:26.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:26 smithi082 ceph-mon[79176]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T02:33:28.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:28 smithi067 ceph-mon[37808]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-30T02:33:28.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:28 smithi170 ceph-mon[78798]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-30T02:33:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:28 smithi082 ceph-mon[79176]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-30T02:33:29.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:29.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:29.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:29.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:29 smithi067 ceph-mon[37808]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-30T02:33:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:29 smithi170 ceph-mon[78798]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-30T02:33:30.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:30.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:30.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:30.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:29 smithi082 ceph-mon[79176]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-30T02:33:31.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.488+0000 7fed77e00700 1 -- 172.21.15.67:0/3434102355 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fed80108eb0 con 0x7fed640a11a0 2024-03-30T02:33:31.489 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 9 on host 'smithi067' 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fed640a11a0 msgr2=0x7fed640a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fed640a11a0 0x7fed640a3660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fed7000b2e0 tx=0x7fed70005fb0 comp rx=0 tx=0).stop 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed80101f60 msgr2=0x7fed80199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed80101f60 0x7fed80199540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00e9f0 tx=0x7fed6c00ed00 comp rx=0 tx=0).stop 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 shutdown_connections 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fed640a11a0 0x7fed640a3660 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed80102920 0x7fed801a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed80100e00 0x7fed80199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 --2- 172.21.15.67:0/3434102355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed80101f60 0x7fed80199540 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 >> 172.21.15.67:0/3434102355 conn(0x7fed800fc3c0 msgr2=0x7fed80107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:33:31.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 shutdown_connections 2024-03-30T02:33:31.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:33:31.490+0000 7fed86400700 1 -- 172.21.15.67:0/3434102355 wait complete. 2024-03-30T02:33:31.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:31.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:31.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:31.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:31.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:31.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:31 smithi170 ceph-mon[78798]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T02:33:31.725 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:31.725 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:31.725 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:31.725 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:31.725 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:31.725 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:31 smithi067 ceph-mon[37808]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T02:33:31.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:31.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:31.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:31.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:31.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:31.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:31 smithi082 ceph-mon[79176]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T02:33:31.924 DEBUG:teuthology.orchestra.run.smithi067:osd.9> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.9.service 2024-03-30T02:33:31.928 INFO:tasks.cephadm:Deploying osd.10 on smithi082 with /dev/vg_nvme/lv_1... 2024-03-30T02:33:31.928 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-30T02:33:32.010 INFO:journalctl@ceph.osd.9.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:33:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:33:32.582 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:32 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:32.582 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:32 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:32.582 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:32 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:32.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:32 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:32.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:32 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:32.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:32 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:32 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:32 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:32.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:32 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:32.955 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:33:32 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:33:32.681+0000 7f79638c7640 -1 osd.9 0 log_to_monitors true 2024-03-30T02:33:33.450 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-30T02:33:33.450 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-30T02:33:33.450 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-03-30T02:33:33.450 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-03-30T02:33:33.450 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0155121 s, 676 MB/s 2024-03-30T02:33:33.451 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-03-30T02:33:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:33 smithi082 ceph-mon[79176]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-30T02:33:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:33 smithi082 ceph-mon[79176]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T02:33:33.827 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2024-03-30T02:33:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:33 smithi067 ceph-mon[37808]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-30T02:33:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:33 smithi067 ceph-mon[37808]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T02:33:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:33 smithi170 ceph-mon[78798]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-30T02:33:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:33 smithi170 ceph-mon[78798]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-30T02:33:34.045 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:33:34.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:34 smithi067 ceph-mon[37808]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-30T02:33:34.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:34 smithi067 ceph-mon[37808]: osdmap e60: 10 total, 9 up, 10 in 2024-03-30T02:33:34.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:34.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:34 smithi067 ceph-mon[37808]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:33:34.664 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:33:34 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:33:34.480+0000 7f7957e00700 -1 osd.9 0 waiting for initial osdmap 2024-03-30T02:33:34.664 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:33:34 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:33:34.496+0000 7f7952c00700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:33:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:34 smithi082 ceph-mon[79176]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-30T02:33:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:34 smithi082 ceph-mon[79176]: osdmap e60: 10 total, 9 up, 10 in 2024-03-30T02:33:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:34 smithi082 ceph-mon[79176]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:33:34.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:34 smithi170 ceph-mon[78798]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-30T02:33:34.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:34 smithi170 ceph-mon[78798]: osdmap e60: 10 total, 9 up, 10 in 2024-03-30T02:33:34.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:34.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:34 smithi170 ceph-mon[78798]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-03-30T02:33:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 -- 172.21.15.82:0/2240415572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b0101890 msgr2=0x7ff9b0108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/2240415572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b0101890 0x7ff9b0108140 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b0d0 tx=0x7ff9a800b3e0 comp rx=0 tx=0).stop 2024-03-30T02:33:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 -- 172.21.15.82:0/2240415572 shutdown_connections 2024-03-30T02:33:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/2240415572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b0101890 0x7ff9b0108140 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/2240415572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b00ffcf0 0x7ff9b0100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/2240415572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0100ef0 0x7ff9b0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 -- 172.21.15.82:0/2240415572 >> 172.21.15.82:0/2240415572 conn(0x7ff9b00fb2b0 msgr2=0x7ff9b00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 -- 172.21.15.82:0/2240415572 shutdown_connections 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 -- 172.21.15.82:0/2240415572 wait complete. 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 Processor -- start 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.025+0000 7ff9b54ca700 1 -- start start 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9b54ca700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b00ffcf0 0x7ff9b0196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9b54ca700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0100ef0 0x7ff9b0197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9b54ca700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b0101890 0x7ff9b019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9b54ca700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b019ea20 con 0x7ff9b0100ef0 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9b54ca700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b019eb60 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9b54ca700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b019ecf0 con 0x7ff9b0101890 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9aeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b00ffcf0 0x7ff9b0196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:35.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9aeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b00ffcf0 0x7ff9b0196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52934/0 (socket says 172.21.15.82:52934) 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9aeffd700 1 -- 172.21.15.82:0/85961992 learned_addr learned my addr 172.21.15.82:0/85961992 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9af7fe700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b0101890 0x7ff9b019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9aeffd700 1 -- 172.21.15.82:0/85961992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b0101890 msgr2=0x7ff9b019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9aeffd700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b0101890 0x7ff9b019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9aeffd700 1 -- 172.21.15.82:0/85961992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0100ef0 msgr2=0x7ff9b0197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9aeffd700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0100ef0 0x7ff9b0197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9aeffd700 1 -- 172.21.15.82:0/85961992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a8009d00 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9af7fe700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b0101890 0x7ff9b019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9aeffd700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b00ffcf0 0x7ff9b0196f30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400b500 tx=0x7ff9a400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff997fff700 1 -- 172.21.15.82:0/85961992 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4011830 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b019efd0 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff997fff700 1 -- 172.21.15.82:0/85961992 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9a4011e50 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff997fff700 1 -- 172.21.15.82:0/85961992 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400f2e0 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.026+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b019f460 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.027+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b004eaf0 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.028+0000 7ff997fff700 1 -- 172.21.15.82:0/85961992 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff9a4010580 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.029+0000 7ff997fff700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9980a11a0 0x7ff9980a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:35.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.029+0000 7ff997fff700 1 -- 172.21.15.82:0/85961992 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5981+0+0 (secure 0 0 0) 0x7ff9a40cbb20 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.029+0000 7ff9ae7fc700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9980a11a0 0x7ff9980a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:35.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.030+0000 7ff9ae7fc700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9980a11a0 0x7ff9980a3660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00097b0 tx=0x7ff9a0006d20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:33:35.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.033+0000 7ff997fff700 1 -- 172.21.15.82:0/85961992 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff9b004eaf0 con 0x7ff9b00ffcf0 2024-03-30T02:33:35.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:35.235+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9b00fd4a0 con 0x7ff9980a11a0 2024-03-30T02:33:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:33:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: osdmap e61: 10 total, 9 up, 10 in 2024-03-30T02:33:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi067 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi067 to 4317M 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:33:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:33:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: osdmap e61: 10 total, 9 up, 10 in 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi067 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi067 to 4317M 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-30T02:33:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:33:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:33:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: osdmap e61: 10 total, 9 up, 10 in 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi067 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi067 to 4317M 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-30T02:33:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:33:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:33:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:36 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:33:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:36 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:33:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:36 smithi082 ceph-mon[79176]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:33:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:36 smithi082 ceph-mon[79176]: osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] boot 2024-03-30T02:33:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:36 smithi082 ceph-mon[79176]: osdmap e62: 10 total, 10 up, 10 in 2024-03-30T02:33:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:36 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:36.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:36 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:33:36.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:36 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:33:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:36 smithi067 ceph-mon[37808]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:33:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:36 smithi067 ceph-mon[37808]: osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] boot 2024-03-30T02:33:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:36 smithi067 ceph-mon[37808]: osdmap e62: 10 total, 10 up, 10 in 2024-03-30T02:33:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:36 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:36.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:36 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:33:36.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:36 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:33:36.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:36 smithi170 ceph-mon[78798]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:33:36.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:36 smithi170 ceph-mon[78798]: osd.9 [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] boot 2024-03-30T02:33:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:36 smithi170 ceph-mon[78798]: osdmap e62: 10 total, 10 up, 10 in 2024-03-30T02:33:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:36 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:33:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:37 smithi082 ceph-mon[79176]: osdmap e63: 10 total, 10 up, 10 in 2024-03-30T02:33:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:37 smithi082 ceph-mon[79176]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:37 smithi082 ceph-mon[79176]: from='client.? 172.21.15.82:0/3873607440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a7ade41-1c90-42e9-a53f-6f75074c9f5b"}]: dispatch 2024-03-30T02:33:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:37 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a7ade41-1c90-42e9-a53f-6f75074c9f5b"}]: dispatch 2024-03-30T02:33:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:37 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a7ade41-1c90-42e9-a53f-6f75074c9f5b"}]': finished 2024-03-30T02:33:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:37 smithi082 ceph-mon[79176]: osdmap e64: 11 total, 10 up, 11 in 2024-03-30T02:33:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:37 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:37.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:37 smithi067 ceph-mon[37808]: osdmap e63: 10 total, 10 up, 10 in 2024-03-30T02:33:37.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:37 smithi067 ceph-mon[37808]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:37.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:37 smithi067 ceph-mon[37808]: from='client.? 172.21.15.82:0/3873607440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a7ade41-1c90-42e9-a53f-6f75074c9f5b"}]: dispatch 2024-03-30T02:33:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:37 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a7ade41-1c90-42e9-a53f-6f75074c9f5b"}]: dispatch 2024-03-30T02:33:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:37 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a7ade41-1c90-42e9-a53f-6f75074c9f5b"}]': finished 2024-03-30T02:33:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:37 smithi067 ceph-mon[37808]: osdmap e64: 11 total, 10 up, 11 in 2024-03-30T02:33:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:37 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:37 smithi170 ceph-mon[78798]: osdmap e63: 10 total, 10 up, 10 in 2024-03-30T02:33:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:37 smithi170 ceph-mon[78798]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:37 smithi170 ceph-mon[78798]: from='client.? 172.21.15.82:0/3873607440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a7ade41-1c90-42e9-a53f-6f75074c9f5b"}]: dispatch 2024-03-30T02:33:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:37 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a7ade41-1c90-42e9-a53f-6f75074c9f5b"}]: dispatch 2024-03-30T02:33:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:37 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a7ade41-1c90-42e9-a53f-6f75074c9f5b"}]': finished 2024-03-30T02:33:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:37 smithi170 ceph-mon[78798]: osdmap e64: 11 total, 10 up, 11 in 2024-03-30T02:33:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:37 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:38 smithi082 ceph-mon[79176]: from='client.? 172.21.15.82:0/1620201168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:33:38.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:38 smithi067 ceph-mon[37808]: from='client.? 172.21.15.82:0/1620201168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:33:38.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:38 smithi170 ceph-mon[78798]: from='client.? 172.21.15.82:0/1620201168' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:33:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:39 smithi082 ceph-mon[79176]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:39 smithi067 ceph-mon[37808]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:39 smithi170 ceph-mon[78798]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:42.196 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:41 smithi082 ceph-mon[79176]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:41 smithi067 ceph-mon[37808]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:42.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:41 smithi170 ceph-mon[78798]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:43.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-30T02:33:43.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:43.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-30T02:33:43.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-30T02:33:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:44.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:43 smithi082 ceph-mon[79176]: Deploying daemon osd.10 on smithi082 2024-03-30T02:33:44.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:43 smithi082 ceph-mon[79176]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:43 smithi067 ceph-mon[37808]: Deploying daemon osd.10 on smithi082 2024-03-30T02:33:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:43 smithi067 ceph-mon[37808]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:43 smithi170 ceph-mon[78798]: Deploying daemon osd.10 on smithi082 2024-03-30T02:33:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:43 smithi170 ceph-mon[78798]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:46.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:46 smithi082 ceph-mon[79176]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:46.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:46 smithi067 ceph-mon[37808]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:46.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:46 smithi170 ceph-mon[78798]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:47.846 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:47.846 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:47.846 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:47.846 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:47 smithi082 ceph-mon[79176]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:47 smithi067 ceph-mon[37808]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:47 smithi170 ceph-mon[78798]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:48.688 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 10 on host 'smithi082' 2024-03-30T02:33:48.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.687+0000 7ff997fff700 1 -- 172.21.15.82:0/85961992 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff9b00fd4a0 con 0x7ff9980a11a0 2024-03-30T02:33:48.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9980a11a0 msgr2=0x7ff9980a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:48.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9980a11a0 0x7ff9980a3660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00097b0 tx=0x7ff9a0006d20 comp rx=0 tx=0).stop 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b00ffcf0 msgr2=0x7ff9b0196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b00ffcf0 0x7ff9b0196f30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400b500 tx=0x7ff9a400b8c0 comp rx=0 tx=0).stop 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 shutdown_connections 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9980a11a0 0x7ff9980a3660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b0101890 0x7ff9b019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b00ffcf0 0x7ff9b0196f30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 --2- 172.21.15.82:0/85961992 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9b0100ef0 0x7ff9b0197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 >> 172.21.15.82:0/85961992 conn(0x7ff9b00fb2b0 msgr2=0x7ff9b00fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 shutdown_connections 2024-03-30T02:33:48.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-30T02:33:48.690+0000 7ff9b54ca700 1 -- 172.21.15.82:0/85961992 wait complete. 2024-03-30T02:33:49.073 DEBUG:teuthology.orchestra.run.smithi082:osd.10> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.10.service 2024-03-30T02:33:49.075 INFO:tasks.cephadm:Deploying osd.11 on smithi170 with /dev/vg_nvme/lv_1... 2024-03-30T02:33:49.075 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-30T02:33:49.104 INFO:journalctl@ceph.osd.10.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:33:49.224 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:49 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.225 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:49 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.225 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:49 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:49.225 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:49 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:49.225 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:49 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.225 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:49 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.225 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:49 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.225 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:49 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:49.225 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:49 smithi082 ceph-mon[79176]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:49.308 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:33:49.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:49 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:49 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:49 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:49.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:49 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:49.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:49 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:49 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:49 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:49 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:49.409 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:49 smithi170 ceph-mon[78798]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:49.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:49 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:49 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:49 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:49.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:49 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:49.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:49 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:49 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:49 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:49 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:33:49.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:49 smithi067 ceph-mon[37808]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-30T02:33:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-30T02:33:50.656 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-03-30T02:33:50.657 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-03-30T02:33:50.657 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00840613 s, 1.2 GB/s 2024-03-30T02:33:50.657 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-03-30T02:33:50.878 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:50 smithi082 ceph-mon[79176]: from='osd.10 [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T02:33:50.878 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:50 smithi082 ceph-mon[79176]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T02:33:50.878 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:33:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:33:50.545+0000 7f04f19cc640 -1 osd.10 0 log_to_monitors true 2024-03-30T02:33:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:50 smithi170 ceph-mon[78798]: from='osd.10 [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T02:33:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:50 smithi170 ceph-mon[78798]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T02:33:51.054 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2024-03-30T02:33:51.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:50 smithi067 ceph-mon[37808]: from='osd.10 [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T02:33:51.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:50 smithi067 ceph-mon[37808]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-30T02:33:51.275 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:33:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-30T02:33:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: osdmap e65: 11 total, 10 up, 11 in 2024-03-30T02:33:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='osd.10 [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:33:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi082 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi082 to 4278M 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:51 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-30T02:33:52.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: osdmap e65: 11 total, 10 up, 11 in 2024-03-30T02:33:52.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:52.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='osd.10 [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:33:52.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:33:52.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:52.076 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi082 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi082 to 4278M 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:52.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:51 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: osdmap e65: 11 total, 10 up, 11 in 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='osd.10 [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi082 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:33:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi082 to 4278M 2024-03-30T02:33:52.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:52.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:33:52.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:51 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:33:52.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.260+0000 7f4d44ebb700 1 -- 172.21.15.170:0/2633463472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40100ef0 msgr2=0x7f4d40101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:52.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.260+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/2633463472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40100ef0 0x7f4d40101350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009990 tx=0x7f4d34009ca0 comp rx=0 tx=0).stop 2024-03-30T02:33:52.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.260+0000 7f4d44ebb700 1 -- 172.21.15.170:0/2633463472 shutdown_connections 2024-03-30T02:33:52.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.260+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/2633463472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d400ffcf0 0x7f4d40100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:52.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.260+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/2633463472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d40101890 0x7f4d40108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.260+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/2633463472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40100ef0 0x7f4d40101350 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.260+0000 7f4d44ebb700 1 -- 172.21.15.170:0/2633463472 >> 172.21.15.170:0/2633463472 conn(0x7f4d400fb2b0 msgr2=0x7f4d400fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.260+0000 7f4d44ebb700 1 -- 172.21.15.170:0/2633463472 shutdown_connections 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.260+0000 7f4d44ebb700 1 -- 172.21.15.170:0/2633463472 wait complete. 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d44ebb700 1 Processor -- start 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d44ebb700 1 -- start start 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d44ebb700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d400ffcf0 0x7f4d40199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d44ebb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d40100ef0 0x7f4d40199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d44ebb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40101890 0x7f4d401a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d44ebb700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d401a0af0 con 0x7f4d400ffcf0 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d44ebb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d401a0c30 con 0x7f4d40100ef0 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d44ebb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d401a0dc0 con 0x7f4d40101890 2024-03-30T02:33:52.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d3ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40101890 0x7f4d401a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d3ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40101890 0x7f4d401a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46426/0 (socket says 172.21.15.170:46426) 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d3ed9d700 1 -- 172.21.15.170:0/1965658423 learned_addr learned my addr 172.21.15.170:0/1965658423 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.261+0000 7f4d3dd9b700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d40100ef0 0x7f4d40199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d3ed9d700 1 -- 172.21.15.170:0/1965658423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d40100ef0 msgr2=0x7f4d40199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d3ed9d700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d40100ef0 0x7f4d40199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d3ed9d700 1 -- 172.21.15.170:0/1965658423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d400ffcf0 msgr2=0x7f4d40199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d3ed9d700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d400ffcf0 0x7f4d40199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d3ed9d700 1 -- 172.21.15.170:0/1965658423 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d340095f0 con 0x7f4d40101890 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d3ed9d700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40101890 0x7f4d401a0520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800add0 tx=0x7f4d3800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d2f7fe700 1 -- 172.21.15.170:0/1965658423 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800f070 con 0x7f4d40101890 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d401a10a0 con 0x7f4d40101890 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d2f7fe700 1 -- 172.21.15.170:0/1965658423 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d38007b30 con 0x7f4d40101890 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d2f7fe700 1 -- 172.21.15.170:0/1965658423 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38005310 con 0x7f4d40101890 2024-03-30T02:33:52.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.262+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d401a15a0 con 0x7f4d40101890 2024-03-30T02:33:52.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.263+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4004ea80 con 0x7f4d40101890 2024-03-30T02:33:52.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.264+0000 7f4d2f7fe700 1 -- 172.21.15.170:0/1965658423 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4d3800c940 con 0x7f4d40101890 2024-03-30T02:33:52.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.265+0000 7f4d2f7fe700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4d280a11a0 0x7f4d280a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:33:52.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.265+0000 7f4d2f7fe700 1 -- 172.21.15.170:0/1965658423 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4d380cbcc0 con 0x7f4d40101890 2024-03-30T02:33:52.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.265+0000 7f4d3e59c700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4d280a11a0 0x7f4d280a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:33:52.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.266+0000 7f4d3e59c700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4d280a11a0 0x7f4d280a3660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4d40100d50 tx=0x7f4d30006d20 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:33:52.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.269+0000 7f4d2f7fe700 1 -- 172.21.15.170:0/1965658423 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4d38095c50 con 0x7f4d40101890 2024-03-30T02:33:52.470 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:33:52.470+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d40068e60 con 0x7f4d280a11a0 2024-03-30T02:33:52.500 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:33:52.150+0000 7f04e7030700 -1 osd.10 0 waiting for initial osdmap 2024-03-30T02:33:52.500 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:33:52.164+0000 7f04dee1d700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:33:52.987 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:52 smithi170 ceph-mon[78798]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:33:52.988 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:52 smithi170 ceph-mon[78798]: osdmap e66: 11 total, 10 up, 11 in 2024-03-30T02:33:52.988 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:52.988 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:52.988 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:52 smithi170 ceph-mon[78798]: from='osd.10 ' entity='osd.10' 2024-03-30T02:33:52.988 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:33:52.988 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:33:52.988 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:52 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-mon[79176]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:33:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-mon[79176]: osdmap e66: 11 total, 10 up, 11 in 2024-03-30T02:33:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-mon[79176]: from='osd.10 ' entity='osd.10' 2024-03-30T02:33:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:33:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:33:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:52 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:53.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:52 smithi067 ceph-mon[37808]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-03-30T02:33:53.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:52 smithi067 ceph-mon[37808]: osdmap e66: 11 total, 10 up, 11 in 2024-03-30T02:33:53.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:53.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:53.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:52 smithi067 ceph-mon[37808]: from='osd.10 ' entity='osd.10' 2024-03-30T02:33:53.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-30T02:33:53.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-30T02:33:53.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:52 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:33:54.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:53 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:33:54.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:53 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:33:54.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:53 smithi082 ceph-mon[79176]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:33:54.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:53 smithi082 ceph-mon[79176]: osd.10 [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] boot 2024-03-30T02:33:54.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:53 smithi082 ceph-mon[79176]: osdmap e67: 11 total, 11 up, 11 in 2024-03-30T02:33:54.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:53 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:54.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:53 smithi082 ceph-mon[79176]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:54.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:53 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:33:54.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:53 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:33:54.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:53 smithi067 ceph-mon[37808]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:33:54.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:53 smithi067 ceph-mon[37808]: osd.10 [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] boot 2024-03-30T02:33:54.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:53 smithi067 ceph-mon[37808]: osdmap e67: 11 total, 11 up, 11 in 2024-03-30T02:33:54.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:53 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:54.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:53 smithi067 ceph-mon[37808]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:54.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:53 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:33:54.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:53 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:33:54.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:53 smithi170 ceph-mon[78798]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:33:54.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:53 smithi170 ceph-mon[78798]: osd.10 [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] boot 2024-03-30T02:33:54.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:53 smithi170 ceph-mon[78798]: osdmap e67: 11 total, 11 up, 11 in 2024-03-30T02:33:54.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:53 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:33:54.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:53 smithi170 ceph-mon[78798]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-03-30T02:33:55.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:54 smithi082 ceph-mon[79176]: osdmap e68: 11 total, 11 up, 11 in 2024-03-30T02:33:55.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:54 smithi082 ceph-mon[79176]: from='client.? 172.21.15.170:0/2806556996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b717af-28b4-4aff-98f9-b652333a9808"}]: dispatch 2024-03-30T02:33:55.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:54 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b717af-28b4-4aff-98f9-b652333a9808"}]: dispatch 2024-03-30T02:33:55.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:54 smithi082 ceph-mon[79176]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7b717af-28b4-4aff-98f9-b652333a9808"}]': finished 2024-03-30T02:33:55.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:54 smithi082 ceph-mon[79176]: osdmap e69: 12 total, 11 up, 12 in 2024-03-30T02:33:55.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:54 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:33:55.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:54 smithi067 ceph-mon[37808]: osdmap e68: 11 total, 11 up, 11 in 2024-03-30T02:33:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:54 smithi067 ceph-mon[37808]: from='client.? 172.21.15.170:0/2806556996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b717af-28b4-4aff-98f9-b652333a9808"}]: dispatch 2024-03-30T02:33:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:54 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b717af-28b4-4aff-98f9-b652333a9808"}]: dispatch 2024-03-30T02:33:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:54 smithi067 ceph-mon[37808]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7b717af-28b4-4aff-98f9-b652333a9808"}]': finished 2024-03-30T02:33:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:54 smithi067 ceph-mon[37808]: osdmap e69: 12 total, 11 up, 12 in 2024-03-30T02:33:55.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:54 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:33:55.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:54 smithi170 ceph-mon[78798]: osdmap e68: 11 total, 11 up, 11 in 2024-03-30T02:33:55.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:54 smithi170 ceph-mon[78798]: from='client.? 172.21.15.170:0/2806556996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b717af-28b4-4aff-98f9-b652333a9808"}]: dispatch 2024-03-30T02:33:55.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:54 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7b717af-28b4-4aff-98f9-b652333a9808"}]: dispatch 2024-03-30T02:33:55.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:54 smithi170 ceph-mon[78798]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7b717af-28b4-4aff-98f9-b652333a9808"}]': finished 2024-03-30T02:33:55.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:54 smithi170 ceph-mon[78798]: osdmap e69: 12 total, 11 up, 12 in 2024-03-30T02:33:55.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:54 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:33:56.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:55 smithi082 ceph-mon[79176]: from='client.? 172.21.15.170:0/1212187768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:33:56.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:55 smithi082 ceph-mon[79176]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:33:56.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:55 smithi082 ceph-mon[79176]: osdmap e70: 12 total, 11 up, 12 in 2024-03-30T02:33:56.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:55 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:33:56.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:55 smithi067 ceph-mon[37808]: from='client.? 172.21.15.170:0/1212187768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:33:56.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:55 smithi067 ceph-mon[37808]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:33:56.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:55 smithi067 ceph-mon[37808]: osdmap e70: 12 total, 11 up, 12 in 2024-03-30T02:33:56.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:55 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:33:56.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:55 smithi170 ceph-mon[78798]: from='client.? 172.21.15.170:0/1212187768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-30T02:33:56.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:55 smithi170 ceph-mon[78798]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:33:56.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:55 smithi170 ceph-mon[78798]: osdmap e70: 12 total, 11 up, 12 in 2024-03-30T02:33:56.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:55 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:33:57.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:57 smithi082 ceph-mon[79176]: osdmap e71: 12 total, 11 up, 12 in 2024-03-30T02:33:57.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:57 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:33:57.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:33:57 smithi082 ceph-mon[79176]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:33:57.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:57 smithi067 ceph-mon[37808]: osdmap e71: 12 total, 11 up, 12 in 2024-03-30T02:33:57.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:33:57.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:33:57 smithi067 ceph-mon[37808]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:33:57.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:57 smithi170 ceph-mon[78798]: osdmap e71: 12 total, 11 up, 12 in 2024-03-30T02:33:57.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:57 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:33:57.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:33:57 smithi170 ceph-mon[78798]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-30T02:34:00.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:00 smithi067 ceph-mon[37808]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-30T02:34:00.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:00 smithi170 ceph-mon[78798]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-30T02:34:00.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:00 smithi082 ceph-mon[79176]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-03-30T02:34:01.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:01 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-30T02:34:01.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:01 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:01.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:01 smithi170 ceph-mon[78798]: Deploying daemon osd.11 on smithi170 2024-03-30T02:34:01.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:01 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-30T02:34:01.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:01 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:01.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:01 smithi067 ceph-mon[37808]: Deploying daemon osd.11 on smithi170 2024-03-30T02:34:01.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:01 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-30T02:34:01.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:01 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:01.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:01 smithi082 ceph-mon[79176]: Deploying daemon osd.11 on smithi170 2024-03-30T02:34:02.342 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:02 smithi170 ceph-mon[78798]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-30T02:34:02.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:02 smithi067 ceph-mon[37808]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-30T02:34:02.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:02 smithi082 ceph-mon[79176]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-30T02:34:04.390 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:04 smithi170 ceph-mon[78798]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T02:34:04.390 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:04.390 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:04.390 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:04 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:04 smithi067 ceph-mon[37808]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T02:34:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:04.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:04 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:04 smithi082 ceph-mon[79176]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-30T02:34:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:04 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:04 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:04 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:05.825 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 11 on host 'smithi170' 2024-03-30T02:34:05.826 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.823+0000 7f4d2f7fe700 1 -- 172.21.15.170:0/1965658423 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4d40068e60 con 0x7f4d280a11a0 2024-03-30T02:34:05.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4d280a11a0 msgr2=0x7f4d280a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:05.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4d280a11a0 0x7f4d280a3660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4d40100d50 tx=0x7f4d30006d20 comp rx=0 tx=0).stop 2024-03-30T02:34:05.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40101890 msgr2=0x7f4d401a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:05.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40101890 0x7f4d401a0520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800add0 tx=0x7f4d3800c330 comp rx=0 tx=0).stop 2024-03-30T02:34:05.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 shutdown_connections 2024-03-30T02:34:05.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4d280a11a0 0x7f4d280a3660 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:05.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d400ffcf0 0x7f4d40199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:05.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40101890 0x7f4d401a0520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:05.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 --2- 172.21.15.170:0/1965658423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d40100ef0 0x7f4d40199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:05.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.825+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 >> 172.21.15.170:0/1965658423 conn(0x7f4d400fb2b0 msgr2=0x7f4d40068750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:05.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.826+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 shutdown_connections 2024-03-30T02:34:05.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:05.826+0000 7f4d44ebb700 1 -- 172.21.15.170:0/1965658423 wait complete. 2024-03-30T02:34:06.223 DEBUG:teuthology.orchestra.run.smithi170:osd.11> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.11.service 2024-03-30T02:34:06.226 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-30T02:34:06.226 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-03-30T02:34:06.255 INFO:journalctl@ceph.osd.11.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:34:06.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:06 smithi170 ceph-mon[78798]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-30T02:34:06.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:06 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:06 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:06 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:06.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:06 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:06.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:06 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:06 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:06 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.312 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:06 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:06.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:06 smithi067 ceph-mon[37808]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-30T02:34:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:06 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:06 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:06 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:06 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:06 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:06 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:06 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:06 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:06 smithi082 ceph-mon[79176]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-30T02:34:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:06 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:06 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:06 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:06 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:06 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:06 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:06 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:06 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:06.603 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:34:07.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.743+0000 7ff6b2800700 1 -- 172.21.15.67:0/1271271436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 msgr2=0x7ff6ac103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:07.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.743+0000 7ff6b2800700 1 --2- 172.21.15.67:0/1271271436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 0x7ff6ac103aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009990 tx=0x7ff6a0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.743+0000 7ff6b2800700 1 -- 172.21.15.67:0/1271271436 shutdown_connections 2024-03-30T02:34:07.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.743+0000 7ff6b2800700 1 --2- 172.21.15.67:0/1271271436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 0x7ff6ac103aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.743+0000 7ff6b2800700 1 --2- 172.21.15.67:0/1271271436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6ac106910 0x7ff6ac108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.743+0000 7ff6b2800700 1 --2- 172.21.15.67:0/1271271436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ac103fe0 0x7ff6ac1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.743+0000 7ff6b2800700 1 -- 172.21.15.67:0/1271271436 >> 172.21.15.67:0/1271271436 conn(0x7ff6ac0fb0c0 msgr2=0x7ff6ac0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:07.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.743+0000 7ff6b2800700 1 -- 172.21.15.67:0/1271271436 shutdown_connections 2024-03-30T02:34:07.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.743+0000 7ff6b2800700 1 -- 172.21.15.67:0/1271271436 wait complete. 2024-03-30T02:34:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.744+0000 7ff6b2800700 1 Processor -- start 2024-03-30T02:34:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b2800700 1 -- start start 2024-03-30T02:34:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 0x7ff6ac198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b2800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6ac103fe0 0x7ff6ac1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ac106910 0x7ff6ac1a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b2800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1a0860 con 0x7ff6ac1016b0 2024-03-30T02:34:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b2800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1a09a0 con 0x7ff6ac106910 2024-03-30T02:34:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b2800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1a0b60 con 0x7ff6ac103fe0 2024-03-30T02:34:07.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 0x7ff6ac198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b0a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6ac103fe0 0x7ff6ac1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b0a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6ac103fe0 0x7ff6ac1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:45600/0 (socket says 172.21.15.67:45600) 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b0a00700 1 -- 172.21.15.67:0/945041723 learned_addr learned my addr 172.21.15.67:0/945041723 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 0x7ff6ac198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37974/0 (socket says 172.21.15.67:37974) 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.745+0000 7ff6b1e00700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ac106910 0x7ff6ac1a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6b0a00700 1 -- 172.21.15.67:0/945041723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ac106910 msgr2=0x7ff6ac1a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6b0a00700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ac106910 0x7ff6ac1a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6b0a00700 1 -- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 msgr2=0x7ff6ac198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6b0a00700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 0x7ff6ac198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6b0a00700 1 -- 172.21.15.67:0/945041723 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a00095f0 con 0x7ff6ac103fe0 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6b1400700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 0x7ff6ac198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:07.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6b0a00700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6ac103fe0 0x7ff6ac1992a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9a0 tx=0x7ff69800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6aa000700 1 -- 172.21.15.67:0/945041723 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698009790 con 0x7ff6ac103fe0 2024-03-30T02:34:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6aa000700 1 -- 172.21.15.67:0/945041723 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff698010430 con 0x7ff6ac103fe0 2024-03-30T02:34:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac1a0e40 con 0x7ff6ac103fe0 2024-03-30T02:34:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6aa000700 1 -- 172.21.15.67:0/945041723 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698018680 con 0x7ff6ac103fe0 2024-03-30T02:34:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.746+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac1a12b0 con 0x7ff6ac103fe0 2024-03-30T02:34:07.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.747+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac1084e0 con 0x7ff6ac103fe0 2024-03-30T02:34:07.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.749+0000 7ff6aa000700 1 -- 172.21.15.67:0/945041723 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff6980105a0 con 0x7ff6ac103fe0 2024-03-30T02:34:07.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.750+0000 7ff6aa000700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff6940a11a0 0x7ff6940a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:07.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.751+0000 7ff6aa000700 1 -- 172.21.15.67:0/945041723 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6809+0+0 (secure 0 0 0) 0x7ff698014070 con 0x7ff6ac103fe0 2024-03-30T02:34:07.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.751+0000 7ff6b1400700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff6940a11a0 0x7ff6940a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:07.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.751+0000 7ff6b1400700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff6940a11a0 0x7ff6940a3660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b680 tx=0x7ff6a0005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:07.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.754+0000 7ff6aa000700 1 -- 172.21.15.67:0/945041723 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff6980529f0 con 0x7ff6ac103fe0 2024-03-30T02:34:07.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:34:07 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:34:07.571+0000 7f9e9fa0a640 -1 osd.11 0 log_to_monitors true 2024-03-30T02:34:07.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.964+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff6ac04ea80 con 0x7ff6ac103fe0 2024-03-30T02:34:07.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.965+0000 7ff6aa000700 1 -- 172.21.15.67:0/945041723 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) v1 ==== 74+0+133 (secure 0 0 0) 0x7ff6980961a0 con 0x7ff6ac103fe0 2024-03-30T02:34:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:07.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff6940a11a0 msgr2=0x7ff6940a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:07.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff6940a11a0 0x7ff6940a3660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000b680 tx=0x7ff6a0005fd0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6ac103fe0 msgr2=0x7ff6ac1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6ac103fe0 0x7ff6ac1992a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff69800e9a0 tx=0x7ff69800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 shutdown_connections 2024-03-30T02:34:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff6940a11a0 0x7ff6940a3660 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6ac1016b0 0x7ff6ac198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6ac106910 0x7ff6ac1a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 --2- 172.21.15.67:0/945041723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6ac103fe0 0x7ff6ac1992a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 >> 172.21.15.67:0/945041723 conn(0x7ff6ac0fb0c0 msgr2=0x7ff6ac100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 shutdown_connections 2024-03-30T02:34:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:07.967+0000 7ff6b2800700 1 -- 172.21.15.67:0/945041723 wait complete. 2024-03-30T02:34:08.250 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:08 smithi170 ceph-mon[78798]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:34:08.250 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:08 smithi170 ceph-mon[78798]: from='osd.11 [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T02:34:08.250 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:08 smithi170 ceph-mon[78798]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T02:34:08.251 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:08 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/945041723' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T02:34:08.355 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:08 smithi067 ceph-mon[37808]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:34:08.355 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:08 smithi067 ceph-mon[37808]: from='osd.11 [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T02:34:08.355 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:08 smithi067 ceph-mon[37808]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T02:34:08.355 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:08 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/945041723' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T02:34:08.381 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1711766032,"num_in_osds":12,"osd_in_since":1711766034,"num_remapped_pgs":0} 2024-03-30T02:34:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:08 smithi082 ceph-mon[79176]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:34:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:08 smithi082 ceph-mon[79176]: from='osd.11 [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T02:34:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:08 smithi082 ceph-mon[79176]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-30T02:34:08.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:08 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/945041723' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T02:34:09.382 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-03-30T02:34:09.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-30T02:34:09.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: osdmap e72: 12 total, 11 up, 12 in 2024-03-30T02:34:09.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:09.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='osd.11 [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:09.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:09 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.439 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:34:09.073+0000 7f9e9506e700 -1 osd.11 0 waiting for initial osdmap 2024-03-30T02:34:09.439 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:34:09.090+0000 7f9e90e63700 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: osdmap e72: 12 total, 11 up, 12 in 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='osd.11 [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:09.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:09.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:09 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: osdmap e72: 12 total, 11 up, 12 in 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='osd.11 [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:34:09.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:09.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:09.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:09 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:09.756 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:34:10.274 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:10 smithi067 ceph-mon[37808]: Detected new or changed devices on smithi170 2024-03-30T02:34:10.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:10 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi170 to 4278M 2024-03-30T02:34:10.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:10 smithi067 ceph-mon[37808]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:34:10.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:10 smithi067 ceph-mon[37808]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:34:10.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:10 smithi067 ceph-mon[37808]: osdmap e73: 12 total, 11 up, 12 in 2024-03-30T02:34:10.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:10 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:10.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:10 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:10.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:10 smithi170 ceph-mon[78798]: Detected new or changed devices on smithi170 2024-03-30T02:34:10.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:10 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi170 to 4278M 2024-03-30T02:34:10.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:10 smithi170 ceph-mon[78798]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:34:10.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:10 smithi170 ceph-mon[78798]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:34:10.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:10 smithi170 ceph-mon[78798]: osdmap e73: 12 total, 11 up, 12 in 2024-03-30T02:34:10.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:10 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:10.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:10 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:10.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:10 smithi082 ceph-mon[79176]: Detected new or changed devices on smithi170 2024-03-30T02:34:10.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:10 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi170 to 4278M 2024-03-30T02:34:10.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:10 smithi082 ceph-mon[79176]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:34:10.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:10 smithi082 ceph-mon[79176]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-03-30T02:34:10.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:10 smithi082 ceph-mon[79176]: osdmap e73: 12 total, 11 up, 12 in 2024-03-30T02:34:10.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:10 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:10.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:10 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:10.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.961+0000 7f2328000700 1 -- 172.21.15.67:0/3117509636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320100d80 msgr2=0x7f23201011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:10.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.961+0000 7f2328000700 1 --2- 172.21.15.67:0/3117509636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320100d80 0x7f23201011a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23140099c0 tx=0x7f2314009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.961+0000 7f2328000700 1 -- 172.21.15.67:0/3117509636 shutdown_connections 2024-03-30T02:34:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.961+0000 7f2328000700 1 --2- 172.21.15.67:0/3117509636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2320102920 0x7f23201091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.961+0000 7f2328000700 1 --2- 172.21.15.67:0/3117509636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320100d80 0x7f23201011a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.961+0000 7f2328000700 1 --2- 172.21.15.67:0/3117509636 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320101f80 0x7f23201023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.961+0000 7f2328000700 1 -- 172.21.15.67:0/3117509636 >> 172.21.15.67:0/3117509636 conn(0x7f23200fc320 msgr2=0x7f23200fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.961+0000 7f2328000700 1 -- 172.21.15.67:0/3117509636 shutdown_connections 2024-03-30T02:34:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.961+0000 7f2328000700 1 -- 172.21.15.67:0/3117509636 wait complete. 2024-03-30T02:34:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.962+0000 7f2328000700 1 Processor -- start 2024-03-30T02:34:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.962+0000 7f2328000700 1 -- start start 2024-03-30T02:34:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.962+0000 7f2328000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320100d80 0x7f2320198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.962+0000 7f2328000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2320101f80 0x7f2320199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.962+0000 7f2328000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320102920 0x7f23201a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.962+0000 7f2328000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23201a0a30 con 0x7f2320102920 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.962+0000 7f2328000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23201a0b70 con 0x7f2320100d80 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.962+0000 7f2328000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23201a0d00 con 0x7f2320101f80 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2326000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320102920 0x7f23201a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2326000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320102920 0x7f23201a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37994/0 (socket says 172.21.15.67:37994) 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2326000700 1 -- 172.21.15.67:0/858666048 learned_addr learned my addr 172.21.15.67:0/858666048 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2324c00700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2320101f80 0x7f2320199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2325600700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320100d80 0x7f2320198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2326000700 1 -- 172.21.15.67:0/858666048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2320101f80 msgr2=0x7f2320199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2326000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2320101f80 0x7f2320199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2326000700 1 -- 172.21.15.67:0/858666048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320100d80 msgr2=0x7f2320198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2326000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320100d80 0x7f2320198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2326000700 1 -- 172.21.15.67:0/858666048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c009520 con 0x7f2320102920 2024-03-30T02:34:10.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.963+0000 7f2326000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320102920 0x7f23201a0460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2318007f10 tx=0x7f231800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:10.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.964+0000 7f231e000700 1 -- 172.21.15.67:0/858666048 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800dd20 con 0x7f2320102920 2024-03-30T02:34:10.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.964+0000 7f231e000700 1 -- 172.21.15.67:0/858666048 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f231800f040 con 0x7f2320102920 2024-03-30T02:34:10.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.964+0000 7f231e000700 1 -- 172.21.15.67:0/858666048 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23180048a0 con 0x7f2320102920 2024-03-30T02:34:10.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.964+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23140095f0 con 0x7f2320102920 2024-03-30T02:34:10.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.964+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23201a1270 con 0x7f2320102920 2024-03-30T02:34:10.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.965+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f232004ea80 con 0x7f2320102920 2024-03-30T02:34:10.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.969+0000 7f231e000700 1 -- 172.21.15.67:0/858666048 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f2318003a00 con 0x7f2320102920 2024-03-30T02:34:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.970+0000 7f231e000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23080a11a0 0x7f23080a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.970+0000 7f231e000700 1 -- 172.21.15.67:0/858666048 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f23180cc450 con 0x7f2320102920 2024-03-30T02:34:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.970+0000 7f2325600700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23080a11a0 0x7f23080a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.971+0000 7f2325600700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23080a11a0 0x7f23080a3660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2314009990 tx=0x7f23140036f0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:10.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:10.973+0000 7f231e000700 1 -- 172.21.15.67:0/858666048 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2318095fe0 con 0x7f2320102920 2024-03-30T02:34:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.188+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f232010c580 con 0x7f2320102920 2024-03-30T02:34:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.189+0000 7f231e000700 1 -- 172.21.15.67:0/858666048 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f23180230a0 con 0x7f2320102920 2024-03-30T02:34:11.189 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23080a11a0 msgr2=0x7f23080a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23080a11a0 0x7f23080a3660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2314009990 tx=0x7f23140036f0 comp rx=0 tx=0).stop 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320102920 msgr2=0x7f23201a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320102920 0x7f23201a0460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2318007f10 tx=0x7f231800d3b0 comp rx=0 tx=0).stop 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 shutdown_connections 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23080a11a0 0x7f23080a3660 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2320102920 0x7f23201a0460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320100d80 0x7f2320198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 --2- 172.21.15.67:0/858666048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2320101f80 0x7f2320199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 >> 172.21.15.67:0/858666048 conn(0x7f23200fc320 msgr2=0x7f2320107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 shutdown_connections 2024-03-30T02:34:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:11.191+0000 7f2328000700 1 -- 172.21.15.67:0/858666048 wait complete. 2024-03-30T02:34:11.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:11 smithi067 ceph-mon[37808]: purged_snaps scrub starts 2024-03-30T02:34:11.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:11 smithi067 ceph-mon[37808]: purged_snaps scrub ok 2024-03-30T02:34:11.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:11 smithi067 ceph-mon[37808]: osd.11 [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] boot 2024-03-30T02:34:11.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:11 smithi067 ceph-mon[37808]: osdmap e74: 12 total, 12 up, 12 in 2024-03-30T02:34:11.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:11 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:11.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:11 smithi170 ceph-mon[78798]: purged_snaps scrub starts 2024-03-30T02:34:11.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:11 smithi170 ceph-mon[78798]: purged_snaps scrub ok 2024-03-30T02:34:11.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:11 smithi170 ceph-mon[78798]: osd.11 [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] boot 2024-03-30T02:34:11.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:11 smithi170 ceph-mon[78798]: osdmap e74: 12 total, 12 up, 12 in 2024-03-30T02:34:11.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:11 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:11.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:11 smithi082 ceph-mon[79176]: purged_snaps scrub starts 2024-03-30T02:34:11.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:11 smithi082 ceph-mon[79176]: purged_snaps scrub ok 2024-03-30T02:34:11.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:11 smithi082 ceph-mon[79176]: osd.11 [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] boot 2024-03-30T02:34:11.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:11 smithi082 ceph-mon[79176]: osdmap e74: 12 total, 12 up, 12 in 2024-03-30T02:34:11.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:11 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:34:11.568 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1711766050,"num_in_osds":12,"osd_in_since":1711766034,"num_remapped_pgs":0} 2024-03-30T02:34:11.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-03-30T02:34:11.970 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:34:12.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:12 smithi067 ceph-mon[37808]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:34:12.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:12 smithi067 ceph-mon[37808]: osdmap e75: 12 total, 12 up, 12 in 2024-03-30T02:34:12.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:12 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/858666048' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T02:34:12.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:12 smithi170 ceph-mon[78798]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:34:12.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:12 smithi170 ceph-mon[78798]: osdmap e75: 12 total, 12 up, 12 in 2024-03-30T02:34:12.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:12 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/858666048' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T02:34:12.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:12 smithi082 ceph-mon[79176]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-30T02:34:12.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:12 smithi082 ceph-mon[79176]: osdmap e75: 12 total, 12 up, 12 in 2024-03-30T02:34:12.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:12 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/858666048' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-30T02:34:13.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.251+0000 7f2edf600700 1 -- 172.21.15.67:0/351464933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ed8100e60 msgr2=0x7f2ed81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:13.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.251+0000 7f2edf600700 1 --2- 172.21.15.67:0/351464933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ed8100e60 0x7f2ed81012e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40099c0 tx=0x7f2ec4009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.251+0000 7f2edf600700 1 -- 172.21.15.67:0/351464933 shutdown_connections 2024-03-30T02:34:13.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.251+0000 7f2edf600700 1 --2- 172.21.15.67:0/351464933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ed8101820 0x7f2ed810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.251+0000 7f2edf600700 1 --2- 172.21.15.67:0/351464933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ed8106ef0 0x7f2ed81072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.251+0000 7f2edf600700 1 --2- 172.21.15.67:0/351464933 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ed8100e60 0x7f2ed81012e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.251+0000 7f2edf600700 1 -- 172.21.15.67:0/351464933 >> 172.21.15.67:0/351464933 conn(0x7f2ed8077fa0 msgr2=0x7f2ed80783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.251+0000 7f2edf600700 1 -- 172.21.15.67:0/351464933 shutdown_connections 2024-03-30T02:34:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.252+0000 7f2edf600700 1 -- 172.21.15.67:0/351464933 wait complete. 2024-03-30T02:34:13.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.252+0000 7f2edf600700 1 Processor -- start 2024-03-30T02:34:13.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edf600700 1 -- start start 2024-03-30T02:34:13.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edf600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ed8100e60 0x7f2ed80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:13.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edf600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ed8101820 0x7f2ed80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:13.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edf600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ed8106ef0 0x7f2ed8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:13.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edf600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed8075070 con 0x7f2ed8101820 2024-03-30T02:34:13.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edf600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81a9b90 con 0x7f2ed8100e60 2024-03-30T02:34:13.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edf600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed81a9d50 con 0x7f2ed8106ef0 2024-03-30T02:34:13.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2ed7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ed8101820 0x7f2ed80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edd600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ed8106ef0 0x7f2ed8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edcc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ed8100e60 0x7f2ed80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edd600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ed8106ef0 0x7f2ed8073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:45654/0 (socket says 172.21.15.67:45654) 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.253+0000 7f2edd600700 1 -- 172.21.15.67:0/993068455 learned_addr learned my addr 172.21.15.67:0/993068455 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2edd600700 1 -- 172.21.15.67:0/993068455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ed8100e60 msgr2=0x7f2ed80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2edd600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ed8100e60 0x7f2ed80730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2edd600700 1 -- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ed8101820 msgr2=0x7f2ed80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2edd600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ed8101820 0x7f2ed80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2edd600700 1 -- 172.21.15.67:0/993068455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40095f0 con 0x7f2ed8106ef0 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2ed7e00700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ed8101820 0x7f2ed80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2edd600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ed8106ef0 0x7f2ed8073b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009ef0 tx=0x7f2ed000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2ed5600700 1 -- 172.21.15.67:0/993068455 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0018070 con 0x7f2ed8106ef0 2024-03-30T02:34:13.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2ed5600700 1 -- 172.21.15.67:0/993068455 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ed000f040 con 0x7f2ed8106ef0 2024-03-30T02:34:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed81a9fd0 con 0x7f2ed8106ef0 2024-03-30T02:34:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.254+0000 7f2ed5600700 1 -- 172.21.15.67:0/993068455 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000cd00 con 0x7f2ed8106ef0 2024-03-30T02:34:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.255+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed81aa440 con 0x7f2ed8106ef0 2024-03-30T02:34:13.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.255+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed81092f0 con 0x7f2ed8106ef0 2024-03-30T02:34:13.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.258+0000 7f2ed5600700 1 -- 172.21.15.67:0/993068455 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f2ed0003a00 con 0x7f2ed8106ef0 2024-03-30T02:34:13.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.259+0000 7f2ed5600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2ec00a11a0 0x7f2ec00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:13.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.260+0000 7f2ed5600700 1 -- 172.21.15.67:0/993068455 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2ed00cbdc0 con 0x7f2ed8106ef0 2024-03-30T02:34:13.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.260+0000 7f2edcc00700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2ec00a11a0 0x7f2ec00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:13.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.260+0000 7f2edcc00700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2ec00a11a0 0x7f2ec00a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009be0 tx=0x7f2ecc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:13.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.264+0000 7f2ed5600700 1 -- 172.21.15.67:0/993068455 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2ed00959e0 con 0x7f2ed8106ef0 2024-03-30T02:34:13.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.476+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ed8074590 con 0x7f2ed8106ef0 2024-03-30T02:34:13.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.478+0000 7f2ed5600700 1 -- 172.21.15.67:0/993068455 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18387 (secure 0 0 0) 0x7f2ed0099000 con 0x7f2ed8106ef0 2024-03-30T02:34:13.479 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:13.479 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:11.078890+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:34:13.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2ec00a11a0 msgr2=0x7f2ec00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2ec00a11a0 0x7f2ec00a3660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009be0 tx=0x7f2ecc009260 comp rx=0 tx=0).stop 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ed8106ef0 msgr2=0x7f2ed8073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ed8106ef0 0x7f2ed8073b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009ef0 tx=0x7f2ed000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 shutdown_connections 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2ec00a11a0 0x7f2ec00a3660 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ed8106ef0 0x7f2ed8073b30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ed8100e60 0x7f2ed80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 --2- 172.21.15.67:0/993068455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ed8101820 0x7f2ed80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.480+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 >> 172.21.15.67:0/993068455 conn(0x7f2ed8077fa0 msgr2=0x7f2ed80ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.481+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 shutdown_connections 2024-03-30T02:34:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:13.481+0000 7f2edf600700 1 -- 172.21.15.67:0/993068455 wait complete. 2024-03-30T02:34:13.845 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T02:34:13.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-03-30T02:34:14.209 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:34:14.358 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:14 smithi067 ceph-mon[37808]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:14.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:14 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/993068455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:14.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:14 smithi170 ceph-mon[78798]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:14.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:14 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/993068455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:14.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:14 smithi082 ceph-mon[79176]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:14.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:14 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/993068455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:15.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.396+0000 7f7051e00700 1 -- 172.21.15.67:0/2106855524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f704c10d7b0 msgr2=0x7f704c10fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.396+0000 7f7051e00700 1 --2- 172.21.15.67:0/2106855524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f704c10d7b0 0x7f704c10fba0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f704400b0a0 tx=0x7f704400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.396+0000 7f7051e00700 1 -- 172.21.15.67:0/2106855524 shutdown_connections 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.396+0000 7f7051e00700 1 --2- 172.21.15.67:0/2106855524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f704c10d7b0 0x7f704c10fba0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.396+0000 7f7051e00700 1 --2- 172.21.15.67:0/2106855524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c072fe0 0x7f704c0733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.396+0000 7f7051e00700 1 --2- 172.21.15.67:0/2106855524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f704c073900 0x7f704c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.396+0000 7f7051e00700 1 -- 172.21.15.67:0/2106855524 >> 172.21.15.67:0/2106855524 conn(0x7f704c077f10 msgr2=0x7f704c078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.396+0000 7f7051e00700 1 -- 172.21.15.67:0/2106855524 shutdown_connections 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.396+0000 7f7051e00700 1 -- 172.21.15.67:0/2106855524 wait complete. 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.397+0000 7f7051e00700 1 Processor -- start 2024-03-30T02:34:15.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.397+0000 7f7051e00700 1 -- start start 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f7051e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c072fe0 0x7f704c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f7051e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f704c073900 0x7f704c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f7051e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f704c10d7b0 0x7f704c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f7051e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c19e420 con 0x7f704c10d7b0 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f7051e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c19e590 con 0x7f704c073900 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f7051e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c19e750 con 0x7f704c072fe0 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f704aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c072fe0 0x7f704c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f704aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c072fe0 0x7f704c199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:45680/0 (socket says 172.21.15.67:45680) 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f704aa00700 1 -- 172.21.15.67:0/1066813241 learned_addr learned my addr 172.21.15.67:0/1066813241 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:15.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f704b400700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f704c10d7b0 0x7f704c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.398+0000 7f7043e00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f704c073900 0x7f704c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f704aa00700 1 -- 172.21.15.67:0/1066813241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f704c073900 msgr2=0x7f704c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f704aa00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f704c073900 0x7f704c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f704aa00700 1 -- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f704c10d7b0 msgr2=0x7f704c19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f704aa00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f704c10d7b0 0x7f704c19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f704aa00700 1 -- 172.21.15.67:0/1066813241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c009520 con 0x7f704c072fe0 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f704aa00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c072fe0 0x7f704c199570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f703c00eb30 tx=0x7f703c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f7050a00700 1 -- 172.21.15.67:0/1066813241 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c009880 con 0x7f704c072fe0 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f7050a00700 1 -- 172.21.15.67:0/1066813241 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f703c0044e0 con 0x7f704c072fe0 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7044009d00 con 0x7f704c072fe0 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f7050a00700 1 -- 172.21.15.67:0/1066813241 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c003d80 con 0x7f704c072fe0 2024-03-30T02:34:15.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.399+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c1a60c0 con 0x7f704c072fe0 2024-03-30T02:34:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.400+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c1089c0 con 0x7f704c072fe0 2024-03-30T02:34:15.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.402+0000 7f7050a00700 1 -- 172.21.15.67:0/1066813241 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f703c003f20 con 0x7f704c072fe0 2024-03-30T02:34:15.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.403+0000 7f7050a00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f70300a1120 0x7f70300a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:15.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.404+0000 7f7050a00700 1 -- 172.21.15.67:0/1066813241 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f703c014070 con 0x7f704c072fe0 2024-03-30T02:34:15.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.404+0000 7f7043e00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f70300a1120 0x7f70300a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:15.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.404+0000 7f7043e00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f70300a1120 0x7f70300a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f704c19aa80 tx=0x7f7034009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:15.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.408+0000 7f7050a00700 1 -- 172.21.15.67:0/1066813241 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f703c0965e0 con 0x7f704c072fe0 2024-03-30T02:34:15.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.622+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f704c19a830 con 0x7f704c072fe0 2024-03-30T02:34:15.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.622+0000 7f7050a00700 1 -- 172.21.15.67:0/1066813241 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f703c099c00 con 0x7f704c072fe0 2024-03-30T02:34:15.623 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:34:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f70300a1120 msgr2=0x7f70300a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f70300a1120 0x7f70300a35e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f704c19aa80 tx=0x7f7034009260 comp rx=0 tx=0).stop 2024-03-30T02:34:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c072fe0 msgr2=0x7f704c199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c072fe0 0x7f704c199570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f703c00eb30 tx=0x7f703c00ee40 comp rx=0 tx=0).stop 2024-03-30T02:34:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 shutdown_connections 2024-03-30T02:34:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f70300a1120 0x7f70300a35e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f704c10d7b0 0x7f704c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c072fe0 0x7f704c199570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 --2- 172.21.15.67:0/1066813241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f704c073900 0x7f704c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:15.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 >> 172.21.15.67:0/1066813241 conn(0x7f704c077f10 msgr2=0x7f704c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:15.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 shutdown_connections 2024-03-30T02:34:15.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:15.625+0000 7f7051e00700 1 -- 172.21.15.67:0/1066813241 wait complete. 2024-03-30T02:34:16.064 INFO:tasks.cephadm:Adding mds.a on smithi067 2024-03-30T02:34:16.064 INFO:tasks.cephadm:Adding mds.d on smithi067 2024-03-30T02:34:16.064 INFO:tasks.cephadm:Adding mds.g on smithi067 2024-03-30T02:34:16.064 INFO:tasks.cephadm:Adding mds.j on smithi067 2024-03-30T02:34:16.064 INFO:tasks.cephadm:Adding mds.b on smithi082 2024-03-30T02:34:16.064 INFO:tasks.cephadm:Adding mds.e on smithi082 2024-03-30T02:34:16.065 INFO:tasks.cephadm:Adding mds.h on smithi082 2024-03-30T02:34:16.065 INFO:tasks.cephadm:Adding mds.k on smithi082 2024-03-30T02:34:16.065 INFO:tasks.cephadm:Adding mds.c on smithi170 2024-03-30T02:34:16.065 INFO:tasks.cephadm:Adding mds.f on smithi170 2024-03-30T02:34:16.065 INFO:tasks.cephadm:Adding mds.i on smithi170 2024-03-30T02:34:16.065 INFO:tasks.cephadm:Adding mds.l on smithi170 2024-03-30T02:34:16.065 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph orch apply mds all '12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l' 2024-03-30T02:34:16.300 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:34:16.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:16 smithi170 ceph-mon[78798]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:16.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:16 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1066813241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:34:16.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:16 smithi067 ceph-mon[37808]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:16.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:16 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1066813241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:34:16.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:16 smithi082 ceph-mon[79176]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:16.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:16 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1066813241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:34:17.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 -- 172.21.15.170:0/4126254419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c01041b0 msgr2=0x7fb7c01065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 --2- 172.21.15.170:0/4126254419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c01041b0 0x7fb7c01065a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009990 tx=0x7fb7b0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 -- 172.21.15.170:0/4126254419 shutdown_connections 2024-03-30T02:34:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 --2- 172.21.15.170:0/4126254419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7c0101880 0x7fb7c0103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 --2- 172.21.15.170:0/4126254419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c0106ae0 0x7fb7c0108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 --2- 172.21.15.170:0/4126254419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c01041b0 0x7fb7c01065a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 -- 172.21.15.170:0/4126254419 >> 172.21.15.170:0/4126254419 conn(0x7fb7c00fb2b0 msgr2=0x7fb7c00fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 -- 172.21.15.170:0/4126254419 shutdown_connections 2024-03-30T02:34:17.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 -- 172.21.15.170:0/4126254419 wait complete. 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.224+0000 7fb7c5172700 1 Processor -- start 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7c5172700 1 -- start start 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7c5172700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c0101880 0x7fb7c0196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7c5172700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c01041b0 0x7fb7c0197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7c5172700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7c0106ae0 0x7fb7c019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7c5172700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c019e8e0 con 0x7fb7c0106ae0 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7c5172700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c019ea20 con 0x7fb7c01041b0 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7c5172700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c019ebe0 con 0x7fb7c0101880 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c0101880 0x7fb7c0196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:17.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c0101880 0x7fb7c0196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54012/0 (socket says 172.21.15.170:54012) 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bed9d700 1 -- 172.21.15.170:0/3660929942 learned_addr learned my addr 172.21.15.170:0/3660929942 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7be59c700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c01041b0 0x7fb7c0197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bf59e700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7c0106ae0 0x7fb7c019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bed9d700 1 -- 172.21.15.170:0/3660929942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c01041b0 msgr2=0x7fb7c0197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bed9d700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c01041b0 0x7fb7c0197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bed9d700 1 -- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7c0106ae0 msgr2=0x7fb7c019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bed9d700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7c0106ae0 0x7fb7c019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bed9d700 1 -- 172.21.15.170:0/3660929942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00095f0 con 0x7fb7c0101880 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bf59e700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7c0106ae0 0x7fb7c019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.225+0000 7fb7bed9d700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c0101880 0x7fb7c0196de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400b530 tx=0x7fb7b400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.226+0000 7fb7a7fff700 1 -- 172.21.15.170:0/3660929942 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4011830 con 0x7fb7c0101880 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.226+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c019eec0 con 0x7fb7c0101880 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.226+0000 7fb7a7fff700 1 -- 172.21.15.170:0/3660929942 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7b4011e50 con 0x7fb7c0101880 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.226+0000 7fb7a7fff700 1 -- 172.21.15.170:0/3660929942 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400f2e0 con 0x7fb7c0101880 2024-03-30T02:34:17.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.226+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c019f2e0 con 0x7fb7c0101880 2024-03-30T02:34:17.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.227+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c00fce30 con 0x7fb7c0101880 2024-03-30T02:34:17.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.228+0000 7fb7a7fff700 1 -- 172.21.15.170:0/3660929942 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fb7b4010380 con 0x7fb7c0101880 2024-03-30T02:34:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.228+0000 7fb7a7fff700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb7a80a1120 0x7fb7a80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.228+0000 7fb7a7fff700 1 -- 172.21.15.170:0/3660929942 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7b40cbf80 con 0x7fb7c0101880 2024-03-30T02:34:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.228+0000 7fb7be59c700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb7a80a1120 0x7fb7a80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:17.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.229+0000 7fb7be59c700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb7a80a1120 0x7fb7a80a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b5c0 tx=0x7fb7b0005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:17.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.232+0000 7fb7a7fff700 1 -- 172.21.15.170:0/3660929942 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb7b40ff9f0 con 0x7fb7c0101880 2024-03-30T02:34:17.434 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.434+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7c002ce20 con 0x7fb7a80a1120 2024-03-30T02:34:17.443 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.442+0000 7fb7a7fff700 1 -- 172.21.15.170:0/3660929942 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fb7c002ce20 con 0x7fb7a80a1120 2024-03-30T02:34:17.443 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mds.all update... 2024-03-30T02:34:17.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.444+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb7a80a1120 msgr2=0x7fb7a80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:17.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.444+0000 7fb7c5172700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb7a80a1120 0x7fb7a80a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b5c0 tx=0x7fb7b0005fd0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c0101880 msgr2=0x7fb7c0196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c0101880 0x7fb7c0196de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400b530 tx=0x7fb7b400b840 comp rx=0 tx=0).stop 2024-03-30T02:34:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 shutdown_connections 2024-03-30T02:34:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb7a80a1120 0x7fb7a80a35e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7c0106ae0 0x7fb7c019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7c0101880 0x7fb7c0196de0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 --2- 172.21.15.170:0/3660929942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c01041b0 0x7fb7c0197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 >> 172.21.15.170:0/3660929942 conn(0x7fb7c00fb2b0 msgr2=0x7fb7c0103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 shutdown_connections 2024-03-30T02:34:17.446 INFO:teuthology.orchestra.run.smithi170.stderr:2024-03-30T02:34:17.445+0000 7fb7c5172700 1 -- 172.21.15.170:0/3660929942 wait complete. 2024-03-30T02:34:17.817 DEBUG:teuthology.orchestra.run.smithi067:mds.a> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.a.service 2024-03-30T02:34:17.821 DEBUG:teuthology.orchestra.run.smithi067:mds.d> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.d.service 2024-03-30T02:34:17.866 DEBUG:teuthology.orchestra.run.smithi067:mds.g> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.g.service 2024-03-30T02:34:17.891 INFO:journalctl@ceph.mds.a.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:34:17.894 DEBUG:teuthology.orchestra.run.smithi067:mds.j> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.j.service 2024-03-30T02:34:17.936 INFO:journalctl@ceph.mds.d.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:34:17.939 DEBUG:teuthology.orchestra.run.smithi082:mds.b> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.b.service 2024-03-30T02:34:17.942 DEBUG:teuthology.orchestra.run.smithi082:mds.e> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.e.service 2024-03-30T02:34:17.961 INFO:journalctl@ceph.mds.g.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:34:17.981 INFO:journalctl@ceph.mds.b.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:34:17.982 DEBUG:teuthology.orchestra.run.smithi082:mds.h> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.h.service 2024-03-30T02:34:18.005 INFO:journalctl@ceph.mds.j.smithi067.stdout:-- Logs begin at Sat 2024-03-30 02:11:46 UTC. -- 2024-03-30T02:34:18.012 INFO:journalctl@ceph.mds.e.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:34:18.013 DEBUG:teuthology.orchestra.run.smithi082:mds.k> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.k.service 2024-03-30T02:34:18.041 INFO:journalctl@ceph.mds.h.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:34:18.042 DEBUG:teuthology.orchestra.run.smithi170:mds.c> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.c.service 2024-03-30T02:34:18.045 DEBUG:teuthology.orchestra.run.smithi170:mds.f> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.f.service 2024-03-30T02:34:18.069 INFO:journalctl@ceph.mds.k.smithi082.stdout:-- Logs begin at Sat 2024-03-30 02:04:59 UTC. -- 2024-03-30T02:34:18.077 INFO:journalctl@ceph.mds.c.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:34:18.078 DEBUG:teuthology.orchestra.run.smithi170:mds.i> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.i.service 2024-03-30T02:34:18.105 INFO:journalctl@ceph.mds.f.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:34:18.106 DEBUG:teuthology.orchestra.run.smithi170:mds.l> sudo journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.l.service 2024-03-30T02:34:18.132 INFO:journalctl@ceph.mds.i.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:34:18.132 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-30T02:34:18.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:34:18.158 INFO:journalctl@ceph.mds.l.smithi170.stdout:-- Logs begin at Sat 2024-03-30 02:04:52 UTC. -- 2024-03-30T02:34:18.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:18 smithi067 ceph-mon[37808]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:18.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:18 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:18.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:18 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:18 smithi082 ceph-mon[79176]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:18 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:18.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:18 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:18.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.357+0000 7f299b400700 1 -- 172.21.15.67:0/88497431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2994074df0 msgr2=0x7f2994073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:18.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.357+0000 7f299b400700 1 --2- 172.21.15.67:0/88497431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2994074df0 0x7f2994073240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2988009990 tx=0x7f2988009ca0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.357+0000 7f299b400700 1 -- 172.21.15.67:0/88497431 shutdown_connections 2024-03-30T02:34:18.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.357+0000 7f299b400700 1 --2- 172.21.15.67:0/88497431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2994108b40 0x7f299410afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.357+0000 7f299b400700 1 --2- 172.21.15.67:0/88497431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2994074df0 0x7f2994073240 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.357+0000 7f299b400700 1 --2- 172.21.15.67:0/88497431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2994073810 0x7f2994073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.357+0000 7f299b400700 1 -- 172.21.15.67:0/88497431 >> 172.21.15.67:0/88497431 conn(0x7f29940fc1f0 msgr2=0x7f29940fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:18.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.357+0000 7f299b400700 1 -- 172.21.15.67:0/88497431 shutdown_connections 2024-03-30T02:34:18.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.357+0000 7f299b400700 1 -- 172.21.15.67:0/88497431 wait complete. 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.358+0000 7f299b400700 1 Processor -- start 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.358+0000 7f299b400700 1 -- start start 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.358+0000 7f299b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2994073810 0x7f2994198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.358+0000 7f299b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2994074df0 0x7f29941993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.358+0000 7f299b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2994108b40 0x7f29941a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.358+0000 7f299b400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29941a0990 con 0x7f2994073810 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.358+0000 7f299b400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29941a0ad0 con 0x7f2994074df0 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.358+0000 7f299b400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29941a0c60 con 0x7f2994108b40 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2998a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2994073810 0x7f2994198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2999400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2994108b40 0x7f29941a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:18.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2999400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2994108b40 0x7f29941a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:46590/0 (socket says 172.21.15.67:46590) 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2999400700 1 -- 172.21.15.67:0/2761048101 learned_addr learned my addr 172.21.15.67:0/2761048101 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2993e00700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2994074df0 0x7f29941993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2993e00700 1 -- 172.21.15.67:0/2761048101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2994108b40 msgr2=0x7f29941a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2993e00700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2994108b40 0x7f29941a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2993e00700 1 -- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2994073810 msgr2=0x7f2994198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2993e00700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2994073810 0x7f2994198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2993e00700 1 -- 172.21.15.67:0/2761048101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980009520 con 0x7f2994074df0 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2999400700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2994108b40 0x7f29941a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2998a00700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2994073810 0x7f2994198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.359+0000 7f2993e00700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2994074df0 0x7f29941993a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f298000e9c0 tx=0x7f298000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.360+0000 7f2991600700 1 -- 172.21.15.67:0/2761048101 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298000cb70 con 0x7f2994074df0 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.360+0000 7f2991600700 1 -- 172.21.15.67:0/2761048101 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f298000ee60 con 0x7f2994074df0 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.360+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29880095f0 con 0x7f2994074df0 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.360+0000 7f2991600700 1 -- 172.21.15.67:0/2761048101 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980018680 con 0x7f2994074df0 2024-03-30T02:34:18.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.360+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29941a1280 con 0x7f2994074df0 2024-03-30T02:34:18.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.361+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2994067a80 con 0x7f2994074df0 2024-03-30T02:34:18.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.362+0000 7f2991600700 1 -- 172.21.15.67:0/2761048101 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f2980010430 con 0x7f2994074df0 2024-03-30T02:34:18.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.364+0000 7f2991600700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f297c0a11a0 0x7f297c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:18.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.364+0000 7f2991600700 1 -- 172.21.15.67:0/2761048101 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2980014070 con 0x7f2994074df0 2024-03-30T02:34:18.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.364+0000 7f2998a00700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f297c0a11a0 0x7f297c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:18.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.364+0000 7f2998a00700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f297c0a11a0 0x7f297c0a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f298800b2e0 tx=0x7f2988005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:18.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.368+0000 7f2991600700 1 -- 172.21.15.67:0/2761048101 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2994067a80 con 0x7f2994074df0 2024-03-30T02:34:18.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:18 smithi170 ceph-mon[78798]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:18.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:18 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:18.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:18 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.614+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2994107900 con 0x7f2994074df0 2024-03-30T02:34:18.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.616+0000 7f2991600700 1 -- 172.21.15.67:0/2761048101 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18387 (secure 0 0 0) 0x7f2980096390 con 0x7f2994074df0 2024-03-30T02:34:18.617 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:18.617 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:11.078890+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:34:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f297c0a11a0 msgr2=0x7f297c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f297c0a11a0 0x7f297c0a3660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f298800b2e0 tx=0x7f2988005fb0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2994074df0 msgr2=0x7f29941993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:18.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2994074df0 0x7f29941993a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f298000e9c0 tx=0x7f298000ecd0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 shutdown_connections 2024-03-30T02:34:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f297c0a11a0 0x7f297c0a3660 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2994073810 0x7f2994198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2994108b40 0x7f29941a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 --2- 172.21.15.67:0/2761048101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2994074df0 0x7f29941993a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 >> 172.21.15.67:0/2761048101 conn(0x7f29940fc1f0 msgr2=0x7f2994104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 shutdown_connections 2024-03-30T02:34:18.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.618+0000 7f299b400700 1 -- 172.21.15.67:0/2761048101 wait complete. 2024-03-30T02:34:18.639 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T02:34:18.640 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:34:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.879+0000 7f02a3000700 1 -- 172.21.15.67:0/1631950046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f029c101db0 msgr2=0x7f029c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.879+0000 7f02a3000700 1 --2- 172.21.15.67:0/1631950046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f029c101db0 0x7f029c102210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f02880099c0 tx=0x7f0288009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.879+0000 7f02a3000700 1 -- 172.21.15.67:0/1631950046 shutdown_connections 2024-03-30T02:34:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.879+0000 7f02a3000700 1 --2- 172.21.15.67:0/1631950046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f029c102750 0x7f029c109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.879+0000 7f02a3000700 1 --2- 172.21.15.67:0/1631950046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029c100bc0 0x7f029c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.879+0000 7f02a3000700 1 --2- 172.21.15.67:0/1631950046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f029c101db0 0x7f029c102210 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.879+0000 7f02a3000700 1 -- 172.21.15.67:0/1631950046 >> 172.21.15.67:0/1631950046 conn(0x7f029c0fc110 msgr2=0x7f029c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.879+0000 7f02a3000700 1 -- 172.21.15.67:0/1631950046 shutdown_connections 2024-03-30T02:34:18.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.879+0000 7f02a3000700 1 -- 172.21.15.67:0/1631950046 wait complete. 2024-03-30T02:34:18.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.881+0000 7f02a3000700 1 Processor -- start 2024-03-30T02:34:18.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.881+0000 7f02a3000700 1 -- start start 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.881+0000 7f02a3000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029c100bc0 0x7f029c196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.881+0000 7f02a3000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f029c101db0 0x7f029c1970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.881+0000 7f02a3000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f029c102750 0x7f029c19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.881+0000 7f02a3000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19e6c0 con 0x7f029c101db0 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.881+0000 7f02a3000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19e800 con 0x7f029c102750 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.881+0000 7f02a3000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19e990 con 0x7f029c100bc0 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029c100bc0 0x7f029c196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f029c101db0 0x7f029c1970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029c100bc0 0x7f029c196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:46610/0 (socket says 172.21.15.67:46610) 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1c00700 1 -- 172.21.15.67:0/101389866 learned_addr learned my addr 172.21.15.67:0/101389866 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a2600700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f029c102750 0x7f029c19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:18.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1200700 1 -- 172.21.15.67:0/101389866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029c100bc0 msgr2=0x7f029c196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1200700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029c100bc0 0x7f029c196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1200700 1 -- 172.21.15.67:0/101389866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f029c102750 msgr2=0x7f029c19e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1200700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f029c102750 0x7f029c19e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1200700 1 -- 172.21.15.67:0/101389866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880095f0 con 0x7f029c101db0 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a2600700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f029c102750 0x7f029c19e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.882+0000 7f02a1200700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f029c101db0 0x7f029c1970d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f02880098e0 tx=0x7f0288009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.883+0000 7f029a000700 1 -- 172.21.15.67:0/101389866 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028801c070 con 0x7f029c101db0 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.883+0000 7f029a000700 1 -- 172.21.15.67:0/101389866 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028800fdc0 con 0x7f029c101db0 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.883+0000 7f029a000700 1 -- 172.21.15.67:0/101389866 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288022ea0 con 0x7f029c101db0 2024-03-30T02:34:18.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.883+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c19ec10 con 0x7f029c101db0 2024-03-30T02:34:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.883+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c19efe0 con 0x7f029c101db0 2024-03-30T02:34:18.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.884+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029c108040 con 0x7f029c101db0 2024-03-30T02:34:18.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.886+0000 7f029a000700 1 -- 172.21.15.67:0/101389866 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0288020070 con 0x7f029c101db0 2024-03-30T02:34:18.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.887+0000 7f029a000700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f02840a11a0 0x7f02840a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:18.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.887+0000 7f029a000700 1 -- 172.21.15.67:0/101389866 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f02880cd420 con 0x7f029c101db0 2024-03-30T02:34:18.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.887+0000 7f02a1c00700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f02840a11a0 0x7f02840a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:18.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.888+0000 7f02a1c00700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f02840a11a0 0x7f02840a3660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0290005950 tx=0x7f0290009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:18.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:18.890+0000 7f029a000700 1 -- 172.21.15.67:0/101389866 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0288097040 con 0x7f029c101db0 2024-03-30T02:34:19.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.112+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f029c0679a0 con 0x7f029c101db0 2024-03-30T02:34:19.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.112+0000 7f029a000700 1 -- 172.21.15.67:0/101389866 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f028809a660 con 0x7f029c101db0 2024-03-30T02:34:19.114 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:34:19.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f02840a11a0 msgr2=0x7f02840a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:19.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f02840a11a0 0x7f02840a3660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0290005950 tx=0x7f0290009260 comp rx=0 tx=0).stop 2024-03-30T02:34:19.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f029c101db0 msgr2=0x7f029c1970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:19.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f029c101db0 0x7f029c1970d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f02880098e0 tx=0x7f0288009580 comp rx=0 tx=0).stop 2024-03-30T02:34:19.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 shutdown_connections 2024-03-30T02:34:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f02840a11a0 0x7f02840a3660 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f029c102750 0x7f029c19e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f029c100bc0 0x7f029c196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 --2- 172.21.15.67:0/101389866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f029c101db0 0x7f029c1970d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 >> 172.21.15.67:0/101389866 conn(0x7f029c0fc110 msgr2=0x7f029c105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 shutdown_connections 2024-03-30T02:34:19.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.115+0000 7f02a3000700 1 -- 172.21.15.67:0/101389866 wait complete. 2024-03-30T02:34:19.127 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mon[78798]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:34:19.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mon[78798]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-03-30T02:34:19.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:19.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:19.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:19.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:19.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:19.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:19.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2761048101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:19.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:19.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.334+0000 7f637d800700 1 -- 172.21.15.67:0/2229018936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 msgr2=0x7f6378103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:19.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.334+0000 7f637d800700 1 --2- 172.21.15.67:0/2229018936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 0x7f6378103b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6368009990 tx=0x7f6368009ca0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.334+0000 7f637d800700 1 -- 172.21.15.67:0/2229018936 shutdown_connections 2024-03-30T02:34:19.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.334+0000 7f637d800700 1 --2- 172.21.15.67:0/2229018936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63781069c0 0x7f6378108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.334+0000 7f637d800700 1 --2- 172.21.15.67:0/2229018936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 0x7f6378103b50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.334+0000 7f637d800700 1 --2- 172.21.15.67:0/2229018936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6378104090 0x7f6378106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.334+0000 7f637d800700 1 -- 172.21.15.67:0/2229018936 >> 172.21.15.67:0/2229018936 conn(0x7f63780fb0d0 msgr2=0x7f63780fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:19.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.334+0000 7f637d800700 1 -- 172.21.15.67:0/2229018936 shutdown_connections 2024-03-30T02:34:19.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.334+0000 7f637d800700 1 -- 172.21.15.67:0/2229018936 wait complete. 2024-03-30T02:34:19.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.336+0000 7f637d800700 1 Processor -- start 2024-03-30T02:34:19.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.336+0000 7f637d800700 1 -- start start 2024-03-30T02:34:19.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.336+0000 7f637d800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 0x7f6378198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:19.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.336+0000 7f637d800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6378104090 0x7f6378199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:19.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.336+0000 7f637d800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63781069c0 0x7f63781a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:19.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.336+0000 7f637d800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63781a08e0 con 0x7f63781069c0 2024-03-30T02:34:19.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.336+0000 7f637d800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63781a0a50 con 0x7f6378104090 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.336+0000 7f637d800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63781a0c10 con 0x7f6378101760 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.336+0000 7f6376a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 0x7f6378198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6376a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 0x7f6378198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:46626/0 (socket says 172.21.15.67:46626) 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6376a00700 1 -- 172.21.15.67:0/2887206853 learned_addr learned my addr 172.21.15.67:0/2887206853 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6377400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63781069c0 0x7f63781a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6376000700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6378104090 0x7f6378199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6377400700 1 -- 172.21.15.67:0/2887206853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 msgr2=0x7f6378198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6377400700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 0x7f6378198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6377400700 1 -- 172.21.15.67:0/2887206853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6378104090 msgr2=0x7f6378199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6377400700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6378104090 0x7f6378199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6377400700 1 -- 172.21.15.67:0/2887206853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680095f0 con 0x7f63781069c0 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6376a00700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 0x7f6378198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:19.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f6377400700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63781069c0 0x7f63781a0310 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f637000efc0 tx=0x7f637000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f636f400700 1 -- 172.21.15.67:0/2887206853 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370018070 con 0x7f63781069c0 2024-03-30T02:34:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f636f400700 1 -- 172.21.15.67:0/2887206853 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f637000f040 con 0x7f63781069c0 2024-03-30T02:34:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63781a0e90 con 0x7f63781069c0 2024-03-30T02:34:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.337+0000 7f636f400700 1 -- 172.21.15.67:0/2887206853 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000cd40 con 0x7f63781069c0 2024-03-30T02:34:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.338+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63781a1310 con 0x7f63781069c0 2024-03-30T02:34:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.338+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6378108540 con 0x7f63781069c0 2024-03-30T02:34:19.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.341+0000 7f636f400700 1 -- 172.21.15.67:0/2887206853 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f6370003a00 con 0x7f63781069c0 2024-03-30T02:34:19.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.342+0000 7f636f400700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f635c0a11a0 0x7f635c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:19.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.342+0000 7f636f400700 1 -- 172.21.15.67:0/2887206853 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f63700cc9a0 con 0x7f63781069c0 2024-03-30T02:34:19.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.342+0000 7f6376a00700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f635c0a11a0 0x7f635c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:19.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.343+0000 7f6376a00700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f635c0a11a0 0x7f635c0a3660 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f636800b2e0 tx=0x7f636800b750 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:19.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.345+0000 7f636f400700 1 -- 172.21.15.67:0/2887206853 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f63700965c0 con 0x7f63781069c0 2024-03-30T02:34:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:19 smithi067 ceph-mon[37808]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:34:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:19 smithi067 ceph-mon[37808]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-03-30T02:34:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:19.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:19.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:19.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:19 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:19.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:19 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2761048101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:19.440 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:34:19 smithi170 systemd[1]: Starting Ceph mds.l for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:19 smithi082 ceph-mon[79176]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:34:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:19 smithi082 ceph-mon[79176]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-03-30T02:34:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:19 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:19.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:19 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2761048101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:19.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.612+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6378067a10 con 0x7f63781069c0 2024-03-30T02:34:19.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.613+0000 7f636f400700 1 -- 172.21.15.67:0/2887206853 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f6370099be0 con 0x7f63781069c0 2024-03-30T02:34:19.614 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:19.614 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-30T02:34:19.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.615+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f635c0a11a0 msgr2=0x7f635c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:19.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.615+0000 7f637d800700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f635c0a11a0 0x7f635c0a3660 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f636800b2e0 tx=0x7f636800b750 comp rx=0 tx=0).stop 2024-03-30T02:34:19.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.615+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63781069c0 msgr2=0x7f63781a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:19.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.615+0000 7f637d800700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63781069c0 0x7f63781a0310 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f637000efc0 tx=0x7f637000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.616+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 shutdown_connections 2024-03-30T02:34:19.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.616+0000 7f637d800700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f635c0a11a0 0x7f635c0a3660 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.616+0000 7f637d800700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63781069c0 0x7f63781a0310 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.616+0000 7f637d800700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6378101760 0x7f6378198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.616+0000 7f637d800700 1 --2- 172.21.15.67:0/2887206853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6378104090 0x7f6378199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:19.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.616+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 >> 172.21.15.67:0/2887206853 conn(0x7f63780fb0d0 msgr2=0x7f6378100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:19.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.616+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 shutdown_connections 2024-03-30T02:34:19.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:19.616+0000 7f637d800700 1 -- 172.21.15.67:0/2887206853 wait complete. 2024-03-30T02:34:19.618 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 1 2024-03-30T02:34:20.103 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-mds[116103]: -- 172.21.15.170:0/2313992150 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5587f12761a0 con 0x5587f124d000 2024-03-30T02:34:20.104 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:34:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-l[116080]: starting mds.l at 2024-03-30T02:34:20.104 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:34:19 smithi170 bash[116065]: 5c5992bdfbf50e99e15fc48961f05476bbad9e32b1a53ee35d4dd4f375e1ed0d 2024-03-30T02:34:20.104 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:34:19 smithi170 systemd[1]: Started Ceph mds.l for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: Deploying daemon mds.l on smithi170 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/101389866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2887206853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6832/3749225376,v1:172.21.15.170:6833/3749225376] up:boot 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: fsmap 1 up:standby 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:20.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:20.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:20 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: Deploying daemon mds.l on smithi170 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/101389866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2887206853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6832/3749225376,v1:172.21.15.170:6833/3749225376] up:boot 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: fsmap 1 up:standby 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:20 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: Deploying daemon mds.l on smithi170 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/101389866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2887206853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6832/3749225376,v1:172.21.15.170:6833/3749225376] up:boot 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: fsmap 1 up:standby 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:20.441 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:20.442 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:20.442 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:20.442 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:20.442 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:20 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:21.107 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:34:21 smithi082 systemd[1]: Starting Ceph mds.k for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:21.381 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:21 smithi082 ceph-mon[79176]: Deploying daemon mds.k on smithi082 2024-03-30T02:34:21.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:21 smithi067 ceph-mon[37808]: Deploying daemon mds.k on smithi082 2024-03-30T02:34:21.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:21 smithi170 ceph-mon[78798]: Deploying daemon mds.k on smithi082 2024-03-30T02:34:21.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:21.750 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:34:21 smithi082 ceph-mds[114840]: -- 172.21.15.82:0/879179162 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5620ae1b21a0 con 0x5620ae188c00 2024-03-30T02:34:21.750 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:34:21 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-k[114817]: starting mds.k at 2024-03-30T02:34:21.750 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:34:21 smithi082 bash[114802]: 8455de682c8205101d23873b80743d4b7fbcc06b6c582b2474c147dbebf0b44a 2024-03-30T02:34:21.750 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:34:21 smithi082 systemd[1]: Started Ceph mds.k for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.839+0000 7fc007600700 1 -- 172.21.15.67:0/4077331358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 msgr2=0x7fc000073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:21.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.839+0000 7fc007600700 1 --2- 172.21.15.67:0/4077331358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 0x7fc000073c90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0098f0 tx=0x7fbfec009c00 comp rx=0 tx=0).stop 2024-03-30T02:34:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.839+0000 7fc007600700 1 -- 172.21.15.67:0/4077331358 shutdown_connections 2024-03-30T02:34:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.839+0000 7fc007600700 1 --2- 172.21.15.67:0/4077331358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc000108b40 0x7fc00010afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.839+0000 7fc007600700 1 --2- 172.21.15.67:0/4077331358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc000074df0 0x7fc000073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.839+0000 7fc007600700 1 --2- 172.21.15.67:0/4077331358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 0x7fc000073c90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.839+0000 7fc007600700 1 -- 172.21.15.67:0/4077331358 >> 172.21.15.67:0/4077331358 conn(0x7fc0000fc1f0 msgr2=0x7fc0000fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.839+0000 7fc007600700 1 -- 172.21.15.67:0/4077331358 shutdown_connections 2024-03-30T02:34:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.839+0000 7fc007600700 1 -- 172.21.15.67:0/4077331358 wait complete. 2024-03-30T02:34:21.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc007600700 1 Processor -- start 2024-03-30T02:34:21.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc007600700 1 -- start start 2024-03-30T02:34:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc007600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 0x7fc000198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc007600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc000074df0 0x7fc000199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:21.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc007600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc000108b40 0x7fc0001a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc007600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001a09f0 con 0x7fc000073810 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc007600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001a0b30 con 0x7fc000074df0 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc007600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001a0cc0 con 0x7fc000108b40 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc004c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 0x7fc000198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc004c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 0x7fc000198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52474/0 (socket says 172.21.15.67:52474) 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.841+0000 7fc004c00700 1 -- 172.21.15.67:0/3081813167 learned_addr learned my addr 172.21.15.67:0/3081813167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fbfffe00700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc000074df0 0x7fc000199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fc005600700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc000108b40 0x7fc0001a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fc004c00700 1 -- 172.21.15.67:0/3081813167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc000108b40 msgr2=0x7fc0001a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fc004c00700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc000108b40 0x7fc0001a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fc004c00700 1 -- 172.21.15.67:0/3081813167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc000074df0 msgr2=0x7fc000199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fc004c00700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc000074df0 0x7fc000199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fc004c00700 1 -- 172.21.15.67:0/3081813167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff8009c40 con 0x7fc000073810 2024-03-30T02:34:21.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fbfffe00700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc000074df0 0x7fc000199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fc004c00700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 0x7fc000198ec0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbff400e9c0 tx=0x7fbff400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fbffd600700 1 -- 172.21.15.67:0/3081813167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4004a30 con 0x7fc000073810 2024-03-30T02:34:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec009520 con 0x7fc000073810 2024-03-30T02:34:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fbffd600700 1 -- 172.21.15.67:0/3081813167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbff400cc80 con 0x7fc000073810 2024-03-30T02:34:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.842+0000 7fbffd600700 1 -- 172.21.15.67:0/3081813167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff40054b0 con 0x7fc000073810 2024-03-30T02:34:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.843+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0001a1230 con 0x7fc000073810 2024-03-30T02:34:21.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.844+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc000107900 con 0x7fc000073810 2024-03-30T02:34:21.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.846+0000 7fbffd600700 1 -- 172.21.15.67:0/3081813167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbff4005610 con 0x7fc000073810 2024-03-30T02:34:21.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.847+0000 7fbffd600700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfe80a1120 0x7fbfe80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:21.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.847+0000 7fbffd600700 1 -- 172.21.15.67:0/3081813167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbff4014070 con 0x7fc000073810 2024-03-30T02:34:21.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.847+0000 7fbfffe00700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfe80a1120 0x7fbfe80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:21.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.848+0000 7fbfffe00700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfe80a1120 0x7fbfe80a35e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0098c0 tx=0x7fbfec019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:21.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:21.851+0000 7fbffd600700 1 -- 172.21.15.67:0/3081813167 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbff4096130 con 0x7fc000073810 2024-03-30T02:34:22.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.141+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc000067a10 con 0x7fc000073810 2024-03-30T02:34:22.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.141+0000 7fbffd600700 1 -- 172.21.15.67:0/3081813167 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7fbff4099750 con 0x7fc000073810 2024-03-30T02:34:22.143 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:22.143 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-30T02:34:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.144+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfe80a1120 msgr2=0x7fbfe80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.144+0000 7fc007600700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfe80a1120 0x7fbfe80a35e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0098c0 tx=0x7fbfec019040 comp rx=0 tx=0).stop 2024-03-30T02:34:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.144+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 msgr2=0x7fc000198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.144+0000 7fc007600700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 0x7fc000198ec0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbff400e9c0 tx=0x7fbff400ecd0 comp rx=0 tx=0).stop 2024-03-30T02:34:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.145+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 shutdown_connections 2024-03-30T02:34:22.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.145+0000 7fc007600700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfe80a1120 0x7fbfe80a35e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.145+0000 7fc007600700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc000073810 0x7fc000198ec0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.145+0000 7fc007600700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc000108b40 0x7fc0001a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.145+0000 7fc007600700 1 --2- 172.21.15.67:0/3081813167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc000074df0 0x7fc000199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.145+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 >> 172.21.15.67:0/3081813167 conn(0x7fc0000fc1f0 msgr2=0x7fc000104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.145+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 shutdown_connections 2024-03-30T02:34:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:22.145+0000 7fc007600700 1 -- 172.21.15.67:0/3081813167 wait complete. 2024-03-30T02:34:22.147 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2024-03-30T02:34:22.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:22 smithi067 ceph-mon[37808]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:22.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:22 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:22 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:22 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:22 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:22 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:22 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:22 smithi067 ceph-mon[37808]: Deploying daemon mds.j on smithi067 2024-03-30T02:34:22.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:22 smithi170 ceph-mon[78798]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:22 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:22 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:22 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:22 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:22 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:22 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:22.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:22 smithi170 ceph-mon[78798]: Deploying daemon mds.j on smithi067 2024-03-30T02:34:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:22 smithi082 ceph-mon[79176]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:22 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:22 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:22 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:22 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:22 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:22 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:22 smithi082 ceph-mon[79176]: Deploying daemon mds.j on smithi067 2024-03-30T02:34:23.256 INFO:journalctl@ceph.mds.j.smithi067.stdout:Mar 30 02:34:23 smithi067 systemd[1]: Starting Ceph mds.j for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:23.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:23 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6832/1365991767,v1:172.21.15.82:6833/1365991767] up:boot 2024-03-30T02:34:23.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:23 smithi067 ceph-mon[37808]: fsmap 2 up:standby 2024-03-30T02:34:23.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:23 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T02:34:23.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:23 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3081813167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:23.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:23 smithi067 ceph-mon[37808]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:23 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6832/1365991767,v1:172.21.15.82:6833/1365991767] up:boot 2024-03-30T02:34:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:23 smithi170 ceph-mon[78798]: fsmap 2 up:standby 2024-03-30T02:34:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:23 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T02:34:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:23 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3081813167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:23 smithi170 ceph-mon[78798]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:23.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:23 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6832/1365991767,v1:172.21.15.82:6833/1365991767] up:boot 2024-03-30T02:34:23.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:23 smithi082 ceph-mon[79176]: fsmap 2 up:standby 2024-03-30T02:34:23.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:23 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T02:34:23.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:23 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3081813167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:23.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:23 smithi082 ceph-mon[79176]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:24.163 INFO:journalctl@ceph.mds.j.smithi067.stdout:Mar 30 02:34:23 smithi067 bash[86367]: 5208eb05e573ea3d45fa13b4e6723d59a76859f58064be2699dcd0dc5b12e644 2024-03-30T02:34:24.163 INFO:journalctl@ceph.mds.j.smithi067.stdout:Mar 30 02:34:23 smithi067 systemd[1]: Started Ceph mds.j for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:24.163 INFO:journalctl@ceph.mds.j.smithi067.stdout:Mar 30 02:34:23 smithi067 ceph-mds[86405]: -- 172.21.15.67:0/4243088556 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55af05d761a0 con 0x55af05d4f000 2024-03-30T02:34:24.163 INFO:journalctl@ceph.mds.j.smithi067.stdout:Mar 30 02:34:23 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-j[86382]: starting mds.j at 2024-03-30T02:34:24.167 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:24.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.391+0000 7f35efc00700 1 -- 172.21.15.67:0/1151811590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8101e70 msgr2=0x7f35e81022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:24.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.391+0000 7f35efc00700 1 --2- 172.21.15.67:0/1151811590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8101e70 0x7f35e81022d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f35d40099c0 tx=0x7f35d4009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.391+0000 7f35efc00700 1 -- 172.21.15.67:0/1151811590 shutdown_connections 2024-03-30T02:34:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.391+0000 7f35efc00700 1 --2- 172.21.15.67:0/1151811590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e8102810 0x7f35e81090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.391+0000 7f35efc00700 1 --2- 172.21.15.67:0/1151811590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e8100c80 0x7f35e81010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.391+0000 7f35efc00700 1 --2- 172.21.15.67:0/1151811590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8101e70 0x7f35e81022d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.391+0000 7f35efc00700 1 -- 172.21.15.67:0/1151811590 >> 172.21.15.67:0/1151811590 conn(0x7f35e80fc1f0 msgr2=0x7f35e80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.391+0000 7f35efc00700 1 -- 172.21.15.67:0/1151811590 shutdown_connections 2024-03-30T02:34:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.391+0000 7f35efc00700 1 -- 172.21.15.67:0/1151811590 wait complete. 2024-03-30T02:34:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.393+0000 7f35efc00700 1 Processor -- start 2024-03-30T02:34:24.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.393+0000 7f35efc00700 1 -- start start 2024-03-30T02:34:24.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.393+0000 7f35efc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e8100c80 0x7f35e8196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.393+0000 7f35efc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e8101e70 0x7f35e8197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.393+0000 7f35efc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8102810 0x7f35e819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.393+0000 7f35efc00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e819e720 con 0x7f35e8102810 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.393+0000 7f35efc00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e819e860 con 0x7f35e8101e70 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.393+0000 7f35efc00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e819e9f0 con 0x7f35e8100c80 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35edc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8102810 0x7f35e819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35edc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8102810 0x7f35e819e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52512/0 (socket says 172.21.15.67:52512) 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35edc00700 1 -- 172.21.15.67:0/2273568583 learned_addr learned my addr 172.21.15.67:0/2273568583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35ed200700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e8100c80 0x7f35e8196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35e7e00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e8101e70 0x7f35e8197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35edc00700 1 -- 172.21.15.67:0/2273568583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e8100c80 msgr2=0x7f35e8196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:24.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35edc00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e8100c80 0x7f35e8196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35edc00700 1 -- 172.21.15.67:0/2273568583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e8101e70 msgr2=0x7f35e8197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35edc00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e8101e70 0x7f35e8197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35edc00700 1 -- 172.21.15.67:0/2273568583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40095f0 con 0x7f35e8102810 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35edc00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8102810 0x7f35e819e150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35e0007ee0 tx=0x7f35e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.394+0000 7f35e5600700 1 -- 172.21.15.67:0/2273568583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0017070 con 0x7f35e8102810 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.395+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e819ecd0 con 0x7f35e8102810 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.395+0000 7f35e5600700 1 -- 172.21.15.67:0/2273568583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e000f040 con 0x7f35e8102810 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.395+0000 7f35e5600700 1 -- 172.21.15.67:0/2273568583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0005580 con 0x7f35e8102810 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.395+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e819f190 con 0x7f35e8102810 2024-03-30T02:34:24.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.396+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e8108100 con 0x7f35e8102810 2024-03-30T02:34:24.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.398+0000 7f35e5600700 1 -- 172.21.15.67:0/2273568583 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f35e000db20 con 0x7f35e8102810 2024-03-30T02:34:24.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.399+0000 7f35e5600700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35d00a1120 0x7f35d00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:24.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.399+0000 7f35e5600700 1 -- 172.21.15.67:0/2273568583 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f35e00cc230 con 0x7f35e8102810 2024-03-30T02:34:24.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.400+0000 7f35ed200700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35d00a1120 0x7f35d00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:24.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.400+0000 7f35ed200700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35d00a1120 0x7f35d00a35e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35e8101cd0 tx=0x7f35dc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:24.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.406+0000 7f35e5600700 1 -- 172.21.15.67:0/2273568583 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f35e0095dc0 con 0x7f35e8102810 2024-03-30T02:34:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.692+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35e8067a10 con 0x7f35e8102810 2024-03-30T02:34:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.692+0000 7f35e5600700 1 -- 172.21.15.67:0/2273568583 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7f35e00993e0 con 0x7f35e8102810 2024-03-30T02:34:24.694 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:24.694 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-30T02:34:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35d00a1120 msgr2=0x7f35d00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35d00a1120 0x7f35d00a35e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35e8101cd0 tx=0x7f35dc009260 comp rx=0 tx=0).stop 2024-03-30T02:34:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8102810 msgr2=0x7f35e819e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8102810 0x7f35e819e150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35e0007ee0 tx=0x7f35e000d3b0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 shutdown_connections 2024-03-30T02:34:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35d00a1120 0x7f35d00a35e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e8102810 0x7f35e819e150 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e8100c80 0x7f35e8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 --2- 172.21.15.67:0/2273568583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e8101e70 0x7f35e8197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 >> 172.21.15.67:0/2273568583 conn(0x7f35e80fc1f0 msgr2=0x7f35e8105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 shutdown_connections 2024-03-30T02:34:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:24.695+0000 7f35efc00700 1 -- 172.21.15.67:0/2273568583 wait complete. 2024-03-30T02:34:24.697 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 4 2024-03-30T02:34:25.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:25.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:25.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:25.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:25.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:25.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:25.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: Deploying daemon mds.d on smithi067 2024-03-30T02:34:25.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.67:6834/2327446464,v1:172.21.15.67:6835/2327446464] up:boot 2024-03-30T02:34:25.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: fsmap 3 up:standby 2024-03-30T02:34:25.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-30T02:34:25.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:24 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2273568583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: Deploying daemon mds.d on smithi067 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.67:6834/2327446464,v1:172.21.15.67:6835/2327446464] up:boot 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: fsmap 3 up:standby 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-30T02:34:25.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:24 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2273568583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:25.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:25.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:25.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:25.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:25.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:25.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:25.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: Deploying daemon mds.d on smithi067 2024-03-30T02:34:25.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.67:6834/2327446464,v1:172.21.15.67:6835/2327446464] up:boot 2024-03-30T02:34:25.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: fsmap 3 up:standby 2024-03-30T02:34:25.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-30T02:34:25.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:24 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2273568583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:25.849 INFO:journalctl@ceph.mds.d.smithi067.stdout:Mar 30 02:34:25 smithi067 systemd[1]: Starting Ceph mds.d for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:26.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:25 smithi067 ceph-mon[37808]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:26.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:25 smithi170 ceph-mon[78798]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:26.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:25 smithi082 ceph-mon[79176]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:26.663 INFO:journalctl@ceph.mds.d.smithi067.stdout:Mar 30 02:34:26 smithi067 ceph-mds[86791]: -- 172.21.15.67:0/564637882 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a04aa221a0 con 0x55a04a9fac00 2024-03-30T02:34:26.663 INFO:journalctl@ceph.mds.d.smithi067.stdout:Mar 30 02:34:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-d[86768]: starting mds.d at 2024-03-30T02:34:26.664 INFO:journalctl@ceph.mds.d.smithi067.stdout:Mar 30 02:34:26 smithi067 bash[86753]: 5a961c55d2b29ba09760ea342d6216d201dcb7a676f0db3f6c229d9504e8429f 2024-03-30T02:34:26.664 INFO:journalctl@ceph.mds.d.smithi067.stdout:Mar 30 02:34:26 smithi067 systemd[1]: Started Ceph mds.d for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:26.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:26.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.920+0000 7fe929800700 1 -- 172.21.15.67:0/669920778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe924101e70 msgr2=0x7fe9241022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:26.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.920+0000 7fe929800700 1 --2- 172.21.15.67:0/669920778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe924101e70 0x7fe9241022d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe9100099c0 tx=0x7fe910009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:26.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.920+0000 7fe929800700 1 -- 172.21.15.67:0/669920778 shutdown_connections 2024-03-30T02:34:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.920+0000 7fe929800700 1 --2- 172.21.15.67:0/669920778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe924102810 0x7fe9241090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.920+0000 7fe929800700 1 --2- 172.21.15.67:0/669920778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe924100c80 0x7fe9241010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.920+0000 7fe929800700 1 --2- 172.21.15.67:0/669920778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe924101e70 0x7fe9241022d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.920+0000 7fe929800700 1 -- 172.21.15.67:0/669920778 >> 172.21.15.67:0/669920778 conn(0x7fe9240fc1f0 msgr2=0x7fe9240fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.920+0000 7fe929800700 1 -- 172.21.15.67:0/669920778 shutdown_connections 2024-03-30T02:34:26.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.920+0000 7fe929800700 1 -- 172.21.15.67:0/669920778 wait complete. 2024-03-30T02:34:26.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.921+0000 7fe929800700 1 Processor -- start 2024-03-30T02:34:26.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.921+0000 7fe929800700 1 -- start start 2024-03-30T02:34:26.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.921+0000 7fe929800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe924100c80 0x7fe924074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:26.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.921+0000 7fe929800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe924101e70 0x7fe9240731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:26.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe929800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe924102810 0x7fe9240736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:26.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe929800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe924073e80 con 0x7fe924101e70 2024-03-30T02:34:26.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe929800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe924073ff0 con 0x7fe924102810 2024-03-30T02:34:26.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe929800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9241a0c20 con 0x7fe924100c80 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe922000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe924101e70 0x7fe9240731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe923400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe924102810 0x7fe9240736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe923400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe924102810 0x7fe9240736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:51844/0 (socket says 172.21.15.67:51844) 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe923400700 1 -- 172.21.15.67:0/1561999760 learned_addr learned my addr 172.21.15.67:0/1561999760 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe922a00700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe924100c80 0x7fe924074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe922000700 1 -- 172.21.15.67:0/1561999760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe924100c80 msgr2=0x7fe924074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe922000700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe924100c80 0x7fe924074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe922000700 1 -- 172.21.15.67:0/1561999760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe924102810 msgr2=0x7fe9240736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe922000700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe924102810 0x7fe9240736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe922000700 1 -- 172.21.15.67:0/1561999760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9100095f0 con 0x7fe924101e70 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe923400700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe924102810 0x7fe9240736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.922+0000 7fe922000700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe924101e70 0x7fe9240731a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe9100098e0 tx=0x7fe910005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.923+0000 7fe91b400700 1 -- 172.21.15.67:0/1561999760 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91001c070 con 0x7fe924101e70 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.923+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9241a0e50 con 0x7fe924101e70 2024-03-30T02:34:26.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.923+0000 7fe91b400700 1 -- 172.21.15.67:0/1561999760 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe91000fd10 con 0x7fe924101e70 2024-03-30T02:34:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.923+0000 7fe91b400700 1 -- 172.21.15.67:0/1561999760 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910022cc0 con 0x7fe924101e70 2024-03-30T02:34:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.923+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9241a11c0 con 0x7fe924101e70 2024-03-30T02:34:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.924+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe924108030 con 0x7fe924101e70 2024-03-30T02:34:26.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.926+0000 7fe91b400700 1 -- 172.21.15.67:0/1561999760 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fe910020070 con 0x7fe924101e70 2024-03-30T02:34:26.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.927+0000 7fe91b400700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe9080a1120 0x7fe9080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:26.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.927+0000 7fe91b400700 1 -- 172.21.15.67:0/1561999760 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe910022e40 con 0x7fe924101e70 2024-03-30T02:34:26.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.927+0000 7fe922a00700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe9080a1120 0x7fe9080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:26.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.928+0000 7fe922a00700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe9080a1120 0x7fe9080a35e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe924101cd0 tx=0x7fe914009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:26.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:26.931+0000 7fe91b400700 1 -- 172.21.15.67:0/1561999760 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe9100970f0 con 0x7fe924101e70 2024-03-30T02:34:27.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.201+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe924067a10 con 0x7fe924101e70 2024-03-30T02:34:27.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.202+0000 7fe91b400700 1 -- 172.21.15.67:0/1561999760 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3528 (secure 0 0 0) 0x7fe910027de0 con 0x7fe924101e70 2024-03-30T02:34:27.203 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:27.203 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-30T02:34:27.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe9080a1120 msgr2=0x7fe9080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:27.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe9080a1120 0x7fe9080a35e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe924101cd0 tx=0x7fe914009260 comp rx=0 tx=0).stop 2024-03-30T02:34:27.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe924101e70 msgr2=0x7fe9240731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:27.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe924101e70 0x7fe9240731a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe9100098e0 tx=0x7fe910005b00 comp rx=0 tx=0).stop 2024-03-30T02:34:27.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 shutdown_connections 2024-03-30T02:34:27.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe9080a1120 0x7fe9080a35e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:27.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe924102810 0x7fe9240736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:27.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe924100c80 0x7fe924074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 --2- 172.21.15.67:0/1561999760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe924101e70 0x7fe9240731a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 >> 172.21.15.67:0/1561999760 conn(0x7fe9240fc1f0 msgr2=0x7fe924105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 shutdown_connections 2024-03-30T02:34:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:27.205+0000 7fe929800700 1 -- 172.21.15.67:0/1561999760 wait complete. 2024-03-30T02:34:27.207 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2024-03-30T02:34:27.535 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: Deploying daemon mds.e on smithi082 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.67:6836/437498289,v1:172.21.15.67:6837/437498289] up:boot 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: fsmap 4 up:standby 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:27.536 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:27 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1561999760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:27.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:27.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: Deploying daemon mds.e on smithi082 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.67:6836/437498289,v1:172.21.15.67:6837/437498289] up:boot 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: fsmap 4 up:standby 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:27.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:27 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1561999760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: Deploying daemon mds.e on smithi082 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.67:6836/437498289,v1:172.21.15.67:6837/437498289] up:boot 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: fsmap 4 up:standby 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:27 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1561999760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:28.250 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:34:27 smithi082 bash[115142]: a4cfa7ba54732cc829359e49bbf8b31c2bf913b391398ce0dc9aee0dcb45eed4 2024-03-30T02:34:28.250 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:34:27 smithi082 systemd[1]: Started Ceph mds.e for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: Deploying daemon mds.f on smithi170 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6834/3401372629,v1:172.21.15.82:6835/3401372629] up:boot 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: fsmap 5 up:standby 2024-03-30T02:34:29.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T02:34:29.226 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: Deploying daemon mds.f on smithi170 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6834/3401372629,v1:172.21.15.82:6835/3401372629] up:boot 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: fsmap 5 up:standby 2024-03-30T02:34:29.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T02:34:29.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:29.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:29.290 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:29.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:29.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:29.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:29.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: Deploying daemon mds.f on smithi170 2024-03-30T02:34:29.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6834/3401372629,v1:172.21.15.82:6835/3401372629] up:boot 2024-03-30T02:34:29.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: fsmap 5 up:standby 2024-03-30T02:34:29.291 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T02:34:29.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.435+0000 7fd8f8400700 1 -- 172.21.15.67:0/1978732564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 msgr2=0x7fd8f01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:29.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.435+0000 7fd8f8400700 1 --2- 172.21.15.67:0/1978732564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 0x7fd8f01010a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40099c0 tx=0x7fd8e4009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.435+0000 7fd8f8400700 1 -- 172.21.15.67:0/1978732564 shutdown_connections 2024-03-30T02:34:29.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.435+0000 7fd8f8400700 1 --2- 172.21.15.67:0/1978732564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 0x7fd8f01010a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.435+0000 7fd8f8400700 1 --2- 172.21.15.67:0/1978732564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8f0102810 0x7fd8f01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.435+0000 7fd8f8400700 1 --2- 172.21.15.67:0/1978732564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f0101e70 0x7fd8f01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.435+0000 7fd8f8400700 1 -- 172.21.15.67:0/1978732564 >> 172.21.15.67:0/1978732564 conn(0x7fd8f00fc1f0 msgr2=0x7fd8f00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.435+0000 7fd8f8400700 1 -- 172.21.15.67:0/1978732564 shutdown_connections 2024-03-30T02:34:29.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.435+0000 7fd8f8400700 1 -- 172.21.15.67:0/1978732564 wait complete. 2024-03-30T02:34:29.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.436+0000 7fd8f8400700 1 Processor -- start 2024-03-30T02:34:29.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.436+0000 7fd8f8400700 1 -- start start 2024-03-30T02:34:29.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.436+0000 7fd8f8400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 0x7fd8f0196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:29.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.436+0000 7fd8f8400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f0101e70 0x7fd8f0197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:29.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.436+0000 7fd8f8400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8f0102810 0x7fd8f019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:29.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.436+0000 7fd8f8400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019e760 con 0x7fd8f0100c80 2024-03-30T02:34:29.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f8400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019e8a0 con 0x7fd8f0102810 2024-03-30T02:34:29.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f8400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019ea30 con 0x7fd8f0101e70 2024-03-30T02:34:29.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 0x7fd8f0196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 0x7fd8f0196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32804/0 (socket says 172.21.15.67:32804) 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5a00700 1 -- 172.21.15.67:0/2774996065 learned_addr learned my addr 172.21.15.67:0/2774996065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5000700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f0101e70 0x7fd8f0197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f6400700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8f0102810 0x7fd8f019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5a00700 1 -- 172.21.15.67:0/2774996065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f0101e70 msgr2=0x7fd8f0197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5a00700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f0101e70 0x7fd8f0197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5a00700 1 -- 172.21.15.67:0/2774996065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8f0102810 msgr2=0x7fd8f019e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5a00700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8f0102810 0x7fd8f019e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5a00700 1 -- 172.21.15.67:0/2774996065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e40095f0 con 0x7fd8f0100c80 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8f5a00700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 0x7fd8f0196c30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4006010 tx=0x7fd8e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.437+0000 7fd8ee000700 1 -- 172.21.15.67:0/2774996065 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e401d070 con 0x7fd8f0100c80 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.438+0000 7fd8ee000700 1 -- 172.21.15.67:0/2774996065 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e4005020 con 0x7fd8f0100c80 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.438+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f019ecb0 con 0x7fd8f0100c80 2024-03-30T02:34:29.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.438+0000 7fd8ee000700 1 -- 172.21.15.67:0/2774996065 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4004b10 con 0x7fd8f0100c80 2024-03-30T02:34:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.438+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f019f070 con 0x7fd8f0100c80 2024-03-30T02:34:29.441 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:34:29 smithi170 systemd[1]: Starting Ceph mds.f for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.439+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f0067a80 con 0x7fd8f0100c80 2024-03-30T02:34:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.441+0000 7fd8ee000700 1 -- 172.21.15.67:0/2774996065 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fd8e4005190 con 0x7fd8f0100c80 2024-03-30T02:34:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.442+0000 7fd8ee000700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd8d80a1120 0x7fd8d80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.442+0000 7fd8ee000700 1 -- 172.21.15.67:0/2774996065 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd8e40ccee0 con 0x7fd8f0100c80 2024-03-30T02:34:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.442+0000 7fd8f5000700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd8d80a1120 0x7fd8d80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:29.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.443+0000 7fd8f5000700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd8d80a1120 0x7fd8d80a35e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009c60 tx=0x7fd8dc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:29.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.446+0000 7fd8ee000700 1 -- 172.21.15.67:0/2774996065 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd8f0067a80 con 0x7fd8f0100c80 2024-03-30T02:34:29.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.714+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8f010c4e0 con 0x7fd8f0100c80 2024-03-30T02:34:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.715+0000 7fd8ee000700 1 -- 172.21.15.67:0/2774996065 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4285 (secure 0 0 0) 0x7fd8e4096b00 con 0x7fd8f0100c80 2024-03-30T02:34:29.717 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:29.717 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-03-30T02:34:29.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd8d80a1120 msgr2=0x7fd8d80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:29.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd8d80a1120 0x7fd8d80a35e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009c60 tx=0x7fd8dc009260 comp rx=0 tx=0).stop 2024-03-30T02:34:29.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 msgr2=0x7fd8f0196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:29.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 0x7fd8f0196c30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4006010 tx=0x7fd8e400f690 comp rx=0 tx=0).stop 2024-03-30T02:34:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 shutdown_connections 2024-03-30T02:34:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd8d80a1120 0x7fd8d80a35e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd8f0100c80 0x7fd8f0196c30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8f0102810 0x7fd8f019e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 --2- 172.21.15.67:0/2774996065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8f0101e70 0x7fd8f0197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 >> 172.21.15.67:0/2774996065 conn(0x7fd8f00fc1f0 msgr2=0x7fd8f0105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 shutdown_connections 2024-03-30T02:34:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:29.717+0000 7fd8f8400700 1 -- 172.21.15.67:0/2774996065 wait complete. 2024-03-30T02:34:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 6 2024-03-30T02:34:29.938 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-mds[116453]: -- 172.21.15.170:0/2230821202 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5651dca661a0 con 0x5651dca3c800 2024-03-30T02:34:29.939 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-f[116430]: starting mds.f at 2024-03-30T02:34:29.939 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:34:29 smithi170 bash[116415]: 77c74bb582faf328fa788bd12b70d858661ba1c547f003139df573bc114bf5b5 2024-03-30T02:34:29.939 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:34:29 smithi170 systemd[1]: Started Ceph mds.f for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:30.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:29 smithi067 ceph-mon[37808]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:30.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:30.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:30.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:30.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:30.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:30.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:29 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:30.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:29 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2774996065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:30.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-mon[78798]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:30.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:30.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:30.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:30.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:30.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:30.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:30.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:29 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2774996065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:30.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:29 smithi082 ceph-mon[79176]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:30.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:30.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:30.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:30.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:30.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:30.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:29 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:30.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:29 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2774996065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:31.128 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:30 smithi170 ceph-mon[78798]: Deploying daemon mds.c on smithi170 2024-03-30T02:34:31.129 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:30 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6834/1685909437,v1:172.21.15.170:6835/1685909437] up:boot 2024-03-30T02:34:31.129 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:30 smithi170 ceph-mon[78798]: fsmap 6 up:standby 2024-03-30T02:34:31.129 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:30 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T02:34:31.129 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:34:30 smithi170 systemd[1]: Starting Ceph mds.c for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:31.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:30 smithi082 ceph-mon[79176]: Deploying daemon mds.c on smithi170 2024-03-30T02:34:31.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:30 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6834/1685909437,v1:172.21.15.170:6835/1685909437] up:boot 2024-03-30T02:34:31.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:30 smithi082 ceph-mon[79176]: fsmap 6 up:standby 2024-03-30T02:34:31.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:30 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T02:34:31.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:30 smithi067 ceph-mon[37808]: Deploying daemon mds.c on smithi170 2024-03-30T02:34:31.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:30 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6834/1685909437,v1:172.21.15.170:6835/1685909437] up:boot 2024-03-30T02:34:31.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:30 smithi067 ceph-mon[37808]: fsmap 6 up:standby 2024-03-30T02:34:31.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:30 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T02:34:31.439 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:34:31 smithi170 ceph-mds[116793]: -- 172.21.15.170:0/932019494 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5583186981a0 con 0x55831866f000 2024-03-30T02:34:31.439 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:34:31 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-c[116770]: starting mds.c at 2024-03-30T02:34:31.439 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:34:31 smithi170 bash[116755]: be0e85205633f51f66885b7b77b7f843413d2d4f48d8b09d25abf5c2f9b21652 2024-03-30T02:34:31.439 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:34:31 smithi170 systemd[1]: Started Ceph mds.c for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:31.738 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.947+0000 7f0ad0400700 1 -- 172.21.15.67:0/747978072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8101e70 msgr2=0x7f0ac81022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.947+0000 7f0ad0400700 1 --2- 172.21.15.67:0/747978072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8101e70 0x7f0ac81022d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40099c0 tx=0x7f0ab4009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.947+0000 7f0ad0400700 1 -- 172.21.15.67:0/747978072 shutdown_connections 2024-03-30T02:34:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.947+0000 7f0ad0400700 1 --2- 172.21.15.67:0/747978072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac8102810 0x7f0ac81090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.947+0000 7f0ad0400700 1 --2- 172.21.15.67:0/747978072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ac8100c80 0x7f0ac81010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.947+0000 7f0ad0400700 1 --2- 172.21.15.67:0/747978072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8101e70 0x7f0ac81022d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.947+0000 7f0ad0400700 1 -- 172.21.15.67:0/747978072 >> 172.21.15.67:0/747978072 conn(0x7f0ac80fc1f0 msgr2=0x7f0ac80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.947+0000 7f0ad0400700 1 -- 172.21.15.67:0/747978072 shutdown_connections 2024-03-30T02:34:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.947+0000 7f0ad0400700 1 -- 172.21.15.67:0/747978072 wait complete. 2024-03-30T02:34:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.948+0000 7f0ad0400700 1 Processor -- start 2024-03-30T02:34:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.948+0000 7f0ad0400700 1 -- start start 2024-03-30T02:34:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.948+0000 7f0ad0400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8100c80 0x7f0ac8196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.948+0000 7f0ad0400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac8101e70 0x7f0ac8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.948+0000 7f0ad0400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ac8102810 0x7f0ac819e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.948+0000 7f0ad0400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819e730 con 0x7f0ac8101e70 2024-03-30T02:34:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.948+0000 7f0ad0400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819e870 con 0x7f0ac8100c80 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.948+0000 7f0ad0400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac819ea00 con 0x7f0ac8102810 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acd000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac8101e70 0x7f0ac8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acd000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac8101e70 0x7f0ac8197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32826/0 (socket says 172.21.15.67:32826) 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acd000700 1 -- 172.21.15.67:0/2765812625 learned_addr learned my addr 172.21.15.67:0/2765812625 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acda00700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8100c80 0x7f0ac8196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0ace400700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ac8102810 0x7f0ac819e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acd000700 1 -- 172.21.15.67:0/2765812625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ac8102810 msgr2=0x7f0ac819e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acd000700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ac8102810 0x7f0ac819e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acd000700 1 -- 172.21.15.67:0/2765812625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8100c80 msgr2=0x7f0ac8196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acd000700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8100c80 0x7f0ac8196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acd000700 1 -- 172.21.15.67:0/2765812625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40095f0 con 0x7f0ac8101e70 2024-03-30T02:34:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0acd000700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac8101e70 0x7f0ac8197140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006010 tx=0x7f0ab400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0ac6000700 1 -- 172.21.15.67:0/2765812625 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab401d070 con 0x7f0ac8101e70 2024-03-30T02:34:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.949+0000 7f0ac6000700 1 -- 172.21.15.67:0/2765812625 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ab4005030 con 0x7f0ac8101e70 2024-03-30T02:34:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.950+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac819ec80 con 0x7f0ac8101e70 2024-03-30T02:34:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.950+0000 7f0ac6000700 1 -- 172.21.15.67:0/2765812625 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4004bb0 con 0x7f0ac8101e70 2024-03-30T02:34:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.950+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac819f100 con 0x7f0ac8101e70 2024-03-30T02:34:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.951+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac8108100 con 0x7f0ac8101e70 2024-03-30T02:34:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.952+0000 7f0ac6000700 1 -- 172.21.15.67:0/2765812625 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0ab40051a0 con 0x7f0ac8101e70 2024-03-30T02:34:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.954+0000 7f0ac6000700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0ab00a11a0 0x7f0ab00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.954+0000 7f0ac6000700 1 -- 172.21.15.67:0/2765812625 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ab40cd190 con 0x7f0ac8101e70 2024-03-30T02:34:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.954+0000 7f0acda00700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0ab00a11a0 0x7f0ab00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.954+0000 7f0acda00700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0ab00a11a0 0x7f0ab00a3660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8101cd0 tx=0x7f0abc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:31.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:31.958+0000 7f0ac6000700 1 -- 172.21.15.67:0/2765812625 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0ab4096db0 con 0x7f0ac8101e70 2024-03-30T02:34:32.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:31 smithi082 ceph-mon[79176]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:32.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:32.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:32.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:32.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:32.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:32.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:31 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:32.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:31 smithi067 ceph-mon[37808]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:32.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:32.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:32.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:31 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:32.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.226+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ac8067a10 con 0x7f0ac8101e70 2024-03-30T02:34:32.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.227+0000 7f0ac6000700 1 -- 172.21.15.67:0/2765812625 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5802 (secure 0 0 0) 0x7f0ab409a3d0 con 0x7f0ac8101e70 2024-03-30T02:34:32.228 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:32.228 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-03-30T02:34:32.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0ab00a11a0 msgr2=0x7f0ab00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:32.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0ab00a11a0 0x7f0ab00a3660 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8101cd0 tx=0x7f0abc009260 comp rx=0 tx=0).stop 2024-03-30T02:34:32.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac8101e70 msgr2=0x7f0ac8197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:32.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac8101e70 0x7f0ac8197140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4006010 tx=0x7f0ab400f690 comp rx=0 tx=0).stop 2024-03-30T02:34:32.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 shutdown_connections 2024-03-30T02:34:32.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0ab00a11a0 0x7f0ab00a3660 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:32.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ac8102810 0x7f0ac819e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:32.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8100c80 0x7f0ac8196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:32.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 --2- 172.21.15.67:0/2765812625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ac8101e70 0x7f0ac8197140 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:32.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 >> 172.21.15.67:0/2765812625 conn(0x7f0ac80fc1f0 msgr2=0x7f0ac8105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:32.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 shutdown_connections 2024-03-30T02:34:32.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:32.229+0000 7f0ad0400700 1 -- 172.21.15.67:0/2765812625 wait complete. 2024-03-30T02:34:32.231 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 8 2024-03-30T02:34:32.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:31 smithi170 ceph-mon[78798]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:32.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:32.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:32.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:32.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:32.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:32.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:31 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:32.603 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:34:32 smithi082 systemd[1]: Starting Ceph mds.h for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:33.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:32 smithi082 ceph-mon[79176]: Deploying daemon mds.h on smithi082 2024-03-30T02:34:33.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:32 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] up:boot 2024-03-30T02:34:33.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:32 smithi082 ceph-mon[79176]: fsmap 7 up:standby 2024-03-30T02:34:33.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:32 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T02:34:33.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:32 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2765812625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:33.250 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:34:33 smithi082 ceph-mds[115523]: -- 172.21.15.82:0/3533677223 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ca016a21a0 con 0x55ca01679000 2024-03-30T02:34:33.251 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:34:33 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-h[115499]: starting mds.h at 2024-03-30T02:34:33.251 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:34:33 smithi082 bash[115485]: e38f3d29778be40bb2ddab13181a0728c3c5916118106dd6d75012184a80b77a 2024-03-30T02:34:33.251 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:34:33 smithi082 systemd[1]: Started Ceph mds.h for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:33.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:32 smithi067 ceph-mon[37808]: Deploying daemon mds.h on smithi082 2024-03-30T02:34:33.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:32 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] up:boot 2024-03-30T02:34:33.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:32 smithi067 ceph-mon[37808]: fsmap 7 up:standby 2024-03-30T02:34:33.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:32 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T02:34:33.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:32 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2765812625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:33.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:32 smithi170 ceph-mon[78798]: Deploying daemon mds.h on smithi082 2024-03-30T02:34:33.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:32 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] up:boot 2024-03-30T02:34:33.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:32 smithi170 ceph-mon[78798]: fsmap 7 up:standby 2024-03-30T02:34:33.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:32 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T02:34:33.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:32 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2765812625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:34.138 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:34.138 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:34.138 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:34.138 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:34.138 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:34.138 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:34.138 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:34.138 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: Deploying daemon mds.i on smithi170 2024-03-30T02:34:34.139 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] up:boot 2024-03-30T02:34:34.139 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: fsmap 8 up:standby 2024-03-30T02:34:34.139 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T02:34:34.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:34.316 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: Deploying daemon mds.i on smithi170 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] up:boot 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: fsmap 8 up:standby 2024-03-30T02:34:34.317 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:34 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T02:34:34.390 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:34:34 smithi170 systemd[1]: Starting Ceph mds.i for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.460+0000 7fc406800700 1 -- 172.21.15.67:0/1897610827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc400102810 msgr2=0x7fc4001090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.460+0000 7fc406800700 1 --2- 172.21.15.67:0/1897610827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc400102810 0x7fc4001090c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800b0a0 tx=0x7fc3f800b3b0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.460+0000 7fc406800700 1 -- 172.21.15.67:0/1897610827 shutdown_connections 2024-03-30T02:34:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.460+0000 7fc406800700 1 --2- 172.21.15.67:0/1897610827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc400100c80 0x7fc4001010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.460+0000 7fc406800700 1 --2- 172.21.15.67:0/1897610827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc400102810 0x7fc4001090c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.460+0000 7fc406800700 1 --2- 172.21.15.67:0/1897610827 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400101e70 0x7fc4001022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.460+0000 7fc406800700 1 -- 172.21.15.67:0/1897610827 >> 172.21.15.67:0/1897610827 conn(0x7fc4000fc1f0 msgr2=0x7fc4000fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:34.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.460+0000 7fc406800700 1 -- 172.21.15.67:0/1897610827 shutdown_connections 2024-03-30T02:34:34.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.460+0000 7fc406800700 1 -- 172.21.15.67:0/1897610827 wait complete. 2024-03-30T02:34:34.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.461+0000 7fc406800700 1 Processor -- start 2024-03-30T02:34:34.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc406800700 1 -- start start 2024-03-30T02:34:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc406800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc400100c80 0x7fc400196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc406800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400101e70 0x7fc4001972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:34.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc406800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc400102810 0x7fc40019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc406800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40019e860 con 0x7fc400100c80 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc406800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40019e9d0 con 0x7fc400101e70 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc406800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40019eb90 con 0x7fc400102810 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc3ff400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc400100c80 0x7fc400196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc3ff400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc400100c80 0x7fc400196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32848/0 (socket says 172.21.15.67:32848) 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc3ff400700 1 -- 172.21.15.67:0/1675177348 learned_addr learned my addr 172.21.15.67:0/1675177348 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc3ffe00700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc400102810 0x7fc40019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.462+0000 7fc3fea00700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400101e70 0x7fc4001972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc3ff400700 1 -- 172.21.15.67:0/1675177348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc400102810 msgr2=0x7fc40019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc3ff400700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc400102810 0x7fc40019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc3ff400700 1 -- 172.21.15.67:0/1675177348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400101e70 msgr2=0x7fc4001972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc3ff400700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400101e70 0x7fc4001972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc3ff400700 1 -- 172.21.15.67:0/1675177348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f8009d00 con 0x7fc400100c80 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc3ff400700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc400100c80 0x7fc400196d70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e970 tx=0x7fc3f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:34.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc3f7e00700 1 -- 172.21.15.67:0/1675177348 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0009790 con 0x7fc400100c80 2024-03-30T02:34:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc3f7e00700 1 -- 172.21.15.67:0/1675177348 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3f000c7e0 con 0x7fc400100c80 2024-03-30T02:34:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc3f7e00700 1 -- 172.21.15.67:0/1675177348 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0003c70 con 0x7fc400100c80 2024-03-30T02:34:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40019ee70 con 0x7fc400100c80 2024-03-30T02:34:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.463+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40019f240 con 0x7fc400100c80 2024-03-30T02:34:34.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.465+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc400108080 con 0x7fc400100c80 2024-03-30T02:34:34.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.466+0000 7fc3f7e00700 1 -- 172.21.15.67:0/1675177348 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc3f000cc50 con 0x7fc400100c80 2024-03-30T02:34:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.468+0000 7fc3f7e00700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc3e40a1120 0x7fc3e40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.469+0000 7fc3f7e00700 1 -- 172.21.15.67:0/1675177348 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc3f0014070 con 0x7fc400100c80 2024-03-30T02:34:34.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.469+0000 7fc3fea00700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc3e40a1120 0x7fc3e40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:34.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.469+0000 7fc3fea00700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc3e40a1120 0x7fc3e40a35e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009c60 tx=0x7fc3ec009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:34.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.472+0000 7fc3f7e00700 1 -- 172.21.15.67:0/1675177348 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc3f0096230 con 0x7fc400100c80 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: Deploying daemon mds.i on smithi170 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] up:boot 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: fsmap 8 up:standby 2024-03-30T02:34:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:34 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T02:34:34.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.734+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc40010c540 con 0x7fc400100c80 2024-03-30T02:34:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.734+0000 7fc3f7e00700 1 -- 172.21.15.67:0/1675177348 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6559 (secure 0 0 0) 0x7fc3f0003dd0 con 0x7fc400100c80 2024-03-30T02:34:34.828 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:34.828 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-03-30T02:34:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc3e40a1120 msgr2=0x7fc3e40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc3e40a1120 0x7fc3e40a35e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009c60 tx=0x7fc3ec009260 comp rx=0 tx=0).stop 2024-03-30T02:34:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc400100c80 msgr2=0x7fc400196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc400100c80 0x7fc400196d70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e970 tx=0x7fc3f000ec80 comp rx=0 tx=0).stop 2024-03-30T02:34:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 shutdown_connections 2024-03-30T02:34:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc3e40a1120 0x7fc3e40a35e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc400100c80 0x7fc400196d70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc400102810 0x7fc40019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 --2- 172.21.15.67:0/1675177348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc400101e70 0x7fc4001972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 >> 172.21.15.67:0/1675177348 conn(0x7fc4000fc1f0 msgr2=0x7fc4001051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 shutdown_connections 2024-03-30T02:34:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:34.737+0000 7fc406800700 1 -- 172.21.15.67:0/1675177348 wait complete. 2024-03-30T02:34:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 9 2024-03-30T02:34:35.066 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-mds[117133]: -- 172.21.15.170:0/3876703107 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584c45b81a0 con 0x5584c458ec00 2024-03-30T02:34:35.066 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:34:34 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-i[117110]: starting mds.i at 2024-03-30T02:34:35.066 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:34:34 smithi170 bash[117095]: 74390ca9f25c3aee77a0dd54e180d681fcde335785fbf282c582ee1c0e190e15 2024-03-30T02:34:35.066 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:34:34 smithi170 systemd[1]: Started Ceph mds.i for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:35.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:35 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1675177348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:35.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:35.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:35.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:35.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:35.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:35.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:35 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:35.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:35 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1675177348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:35.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:35.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:35.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:35.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:35.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:35.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:35 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:35.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:35 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1675177348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:35.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:35.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:35.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:35.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:35.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:35.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:35 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:36.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:36 smithi067 ceph-mon[37808]: Deploying daemon mds.g on smithi067 2024-03-30T02:34:36.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:36 smithi067 ceph-mon[37808]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:36.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:36 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] up:boot 2024-03-30T02:34:36.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:36 smithi067 ceph-mon[37808]: fsmap 9 up:standby 2024-03-30T02:34:36.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:36 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T02:34:36.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:36 smithi170 ceph-mon[78798]: Deploying daemon mds.g on smithi067 2024-03-30T02:34:36.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:36 smithi170 ceph-mon[78798]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:36.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:36 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] up:boot 2024-03-30T02:34:36.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:36 smithi170 ceph-mon[78798]: fsmap 9 up:standby 2024-03-30T02:34:36.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:36 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T02:34:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:36 smithi082 ceph-mon[79176]: Deploying daemon mds.g on smithi067 2024-03-30T02:34:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:36 smithi082 ceph-mon[79176]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:36 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] up:boot 2024-03-30T02:34:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:36 smithi082 ceph-mon[79176]: fsmap 9 up:standby 2024-03-30T02:34:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:36 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T02:34:36.693 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:34:36 smithi067 systemd[1]: Starting Ceph mds.g for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:36.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:37.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.056+0000 7f2ffa200700 1 -- 172.21.15.67:0/291247126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 msgr2=0x7f2ff41010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:37.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.056+0000 7f2ffa200700 1 --2- 172.21.15.67:0/291247126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 0x7f2ff41010a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40099c0 tx=0x7f2fe4009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.056+0000 7f2ffa200700 1 -- 172.21.15.67:0/291247126 shutdown_connections 2024-03-30T02:34:37.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.056+0000 7f2ffa200700 1 --2- 172.21.15.67:0/291247126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 0x7f2ff41010a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.056+0000 7f2ffa200700 1 --2- 172.21.15.67:0/291247126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff4102810 0x7f2ff41090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.056+0000 7f2ffa200700 1 --2- 172.21.15.67:0/291247126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff4101e70 0x7f2ff41022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.056+0000 7f2ffa200700 1 -- 172.21.15.67:0/291247126 >> 172.21.15.67:0/291247126 conn(0x7f2ff40fc1f0 msgr2=0x7f2ff40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:37.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.057+0000 7f2ffa200700 1 -- 172.21.15.67:0/291247126 shutdown_connections 2024-03-30T02:34:37.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.057+0000 7f2ffa200700 1 -- 172.21.15.67:0/291247126 wait complete. 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.058+0000 7f2ffa200700 1 Processor -- start 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.058+0000 7f2ffa200700 1 -- start start 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.058+0000 7f2ffa200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 0x7f2ff4196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.058+0000 7f2ffa200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff4101e70 0x7f2ff4197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.058+0000 7f2ffa200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff4102810 0x7f2ff419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.058+0000 7f2ffa200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff419e770 con 0x7f2ff4100c80 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.058+0000 7f2ffa200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff419e8b0 con 0x7f2ff4101e70 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.058+0000 7f2ffa200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff419ea40 con 0x7f2ff4102810 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 0x7f2ff4196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 0x7f2ff4196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55446/0 (socket says 172.21.15.67:55446) 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3400700 1 -- 172.21.15.67:0/1410334659 learned_addr learned my addr 172.21.15.67:0/1410334659 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff2a00700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff4101e70 0x7f2ff4197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:37.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3e00700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff4102810 0x7f2ff419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3400700 1 -- 172.21.15.67:0/1410334659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff4102810 msgr2=0x7f2ff419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3400700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff4102810 0x7f2ff419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3400700 1 -- 172.21.15.67:0/1410334659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff4101e70 msgr2=0x7f2ff4197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3400700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff4101e70 0x7f2ff4197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3400700 1 -- 172.21.15.67:0/1410334659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe40095f0 con 0x7f2ff4100c80 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff3e00700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff4102810 0x7f2ff419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.059+0000 7f2ff2a00700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff4101e70 0x7f2ff4197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.060+0000 7f2ff3400700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 0x7f2ff4196c40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4003730 tx=0x7f2fe400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.060+0000 7f2febe00700 1 -- 172.21.15.67:0/1410334659 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe401d070 con 0x7f2ff4100c80 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.060+0000 7f2febe00700 1 -- 172.21.15.67:0/1410334659 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fe400bda0 con 0x7f2ff4100c80 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.060+0000 7f2febe00700 1 -- 172.21.15.67:0/1410334659 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40173f0 con 0x7f2ff4100c80 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.060+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff419ecc0 con 0x7f2ff4100c80 2024-03-30T02:34:37.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.060+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff419f090 con 0x7f2ff4100c80 2024-03-30T02:34:37.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.061+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff41080f0 con 0x7f2ff4100c80 2024-03-30T02:34:37.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.064+0000 7f2febe00700 1 -- 172.21.15.67:0/1410334659 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f2fe4022470 con 0x7f2ff4100c80 2024-03-30T02:34:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.066+0000 7f2febe00700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80a11a0 0x7f2fd80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.066+0000 7f2febe00700 1 -- 172.21.15.67:0/1410334659 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2fe40cd9e0 con 0x7f2ff4100c80 2024-03-30T02:34:37.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.066+0000 7f2ff2a00700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80a11a0 0x7f2fd80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.067+0000 7f2ff2a00700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80a11a0 0x7f2fd80a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0011490 tx=0x7f2fe0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:37.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.068+0000 7f2febe00700 1 -- 172.21.15.67:0/1410334659 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2fe4097640 con 0x7f2ff4100c80 2024-03-30T02:34:37.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.352+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ff4067a10 con 0x7f2ff4100c80 2024-03-30T02:34:37.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.353+0000 7f2febe00700 1 -- 172.21.15.67:0/1410334659 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7321 (secure 0 0 0) 0x7f2fe409ac60 con 0x7f2ff4100c80 2024-03-30T02:34:37.354 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:37.354 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.355+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80a11a0 msgr2=0x7f2fd80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.355+0000 7f2ffa200700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80a11a0 0x7f2fd80a3660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0011490 tx=0x7f2fe0009260 comp rx=0 tx=0).stop 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.355+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 msgr2=0x7f2ff4196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.355+0000 7f2ffa200700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 0x7f2ff4196c40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4003730 tx=0x7f2fe400fa00 comp rx=0 tx=0).stop 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.355+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 shutdown_connections 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.355+0000 7f2ffa200700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f2fd80a11a0 0x7f2fd80a3660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.356+0000 7f2ffa200700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2ff4100c80 0x7f2ff4196c40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.356+0000 7f2ffa200700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ff4102810 0x7f2ff419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.356+0000 7f2ffa200700 1 --2- 172.21.15.67:0/1410334659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff4101e70 0x7f2ff4197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.356+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 >> 172.21.15.67:0/1410334659 conn(0x7f2ff40fc1f0 msgr2=0x7f2ff4105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:37.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.356+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 shutdown_connections 2024-03-30T02:34:37.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:37.356+0000 7f2ffa200700 1 -- 172.21.15.67:0/1410334659 wait complete. 2024-03-30T02:34:37.358 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 10 2024-03-30T02:34:37.663 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:34:37 smithi067 bash[87293]: f8df9238ab2171fa01a9860100569cce2267f166ab75306dd3440dbfbc5f804d 2024-03-30T02:34:37.663 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:34:37 smithi067 systemd[1]: Started Ceph mds.g for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:37.663 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:34:37 smithi067 ceph-mds[87355]: -- 172.21.15.67:0/3453643024 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d6a16fc1a0 con 0x55d6a16d4800 2024-03-30T02:34:37.663 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:34:37 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-g[87313]: starting mds.g at 2024-03-30T02:34:38.288 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:38 smithi067 ceph-mon[37808]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:38.288 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:38.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:38.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:38.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:38.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:38.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:38 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:38.289 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:38 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1410334659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:38.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:38 smithi170 ceph-mon[78798]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:38.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:38.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:38 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:38.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:38 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1410334659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:38 smithi082 ceph-mon[79176]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:38 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:38 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1410334659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:39.093 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:34:39 smithi067 systemd[1]: Starting Ceph mds.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:39.093 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:39 smithi067 ceph-mon[37808]: Deploying daemon mds.a on smithi067 2024-03-30T02:34:39.093 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:39 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] up:boot 2024-03-30T02:34:39.093 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:39 smithi067 ceph-mon[37808]: fsmap 10 up:standby 2024-03-30T02:34:39.093 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:39 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T02:34:39.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:39.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:39 smithi170 ceph-mon[78798]: Deploying daemon mds.a on smithi067 2024-03-30T02:34:39.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:39 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] up:boot 2024-03-30T02:34:39.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:39 smithi170 ceph-mon[78798]: fsmap 10 up:standby 2024-03-30T02:34:39.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:39 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T02:34:39.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:39 smithi082 ceph-mon[79176]: Deploying daemon mds.a on smithi067 2024-03-30T02:34:39.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:39 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] up:boot 2024-03-30T02:34:39.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:39 smithi082 ceph-mon[79176]: fsmap 10 up:standby 2024-03-30T02:34:39.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:39 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T02:34:39.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.594+0000 7fa173c00700 1 -- 172.21.15.67:0/3113524688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa16c108b60 msgr2=0x7fa16c10afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:39.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.594+0000 7fa173c00700 1 --2- 172.21.15.67:0/3113524688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa16c108b60 0x7fa16c10afd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa16400b0a0 tx=0x7fa16400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.594+0000 7fa173c00700 1 -- 172.21.15.67:0/3113524688 shutdown_connections 2024-03-30T02:34:39.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.594+0000 7fa173c00700 1 --2- 172.21.15.67:0/3113524688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa16c108b60 0x7fa16c10afd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.594+0000 7fa173c00700 1 --2- 172.21.15.67:0/3113524688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa16c074d90 0x7fa16c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.594+0000 7fa173c00700 1 --2- 172.21.15.67:0/3113524688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16c0737b0 0x7fa16c073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.594+0000 7fa173c00700 1 -- 172.21.15.67:0/3113524688 >> 172.21.15.67:0/3113524688 conn(0x7fa16c0fc200 msgr2=0x7fa16c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:39.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.594+0000 7fa173c00700 1 -- 172.21.15.67:0/3113524688 shutdown_connections 2024-03-30T02:34:39.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.595+0000 7fa173c00700 1 -- 172.21.15.67:0/3113524688 wait complete. 2024-03-30T02:34:39.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.596+0000 7fa173c00700 1 Processor -- start 2024-03-30T02:34:39.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.596+0000 7fa173c00700 1 -- start start 2024-03-30T02:34:39.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa173c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa16c0737b0 0x7fa16c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa173c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16c074d90 0x7fa16c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa173c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa16c108b60 0x7fa16c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa173c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1a0a10 con 0x7fa16c0737b0 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa173c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1a0b50 con 0x7fa16c074d90 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa173c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1a0ce0 con 0x7fa16c108b60 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa171200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa16c0737b0 0x7fa16c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa16be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16c074d90 0x7fa16c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa16be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16c074d90 0x7fa16c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35322/0 (socket says 172.21.15.67:35322) 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa16be00700 1 -- 172.21.15.67:0/2133713397 learned_addr learned my addr 172.21.15.67:0/2133713397 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa171200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa16c0737b0 0x7fa16c198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55478/0 (socket says 172.21.15.67:55478) 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.597+0000 7fa171c00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa16c108b60 0x7fa16c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:39.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa16be00700 1 -- 172.21.15.67:0/2133713397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa16c108b60 msgr2=0x7fa16c1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:39.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa16be00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa16c108b60 0x7fa16c1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa16be00700 1 -- 172.21.15.67:0/2133713397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa16c0737b0 msgr2=0x7fa16c198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:39.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa16be00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa16c0737b0 0x7fa16c198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa16be00700 1 -- 172.21.15.67:0/2133713397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164009d00 con 0x7fa16c074d90 2024-03-30T02:34:39.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa16be00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16c074d90 0x7fa16c199420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa15800e9a0 tx=0x7fa15800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:39.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa169600700 1 -- 172.21.15.67:0/2133713397 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158009790 con 0x7fa16c074d90 2024-03-30T02:34:39.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa169600700 1 -- 172.21.15.67:0/2133713397 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa158010430 con 0x7fa16c074d90 2024-03-30T02:34:39.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa169600700 1 -- 172.21.15.67:0/2133713397 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158018680 con 0x7fa16c074d90 2024-03-30T02:34:39.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.598+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c1a0fc0 con 0x7fa16c074d90 2024-03-30T02:34:39.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.599+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c1a1440 con 0x7fa16c074d90 2024-03-30T02:34:39.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.600+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c107970 con 0x7fa16c074d90 2024-03-30T02:34:39.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.602+0000 7fa169600700 1 -- 172.21.15.67:0/2133713397 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa1580105a0 con 0x7fa16c074d90 2024-03-30T02:34:39.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.604+0000 7fa169600700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa1540a11a0 0x7fa1540a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:39.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.604+0000 7fa169600700 1 -- 172.21.15.67:0/2133713397 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa158014070 con 0x7fa16c074d90 2024-03-30T02:34:39.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.605+0000 7fa171200700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa1540a11a0 0x7fa1540a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:39.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.605+0000 7fa171200700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa1540a11a0 0x7fa1540a3660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa160009c60 tx=0x7fa160009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:39.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.608+0000 7fa169600700 1 -- 172.21.15.67:0/2133713397 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa1580529f0 con 0x7fa16c074d90 2024-03-30T02:34:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.878+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa16c067a60 con 0x7fa16c074d90 2024-03-30T02:34:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.879+0000 7fa169600700 1 -- 172.21.15.67:0/2133713397 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8079 (secure 0 0 0) 0x7fa1580961a0 con 0x7fa16c074d90 2024-03-30T02:34:39.881 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:39.881 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[]} 2024-03-30T02:34:39.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa1540a11a0 msgr2=0x7fa1540a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa1540a11a0 0x7fa1540a3660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa160009c60 tx=0x7fa160009260 comp rx=0 tx=0).stop 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16c074d90 msgr2=0x7fa16c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16c074d90 0x7fa16c199420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa15800e9a0 tx=0x7fa15800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 shutdown_connections 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa1540a11a0 0x7fa1540a3660 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa16c0737b0 0x7fa16c198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa16c108b60 0x7fa16c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 --2- 172.21.15.67:0/2133713397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16c074d90 0x7fa16c199420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 >> 172.21.15.67:0/2133713397 conn(0x7fa16c0fc200 msgr2=0x7fa16c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 shutdown_connections 2024-03-30T02:34:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:39.882+0000 7fa173c00700 1 -- 172.21.15.67:0/2133713397 wait complete. 2024-03-30T02:34:39.885 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 11 2024-03-30T02:34:39.900 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:34:39 smithi067 bash[87659]: 0596a55c05a0bc6fbfa0f71f373b9a3f15cd077dc25f7aaf81f7ebd8d78e1460 2024-03-30T02:34:39.900 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:34:39 smithi067 systemd[1]: Started Ceph mds.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:39.900 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:34:39 smithi067 ceph-mds[87720]: -- 172.21.15.67:0/2685137382 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ba2229c1a0 con 0x55ba22275000 2024-03-30T02:34:39.900 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:34:39 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-a[87694]: starting mds.a at 2024-03-30T02:34:40.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:40 smithi067 ceph-mon[37808]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:40.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:40.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:40.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:40.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:40.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:40.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:40 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:40.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:40 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2133713397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:40.382 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:40 smithi082 ceph-mon[79176]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:40.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:40 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:40.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:40 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:40.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:40 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:40.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:40 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:40.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:40 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:40.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:40 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:40.383 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:40 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2133713397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:40.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:40 smithi170 ceph-mon[78798]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:40.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:40 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:40.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:40 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:40 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:40 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-30T02:34:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:40 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-30T02:34:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:40 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:40.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:40 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2133713397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:40.922 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:34:40 smithi082 systemd[1]: Starting Ceph mds.b for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:34:41.360 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:41 smithi082 ceph-mon[79176]: Deploying daemon mds.b on smithi082 2024-03-30T02:34:41.360 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:41 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.67:6840/853027854,v1:172.21.15.67:6841/853027854] up:boot 2024-03-30T02:34:41.360 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:41 smithi082 ceph-mon[79176]: fsmap 11 up:standby 2024-03-30T02:34:41.360 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:41 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T02:34:41.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:41 smithi067 ceph-mon[37808]: Deploying daemon mds.b on smithi082 2024-03-30T02:34:41.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:41 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.67:6840/853027854,v1:172.21.15.67:6841/853027854] up:boot 2024-03-30T02:34:41.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:41 smithi067 ceph-mon[37808]: fsmap 11 up:standby 2024-03-30T02:34:41.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:41 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T02:34:41.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:41 smithi170 ceph-mon[78798]: Deploying daemon mds.b on smithi082 2024-03-30T02:34:41.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:41 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.67:6840/853027854,v1:172.21.15.67:6841/853027854] up:boot 2024-03-30T02:34:41.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:41 smithi170 ceph-mon[78798]: fsmap 11 up:standby 2024-03-30T02:34:41.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:41 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T02:34:41.750 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:34:41 smithi082 ceph-mds[115865]: -- 172.21.15.82:0/1980282249 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562b068f01a0 con 0x562b068c7000 2024-03-30T02:34:41.750 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:34:41 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: starting mds.b at 2024-03-30T02:34:41.750 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:34:41 smithi082 bash[115826]: 0aa26d63945d4084c14144e5e3e2b236d9a876682eae5512673dc4739522bc5c 2024-03-30T02:34:41.750 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:34:41 smithi082 systemd[1]: Started Ceph mds.b for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4. 2024-03-30T02:34:41.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:42.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.120+0000 7f9151800700 1 -- 172.21.15.67:0/2169896175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 msgr2=0x7f914c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.120+0000 7f9151800700 1 --2- 172.21.15.67:0/2169896175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 0x7f914c073240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f913c009990 tx=0x7f913c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.120+0000 7f9151800700 1 -- 172.21.15.67:0/2169896175 shutdown_connections 2024-03-30T02:34:42.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.120+0000 7f9151800700 1 --2- 172.21.15.67:0/2169896175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f914c108b40 0x7f914c10afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.120+0000 7f9151800700 1 --2- 172.21.15.67:0/2169896175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 0x7f914c073240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.120+0000 7f9151800700 1 --2- 172.21.15.67:0/2169896175 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c073810 0x7f914c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.120+0000 7f9151800700 1 -- 172.21.15.67:0/2169896175 >> 172.21.15.67:0/2169896175 conn(0x7f914c0fc1f0 msgr2=0x7f914c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:42.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.120+0000 7f9151800700 1 -- 172.21.15.67:0/2169896175 shutdown_connections 2024-03-30T02:34:42.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.120+0000 7f9151800700 1 -- 172.21.15.67:0/2169896175 wait complete. 2024-03-30T02:34:42.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.121+0000 7f9151800700 1 Processor -- start 2024-03-30T02:34:42.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f9151800700 1 -- start start 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f9151800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c073810 0x7f914c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f9151800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 0x7f914c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f9151800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f914c108b40 0x7f914c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f9151800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c1a0970 con 0x7f914c073810 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f9151800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c1a0ab0 con 0x7f914c074df0 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f9151800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c1a0c40 con 0x7f914c108b40 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f914a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 0x7f914c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f914aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c073810 0x7f914c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f914a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 0x7f914c199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35354/0 (socket says 172.21.15.67:35354) 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f914a000700 1 -- 172.21.15.67:0/4143429172 learned_addr learned my addr 172.21.15.67:0/4143429172 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.122+0000 7f914b400700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f914c108b40 0x7f914c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f914a000700 1 -- 172.21.15.67:0/4143429172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f914c108b40 msgr2=0x7f914c1a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f914a000700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f914c108b40 0x7f914c1a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f914a000700 1 -- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c073810 msgr2=0x7f914c198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f914a000700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c073810 0x7f914c198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f914a000700 1 -- 172.21.15.67:0/4143429172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c0095f0 con 0x7f914c074df0 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f914b400700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f914c108b40 0x7f914c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f914aa00700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c073810 0x7f914c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f914a000700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 0x7f914c199380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f913800e970 tx=0x7f913800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f9143400700 1 -- 172.21.15.67:0/4143429172 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138009790 con 0x7f914c074df0 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f9143400700 1 -- 172.21.15.67:0/4143429172 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f913800eec0 con 0x7f914c074df0 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f9143400700 1 -- 172.21.15.67:0/4143429172 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138003cf0 con 0x7f914c074df0 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914c1a0f20 con 0x7f914c074df0 2024-03-30T02:34:42.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.123+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914c1a1350 con 0x7f914c074df0 2024-03-30T02:34:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.124+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914c107900 con 0x7f914c074df0 2024-03-30T02:34:42.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.126+0000 7f9143400700 1 -- 172.21.15.67:0/4143429172 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f913800ccd0 con 0x7f914c074df0 2024-03-30T02:34:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.128+0000 7f9143400700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91300a11a0 0x7f91300a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.128+0000 7f9143400700 1 -- 172.21.15.67:0/4143429172 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9138014070 con 0x7f914c074df0 2024-03-30T02:34:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.128+0000 7f914aa00700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91300a11a0 0x7f91300a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:42.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.128+0000 7f914aa00700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91300a11a0 0x7f91300a3660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f913c00b2e0 tx=0x7f913c005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:42.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.132+0000 7f9143400700 1 -- 172.21.15.67:0/4143429172 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9138096230 con 0x7f914c074df0 2024-03-30T02:34:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:42 smithi067 ceph-mon[37808]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:42.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:42.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:42 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.426+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f914c067a60 con 0x7f914c074df0 2024-03-30T02:34:42.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.426+0000 7f9143400700 1 -- 172.21.15.67:0/4143429172 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9589 (secure 0 0 0) 0x7f9138099850 con 0x7f914c074df0 2024-03-30T02:34:42.428 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:42.428 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91300a11a0 msgr2=0x7f91300a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91300a11a0 0x7f91300a3660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f913c00b2e0 tx=0x7f913c005fd0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 msgr2=0x7f914c199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 0x7f914c199380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f913800e970 tx=0x7f913800ec80 comp rx=0 tx=0).stop 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 shutdown_connections 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91300a11a0 0x7f91300a3660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f914c073810 0x7f914c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f914c108b40 0x7f914c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 --2- 172.21.15.67:0/4143429172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c074df0 0x7f914c199380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.429+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 >> 172.21.15.67:0/4143429172 conn(0x7f914c0fc1f0 msgr2=0x7f914c104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.430+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 shutdown_connections 2024-03-30T02:34:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.430+0000 7f9151800700 1 -- 172.21.15.67:0/4143429172 wait complete. 2024-03-30T02:34:42.432 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 13 2024-03-30T02:34:42.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:42 smithi170 ceph-mon[78798]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:42.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:42.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:42 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:34:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:42 smithi082 ceph-mon[79176]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:42.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:42 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.656+0000 7f164a800700 1 -- 172.21.15.67:0/1733486690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1644100c80 msgr2=0x7f16441010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.656+0000 7f164a800700 1 --2- 172.21.15.67:0/1733486690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1644100c80 0x7f16441010a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16340099c0 tx=0x7f1634009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.656+0000 7f164a800700 1 -- 172.21.15.67:0/1733486690 shutdown_connections 2024-03-30T02:34:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.656+0000 7f164a800700 1 --2- 172.21.15.67:0/1733486690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1644102810 0x7f16441090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.656+0000 7f164a800700 1 --2- 172.21.15.67:0/1733486690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1644100c80 0x7f16441010a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.656+0000 7f164a800700 1 --2- 172.21.15.67:0/1733486690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1644101e70 0x7f16441022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.656+0000 7f164a800700 1 -- 172.21.15.67:0/1733486690 >> 172.21.15.67:0/1733486690 conn(0x7f16440fc1f0 msgr2=0x7f16440fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.656+0000 7f164a800700 1 -- 172.21.15.67:0/1733486690 shutdown_connections 2024-03-30T02:34:42.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.656+0000 7f164a800700 1 -- 172.21.15.67:0/1733486690 wait complete. 2024-03-30T02:34:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.657+0000 7f164a800700 1 Processor -- start 2024-03-30T02:34:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.657+0000 7f164a800700 1 -- start start 2024-03-30T02:34:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.657+0000 7f164a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1644100c80 0x7f1644198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.657+0000 7f164a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1644101e70 0x7f1644199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.657+0000 7f164a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1644102810 0x7f16441a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.657+0000 7f164a800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441a0930 con 0x7f1644101e70 2024-03-30T02:34:42.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.657+0000 7f164a800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441a0a70 con 0x7f1644102810 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.657+0000 7f164a800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441a0c00 con 0x7f1644100c80 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1644100c80 0x7f1644198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1642a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1644101e70 0x7f1644199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1644102810 0x7f16441a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1644100c80 0x7f1644198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40236/0 (socket says 172.21.15.67:40236) 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643400700 1 -- 172.21.15.67:0/3021597626 learned_addr learned my addr 172.21.15.67:0/3021597626 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643400700 1 -- 172.21.15.67:0/3021597626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1644102810 msgr2=0x7f16441a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643400700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1644102810 0x7f16441a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643400700 1 -- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1644101e70 msgr2=0x7f1644199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643400700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1644101e70 0x7f1644199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643400700 1 -- 172.21.15.67:0/3021597626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1630009520 con 0x7f1644100c80 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1642a00700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1644101e70 0x7f1644199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643e00700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1644102810 0x7f16441a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:42.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.658+0000 7f1643400700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1644100c80 0x7f1644198e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1634003730 tx=0x7f163400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.659+0000 7f163be00700 1 -- 172.21.15.67:0/3021597626 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163401d070 con 0x7f1644100c80 2024-03-30T02:34:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.659+0000 7f163be00700 1 -- 172.21.15.67:0/3021597626 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1634005050 con 0x7f1644100c80 2024-03-30T02:34:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.659+0000 7f163be00700 1 -- 172.21.15.67:0/3021597626 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634004970 con 0x7f1644100c80 2024-03-30T02:34:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.659+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16340095f0 con 0x7f1644100c80 2024-03-30T02:34:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.659+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16441a1150 con 0x7f1644100c80 2024-03-30T02:34:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.660+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1644067a10 con 0x7f1644100c80 2024-03-30T02:34:42.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.663+0000 7f163be00700 1 -- 172.21.15.67:0/3021597626 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1634005470 con 0x7f1644100c80 2024-03-30T02:34:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.663+0000 7f163be00700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16280a1120 0x7f16280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.663+0000 7f163be00700 1 -- 172.21.15.67:0/3021597626 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f16340cd150 con 0x7f1644100c80 2024-03-30T02:34:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.663+0000 7f1642a00700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16280a1120 0x7f16280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:42.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.664+0000 7f1642a00700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16280a1120 0x7f16280a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1630009fd0 tx=0x7f1630009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:42.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.665+0000 7f163be00700 1 -- 172.21.15.67:0/3021597626 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1634096d70 con 0x7f1644100c80 2024-03-30T02:34:42.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.946+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f164410c490 con 0x7f1644100c80 2024-03-30T02:34:42.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.947+0000 7f163be00700 1 -- 172.21.15.67:0/3021597626 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18387 (secure 0 0 0) 0x7f163409a390 con 0x7f1644100c80 2024-03-30T02:34:42.949 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:42.949 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":75,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:11.078890+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:34:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16280a1120 msgr2=0x7f16280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16280a1120 0x7f16280a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1630009fd0 tx=0x7f1630009260 comp rx=0 tx=0).stop 2024-03-30T02:34:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1644100c80 msgr2=0x7f1644198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1644100c80 0x7f1644198e00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1634003730 tx=0x7f163400f690 comp rx=0 tx=0).stop 2024-03-30T02:34:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 shutdown_connections 2024-03-30T02:34:42.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16280a1120 0x7f16280a35e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1644102810 0x7f16441a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1644100c80 0x7f1644198e00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 --2- 172.21.15.67:0/3021597626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1644101e70 0x7f1644199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 >> 172.21.15.67:0/3021597626 conn(0x7f16440fc1f0 msgr2=0x7f1644072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 shutdown_connections 2024-03-30T02:34:42.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:42.950+0000 7f164a800700 1 -- 172.21.15.67:0/3021597626 wait complete. 2024-03-30T02:34:42.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T02:34:42.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:34:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.190+0000 7fbe5c800700 1 -- 172.21.15.67:0/2170875878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe54101810 msgr2=0x7fbe54103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.190+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2170875878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe54101810 0x7fbe54103c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe480098f0 tx=0x7fbe48009c00 comp rx=0 tx=0).stop 2024-03-30T02:34:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.190+0000 7fbe5c800700 1 -- 172.21.15.67:0/2170875878 shutdown_connections 2024-03-30T02:34:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.190+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2170875878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe54101810 0x7fbe54103c00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.190+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2170875878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe54106a70 0x7fbe54108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.190+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2170875878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54104140 0x7fbe54106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.190+0000 7fbe5c800700 1 -- 172.21.15.67:0/2170875878 >> 172.21.15.67:0/2170875878 conn(0x7fbe540fb200 msgr2=0x7fbe540fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.191+0000 7fbe5c800700 1 -- 172.21.15.67:0/2170875878 shutdown_connections 2024-03-30T02:34:43.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.191+0000 7fbe5c800700 1 -- 172.21.15.67:0/2170875878 wait complete. 2024-03-30T02:34:43.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.192+0000 7fbe5c800700 1 Processor -- start 2024-03-30T02:34:43.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.192+0000 7fbe5c800700 1 -- start start 2024-03-30T02:34:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5c800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe54101810 0x7fbe54198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5c800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54104140 0x7fbe541994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5c800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe54106a70 0x7fbe541a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5c800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe541a0ab0 con 0x7fbe54101810 2024-03-30T02:34:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5c800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe541a0bf0 con 0x7fbe54106a70 2024-03-30T02:34:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5c800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe541a0d80 con 0x7fbe54104140 2024-03-30T02:34:43.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe54106a70 0x7fbe541a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe59e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe54101810 0x7fbe54198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe54106a70 0x7fbe541a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35388/0 (socket says 172.21.15.67:35388) 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5a800700 1 -- 172.21.15.67:0/2093366369 learned_addr learned my addr 172.21.15.67:0/2093366369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe59400700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54104140 0x7fbe541994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5a800700 1 -- 172.21.15.67:0/2093366369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54104140 msgr2=0x7fbe541994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5a800700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54104140 0x7fbe541994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5a800700 1 -- 172.21.15.67:0/2093366369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe54101810 msgr2=0x7fbe54198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5a800700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe54101810 0x7fbe54198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5a800700 1 -- 172.21.15.67:0/2093366369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe48009520 con 0x7fbe54106a70 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.193+0000 7fbe5a800700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe54106a70 0x7fbe541a04e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000ef30 tx=0x7fbe5000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.194+0000 7fbe4ea00700 1 -- 172.21.15.67:0/2093366369 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50018070 con 0x7fbe54106a70 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.194+0000 7fbe4ea00700 1 -- 172.21.15.67:0/2093366369 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe5000f040 con 0x7fbe54106a70 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.194+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe541a1000 con 0x7fbe54106a70 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.194+0000 7fbe4ea00700 1 -- 172.21.15.67:0/2093366369 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000ccf0 con 0x7fbe54106a70 2024-03-30T02:34:43.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.194+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe541a1510 con 0x7fbe54106a70 2024-03-30T02:34:43.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.195+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe54067260 con 0x7fbe54106a70 2024-03-30T02:34:43.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.196+0000 7fbe4ea00700 1 -- 172.21.15.67:0/2093366369 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbe50003a00 con 0x7fbe54106a70 2024-03-30T02:34:43.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.199+0000 7fbe4ea00700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbe3c0a1120 0x7fbe3c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.199+0000 7fbe4ea00700 1 -- 172.21.15.67:0/2093366369 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbe500cbfe0 con 0x7fbe54106a70 2024-03-30T02:34:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.199+0000 7fbe59e00700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbe3c0a1120 0x7fbe3c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:43.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.200+0000 7fbe59e00700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbe3c0a1120 0x7fbe3c0a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800fa40 tx=0x7fbe48019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:43.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.202+0000 7fbe4ea00700 1 -- 172.21.15.67:0/2093366369 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbe50095c20 con 0x7fbe54106a70 2024-03-30T02:34:43.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:43 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] up:boot 2024-03-30T02:34:43.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:43 smithi067 ceph-mon[37808]: fsmap 12 up:standby 2024-03-30T02:34:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T02:34:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:43 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4143429172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:43 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:43 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3021597626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:43.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.430+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe5410c5c0 con 0x7fbe54106a70 2024-03-30T02:34:43.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.431+0000 7fbe4ea00700 1 -- 172.21.15.67:0/2093366369 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe50099240 con 0x7fbe54106a70 2024-03-30T02:34:43.431 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:34:43.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbe3c0a1120 msgr2=0x7fbe3c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbe3c0a1120 0x7fbe3c0a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800fa40 tx=0x7fbe48019040 comp rx=0 tx=0).stop 2024-03-30T02:34:43.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe54106a70 msgr2=0x7fbe541a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe54106a70 0x7fbe541a04e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000ef30 tx=0x7fbe5000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 shutdown_connections 2024-03-30T02:34:43.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbe3c0a1120 0x7fbe3c0a35e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe54101810 0x7fbe54198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe54106a70 0x7fbe541a04e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 --2- 172.21.15.67:0/2093366369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54104140 0x7fbe541994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 >> 172.21.15.67:0/2093366369 conn(0x7fbe540fb200 msgr2=0x7fbe540ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:43.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 shutdown_connections 2024-03-30T02:34:43.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.434+0000 7fbe5c800700 1 -- 172.21.15.67:0/2093366369 wait complete. 2024-03-30T02:34:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:43 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] up:boot 2024-03-30T02:34:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:43 smithi170 ceph-mon[78798]: fsmap 12 up:standby 2024-03-30T02:34:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T02:34:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:43 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4143429172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:43 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:43 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3021597626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:43.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T02:34:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:43 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] up:boot 2024-03-30T02:34:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:43 smithi082 ceph-mon[79176]: fsmap 12 up:standby 2024-03-30T02:34:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T02:34:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:43 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4143429172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:43 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:43 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3021597626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:43.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.662+0000 7f6a97a00700 1 -- 172.21.15.67:0/72643458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a900737b0 msgr2=0x7f6a90073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.662+0000 7f6a97a00700 1 --2- 172.21.15.67:0/72643458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a900737b0 0x7f6a90073c30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009990 tx=0x7f6a7c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.662+0000 7f6a97a00700 1 -- 172.21.15.67:0/72643458 shutdown_connections 2024-03-30T02:34:43.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.662+0000 7f6a97a00700 1 --2- 172.21.15.67:0/72643458 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a90108b40 0x7f6a9010afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.662+0000 7f6a97a00700 1 --2- 172.21.15.67:0/72643458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a90074d90 0x7f6a900731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.662+0000 7f6a97a00700 1 --2- 172.21.15.67:0/72643458 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a900737b0 0x7f6a90073c30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.662+0000 7f6a97a00700 1 -- 172.21.15.67:0/72643458 >> 172.21.15.67:0/72643458 conn(0x7f6a900fc1e0 msgr2=0x7f6a900fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:43.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.662+0000 7f6a97a00700 1 -- 172.21.15.67:0/72643458 shutdown_connections 2024-03-30T02:34:43.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.662+0000 7f6a97a00700 1 -- 172.21.15.67:0/72643458 wait complete. 2024-03-30T02:34:43.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.663+0000 7f6a97a00700 1 Processor -- start 2024-03-30T02:34:43.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.663+0000 7f6a97a00700 1 -- start start 2024-03-30T02:34:43.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.663+0000 7f6a97a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a900737b0 0x7f6a90198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:43.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a97a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a90074d90 0x7f6a90199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:43.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a97a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a90108b40 0x7f6a901a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:43.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a97a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901a0930 con 0x7f6a90074d90 2024-03-30T02:34:43.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a97a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901a0a70 con 0x7f6a900737b0 2024-03-30T02:34:43.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a97a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901a0c00 con 0x7f6a90108b40 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a900737b0 0x7f6a90198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a900737b0 0x7f6a90198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35398/0 (socket says 172.21.15.67:35398) 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95000700 1 -- 172.21.15.67:0/2095913168 learned_addr learned my addr 172.21.15.67:0/2095913168 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a8fe00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a90074d90 0x7f6a90199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95a00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a90108b40 0x7f6a901a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95000700 1 -- 172.21.15.67:0/2095913168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a90108b40 msgr2=0x7f6a901a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95000700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a90108b40 0x7f6a901a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95000700 1 -- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a90074d90 msgr2=0x7f6a90199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95000700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a90074d90 0x7f6a90199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95000700 1 -- 172.21.15.67:0/2095913168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0095f0 con 0x7f6a900737b0 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.664+0000 7f6a95000700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a900737b0 0x7f6a90198e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400d950 tx=0x7f6a8400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.665+0000 7f6a8d600700 1 -- 172.21.15.67:0/2095913168 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84009940 con 0x7f6a900737b0 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.665+0000 7f6a8d600700 1 -- 172.21.15.67:0/2095913168 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a84010460 con 0x7f6a900737b0 2024-03-30T02:34:43.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.665+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a901a0ee0 con 0x7f6a900737b0 2024-03-30T02:34:43.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.665+0000 7f6a8d600700 1 -- 172.21.15.67:0/2095913168 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400f620 con 0x7f6a900737b0 2024-03-30T02:34:43.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.665+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a901a1310 con 0x7f6a900737b0 2024-03-30T02:34:43.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.666+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a90107950 con 0x7f6a900737b0 2024-03-30T02:34:43.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.667+0000 7f6a8d600700 1 -- 172.21.15.67:0/2095913168 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f6a840108d0 con 0x7f6a900737b0 2024-03-30T02:34:43.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.669+0000 7f6a8d600700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780a11a0 0x7f6a780a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:43.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.669+0000 7f6a8d600700 1 -- 172.21.15.67:0/2095913168 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6a840cc290 con 0x7f6a900737b0 2024-03-30T02:34:43.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.669+0000 7f6a8fe00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780a11a0 0x7f6a780a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:43.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.670+0000 7f6a8fe00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780a11a0 0x7f6a780a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00b2e0 tx=0x7f6a7c005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:43.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.673+0000 7f6a8d600700 1 -- 172.21.15.67:0/2095913168 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6a84095e20 con 0x7f6a900737b0 2024-03-30T02:34:43.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.955+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6a90067e60 con 0x7f6a900737b0 2024-03-30T02:34:43.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.955+0000 7f6a8d600700 1 -- 172.21.15.67:0/2095913168 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6a84099440 con 0x7f6a900737b0 2024-03-30T02:34:43.956 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:43.956 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-03-30T02:34:43.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:34:43.956 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-03-30T02:34:43.956 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-03-30T02:34:43.956 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:34:43.956 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780a11a0 msgr2=0x7f6a780a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780a11a0 0x7f6a780a3660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00b2e0 tx=0x7f6a7c005fd0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a900737b0 msgr2=0x7f6a90198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a900737b0 0x7f6a90198e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400d950 tx=0x7f6a8400dc60 comp rx=0 tx=0).stop 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 shutdown_connections 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6a780a11a0 0x7f6a780a3660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a90108b40 0x7f6a901a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a900737b0 0x7f6a90198e00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 --2- 172.21.15.67:0/2095913168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a90074d90 0x7f6a90199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.958+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 >> 172.21.15.67:0/2095913168 conn(0x7f6a900fc1e0 msgr2=0x7f6a90104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.959+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 shutdown_connections 2024-03-30T02:34:43.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:43.959+0000 7f6a97a00700 1 -- 172.21.15.67:0/2095913168 wait complete. 2024-03-30T02:34:43.978 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T02:34:43.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T02:34:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.204+0000 7f8fb7600700 1 -- 172.21.15.67:0/1683212495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fb0101f70 msgr2=0x7f8fb01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.204+0000 7f8fb7600700 1 --2- 172.21.15.67:0/1683212495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fb0101f70 0x7f8fb01023d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40099c0 tx=0x7f8fa4009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.204+0000 7f8fb7600700 1 -- 172.21.15.67:0/1683212495 shutdown_connections 2024-03-30T02:34:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.204+0000 7f8fb7600700 1 --2- 172.21.15.67:0/1683212495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb0102910 0x7f8fb01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.204+0000 7f8fb7600700 1 --2- 172.21.15.67:0/1683212495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fb0100d80 0x7f8fb01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.204+0000 7f8fb7600700 1 --2- 172.21.15.67:0/1683212495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fb0101f70 0x7f8fb01023d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.204+0000 7f8fb7600700 1 -- 172.21.15.67:0/1683212495 >> 172.21.15.67:0/1683212495 conn(0x7f8fb00fc2f0 msgr2=0x7f8fb00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.204+0000 7f8fb7600700 1 -- 172.21.15.67:0/1683212495 shutdown_connections 2024-03-30T02:34:44.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.204+0000 7f8fb7600700 1 -- 172.21.15.67:0/1683212495 wait complete. 2024-03-30T02:34:44.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.205+0000 7f8fb7600700 1 Processor -- start 2024-03-30T02:34:44.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb7600700 1 -- start start 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb7600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fb0100d80 0x7f8fb0198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb7600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fb0101f70 0x7f8fb0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb7600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb0102910 0x7f8fb01a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb7600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01a0a60 con 0x7f8fb0102910 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb7600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01a0ba0 con 0x7f8fb0101f70 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb7600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb01a0d30 con 0x7f8fb0100d80 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb5600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb0102910 0x7f8fb01a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fafe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fb0101f70 0x7f8fb0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb4c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fb0100d80 0x7f8fb0198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb4c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fb0100d80 0x7f8fb0198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40292/0 (socket says 172.21.15.67:40292) 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb5600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb0102910 0x7f8fb01a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55556/0 (socket says 172.21.15.67:55556) 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.206+0000 7f8fb4c00700 1 -- 172.21.15.67:0/2782112772 learned_addr learned my addr 172.21.15.67:0/2782112772 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fb4c00700 1 -- 172.21.15.67:0/2782112772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fb0101f70 msgr2=0x7f8fb0199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:44.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fb4c00700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fb0101f70 0x7f8fb0199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fb4c00700 1 -- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb0102910 msgr2=0x7f8fb01a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fb4c00700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb0102910 0x7f8fb01a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fb4c00700 1 -- 172.21.15.67:0/2782112772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa8009c40 con 0x7f8fb0100d80 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fb5600700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb0102910 0x7f8fb01a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fb4c00700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fb0100d80 0x7f8fb0198f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e970 tx=0x7f8fa000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fad600700 1 -- 172.21.15.67:0/2782112772 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0009790 con 0x7f8fb0100d80 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fad600700 1 -- 172.21.15.67:0/2782112772 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fa000c7c0 con 0x7f8fb0100d80 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa40095f0 con 0x7f8fb0100d80 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fad600700 1 -- 172.21.15.67:0/2782112772 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0003c20 con 0x7f8fb0100d80 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.207+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb01a12a0 con 0x7f8fb0100d80 2024-03-30T02:34:44.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.208+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb0067af0 con 0x7f8fb0100d80 2024-03-30T02:34:44.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.210+0000 7f8fad600700 1 -- 172.21.15.67:0/2782112772 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8fa000c930 con 0x7f8fb0100d80 2024-03-30T02:34:44.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.211+0000 7f8fad600700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f980a1120 0x7f8f980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.212+0000 7f8fad600700 1 -- 172.21.15.67:0/2782112772 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8fa0014070 con 0x7f8fb0100d80 2024-03-30T02:34:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.212+0000 7f8fafe00700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f980a1120 0x7f8f980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:44.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.212+0000 7f8fafe00700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f980a1120 0x7f8f980a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4003730 tx=0x7f8fa400b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:44.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.213+0000 7f8fad600700 1 -- 172.21.15.67:0/2782112772 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8fa0096900 con 0x7f8fb0100d80 2024-03-30T02:34:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:44 smithi067 ceph-mon[37808]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:44 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2093366369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:34:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:44.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:44.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:44.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:44 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:44.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:44 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2095913168' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-30T02:34:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:44 smithi170 ceph-mon[78798]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:44 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2093366369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:34:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:44 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:44 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:44 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:44 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:44 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:44 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:44.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:44 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2095913168' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-30T02:34:44.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:44.466+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8fb010c5e0 con 0x7f8fb0100d80 2024-03-30T02:34:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:44 smithi082 ceph-mon[79176]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:44 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2093366369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:34:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:44 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:44 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:44 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:44 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:44 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:44 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:34:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:44 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2095913168' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-30T02:34:45.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.125+0000 7f8fad600700 1 -- 172.21.15.67:0/2782112772 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8fa0099f20 con 0x7f8fb0100d80 2024-03-30T02:34:45.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.130+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f980a1120 msgr2=0x7f8f980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:45.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.130+0000 7f8fb7600700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f980a1120 0x7f8f980a35e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4003730 tx=0x7f8fa400b540 comp rx=0 tx=0).stop 2024-03-30T02:34:45.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.130+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fb0100d80 msgr2=0x7f8fb0198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:45.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.130+0000 7f8fb7600700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fb0100d80 0x7f8fb0198f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e970 tx=0x7f8fa000ec80 comp rx=0 tx=0).stop 2024-03-30T02:34:45.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.131+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 shutdown_connections 2024-03-30T02:34:45.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.131+0000 7f8fb7600700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f980a1120 0x7f8f980a35e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.131+0000 7f8fb7600700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fb0102910 0x7f8fb01a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.131+0000 7f8fb7600700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fb0100d80 0x7f8fb0198f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.131+0000 7f8fb7600700 1 --2- 172.21.15.67:0/2782112772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fb0101f70 0x7f8fb0199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.131+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 >> 172.21.15.67:0/2782112772 conn(0x7f8fb00fc2f0 msgr2=0x7f8fb0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:45.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.131+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 shutdown_connections 2024-03-30T02:34:45.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.131+0000 7f8fb7600700 1 -- 172.21.15.67:0/2782112772 wait complete. 2024-03-30T02:34:45.134 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-03-30T02:34:45.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-30T02:34:45.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.379+0000 7f879b200700 1 -- 172.21.15.67:0/2892110030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794104140 msgr2=0x7f8794106530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:45.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.379+0000 7f879b200700 1 --2- 172.21.15.67:0/2892110030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794104140 0x7f8794106530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8780009990 tx=0x7f8780009ca0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.380+0000 7f879b200700 1 -- 172.21.15.67:0/2892110030 shutdown_connections 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.380+0000 7f879b200700 1 --2- 172.21.15.67:0/2892110030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8794101810 0x7f8794103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.380+0000 7f879b200700 1 --2- 172.21.15.67:0/2892110030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8794106a70 0x7f8794108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.380+0000 7f879b200700 1 --2- 172.21.15.67:0/2892110030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794104140 0x7f8794106530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.380+0000 7f879b200700 1 -- 172.21.15.67:0/2892110030 >> 172.21.15.67:0/2892110030 conn(0x7f87940fb200 msgr2=0x7f87940fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.380+0000 7f879b200700 1 -- 172.21.15.67:0/2892110030 shutdown_connections 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.380+0000 7f879b200700 1 -- 172.21.15.67:0/2892110030 wait complete. 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.381+0000 7f879b200700 1 Processor -- start 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.381+0000 7f879b200700 1 -- start start 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.381+0000 7f879b200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794101810 0x7f8794196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f879b200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8794104140 0x7f8794197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f879b200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8794106a70 0x7f879419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f879b200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879419e840 con 0x7f8794104140 2024-03-30T02:34:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f879b200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879419e9b0 con 0x7f8794106a70 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f879b200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879419eb70 con 0x7f8794101810 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8799200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8794106a70 0x7f879419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794101810 0x7f8794196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794101810 0x7f8794196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40318/0 (socket says 172.21.15.67:40318) 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8799200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8794106a70 0x7f879419e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35432/0 (socket says 172.21.15.67:35432) 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793e00700 1 -- 172.21.15.67:0/2140600181 learned_addr learned my addr 172.21.15.67:0/2140600181 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793400700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8794104140 0x7f8794197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793e00700 1 -- 172.21.15.67:0/2140600181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8794106a70 msgr2=0x7f879419e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793e00700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8794106a70 0x7f879419e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793e00700 1 -- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8794104140 msgr2=0x7f8794197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793e00700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8794104140 0x7f8794197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793e00700 1 -- 172.21.15.67:0/2140600181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87800095f0 con 0x7f8794101810 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8793400700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8794104140 0x7f8794197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:34:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.382+0000 7f8799200700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8794106a70 0x7f879419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.383+0000 7f8793e00700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794101810 0x7f8794196d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f878400e9a0 tx=0x7f878400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.383+0000 7f8790c00700 1 -- 172.21.15.67:0/2140600181 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784009790 con 0x7f8794101810 2024-03-30T02:34:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.383+0000 7f8790c00700 1 -- 172.21.15.67:0/2140600181 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f878400cbd0 con 0x7f8794101810 2024-03-30T02:34:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.383+0000 7f8790c00700 1 -- 172.21.15.67:0/2140600181 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784018680 con 0x7f8794101810 2024-03-30T02:34:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.383+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879419ee50 con 0x7f8794101810 2024-03-30T02:34:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.383+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879419f220 con 0x7f8794101810 2024-03-30T02:34:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.385+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87940fcd80 con 0x7f8794101810 2024-03-30T02:34:45.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.387+0000 7f8790c00700 1 -- 172.21.15.67:0/2140600181 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f878400cd40 con 0x7f8794101810 2024-03-30T02:34:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.389+0000 7f8790c00700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87780a11a0 0x7f87780a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:45.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.389+0000 7f8790c00700 1 -- 172.21.15.67:0/2140600181 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f8784014070 con 0x7f8794101810 2024-03-30T02:34:45.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.389+0000 7f8793400700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87780a11a0 0x7f87780a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:45.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.390+0000 7f8793400700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87780a11a0 0x7f87780a3660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f878000b2e0 tx=0x7f8780005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:45.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.390+0000 7f8790c00700 1 -- 172.21.15.67:0/2140600181 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f87840962c0 con 0x7f8794101810 2024-03-30T02:34:45.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:45 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2782112772' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T02:34:45.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:45 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T02:34:45.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:45 smithi067 ceph-mon[37808]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:45.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:45 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:45 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:45 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:45 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:45 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2782112772' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T02:34:45.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:45 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T02:34:45.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:45 smithi170 ceph-mon[78798]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:45.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:45 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:45 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:45 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:45 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:45 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2782112772' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T02:34:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:45 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-30T02:34:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:45 smithi082 ceph-mon[79176]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:45 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:45 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:45 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:45 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:45.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:45.653+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f879410c850 con 0x7f8794101810 2024-03-30T02:34:46.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.197+0000 7f8790c00700 1 -- 172.21.15.67:0/2140600181 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) v1 ==== 171+0+0 (secure 0 0 0) 0x7f87840998e0 con 0x7f8794101810 2024-03-30T02:34:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87780a11a0 msgr2=0x7f87780a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87780a11a0 0x7f87780a3660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f878000b2e0 tx=0x7f8780005fb0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794101810 msgr2=0x7f8794196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794101810 0x7f8794196d50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f878400e9a0 tx=0x7f878400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 shutdown_connections 2024-03-30T02:34:46.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87780a11a0 0x7f87780a3660 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8794106a70 0x7f879419e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8794101810 0x7f8794196d50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 --2- 172.21.15.67:0/2140600181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8794104140 0x7f8794197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.202+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 >> 172.21.15.67:0/2140600181 conn(0x7f87940fb200 msgr2=0x7f87941036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.203+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 shutdown_connections 2024-03-30T02:34:46.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.203+0000 7f879b200700 1 -- 172.21.15.67:0/2140600181 wait complete. 2024-03-30T02:34:46.205 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-03-30T02:34:46.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T02:34:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.437+0000 7f13dd800700 1 -- 172.21.15.67:0/2680037244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13d8104110 msgr2=0x7f13d8106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.437+0000 7f13dd800700 1 --2- 172.21.15.67:0/2680037244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13d8104110 0x7f13d8106500 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f13c40098f0 tx=0x7f13c4009c00 comp rx=0 tx=0).stop 2024-03-30T02:34:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.437+0000 7f13dd800700 1 -- 172.21.15.67:0/2680037244 shutdown_connections 2024-03-30T02:34:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.437+0000 7f13dd800700 1 --2- 172.21.15.67:0/2680037244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d81017e0 0x7f13d8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.437+0000 7f13dd800700 1 --2- 172.21.15.67:0/2680037244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13d8106a40 0x7f13d8108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.437+0000 7f13dd800700 1 --2- 172.21.15.67:0/2680037244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13d8104110 0x7f13d8106500 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.437+0000 7f13dd800700 1 -- 172.21.15.67:0/2680037244 >> 172.21.15.67:0/2680037244 conn(0x7f13d80fb1d0 msgr2=0x7f13d80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.437+0000 7f13dd800700 1 -- 172.21.15.67:0/2680037244 shutdown_connections 2024-03-30T02:34:46.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.437+0000 7f13dd800700 1 -- 172.21.15.67:0/2680037244 wait complete. 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.438+0000 7f13dd800700 1 Processor -- start 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.438+0000 7f13dd800700 1 -- start start 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13dd800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13d81017e0 0x7f13d8101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13dd800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d8104110 0x7f13d80ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13dd800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13d8106a40 0x7f13d80ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13dd800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8100470 con 0x7f13d8104110 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13dd800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d81005e0 con 0x7f13d8106a40 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13dd800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d81a0dc0 con 0x7f13d81017e0 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d6000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d8104110 0x7f13d80ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:46.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d6000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d8104110 0x7f13d80ff790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42514/0 (socket says 172.21.15.67:42514) 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d6000700 1 -- 172.21.15.67:0/2057443151 learned_addr learned my addr 172.21.15.67:0/2057443151 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d6a00700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13d81017e0 0x7f13d8101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d7400700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13d8106a40 0x7f13d80ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d6000700 1 -- 172.21.15.67:0/2057443151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13d81017e0 msgr2=0x7f13d8101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d6000700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13d81017e0 0x7f13d8101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d6000700 1 -- 172.21.15.67:0/2057443151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13d8106a40 msgr2=0x7f13d80ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d6000700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13d8106a40 0x7f13d80ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.439+0000 7f13d6000700 1 -- 172.21.15.67:0/2057443151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80095f0 con 0x7f13d8104110 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.440+0000 7f13d6000700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d8104110 0x7f13d80ff790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009810 tx=0x7f13c400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.440+0000 7f13cf400700 1 -- 172.21.15.67:0/2057443151 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c401c070 con 0x7f13d8104110 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.440+0000 7f13cf400700 1 -- 172.21.15.67:0/2057443151 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13c400f840 con 0x7f13d8104110 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.440+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c4009520 con 0x7f13d8104110 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.440+0000 7f13cf400700 1 -- 172.21.15.67:0/2057443151 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4003a40 con 0x7f13d8104110 2024-03-30T02:34:46.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.440+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d81a11d0 con 0x7f13d8104110 2024-03-30T02:34:46.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.441+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d80fcd50 con 0x7f13d8104110 2024-03-30T02:34:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.443+0000 7f13cf400700 1 -- 172.21.15.67:0/2057443151 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f13c4020030 con 0x7f13d8104110 2024-03-30T02:34:46.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.444+0000 7f13cf400700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f13bc0a1120 0x7f13bc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:46.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.444+0000 7f13cf400700 1 -- 172.21.15.67:0/2057443151 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f13c40cdc30 con 0x7f13d8104110 2024-03-30T02:34:46.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.445+0000 7f13d6a00700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f13bc0a1120 0x7f13bc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:46.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.445+0000 7f13d6a00700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f13bc0a1120 0x7f13bc0a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005fd0 tx=0x7f13c8009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:46.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.447+0000 7f13cf400700 1 -- 172.21.15.67:0/2057443151 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f13c4097560 con 0x7f13d8104110 2024-03-30T02:34:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:46 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-30T02:34:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:46 smithi082 ceph-mon[79176]: osdmap e76: 12 total, 12 up, 12 in 2024-03-30T02:34:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:46 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:46 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:46 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:46.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:46 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:46.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:46 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:46 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-30T02:34:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:46 smithi067 ceph-mon[37808]: osdmap e76: 12 total, 12 up, 12 in 2024-03-30T02:34:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:46 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:46 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:46 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:46 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:46 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:46 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-30T02:34:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:46 smithi170 ceph-mon[78798]: osdmap e76: 12 total, 12 up, 12 in 2024-03-30T02:34:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:46 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:46 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:46 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:34:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:46 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:34:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:46 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:34:46.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.711+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f13d8067ac0 con 0x7f13d8104110 2024-03-30T02:34:46.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.721+0000 7f13cf400700 1 -- 172.21.15.67:0/2057443151 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f13c409ab80 con 0x7f13d8104110 2024-03-30T02:34:46.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.726+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f13bc0a1120 msgr2=0x7f13bc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.726+0000 7f13dd800700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f13bc0a1120 0x7f13bc0a35e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005fd0 tx=0x7f13c8009310 comp rx=0 tx=0).stop 2024-03-30T02:34:46.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.726+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d8104110 msgr2=0x7f13d80ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.726+0000 7f13dd800700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d8104110 0x7f13d80ff790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009810 tx=0x7f13c400b790 comp rx=0 tx=0).stop 2024-03-30T02:34:46.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.727+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 shutdown_connections 2024-03-30T02:34:46.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.727+0000 7f13dd800700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f13bc0a1120 0x7f13bc0a35e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.727+0000 7f13dd800700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13d8106a40 0x7f13d80ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.727+0000 7f13dd800700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13d81017e0 0x7f13d8101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.727+0000 7f13dd800700 1 --2- 172.21.15.67:0/2057443151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d8104110 0x7f13d80ff790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.727+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 >> 172.21.15.67:0/2057443151 conn(0x7f13d80fb1d0 msgr2=0x7f13d8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:46.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.728+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 shutdown_connections 2024-03-30T02:34:46.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.728+0000 7f13dd800700 1 -- 172.21.15.67:0/2057443151 wait complete. 2024-03-30T02:34:46.731 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2024-03-30T02:34:46.762 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-30T02:34:46.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-30T02:34:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.966+0000 7f07cd600700 1 -- 172.21.15.67:0/3940615504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8101f70 msgr2=0x7f07c81023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.966+0000 7f07cd600700 1 --2- 172.21.15.67:0/3940615504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8101f70 0x7f07c81023d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f07b40099c0 tx=0x7f07b4009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.966+0000 7f07cd600700 1 -- 172.21.15.67:0/3940615504 shutdown_connections 2024-03-30T02:34:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.966+0000 7f07cd600700 1 --2- 172.21.15.67:0/3940615504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c8102910 0x7f07c81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.966+0000 7f07cd600700 1 --2- 172.21.15.67:0/3940615504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c8100d80 0x7f07c81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.966+0000 7f07cd600700 1 --2- 172.21.15.67:0/3940615504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8101f70 0x7f07c81023d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.966+0000 7f07cd600700 1 -- 172.21.15.67:0/3940615504 >> 172.21.15.67:0/3940615504 conn(0x7f07c80fc2f0 msgr2=0x7f07c80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.967+0000 7f07cd600700 1 -- 172.21.15.67:0/3940615504 shutdown_connections 2024-03-30T02:34:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.967+0000 7f07cd600700 1 -- 172.21.15.67:0/3940615504 wait complete. 2024-03-30T02:34:46.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.967+0000 7f07cd600700 1 Processor -- start 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07cd600700 1 -- start start 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07cd600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c8100d80 0x7f07c8198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07cd600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c8101f70 0x7f07c8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07cd600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8102910 0x7f07c81a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07cd600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81a0a60 con 0x7f07c8102910 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07cd600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81a0ba0 con 0x7f07c8100d80 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07cd600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81a0d30 con 0x7f07c8101f70 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07c6000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c8101f70 0x7f07c8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07c6000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c8101f70 0x7f07c8199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:46538/0 (socket says 172.21.15.67:46538) 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07c6000700 1 -- 172.21.15.67:0/1371327050 learned_addr learned my addr 172.21.15.67:0/1371327050 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07c6a00700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c8100d80 0x7f07c8198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.968+0000 7f07c7400700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8102910 0x7f07c81a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07c6000700 1 -- 172.21.15.67:0/1371327050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c8100d80 msgr2=0x7f07c8198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07c6000700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c8100d80 0x7f07c8198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07c6000700 1 -- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8102910 msgr2=0x7f07c81a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07c6000700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8102910 0x7f07c81a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07c6000700 1 -- 172.21.15.67:0/1371327050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b40095f0 con 0x7f07c8101f70 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07c6a00700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c8100d80 0x7f07c8198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07c7400700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8102910 0x7f07c81a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07c6000700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c8101f70 0x7f07c8199470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07b4006010 tx=0x7f07b400f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07bf400700 1 -- 172.21.15.67:0/1371327050 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b401d070 con 0x7f07c8101f70 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07bf400700 1 -- 172.21.15.67:0/1371327050 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b40054f0 con 0x7f07c8101f70 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c81a0fb0 con 0x7f07c8101f70 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07bf400700 1 -- 172.21.15.67:0/1371327050 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4004c70 con 0x7f07c8101f70 2024-03-30T02:34:46.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.969+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c81a1370 con 0x7f07c8101f70 2024-03-30T02:34:46.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.970+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c8067af0 con 0x7f07c8101f70 2024-03-30T02:34:46.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.972+0000 7f07bf400700 1 -- 172.21.15.67:0/1371327050 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f07b400fbb0 con 0x7f07c8101f70 2024-03-30T02:34:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.974+0000 7f07bf400700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f07ac0a1120 0x7f07ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.974+0000 7f07bf400700 1 -- 172.21.15.67:0/1371327050 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07b40cd9f0 con 0x7f07c8101f70 2024-03-30T02:34:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.974+0000 7f07c6a00700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f07ac0a1120 0x7f07ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:46.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.975+0000 7f07c6a00700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f07ac0a1120 0x7f07ac0a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07c8101dd0 tx=0x7f07b8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:46.976+0000 7f07bf400700 1 -- 172.21.15.67:0/1371327050 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f07b40973e0 con 0x7f07c8101f70 2024-03-30T02:34:47.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.206+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f07c810c5e0 con 0x7f07c8101f70 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2140600181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: osdmap e77: 12 total, 12 up, 12 in 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2057443151' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: osdmap e78: 12 total, 12 up, 12 in 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2057443151' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-30T02:34:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-30T02:34:47.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:47 smithi082 ceph-mon[79176]: pgmap v228: 97 pgs: 9 creating+peering, 87 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2140600181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-30T02:34:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-30T02:34:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-30T02:34:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: osdmap e77: 12 total, 12 up, 12 in 2024-03-30T02:34:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2057443151' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-30T02:34:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-30T02:34:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: osdmap e78: 12 total, 12 up, 12 in 2024-03-30T02:34:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2057443151' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-30T02:34:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-30T02:34:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-30T02:34:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:47 smithi067 ceph-mon[37808]: pgmap v228: 97 pgs: 9 creating+peering, 87 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2140600181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-30T02:34:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-30T02:34:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-30T02:34:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: osdmap e77: 12 total, 12 up, 12 in 2024-03-30T02:34:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2057443151' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-30T02:34:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-30T02:34:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: osdmap e78: 12 total, 12 up, 12 in 2024-03-30T02:34:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2057443151' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-30T02:34:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-30T02:34:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-30T02:34:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:47 smithi170 ceph-mon[78798]: pgmap v228: 97 pgs: 9 creating+peering, 87 unknown, 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:34:47.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.724+0000 7f07bf400700 1 -- 172.21.15.67:0/1371327050 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) v1 ==== 157+0+0 (secure 0 0 0) 0x7f07b409aa00 con 0x7f07c8101f70 2024-03-30T02:34:47.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f07ac0a1120 msgr2=0x7f07ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:47.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f07ac0a1120 0x7f07ac0a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07c8101dd0 tx=0x7f07b8009260 comp rx=0 tx=0).stop 2024-03-30T02:34:47.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c8101f70 msgr2=0x7f07c8199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:47.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c8101f70 0x7f07c8199470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07b4006010 tx=0x7f07b400f740 comp rx=0 tx=0).stop 2024-03-30T02:34:47.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 shutdown_connections 2024-03-30T02:34:47.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f07ac0a1120 0x7f07ac0a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f07c8102910 0x7f07c81a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c8100d80 0x7f07c8198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 --2- 172.21.15.67:0/1371327050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c8101f70 0x7f07c8199470 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 >> 172.21.15.67:0/1371327050 conn(0x7f07c80fc2f0 msgr2=0x7f07c8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 shutdown_connections 2024-03-30T02:34:47.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.728+0000 7f07cd600700 1 -- 172.21.15.67:0/1371327050 wait complete. 2024-03-30T02:34:47.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T02:34:47.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.957+0000 7f7f1f800700 1 -- 172.21.15.67:0/3255576506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18102840 msgr2=0x7f7f181090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:47.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.957+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3255576506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18102840 0x7f7f181090f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b0d0 tx=0x7f7f1000b3e0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.957+0000 7f7f1f800700 1 -- 172.21.15.67:0/3255576506 shutdown_connections 2024-03-30T02:34:47.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.957+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3255576506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18102840 0x7f7f181090f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.957+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3255576506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f18100cb0 0x7f7f181010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.957+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3255576506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f18101ea0 0x7f7f18102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.957+0000 7f7f1f800700 1 -- 172.21.15.67:0/3255576506 >> 172.21.15.67:0/3255576506 conn(0x7f7f180fc220 msgr2=0x7f7f180fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:47.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.958+0000 7f7f1f800700 1 -- 172.21.15.67:0/3255576506 shutdown_connections 2024-03-30T02:34:47.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.958+0000 7f7f1f800700 1 -- 172.21.15.67:0/3255576506 wait complete. 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.959+0000 7f7f1f800700 1 Processor -- start 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.959+0000 7f7f1f800700 1 -- start start 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.959+0000 7f7f1f800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f18100cb0 0x7f7f18074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.959+0000 7f7f1f800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18101ea0 0x7f7f180731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.959+0000 7f7f1f800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f18102840 0x7f7f180736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.959+0000 7f7f1f800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f18073e80 con 0x7f7f18101ea0 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.959+0000 7f7f1f800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f18073ff0 con 0x7f7f18100cb0 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.959+0000 7f7f1f800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f181a0d60 con 0x7f7f18102840 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.959+0000 7f7f17e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18101ea0 0x7f7f180731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f17e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18101ea0 0x7f7f180731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42552/0 (socket says 172.21.15.67:42552) 2024-03-30T02:34:47.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f17e00700 1 -- 172.21.15.67:0/3702882944 learned_addr learned my addr 172.21.15.67:0/3702882944 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f1d800700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f18102840 0x7f7f180736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f1ce00700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f18100cb0 0x7f7f18074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f17e00700 1 -- 172.21.15.67:0/3702882944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f18102840 msgr2=0x7f7f180736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f17e00700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f18102840 0x7f7f180736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f17e00700 1 -- 172.21.15.67:0/3702882944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f18100cb0 msgr2=0x7f7f18074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f17e00700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f18100cb0 0x7f7f18074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f17e00700 1 -- 172.21.15.67:0/3702882944 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f10009d00 con 0x7f7f18101ea0 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f1d800700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f18102840 0x7f7f180736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f17e00700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18101ea0 0x7f7f180731a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400e9a0 tx=0x7f7f0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f15600700 1 -- 172.21.15.67:0/3702882944 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04009790 con 0x7f7f18101ea0 2024-03-30T02:34:47.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f15600700 1 -- 172.21.15.67:0/3702882944 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f0400ce70 con 0x7f7f18101ea0 2024-03-30T02:34:47.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f181a0ff0 con 0x7f7f18101ea0 2024-03-30T02:34:47.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.960+0000 7f7f15600700 1 -- 172.21.15.67:0/3702882944 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04018420 con 0x7f7f18101ea0 2024-03-30T02:34:47.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.961+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f181a1470 con 0x7f7f18101ea0 2024-03-30T02:34:47.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.962+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f18108060 con 0x7f7f18101ea0 2024-03-30T02:34:47.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.963+0000 7f7f15600700 1 -- 172.21.15.67:0/3702882944 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7f040106f0 con 0x7f7f18101ea0 2024-03-30T02:34:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.965+0000 7f7f15600700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f000a1110 0x7f7f000a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.965+0000 7f7f15600700 1 -- 172.21.15.67:0/3702882944 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f7f04014070 con 0x7f7f18101ea0 2024-03-30T02:34:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.965+0000 7f7f1ce00700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f000a1110 0x7f7f000a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:47.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.965+0000 7f7f1ce00700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f000a1110 0x7f7f000a35d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f18101d00 tx=0x7f7f0c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:47.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:47.969+0000 7f7f15600700 1 -- 172.21.15.67:0/3702882944 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7f040529f0 con 0x7f7f18101ea0 2024-03-30T02:34:48.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:48.185+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7f1810c7d0 con 0x7f7f18101ea0 2024-03-30T02:34:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:48 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1371327050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-30T02:34:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:48 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-30T02:34:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:48 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] up:active 2024-03-30T02:34:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:48 smithi082 ceph-mon[79176]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:34:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:48 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-30T02:34:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:48 smithi082 ceph-mon[79176]: osdmap e79: 12 total, 12 up, 12 in 2024-03-30T02:34:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:48 smithi082 ceph-mon[79176]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:34:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:48 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3702882944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-30T02:34:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:48 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1371327050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-30T02:34:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:48 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-30T02:34:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:48 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] up:active 2024-03-30T02:34:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:48 smithi067 ceph-mon[37808]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:34:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:48 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-30T02:34:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:48 smithi067 ceph-mon[37808]: osdmap e79: 12 total, 12 up, 12 in 2024-03-30T02:34:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:48 smithi067 ceph-mon[37808]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:34:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:48 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3702882944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-30T02:34:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:48 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1371327050' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-30T02:34:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:48 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-30T02:34:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:48 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] up:active 2024-03-30T02:34:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:48 smithi170 ceph-mon[78798]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:34:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:48 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-30T02:34:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:48 smithi170 ceph-mon[78798]: osdmap e79: 12 total, 12 up, 12 in 2024-03-30T02:34:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:48 smithi170 ceph-mon[78798]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:34:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:48 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3702882944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-30T02:34:49.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.740+0000 7f7f15600700 1 -- 172.21.15.67:0/3702882944 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) v1 ==== 228+0+0 (secure 0 0 0) 0x7f7f04096200 con 0x7f7f18101ea0 2024-03-30T02:34:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.748+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f000a1110 msgr2=0x7f7f000a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.748+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f000a1110 0x7f7f000a35d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f18101d00 tx=0x7f7f0c009260 comp rx=0 tx=0).stop 2024-03-30T02:34:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.748+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18101ea0 msgr2=0x7f7f180731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:49.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.748+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18101ea0 0x7f7f180731a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400e9a0 tx=0x7f7f0400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.749+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 shutdown_connections 2024-03-30T02:34:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.749+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f000a1110 0x7f7f000a35d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.749+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f18102840 0x7f7f180736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.749+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f18100cb0 0x7f7f18074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.749+0000 7f7f1f800700 1 --2- 172.21.15.67:0/3702882944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f18101ea0 0x7f7f180731a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.749+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 >> 172.21.15.67:0/3702882944 conn(0x7f7f180fc220 msgr2=0x7f7f181051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:49.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.749+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 shutdown_connections 2024-03-30T02:34:49.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.750+0000 7f7f1f800700 1 -- 172.21.15.67:0/3702882944 wait complete. 2024-03-30T02:34:49.752 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' created 2024-03-30T02:34:49.774 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-30T02:34:49.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.978+0000 7f50a3200700 1 -- 172.21.15.67:0/571757594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f509c104110 msgr2=0x7f509c106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.978+0000 7f50a3200700 1 --2- 172.21.15.67:0/571757594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f509c104110 0x7f509c106500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5088009990 tx=0x7f5088009ca0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.978+0000 7f50a3200700 1 -- 172.21.15.67:0/571757594 shutdown_connections 2024-03-30T02:34:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.978+0000 7f50a3200700 1 --2- 172.21.15.67:0/571757594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509c106a40 0x7f509c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.978+0000 7f50a3200700 1 --2- 172.21.15.67:0/571757594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f509c1017e0 0x7f509c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.978+0000 7f50a3200700 1 --2- 172.21.15.67:0/571757594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f509c104110 0x7f509c106500 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.978+0000 7f50a3200700 1 -- 172.21.15.67:0/571757594 >> 172.21.15.67:0/571757594 conn(0x7f509c0fb1d0 msgr2=0x7f509c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.978+0000 7f50a3200700 1 -- 172.21.15.67:0/571757594 shutdown_connections 2024-03-30T02:34:49.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.978+0000 7f50a3200700 1 -- 172.21.15.67:0/571757594 wait complete. 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.979+0000 7f50a3200700 1 Processor -- start 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.979+0000 7f50a3200700 1 -- start start 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f50a3200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509c1017e0 0x7f509c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f50a3200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f509c104110 0x7f509c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f50a3200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f509c106a40 0x7f509c19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f50a3200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c19e820 con 0x7f509c104110 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f50a3200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c19e990 con 0x7f509c1017e0 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f50a3200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c19eb50 con 0x7f509c106a40 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f509b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f509c104110 0x7f509c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f509be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509c1017e0 0x7f509c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f509be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509c1017e0 0x7f509c196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:36300/0 (socket says 172.21.15.67:36300) 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f509be00700 1 -- 172.21.15.67:0/2644366481 learned_addr learned my addr 172.21.15.67:0/2644366481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:49.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f50a1200700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f509c106a40 0x7f509c19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f509be00700 1 -- 172.21.15.67:0/2644366481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f509c106a40 msgr2=0x7f509c19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f509be00700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f509c106a40 0x7f509c19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f509be00700 1 -- 172.21.15.67:0/2644366481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f509c104110 msgr2=0x7f509c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f509be00700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f509c104110 0x7f509c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.980+0000 7f509be00700 1 -- 172.21.15.67:0/2644366481 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094009c40 con 0x7f509c1017e0 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.981+0000 7f509be00700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509c1017e0 0x7f509c196d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f508c00e9a0 tx=0x7f508c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.981+0000 7f5098c00700 1 -- 172.21.15.67:0/2644366481 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c009790 con 0x7f509c1017e0 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.981+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50880095f0 con 0x7f509c1017e0 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.981+0000 7f5098c00700 1 -- 172.21.15.67:0/2644366481 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f508c00cbd0 con 0x7f509c1017e0 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.981+0000 7f5098c00700 1 -- 172.21.15.67:0/2644366481 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c018680 con 0x7f509c1017e0 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.981+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c19f1d0 con 0x7f509c1017e0 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.982+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c0fcd50 con 0x7f509c1017e0 2024-03-30T02:34:49.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.984+0000 7f5098c00700 1 -- 172.21.15.67:0/2644366481 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f508c00cd40 con 0x7f509c1017e0 2024-03-30T02:34:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.985+0000 7f5098c00700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f50800a1120 0x7f50800a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.985+0000 7f5098c00700 1 -- 172.21.15.67:0/2644366481 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f508c014070 con 0x7f509c1017e0 2024-03-30T02:34:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.985+0000 7f509b400700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f50800a1120 0x7f50800a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:49.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.986+0000 7f509b400700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f50800a1120 0x7f50800a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f508800b2e0 tx=0x7f5088005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:49.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:49.987+0000 7f5098c00700 1 -- 172.21.15.67:0/2644366481 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f508c0965c0 con 0x7f509c1017e0 2024-03-30T02:34:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:49 smithi082 ceph-mon[79176]: pgmap v231: 97 pgs: 9 creating+peering, 35 unknown, 53 active+clean; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 8 op/s 2024-03-30T02:34:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:49 smithi082 ceph-mon[79176]: osdmap e80: 12 total, 12 up, 12 in 2024-03-30T02:34:50.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:49 smithi067 ceph-mon[37808]: pgmap v231: 97 pgs: 9 creating+peering, 35 unknown, 53 active+clean; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 8 op/s 2024-03-30T02:34:50.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:49 smithi067 ceph-mon[37808]: osdmap e80: 12 total, 12 up, 12 in 2024-03-30T02:34:50.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:49 smithi170 ceph-mon[78798]: pgmap v231: 97 pgs: 9 creating+peering, 35 unknown, 53 active+clean; 578 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s wr, 8 op/s 2024-03-30T02:34:50.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:49 smithi170 ceph-mon[78798]: osdmap e80: 12 total, 12 up, 12 in 2024-03-30T02:34:50.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.205+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f509c10c7d0 con 0x7f509c1017e0 2024-03-30T02:34:50.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.744+0000 7f5098c00700 1 -- 172.21.15.67:0/2644366481 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v82) v1 ==== 169+0+0 (secure 0 0 0) 0x7f508c099be0 con 0x7f509c1017e0 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.753+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f50800a1120 msgr2=0x7f50800a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.753+0000 7f50a3200700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f50800a1120 0x7f50800a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f508800b2e0 tx=0x7f5088005fd0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.753+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509c1017e0 msgr2=0x7f509c196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.753+0000 7f50a3200700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509c1017e0 0x7f509c196d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f508c00e9a0 tx=0x7f508c00ecb0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.754+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 shutdown_connections 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.754+0000 7f50a3200700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f50800a1120 0x7f50800a35e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.754+0000 7f50a3200700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f509c106a40 0x7f509c19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.754+0000 7f50a3200700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f509c1017e0 0x7f509c196d00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.754+0000 7f50a3200700 1 --2- 172.21.15.67:0/2644366481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f509c104110 0x7f509c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.754+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 >> 172.21.15.67:0/2644366481 conn(0x7f509c0fb1d0 msgr2=0x7f509c102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:50.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.754+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 shutdown_connections 2024-03-30T02:34:50.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.755+0000 7f50a3200700 1 -- 172.21.15.67:0/2644366481 wait complete. 2024-03-30T02:34:50.758 INFO:teuthology.orchestra.run.smithi067.stderr:set pool 4 allow_ec_overwrites to true 2024-03-30T02:34:50.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-30T02:34:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.989+0000 7f8423600700 1 -- 172.21.15.67:0/2936736791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c102910 msgr2=0x7f841c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.989+0000 7f8423600700 1 --2- 172.21.15.67:0/2936736791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c102910 0x7f841c1091c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f841400b0d0 tx=0x7f841400b3e0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.989+0000 7f8423600700 1 -- 172.21.15.67:0/2936736791 shutdown_connections 2024-03-30T02:34:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.989+0000 7f8423600700 1 --2- 172.21.15.67:0/2936736791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c102910 0x7f841c1091c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.989+0000 7f8423600700 1 --2- 172.21.15.67:0/2936736791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f841c100d80 0x7f841c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.989+0000 7f8423600700 1 --2- 172.21.15.67:0/2936736791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c101f70 0x7f841c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.989+0000 7f8423600700 1 -- 172.21.15.67:0/2936736791 >> 172.21.15.67:0/2936736791 conn(0x7f841c0fc2f0 msgr2=0x7f841c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:50.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.989+0000 7f8423600700 1 -- 172.21.15.67:0/2936736791 shutdown_connections 2024-03-30T02:34:50.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.989+0000 7f8423600700 1 -- 172.21.15.67:0/2936736791 wait complete. 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8423600700 1 Processor -- start 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8423600700 1 -- start start 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8423600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c100d80 0x7f841c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8423600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c101f70 0x7f841c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8423600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f841c102910 0x7f841c1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8423600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c1a0b50 con 0x7f841c100d80 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8423600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c1a0c90 con 0x7f841c101f70 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8423600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c1a0e20 con 0x7f841c102910 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8420c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c100d80 0x7f841c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8420c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c100d80 0x7f841c199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42588/0 (socket says 172.21.15.67:42588) 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.990+0000 7f8420c00700 1 -- 172.21.15.67:0/898460609 learned_addr learned my addr 172.21.15.67:0/898460609 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:50.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8421600700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f841c102910 0x7f841c1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f841be00700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c101f70 0x7f841c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8420c00700 1 -- 172.21.15.67:0/898460609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f841c102910 msgr2=0x7f841c1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8420c00700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f841c102910 0x7f841c1a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8420c00700 1 -- 172.21.15.67:0/898460609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c101f70 msgr2=0x7f841c199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8420c00700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c101f70 0x7f841c199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8420c00700 1 -- 172.21.15.67:0/898460609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8414009d00 con 0x7f841c100d80 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8421600700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f841c102910 0x7f841c1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8420c00700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c100d80 0x7f841c199020 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f841000e970 tx=0x7f841000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8419600700 1 -- 172.21.15.67:0/898460609 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410009790 con 0x7f841c100d80 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8419600700 1 -- 172.21.15.67:0/898460609 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f841000c7e0 con 0x7f841c100d80 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c1a1100 con 0x7f841c100d80 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.991+0000 7f8419600700 1 -- 172.21.15.67:0/898460609 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410003c70 con 0x7f841c100d80 2024-03-30T02:34:50.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.992+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c1a1610 con 0x7f841c100d80 2024-03-30T02:34:50.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.993+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841c067af0 con 0x7f841c100d80 2024-03-30T02:34:50.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.995+0000 7f8419600700 1 -- 172.21.15.67:0/898460609 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f841000cc50 con 0x7f841c100d80 2024-03-30T02:34:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.996+0000 7f8419600700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f84040a1120 0x7f84040a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.996+0000 7f8419600700 1 -- 172.21.15.67:0/898460609 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f8410014070 con 0x7f841c100d80 2024-03-30T02:34:50.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.996+0000 7f841be00700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f84040a1120 0x7f84040a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:50.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.997+0000 7f841be00700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f84040a1120 0x7f84040a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8408009c60 tx=0x7f8408009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:50.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:50.999+0000 7f8419600700 1 -- 172.21.15.67:0/898460609 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8410096350 con 0x7f841c100d80 2024-03-30T02:34:51.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:50 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3702882944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-30T02:34:51.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:50 smithi067 ceph-mon[37808]: osdmap e81: 12 total, 12 up, 12 in 2024-03-30T02:34:51.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:50 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2644366481' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-30T02:34:51.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:50 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-30T02:34:51.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:50 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3702882944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-30T02:34:51.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:50 smithi170 ceph-mon[78798]: osdmap e81: 12 total, 12 up, 12 in 2024-03-30T02:34:51.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:50 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2644366481' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-30T02:34:51.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:50 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-30T02:34:51.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:50 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3702882944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-30T02:34:51.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:50 smithi082 ceph-mon[79176]: osdmap e81: 12 total, 12 up, 12 in 2024-03-30T02:34:51.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:50 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2644366481' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-30T02:34:51.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:50 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-30T02:34:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.267+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f841c10c5f0 con 0x7f841c100d80 2024-03-30T02:34:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.273+0000 7f8419600700 1 -- 172.21.15.67:0/898460609 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f8410099970 con 0x7f841c100d80 2024-03-30T02:34:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.275+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f84040a1120 msgr2=0x7f84040a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.275+0000 7f8423600700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f84040a1120 0x7f84040a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8408009c60 tx=0x7f8408009260 comp rx=0 tx=0).stop 2024-03-30T02:34:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.275+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c100d80 msgr2=0x7f841c199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.275+0000 7f8423600700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c100d80 0x7f841c199020 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f841000e970 tx=0x7f841000ec80 comp rx=0 tx=0).stop 2024-03-30T02:34:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.276+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 shutdown_connections 2024-03-30T02:34:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.276+0000 7f8423600700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f84040a1120 0x7f84040a35e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.276+0000 7f8423600700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f841c100d80 0x7f841c199020 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.276+0000 7f8423600700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f841c102910 0x7f841c1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.276+0000 7f8423600700 1 --2- 172.21.15.67:0/898460609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c101f70 0x7f841c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.276+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 >> 172.21.15.67:0/898460609 conn(0x7f841c0fc2f0 msgr2=0x7f841c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.276+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 shutdown_connections 2024-03-30T02:34:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.276+0000 7f8423600700 1 -- 172.21.15.67:0/898460609 wait complete. 2024-03-30T02:34:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:added data pool 4 to fsmap 2024-03-30T02:34:51.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:51.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.494+0000 7f0bba600700 1 -- 172.21.15.67:0/695160397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 msgr2=0x7f0bb41090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.494+0000 7f0bba600700 1 --2- 172.21.15.67:0/695160397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 0x7f0bb41090b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00b0d0 tx=0x7f0bac00b3e0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.494+0000 7f0bba600700 1 -- 172.21.15.67:0/695160397 shutdown_connections 2024-03-30T02:34:51.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.494+0000 7f0bba600700 1 --2- 172.21.15.67:0/695160397 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 0x7f0bb41090b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.494+0000 7f0bba600700 1 --2- 172.21.15.67:0/695160397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bb4100c70 0x7f0bb4101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.494+0000 7f0bba600700 1 --2- 172.21.15.67:0/695160397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb4101e60 0x7f0bb41022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.494+0000 7f0bba600700 1 -- 172.21.15.67:0/695160397 >> 172.21.15.67:0/695160397 conn(0x7f0bb40fc1e0 msgr2=0x7f0bb40fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:51.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.494+0000 7f0bba600700 1 -- 172.21.15.67:0/695160397 shutdown_connections 2024-03-30T02:34:51.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.494+0000 7f0bba600700 1 -- 172.21.15.67:0/695160397 wait complete. 2024-03-30T02:34:51.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.495+0000 7f0bba600700 1 Processor -- start 2024-03-30T02:34:51.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.495+0000 7f0bba600700 1 -- start start 2024-03-30T02:34:51.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.495+0000 7f0bba600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb4100c70 0x7f0bb4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:51.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.495+0000 7f0bba600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bb4101e60 0x7f0bb40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:51.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.495+0000 7f0bba600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 0x7f0bb40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:51.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.495+0000 7f0bba600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb4073e80 con 0x7f0bb4102800 2024-03-30T02:34:51.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.495+0000 7f0bba600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb4073ff0 con 0x7f0bb4100c70 2024-03-30T02:34:51.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.495+0000 7f0bba600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb41a0d70 con 0x7f0bb4101e60 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 0x7f0bb40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 0x7f0bb40736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42612/0 (socket says 172.21.15.67:42612) 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3e00700 1 -- 172.21.15.67:0/2617310720 learned_addr learned my addr 172.21.15.67:0/2617310720 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb4100c70 0x7f0bb4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb2a00700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bb4101e60 0x7f0bb40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3e00700 1 -- 172.21.15.67:0/2617310720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bb4101e60 msgr2=0x7f0bb40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3e00700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bb4101e60 0x7f0bb40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3e00700 1 -- 172.21.15.67:0/2617310720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb4100c70 msgr2=0x7f0bb4074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3e00700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb4100c70 0x7f0bb4074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3e00700 1 -- 172.21.15.67:0/2617310720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac009d00 con 0x7f0bb4102800 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0bb3e00700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 0x7f0bb40736e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0bac007b60 tx=0x7f0bac008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.496+0000 7f0babe00700 1 -- 172.21.15.67:0/2617310720 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00e040 con 0x7f0bb4102800 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.497+0000 7f0babe00700 1 -- 172.21.15.67:0/2617310720 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bac01d930 con 0x7f0bb4102800 2024-03-30T02:34:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.497+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb41a0fa0 con 0x7f0bb4102800 2024-03-30T02:34:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.497+0000 7f0babe00700 1 -- 172.21.15.67:0/2617310720 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac01b9f0 con 0x7f0bb4102800 2024-03-30T02:34:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.497+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb41a1310 con 0x7f0bb4102800 2024-03-30T02:34:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.498+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb4108020 con 0x7f0bb4102800 2024-03-30T02:34:51.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.500+0000 7f0babe00700 1 -- 172.21.15.67:0/2617310720 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0bac01daa0 con 0x7f0bb4102800 2024-03-30T02:34:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.501+0000 7f0babe00700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b980a1120 0x7f0b980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.501+0000 7f0babe00700 1 -- 172.21.15.67:0/2617310720 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0bac0cdfa0 con 0x7f0bb4102800 2024-03-30T02:34:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.501+0000 7f0bb3400700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b980a1120 0x7f0b980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:51.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.502+0000 7f0bb3400700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b980a1120 0x7f0b980a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4101cc0 tx=0x7f0ba4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.505+0000 7f0babe00700 1 -- 172.21.15.67:0/2617310720 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0bac0975d0 con 0x7f0bb4102800 2024-03-30T02:34:51.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.769+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bb4067e60 con 0x7f0bb4102800 2024-03-30T02:34:51.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.770+0000 7f0babe00700 1 -- 172.21.15.67:0/2617310720 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10798 (secure 0 0 0) 0x7f0bac09abf0 con 0x7f0bb4102800 2024-03-30T02:34:51.770 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:51.770 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:34:51.268448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-30T02:34:51.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b980a1120 msgr2=0x7f0b980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b980a1120 0x7f0b980a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4101cc0 tx=0x7f0ba4009260 comp rx=0 tx=0).stop 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 msgr2=0x7f0bb40736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 0x7f0bb40736e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0bac007b60 tx=0x7f0bac008e40 comp rx=0 tx=0).stop 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 shutdown_connections 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b980a1120 0x7f0b980a35e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bb4102800 0x7f0bb40736e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb4100c70 0x7f0bb4074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 --2- 172.21.15.67:0/2617310720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0bb4101e60 0x7f0bb40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 >> 172.21.15.67:0/2617310720 conn(0x7f0bb40fc1e0 msgr2=0x7f0bb4105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 shutdown_connections 2024-03-30T02:34:51.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.772+0000 7f0bba600700 1 -- 172.21.15.67:0/2617310720 wait complete. 2024-03-30T02:34:51.774 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-03-30T02:34:51.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:34:51.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.989+0000 7f1440400700 1 -- 172.21.15.67:0/3517126528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438102810 msgr2=0x7f14381090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.989+0000 7f1440400700 1 --2- 172.21.15.67:0/3517126528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438102810 0x7f14381090c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f143000b0d0 tx=0x7f143000b3e0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.989+0000 7f1440400700 1 -- 172.21.15.67:0/3517126528 shutdown_connections 2024-03-30T02:34:51.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.989+0000 7f1440400700 1 --2- 172.21.15.67:0/3517126528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438102810 0x7f14381090c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.989+0000 7f1440400700 1 --2- 172.21.15.67:0/3517126528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1438100c80 0x7f14381010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.989+0000 7f1440400700 1 --2- 172.21.15.67:0/3517126528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1438101e70 0x7f14381022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.989+0000 7f1440400700 1 -- 172.21.15.67:0/3517126528 >> 172.21.15.67:0/3517126528 conn(0x7f14380fc1f0 msgr2=0x7f14380fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:51.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.989+0000 7f1440400700 1 -- 172.21.15.67:0/3517126528 shutdown_connections 2024-03-30T02:34:51.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.989+0000 7f1440400700 1 -- 172.21.15.67:0/3517126528 wait complete. 2024-03-30T02:34:51.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.990+0000 7f1440400700 1 Processor -- start 2024-03-30T02:34:51.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.990+0000 7f1440400700 1 -- start start 2024-03-30T02:34:51.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f1440400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1438100c80 0x7f1438074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f1440400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438101e70 0x7f14380731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f1440400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1438102810 0x7f14380736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f1440400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1438073e80 con 0x7f1438101e70 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f1440400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1438073ff0 con 0x7f1438102810 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f1440400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14381a0d50 con 0x7f1438100c80 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143d000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438101e70 0x7f14380731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143d000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438101e70 0x7f14380731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42636/0 (socket says 172.21.15.67:42636) 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143d000700 1 -- 172.21.15.67:0/168534549 learned_addr learned my addr 172.21.15.67:0/168534549 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143da00700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1438100c80 0x7f1438074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143e400700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1438102810 0x7f14380736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143d000700 1 -- 172.21.15.67:0/168534549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1438100c80 msgr2=0x7f1438074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143d000700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1438100c80 0x7f1438074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143d000700 1 -- 172.21.15.67:0/168534549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1438102810 msgr2=0x7f14380736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143d000700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1438102810 0x7f14380736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:51.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.991+0000 7f143d000700 1 -- 172.21.15.67:0/168534549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1430009d00 con 0x7f1438101e70 2024-03-30T02:34:51.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.992+0000 7f143d000700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438101e70 0x7f14380731a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f142400e9a0 tx=0x7f142400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:51.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.992+0000 7f1436000700 1 -- 172.21.15.67:0/168534549 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424009790 con 0x7f1438101e70 2024-03-30T02:34:51.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.992+0000 7f1436000700 1 -- 172.21.15.67:0/168534549 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f142400ce70 con 0x7f1438101e70 2024-03-30T02:34:51.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.992+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14381a0fe0 con 0x7f1438101e70 2024-03-30T02:34:51.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.992+0000 7f1436000700 1 -- 172.21.15.67:0/168534549 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424018420 con 0x7f1438101e70 2024-03-30T02:34:51.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.992+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14381a1460 con 0x7f1438101e70 2024-03-30T02:34:51.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.993+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14381080b0 con 0x7f1438101e70 2024-03-30T02:34:51.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.995+0000 7f1436000700 1 -- 172.21.15.67:0/168534549 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f14240106f0 con 0x7f1438101e70 2024-03-30T02:34:51.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.996+0000 7f1436000700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14200a1120 0x7f14200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:51.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.997+0000 7f1436000700 1 -- 172.21.15.67:0/168534549 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1424014070 con 0x7f1438101e70 2024-03-30T02:34:51.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.997+0000 7f143da00700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14200a1120 0x7f14200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:51.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:51.997+0000 7f143da00700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14200a1120 0x7f14200a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1438101cd0 tx=0x7f142c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:52.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.000+0000 7f1436000700 1 -- 172.21.15.67:0/168534549 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f14240529f0 con 0x7f1438101e70 2024-03-30T02:34:52.114 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:51 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-30T02:34:52.114 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:51 smithi082 ceph-mon[79176]: osdmap e82: 12 total, 12 up, 12 in 2024-03-30T02:34:52.114 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:51 smithi082 ceph-mon[79176]: pgmap v234: 129 pgs: 14 creating+peering, 41 unknown, 74 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-30T02:34:52.114 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:51 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/898460609' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-30T02:34:52.114 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:51 smithi082 ceph-mon[79176]: osdmap e83: 12 total, 12 up, 12 in 2024-03-30T02:34:52.114 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:51 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/898460609' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-30T02:34:52.114 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:51 smithi082 ceph-mon[79176]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:34:52.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:51 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-30T02:34:52.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:51 smithi067 ceph-mon[37808]: osdmap e82: 12 total, 12 up, 12 in 2024-03-30T02:34:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:51 smithi067 ceph-mon[37808]: pgmap v234: 129 pgs: 14 creating+peering, 41 unknown, 74 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-30T02:34:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:51 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/898460609' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-30T02:34:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:51 smithi067 ceph-mon[37808]: osdmap e83: 12 total, 12 up, 12 in 2024-03-30T02:34:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:51 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/898460609' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-30T02:34:52.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:51 smithi067 ceph-mon[37808]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:34:52.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:51 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-30T02:34:52.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:51 smithi170 ceph-mon[78798]: osdmap e82: 12 total, 12 up, 12 in 2024-03-30T02:34:52.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:51 smithi170 ceph-mon[78798]: pgmap v234: 129 pgs: 14 creating+peering, 41 unknown, 74 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2024-03-30T02:34:52.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:51 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/898460609' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-30T02:34:52.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:51 smithi170 ceph-mon[78798]: osdmap e83: 12 total, 12 up, 12 in 2024-03-30T02:34:52.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:51 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/898460609' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-30T02:34:52.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:51 smithi170 ceph-mon[78798]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:34:52.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.255+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1438067a10 con 0x7f1438101e70 2024-03-30T02:34:52.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.257+0000 7f1436000700 1 -- 172.21.15.67:0/168534549 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f1424099c70 con 0x7f1438101e70 2024-03-30T02:34:52.257 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:52.258 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:51.268204+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14200a1120 msgr2=0x7f14200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14200a1120 0x7f14200a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1438101cd0 tx=0x7f142c009260 comp rx=0 tx=0).stop 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438101e70 msgr2=0x7f14380731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438101e70 0x7f14380731a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f142400e9a0 tx=0x7f142400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 shutdown_connections 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14200a1120 0x7f14200a35e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1438102810 0x7f14380736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1438100c80 0x7f1438074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 --2- 172.21.15.67:0/168534549 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1438101e70 0x7f14380731a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.259+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 >> 172.21.15.67:0/168534549 conn(0x7f14380fc1f0 msgr2=0x7f14381051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.260+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 shutdown_connections 2024-03-30T02:34:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.260+0000 7f1440400700 1 -- 172.21.15.67:0/168534549 wait complete. 2024-03-30T02:34:52.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:34:52.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.478+0000 7f2366600700 1 -- 172.21.15.67:0/3422161931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2360101e40 msgr2=0x7f23601022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.478+0000 7f2366600700 1 --2- 172.21.15.67:0/3422161931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2360101e40 0x7f23601022a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f234c0099c0 tx=0x7f234c009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.479+0000 7f2366600700 1 -- 172.21.15.67:0/3422161931 shutdown_connections 2024-03-30T02:34:52.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.479+0000 7f2366600700 1 --2- 172.21.15.67:0/3422161931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23601027e0 0x7f2360109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.479+0000 7f2366600700 1 --2- 172.21.15.67:0/3422161931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2360100c50 0x7f2360101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.479+0000 7f2366600700 1 --2- 172.21.15.67:0/3422161931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2360101e40 0x7f23601022a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.479+0000 7f2366600700 1 -- 172.21.15.67:0/3422161931 >> 172.21.15.67:0/3422161931 conn(0x7f23600fc1c0 msgr2=0x7f23600fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.479+0000 7f2366600700 1 -- 172.21.15.67:0/3422161931 shutdown_connections 2024-03-30T02:34:52.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.479+0000 7f2366600700 1 -- 172.21.15.67:0/3422161931 wait complete. 2024-03-30T02:34:52.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2366600700 1 Processor -- start 2024-03-30T02:34:52.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2366600700 1 -- start start 2024-03-30T02:34:52.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2366600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2360100c50 0x7f2360198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2366600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2360101e40 0x7f2360199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2366600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23601027e0 0x7f23601a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2366600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23601a0960 con 0x7f23601027e0 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2366600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23601a0aa0 con 0x7f2360101e40 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2366600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23601a0c30 con 0x7f2360100c50 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2365c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23601027e0 0x7f23601a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2365200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2360100c50 0x7f2360198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2365c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23601027e0 0x7f23601a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42664/0 (socket says 172.21.15.67:42664) 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f2365c00700 1 -- 172.21.15.67:0/3637464849 learned_addr learned my addr 172.21.15.67:0/3637464849 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:52.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.480+0000 7f235fe00700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2360101e40 0x7f2360199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f2365200700 1 -- 172.21.15.67:0/3637464849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2360101e40 msgr2=0x7f2360199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f2365200700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2360101e40 0x7f2360199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f2365200700 1 -- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23601027e0 msgr2=0x7f23601a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f2365200700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23601027e0 0x7f23601a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f2365200700 1 -- 172.21.15.67:0/3637464849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c0095f0 con 0x7f2360100c50 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f2365c00700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23601027e0 0x7f23601a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f2365200700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2360100c50 0x7f2360198e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f235400e9c0 tx=0x7f235400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f235d600700 1 -- 172.21.15.67:0/3637464849 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354004220 con 0x7f2360100c50 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f235d600700 1 -- 172.21.15.67:0/3637464849 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2354004380 con 0x7f2360100c50 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23601a0f10 con 0x7f2360100c50 2024-03-30T02:34:52.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f235d600700 1 -- 172.21.15.67:0/3637464849 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354003be0 con 0x7f2360100c50 2024-03-30T02:34:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.481+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23601a1330 con 0x7f2360100c50 2024-03-30T02:34:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.482+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23600679a0 con 0x7f2360100c50 2024-03-30T02:34:52.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.485+0000 7f235d600700 1 -- 172.21.15.67:0/3637464849 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f2354003d40 con 0x7f2360100c50 2024-03-30T02:34:52.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.486+0000 7f235d600700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23480a1120 0x7f23480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:52.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.486+0000 7f235d600700 1 -- 172.21.15.67:0/3637464849 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2354014070 con 0x7f2360100c50 2024-03-30T02:34:52.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.486+0000 7f235fe00700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23480a1120 0x7f23480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:52.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.487+0000 7f235fe00700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23480a1120 0x7f23480a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f234c006010 tx=0x7f234c00b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:52.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.488+0000 7f235d600700 1 -- 172.21.15.67:0/3637464849 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2354096440 con 0x7f2360100c50 2024-03-30T02:34:52.500 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:34:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: 2024-03-30T02:34:52.112+0000 7f86ad81d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T02:34:52.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.745+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f236010c6a0 con 0x7f2360100c50 2024-03-30T02:34:52.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.747+0000 7f235d600700 1 -- 172.21.15.67:0/3637464849 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f236010c6a0 con 0x7f2360100c50 2024-03-30T02:34:52.748 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:52.748 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:51.268204+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:34:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23480a1120 msgr2=0x7f23480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23480a1120 0x7f23480a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f234c006010 tx=0x7f234c00b540 comp rx=0 tx=0).stop 2024-03-30T02:34:52.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2360100c50 msgr2=0x7f2360198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2360100c50 0x7f2360198e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f235400e9c0 tx=0x7f235400ecd0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 shutdown_connections 2024-03-30T02:34:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f23480a1120 0x7f23480a35e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23601027e0 0x7f23601a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2360100c50 0x7f2360198e30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 --2- 172.21.15.67:0/3637464849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2360101e40 0x7f2360199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 >> 172.21.15.67:0/3637464849 conn(0x7f23600fc1c0 msgr2=0x7f2360072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 shutdown_connections 2024-03-30T02:34:52.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.750+0000 7f2366600700 1 -- 172.21.15.67:0/3637464849 wait complete. 2024-03-30T02:34:52.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.983+0000 7f6c0f800700 1 -- 172.21.15.67:0/3521506349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c08100c80 msgr2=0x7f6c081010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.983+0000 7f6c0f800700 1 --2- 172.21.15.67:0/3521506349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c08100c80 0x7f6c081010a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0099c0 tx=0x7f6bfc009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.984+0000 7f6c0f800700 1 -- 172.21.15.67:0/3521506349 shutdown_connections 2024-03-30T02:34:52.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.984+0000 7f6c0f800700 1 --2- 172.21.15.67:0/3521506349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c08100c80 0x7f6c081010a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.984+0000 7f6c0f800700 1 --2- 172.21.15.67:0/3521506349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08102810 0x7f6c081090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.984+0000 7f6c0f800700 1 --2- 172.21.15.67:0/3521506349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c08101e70 0x7f6c081022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.984+0000 7f6c0f800700 1 -- 172.21.15.67:0/3521506349 >> 172.21.15.67:0/3521506349 conn(0x7f6c080fc1f0 msgr2=0x7f6c080fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:52.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.984+0000 7f6c0f800700 1 -- 172.21.15.67:0/3521506349 shutdown_connections 2024-03-30T02:34:52.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.984+0000 7f6c0f800700 1 -- 172.21.15.67:0/3521506349 wait complete. 2024-03-30T02:34:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.985+0000 7f6c0f800700 1 Processor -- start 2024-03-30T02:34:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.985+0000 7f6c0f800700 1 -- start start 2024-03-30T02:34:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.985+0000 7f6c0f800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c08100c80 0x7f6c08074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.985+0000 7f6c0f800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08101e70 0x7f6c080731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.985+0000 7f6c0f800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c08102810 0x7f6c080736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.985+0000 7f6c0f800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08073e80 con 0x7f6c08102810 2024-03-30T02:34:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.985+0000 7f6c0f800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08073ff0 con 0x7f6c08100c80 2024-03-30T02:34:52.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.985+0000 7f6c0f800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c081a0c60 con 0x7f6c08101e70 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0ce00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c08100c80 0x7f6c08074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0d800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c08102810 0x7f6c080736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0ce00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c08100c80 0x7f6c08074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:36374/0 (socket says 172.21.15.67:36374) 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0ce00700 1 -- 172.21.15.67:0/2440274978 learned_addr learned my addr 172.21.15.67:0/2440274978 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c07e00700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08101e70 0x7f6c080731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0ce00700 1 -- 172.21.15.67:0/2440274978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08101e70 msgr2=0x7f6c080731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0ce00700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08101e70 0x7f6c080731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0ce00700 1 -- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c08102810 msgr2=0x7f6c080736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0ce00700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c08102810 0x7f6c080736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0ce00700 1 -- 172.21.15.67:0/2440274978 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0095f0 con 0x7f6c08100c80 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0d800700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c08102810 0x7f6c080736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c0ce00700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c08100c80 0x7f6c08074b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc000c00 tx=0x7f6bfc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.986+0000 7f6c05600700 1 -- 172.21.15.67:0/2440274978 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc01d070 con 0x7f6c08100c80 2024-03-30T02:34:52.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.987+0000 7f6c05600700 1 -- 172.21.15.67:0/2440274978 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bfc005050 con 0x7f6c08100c80 2024-03-30T02:34:52.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.987+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c081a0e90 con 0x7f6c08100c80 2024-03-30T02:34:52.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.987+0000 7f6c05600700 1 -- 172.21.15.67:0/2440274978 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc004b10 con 0x7f6c08100c80 2024-03-30T02:34:52.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.987+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c081a1200 con 0x7f6c08100c80 2024-03-30T02:34:52.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.988+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c08067a80 con 0x7f6c08100c80 2024-03-30T02:34:52.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.989+0000 7f6c05600700 1 -- 172.21.15.67:0/2440274978 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f6bfc0051c0 con 0x7f6c08100c80 2024-03-30T02:34:52.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.991+0000 7f6c05600700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6bf00a11a0 0x7f6bf00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:52.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.991+0000 7f6c05600700 1 -- 172.21.15.67:0/2440274978 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6bfc0cd8e0 con 0x7f6c08100c80 2024-03-30T02:34:52.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.991+0000 7f6c07e00700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6bf00a11a0 0x7f6bf00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:52.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.992+0000 7f6c07e00700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6bf00a11a0 0x7f6bf00a3660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009c60 tx=0x7f6bf4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:52.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:52.995+0000 7f6c05600700 1 -- 172.21.15.67:0/2440274978 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6c08067a80 con 0x7f6c08100c80 2024-03-30T02:34:53.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:52 smithi067 ceph-mon[37808]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-30T02:34:53.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:52 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2617310720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:53.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:52 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/168534549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:53.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:52 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3637464849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:53.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:52 smithi170 ceph-mon[78798]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-30T02:34:53.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:52 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2617310720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:53.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:52 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/168534549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:53.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:52 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3637464849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:53.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:52 smithi082 ceph-mon[79176]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-30T02:34:53.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:52 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2617310720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:53.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:52 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/168534549' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:53.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:52 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3637464849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:53.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.259+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c08067a80 con 0x7f6c08100c80 2024-03-30T02:34:53.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.260+0000 7f6c05600700 1 -- 172.21.15.67:0/2440274978 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10798 (secure 0 0 0) 0x7f6c08067a80 con 0x7f6c08100c80 2024-03-30T02:34:53.260 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:53.260 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24512,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24514,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24541,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24560,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:34:51.268448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-30T02:34:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6bf00a11a0 msgr2=0x7f6bf00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6bf00a11a0 0x7f6bf00a3660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009c60 tx=0x7f6bf4009260 comp rx=0 tx=0).stop 2024-03-30T02:34:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c08100c80 msgr2=0x7f6c08074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c08100c80 0x7f6c08074b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc000c00 tx=0x7f6bfc00f690 comp rx=0 tx=0).stop 2024-03-30T02:34:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 shutdown_connections 2024-03-30T02:34:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6bf00a11a0 0x7f6bf00a3660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c08102810 0x7f6c080736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c08100c80 0x7f6c08074b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 --2- 172.21.15.67:0/2440274978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08101e70 0x7f6c080731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 >> 172.21.15.67:0/2440274978 conn(0x7f6c080fc1f0 msgr2=0x7f6c08105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:53.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 shutdown_connections 2024-03-30T02:34:53.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.263+0000 7f6c0f800700 1 -- 172.21.15.67:0/2440274978 wait complete. 2024-03-30T02:34:53.264 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-03-30T02:34:53.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:34:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.480+0000 7f35e4a00700 1 -- 172.21.15.67:0/3328229615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 msgr2=0x7f35e01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.480+0000 7f35e4a00700 1 --2- 172.21.15.67:0/3328229615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 0x7f35e01010a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f35d00099c0 tx=0x7f35d0009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.480+0000 7f35e4a00700 1 -- 172.21.15.67:0/3328229615 shutdown_connections 2024-03-30T02:34:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.480+0000 7f35e4a00700 1 --2- 172.21.15.67:0/3328229615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 0x7f35e01010a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.480+0000 7f35e4a00700 1 --2- 172.21.15.67:0/3328229615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e0102810 0x7f35e01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.480+0000 7f35e4a00700 1 --2- 172.21.15.67:0/3328229615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0101e70 0x7f35e01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.480+0000 7f35e4a00700 1 -- 172.21.15.67:0/3328229615 >> 172.21.15.67:0/3328229615 conn(0x7f35e00fc1f0 msgr2=0x7f35e00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.480+0000 7f35e4a00700 1 -- 172.21.15.67:0/3328229615 shutdown_connections 2024-03-30T02:34:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.481+0000 7f35e4a00700 1 -- 172.21.15.67:0/3328229615 wait complete. 2024-03-30T02:34:53.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.482+0000 7f35e4a00700 1 Processor -- start 2024-03-30T02:34:53.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.482+0000 7f35e4a00700 1 -- start start 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35e4a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 0x7f35e0196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35e4a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e0101e70 0x7f35e01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35e4a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102810 0x7f35e019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35e4a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e019e760 con 0x7f35e0100c80 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35e4a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e019e8d0 con 0x7f35e0102810 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35e4a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e019ea90 con 0x7f35e0101e70 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35dde00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 0x7f35e0196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35dde00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 0x7f35e0196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42692/0 (socket says 172.21.15.67:42692) 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35dde00700 1 -- 172.21.15.67:0/2726161456 learned_addr learned my addr 172.21.15.67:0/2726161456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:53.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35de800700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102810 0x7f35e019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35dd400700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e0101e70 0x7f35e01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35dde00700 1 -- 172.21.15.67:0/2726161456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e0101e70 msgr2=0x7f35e01971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35dde00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e0101e70 0x7f35e01971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35dde00700 1 -- 172.21.15.67:0/2726161456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102810 msgr2=0x7f35e019e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35dde00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102810 0x7f35e019e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.483+0000 7f35dde00700 1 -- 172.21.15.67:0/2726161456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00095f0 con 0x7f35e0100c80 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.484+0000 7f35dde00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 0x7f35e0196c70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f35d0003730 tx=0x7f35d000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.484+0000 7f35d6a00700 1 -- 172.21.15.67:0/2726161456 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d001d070 con 0x7f35e0100c80 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.484+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e019ed10 con 0x7f35e0100c80 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.484+0000 7f35d6a00700 1 -- 172.21.15.67:0/2726161456 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35d000bda0 con 0x7f35e0100c80 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.484+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e019f140 con 0x7f35e0100c80 2024-03-30T02:34:53.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.484+0000 7f35d6a00700 1 -- 172.21.15.67:0/2726161456 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d00173f0 con 0x7f35e0100c80 2024-03-30T02:34:53.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.485+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e0108150 con 0x7f35e0100c80 2024-03-30T02:34:53.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.487+0000 7f35d6a00700 1 -- 172.21.15.67:0/2726161456 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f35d0022470 con 0x7f35e0100c80 2024-03-30T02:34:53.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.488+0000 7f35d6a00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35c40a1120 0x7f35c40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.488+0000 7f35d6a00700 1 -- 172.21.15.67:0/2726161456 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f35d00cdb90 con 0x7f35e0100c80 2024-03-30T02:34:53.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.488+0000 7f35dd400700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35c40a1120 0x7f35c40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.489+0000 7f35dd400700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35c40a1120 0x7f35c40a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009c60 tx=0x7f35cc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:53.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.492+0000 7f35d6a00700 1 -- 172.21.15.67:0/2726161456 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f35d0097160 con 0x7f35e0100c80 2024-03-30T02:34:53.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.753+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35e0067a10 con 0x7f35e0100c80 2024-03-30T02:34:53.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.755+0000 7f35d6a00700 1 -- 172.21.15.67:0/2726161456 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f35d009a780 con 0x7f35e0100c80 2024-03-30T02:34:53.755 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:34:53.755 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:51.268204+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:34:53.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35c40a1120 msgr2=0x7f35c40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35c40a1120 0x7f35c40a35e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009c60 tx=0x7f35cc009260 comp rx=0 tx=0).stop 2024-03-30T02:34:53.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 msgr2=0x7f35e0196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 0x7f35e0196c70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f35d0003730 tx=0x7f35d000fa00 comp rx=0 tx=0).stop 2024-03-30T02:34:53.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 shutdown_connections 2024-03-30T02:34:53.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35c40a1120 0x7f35c40a35e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35e0100c80 0x7f35e0196c70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35e0102810 0x7f35e019e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 --2- 172.21.15.67:0/2726161456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35e0101e70 0x7f35e01971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 >> 172.21.15.67:0/2726161456 conn(0x7f35e00fc1f0 msgr2=0x7f35e0105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:53.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 shutdown_connections 2024-03-30T02:34:53.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.757+0000 7f35e4a00700 1 -- 172.21.15.67:0/2726161456 wait complete. 2024-03-30T02:34:53.781 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T02:34:46.713132+0000', 'modified': '2024-03-30T02:34:51.268448+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6839/604601206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 604601206}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 604601206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-30T02:34:53.781 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T02:34:53.781 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-30T02:34:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.979+0000 7f347d400700 1 -- 172.21.15.67:0/3825512711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478101f70 msgr2=0x7f34781023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.979+0000 7f347d400700 1 --2- 172.21.15.67:0/3825512711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478101f70 0x7f34781023d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f34640099c0 tx=0x7f3464009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.979+0000 7f347d400700 1 -- 172.21.15.67:0/3825512711 shutdown_connections 2024-03-30T02:34:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.979+0000 7f347d400700 1 --2- 172.21.15.67:0/3825512711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3478102910 0x7f34781091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.980+0000 7f347d400700 1 --2- 172.21.15.67:0/3825512711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3478100d80 0x7f34781011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.980+0000 7f347d400700 1 --2- 172.21.15.67:0/3825512711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478101f70 0x7f34781023d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.980+0000 7f347d400700 1 -- 172.21.15.67:0/3825512711 >> 172.21.15.67:0/3825512711 conn(0x7f34780fc2f0 msgr2=0x7f34780fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.980+0000 7f347d400700 1 -- 172.21.15.67:0/3825512711 shutdown_connections 2024-03-30T02:34:53.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.980+0000 7f347d400700 1 -- 172.21.15.67:0/3825512711 wait complete. 2024-03-30T02:34:53.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f347d400700 1 Processor -- start 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f347d400700 1 -- start start 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f347d400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3478100d80 0x7f3478196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f347d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3478101f70 0x7f3478197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f347d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478102910 0x7f347819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f347d400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347819e830 con 0x7f3478102910 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f347d400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347819e9a0 con 0x7f3478101f70 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f347d400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347819eb60 con 0x7f3478100d80 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f3477200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478102910 0x7f347819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f3477200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478102910 0x7f347819e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42712/0 (socket says 172.21.15.67:42712) 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f3477200700 1 -- 172.21.15.67:0/4247455349 learned_addr learned my addr 172.21.15.67:0/4247455349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f3476800700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3478100d80 0x7f3478196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.981+0000 7f3475e00700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3478101f70 0x7f3478197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f3477200700 1 -- 172.21.15.67:0/4247455349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3478100d80 msgr2=0x7f3478196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f3477200700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3478100d80 0x7f3478196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f3477200700 1 -- 172.21.15.67:0/4247455349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3478101f70 msgr2=0x7f3478197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f3477200700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3478101f70 0x7f3478197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f3477200700 1 -- 172.21.15.67:0/4247455349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3468009520 con 0x7f3478102910 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f3477200700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478102910 0x7f347819e260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f347000ef30 tx=0x7f347000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f346f400700 1 -- 172.21.15.67:0/4247455349 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470018070 con 0x7f3478102910 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f346f400700 1 -- 172.21.15.67:0/4247455349 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f347000f040 con 0x7f3478102910 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f347d400700 1 -- 172.21.15.67:0/4247455349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34640095f0 con 0x7f3478102910 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.982+0000 7f346f400700 1 -- 172.21.15.67:0/4247455349 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347000cd00 con 0x7f3478102910 2024-03-30T02:34:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.983+0000 7f347d400700 1 -- 172.21.15.67:0/4247455349 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347819f130 con 0x7f3478102910 2024-03-30T02:34:53.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.986+0000 7f346f400700 1 -- 172.21.15.67:0/4247455349 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f3470003a00 con 0x7f3478102910 2024-03-30T02:34:53.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f3490f0db80 1 Processor -- start 2024-03-30T02:34:53.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f3490f0db80 1 -- start start 2024-03-30T02:34:53.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f346f400700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f345c0a5530 0x7f345c0a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f346f400700 1 -- 172.21.15.67:0/4247455349 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f34700cc620 con 0x7f3478102910 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f3f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f3475e00700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f345c0a5530 0x7f345c0a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 0x56185f3f6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f3490f0db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f34c940 con 0x56185f3f50e0 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f3490f0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f34a300 con 0x56185f3f5f80 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.987+0000 7f3490f0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f345bf0 con 0x56185f3f4780 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3476800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3475e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f3f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3476800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:46690/0 (socket says 172.21.15.67:46690) 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3476800700 1 -- 172.21.15.67:0/3669554317 learned_addr learned my addr 172.21.15.67:0/3669554317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3477200700 1 --2- 172.21.15.67:0/3669554317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 0x56185f3f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3476800700 1 -- 172.21.15.67:0/3669554317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 msgr2=0x56185f3f6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:34:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3476800700 1 --2- 172.21.15.67:0/3669554317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 0x56185f3f6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3476800700 1 -- 172.21.15.67:0/3669554317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 msgr2=0x56185f3f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:34:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3476800700 1 --2- 172.21.15.67:0/3669554317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f3f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3476800700 1 -- 172.21.15.67:0/3669554317 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56185f1c7360 con 0x56185f3f4780 2024-03-30T02:34:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.988+0000 7f3475e00700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f345c0a5530 0x7f345c0a79f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3464006010 tx=0x7f346400b790 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.989+0000 7f3476800700 1 --2- 172.21.15.67:0/3669554317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4ba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3468009e70 tx=0x7f3468011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2abcd2a3fa5f7e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.989+0000 7f3463400700 1 -- 172.21.15.67:0/3669554317 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468004310 con 0x56185f3f4780 2024-03-30T02:34:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.989+0000 7f3463400700 1 -- 172.21.15.67:0/3669554317 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3468004470 con 0x56185f3f4780 2024-03-30T02:34:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.989+0000 7f3463400700 1 -- 172.21.15.67:0/3669554317 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680198f0 con 0x56185f3f4780 2024-03-30T02:34:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.989+0000 7f3490f0db80 1 -- 172.21.15.67:0/3669554317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 msgr2=0x56185f3f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.989+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3669554317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4ba0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3468009e70 tx=0x7f3468011720 comp rx=0 tx=0).stop 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.990+0000 7f3490f0db80 1 -- 172.21.15.67:0/3669554317 shutdown_connections 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.990+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3669554317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 0x56185f3f6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.990+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3669554317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4ba0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.990+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3669554317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f3f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.990+0000 7f3490f0db80 1 -- 172.21.15.67:0/3669554317 >> 172.21.15.67:0/3669554317 conn(0x56185f353650 msgr2=0x56185f3ef950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.990+0000 7f3490f0db80 1 -- 172.21.15.67:0/3669554317 shutdown_connections 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.990+0000 7f3490f0db80 1 -- 172.21.15.67:0/3669554317 wait complete. 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.990+0000 7f3490f0db80 1 -- start start 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.990+0000 7f3490f0db80 1 Processor -- start 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f3fe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f3feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 0x56185f405b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3490f0db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f2e1d30 con 0x56185f3f4780 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3490f0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f2e19d0 con 0x56185f3f50e0 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3490f0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f1c9650 con 0x56185f3f5f80 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3490f0db80 10 client.0 _ll_register_callbacks cb 0x56185f2eb420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3476800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f3fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3475e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f3feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3475e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f3feb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:36422/0 (socket says 172.21.15.67:36422) 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3476800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f3fe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42720/0 (socket says 172.21.15.67:42720) 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3475e00700 1 -- 172.21.15.67:0/3693076678 learned_addr learned my addr 172.21.15.67:0/3693076678 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3477200700 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 0x56185f405b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3476800700 1 -- 172.21.15.67:0/3693076678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 msgr2=0x56185f405b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3476800700 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 0x56185f405b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3476800700 1 -- 172.21.15.67:0/3693076678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 msgr2=0x56185f3feb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3476800700 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f3feb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3476800700 1 -- 172.21.15.67:0/3693076678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3468009ad0 con 0x56185f3f4780 2024-03-30T02:34:53.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.991+0000 7f3475e00700 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f3feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.992+0000 7f3476800700 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f3fe610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3468011a70 tx=0x7f3468004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41eaadac24f40d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.992+0000 7f3457e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.992+0000 7f3457e00700 1 -- 172.21.15.67:0/3693076678 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468003890 con 0x56185f3f4780 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.992+0000 7f3457e00700 1 -- 172.21.15.67:0/3693076678 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f34680039f0 con 0x56185f3f4780 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.992+0000 7f3490f0db80 1 -- 172.21.15.67:0/3693076678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56185f306330 con 0x56185f3f4780 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.992+0000 7f3457e00700 1 -- 172.21.15.67:0/3693076678 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3468003bb0 con 0x56185f3f4780 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.992+0000 7f3490f0db80 10 client.14634 fetch_fsmap learned FSMap version 17 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.992+0000 7f3490f0db80 1 -- 172.21.15.67:0/3693076678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56185f173b10 con 0x56185f3f4780 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.993+0000 7f3457e00700 1 -- 172.21.15.67:0/3693076678 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56185f306330 con 0x56185f3f4780 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.993+0000 7f3457e00700 1 -- 172.21.15.67:0/3693076678 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f346801a780 con 0x56185f3f4780 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.993+0000 7f3490f0db80 10 client.14634 fetch_fsmap finished waiting for FSMap version 17 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.993+0000 7f3490f0db80 10 client.14634 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.993+0000 7f3490f0db80 20 client.14634 populate_metadata read hostname 'smithi067' 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.993+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56185f409730 0x56185f40ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:53.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.993+0000 7f3490f0db80 4 client.14634 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:34:53.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.993+0000 7f3490f0db80 1 -- 172.21.15.67:0/3693076678 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56185f33e6b0 con 0x56185f409730 2024-03-30T02:34:53.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.993+0000 7f3475e00700 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56185f409730 0x56185f40ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.994+0000 7f3475e00700 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56185f409730 0x56185f40ea50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16a3b600519fc83 server_cookie=4911c1a7b53c0277 in_seq=0 out_seq=0 2024-03-30T02:34:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.994+0000 7f3457e00700 10 client.14634 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:34:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.995+0000 7f3457e00700 1 -- 172.21.15.67:0/3693076678 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56185f33e6b0 con 0x56185f409730 2024-03-30T02:34:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.995+0000 7f3457e00700 10 client.14634 handle_command_reply: tid=0 2024-03-30T02:34:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.995+0000 7f3490f0db80 1 client.14634 shutdown 2024-03-30T02:34:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 -- 172.21.15.67:0/3693076678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 msgr2=0x56185f3fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f3fe610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3468011a70 tx=0x7f3468004670 comp rx=0 tx=0).stop 2024-03-30T02:34:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 -- 172.21.15.67:0/3693076678 shutdown_connections 2024-03-30T02:34:53.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56185f409730 0x56185f40ea50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f3fe610 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 0x56185f405b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 --2- 172.21.15.67:0/3693076678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f3feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3457e00700 0 client.14634 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:34:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 -- 172.21.15.67:0/3693076678 >> 172.21.15.67:0/3693076678 conn(0x56185f353650 msgr2=0x56185f3f3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 -- 172.21.15.67:0/3693076678 shutdown_connections 2024-03-30T02:34:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 1 -- 172.21.15.67:0/3693076678 wait complete. 2024-03-30T02:34:53.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:53.996+0000 7f3490f0db80 20 client.14634 trim_cache size 0 max 16384 2024-03-30T02:34:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.014+0000 7f3490f0db80 1 Processor -- start 2024-03-30T02:34:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.014+0000 7f3490f0db80 1 -- start start 2024-03-30T02:34:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.014+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.014+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f405b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.014+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 0x56185f3f6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.014+0000 7f3490f0db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f3f98a0 con 0x56185f3f50e0 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.014+0000 7f3490f0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f303ad0 con 0x56185f3f5f80 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.014+0000 7f3490f0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f33e990 con 0x56185f3f4780 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3475e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f405b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3475e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f405b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42732/0 (socket says 172.21.15.67:42732) 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3475e00700 1 -- 172.21.15.67:0/1484060473 learned_addr learned my addr 172.21.15.67:0/1484060473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3476800700 1 --2- 172.21.15.67:0/1484060473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3477200700 1 --2- 172.21.15.67:0/1484060473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 0x56185f3f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3475e00700 1 -- 172.21.15.67:0/1484060473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 msgr2=0x56185f3f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3475e00700 1 --2- 172.21.15.67:0/1484060473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3475e00700 1 -- 172.21.15.67:0/1484060473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 msgr2=0x56185f3f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3475e00700 1 --2- 172.21.15.67:0/1484060473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 0x56185f3f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3475e00700 1 -- 172.21.15.67:0/1484060473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56185f30be50 con 0x56185f3f50e0 2024-03-30T02:34:54.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3475e00700 1 --2- 172.21.15.67:0/1484060473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f405b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3464005b00 tx=0x7f346402b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9416ff7a93962f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:54.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.015+0000 7f3463e00700 1 -- 172.21.15.67:0/1484060473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640343e0 con 0x56185f3f50e0 2024-03-30T02:34:54.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3463e00700 1 -- 172.21.15.67:0/1484060473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f346401e070 con 0x56185f3f50e0 2024-03-30T02:34:54.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3463e00700 1 -- 172.21.15.67:0/1484060473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346400fe50 con 0x56185f3f50e0 2024-03-30T02:34:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3490f0db80 1 -- 172.21.15.67:0/1484060473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 msgr2=0x56185f405b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3490f0db80 1 --2- 172.21.15.67:0/1484060473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f405b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3464005b00 tx=0x7f346402b000 comp rx=0 tx=0).stop 2024-03-30T02:34:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3490f0db80 1 -- 172.21.15.67:0/1484060473 shutdown_connections 2024-03-30T02:34:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3490f0db80 1 --2- 172.21.15.67:0/1484060473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f5f80 0x56185f3f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3490f0db80 1 --2- 172.21.15.67:0/1484060473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f4780 0x56185f3f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3490f0db80 1 --2- 172.21.15.67:0/1484060473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f50e0 0x56185f405b50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3490f0db80 1 -- 172.21.15.67:0/1484060473 >> 172.21.15.67:0/1484060473 conn(0x56185f353650 msgr2=0x56185f3f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3490f0db80 1 -- 172.21.15.67:0/1484060473 shutdown_connections 2024-03-30T02:34:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.016+0000 7f3490f0db80 1 -- 172.21.15.67:0/1484060473 wait complete. 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.017+0000 7f3490f0db80 1 -- start start 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.017+0000 7f3490f0db80 1 Processor -- start 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.017+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f405910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.017+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f402400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.017+0000 7f3490f0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 0x56185f402940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.017+0000 7f3490f0db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f402ed0 con 0x56185f3f4780 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.017+0000 7f3490f0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f403040 con 0x56185f3f50e0 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.017+0000 7f3490f0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56185f45e090 con 0x56185f3f5f80 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.017+0000 7f3490f0db80 10 client.0 _ll_register_callbacks cb 0x56185f2eb420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3476800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f405910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3475e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f402400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3476800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f405910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42746/0 (socket says 172.21.15.67:42746) 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3476800700 1 -- 172.21.15.67:0/2892376507 learned_addr learned my addr 172.21.15.67:0/2892376507 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3475e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f402400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:36436/0 (socket says 172.21.15.67:36436) 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3477200700 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 0x56185f402940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3475e00700 1 -- 172.21.15.67:0/2892376507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 msgr2=0x56185f402940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3475e00700 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 0x56185f402940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3475e00700 1 -- 172.21.15.67:0/2892376507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 msgr2=0x56185f405910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3475e00700 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f405910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3475e00700 1 -- 172.21.15.67:0/2892376507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346402f040 con 0x56185f3f50e0 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3476800700 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f405910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3475e00700 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f402400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3464003730 tx=0x7f3464005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7899cc05934da8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3457e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3457e00700 1 -- 172.21.15.67:0/2892376507 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3464034730 con 0x56185f3f50e0 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.018+0000 7f3490f0db80 1 -- 172.21.15.67:0/2892376507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3468004470 con 0x56185f3f50e0 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3457e00700 1 -- 172.21.15.67:0/2892376507 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f346400fdd0 con 0x56185f3f50e0 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3457e00700 1 -- 172.21.15.67:0/2892376507 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34640343e0 con 0x56185f3f50e0 2024-03-30T02:34:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3490f0db80 10 client.24692 fetch_fsmap learned FSMap version 17 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3490f0db80 1 -- 172.21.15.67:0/2892376507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x56185f1cbc80 con 0x56185f3f50e0 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3457e00700 1 -- 172.21.15.67:0/2892376507 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3468004470 con 0x56185f3f50e0 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3457e00700 1 -- 172.21.15.67:0/2892376507 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f346403fab0 con 0x56185f3f50e0 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3490f0db80 10 client.24692 fetch_fsmap finished waiting for FSMap version 17 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3490f0db80 10 client.24692 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3490f0db80 20 client.24692 populate_metadata read hostname 'smithi067' 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3490f0db80 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56185f462a20 0x56185f45f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3490f0db80 4 client.24692 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.019+0000 7f3490f0db80 1 -- 172.21.15.67:0/2892376507 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56185f194ca0 con 0x56185f462a20 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.020+0000 7f3476800700 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56185f462a20 0x56185f45f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.020+0000 7f3476800700 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56185f462a20 0x56185f45f3d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1496fd1139dadc server_cookie=558bb5f8b8826353 in_seq=0 out_seq=0 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.020+0000 7f3457e00700 10 client.24692 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.020+0000 7f3457e00700 1 -- 172.21.15.67:0/2892376507 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56185f194ca0 con 0x56185f462a20 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.020+0000 7f3457e00700 10 client.24692 handle_command_reply: tid=0 2024-03-30T02:34:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.020+0000 7f3490f0db80 1 client.24692 shutdown 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.020+0000 7f3490f0db80 1 -- 172.21.15.67:0/2892376507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 msgr2=0x56185f402400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.020+0000 7f3490f0db80 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f402400 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3464003730 tx=0x7f3464005f00 comp rx=0 tx=0).stop 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3490f0db80 1 -- 172.21.15.67:0/2892376507 shutdown_connections 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3490f0db80 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56185f462a20 0x56185f45f3d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3490f0db80 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56185f3f4780 0x56185f405910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3490f0db80 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56185f3f5f80 0x56185f402940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3490f0db80 1 --2- 172.21.15.67:0/2892376507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56185f3f50e0 0x56185f402400 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3457e00700 0 client.24692 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3490f0db80 1 -- 172.21.15.67:0/2892376507 >> 172.21.15.67:0/2892376507 conn(0x56185f353650 msgr2=0x56185f3fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3490f0db80 1 -- 172.21.15.67:0/2892376507 shutdown_connections 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3490f0db80 1 -- 172.21.15.67:0/2892376507 wait complete. 2024-03-30T02:34:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f3490f0db80 20 client.24692 trim_cache size 0 max 16384 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24578, 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 7.303366477, 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 17, 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 83, 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 82, 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 12.657835667000001 2024-03-30T02:34:54.026 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:34:54.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f347d400700 1 -- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f345c0a5530 msgr2=0x7f345c0a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:54.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f347d400700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f345c0a5530 0x7f345c0a79f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3464006010 tx=0x7f346400b790 comp rx=0 tx=0).stop 2024-03-30T02:34:54.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f347d400700 1 -- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478102910 msgr2=0x7f347819e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:54.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.021+0000 7f347d400700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478102910 0x7f347819e260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f347000ef30 tx=0x7f347000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.022+0000 7f347d400700 1 -- 172.21.15.67:0/4247455349 shutdown_connections 2024-03-30T02:34:54.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.022+0000 7f347d400700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f345c0a5530 0x7f345c0a79f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.022+0000 7f347d400700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3478102910 0x7f347819e260 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.022+0000 7f347d400700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3478100d80 0x7f3478196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.022+0000 7f347d400700 1 --2- 172.21.15.67:0/4247455349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3478101f70 0x7f3478197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:54.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.022+0000 7f347d400700 1 -- 172.21.15.67:0/4247455349 >> 172.21.15.67:0/4247455349 conn(0x7f34780fc2f0 msgr2=0x7f3478105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:54.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.022+0000 7f347d400700 1 -- 172.21.15.67:0/4247455349 shutdown_connections 2024-03-30T02:34:54.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:54.022+0000 7f347d400700 1 -- 172.21.15.67:0/4247455349 wait complete. 2024-03-30T02:34:54.038 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:53 smithi067 ceph-mon[37808]: pgmap v236: 129 pgs: 8 creating+peering, 5 unknown, 116 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 479 B/s wr, 7 op/s 2024-03-30T02:34:54.038 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:53 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2440274978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:54.038 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:53 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2726161456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:54.040 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T02:34:54.040 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T02:34:54.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-03-30T02:34:54.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:53 smithi170 ceph-mon[78798]: pgmap v236: 129 pgs: 8 creating+peering, 5 unknown, 116 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 479 B/s wr, 7 op/s 2024-03-30T02:34:54.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:53 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2440274978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:54.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:53 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2726161456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:54.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:53 smithi082 ceph-mon[79176]: pgmap v236: 129 pgs: 8 creating+peering, 5 unknown, 116 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 479 B/s wr, 7 op/s 2024-03-30T02:34:54.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:53 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2440274978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:34:54.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:53 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2726161456' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:34:54.408 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:34:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:54 smithi067 ceph-mon[37808]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-30T02:34:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:54 smithi067 ceph-mon[37808]: Cluster is now healthy 2024-03-30T02:34:55.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:54 smithi170 ceph-mon[78798]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-30T02:34:55.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:54 smithi170 ceph-mon[78798]: Cluster is now healthy 2024-03-30T02:34:55.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:54 smithi082 ceph-mon[79176]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-30T02:34:55.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:54 smithi082 ceph-mon[79176]: Cluster is now healthy 2024-03-30T02:34:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.603+0000 7f4be6c00700 1 -- 172.21.15.67:0/3533319542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0106ef0 msgr2=0x7f4be01072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.603+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3533319542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0106ef0 0x7f4be01072d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00099c0 tx=0x7f4bd0009cd0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.603+0000 7f4be6c00700 1 -- 172.21.15.67:0/3533319542 shutdown_connections 2024-03-30T02:34:55.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.603+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3533319542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0106ef0 0x7f4be01072d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.603+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3533319542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4be0101820 0x7f4be010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.603+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3533319542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be0100e60 0x7f4be01012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.603+0000 7f4be6c00700 1 -- 172.21.15.67:0/3533319542 >> 172.21.15.67:0/3533319542 conn(0x7f4be0077fa0 msgr2=0x7f4be00783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:55.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.603+0000 7f4be6c00700 1 -- 172.21.15.67:0/3533319542 shutdown_connections 2024-03-30T02:34:55.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.603+0000 7f4be6c00700 1 -- 172.21.15.67:0/3533319542 wait complete. 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.604+0000 7f4be6c00700 1 Processor -- start 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.604+0000 7f4be6c00700 1 -- start start 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.604+0000 7f4be6c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0100e60 0x7f4be01971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.604+0000 7f4be6c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4be0101820 0x7f4be0197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.604+0000 7f4be6c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be0106ef0 0x7f4be019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.604+0000 7f4be6c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be019c0e0 con 0x7f4be0100e60 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.604+0000 7f4be6c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be019c250 con 0x7f4be0101820 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.604+0000 7f4be6c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be019c410 con 0x7f4be0106ef0 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdfe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0100e60 0x7f4be01971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdfe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0100e60 0x7f4be01971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42770/0 (socket says 172.21.15.67:42770) 2024-03-30T02:34:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdfe00700 1 -- 172.21.15.67:0/3568430590 learned_addr learned my addr 172.21.15.67:0/3568430590 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdf400700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4be0101820 0x7f4be0197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4be4c00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be0106ef0 0x7f4be019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdfe00700 1 -- 172.21.15.67:0/3568430590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be0106ef0 msgr2=0x7f4be019bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdfe00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be0106ef0 0x7f4be019bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdfe00700 1 -- 172.21.15.67:0/3568430590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4be0101820 msgr2=0x7f4be0197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdfe00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4be0101820 0x7f4be0197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdfe00700 1 -- 172.21.15.67:0/3568430590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd00095f0 con 0x7f4be0100e60 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdf400700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4be0101820 0x7f4be0197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.605+0000 7f4bdfe00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0100e60 0x7f4be01971e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b580 tx=0x7f4bd000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.606+0000 7f4bdcc00700 1 -- 172.21.15.67:0/3568430590 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd001c070 con 0x7f4be0100e60 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.606+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be019c690 con 0x7f4be0100e60 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.606+0000 7f4bdcc00700 1 -- 172.21.15.67:0/3568430590 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bd000fd80 con 0x7f4be0100e60 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.606+0000 7f4bdcc00700 1 -- 172.21.15.67:0/3568430590 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0022c40 con 0x7f4be0100e60 2024-03-30T02:34:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.606+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be01a3f50 con 0x7f4be0100e60 2024-03-30T02:34:55.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.607+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be01092d0 con 0x7f4be0100e60 2024-03-30T02:34:55.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.609+0000 7f4bdcc00700 1 -- 172.21.15.67:0/3568430590 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4bd0020070 con 0x7f4be0100e60 2024-03-30T02:34:55.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.610+0000 7f4bdcc00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4bc40a1120 0x7f4bc40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:34:55.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.610+0000 7f4bdcc00700 1 -- 172.21.15.67:0/3568430590 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4bd00cd840 con 0x7f4be0100e60 2024-03-30T02:34:55.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.610+0000 7f4bdf400700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4bc40a1120 0x7f4bc40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:34:55.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.611+0000 7f4bdf400700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4bc40a1120 0x7f4bc40a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4be0198740 tx=0x7f4bcc00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:34:55.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.614+0000 7f4bdcc00700 1 -- 172.21.15.67:0/3568430590 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4bd0096e10 con 0x7f4be0100e60 2024-03-30T02:34:55.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.874+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4be004ead0 con 0x7f4be0100e60 2024-03-30T02:34:55.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.875+0000 7f4bdcc00700 1 -- 172.21.15.67:0/3568430590 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4bd009a430 con 0x7f4be0100e60 2024-03-30T02:34:55.875 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T02:34:55.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4bc40a1120 msgr2=0x7f4bc40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:55.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4bc40a1120 0x7f4bc40a35e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4be0198740 tx=0x7f4bcc00b410 comp rx=0 tx=0).stop 2024-03-30T02:34:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0100e60 msgr2=0x7f4be01971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:34:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0100e60 0x7f4be01971e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b580 tx=0x7f4bd000f690 comp rx=0 tx=0).stop 2024-03-30T02:34:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 shutdown_connections 2024-03-30T02:34:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4bc40a1120 0x7f4bc40a35e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4be0100e60 0x7f4be01971e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be0106ef0 0x7f4be019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 --2- 172.21.15.67:0/3568430590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4be0101820 0x7f4be0197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:34:55.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 >> 172.21.15.67:0/3568430590 conn(0x7f4be0077fa0 msgr2=0x7f4be00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:34:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 shutdown_connections 2024-03-30T02:34:55.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:34:55.877+0000 7f4be6c00700 1 -- 172.21.15.67:0/3568430590 wait complete. 2024-03-30T02:34:56.078 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:55 smithi067 ceph-mon[37808]: pgmap v237: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-03-30T02:34:56.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:55 smithi170 ceph-mon[78798]: pgmap v237: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-03-30T02:34:56.248 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T02:34:56.248 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T02:34:56.248 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-03-30T02:34:56.249 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-30T02:34:56.249 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T02:34:56.249 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T02:34:56.249 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T02:34:56.249 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T02:34:56.249 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T02:34:56.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:56.250 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-03-30T02:34:56.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:55 smithi082 ceph-mon[79176]: pgmap v237: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-03-30T02:34:56.293 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T02:34:56.293 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T02:34:56.293 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T02:34:56.293 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T02:34:56.294 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41828sec preferred_lft 41828sec 2024-03-30T02:34:56.295 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-03-30T02:34:56.295 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-03-30T02:34:56.296 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T02:34:56.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:56.296 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:34:56.296 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:34:56.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-03-30T02:34:56.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-03-30T02:34:56.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-03-30T02:34:56.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T02:34:56.297 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:34:56.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:56.747 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T02:34:56.818 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-03-30T02:34:56.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:56.821 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-03-30T02:34:56.886 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-03-30T02:34:56.886 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T02:34:56.886 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T02:34:56.887 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T02:34:56.887 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T02:34:56.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:56.888 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:34:56.888 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:34:56.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T02:34:56.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T02:34:56.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T02:34:56.888 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:34:56.986 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:56 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3568430590' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-30T02:34:57.189 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:56 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3568430590' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-30T02:34:57.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:56 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3568430590' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-30T02:34:57.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:57.606 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-03-30T02:34:57.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:57.649 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-03-30T02:34:57.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:57.731 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:34:57.732 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:34:57.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T02:34:57.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-30T02:34:57.732 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:34:57.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:34:57 smithi067 ceph-mon[37808]: pgmap v238: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 141 B/s wr, 1 op/s 2024-03-30T02:34:57.911 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-30T02:34:57.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:57.911 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:34:57.911 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:34:57.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-30T02:34:57.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T02:34:57.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-30T02:34:57.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-30T02:34:57.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-30T02:34:57.912 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:34:58.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:58.184 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:34:58.184 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:34:58.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-03-30T02:34:58.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T02:34:58.184 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:34:58.190 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:34:57 smithi170 ceph-mon[78798]: pgmap v238: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 141 B/s wr, 1 op/s 2024-03-30T02:34:58.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:34:57 smithi082 ceph-mon[79176]: pgmap v238: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 141 B/s wr, 1 op/s 2024-03-30T02:34:58.355 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-30T02:34:58.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T02:34:58.355 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-30T02:34:58.397 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T02:34:58.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T02:34:58.398 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-30T02:34:58.479 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-03-30T02:34:58.586 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T02:34:58.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T02:34:58.652 INFO:teuthology.orchestra.run.smithi067.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T02:34:58.654 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T02:34:58.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:58.655 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-03-30T02:34:58.697 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T02:34:58.697 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-30T02:34:58.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-03-30T02:34:58.813 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T02:34:58.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T02:34:58.880 INFO:teuthology.orchestra.run.smithi067.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T02:34:58.883 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T02:34:58.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:34:58.884 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-03-30T02:34:58.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:2024-03-30T02:34:58.927+0000 7f6dbf44f5c0 -1 init, newargv = 0x55e2cd6021c0 newargc=15 2024-03-30T02:34:58.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[90111]: starting ceph client 2024-03-30T02:34:58.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[90111]: starting fuse 2024-03-30T02:35:00.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-03-30T02:35:00.103 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-30T02:35:00.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-30T02:35:00.171 INFO:teuthology.orchestra.run.smithi067.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-30T02:35:00.174 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-30T02:35:00.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:35:00.174 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-03-30T02:35:00.216 INFO:teuthology.orchestra.run.smithi067.stdout:122 2024-03-30T02:35:00.216 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [122] 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-03-30T02:35:00.217 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.admin")) 2024-03-30T02:35:00.218 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-03-30T02:35:00.314 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:00 smithi067 ceph-mon[37808]: pgmap v239: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-30T02:35:00.400 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.admin.90111.asok 2024-03-30T02:35:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:00 smithi170 ceph-mon[78798]: pgmap v239: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-30T02:35:00.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:00 smithi082 ceph-mon[79176]: pgmap v239: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-30T02:35:00.577 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.90111.asok 2024-03-30T02:35:00.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:35:00.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.90111.asok status 2024-03-30T02:35:00.807 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:35:00.807 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-03-30T02:35:00.807 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "470100a4ccbcf7769052bf09a5413881ccf1b97f", 2024-03-30T02:35:00.807 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 18.2.2-1018-g470100a4 (470100a4ccbcf7769052bf09a5413881ccf1b97f) reef (stable)", 2024-03-30T02:35:00.807 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "admin", 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "90111", 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24704, 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 24704 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-03-30T02:35:00.808 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 151979686 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 151979686 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.24704 172.21.15.67:0/151979686", 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "172.21.15.67:0/151979686", 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 17, 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 83, 2024-03-30T02:35:00.809 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-03-30T02:35:00.810 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-03-30T02:35:00.810 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-03-30T02:35:00.810 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:35:00.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:35:00.822 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-03-30T02:35:00.822 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-30T02:35:00.865 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-03-30T02:35:00.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-30T02:35:00.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:35:00.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-30T02:35:00.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:35:00.972 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-30T02:35:01.090 INFO:teuthology.orchestra.run.smithi067.stdout:# file: . 2024-03-30T02:35:01.090 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-30T02:35:01.090 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:01.091 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:35:01.091 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T02:35:01.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:35:01.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.372+0000 7f860de00700 1 -- 172.21.15.67:0/3126925349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608101e70 msgr2=0x7f86081022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.372+0000 7f860de00700 1 --2- 172.21.15.67:0/3126925349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608101e70 0x7f86081022d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f85f40099c0 tx=0x7f85f4009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.372+0000 7f860de00700 1 -- 172.21.15.67:0/3126925349 shutdown_connections 2024-03-30T02:35:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.372+0000 7f860de00700 1 --2- 172.21.15.67:0/3126925349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8608102810 0x7f86081090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.372+0000 7f860de00700 1 --2- 172.21.15.67:0/3126925349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608100c80 0x7f86081010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.372+0000 7f860de00700 1 --2- 172.21.15.67:0/3126925349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608101e70 0x7f86081022d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.372+0000 7f860de00700 1 -- 172.21.15.67:0/3126925349 >> 172.21.15.67:0/3126925349 conn(0x7f86080fc1f0 msgr2=0x7f86080fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.372+0000 7f860de00700 1 -- 172.21.15.67:0/3126925349 shutdown_connections 2024-03-30T02:35:01.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.372+0000 7f860de00700 1 -- 172.21.15.67:0/3126925349 wait complete. 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.373+0000 7f860de00700 1 Processor -- start 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.373+0000 7f860de00700 1 -- start start 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f860de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608100c80 0x7f8608198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f860de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8608101e70 0x7f86081993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f860de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608102810 0x7f86081a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f860de00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081a0960 con 0x7f8608100c80 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f860de00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081a0aa0 con 0x7f8608102810 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f860de00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081a0c30 con 0x7f8608101e70 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8606a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608100c80 0x7f8608198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8607400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608102810 0x7f86081a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8606a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608100c80 0x7f8608198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42056/0 (socket says 172.21.15.67:42056) 2024-03-30T02:35:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8606a00700 1 -- 172.21.15.67:0/973112306 learned_addr learned my addr 172.21.15.67:0/973112306 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8606000700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8608101e70 0x7f86081993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8606a00700 1 -- 172.21.15.67:0/973112306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8608101e70 msgr2=0x7f86081993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8606a00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8608101e70 0x7f86081993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8606a00700 1 -- 172.21.15.67:0/973112306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608102810 msgr2=0x7f86081a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8606a00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608102810 0x7f86081a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.374+0000 7f8606a00700 1 -- 172.21.15.67:0/973112306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40095f0 con 0x7f8608100c80 2024-03-30T02:35:01.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.375+0000 7f8606a00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608100c80 0x7f8608198e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f85f800e970 tx=0x7f85f800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.375+0000 7f860ca00700 1 -- 172.21.15.67:0/973112306 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8009790 con 0x7f8608100c80 2024-03-30T02:35:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.375+0000 7f860ca00700 1 -- 172.21.15.67:0/973112306 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85f800c7e0 con 0x7f8608100c80 2024-03-30T02:35:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.375+0000 7f860ca00700 1 -- 172.21.15.67:0/973112306 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8003c70 con 0x7f8608100c80 2024-03-30T02:35:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.375+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86081a0f10 con 0x7f8608100c80 2024-03-30T02:35:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.376+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86081a1280 con 0x7f8608100c80 2024-03-30T02:35:01.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.377+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8608067a10 con 0x7f8608100c80 2024-03-30T02:35:01.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.378+0000 7f860ca00700 1 -- 172.21.15.67:0/973112306 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f85f800cc50 con 0x7f8608100c80 2024-03-30T02:35:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.380+0000 7f860ca00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f85ec0a1120 0x7f85ec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.380+0000 7f860ca00700 1 -- 172.21.15.67:0/973112306 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f85f8014070 con 0x7f8608100c80 2024-03-30T02:35:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.380+0000 7f8606000700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f85ec0a1120 0x7f85ec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:01.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.381+0000 7f8606000700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f85ec0a1120 0x7f85ec0a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006010 tx=0x7f85f400b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:01.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.384+0000 7f860ca00700 1 -- 172.21.15.67:0/973112306 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f85f8096250 con 0x7f8608100c80 2024-03-30T02:35:01.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.625+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86081a17f0 con 0x7f8608100c80 2024-03-30T02:35:01.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.627+0000 7f860ca00700 1 -- 172.21.15.67:0/973112306 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f85f8003dd0 con 0x7f8608100c80 2024-03-30T02:35:01.628 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:01.628 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:51.268204+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:35:01.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f85ec0a1120 msgr2=0x7f85ec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:01.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f85ec0a1120 0x7f85ec0a35e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006010 tx=0x7f85f400b540 comp rx=0 tx=0).stop 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608100c80 msgr2=0x7f8608198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608100c80 0x7f8608198e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f85f800e970 tx=0x7f85f800ec80 comp rx=0 tx=0).stop 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 shutdown_connections 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f85ec0a1120 0x7f85ec0a35e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8608100c80 0x7f8608198e80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8608102810 0x7f86081a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 --2- 172.21.15.67:0/973112306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8608101e70 0x7f86081993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 >> 172.21.15.67:0/973112306 conn(0x7f86080fc1f0 msgr2=0x7f8608072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.629+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 shutdown_connections 2024-03-30T02:35:01.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.630+0000 7f860de00700 1 -- 172.21.15.67:0/973112306 wait complete. 2024-03-30T02:35:01.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:35:01.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:35:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.850+0000 7f67a7800700 1 -- 172.21.15.67:0/4294516404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 msgr2=0x7f67a0073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.850+0000 7f67a7800700 1 --2- 172.21.15.67:0/4294516404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 0x7f67a0073c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f678c0098f0 tx=0x7f678c009c00 comp rx=0 tx=0).stop 2024-03-30T02:35:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.850+0000 7f67a7800700 1 -- 172.21.15.67:0/4294516404 shutdown_connections 2024-03-30T02:35:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.850+0000 7f67a7800700 1 --2- 172.21.15.67:0/4294516404 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a0074df0 0x7f67a0073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.850+0000 7f67a7800700 1 --2- 172.21.15.67:0/4294516404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a0108c40 0x7f67a010b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.850+0000 7f67a7800700 1 --2- 172.21.15.67:0/4294516404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 0x7f67a0073c90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.850+0000 7f67a7800700 1 -- 172.21.15.67:0/4294516404 >> 172.21.15.67:0/4294516404 conn(0x7f67a00fc2f0 msgr2=0x7f67a00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.850+0000 7f67a7800700 1 -- 172.21.15.67:0/4294516404 shutdown_connections 2024-03-30T02:35:01.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.850+0000 7f67a7800700 1 -- 172.21.15.67:0/4294516404 wait complete. 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.851+0000 7f67a7800700 1 Processor -- start 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a7800700 1 -- start start 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a7800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 0x7f67a0198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a7800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a0074df0 0x7f67a0199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a7800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a0108c40 0x7f67a01a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a7800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01a0ab0 con 0x7f67a0074df0 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a7800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01a0bf0 con 0x7f67a0073810 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a7800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01a0d80 con 0x7f67a0108c40 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 0x7f67a0198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f679fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a0074df0 0x7f67a0199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 0x7f67a0198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:50832/0 (socket says 172.21.15.67:50832) 2024-03-30T02:35:01.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a4e00700 1 -- 172.21.15.67:0/201550350 learned_addr learned my addr 172.21.15.67:0/201550350 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.852+0000 7f67a5800700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a0108c40 0x7f67a01a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f679fe00700 1 -- 172.21.15.67:0/201550350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a0108c40 msgr2=0x7f67a01a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f679fe00700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a0108c40 0x7f67a01a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f679fe00700 1 -- 172.21.15.67:0/201550350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 msgr2=0x7f67a0198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f679fe00700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 0x7f67a0198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f679fe00700 1 -- 172.21.15.67:0/201550350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67940095f0 con 0x7f67a0074df0 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f67a5800700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a0108c40 0x7f67a01a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f67a4e00700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 0x7f67a0198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f679fe00700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a0074df0 0x7f67a0199510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f678c009230 tx=0x7f678c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f679d600700 1 -- 172.21.15.67:0/201550350 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c01c070 con 0x7f67a0074df0 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f679d600700 1 -- 172.21.15.67:0/201550350 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f678c00f7d0 con 0x7f67a0074df0 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c009520 con 0x7f67a0074df0 2024-03-30T02:35:01.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f679d600700 1 -- 172.21.15.67:0/201550350 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c003990 con 0x7f67a0074df0 2024-03-30T02:35:01.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.853+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a01a12f0 con 0x7f67a0074df0 2024-03-30T02:35:01.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.854+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a0107a00 con 0x7f67a0074df0 2024-03-30T02:35:01.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.856+0000 7f679d600700 1 -- 172.21.15.67:0/201550350 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f678c020070 con 0x7f67a0074df0 2024-03-30T02:35:01.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.857+0000 7f679d600700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f67880a11a0 0x7f67880a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:01.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.857+0000 7f679d600700 1 -- 172.21.15.67:0/201550350 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f678c0cd3d0 con 0x7f67a0074df0 2024-03-30T02:35:01.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.858+0000 7f67a4e00700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f67880a11a0 0x7f67880a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:01.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.858+0000 7f67a4e00700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f67880a11a0 0x7f67880a3660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6794005d10 tx=0x7f6794009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:01.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:01.862+0000 7f679d600700 1 -- 172.21.15.67:0/201550350 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f678c0969a0 con 0x7f67a0074df0 2024-03-30T02:35:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.082+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67a010e970 con 0x7f67a0074df0 2024-03-30T02:35:02.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.082+0000 7f679d600700 1 -- 172.21.15.67:0/201550350 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f678c099fc0 con 0x7f67a0074df0 2024-03-30T02:35:02.084 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:35:02.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f67880a11a0 msgr2=0x7f67880a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f67880a11a0 0x7f67880a3660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6794005d10 tx=0x7f6794009310 comp rx=0 tx=0).stop 2024-03-30T02:35:02.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a0074df0 msgr2=0x7f67a0199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a0074df0 0x7f67a0199510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f678c009230 tx=0x7f678c00b790 comp rx=0 tx=0).stop 2024-03-30T02:35:02.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 shutdown_connections 2024-03-30T02:35:02.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f67880a11a0 0x7f67880a3660 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a0108c40 0x7f67a01a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a0073810 0x7f67a0198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 --2- 172.21.15.67:0/201550350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a0074df0 0x7f67a0199510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 >> 172.21.15.67:0/201550350 conn(0x7f67a00fc2f0 msgr2=0x7f67a0104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:02.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 shutdown_connections 2024-03-30T02:35:02.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.085+0000 7f67a7800700 1 -- 172.21.15.67:0/201550350 wait complete. 2024-03-30T02:35:02.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:35:02.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.308+0000 7f5c57600700 1 -- 172.21.15.67:0/1359644286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50108c40 msgr2=0x7f5c5010b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.308+0000 7f5c57600700 1 --2- 172.21.15.67:0/1359644286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50108c40 0x7f5c5010b0d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800b0a0 tx=0x7f5c4800b3b0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.308+0000 7f5c57600700 1 -- 172.21.15.67:0/1359644286 shutdown_connections 2024-03-30T02:35:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.308+0000 7f5c57600700 1 --2- 172.21.15.67:0/1359644286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50108c40 0x7f5c5010b0d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.308+0000 7f5c57600700 1 --2- 172.21.15.67:0/1359644286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c50074df0 0x7f5c50073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.308+0000 7f5c57600700 1 --2- 172.21.15.67:0/1359644286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c50073810 0x7f5c50073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.308+0000 7f5c57600700 1 -- 172.21.15.67:0/1359644286 >> 172.21.15.67:0/1359644286 conn(0x7f5c500fc2f0 msgr2=0x7f5c500fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.308+0000 7f5c57600700 1 -- 172.21.15.67:0/1359644286 shutdown_connections 2024-03-30T02:35:02.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.308+0000 7f5c57600700 1 -- 172.21.15.67:0/1359644286 wait complete. 2024-03-30T02:35:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.309+0000 7f5c57600700 1 Processor -- start 2024-03-30T02:35:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.309+0000 7f5c57600700 1 -- start start 2024-03-30T02:35:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.309+0000 7f5c57600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c50073810 0x7f5c50199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.309+0000 7f5c57600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50074df0 0x7f5c501995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.309+0000 7f5c57600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c50108c40 0x7f5c501a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:02.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.309+0000 7f5c57600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c501a0b50 con 0x7f5c50074df0 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.309+0000 7f5c57600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c501a0c90 con 0x7f5c50108c40 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c57600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c501a0e20 con 0x7f5c50073810 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c4fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50074df0 0x7f5c501995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c54c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c50073810 0x7f5c50199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c4fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50074df0 0x7f5c501995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42082/0 (socket says 172.21.15.67:42082) 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c54c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c50073810 0x7f5c50199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47384/0 (socket says 172.21.15.67:47384) 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c4fe00700 1 -- 172.21.15.67:0/3278885278 learned_addr learned my addr 172.21.15.67:0/3278885278 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c55600700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c50108c40 0x7f5c501a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c4fe00700 1 -- 172.21.15.67:0/3278885278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c50073810 msgr2=0x7f5c50199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c4fe00700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c50073810 0x7f5c50199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c4fe00700 1 -- 172.21.15.67:0/3278885278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c50108c40 msgr2=0x7f5c501a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c4fe00700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c50108c40 0x7f5c501a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c4fe00700 1 -- 172.21.15.67:0/3278885278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009d00 con 0x7f5c50074df0 2024-03-30T02:35:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.310+0000 7f5c4fe00700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50074df0 0x7f5c501995b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e970 tx=0x7f5c3c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.311+0000 7f5c4d600700 1 -- 172.21.15.67:0/3278885278 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c009790 con 0x7f5c50074df0 2024-03-30T02:35:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.311+0000 7f5c4d600700 1 -- 172.21.15.67:0/3278885278 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c3c00c7e0 con 0x7f5c50074df0 2024-03-30T02:35:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.311+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c501a1100 con 0x7f5c50074df0 2024-03-30T02:35:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.311+0000 7f5c4d600700 1 -- 172.21.15.67:0/3278885278 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c003c70 con 0x7f5c50074df0 2024-03-30T02:35:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.311+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c501a1660 con 0x7f5c50074df0 2024-03-30T02:35:02.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.312+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c50107a00 con 0x7f5c50074df0 2024-03-30T02:35:02.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.315+0000 7f5c4d600700 1 -- 172.21.15.67:0/3278885278 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f5c3c00cc50 con 0x7f5c50074df0 2024-03-30T02:35:02.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.316+0000 7f5c4d600700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5c380a11a0 0x7f5c380a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:02.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.317+0000 7f5c4d600700 1 -- 172.21.15.67:0/3278885278 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5c3c014070 con 0x7f5c50074df0 2024-03-30T02:35:02.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.317+0000 7f5c54c00700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5c380a11a0 0x7f5c380a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:02.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.317+0000 7f5c54c00700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5c380a11a0 0x7f5c380a3660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009c60 tx=0x7f5c44009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:02.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.319+0000 7f5c4d600700 1 -- 172.21.15.67:0/3278885278 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5c3c0962d0 con 0x7f5c50074df0 2024-03-30T02:35:02.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:02 smithi067 ceph-mon[37808]: pgmap v240: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-30T02:35:02.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:02 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/973112306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:02.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:02 smithi170 ceph-mon[78798]: pgmap v240: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-30T02:35:02.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:02 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/973112306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:02.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:02 smithi082 ceph-mon[79176]: pgmap v240: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-30T02:35:02.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:02 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/973112306' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:02.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.540+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c5010e7c0 con 0x7f5c50074df0 2024-03-30T02:35:02.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.540+0000 7f5c4d600700 1 -- 172.21.15.67:0/3278885278 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c3c0998f0 con 0x7f5c50074df0 2024-03-30T02:35:02.541 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:35:02.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5c380a11a0 msgr2=0x7f5c380a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5c380a11a0 0x7f5c380a3660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009c60 tx=0x7f5c44009260 comp rx=0 tx=0).stop 2024-03-30T02:35:02.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50074df0 msgr2=0x7f5c501995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50074df0 0x7f5c501995b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e970 tx=0x7f5c3c00ec80 comp rx=0 tx=0).stop 2024-03-30T02:35:02.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 shutdown_connections 2024-03-30T02:35:02.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5c380a11a0 0x7f5c380a3660 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c50108c40 0x7f5c501a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c50073810 0x7f5c50199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 --2- 172.21.15.67:0/3278885278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5c50074df0 0x7f5c501995b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 >> 172.21.15.67:0/3278885278 conn(0x7f5c500fc2f0 msgr2=0x7f5c50104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:02.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 shutdown_connections 2024-03-30T02:35:02.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.543+0000 7f5c57600700 1 -- 172.21.15.67:0/3278885278 wait complete. 2024-03-30T02:35:02.563 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:35:02.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.766+0000 7f6d2d000700 1 -- 172.21.15.67:0/1079361979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d28101f70 msgr2=0x7f6d281023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.766+0000 7f6d2d000700 1 --2- 172.21.15.67:0/1079361979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d28101f70 0x7f6d281023d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d140099c0 tx=0x7f6d14009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.766+0000 7f6d2d000700 1 -- 172.21.15.67:0/1079361979 shutdown_connections 2024-03-30T02:35:02.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.766+0000 7f6d2d000700 1 --2- 172.21.15.67:0/1079361979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28102910 0x7f6d281091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.766+0000 7f6d2d000700 1 --2- 172.21.15.67:0/1079361979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d28100d80 0x7f6d281011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.766+0000 7f6d2d000700 1 --2- 172.21.15.67:0/1079361979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d28101f70 0x7f6d281023d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.766+0000 7f6d2d000700 1 -- 172.21.15.67:0/1079361979 >> 172.21.15.67:0/1079361979 conn(0x7f6d280fc2f0 msgr2=0x7f6d280fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:02.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.767+0000 7f6d2d000700 1 -- 172.21.15.67:0/1079361979 shutdown_connections 2024-03-30T02:35:02.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.767+0000 7f6d2d000700 1 -- 172.21.15.67:0/1079361979 wait complete. 2024-03-30T02:35:02.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d2d000700 1 Processor -- start 2024-03-30T02:35:02.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d2d000700 1 -- start start 2024-03-30T02:35:02.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d2d000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28100d80 0x7f6d28074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:02.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d2d000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d28101f70 0x7f6d280731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:02.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d2d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d28102910 0x7f6d280736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:02.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d2d000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28073e80 con 0x7f6d28101f70 2024-03-30T02:35:02.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d2d000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28073ff0 con 0x7f6d28102910 2024-03-30T02:35:02.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d2d000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281a0d80 con 0x7f6d28100d80 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d25e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28100d80 0x7f6d28074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d25400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d28101f70 0x7f6d280731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d25e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28100d80 0x7f6d28074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47406/0 (socket says 172.21.15.67:47406) 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.768+0000 7f6d25e00700 1 -- 172.21.15.67:0/3474900060 learned_addr learned my addr 172.21.15.67:0/3474900060 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d26800700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d28102910 0x7f6d280736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d25e00700 1 -- 172.21.15.67:0/3474900060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d28102910 msgr2=0x7f6d280736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d25e00700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d28102910 0x7f6d280736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d25e00700 1 -- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d28101f70 msgr2=0x7f6d280731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d25e00700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d28101f70 0x7f6d280731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d25e00700 1 -- 172.21.15.67:0/3474900060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d140095f0 con 0x7f6d28100d80 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d26800700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d28102910 0x7f6d280736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d25400700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d28101f70 0x7f6d280731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:02.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d25e00700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28100d80 0x7f6d28074b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800e970 tx=0x7f6d1800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:02.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.769+0000 7f6d1ea00700 1 -- 172.21.15.67:0/3474900060 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18009790 con 0x7f6d28100d80 2024-03-30T02:35:02.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.770+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d281a0fb0 con 0x7f6d28100d80 2024-03-30T02:35:02.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.770+0000 7f6d1ea00700 1 -- 172.21.15.67:0/3474900060 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d1800eec0 con 0x7f6d28100d80 2024-03-30T02:35:02.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.770+0000 7f6d1ea00700 1 -- 172.21.15.67:0/3474900060 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18003cf0 con 0x7f6d28100d80 2024-03-30T02:35:02.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.770+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d281a13e0 con 0x7f6d28100d80 2024-03-30T02:35:02.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.771+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d28108200 con 0x7f6d28100d80 2024-03-30T02:35:02.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.773+0000 7f6d1ea00700 1 -- 172.21.15.67:0/3474900060 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f6d1800ccd0 con 0x7f6d28100d80 2024-03-30T02:35:02.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.774+0000 7f6d1ea00700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6d0c0a1120 0x7f6d0c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:02.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.774+0000 7f6d1ea00700 1 -- 172.21.15.67:0/3474900060 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6d18014070 con 0x7f6d28100d80 2024-03-30T02:35:02.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.774+0000 7f6d25400700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6d0c0a1120 0x7f6d0c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:02.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.775+0000 7f6d25400700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6d0c0a1120 0x7f6d0c0a35e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d14006010 tx=0x7f6d1400b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:02.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.776+0000 7f6d1ea00700 1 -- 172.21.15.67:0/3474900060 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6d180963e0 con 0x7f6d28100d80 2024-03-30T02:35:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.995+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d2810c890 con 0x7f6d28100d80 2024-03-30T02:35:02.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.996+0000 7f6d1ea00700 1 -- 172.21.15.67:0/3474900060 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d18099a00 con 0x7f6d28100d80 2024-03-30T02:35:02.997 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:35:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6d0c0a1120 msgr2=0x7f6d0c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6d0c0a1120 0x7f6d0c0a35e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d14006010 tx=0x7f6d1400b540 comp rx=0 tx=0).stop 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28100d80 msgr2=0x7f6d28074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28100d80 0x7f6d28074b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800e970 tx=0x7f6d1800ec80 comp rx=0 tx=0).stop 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 shutdown_connections 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6d0c0a1120 0x7f6d0c0a35e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d28102910 0x7f6d280736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28100d80 0x7f6d28074b10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 --2- 172.21.15.67:0/3474900060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d28101f70 0x7f6d280731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 >> 172.21.15.67:0/3474900060 conn(0x7f6d280fc2f0 msgr2=0x7f6d28105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 shutdown_connections 2024-03-30T02:35:03.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:02.998+0000 7f6d2d000700 1 -- 172.21.15.67:0/3474900060 wait complete. 2024-03-30T02:35:03.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:35:03.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.219+0000 7f6fbba00700 1 -- 172.21.15.67:0/1309946731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb4100d80 msgr2=0x7f6fb41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.219+0000 7f6fbba00700 1 --2- 172.21.15.67:0/1309946731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb4100d80 0x7f6fb41011a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80099c0 tx=0x7f6fa8009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.219+0000 7f6fbba00700 1 -- 172.21.15.67:0/1309946731 shutdown_connections 2024-03-30T02:35:03.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.219+0000 7f6fbba00700 1 --2- 172.21.15.67:0/1309946731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb4100d80 0x7f6fb41011a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.219+0000 7f6fbba00700 1 --2- 172.21.15.67:0/1309946731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fb4102910 0x7f6fb41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.219+0000 7f6fbba00700 1 --2- 172.21.15.67:0/1309946731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fb4101f70 0x7f6fb41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.219+0000 7f6fbba00700 1 -- 172.21.15.67:0/1309946731 >> 172.21.15.67:0/1309946731 conn(0x7f6fb40fc2f0 msgr2=0x7f6fb40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:03.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.219+0000 7f6fbba00700 1 -- 172.21.15.67:0/1309946731 shutdown_connections 2024-03-30T02:35:03.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.219+0000 7f6fbba00700 1 -- 172.21.15.67:0/1309946731 wait complete. 2024-03-30T02:35:03.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.220+0000 7f6fbba00700 1 Processor -- start 2024-03-30T02:35:03.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.220+0000 7f6fbba00700 1 -- start start 2024-03-30T02:35:03.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fbba00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fb4100d80 0x7f6fb4198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:03.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fbba00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb4101f70 0x7f6fb4199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fbba00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fb4102910 0x7f6fb41a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fbba00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41a0a60 con 0x7f6fb4101f70 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fbba00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41a0ba0 con 0x7f6fb4102910 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fbba00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41a0d30 con 0x7f6fb4100d80 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb4101f70 0x7f6fb4199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb9a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fb4102910 0x7f6fb41a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb9000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fb4100d80 0x7f6fb4198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb9000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fb4100d80 0x7f6fb4198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47420/0 (socket says 172.21.15.67:47420) 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb9a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fb4102910 0x7f6fb41a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:50886/0 (socket says 172.21.15.67:50886) 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb9000700 1 -- 172.21.15.67:0/2484876985 learned_addr learned my addr 172.21.15.67:0/2484876985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb3e00700 1 -- 172.21.15.67:0/2484876985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fb4100d80 msgr2=0x7f6fb4198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb3e00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fb4100d80 0x7f6fb4198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb3e00700 1 -- 172.21.15.67:0/2484876985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fb4102910 msgr2=0x7f6fb41a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb3e00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fb4102910 0x7f6fb41a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb3e00700 1 -- 172.21.15.67:0/2484876985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80095f0 con 0x7f6fb4101f70 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.221+0000 7f6fb9a00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fb4102910 0x7f6fb41a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.222+0000 7f6fb3e00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb4101f70 0x7f6fb4199470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000e9f0 tx=0x7f6fa000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.222+0000 7f6fb1600700 1 -- 172.21.15.67:0/2484876985 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000c9c0 con 0x7f6fb4101f70 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.222+0000 7f6fb1600700 1 -- 172.21.15.67:0/2484876985 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa0010430 con 0x7f6fb4101f70 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.222+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb41a0fb0 con 0x7f6fb4101f70 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.222+0000 7f6fb1600700 1 -- 172.21.15.67:0/2484876985 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0018680 con 0x7f6fb4101f70 2024-03-30T02:35:03.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.222+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb41a14e0 con 0x7f6fb4101f70 2024-03-30T02:35:03.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.223+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4067af0 con 0x7f6fb4101f70 2024-03-30T02:35:03.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.225+0000 7f6fb1600700 1 -- 172.21.15.67:0/2484876985 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f6fa0010860 con 0x7f6fb4101f70 2024-03-30T02:35:03.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.226+0000 7f6fb1600700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f9c0a11a0 0x7f6f9c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:03.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.226+0000 7f6fb1600700 1 -- 172.21.15.67:0/2484876985 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6fa0014070 con 0x7f6fb4101f70 2024-03-30T02:35:03.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.227+0000 7f6fb9000700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f9c0a11a0 0x7f6f9c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:03.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.227+0000 7f6fb9000700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f9c0a11a0 0x7f6f9c0a3660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000c00 tx=0x7f6fa8009530 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:03.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.228+0000 7f6fb1600700 1 -- 172.21.15.67:0/2484876985 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6fa00962a0 con 0x7f6fb4101f70 2024-03-30T02:35:03.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:03 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/201550350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:35:03.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:03 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3278885278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:35:03.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:03 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3474900060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:35:03.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:03 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/201550350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:35:03.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:03 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3278885278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:35:03.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:03 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3474900060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:35:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.447+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6fb41a1780 con 0x7f6fb4101f70 2024-03-30T02:35:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.448+0000 7f6fb1600700 1 -- 172.21.15.67:0/2484876985 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6fa00260a0 con 0x7f6fb4101f70 2024-03-30T02:35:03.449 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.450+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f9c0a11a0 msgr2=0x7f6f9c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.450+0000 7f6fbba00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f9c0a11a0 0x7f6f9c0a3660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8000c00 tx=0x7f6fa8009530 comp rx=0 tx=0).stop 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.450+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb4101f70 msgr2=0x7f6fb4199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.450+0000 7f6fbba00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb4101f70 0x7f6fb4199470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000e9f0 tx=0x7f6fa000ed00 comp rx=0 tx=0).stop 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.450+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 shutdown_connections 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.451+0000 7f6fbba00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f9c0a11a0 0x7f6f9c0a3660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.451+0000 7f6fbba00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fb4102910 0x7f6fb41a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.451+0000 7f6fbba00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fb4100d80 0x7f6fb4198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.451+0000 7f6fbba00700 1 --2- 172.21.15.67:0/2484876985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6fb4101f70 0x7f6fb4199470 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.451+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 >> 172.21.15.67:0/2484876985 conn(0x7f6fb40fc2f0 msgr2=0x7f6fb4107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.451+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 shutdown_connections 2024-03-30T02:35:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.451+0000 7f6fbba00700 1 -- 172.21.15.67:0/2484876985 wait complete. 2024-03-30T02:35:03.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T02:35:03.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:03 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/201550350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:35:03.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:03 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3278885278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:35:03.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:03 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3474900060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:35:03.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.668+0000 7fc1fba00700 1 -- 172.21.15.67:0/4080251243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4102910 msgr2=0x7fc1f41091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.668+0000 7fc1fba00700 1 --2- 172.21.15.67:0/4080251243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4102910 0x7fc1f41091c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00b0d0 tx=0x7fc1ec00b3e0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.668+0000 7fc1fba00700 1 -- 172.21.15.67:0/4080251243 shutdown_connections 2024-03-30T02:35:03.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.668+0000 7fc1fba00700 1 --2- 172.21.15.67:0/4080251243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f4100d80 0x7fc1f41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.668+0000 7fc1fba00700 1 --2- 172.21.15.67:0/4080251243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4102910 0x7fc1f41091c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.668+0000 7fc1fba00700 1 --2- 172.21.15.67:0/4080251243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4101f70 0x7fc1f41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.668+0000 7fc1fba00700 1 -- 172.21.15.67:0/4080251243 >> 172.21.15.67:0/4080251243 conn(0x7fc1f40fc2f0 msgr2=0x7fc1f40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:03.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.668+0000 7fc1fba00700 1 -- 172.21.15.67:0/4080251243 shutdown_connections 2024-03-30T02:35:03.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.668+0000 7fc1fba00700 1 -- 172.21.15.67:0/4080251243 wait complete. 2024-03-30T02:35:03.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.669+0000 7fc1fba00700 1 Processor -- start 2024-03-30T02:35:03.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.669+0000 7fc1fba00700 1 -- start start 2024-03-30T02:35:03.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1fba00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f4100d80 0x7fc1f4074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1fba00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4101f70 0x7fc1f40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1fba00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4102910 0x7fc1f40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1fba00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f4073e80 con 0x7fc1f4100d80 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1fba00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f4073ff0 con 0x7fc1f4101f70 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1fba00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f41a0e80 con 0x7fc1f4102910 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f9000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f4100d80 0x7fc1f4074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4101f70 0x7fc1f40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4101f70 0x7fc1f40731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:50892/0 (socket says 172.21.15.67:50892) 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f3e00700 1 -- 172.21.15.67:0/3848379329 learned_addr learned my addr 172.21.15.67:0/3848379329 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f9a00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4102910 0x7fc1f40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f9000700 1 -- 172.21.15.67:0/3848379329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4102910 msgr2=0x7fc1f40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f9000700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4102910 0x7fc1f40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f9000700 1 -- 172.21.15.67:0/3848379329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4101f70 msgr2=0x7fc1f40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f9000700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4101f70 0x7fc1f40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f9000700 1 -- 172.21.15.67:0/3848379329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec009d00 con 0x7fc1f4100d80 2024-03-30T02:35:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.670+0000 7fc1f3e00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4101f70 0x7fc1f40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.671+0000 7fc1f9000700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f4100d80 0x7fc1f4074b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800e9c0 tx=0x7fc1e800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.671+0000 7fc1f1600700 1 -- 172.21.15.67:0/3848379329 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8004220 con 0x7fc1f4100d80 2024-03-30T02:35:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.671+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f41a1110 con 0x7fc1f4100d80 2024-03-30T02:35:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.671+0000 7fc1f1600700 1 -- 172.21.15.67:0/3848379329 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e8004380 con 0x7fc1f4100d80 2024-03-30T02:35:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.671+0000 7fc1f1600700 1 -- 172.21.15.67:0/3848379329 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8003b20 con 0x7fc1f4100d80 2024-03-30T02:35:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.671+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f41a1670 con 0x7fc1f4100d80 2024-03-30T02:35:03.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.672+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f4108130 con 0x7fc1f4100d80 2024-03-30T02:35:03.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.674+0000 7fc1f1600700 1 -- 172.21.15.67:0/3848379329 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc1e8003d50 con 0x7fc1f4100d80 2024-03-30T02:35:03.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.675+0000 7fc1f1600700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc1dc0a1120 0x7fc1dc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:03.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.675+0000 7fc1f1600700 1 -- 172.21.15.67:0/3848379329 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc1e8014070 con 0x7fc1f4100d80 2024-03-30T02:35:03.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.675+0000 7fc1f3e00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc1dc0a1120 0x7fc1dc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:03.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.676+0000 7fc1f3e00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc1dc0a1120 0x7fc1dc0a35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0011550 tx=0x7fc1e0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:03.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.679+0000 7fc1f1600700 1 -- 172.21.15.67:0/3848379329 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc1e80962a0 con 0x7fc1f4100d80 2024-03-30T02:35:03.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.958+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc1f4067af0 con 0x7fc1f4100d80 2024-03-30T02:35:03.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.959+0000 7fc1f1600700 1 -- 172.21.15.67:0/3848379329 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) v1 ==== 81+0+591 (secure 0 0 0) 0x7fc1e80998c0 con 0x7fc1f4100d80 2024-03-30T02:35:03.960 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2119993048 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:03.960 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2682757440 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:03.960 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2682757440 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:03.960 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/598232861 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:03.960 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/4186481961 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:03.960 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3533289986 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:03.960 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/4186481961 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:03.961 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1250300809 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:03.961 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2693891742 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:03.961 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3435746414 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc1dc0a1120 msgr2=0x7fc1dc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc1dc0a1120 0x7fc1dc0a35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0011550 tx=0x7fc1e0009260 comp rx=0 tx=0).stop 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f4100d80 msgr2=0x7fc1f4074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f4100d80 0x7fc1f4074b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800e9c0 tx=0x7fc1e800ecd0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 shutdown_connections 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc1dc0a1120 0x7fc1dc0a35e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1f4100d80 0x7fc1f4074b10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4102910 0x7fc1f40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 --2- 172.21.15.67:0/3848379329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1f4101f70 0x7fc1f40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.961+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 >> 172.21.15.67:0/3848379329 conn(0x7fc1f40fc2f0 msgr2=0x7fc1f4105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.962+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 shutdown_connections 2024-03-30T02:35:03.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:03.962+0000 7fc1fba00700 1 -- 172.21.15.67:0/3848379329 wait complete. 2024-03-30T02:35:03.964 INFO:teuthology.orchestra.run.smithi067.stderr:listed 10 entries 2024-03-30T02:35:03.980 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:35:03.981 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T02:35:04.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:35:04.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.259+0000 7fbc9a400700 1 -- 172.21.15.67:0/599924862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 msgr2=0x7fbc941010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.259+0000 7fbc9a400700 1 --2- 172.21.15.67:0/599924862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 0x7fbc941010a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbc840099c0 tx=0x7fbc84009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.260+0000 7fbc9a400700 1 -- 172.21.15.67:0/599924862 shutdown_connections 2024-03-30T02:35:04.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.260+0000 7fbc9a400700 1 --2- 172.21.15.67:0/599924862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 0x7fbc941010a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.260+0000 7fbc9a400700 1 --2- 172.21.15.67:0/599924862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc94102810 0x7fbc941090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.260+0000 7fbc9a400700 1 --2- 172.21.15.67:0/599924862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc94101e70 0x7fbc941022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.260+0000 7fbc9a400700 1 -- 172.21.15.67:0/599924862 >> 172.21.15.67:0/599924862 conn(0x7fbc940fc1f0 msgr2=0x7fbc940fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:04.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.260+0000 7fbc9a400700 1 -- 172.21.15.67:0/599924862 shutdown_connections 2024-03-30T02:35:04.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.260+0000 7fbc9a400700 1 -- 172.21.15.67:0/599924862 wait complete. 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc9a400700 1 Processor -- start 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc9a400700 1 -- start start 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc9a400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 0x7fbc94198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc9a400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc94101e70 0x7fbc941993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc9a400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc94102810 0x7fbc941a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc9a400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941a0990 con 0x7fbc94100c80 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc9a400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941a0ad0 con 0x7fbc94102810 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc9a400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941a0c60 con 0x7fbc94101e70 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc93400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 0x7fbc94198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc93400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 0x7fbc94198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42162/0 (socket says 172.21.15.67:42162) 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.261+0000 7fbc93400700 1 -- 172.21.15.67:0/2433335638 learned_addr learned my addr 172.21.15.67:0/2433335638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:04.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc92a00700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc94101e70 0x7fbc941993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc93e00700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc94102810 0x7fbc941a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc93400700 1 -- 172.21.15.67:0/2433335638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc94101e70 msgr2=0x7fbc941993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc93400700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc94101e70 0x7fbc941993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc93400700 1 -- 172.21.15.67:0/2433335638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc94102810 msgr2=0x7fbc941a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc93400700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc94102810 0x7fbc941a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc93400700 1 -- 172.21.15.67:0/2433335638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840095f0 con 0x7fbc94100c80 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc92a00700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc94101e70 0x7fbc941993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc93400700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 0x7fbc94198eb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc84006010 tx=0x7fbc8400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc8be00700 1 -- 172.21.15.67:0/2433335638 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8401d070 con 0x7fbc94100c80 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc8be00700 1 -- 172.21.15.67:0/2433335638 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc84005020 con 0x7fbc94100c80 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc8be00700 1 -- 172.21.15.67:0/2433335638 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84004b10 con 0x7fbc94100c80 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.262+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc941a0ee0 con 0x7fbc94100c80 2024-03-30T02:35:04.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.263+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc941a1250 con 0x7fbc94100c80 2024-03-30T02:35:04.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.264+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc94067a80 con 0x7fbc94100c80 2024-03-30T02:35:04.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.265+0000 7fbc8be00700 1 -- 172.21.15.67:0/2433335638 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbc84005190 con 0x7fbc94100c80 2024-03-30T02:35:04.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.267+0000 7fbc8be00700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbc780a11a0 0x7fbc780a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:04.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.267+0000 7fbc8be00700 1 -- 172.21.15.67:0/2433335638 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbc840cd4b0 con 0x7fbc94100c80 2024-03-30T02:35:04.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.267+0000 7fbc92a00700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbc780a11a0 0x7fbc780a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:04.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.267+0000 7fbc92a00700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbc780a11a0 0x7fbc780a3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc80011490 tx=0x7fbc80009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:04.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.271+0000 7fbc8be00700 1 -- 172.21.15.67:0/2433335638 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbc94067a80 con 0x7fbc94100c80 2024-03-30T02:35:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:04 smithi067 ceph-mon[37808]: pgmap v241: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-30T02:35:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:04 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2484876985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:35:04.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:04 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3848379329' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T02:35:04.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:04 smithi170 ceph-mon[78798]: pgmap v241: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-30T02:35:04.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:04 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2484876985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:35:04.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:04 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3848379329' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T02:35:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:04 smithi082 ceph-mon[79176]: pgmap v241: 129 pgs: 129 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-30T02:35:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:04 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2484876985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:35:04.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:04 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3848379329' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T02:35:04.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.525+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc9410c4e0 con 0x7fbc94100c80 2024-03-30T02:35:04.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.527+0000 7fbc8be00700 1 -- 172.21.15.67:0/2433335638 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+23925 (secure 0 0 0) 0x7fbc84096a80 con 0x7fbc94100c80 2024-03-30T02:35:04.528 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:04.528 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:51.268204+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:35:04.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbc780a11a0 msgr2=0x7fbc780a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbc780a11a0 0x7fbc780a3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc80011490 tx=0x7fbc80009260 comp rx=0 tx=0).stop 2024-03-30T02:35:04.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 msgr2=0x7fbc94198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 0x7fbc94198eb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc84006010 tx=0x7fbc8400f690 comp rx=0 tx=0).stop 2024-03-30T02:35:04.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 shutdown_connections 2024-03-30T02:35:04.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbc780a11a0 0x7fbc780a3660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc94100c80 0x7fbc94198eb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc94102810 0x7fbc941a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 --2- 172.21.15.67:0/2433335638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc94101e70 0x7fbc941993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 >> 172.21.15.67:0/2433335638 conn(0x7fbc940fc1f0 msgr2=0x7fbc94072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:04.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 shutdown_connections 2024-03-30T02:35:04.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.529+0000 7fbc9a400700 1 -- 172.21.15.67:0/2433335638 wait complete. 2024-03-30T02:35:04.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:35:04.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:35:04.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.750+0000 7f4103800700 1 -- 172.21.15.67:0/2746589124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc101f70 msgr2=0x7f40fc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.750+0000 7f4103800700 1 --2- 172.21.15.67:0/2746589124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc101f70 0x7f40fc1023d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009920 tx=0x7f40e8009c30 comp rx=0 tx=0).stop 2024-03-30T02:35:04.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.750+0000 7f4103800700 1 -- 172.21.15.67:0/2746589124 shutdown_connections 2024-03-30T02:35:04.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.750+0000 7f4103800700 1 --2- 172.21.15.67:0/2746589124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40fc100d80 0x7f40fc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.750+0000 7f4103800700 1 --2- 172.21.15.67:0/2746589124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc102910 0x7f40fc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.750+0000 7f4103800700 1 --2- 172.21.15.67:0/2746589124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc101f70 0x7f40fc1023d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.750+0000 7f4103800700 1 -- 172.21.15.67:0/2746589124 >> 172.21.15.67:0/2746589124 conn(0x7f40fc0fc2f0 msgr2=0x7f40fc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.750+0000 7f4103800700 1 -- 172.21.15.67:0/2746589124 shutdown_connections 2024-03-30T02:35:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.750+0000 7f4103800700 1 -- 172.21.15.67:0/2746589124 wait complete. 2024-03-30T02:35:04.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.751+0000 7f4103800700 1 Processor -- start 2024-03-30T02:35:04.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.751+0000 7f4103800700 1 -- start start 2024-03-30T02:35:04.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.751+0000 7f4103800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc100d80 0x7f40fc196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:04.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f4103800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40fc101f70 0x7f40fc197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:04.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f4103800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc102910 0x7f40fc19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:04.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f4103800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc19e880 con 0x7f40fc101f70 2024-03-30T02:35:04.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f4103800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc19e9f0 con 0x7f40fc102910 2024-03-30T02:35:04.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f4103800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc19ebb0 con 0x7f40fc100d80 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f40fbe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40fc101f70 0x7f40fc197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f40fbe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40fc101f70 0x7f40fc197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42174/0 (socket says 172.21.15.67:42174) 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f40fbe00700 1 -- 172.21.15.67:0/856673232 learned_addr learned my addr 172.21.15.67:0/856673232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f4100e00700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc100d80 0x7f40fc196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f4101800700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc102910 0x7f40fc19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f40fbe00700 1 -- 172.21.15.67:0/856673232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc100d80 msgr2=0x7f40fc196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f40fbe00700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc100d80 0x7f40fc196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f40fbe00700 1 -- 172.21.15.67:0/856673232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc102910 msgr2=0x7f40fc19e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f40fbe00700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc102910 0x7f40fc19e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f40fbe00700 1 -- 172.21.15.67:0/856673232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e8009520 con 0x7f40fc101f70 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.752+0000 7f40fbe00700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40fc101f70 0x7f40fc197320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f40e8005e90 tx=0x7f40e800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.753+0000 7f40f9600700 1 -- 172.21.15.67:0/856673232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e801c070 con 0x7f40fc101f70 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.753+0000 7f40f9600700 1 -- 172.21.15.67:0/856673232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e800f7f0 con 0x7f40fc101f70 2024-03-30T02:35:04.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.753+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc19ee30 con 0x7f40fc101f70 2024-03-30T02:35:04.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.753+0000 7f40f9600700 1 -- 172.21.15.67:0/856673232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8003cc0 con 0x7f40fc101f70 2024-03-30T02:35:04.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.753+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc19f1a0 con 0x7f40fc101f70 2024-03-30T02:35:04.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.754+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc108290 con 0x7f40fc101f70 2024-03-30T02:35:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.756+0000 7f40f9600700 1 -- 172.21.15.67:0/856673232 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f40e8005590 con 0x7f40fc101f70 2024-03-30T02:35:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.757+0000 7f40f9600700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e40a1120 0x7f40e40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.757+0000 7f40f9600700 1 -- 172.21.15.67:0/856673232 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f40e80cdb90 con 0x7f40fc101f70 2024-03-30T02:35:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.757+0000 7f4100e00700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e40a1120 0x7f40e40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:04.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.758+0000 7f4100e00700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e40a1120 0x7f40e40a35e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40fc198210 tx=0x7f40f0009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:04.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.759+0000 7f40f9600700 1 -- 172.21.15.67:0/856673232 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f40e8097160 con 0x7f40fc101f70 2024-03-30T02:35:04.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.976+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40fc10c590 con 0x7f40fc101f70 2024-03-30T02:35:04.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.976+0000 7f40f9600700 1 -- 172.21.15.67:0/856673232 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40e809a780 con 0x7f40fc101f70 2024-03-30T02:35:04.977 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.978+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e40a1120 msgr2=0x7f40e40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.978+0000 7f4103800700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e40a1120 0x7f40e40a35e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40fc198210 tx=0x7f40f0009310 comp rx=0 tx=0).stop 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.978+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40fc101f70 msgr2=0x7f40fc197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.978+0000 7f4103800700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40fc101f70 0x7f40fc197320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f40e8005e90 tx=0x7f40e800b790 comp rx=0 tx=0).stop 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.979+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 shutdown_connections 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.979+0000 7f4103800700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e40a1120 0x7f40e40a35e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.979+0000 7f4103800700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc102910 0x7f40fc19e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.979+0000 7f4103800700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40fc100d80 0x7f40fc196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.979+0000 7f4103800700 1 --2- 172.21.15.67:0/856673232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40fc101f70 0x7f40fc197320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.979+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 >> 172.21.15.67:0/856673232 conn(0x7f40fc0fc2f0 msgr2=0x7f40fc1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.979+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 shutdown_connections 2024-03-30T02:35:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:04.979+0000 7f4103800700 1 -- 172.21.15.67:0/856673232 wait complete. 2024-03-30T02:35:04.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:35:05.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.199+0000 7f632c000700 1 -- 172.21.15.67:0/3380534069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324100d80 msgr2=0x7f63241011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.199+0000 7f632c000700 1 --2- 172.21.15.67:0/3380534069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324100d80 0x7f63241011a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f63180099c0 tx=0x7f6318009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.199+0000 7f632c000700 1 -- 172.21.15.67:0/3380534069 shutdown_connections 2024-03-30T02:35:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.199+0000 7f632c000700 1 --2- 172.21.15.67:0/3380534069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324100d80 0x7f63241011a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.199+0000 7f632c000700 1 --2- 172.21.15.67:0/3380534069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6324102910 0x7f63241091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.199+0000 7f632c000700 1 --2- 172.21.15.67:0/3380534069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6324101f70 0x7f63241023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.199+0000 7f632c000700 1 -- 172.21.15.67:0/3380534069 >> 172.21.15.67:0/3380534069 conn(0x7f63240fc2f0 msgr2=0x7f63240fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.199+0000 7f632c000700 1 -- 172.21.15.67:0/3380534069 shutdown_connections 2024-03-30T02:35:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.199+0000 7f632c000700 1 -- 172.21.15.67:0/3380534069 wait complete. 2024-03-30T02:35:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.200+0000 7f632c000700 1 Processor -- start 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.200+0000 7f632c000700 1 -- start start 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f632c000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6324100d80 0x7f6324198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f632c000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324101f70 0x7f63241994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f632c000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6324102910 0x7f63241a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f632c000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241a0ab0 con 0x7f6324101f70 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f632c000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241a0bf0 con 0x7f6324100d80 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f632c000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63241a0d80 con 0x7f6324102910 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6328c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324101f70 0x7f63241994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6328c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324101f70 0x7f63241994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42192/0 (socket says 172.21.15.67:42192) 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6329600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6324100d80 0x7f6324198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6328c00700 1 -- 172.21.15.67:0/2332968574 learned_addr learned my addr 172.21.15.67:0/2332968574 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f632a000700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6324102910 0x7f63241a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6328c00700 1 -- 172.21.15.67:0/2332968574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6324102910 msgr2=0x7f63241a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6328c00700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6324102910 0x7f63241a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6328c00700 1 -- 172.21.15.67:0/2332968574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6324100d80 msgr2=0x7f6324198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6328c00700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6324100d80 0x7f6324198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6328c00700 1 -- 172.21.15.67:0/2332968574 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63180095f0 con 0x7f6324101f70 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.201+0000 7f6329600700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6324100d80 0x7f6324198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.202+0000 7f6328c00700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324101f70 0x7f63241994c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f631000e9a0 tx=0x7f631000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.202+0000 7f6322000700 1 -- 172.21.15.67:0/2332968574 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310009790 con 0x7f6324101f70 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.202+0000 7f6322000700 1 -- 172.21.15.67:0/2332968574 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f631000cbd0 con 0x7f6324101f70 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.202+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63241a1060 con 0x7f6324101f70 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.202+0000 7f6322000700 1 -- 172.21.15.67:0/2332968574 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310018680 con 0x7f6324101f70 2024-03-30T02:35:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.202+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63241a1570 con 0x7f6324101f70 2024-03-30T02:35:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.203+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6324067af0 con 0x7f6324101f70 2024-03-30T02:35:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.205+0000 7f6322000700 1 -- 172.21.15.67:0/2332968574 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f631000cd40 con 0x7f6324101f70 2024-03-30T02:35:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.206+0000 7f6322000700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f630c0a1120 0x7f630c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.206+0000 7f6322000700 1 -- 172.21.15.67:0/2332968574 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6310014070 con 0x7f6324101f70 2024-03-30T02:35:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.207+0000 7f6329600700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f630c0a1120 0x7f630c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.207+0000 7f6329600700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f630c0a1120 0x7f630c0a35e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6318009990 tx=0x7f631801a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:05.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.210+0000 7f6322000700 1 -- 172.21.15.67:0/2332968574 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f63100529f0 con 0x7f6324101f70 2024-03-30T02:35:05.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:05 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2433335638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:05.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:05 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/856673232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:35:05.609 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:05 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2433335638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:05.609 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:05 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/856673232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:35:05.609 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:05 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2433335638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:05.609 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:05 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/856673232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:35:05.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.425+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63241a1890 con 0x7f6324101f70 2024-03-30T02:35:05.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.426+0000 7f6322000700 1 -- 172.21.15.67:0/2332968574 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63100962d0 con 0x7f6324101f70 2024-03-30T02:35:05.610 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:35:05.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f630c0a1120 msgr2=0x7f630c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:05.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f630c0a1120 0x7f630c0a35e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6318009990 tx=0x7f631801a040 comp rx=0 tx=0).stop 2024-03-30T02:35:05.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324101f70 msgr2=0x7f63241994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:05.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324101f70 0x7f63241994c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f631000e9a0 tx=0x7f631000ecb0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 shutdown_connections 2024-03-30T02:35:05.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f630c0a1120 0x7f630c0a35e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6324102910 0x7f63241a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6324100d80 0x7f6324198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 --2- 172.21.15.67:0/2332968574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6324101f70 0x7f63241994c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 >> 172.21.15.67:0/2332968574 conn(0x7f63240fc2f0 msgr2=0x7f6324072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:05.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 shutdown_connections 2024-03-30T02:35:05.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.428+0000 7f632c000700 1 -- 172.21.15.67:0/2332968574 wait complete. 2024-03-30T02:35:05.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:35:05.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.816+0000 7f7834a00700 1 -- 172.21.15.67:0/2609678697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830102910 msgr2=0x7f78301091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:05.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.816+0000 7f7834a00700 1 --2- 172.21.15.67:0/2609678697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830102910 0x7f78301091c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f782800b0d0 tx=0x7f782800b3e0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.816+0000 7f7834a00700 1 -- 172.21.15.67:0/2609678697 shutdown_connections 2024-03-30T02:35:05.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.816+0000 7f7834a00700 1 --2- 172.21.15.67:0/2609678697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830102910 0x7f78301091c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.816+0000 7f7834a00700 1 --2- 172.21.15.67:0/2609678697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7830100d80 0x7f78301011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.816+0000 7f7834a00700 1 --2- 172.21.15.67:0/2609678697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7830101f70 0x7f78301023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.816+0000 7f7834a00700 1 -- 172.21.15.67:0/2609678697 >> 172.21.15.67:0/2609678697 conn(0x7f78300fc2f0 msgr2=0x7f78300fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:05.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.816+0000 7f7834a00700 1 -- 172.21.15.67:0/2609678697 shutdown_connections 2024-03-30T02:35:05.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.816+0000 7f7834a00700 1 -- 172.21.15.67:0/2609678697 wait complete. 2024-03-30T02:35:05.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.817+0000 7f7834a00700 1 Processor -- start 2024-03-30T02:35:05.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.817+0000 7f7834a00700 1 -- start start 2024-03-30T02:35:05.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f7834a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830100d80 0x7f7830196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:05.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f7834a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7830101f70 0x7f7830197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:05.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f7834a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7830102910 0x7f783019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:05.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f7834a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783019e980 con 0x7f7830100d80 2024-03-30T02:35:05.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f7834a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783019eac0 con 0x7f7830101f70 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f7834a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783019ec50 con 0x7f7830102910 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830100d80 0x7f7830196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7830101f70 0x7f7830197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830100d80 0x7f7830196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42202/0 (socket says 172.21.15.67:42202) 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7830101f70 0x7f7830197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:50962/0 (socket says 172.21.15.67:50962) 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782de00700 1 -- 172.21.15.67:0/2587092375 learned_addr learned my addr 172.21.15.67:0/2587092375 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782e800700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7830102910 0x7f783019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782d400700 1 -- 172.21.15.67:0/2587092375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7830102910 msgr2=0x7f783019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782d400700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7830102910 0x7f783019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782d400700 1 -- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830100d80 msgr2=0x7f7830196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782d400700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830100d80 0x7f7830196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782d400700 1 -- 172.21.15.67:0/2587092375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7828009d00 con 0x7f7830101f70 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.818+0000 7f782de00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830100d80 0x7f7830196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.819+0000 7f782d400700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7830101f70 0x7f7830197390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f781c00e9f0 tx=0x7f781c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.819+0000 7f7826a00700 1 -- 172.21.15.67:0/2587092375 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c00c9c0 con 0x7f7830101f70 2024-03-30T02:35:05.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.819+0000 7f7826a00700 1 -- 172.21.15.67:0/2587092375 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f781c010430 con 0x7f7830101f70 2024-03-30T02:35:05.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.819+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783019ef30 con 0x7f7830101f70 2024-03-30T02:35:05.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.819+0000 7f7826a00700 1 -- 172.21.15.67:0/2587092375 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c018680 con 0x7f7830101f70 2024-03-30T02:35:05.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.819+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783019f3b0 con 0x7f7830101f70 2024-03-30T02:35:05.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.820+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7830108180 con 0x7f7830101f70 2024-03-30T02:35:05.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.822+0000 7f7826a00700 1 -- 172.21.15.67:0/2587092375 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f781c010860 con 0x7f7830101f70 2024-03-30T02:35:05.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.823+0000 7f7826a00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f78140a1120 0x7f78140a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:05.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.824+0000 7f7826a00700 1 -- 172.21.15.67:0/2587092375 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f781c014070 con 0x7f7830101f70 2024-03-30T02:35:05.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.824+0000 7f782de00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f78140a1120 0x7f78140a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:05.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.824+0000 7f782de00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f78140a1120 0x7f78140a35e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7820005950 tx=0x7f7820009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:05.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:05.826+0000 7f7826a00700 1 -- 172.21.15.67:0/2587092375 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f781c0962d0 con 0x7f7830101f70 2024-03-30T02:35:06.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.044+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f783002ccb0 con 0x7f7830101f70 2024-03-30T02:35:06.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.045+0000 7f7826a00700 1 -- 172.21.15.67:0/2587092375 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f781c026020 con 0x7f7830101f70 2024-03-30T02:35:06.046 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f78140a1120 msgr2=0x7f78140a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f78140a1120 0x7f78140a35e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7820005950 tx=0x7f7820009260 comp rx=0 tx=0).stop 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7830101f70 msgr2=0x7f7830197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7830101f70 0x7f7830197390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f781c00e9f0 tx=0x7f781c00ed00 comp rx=0 tx=0).stop 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 shutdown_connections 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f78140a1120 0x7f78140a35e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7830100d80 0x7f7830196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7830102910 0x7f783019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 --2- 172.21.15.67:0/2587092375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7830101f70 0x7f7830197390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.047+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 >> 172.21.15.67:0/2587092375 conn(0x7f78300fc2f0 msgr2=0x7f78301052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.048+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 shutdown_connections 2024-03-30T02:35:06.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.048+0000 7f7834a00700 1 -- 172.21.15.67:0/2587092375 wait complete. 2024-03-30T02:35:06.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:35:06.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.268+0000 7f1d87200700 1 -- 172.21.15.67:0/2762157068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d801017e0 msgr2=0x7f1d80103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.268+0000 7f1d87200700 1 --2- 172.21.15.67:0/2762157068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d801017e0 0x7f1d80103bd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009990 tx=0x7f1d70009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.268+0000 7f1d87200700 1 -- 172.21.15.67:0/2762157068 shutdown_connections 2024-03-30T02:35:06.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.268+0000 7f1d87200700 1 --2- 172.21.15.67:0/2762157068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d80106a40 0x7f1d80108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.268+0000 7f1d87200700 1 --2- 172.21.15.67:0/2762157068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d801017e0 0x7f1d80103bd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.268+0000 7f1d87200700 1 --2- 172.21.15.67:0/2762157068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d80104110 0x7f1d80106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.268+0000 7f1d87200700 1 -- 172.21.15.67:0/2762157068 >> 172.21.15.67:0/2762157068 conn(0x7f1d800fb1d0 msgr2=0x7f1d800fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:06.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.268+0000 7f1d87200700 1 -- 172.21.15.67:0/2762157068 shutdown_connections 2024-03-30T02:35:06.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.268+0000 7f1d87200700 1 -- 172.21.15.67:0/2762157068 wait complete. 2024-03-30T02:35:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.270+0000 7f1d87200700 1 Processor -- start 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.270+0000 7f1d87200700 1 -- start start 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.270+0000 7f1d87200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d801017e0 0x7f1d80196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.270+0000 7f1d87200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d80104110 0x7f1d801971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.270+0000 7f1d87200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80106a40 0x7f1d8019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.270+0000 7f1d87200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8019e780 con 0x7f1d80104110 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.270+0000 7f1d87200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8019e8f0 con 0x7f1d801017e0 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.270+0000 7f1d87200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8019eab0 con 0x7f1d80106a40 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d80104110 0x7f1d801971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d801017e0 0x7f1d80196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d80104110 0x7f1d801971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58352/0 (socket says 172.21.15.67:58352) 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7f400700 1 -- 172.21.15.67:0/4263503384 learned_addr learned my addr 172.21.15.67:0/4263503384 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d85200700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80106a40 0x7f1d8019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7f400700 1 -- 172.21.15.67:0/4263503384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80106a40 msgr2=0x7f1d8019e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7f400700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80106a40 0x7f1d8019e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7f400700 1 -- 172.21.15.67:0/4263503384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d801017e0 msgr2=0x7f1d80196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7f400700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d801017e0 0x7f1d80196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7f400700 1 -- 172.21.15.67:0/4263503384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d700095f0 con 0x7f1d80104110 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d85200700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80106a40 0x7f1d8019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.271+0000 7f1d7f400700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d80104110 0x7f1d801971d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00e9a0 tx=0x7f1d6c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.272+0000 7f1d7cc00700 1 -- 172.21.15.67:0/4263503384 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c009790 con 0x7f1d80104110 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.272+0000 7f1d7cc00700 1 -- 172.21.15.67:0/4263503384 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d6c00cbd0 con 0x7f1d80104110 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.272+0000 7f1d7cc00700 1 -- 172.21.15.67:0/4263503384 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c018680 con 0x7f1d80104110 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.272+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8019ed90 con 0x7f1d80104110 2024-03-30T02:35:06.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.272+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8019f190 con 0x7f1d80104110 2024-03-30T02:35:06.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.273+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d800fcd50 con 0x7f1d80104110 2024-03-30T02:35:06.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.275+0000 7f1d7cc00700 1 -- 172.21.15.67:0/4263503384 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f1d6c00cd40 con 0x7f1d80104110 2024-03-30T02:35:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.276+0000 7f1d7cc00700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1d640a1120 0x7f1d640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.276+0000 7f1d7cc00700 1 -- 172.21.15.67:0/4263503384 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1d6c014070 con 0x7f1d80104110 2024-03-30T02:35:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.276+0000 7f1d7fe00700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1d640a1120 0x7f1d640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.277+0000 7f1d7fe00700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1d640a1120 0x7f1d640a35e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000b680 tx=0x7f1d7000b750 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:06.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.280+0000 7f1d7cc00700 1 -- 172.21.15.67:0/4263503384 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1d6c0529f0 con 0x7f1d80104110 2024-03-30T02:35:06.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:06 smithi067 ceph-mon[37808]: pgmap v242: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-30T02:35:06.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:06 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2332968574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:35:06.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:06 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2587092375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:35:06.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:06 smithi170 ceph-mon[78798]: pgmap v242: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-30T02:35:06.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:06 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2332968574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:35:06.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:06 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2587092375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:35:06.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.496+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1d8002ccb0 con 0x7f1d80104110 2024-03-30T02:35:06.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.497+0000 7f1d7cc00700 1 -- 172.21.15.67:0/4263503384 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1d6c0962c0 con 0x7f1d80104110 2024-03-30T02:35:06.498 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:35:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:06 smithi082 ceph-mon[79176]: pgmap v242: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-30T02:35:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:06 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2332968574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:35:06.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:06 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2587092375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.499+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1d640a1120 msgr2=0x7f1d640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.499+0000 7f1d87200700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1d640a1120 0x7f1d640a35e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000b680 tx=0x7f1d7000b750 comp rx=0 tx=0).stop 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.499+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d80104110 msgr2=0x7f1d801971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.499+0000 7f1d87200700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d80104110 0x7f1d801971d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00e9a0 tx=0x7f1d6c00ecb0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.499+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 shutdown_connections 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.499+0000 7f1d87200700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1d640a1120 0x7f1d640a35e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.499+0000 7f1d87200700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80106a40 0x7f1d8019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.500+0000 7f1d87200700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d801017e0 0x7f1d80196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.500+0000 7f1d87200700 1 --2- 172.21.15.67:0/4263503384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1d80104110 0x7f1d801971d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.500+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 >> 172.21.15.67:0/4263503384 conn(0x7f1d800fb1d0 msgr2=0x7f1d80103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.500+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 shutdown_connections 2024-03-30T02:35:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.500+0000 7f1d87200700 1 -- 172.21.15.67:0/4263503384 wait complete. 2024-03-30T02:35:06.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T02:35:06.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.716+0000 7fa05e600700 1 -- 172.21.15.67:0/2903513857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05810d350 msgr2=0x7fa05810f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.716+0000 7fa05e600700 1 --2- 172.21.15.67:0/2903513857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05810d350 0x7fa05810f740 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b0a0 tx=0x7fa05000b3b0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.717+0000 7fa05e600700 1 -- 172.21.15.67:0/2903513857 shutdown_connections 2024-03-30T02:35:06.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.717+0000 7fa05e600700 1 --2- 172.21.15.67:0/2903513857 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa05810d350 0x7fa05810f740 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.717+0000 7fa05e600700 1 --2- 172.21.15.67:0/2903513857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa058074df0 0x7fa058073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.717+0000 7fa05e600700 1 --2- 172.21.15.67:0/2903513857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058073810 0x7fa058073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.717+0000 7fa05e600700 1 -- 172.21.15.67:0/2903513857 >> 172.21.15.67:0/2903513857 conn(0x7fa0580fc2f0 msgr2=0x7fa0580fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:06.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.717+0000 7fa05e600700 1 -- 172.21.15.67:0/2903513857 shutdown_connections 2024-03-30T02:35:06.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.717+0000 7fa05e600700 1 -- 172.21.15.67:0/2903513857 wait complete. 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa05e600700 1 Processor -- start 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa05e600700 1 -- start start 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa05e600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058073810 0x7fa05819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa05e600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa058074df0 0x7fa05819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa05e600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa05810d350 0x7fa0581a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa05e600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0581a4f30 con 0x7fa058074df0 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa05e600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0581a5070 con 0x7fa05810d350 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa05e600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0581a5200 con 0x7fa058073810 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa057400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058073810 0x7fa05819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:06.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa056a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa058074df0 0x7fa05819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa057400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058073810 0x7fa05819d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55636/0 (socket says 172.21.15.67:55636) 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa057400700 1 -- 172.21.15.67:0/3994696284 learned_addr learned my addr 172.21.15.67:0/3994696284 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.718+0000 7fa057e00700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa05810d350 0x7fa0581a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa057400700 1 -- 172.21.15.67:0/3994696284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa05810d350 msgr2=0x7fa0581a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa057400700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa05810d350 0x7fa0581a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa057400700 1 -- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa058074df0 msgr2=0x7fa05819d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa057400700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa058074df0 0x7fa05819d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa057400700 1 -- 172.21.15.67:0/3994696284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa044009520 con 0x7fa058073810 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa056a00700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa058074df0 0x7fa05819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa057400700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058073810 0x7fa05819d400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa04800eb30 tx=0x7fa04800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa04fe00700 1 -- 172.21.15.67:0/3994696284 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048009880 con 0x7fa058073810 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa048009520 con 0x7fa058073810 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa04fe00700 1 -- 172.21.15.67:0/3994696284 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0480044e0 con 0x7fa058073810 2024-03-30T02:35:06.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.719+0000 7fa04fe00700 1 -- 172.21.15.67:0/3994696284 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa048003d80 con 0x7fa058073810 2024-03-30T02:35:06.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.720+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa050009d00 con 0x7fa058073810 2024-03-30T02:35:06.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.720+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0581035b0 con 0x7fa058073810 2024-03-30T02:35:06.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.722+0000 7fa04fe00700 1 -- 172.21.15.67:0/3994696284 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa048003f00 con 0x7fa058073810 2024-03-30T02:35:06.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.724+0000 7fa04fe00700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa03c0a1120 0x7fa03c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:06.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.724+0000 7fa04fe00700 1 -- 172.21.15.67:0/3994696284 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa048014070 con 0x7fa058073810 2024-03-30T02:35:06.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.724+0000 7fa056a00700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa03c0a1120 0x7fa03c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:06.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.724+0000 7fa056a00700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa03c0a1120 0x7fa03c0a35e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa044009fd0 tx=0x7fa044009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:06.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:06.726+0000 7fa04fe00700 1 -- 172.21.15.67:0/3994696284 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa0480966f0 con 0x7fa058073810 2024-03-30T02:35:07.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.014+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa058112e10 con 0x7fa058073810 2024-03-30T02:35:07.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.015+0000 7fa04fe00700 1 -- 172.21.15.67:0/3994696284 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) v1 ==== 81+0+591 (secure 0 0 0) 0x7fa04800fe60 con 0x7fa058073810 2024-03-30T02:35:07.016 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2119993048 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:07.016 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2682757440 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:07.016 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2682757440 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:07.016 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/598232861 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:07.017 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/4186481961 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:07.017 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3533289986 2024-03-31T02:29:26.929980+0000 2024-03-30T02:35:07.017 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/4186481961 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:07.017 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1250300809 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:07.017 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2693891742 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:07.017 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3435746414 2024-03-31T02:29:01.908985+0000 2024-03-30T02:35:07.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa03c0a1120 msgr2=0x7fa03c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:07.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa03c0a1120 0x7fa03c0a35e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa044009fd0 tx=0x7fa044009260 comp rx=0 tx=0).stop 2024-03-30T02:35:07.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058073810 msgr2=0x7fa05819d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:07.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058073810 0x7fa05819d400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa04800eb30 tx=0x7fa04800ee40 comp rx=0 tx=0).stop 2024-03-30T02:35:07.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 shutdown_connections 2024-03-30T02:35:07.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa03c0a1120 0x7fa03c0a35e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:07.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa05810d350 0x7fa0581a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:07.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058073810 0x7fa05819d400 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:07.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 --2- 172.21.15.67:0/3994696284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa058074df0 0x7fa05819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:07.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 >> 172.21.15.67:0/3994696284 conn(0x7fa0580fc2f0 msgr2=0x7fa0581006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:07.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 shutdown_connections 2024-03-30T02:35:07.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:07.018+0000 7fa05e600700 1 -- 172.21.15.67:0/3994696284 wait complete. 2024-03-30T02:35:07.020 INFO:teuthology.orchestra.run.smithi067.stderr:listed 10 entries 2024-03-30T02:35:07.037 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-03-30T02:35:07.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:35:07.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-30T02:35:07.102 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[90111]: fuse finished with error 0 and tester_r 0 2024-03-30T02:35:07.106 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T02:35:07.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:07 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4263503384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:35:07.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:07 smithi067 ceph-mon[37808]: pgmap v243: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-30T02:35:07.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:07 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3994696284' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T02:35:07.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:07 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4263503384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:35:07.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:07 smithi082 ceph-mon[79176]: pgmap v243: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-30T02:35:07.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:07 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3994696284' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T02:35:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:07 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4263503384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:35:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:07 smithi170 ceph-mon[78798]: pgmap v243: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-30T02:35:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:07 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3994696284' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T02:35:10.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:10 smithi067 ceph-mon[37808]: pgmap v244: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:10.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:10 smithi170 ceph-mon[78798]: pgmap v244: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:10.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:10 smithi082 ceph-mon[79176]: pgmap v244: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:12.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:12 smithi067 ceph-mon[37808]: pgmap v245: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:12.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:12 smithi170 ceph-mon[78798]: pgmap v245: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:12.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:12 smithi082 ceph-mon[79176]: pgmap v245: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:13.208 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:35:13.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:35:13.208 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-30T02:35:13.273 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:35:13.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.477+0000 7f5253400700 1 -- 172.21.15.67:0/1689040241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c101e70 msgr2=0x7f524c1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:13.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.477+0000 7f5253400700 1 --2- 172.21.15.67:0/1689040241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c101e70 0x7f524c1022d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f52380099c0 tx=0x7f5238009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.477+0000 7f5253400700 1 -- 172.21.15.67:0/1689040241 shutdown_connections 2024-03-30T02:35:13.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.477+0000 7f5253400700 1 --2- 172.21.15.67:0/1689040241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c102810 0x7f524c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.477+0000 7f5253400700 1 --2- 172.21.15.67:0/1689040241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c100c80 0x7f524c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.477+0000 7f5253400700 1 --2- 172.21.15.67:0/1689040241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c101e70 0x7f524c1022d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.477+0000 7f5253400700 1 -- 172.21.15.67:0/1689040241 >> 172.21.15.67:0/1689040241 conn(0x7f524c0fc1f0 msgr2=0x7f524c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:13.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.477+0000 7f5253400700 1 -- 172.21.15.67:0/1689040241 shutdown_connections 2024-03-30T02:35:13.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.477+0000 7f5253400700 1 -- 172.21.15.67:0/1689040241 wait complete. 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.478+0000 7f5253400700 1 Processor -- start 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.478+0000 7f5253400700 1 -- start start 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.478+0000 7f5253400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c100c80 0x7f524c198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.478+0000 7f5253400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c101e70 0x7f524c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.478+0000 7f5253400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c102810 0x7f524c1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.478+0000 7f5253400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1a0940 con 0x7f524c100c80 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.478+0000 7f5253400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1a0a80 con 0x7f524c102810 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.478+0000 7f5253400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1a0c10 con 0x7f524c101e70 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5250a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c100c80 0x7f524c198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5250a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c100c80 0x7f524c198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58392/0 (socket says 172.21.15.67:58392) 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5250a00700 1 -- 172.21.15.67:0/4177131324 learned_addr learned my addr 172.21.15.67:0/4177131324 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f524be00700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c101e70 0x7f524c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5251400700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c102810 0x7f524c1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:13.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5250a00700 1 -- 172.21.15.67:0/4177131324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c101e70 msgr2=0x7f524c1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5250a00700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c101e70 0x7f524c1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5250a00700 1 -- 172.21.15.67:0/4177131324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c102810 msgr2=0x7f524c1a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5250a00700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c102810 0x7f524c1a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5250a00700 1 -- 172.21.15.67:0/4177131324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380095f0 con 0x7f524c100c80 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5250a00700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c100c80 0x7f524c198e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f524000e970 tx=0x7f524000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.479+0000 7f5249600700 1 -- 172.21.15.67:0/4177131324 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240009790 con 0x7f524c100c80 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.480+0000 7f5249600700 1 -- 172.21.15.67:0/4177131324 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f524000c7e0 con 0x7f524c100c80 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.480+0000 7f5249600700 1 -- 172.21.15.67:0/4177131324 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240003c70 con 0x7f524c100c80 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.480+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c1a0ef0 con 0x7f524c100c80 2024-03-30T02:35:13.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.480+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c1a12c0 con 0x7f524c100c80 2024-03-30T02:35:13.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.481+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c067a80 con 0x7f524c100c80 2024-03-30T02:35:13.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.483+0000 7f5249600700 1 -- 172.21.15.67:0/4177131324 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f524000cc50 con 0x7f524c100c80 2024-03-30T02:35:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.484+0000 7f5249600700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f52340a11a0 0x7f52340a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.484+0000 7f5249600700 1 -- 172.21.15.67:0/4177131324 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5240014070 con 0x7f524c100c80 2024-03-30T02:35:13.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.484+0000 7f524be00700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f52340a11a0 0x7f52340a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:13.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.485+0000 7f524be00700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f52340a11a0 0x7f52340a3660 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5238009990 tx=0x7f5238000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:13.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.488+0000 7f5249600700 1 -- 172.21.15.67:0/4177131324 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f524c067a80 con 0x7f524c100c80 2024-03-30T02:35:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.742+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f524c067a80 con 0x7f524c100c80 2024-03-30T02:35:13.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.743+0000 7f5249600700 1 -- 172.21.15.67:0/4177131324 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f524c067a80 con 0x7f524c100c80 2024-03-30T02:35:13.745 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:13.745 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:51.268204+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:35:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f52340a11a0 msgr2=0x7f52340a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f52340a11a0 0x7f52340a3660 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5238009990 tx=0x7f5238000bc0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c100c80 msgr2=0x7f524c198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c100c80 0x7f524c198e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f524000e970 tx=0x7f524000ec80 comp rx=0 tx=0).stop 2024-03-30T02:35:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 shutdown_connections 2024-03-30T02:35:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f52340a11a0 0x7f52340a3660 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f524c100c80 0x7f524c198e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c102810 0x7f524c1a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 --2- 172.21.15.67:0/4177131324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c101e70 0x7f524c1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 >> 172.21.15.67:0/4177131324 conn(0x7f524c0fc1f0 msgr2=0x7f524c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 shutdown_connections 2024-03-30T02:35:13.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.746+0000 7f5253400700 1 -- 172.21.15.67:0/4177131324 wait complete. 2024-03-30T02:35:13.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:35:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.963+0000 7f489a600700 1 -- 172.21.15.67:0/2313002186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48941027e0 msgr2=0x7f4894109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.963+0000 7f489a600700 1 --2- 172.21.15.67:0/2313002186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48941027e0 0x7f4894109090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f488c00b0d0 tx=0x7f488c00b3e0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.963+0000 7f489a600700 1 -- 172.21.15.67:0/2313002186 shutdown_connections 2024-03-30T02:35:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.963+0000 7f489a600700 1 --2- 172.21.15.67:0/2313002186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48941027e0 0x7f4894109090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.963+0000 7f489a600700 1 --2- 172.21.15.67:0/2313002186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4894100c50 0x7f4894101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.963+0000 7f489a600700 1 --2- 172.21.15.67:0/2313002186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4894101e40 0x7f48941022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.963+0000 7f489a600700 1 -- 172.21.15.67:0/2313002186 >> 172.21.15.67:0/2313002186 conn(0x7f48940fc1c0 msgr2=0x7f48940fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.963+0000 7f489a600700 1 -- 172.21.15.67:0/2313002186 shutdown_connections 2024-03-30T02:35:13.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.963+0000 7f489a600700 1 -- 172.21.15.67:0/2313002186 wait complete. 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.964+0000 7f489a600700 1 Processor -- start 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.964+0000 7f489a600700 1 -- start start 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f489a600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4894100c50 0x7f4894074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f489a600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4894101e40 0x7f48940731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f489a600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48941027e0 0x7f48940736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f489a600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894073e80 con 0x7f48941027e0 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f489a600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894073ff0 con 0x7f4894101e40 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f489a600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48941a0d60 con 0x7f4894100c50 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f4893e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4894101e40 0x7f48940731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:13.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f4899c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48941027e0 0x7f48940736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f4893e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4894101e40 0x7f48940731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55988/0 (socket says 172.21.15.67:55988) 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f4893e00700 1 -- 172.21.15.67:0/2220836821 learned_addr learned my addr 172.21.15.67:0/2220836821 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f4899200700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4894100c50 0x7f4894074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f4899c00700 1 -- 172.21.15.67:0/2220836821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4894100c50 msgr2=0x7f4894074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.965+0000 7f4899c00700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4894100c50 0x7f4894074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f4899c00700 1 -- 172.21.15.67:0/2220836821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4894101e40 msgr2=0x7f48940731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f4899c00700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4894101e40 0x7f48940731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f4899c00700 1 -- 172.21.15.67:0/2220836821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4880009520 con 0x7f48941027e0 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f4893e00700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4894101e40 0x7f48940731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f4899c00700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48941027e0 0x7f48940736e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f488c007b60 tx=0x7f488c008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f4891600700 1 -- 172.21.15.67:0/2220836821 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00e040 con 0x7f48941027e0 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f4891600700 1 -- 172.21.15.67:0/2220836821 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f488c005070 con 0x7f48941027e0 2024-03-30T02:35:13.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f4891600700 1 -- 172.21.15.67:0/2220836821 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c01d6d0 con 0x7f48941027e0 2024-03-30T02:35:13.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c009d00 con 0x7f48941027e0 2024-03-30T02:35:13.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.966+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48941a11d0 con 0x7f48941027e0 2024-03-30T02:35:13.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.967+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4894108070 con 0x7f48941027e0 2024-03-30T02:35:13.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.969+0000 7f4891600700 1 -- 172.21.15.67:0/2220836821 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f488c0051e0 con 0x7f48941027e0 2024-03-30T02:35:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.970+0000 7f4891600700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f487c0a1120 0x7f487c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.971+0000 7f4891600700 1 -- 172.21.15.67:0/2220836821 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f488c0cd9b0 con 0x7f48941027e0 2024-03-30T02:35:13.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.971+0000 7f4899200700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f487c0a1120 0x7f487c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:13.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.971+0000 7f4899200700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f487c0a1120 0x7f487c0a35e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4894101ca0 tx=0x7f4888009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:13.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:13.975+0000 7f4891600700 1 -- 172.21.15.67:0/2220836821 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f488c096f80 con 0x7f48941027e0 2024-03-30T02:35:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.226+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f489410c4c0 con 0x7f48941027e0 2024-03-30T02:35:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.228+0000 7f4891600700 1 -- 172.21.15.67:0/2220836821 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f488c09a5a0 con 0x7f48941027e0 2024-03-30T02:35:14.229 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:14.229 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:51.268204+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:35:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f487c0a1120 msgr2=0x7f487c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f487c0a1120 0x7f487c0a35e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4894101ca0 tx=0x7f4888009260 comp rx=0 tx=0).stop 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48941027e0 msgr2=0x7f48940736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48941027e0 0x7f48940736e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f488c007b60 tx=0x7f488c008e40 comp rx=0 tx=0).stop 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 shutdown_connections 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f487c0a1120 0x7f487c0a35e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f48941027e0 0x7f48940736e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4894100c50 0x7f4894074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 --2- 172.21.15.67:0/2220836821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4894101e40 0x7f48940731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.230+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 >> 172.21.15.67:0/2220836821 conn(0x7f48940fc1c0 msgr2=0x7f48941051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.231+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 shutdown_connections 2024-03-30T02:35:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.231+0000 7f489a600700 1 -- 172.21.15.67:0/2220836821 wait complete. 2024-03-30T02:35:14.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T02:35:14.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:14 smithi067 ceph-mon[37808]: pgmap v246: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:14.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:14 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4177131324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:14.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:14 smithi170 ceph-mon[78798]: pgmap v246: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:14.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:14 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4177131324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.450+0000 7fef97e00700 1 -- 172.21.15.67:0/3913681239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef90100d80 msgr2=0x7fef901011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.450+0000 7fef97e00700 1 --2- 172.21.15.67:0/3913681239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef90100d80 0x7fef901011a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef84009920 tx=0x7fef84009c30 comp rx=0 tx=0).stop 2024-03-30T02:35:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.451+0000 7fef97e00700 1 -- 172.21.15.67:0/3913681239 shutdown_connections 2024-03-30T02:35:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.451+0000 7fef97e00700 1 --2- 172.21.15.67:0/3913681239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef90102910 0x7fef901091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.451+0000 7fef97e00700 1 --2- 172.21.15.67:0/3913681239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef90100d80 0x7fef901011a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.451+0000 7fef97e00700 1 --2- 172.21.15.67:0/3913681239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef90101f70 0x7fef901023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.451+0000 7fef97e00700 1 -- 172.21.15.67:0/3913681239 >> 172.21.15.67:0/3913681239 conn(0x7fef900fc2f0 msgr2=0x7fef900fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.451+0000 7fef97e00700 1 -- 172.21.15.67:0/3913681239 shutdown_connections 2024-03-30T02:35:14.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.451+0000 7fef97e00700 1 -- 172.21.15.67:0/3913681239 wait complete. 2024-03-30T02:35:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.452+0000 7fef97e00700 1 Processor -- start 2024-03-30T02:35:14.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.452+0000 7fef97e00700 1 -- start start 2024-03-30T02:35:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef97e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef90100d80 0x7fef90199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef97e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef90101f70 0x7fef90199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef97e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef90102910 0x7fef901a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef97e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901a0b30 con 0x7fef90101f70 2024-03-30T02:35:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef97e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901a0c70 con 0x7fef90102910 2024-03-30T02:35:14.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef97e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef901a0e00 con 0x7fef90100d80 2024-03-30T02:35:14.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef95400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef90100d80 0x7fef90199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:14.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef94a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef90101f70 0x7fef90199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef95400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef90100d80 0x7fef90199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55708/0 (socket says 172.21.15.67:55708) 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef95400700 1 -- 172.21.15.67:0/1016976717 learned_addr learned my addr 172.21.15.67:0/1016976717 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.453+0000 7fef95e00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef90102910 0x7fef901a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef94a00700 1 -- 172.21.15.67:0/1016976717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef90100d80 msgr2=0x7fef90199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef94a00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef90100d80 0x7fef90199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef94a00700 1 -- 172.21.15.67:0/1016976717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef90102910 msgr2=0x7fef901a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef94a00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef90102910 0x7fef901a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef94a00700 1 -- 172.21.15.67:0/1016976717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef84009520 con 0x7fef90101f70 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef95400700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef90100d80 0x7fef90199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef94a00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef90101f70 0x7fef90199540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9f0 tx=0x7fef7c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef8e000700 1 -- 172.21.15.67:0/1016976717 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00c9c0 con 0x7fef90101f70 2024-03-30T02:35:14.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef8e000700 1 -- 172.21.15.67:0/1016976717 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef7c010430 con 0x7fef90101f70 2024-03-30T02:35:14.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef901a10e0 con 0x7fef90101f70 2024-03-30T02:35:14.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef8e000700 1 -- 172.21.15.67:0/1016976717 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c018680 con 0x7fef90101f70 2024-03-30T02:35:14.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.454+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef901a15f0 con 0x7fef90101f70 2024-03-30T02:35:14.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.455+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef90067af0 con 0x7fef90101f70 2024-03-30T02:35:14.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.457+0000 7fef8e000700 1 -- 172.21.15.67:0/1016976717 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fef7c010860 con 0x7fef90101f70 2024-03-30T02:35:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.459+0000 7fef8e000700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef780a1120 0x7fef780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:14.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.460+0000 7fef8e000700 1 -- 172.21.15.67:0/1016976717 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fef7c014070 con 0x7fef90101f70 2024-03-30T02:35:14.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.460+0000 7fef95400700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef780a1120 0x7fef780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:14.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.460+0000 7fef95400700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef780a1120 0x7fef780a35e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef8400fb70 tx=0x7fef84019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:14.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.462+0000 7fef8e000700 1 -- 172.21.15.67:0/1016976717 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fef7c096320 con 0x7fef90101f70 2024-03-30T02:35:14.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:14 smithi082 ceph-mon[79176]: pgmap v246: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:14.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:14 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4177131324' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:14.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:14.730+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fef9010c590 con 0x7fef90101f70 2024-03-30T02:35:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.074+0000 7fef8e000700 1 -- 172.21.15.67:0/1016976717 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fef7c099940 con 0x7fef90101f70 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.076+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef780a1120 msgr2=0x7fef780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.076+0000 7fef97e00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef780a1120 0x7fef780a35e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef8400fb70 tx=0x7fef84019040 comp rx=0 tx=0).stop 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.076+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef90101f70 msgr2=0x7fef90199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.076+0000 7fef97e00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef90101f70 0x7fef90199540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9f0 tx=0x7fef7c00ed00 comp rx=0 tx=0).stop 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.077+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 shutdown_connections 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.077+0000 7fef97e00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef780a1120 0x7fef780a35e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.077+0000 7fef97e00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef90102910 0x7fef901a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.077+0000 7fef97e00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef90100d80 0x7fef90199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.077+0000 7fef97e00700 1 --2- 172.21.15.67:0/1016976717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef90101f70 0x7fef90199540 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.077+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 >> 172.21.15.67:0/1016976717 conn(0x7fef900fc2f0 msgr2=0x7fef901077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.077+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 shutdown_connections 2024-03-30T02:35:15.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.077+0000 7fef97e00700 1 -- 172.21.15.67:0/1016976717 wait complete. 2024-03-30T02:35:15.097 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-30T02:35:15.097 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-30T02:35:15.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.299+0000 7f8b5de00700 1 -- 172.21.15.67:0/3008625982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 msgr2=0x7f8b581023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:15.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.299+0000 7f8b5de00700 1 --2- 172.21.15.67:0/3008625982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 0x7f8b581023d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b440099c0 tx=0x7f8b44009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.299+0000 7f8b5de00700 1 -- 172.21.15.67:0/3008625982 shutdown_connections 2024-03-30T02:35:15.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.299+0000 7f8b5de00700 1 --2- 172.21.15.67:0/3008625982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b58102910 0x7f8b581091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.299+0000 7f8b5de00700 1 --2- 172.21.15.67:0/3008625982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b58100d80 0x7f8b581011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.299+0000 7f8b5de00700 1 --2- 172.21.15.67:0/3008625982 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 0x7f8b581023d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.299+0000 7f8b5de00700 1 -- 172.21.15.67:0/3008625982 >> 172.21.15.67:0/3008625982 conn(0x7f8b580fc2f0 msgr2=0x7f8b580fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:15.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.299+0000 7f8b5de00700 1 -- 172.21.15.67:0/3008625982 shutdown_connections 2024-03-30T02:35:15.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.299+0000 7f8b5de00700 1 -- 172.21.15.67:0/3008625982 wait complete. 2024-03-30T02:35:15.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.300+0000 7f8b5de00700 1 Processor -- start 2024-03-30T02:35:15.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.300+0000 7f8b5de00700 1 -- start start 2024-03-30T02:35:15.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b5de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b58100d80 0x7f8b58198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:15.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b5de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 0x7f8b58199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:15.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b5de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b58102910 0x7f8b581a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:15.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b5de00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581a0a50 con 0x7f8b58101f70 2024-03-30T02:35:15.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b5de00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581a0b90 con 0x7f8b58100d80 2024-03-30T02:35:15.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b5de00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581a0d20 con 0x7f8b58102910 2024-03-30T02:35:15.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 0x7f8b58199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:15.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b58100d80 0x7f8b58198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:15.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 0x7f8b58199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58470/0 (socket says 172.21.15.67:58470) 2024-03-30T02:35:15.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56000700 1 -- 172.21.15.67:0/2445190744 learned_addr learned my addr 172.21.15.67:0/2445190744 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:15.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b57400700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b58102910 0x7f8b581a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:15.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56000700 1 -- 172.21.15.67:0/2445190744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b58102910 msgr2=0x7f8b581a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:15.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56000700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b58102910 0x7f8b581a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56000700 1 -- 172.21.15.67:0/2445190744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b58100d80 msgr2=0x7f8b58198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56000700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b58100d80 0x7f8b58198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56000700 1 -- 172.21.15.67:0/2445190744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440095f0 con 0x7f8b58101f70 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b57400700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b58102910 0x7f8b581a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.301+0000 7f8b56a00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b58100d80 0x7f8b58198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.302+0000 7f8b56000700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 0x7f8b58199460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8b44003730 tx=0x7f8b4400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.302+0000 7f8b5ca00700 1 -- 172.21.15.67:0/2445190744 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4401d070 con 0x7f8b58101f70 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.302+0000 7f8b5ca00700 1 -- 172.21.15.67:0/2445190744 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4400fe70 con 0x7f8b58101f70 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.302+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b581a0fa0 con 0x7f8b58101f70 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.302+0000 7f8b5ca00700 1 -- 172.21.15.67:0/2445190744 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44004bd0 con 0x7f8b58101f70 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.302+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b581a1360 con 0x7f8b58101f70 2024-03-30T02:35:15.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.303+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b58067af0 con 0x7f8b58101f70 2024-03-30T02:35:15.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.305+0000 7f8b5ca00700 1 -- 172.21.15.67:0/2445190744 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f8b44004d30 con 0x7f8b58101f70 2024-03-30T02:35:15.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.306+0000 7f8b5ca00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8b3c0a1120 0x7f8b3c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:15.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.306+0000 7f8b5ca00700 1 -- 172.21.15.67:0/2445190744 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8b440cda70 con 0x7f8b58101f70 2024-03-30T02:35:15.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.307+0000 7f8b56a00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8b3c0a1120 0x7f8b3c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:15.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.307+0000 7f8b56a00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8b3c0a1120 0x7f8b3c0a35e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8b58101dd0 tx=0x7f8b4800b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:15.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.310+0000 7f8b5ca00700 1 -- 172.21.15.67:0/2445190744 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8b44097040 con 0x7f8b58101f70 2024-03-30T02:35:15.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:15 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2220836821' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:15.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:15 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1016976717' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-30T02:35:15.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:15 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2220836821' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:15.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:15 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1016976717' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-30T02:35:15.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:15 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2220836821' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:15.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:15 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1016976717' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-30T02:35:15.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:15.577+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f8b581a1690 con 0x7f8b58101f70 2024-03-30T02:35:16.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.084+0000 7f8b5ca00700 1 -- 172.21.15.67:0/2445190744 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8b4409a660 con 0x7f8b58101f70 2024-03-30T02:35:16.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8b3c0a1120 msgr2=0x7f8b3c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:16.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8b3c0a1120 0x7f8b3c0a35e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8b58101dd0 tx=0x7f8b4800b410 comp rx=0 tx=0).stop 2024-03-30T02:35:16.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 msgr2=0x7f8b58199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:16.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 0x7f8b58199460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8b44003730 tx=0x7f8b4400fa00 comp rx=0 tx=0).stop 2024-03-30T02:35:16.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 shutdown_connections 2024-03-30T02:35:16.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8b3c0a1120 0x7f8b3c0a35e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:16.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b58102910 0x7f8b581a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:16.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b58100d80 0x7f8b58198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:16.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 --2- 172.21.15.67:0/2445190744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b58101f70 0x7f8b58199460 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:16.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 >> 172.21.15.67:0/2445190744 conn(0x7f8b580fc2f0 msgr2=0x7f8b58107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:16.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 shutdown_connections 2024-03-30T02:35:16.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.087+0000 7f8b5de00700 1 -- 172.21.15.67:0/2445190744 wait complete. 2024-03-30T02:35:16.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-30T02:35:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.328+0000 7fee5c400700 1 -- 172.21.15.67:0/2003314537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee54106a40 msgr2=0x7fee54108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.328+0000 7fee5c400700 1 --2- 172.21.15.67:0/2003314537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee54106a40 0x7fee54108eb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009f90 tx=0x7fee4c00b330 comp rx=0 tx=0).stop 2024-03-30T02:35:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.328+0000 7fee5c400700 1 -- 172.21.15.67:0/2003314537 shutdown_connections 2024-03-30T02:35:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.328+0000 7fee5c400700 1 --2- 172.21.15.67:0/2003314537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee54106a40 0x7fee54108eb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.328+0000 7fee5c400700 1 --2- 172.21.15.67:0/2003314537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee541017e0 0x7fee54103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.328+0000 7fee5c400700 1 --2- 172.21.15.67:0/2003314537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54104110 0x7fee54106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.328+0000 7fee5c400700 1 -- 172.21.15.67:0/2003314537 >> 172.21.15.67:0/2003314537 conn(0x7fee540fb1d0 msgr2=0x7fee540fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.328+0000 7fee5c400700 1 -- 172.21.15.67:0/2003314537 shutdown_connections 2024-03-30T02:35:16.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.328+0000 7fee5c400700 1 -- 172.21.15.67:0/2003314537 wait complete. 2024-03-30T02:35:16.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.329+0000 7fee5c400700 1 Processor -- start 2024-03-30T02:35:16.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee5c400700 1 -- start start 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee5c400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee541017e0 0x7fee54196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee5c400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee54104110 0x7fee541972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee5c400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54106a40 0x7fee5419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee5c400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5419e870 con 0x7fee541017e0 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee5c400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5419e9e0 con 0x7fee54104110 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee5c400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5419eba0 con 0x7fee54106a40 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee59a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee541017e0 0x7fee54196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee59a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee541017e0 0x7fee54196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57404/0 (socket says 172.21.15.67:57404) 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee59a00700 1 -- 172.21.15.67:0/4125451052 learned_addr learned my addr 172.21.15.67:0/4125451052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee5a400700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54106a40 0x7fee5419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.330+0000 7fee59000700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee54104110 0x7fee541972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee59a00700 1 -- 172.21.15.67:0/4125451052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54106a40 msgr2=0x7fee5419e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:16.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee59a00700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54106a40 0x7fee5419e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee59a00700 1 -- 172.21.15.67:0/4125451052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee54104110 msgr2=0x7fee541972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee59a00700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee54104110 0x7fee541972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee59a00700 1 -- 172.21.15.67:0/4125451052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c009c40 con 0x7fee541017e0 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee59a00700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee541017e0 0x7fee54196d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee4800e9c0 tx=0x7fee4800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee52000700 1 -- 172.21.15.67:0/4125451052 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48004a30 con 0x7fee541017e0 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee52000700 1 -- 172.21.15.67:0/4125451052 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee4800cc80 con 0x7fee541017e0 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5419ee80 con 0x7fee541017e0 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee52000700 1 -- 172.21.15.67:0/4125451052 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee480054e0 con 0x7fee541017e0 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.331+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5419f2a0 con 0x7fee541017e0 2024-03-30T02:35:16.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.332+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee540fcd50 con 0x7fee541017e0 2024-03-30T02:35:16.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.334+0000 7fee52000700 1 -- 172.21.15.67:0/4125451052 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fee48004b90 con 0x7fee541017e0 2024-03-30T02:35:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.336+0000 7fee52000700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fee3c0a1120 0x7fee3c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.336+0000 7fee52000700 1 -- 172.21.15.67:0/4125451052 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fee48014070 con 0x7fee541017e0 2024-03-30T02:35:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.336+0000 7fee59000700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fee3c0a1120 0x7fee3c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:16.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.337+0000 7fee59000700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fee3c0a1120 0x7fee3c0a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee40009c60 tx=0x7fee40009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:16.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.338+0000 7fee52000700 1 -- 172.21.15.67:0/4125451052 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee480962e0 con 0x7fee541017e0 2024-03-30T02:35:16.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:16 smithi067 ceph-mon[37808]: pgmap v247: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:16.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:16 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1016976717' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-30T02:35:16.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:16 smithi067 ceph-mon[37808]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:35:16.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:16 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2445190744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-30T02:35:16.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:16 smithi170 ceph-mon[78798]: pgmap v247: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:16.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:16 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1016976717' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-30T02:35:16.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:16 smithi170 ceph-mon[78798]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:35:16.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:16 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2445190744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-30T02:35:16.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:16 smithi082 ceph-mon[79176]: pgmap v247: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-30T02:35:16.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:16 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1016976717' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-30T02:35:16.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:16 smithi082 ceph-mon[79176]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:35:16.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:16 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2445190744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-30T02:35:16.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:16.610+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fee5410c540 con 0x7fee541017e0 2024-03-30T02:35:17.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.093+0000 7fee52000700 1 -- 172.21.15.67:0/4125451052 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fee48099900 con 0x7fee541017e0 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fee3c0a1120 msgr2=0x7fee3c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fee3c0a1120 0x7fee3c0a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee40009c60 tx=0x7fee40009260 comp rx=0 tx=0).stop 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee541017e0 msgr2=0x7fee54196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee541017e0 0x7fee54196d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee4800e9c0 tx=0x7fee4800ecd0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 shutdown_connections 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fee3c0a1120 0x7fee3c0a35e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee541017e0 0x7fee54196d80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54106a40 0x7fee5419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 --2- 172.21.15.67:0/4125451052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee54104110 0x7fee541972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.096+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 >> 172.21.15.67:0/4125451052 conn(0x7fee540fb1d0 msgr2=0x7fee541022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.097+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 shutdown_connections 2024-03-30T02:35:17.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.097+0000 7fee5c400700 1 -- 172.21.15.67:0/4125451052 wait complete. 2024-03-30T02:35:17.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-30T02:35:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.325+0000 7f588ba00700 1 -- 172.21.15.67:0/1606341572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884100d80 msgr2=0x7f58841011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.325+0000 7f588ba00700 1 --2- 172.21.15.67:0/1606341572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884100d80 0x7f58841011a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f58780099c0 tx=0x7f5878009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.326+0000 7f588ba00700 1 -- 172.21.15.67:0/1606341572 shutdown_connections 2024-03-30T02:35:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.326+0000 7f588ba00700 1 --2- 172.21.15.67:0/1606341572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884100d80 0x7f58841011a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.326+0000 7f588ba00700 1 --2- 172.21.15.67:0/1606341572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5884102910 0x7f58841091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.326+0000 7f588ba00700 1 --2- 172.21.15.67:0/1606341572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5884101f70 0x7f58841023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.326+0000 7f588ba00700 1 -- 172.21.15.67:0/1606341572 >> 172.21.15.67:0/1606341572 conn(0x7f58840fc2f0 msgr2=0x7f58840fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.326+0000 7f588ba00700 1 -- 172.21.15.67:0/1606341572 shutdown_connections 2024-03-30T02:35:17.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.326+0000 7f588ba00700 1 -- 172.21.15.67:0/1606341572 wait complete. 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.327+0000 7f588ba00700 1 Processor -- start 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.327+0000 7f588ba00700 1 -- start start 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.327+0000 7f588ba00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5884100d80 0x7f5884074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.327+0000 7f588ba00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884101f70 0x7f58840731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.327+0000 7f588ba00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5884102910 0x7f58840736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.327+0000 7f588ba00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5884073e80 con 0x7f5884101f70 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.327+0000 7f588ba00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5884073ff0 con 0x7f5884100d80 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.327+0000 7f588ba00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58841a0d70 con 0x7f5884102910 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.327+0000 7f5883e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884101f70 0x7f58840731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5883e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884101f70 0x7f58840731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57416/0 (socket says 172.21.15.67:57416) 2024-03-30T02:35:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5883e00700 1 -- 172.21.15.67:0/1192122987 learned_addr learned my addr 172.21.15.67:0/1192122987 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5889000700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5884100d80 0x7f5884074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5889a00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5884102910 0x7f58840736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5883e00700 1 -- 172.21.15.67:0/1192122987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5884102910 msgr2=0x7f58840736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5883e00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5884102910 0x7f58840736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5883e00700 1 -- 172.21.15.67:0/1192122987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5884100d80 msgr2=0x7f5884074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5883e00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5884100d80 0x7f5884074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5883e00700 1 -- 172.21.15.67:0/1192122987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58780095f0 con 0x7f5884101f70 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5883e00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884101f70 0x7f58840731a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f587000d980 tx=0x7f587000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5881600700 1 -- 172.21.15.67:0/1192122987 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870004220 con 0x7f5884101f70 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.328+0000 7f5881600700 1 -- 172.21.15.67:0/1192122987 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5870007640 con 0x7f5884101f70 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.329+0000 7f5881600700 1 -- 172.21.15.67:0/1192122987 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870005690 con 0x7f5884101f70 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.329+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58841a0fa0 con 0x7f5884101f70 2024-03-30T02:35:17.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.329+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58841a13d0 con 0x7f5884101f70 2024-03-30T02:35:17.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.330+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5884108250 con 0x7f5884101f70 2024-03-30T02:35:17.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.332+0000 7f5881600700 1 -- 172.21.15.67:0/1192122987 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f5870004750 con 0x7f5884101f70 2024-03-30T02:35:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.333+0000 7f5881600700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f586c0a1120 0x7f586c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.333+0000 7f5881600700 1 -- 172.21.15.67:0/1192122987 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f58700cc910 con 0x7f5884101f70 2024-03-30T02:35:17.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.333+0000 7f5889000700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f586c0a1120 0x7f586c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:17.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.334+0000 7f5889000700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f586c0a1120 0x7f586c0a35e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5878006010 tx=0x7f5878000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:17.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.337+0000 7f5881600700 1 -- 172.21.15.67:0/1192122987 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5870095e50 con 0x7f5884101f70 2024-03-30T02:35:17.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:17 smithi067 ceph-mon[37808]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-30T02:35:17.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:17 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2445190744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-30T02:35:17.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:17 smithi067 ceph-mon[37808]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-30T02:35:17.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:17 smithi067 ceph-mon[37808]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:35:17.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:17 smithi067 ceph-mon[37808]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-30T02:35:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:17 smithi067 ceph-mon[37808]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-30T02:35:17.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:17 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4125451052' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-30T02:35:17.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:17 smithi170 ceph-mon[78798]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-30T02:35:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:17 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2445190744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-30T02:35:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:17 smithi170 ceph-mon[78798]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-30T02:35:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:17 smithi170 ceph-mon[78798]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:35:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:17 smithi170 ceph-mon[78798]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-30T02:35:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:17 smithi170 ceph-mon[78798]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-30T02:35:17.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:17 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4125451052' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-30T02:35:17.500 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:35:17 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: 2024-03-30T02:35:17.113+0000 7f86ad81d700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-30T02:35:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:17 smithi082 ceph-mon[79176]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-30T02:35:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:17 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2445190744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-30T02:35:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:17 smithi082 ceph-mon[79176]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-30T02:35:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:17 smithi082 ceph-mon[79176]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-30T02:35:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:17 smithi082 ceph-mon[79176]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-03-30T02:35:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:17 smithi082 ceph-mon[79176]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-03-30T02:35:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:17 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4125451052' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-30T02:35:17.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:17.604+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f5884067af0 con 0x7f5884101f70 2024-03-30T02:35:18.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.100+0000 7f5881600700 1 -- 172.21.15.67:0/1192122987 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5870016020 con 0x7f5884101f70 2024-03-30T02:35:18.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f586c0a1120 msgr2=0x7f586c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f586c0a1120 0x7f586c0a35e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5878006010 tx=0x7f5878000bc0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884101f70 msgr2=0x7f58840731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884101f70 0x7f58840731a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f587000d980 tx=0x7f587000dc90 comp rx=0 tx=0).stop 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 shutdown_connections 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f586c0a1120 0x7f586c0a35e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5884102910 0x7f58840736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5884100d80 0x7f5884074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 --2- 172.21.15.67:0/1192122987 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5884101f70 0x7f58840731a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.103+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 >> 172.21.15.67:0/1192122987 conn(0x7f58840fc2f0 msgr2=0x7f5884105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.104+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 shutdown_connections 2024-03-30T02:35:18.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.104+0000 7f588ba00700 1 -- 172.21.15.67:0/1192122987 wait complete. 2024-03-30T02:35:18.129 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-30T02:35:18.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-30T02:35:18.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.333+0000 7f55c2a00700 1 -- 172.21.15.67:0/2133095772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 msgr2=0x7f55bc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:18.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.334+0000 7f55c2a00700 1 --2- 172.21.15.67:0/2133095772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 0x7f55bc1023d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f55a80099c0 tx=0x7f55a8009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.334+0000 7f55c2a00700 1 -- 172.21.15.67:0/2133095772 shutdown_connections 2024-03-30T02:35:18.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.334+0000 7f55c2a00700 1 --2- 172.21.15.67:0/2133095772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55bc102910 0x7f55bc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.334+0000 7f55c2a00700 1 --2- 172.21.15.67:0/2133095772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55bc100d80 0x7f55bc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.334+0000 7f55c2a00700 1 --2- 172.21.15.67:0/2133095772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 0x7f55bc1023d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.334+0000 7f55c2a00700 1 -- 172.21.15.67:0/2133095772 >> 172.21.15.67:0/2133095772 conn(0x7f55bc0fc2f0 msgr2=0x7f55bc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:18.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.334+0000 7f55c2a00700 1 -- 172.21.15.67:0/2133095772 shutdown_connections 2024-03-30T02:35:18.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.334+0000 7f55c2a00700 1 -- 172.21.15.67:0/2133095772 wait complete. 2024-03-30T02:35:18.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.335+0000 7f55c2a00700 1 Processor -- start 2024-03-30T02:35:18.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.335+0000 7f55c2a00700 1 -- start start 2024-03-30T02:35:18.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.335+0000 7f55c2a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55bc100d80 0x7f55bc198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:18.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.335+0000 7f55c2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 0x7f55bc1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:18.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.335+0000 7f55c2a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55bc102910 0x7f55bc1a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:18.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.335+0000 7f55c2a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc1a0a60 con 0x7f55bc101f70 2024-03-30T02:35:18.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.335+0000 7f55c2a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc1a0ba0 con 0x7f55bc100d80 2024-03-30T02:35:18.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.335+0000 7f55c2a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc1a0d30 con 0x7f55bc102910 2024-03-30T02:35:18.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 0x7f55bc1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 0x7f55bc1994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57436/0 (socket says 172.21.15.67:57436) 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bb400700 1 -- 172.21.15.67:0/1066115311 learned_addr learned my addr 172.21.15.67:0/1066115311 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bbe00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55bc100d80 0x7f55bc198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55c0a00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55bc102910 0x7f55bc1a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bb400700 1 -- 172.21.15.67:0/1066115311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55bc102910 msgr2=0x7f55bc1a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bb400700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55bc102910 0x7f55bc1a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bb400700 1 -- 172.21.15.67:0/1066115311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55bc100d80 msgr2=0x7f55bc198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bb400700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55bc100d80 0x7f55bc198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bb400700 1 -- 172.21.15.67:0/1066115311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80095f0 con 0x7f55bc101f70 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.336+0000 7f55bb400700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 0x7f55bc1994c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006010 tx=0x7f55a800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.337+0000 7f55b8c00700 1 -- 172.21.15.67:0/1066115311 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a801d070 con 0x7f55bc101f70 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.337+0000 7f55b8c00700 1 -- 172.21.15.67:0/1066115311 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a8005030 con 0x7f55bc101f70 2024-03-30T02:35:18.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.337+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc1a0fb0 con 0x7f55bc101f70 2024-03-30T02:35:18.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.337+0000 7f55b8c00700 1 -- 172.21.15.67:0/1066115311 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a8004bb0 con 0x7f55bc101f70 2024-03-30T02:35:18.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.337+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc1a1370 con 0x7f55bc101f70 2024-03-30T02:35:18.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.338+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc067af0 con 0x7f55bc101f70 2024-03-30T02:35:18.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.340+0000 7f55b8c00700 1 -- 172.21.15.67:0/1066115311 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f55a80051a0 con 0x7f55bc101f70 2024-03-30T02:35:18.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.341+0000 7f55b8c00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f55a00a1120 0x7f55a00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:18.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.341+0000 7f55b8c00700 1 -- 172.21.15.67:0/1066115311 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f55a80cdc00 con 0x7f55bc101f70 2024-03-30T02:35:18.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.341+0000 7f55bbe00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f55a00a1120 0x7f55a00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:18.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.342+0000 7f55bbe00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f55a00a1120 0x7f55a00a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55bc101dd0 tx=0x7f55ac009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:18.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.344+0000 7f55b8c00700 1 -- 172.21.15.67:0/1066115311 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f55a8097230 con 0x7f55bc101f70 2024-03-30T02:35:18.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-mon[37808]: pgmap v248: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 0 op/s 2024-03-30T02:35:18.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] up:active 2024-03-30T02:35:18.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4125451052' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-30T02:35:18.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-mon[37808]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-30T02:35:18.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-mon[37808]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-30T02:35:18.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-mon[37808]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-03-30T02:35:18.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-mon[37808]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-30T02:35:18.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1192122987' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-30T02:35:18.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:18 smithi170 ceph-mon[78798]: pgmap v248: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 0 op/s 2024-03-30T02:35:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:18 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] up:active 2024-03-30T02:35:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:18 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4125451052' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-30T02:35:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:18 smithi170 ceph-mon[78798]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-30T02:35:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:18 smithi170 ceph-mon[78798]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-30T02:35:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:18 smithi170 ceph-mon[78798]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-03-30T02:35:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:18 smithi170 ceph-mon[78798]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-30T02:35:18.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:18 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1192122987' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-30T02:35:18.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:18 smithi082 ceph-mon[79176]: pgmap v248: 129 pgs: 129 active+clean; 581 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 0 op/s 2024-03-30T02:35:18.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:18 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] up:active 2024-03-30T02:35:18.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:18 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4125451052' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-30T02:35:18.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:18 smithi082 ceph-mon[79176]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-30T02:35:18.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:18 smithi082 ceph-mon[79176]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-03-30T02:35:18.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:18 smithi082 ceph-mon[79176]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-03-30T02:35:18.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:18 smithi082 ceph-mon[79176]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-03-30T02:35:18.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:18 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1192122987' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-30T02:35:18.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:18.597+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f55bc074700 con 0x7f55a00a1120 2024-03-30T02:35:18.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:35:18.603+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:35:18.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:35:18.603+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:35:18.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:35:18.603+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:35:18.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:35:18.603+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:35:18.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:35:18 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:35:18.603+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:35:19.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.131+0000 7f55b8c00700 1 -- 172.21.15.67:0/1066115311 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f55bc074700 con 0x7f55a00a1120 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f55a00a1120 msgr2=0x7f55a00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f55a00a1120 0x7f55a00a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55bc101dd0 tx=0x7f55ac009260 comp rx=0 tx=0).stop 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 msgr2=0x7f55bc1994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 0x7f55bc1994c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006010 tx=0x7f55a800f690 comp rx=0 tx=0).stop 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 shutdown_connections 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f55a00a1120 0x7f55a00a35e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55bc102910 0x7f55bc1a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55bc100d80 0x7f55bc198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 --2- 172.21.15.67:0/1066115311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55bc101f70 0x7f55bc1994c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 >> 172.21.15.67:0/1066115311 conn(0x7f55bc0fc2f0 msgr2=0x7f55bc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 shutdown_connections 2024-03-30T02:35:19.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.134+0000 7f55c2a00700 1 -- 172.21.15.67:0/1066115311 wait complete. 2024-03-30T02:35:19.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-30T02:35:19.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.360+0000 7f38ac600700 1 -- 172.21.15.67:0/181562764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 msgr2=0x7f38a41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.360+0000 7f38ac600700 1 --2- 172.21.15.67:0/181562764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 0x7f38a41011a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f38980099c0 tx=0x7f3898009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.360+0000 7f38ac600700 1 -- 172.21.15.67:0/181562764 shutdown_connections 2024-03-30T02:35:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.360+0000 7f38ac600700 1 --2- 172.21.15.67:0/181562764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 0x7f38a41011a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.360+0000 7f38ac600700 1 --2- 172.21.15.67:0/181562764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4102910 0x7f38a41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.360+0000 7f38ac600700 1 --2- 172.21.15.67:0/181562764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a4101f70 0x7f38a41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.360+0000 7f38ac600700 1 -- 172.21.15.67:0/181562764 >> 172.21.15.67:0/181562764 conn(0x7f38a40fc2f0 msgr2=0x7f38a40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.360+0000 7f38ac600700 1 -- 172.21.15.67:0/181562764 shutdown_connections 2024-03-30T02:35:19.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.360+0000 7f38ac600700 1 -- 172.21.15.67:0/181562764 wait complete. 2024-03-30T02:35:19.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.361+0000 7f38ac600700 1 Processor -- start 2024-03-30T02:35:19.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.361+0000 7f38ac600700 1 -- start start 2024-03-30T02:35:19.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38ac600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 0x7f38a4198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38ac600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4101f70 0x7f38a41994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38ac600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a4102910 0x7f38a41a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38ac600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41a0ab0 con 0x7f38a4100d80 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38ac600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41a0bf0 con 0x7f38a4101f70 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38ac600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a41a0d80 con 0x7f38a4102910 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38a9c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 0x7f38a4198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38a9c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 0x7f38a4198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57472/0 (socket says 172.21.15.67:57472) 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38a9c00700 1 -- 172.21.15.67:0/552351795 learned_addr learned my addr 172.21.15.67:0/552351795 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38a9200700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4101f70 0x7f38a41994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38aa600700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a4102910 0x7f38a41a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38a9c00700 1 -- 172.21.15.67:0/552351795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a4102910 msgr2=0x7f38a41a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38a9c00700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a4102910 0x7f38a41a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38a9c00700 1 -- 172.21.15.67:0/552351795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4101f70 msgr2=0x7f38a41994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38a9c00700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4101f70 0x7f38a41994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38a9c00700 1 -- 172.21.15.67:0/552351795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980095f0 con 0x7f38a4100d80 2024-03-30T02:35:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.362+0000 7f38aa600700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a4102910 0x7f38a41a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:35:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.363+0000 7f38a9c00700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 0x7f38a4198f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f389800f690 tx=0x7f3898005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.363+0000 7f38a2000700 1 -- 172.21.15.67:0/552351795 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389801c070 con 0x7f38a4100d80 2024-03-30T02:35:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.363+0000 7f38a2000700 1 -- 172.21.15.67:0/552351795 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f389800fe20 con 0x7f38a4100d80 2024-03-30T02:35:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.363+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a41a1000 con 0x7f38a4100d80 2024-03-30T02:35:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.363+0000 7f38a2000700 1 -- 172.21.15.67:0/552351795 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3898022dc0 con 0x7f38a4100d80 2024-03-30T02:35:19.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.363+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a41a1420 con 0x7f38a4100d80 2024-03-30T02:35:19.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.364+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4067af0 con 0x7f38a4100d80 2024-03-30T02:35:19.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.366+0000 7f38a2000700 1 -- 172.21.15.67:0/552351795 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f3898020070 con 0x7f38a4100d80 2024-03-30T02:35:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.367+0000 7f38a2000700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f388c0a1120 0x7f388c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.367+0000 7f38a2000700 1 -- 172.21.15.67:0/552351795 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f38980cd930 con 0x7f38a4100d80 2024-03-30T02:35:19.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.367+0000 7f38a9200700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f388c0a1120 0x7f388c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:19.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.368+0000 7f38a9200700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f388c0a1120 0x7f388c0a35e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3890009c60 tx=0x7f3890009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:19.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.371+0000 7f38a2000700 1 -- 172.21.15.67:0/552351795 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3898096f00 con 0x7f38a4100d80 2024-03-30T02:35:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:19 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] up:active 2024-03-30T02:35:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:19 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1192122987' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-30T02:35:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:19 smithi067 ceph-mon[37808]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-30T02:35:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:19 smithi067 ceph-mon[37808]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-03-30T02:35:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:19 smithi067 ceph-mon[37808]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-03-30T02:35:19.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:19 smithi067 ceph-mon[37808]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-30T02:35:19.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:19 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] up:active 2024-03-30T02:35:19.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:19 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1192122987' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-30T02:35:19.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:19 smithi170 ceph-mon[78798]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-30T02:35:19.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:19 smithi170 ceph-mon[78798]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-03-30T02:35:19.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:19 smithi170 ceph-mon[78798]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-03-30T02:35:19.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:19 smithi170 ceph-mon[78798]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-30T02:35:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:19 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] up:active 2024-03-30T02:35:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:19 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1192122987' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-30T02:35:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:19 smithi082 ceph-mon[79176]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-30T02:35:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:19 smithi082 ceph-mon[79176]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-03-30T02:35:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:19 smithi082 ceph-mon[79176]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-03-30T02:35:19.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:19 smithi082 ceph-mon[79176]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-03-30T02:35:19.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.625+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f38a4074700 con 0x7f388c0a1120 2024-03-30T02:35:19.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.651+0000 7f38a2000700 1 -- 172.21.15.67:0/552351795 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f38a4074700 con 0x7f388c0a1120 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f388c0a1120 msgr2=0x7f388c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f388c0a1120 0x7f388c0a35e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3890009c60 tx=0x7f3890009260 comp rx=0 tx=0).stop 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 msgr2=0x7f38a4198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 0x7f38a4198f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f389800f690 tx=0x7f3898005b00 comp rx=0 tx=0).stop 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 shutdown_connections 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f388c0a1120 0x7f388c0a35e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38a4100d80 0x7f38a4198f80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38a4102910 0x7f38a41a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 --2- 172.21.15.67:0/552351795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a4101f70 0x7f38a41994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 >> 172.21.15.67:0/552351795 conn(0x7f38a40fc2f0 msgr2=0x7f38a4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:19.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 shutdown_connections 2024-03-30T02:35:19.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.654+0000 7f38ac600700 1 -- 172.21.15.67:0/552351795 wait complete. 2024-03-30T02:35:19.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-30T02:35:19.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.878+0000 7f11c9e00700 1 -- 172.21.15.67:0/2193010375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11c4073810 msgr2=0x7f11c4073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.878+0000 7f11c9e00700 1 --2- 172.21.15.67:0/2193010375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11c4073810 0x7f11c4073c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009990 tx=0x7f11b0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.878+0000 7f11c9e00700 1 -- 172.21.15.67:0/2193010375 shutdown_connections 2024-03-30T02:35:19.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.878+0000 7f11c9e00700 1 --2- 172.21.15.67:0/2193010375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108c40 0x7f11c410b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.878+0000 7f11c9e00700 1 --2- 172.21.15.67:0/2193010375 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11c4074df0 0x7f11c4073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.878+0000 7f11c9e00700 1 --2- 172.21.15.67:0/2193010375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11c4073810 0x7f11c4073c90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.878+0000 7f11c9e00700 1 -- 172.21.15.67:0/2193010375 >> 172.21.15.67:0/2193010375 conn(0x7f11c40fc2f0 msgr2=0x7f11c40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:19.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.878+0000 7f11c9e00700 1 -- 172.21.15.67:0/2193010375 shutdown_connections 2024-03-30T02:35:19.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.878+0000 7f11c9e00700 1 -- 172.21.15.67:0/2193010375 wait complete. 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.879+0000 7f11c9e00700 1 Processor -- start 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c9e00700 1 -- start start 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11c4073810 0x7f11c4198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c9e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11c4074df0 0x7f11c41994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108c40 0x7f11c41a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c9e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c41a0aa0 con 0x7f11c4108c40 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c9e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c41a0be0 con 0x7f11c4073810 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c9e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c41a0d70 con 0x7f11c4074df0 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108c40 0x7f11c41a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108c40 0x7f11c41a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57486/0 (socket says 172.21.15.67:57486) 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c3400700 1 -- 172.21.15.67:0/3151893964 learned_addr learned my addr 172.21.15.67:0/3151893964 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.880+0000 7f11c2000700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11c4074df0 0x7f11c41994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:19.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c2a00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11c4073810 0x7f11c4198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c3400700 1 -- 172.21.15.67:0/3151893964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11c4074df0 msgr2=0x7f11c41994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c3400700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11c4074df0 0x7f11c41994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c3400700 1 -- 172.21.15.67:0/3151893964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11c4073810 msgr2=0x7f11c4198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c3400700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11c4073810 0x7f11c4198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c3400700 1 -- 172.21.15.67:0/3151893964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b00095f0 con 0x7f11c4108c40 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c2000700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11c4074df0 0x7f11c41994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c3400700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108c40 0x7f11c41a04d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00ef30 tx=0x7f11bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c8a00700 1 -- 172.21.15.67:0/3151893964 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc018070 con 0x7f11c4108c40 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c8a00700 1 -- 172.21.15.67:0/3151893964 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11bc00f040 con 0x7f11c4108c40 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c41a0ff0 con 0x7f11c4108c40 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c8a00700 1 -- 172.21.15.67:0/3151893964 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc00cdb0 con 0x7f11c4108c40 2024-03-30T02:35:19.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.881+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c41a1470 con 0x7f11c4108c40 2024-03-30T02:35:19.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.882+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c4107a00 con 0x7f11c4108c40 2024-03-30T02:35:19.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.884+0000 7f11c8a00700 1 -- 172.21.15.67:0/3151893964 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f11bc003a00 con 0x7f11c4108c40 2024-03-30T02:35:19.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.885+0000 7f11c8a00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f11a80a1120 0x7f11a80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:19.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.886+0000 7f11c8a00700 1 -- 172.21.15.67:0/3151893964 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f11bc0cd090 con 0x7f11c4108c40 2024-03-30T02:35:19.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.886+0000 7f11c2a00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f11a80a1120 0x7f11a80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:19.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.886+0000 7f11c2a00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f11a80a1120 0x7f11a80a35e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009bf0 tx=0x7f11b4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:19.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:19.888+0000 7f11c8a00700 1 -- 172.21.15.67:0/3151893964 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f11bc096660 con 0x7f11c4108c40 2024-03-30T02:35:20.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.154+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f11c402cd20 con 0x7f11a80a1120 2024-03-30T02:35:20.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.159+0000 7f11c8a00700 1 -- 172.21.15.67:0/3151893964 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f11c402cd20 con 0x7f11a80a1120 2024-03-30T02:35:20.160 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_0/8f0a9cf8-2f84-4448-bef9-4059bbf90e0a 2024-03-30T02:35:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f11a80a1120 msgr2=0x7f11a80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f11a80a1120 0x7f11a80a35e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009bf0 tx=0x7f11b4009260 comp rx=0 tx=0).stop 2024-03-30T02:35:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108c40 msgr2=0x7f11c41a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108c40 0x7f11c41a04d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00ef30 tx=0x7f11bc00c3a0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 shutdown_connections 2024-03-30T02:35:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f11a80a1120 0x7f11a80a35e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11c4108c40 0x7f11c41a04d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11c4073810 0x7f11c4198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 --2- 172.21.15.67:0/3151893964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11c4074df0 0x7f11c41994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 >> 172.21.15.67:0/3151893964 conn(0x7f11c40fc2f0 msgr2=0x7f11c4104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 shutdown_connections 2024-03-30T02:35:20.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.162+0000 7f11c9e00700 1 -- 172.21.15.67:0/3151893964 wait complete. 2024-03-30T02:35:20.181 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-30T02:35:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.387+0000 7f492d600700 1 -- 172.21.15.67:0/2965467180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102910 msgr2=0x7f49281091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.387+0000 7f492d600700 1 --2- 172.21.15.67:0/2965467180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102910 0x7f49281091c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f492000b0d0 tx=0x7f492000b3e0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.387+0000 7f492d600700 1 -- 172.21.15.67:0/2965467180 shutdown_connections 2024-03-30T02:35:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.387+0000 7f492d600700 1 --2- 172.21.15.67:0/2965467180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928102910 0x7f49281091c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.387+0000 7f492d600700 1 --2- 172.21.15.67:0/2965467180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4928100d80 0x7f49281011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.387+0000 7f492d600700 1 --2- 172.21.15.67:0/2965467180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928101f70 0x7f49281023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.387+0000 7f492d600700 1 -- 172.21.15.67:0/2965467180 >> 172.21.15.67:0/2965467180 conn(0x7f49280fc2f0 msgr2=0x7f49280fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.387+0000 7f492d600700 1 -- 172.21.15.67:0/2965467180 shutdown_connections 2024-03-30T02:35:20.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.387+0000 7f492d600700 1 -- 172.21.15.67:0/2965467180 wait complete. 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.388+0000 7f492d600700 1 Processor -- start 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f492d600700 1 -- start start 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f492d600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928100d80 0x7f4928074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f492d600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928101f70 0x7f49280731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f492d600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4928102910 0x7f49280736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f492d600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4928073d90 con 0x7f4928101f70 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f492d600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4928073f00 con 0x7f4928102910 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f492d600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49281a0f10 con 0x7f4928100d80 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f4926000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928101f70 0x7f49280731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f4926000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928101f70 0x7f49280731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57516/0 (socket says 172.21.15.67:57516) 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f4926000700 1 -- 172.21.15.67:0/3631476569 learned_addr learned my addr 172.21.15.67:0/3631476569 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f4927400700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4928102910 0x7f49280736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:20.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f4926a00700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928100d80 0x7f4928074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f4926000700 1 -- 172.21.15.67:0/3631476569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928100d80 msgr2=0x7f4928074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f4926000700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928100d80 0x7f4928074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.389+0000 7f4926000700 1 -- 172.21.15.67:0/3631476569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4928102910 msgr2=0x7f49280736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.390+0000 7f4926000700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4928102910 0x7f49280736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.390+0000 7f4926000700 1 -- 172.21.15.67:0/3631476569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4920009d00 con 0x7f4928101f70 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.390+0000 7f4927400700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4928102910 0x7f49280736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.390+0000 7f4926000700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928101f70 0x7f49280731a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9a0 tx=0x7f491400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.390+0000 7f491f400700 1 -- 172.21.15.67:0/3631476569 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914009790 con 0x7f4928101f70 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.390+0000 7f491f400700 1 -- 172.21.15.67:0/3631476569 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4914010430 con 0x7f4928101f70 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.390+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49281a1110 con 0x7f4928101f70 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.390+0000 7f491f400700 1 -- 172.21.15.67:0/3631476569 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914018680 con 0x7f4928101f70 2024-03-30T02:35:20.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.390+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49281a1620 con 0x7f4928101f70 2024-03-30T02:35:20.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.391+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4928108130 con 0x7f4928101f70 2024-03-30T02:35:20.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.393+0000 7f491f400700 1 -- 172.21.15.67:0/3631476569 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f49140105a0 con 0x7f4928101f70 2024-03-30T02:35:20.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.394+0000 7f491f400700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f490c0a1120 0x7f490c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:20.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.394+0000 7f491f400700 1 -- 172.21.15.67:0/3631476569 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4914014070 con 0x7f4928101f70 2024-03-30T02:35:20.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.395+0000 7f4926a00700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f490c0a1120 0x7f490c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:20.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.395+0000 7f4926a00700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f490c0a1120 0x7f490c0a35e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4928101dd0 tx=0x7f4918009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:20.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.398+0000 7f491f400700 1 -- 172.21.15.67:0/3631476569 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f49140529f0 con 0x7f4928101f70 2024-03-30T02:35:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:20 smithi067 ceph-mon[37808]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:20 smithi067 ceph-mon[37808]: pgmap v249: 129 pgs: 129 active+clean; 583 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-03-30T02:35:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:20 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] up:active 2024-03-30T02:35:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:20 smithi067 ceph-mon[37808]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-30T02:35:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:20 smithi067 ceph-mon[37808]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-30T02:35:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:20 smithi067 ceph-mon[37808]: Cluster is now healthy 2024-03-30T02:35:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:20 smithi067 ceph-mon[37808]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-03-30T02:35:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:20 smithi067 ceph-mon[37808]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-30T02:35:20.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:20 smithi067 ceph-mon[37808]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-03-30T02:35:20.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:20 smithi170 ceph-mon[78798]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:20.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:20 smithi170 ceph-mon[78798]: pgmap v249: 129 pgs: 129 active+clean; 583 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-03-30T02:35:20.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:20 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] up:active 2024-03-30T02:35:20.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:20 smithi170 ceph-mon[78798]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-30T02:35:20.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:20 smithi170 ceph-mon[78798]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-30T02:35:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:20 smithi170 ceph-mon[78798]: Cluster is now healthy 2024-03-30T02:35:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:20 smithi170 ceph-mon[78798]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-03-30T02:35:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:20 smithi170 ceph-mon[78798]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-30T02:35:20.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:20 smithi170 ceph-mon[78798]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-03-30T02:35:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:20 smithi082 ceph-mon[79176]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:20 smithi082 ceph-mon[79176]: pgmap v249: 129 pgs: 129 active+clean; 583 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-03-30T02:35:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:20 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] up:active 2024-03-30T02:35:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:20 smithi082 ceph-mon[79176]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-30T02:35:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:20 smithi082 ceph-mon[79176]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-30T02:35:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:20 smithi082 ceph-mon[79176]: Cluster is now healthy 2024-03-30T02:35:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:20 smithi082 ceph-mon[79176]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-03-30T02:35:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:20 smithi082 ceph-mon[79176]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-30T02:35:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:20 smithi082 ceph-mon[79176]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-03-30T02:35:20.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.654+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f49280611f0 con 0x7f490c0a1120 2024-03-30T02:35:20.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.678+0000 7f491f400700 1 -- 172.21.15.67:0/3631476569 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f49280611f0 con 0x7f490c0a1120 2024-03-30T02:35:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f490c0a1120 msgr2=0x7f490c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f490c0a1120 0x7f490c0a35e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4928101dd0 tx=0x7f4918009260 comp rx=0 tx=0).stop 2024-03-30T02:35:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928101f70 msgr2=0x7f49280731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928101f70 0x7f49280731a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9a0 tx=0x7f491400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 shutdown_connections 2024-03-30T02:35:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f490c0a1120 0x7f490c0a35e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4928102910 0x7f49280736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4928100d80 0x7f4928074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 --2- 172.21.15.67:0/3631476569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4928101f70 0x7f49280731a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 >> 172.21.15.67:0/3631476569 conn(0x7f49280fc2f0 msgr2=0x7f4928105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 shutdown_connections 2024-03-30T02:35:20.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.680+0000 7f492d600700 1 -- 172.21.15.67:0/3631476569 wait complete. 2024-03-30T02:35:20.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-30T02:35:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.903+0000 7f4f83a00700 1 -- 172.21.15.67:0/2749566423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 msgr2=0x7f4f7c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.903+0000 7f4f83a00700 1 --2- 172.21.15.67:0/2749566423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 0x7f4f7c1023d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4f680099c0 tx=0x7f4f68009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.903+0000 7f4f83a00700 1 -- 172.21.15.67:0/2749566423 shutdown_connections 2024-03-30T02:35:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.903+0000 7f4f83a00700 1 --2- 172.21.15.67:0/2749566423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f7c102910 0x7f4f7c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.903+0000 7f4f83a00700 1 --2- 172.21.15.67:0/2749566423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f7c100d80 0x7f4f7c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.903+0000 7f4f83a00700 1 --2- 172.21.15.67:0/2749566423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 0x7f4f7c1023d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.903+0000 7f4f83a00700 1 -- 172.21.15.67:0/2749566423 >> 172.21.15.67:0/2749566423 conn(0x7f4f7c0fc2f0 msgr2=0x7f4f7c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.903+0000 7f4f83a00700 1 -- 172.21.15.67:0/2749566423 shutdown_connections 2024-03-30T02:35:20.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.903+0000 7f4f83a00700 1 -- 172.21.15.67:0/2749566423 wait complete. 2024-03-30T02:35:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.904+0000 7f4f83a00700 1 Processor -- start 2024-03-30T02:35:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.904+0000 7f4f83a00700 1 -- start start 2024-03-30T02:35:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f83a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f7c100d80 0x7f4f7c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:20.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f83a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 0x7f4f7c0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f83a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f7c102910 0x7f4f7c0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f83a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c073e80 con 0x7f4f7c101f70 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f83a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c073ff0 con 0x7f4f7c100d80 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f83a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c1a0d70 con 0x7f4f7c102910 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f7be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 0x7f4f7c0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f7be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 0x7f4f7c0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57522/0 (socket says 172.21.15.67:57522) 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f7be00700 1 -- 172.21.15.67:0/3036387457 learned_addr learned my addr 172.21.15.67:0/3036387457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f81000700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f7c100d80 0x7f4f7c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f81a00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f7c102910 0x7f4f7c0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f7be00700 1 -- 172.21.15.67:0/3036387457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f7c102910 msgr2=0x7f4f7c0736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f7be00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f7c102910 0x7f4f7c0736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f7be00700 1 -- 172.21.15.67:0/3036387457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f7c100d80 msgr2=0x7f4f7c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f7be00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f7c100d80 0x7f4f7c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:20.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f7be00700 1 -- 172.21.15.67:0/3036387457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f680095f0 con 0x7f4f7c101f70 2024-03-30T02:35:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.905+0000 7f4f81000700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f7c100d80 0x7f4f7c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.906+0000 7f4f7be00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 0x7f4f7c0731a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f68003730 tx=0x7f4f6800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.906+0000 7f4f79600700 1 -- 172.21.15.67:0/3036387457 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6801d070 con 0x7f4f7c101f70 2024-03-30T02:35:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.906+0000 7f4f79600700 1 -- 172.21.15.67:0/3036387457 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f68005050 con 0x7f4f7c101f70 2024-03-30T02:35:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.906+0000 7f4f79600700 1 -- 172.21.15.67:0/3036387457 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68004b10 con 0x7f4f7c101f70 2024-03-30T02:35:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.906+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c1a0fa0 con 0x7f4f7c101f70 2024-03-30T02:35:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.906+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7c1a1310 con 0x7f4f7c101f70 2024-03-30T02:35:20.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.907+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7c108250 con 0x7f4f7c101f70 2024-03-30T02:35:20.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.909+0000 7f4f79600700 1 -- 172.21.15.67:0/3036387457 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f4f680051c0 con 0x7f4f7c101f70 2024-03-30T02:35:20.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.911+0000 7f4f79600700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4f640a11a0 0x7f4f640a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:20.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.911+0000 7f4f79600700 1 -- 172.21.15.67:0/3036387457 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4f680cd7b0 con 0x7f4f7c101f70 2024-03-30T02:35:20.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.911+0000 7f4f81000700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4f640a11a0 0x7f4f640a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.912+0000 7f4f81000700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4f640a11a0 0x7f4f640a3660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c101dd0 tx=0x7f4f70009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:20.915+0000 7f4f79600700 1 -- 172.21.15.67:0/3036387457 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4f68096de0 con 0x7f4f7c101f70 2024-03-30T02:35:21.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.136+0000 7f4f79600700 1 -- 172.21.15.67:0/3036387457 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4f680971b0 con 0x7f4f7c101f70 2024-03-30T02:35:21.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.181+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f7c0611f0 con 0x7f4f640a11a0 2024-03-30T02:35:21.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.185+0000 7f4f79600700 1 -- 172.21.15.67:0/3036387457 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f4f7c0611f0 con 0x7f4f640a11a0 2024-03-30T02:35:21.186 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_1/293e8158-1cf3-4427-b5aa-a50888e9e995 2024-03-30T02:35:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4f640a11a0 msgr2=0x7f4f640a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4f640a11a0 0x7f4f640a3660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c101dd0 tx=0x7f4f70009260 comp rx=0 tx=0).stop 2024-03-30T02:35:21.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 msgr2=0x7f4f7c0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 0x7f4f7c0731a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f68003730 tx=0x7f4f6800f690 comp rx=0 tx=0).stop 2024-03-30T02:35:21.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 shutdown_connections 2024-03-30T02:35:21.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4f640a11a0 0x7f4f640a3660 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f7c102910 0x7f4f7c0736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f7c100d80 0x7f4f7c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 --2- 172.21.15.67:0/3036387457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4f7c101f70 0x7f4f7c0731a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 >> 172.21.15.67:0/3036387457 conn(0x7f4f7c0fc2f0 msgr2=0x7f4f7c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:21.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 shutdown_connections 2024-03-30T02:35:21.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.188+0000 7f4f83a00700 1 -- 172.21.15.67:0/3036387457 wait complete. 2024-03-30T02:35:21.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:35:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.406+0000 7f4c23a00700 1 -- 172.21.15.67:0/2159835866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c102810 msgr2=0x7f4c1c1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.406+0000 7f4c23a00700 1 --2- 172.21.15.67:0/2159835866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c102810 0x7f4c1c1090c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400b0d0 tx=0x7f4c1400b3e0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.406+0000 7f4c23a00700 1 -- 172.21.15.67:0/2159835866 shutdown_connections 2024-03-30T02:35:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.406+0000 7f4c23a00700 1 --2- 172.21.15.67:0/2159835866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c102810 0x7f4c1c1090c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.406+0000 7f4c23a00700 1 --2- 172.21.15.67:0/2159835866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c100c80 0x7f4c1c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.406+0000 7f4c23a00700 1 --2- 172.21.15.67:0/2159835866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c101e70 0x7f4c1c1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.406+0000 7f4c23a00700 1 -- 172.21.15.67:0/2159835866 >> 172.21.15.67:0/2159835866 conn(0x7f4c1c0fc1f0 msgr2=0x7f4c1c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.406+0000 7f4c23a00700 1 -- 172.21.15.67:0/2159835866 shutdown_connections 2024-03-30T02:35:21.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.406+0000 7f4c23a00700 1 -- 172.21.15.67:0/2159835866 wait complete. 2024-03-30T02:35:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.407+0000 7f4c23a00700 1 Processor -- start 2024-03-30T02:35:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.407+0000 7f4c23a00700 1 -- start start 2024-03-30T02:35:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.407+0000 7f4c23a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c100c80 0x7f4c1c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:21.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.407+0000 7f4c23a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c101e70 0x7f4c1c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.407+0000 7f4c23a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102810 0x7f4c1c1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.407+0000 7f4c23a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1a0a00 con 0x7f4c1c100c80 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c23a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1a0b40 con 0x7f4c1c101e70 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c23a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1a0cd0 con 0x7f4c1c102810 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c100c80 0x7f4c1c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c1be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c101e70 0x7f4c1c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c100c80 0x7f4c1c198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57536/0 (socket says 172.21.15.67:57536) 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21000700 1 -- 172.21.15.67:0/287330702 learned_addr learned my addr 172.21.15.67:0/287330702 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21a00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102810 0x7f4c1c1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21000700 1 -- 172.21.15.67:0/287330702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102810 msgr2=0x7f4c1c1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21000700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102810 0x7f4c1c1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21000700 1 -- 172.21.15.67:0/287330702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c101e70 msgr2=0x7f4c1c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21000700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c101e70 0x7f4c1c199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21000700 1 -- 172.21.15.67:0/287330702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c08009520 con 0x7f4c1c100c80 2024-03-30T02:35:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c1be00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c101e70 0x7f4c1c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.408+0000 7f4c21000700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c100c80 0x7f4c1c198ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e970 tx=0x7f4c1000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.409+0000 7f4c19600700 1 -- 172.21.15.67:0/287330702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10009790 con 0x7f4c1c100c80 2024-03-30T02:35:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.409+0000 7f4c19600700 1 -- 172.21.15.67:0/287330702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c1000c7e0 con 0x7f4c1c100c80 2024-03-30T02:35:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.409+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c14009d00 con 0x7f4c1c100c80 2024-03-30T02:35:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.409+0000 7f4c19600700 1 -- 172.21.15.67:0/287330702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c10003c70 con 0x7f4c1c100c80 2024-03-30T02:35:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.409+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c1a1240 con 0x7f4c1c100c80 2024-03-30T02:35:21.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.410+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c067a10 con 0x7f4c1c100c80 2024-03-30T02:35:21.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.412+0000 7f4c19600700 1 -- 172.21.15.67:0/287330702 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f4c1000cc50 con 0x7f4c1c100c80 2024-03-30T02:35:21.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:21 smithi067 ceph-mon[37808]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:21.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:21 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] up:active 2024-03-30T02:35:21.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:21 smithi067 ceph-mon[37808]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-30T02:35:21.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:21 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:35:21.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.413+0000 7f4c19600700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4c040a11b0 0x7f4c040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:21.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.413+0000 7f4c19600700 1 -- 172.21.15.67:0/287330702 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4c10014070 con 0x7f4c1c100c80 2024-03-30T02:35:21.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.413+0000 7f4c1be00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4c040a11b0 0x7f4c040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:21.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.414+0000 7f4c1be00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4c040a11b0 0x7f4c040a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009fd0 tx=0x7f4c08009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:21.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.418+0000 7f4c19600700 1 -- 172.21.15.67:0/287330702 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f4c10096260 con 0x7f4c1c100c80 2024-03-30T02:35:21.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:21 smithi170 ceph-mon[78798]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:21.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:21 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] up:active 2024-03-30T02:35:21.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:21 smithi170 ceph-mon[78798]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-30T02:35:21.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:21 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:35:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:21 smithi082 ceph-mon[79176]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:21 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] up:active 2024-03-30T02:35:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:21 smithi082 ceph-mon[79176]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-30T02:35:21.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:21 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:35:21.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.681+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c1c1a1490 con 0x7f4c1c100c80 2024-03-30T02:35:21.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.682+0000 7f4c19600700 1 -- 172.21.15.67:0/287330702 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f4c10003dd0 con 0x7f4c1c100c80 2024-03-30T02:35:21.683 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:21.683 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:35:20.119112+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:35:21.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4c040a11b0 msgr2=0x7f4c040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4c040a11b0 0x7f4c040a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009fd0 tx=0x7f4c08009260 comp rx=0 tx=0).stop 2024-03-30T02:35:21.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c100c80 msgr2=0x7f4c1c198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c100c80 0x7f4c1c198ed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e970 tx=0x7f4c1000ec80 comp rx=0 tx=0).stop 2024-03-30T02:35:21.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 shutdown_connections 2024-03-30T02:35:21.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f4c040a11b0 0x7f4c040a3670 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c100c80 0x7f4c1c198ed0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c102810 0x7f4c1c1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 --2- 172.21.15.67:0/287330702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c101e70 0x7f4c1c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 >> 172.21.15.67:0/287330702 conn(0x7f4c1c0fc1f0 msgr2=0x7f4c1c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:21.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 shutdown_connections 2024-03-30T02:35:21.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.685+0000 7f4c23a00700 1 -- 172.21.15.67:0/287330702 wait complete. 2024-03-30T02:35:21.687 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2024-03-30T02:35:21.706 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:35:21.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.903+0000 7ff1b3800700 1 -- 172.21.15.67:0/260788494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac101e70 msgr2=0x7ff1ac1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.903+0000 7ff1b3800700 1 --2- 172.21.15.67:0/260788494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac101e70 0x7ff1ac1022d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff1980099c0 tx=0x7ff198009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.903+0000 7ff1b3800700 1 -- 172.21.15.67:0/260788494 shutdown_connections 2024-03-30T02:35:21.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.903+0000 7ff1b3800700 1 --2- 172.21.15.67:0/260788494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1ac102810 0x7ff1ac1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.903+0000 7ff1b3800700 1 --2- 172.21.15.67:0/260788494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ac100c80 0x7ff1ac1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.903+0000 7ff1b3800700 1 --2- 172.21.15.67:0/260788494 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac101e70 0x7ff1ac1022d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.903+0000 7ff1b3800700 1 -- 172.21.15.67:0/260788494 >> 172.21.15.67:0/260788494 conn(0x7ff1ac0fc1f0 msgr2=0x7ff1ac0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:21.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.903+0000 7ff1b3800700 1 -- 172.21.15.67:0/260788494 shutdown_connections 2024-03-30T02:35:21.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.903+0000 7ff1b3800700 1 -- 172.21.15.67:0/260788494 wait complete. 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.904+0000 7ff1b3800700 1 Processor -- start 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.904+0000 7ff1b3800700 1 -- start start 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b3800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac100c80 0x7ff1ac198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b3800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ac101e70 0x7ff1ac199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b3800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1ac102810 0x7ff1ac1a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b3800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1a0900 con 0x7ff1ac100c80 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b3800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1a0a70 con 0x7ff1ac101e70 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b3800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1a0c30 con 0x7ff1ac102810 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b0e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac100c80 0x7ff1ac198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1abe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ac101e70 0x7ff1ac199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:21.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b0e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac100c80 0x7ff1ac198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57550/0 (socket says 172.21.15.67:57550) 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b0e00700 1 -- 172.21.15.67:0/977433592 learned_addr learned my addr 172.21.15.67:0/977433592 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b1800700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1ac102810 0x7ff1ac1a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b0e00700 1 -- 172.21.15.67:0/977433592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1ac102810 msgr2=0x7ff1ac1a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b0e00700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1ac102810 0x7ff1ac1a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b0e00700 1 -- 172.21.15.67:0/977433592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ac101e70 msgr2=0x7ff1ac199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.905+0000 7ff1b0e00700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ac101e70 0x7ff1ac199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.906+0000 7ff1b0e00700 1 -- 172.21.15.67:0/977433592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980095f0 con 0x7ff1ac100c80 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.906+0000 7ff1abe00700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ac101e70 0x7ff1ac199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.906+0000 7ff1b0e00700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac100c80 0x7ff1ac198e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000e970 tx=0x7ff1a000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.906+0000 7ff1a9600700 1 -- 172.21.15.67:0/977433592 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0009790 con 0x7ff1ac100c80 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.906+0000 7ff1a9600700 1 -- 172.21.15.67:0/977433592 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1a000c7e0 con 0x7ff1ac100c80 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.906+0000 7ff1a9600700 1 -- 172.21.15.67:0/977433592 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0003c70 con 0x7ff1ac100c80 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.906+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ac1a0f10 con 0x7ff1ac100c80 2024-03-30T02:35:21.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.906+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ac1a12e0 con 0x7ff1ac100c80 2024-03-30T02:35:21.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.907+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ac067a10 con 0x7ff1ac100c80 2024-03-30T02:35:21.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.909+0000 7ff1a9600700 1 -- 172.21.15.67:0/977433592 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff1a000cc50 con 0x7ff1ac100c80 2024-03-30T02:35:21.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.910+0000 7ff1a9600700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1940a11b0 0x7ff1940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:21.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.911+0000 7ff1a9600700 1 -- 172.21.15.67:0/977433592 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff1a0014070 con 0x7ff1ac100c80 2024-03-30T02:35:21.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.911+0000 7ff1abe00700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1940a11b0 0x7ff1940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:21.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.911+0000 7ff1abe00700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1940a11b0 0x7ff1940a3670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff198006010 tx=0x7ff19800b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:21.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:21.915+0000 7ff1a9600700 1 -- 172.21.15.67:0/977433592 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff1a0096260 con 0x7ff1ac100c80 2024-03-30T02:35:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.169+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1ac10c4e0 con 0x7ff1ac100c80 2024-03-30T02:35:22.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.170+0000 7ff1a9600700 1 -- 172.21.15.67:0/977433592 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+23925 (secure 0 0 0) 0x7ff1a0003dd0 con 0x7ff1ac100c80 2024-03-30T02:35:22.172 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:22.172 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":83,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:34:51.268204+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:35:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1940a11b0 msgr2=0x7ff1940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1940a11b0 0x7ff1940a3670 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff198006010 tx=0x7ff19800b540 comp rx=0 tx=0).stop 2024-03-30T02:35:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac100c80 msgr2=0x7ff1ac198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac100c80 0x7ff1ac198e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000e970 tx=0x7ff1a000ec80 comp rx=0 tx=0).stop 2024-03-30T02:35:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 shutdown_connections 2024-03-30T02:35:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1940a11b0 0x7ff1940a3670 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1ac100c80 0x7ff1ac198e10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1ac102810 0x7ff1ac1a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 --2- 172.21.15.67:0/977433592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1ac101e70 0x7ff1ac199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 >> 172.21.15.67:0/977433592 conn(0x7ff1ac0fc1f0 msgr2=0x7ff1ac072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 shutdown_connections 2024-03-30T02:35:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.173+0000 7ff1b3800700 1 -- 172.21.15.67:0/977433592 wait complete. 2024-03-30T02:35:22.194 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T02:35:22.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T02:35:22.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.392+0000 7f1c88400700 1 -- 172.21.15.67:0/205361418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80102810 msgr2=0x7f1c801090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.392+0000 7f1c88400700 1 --2- 172.21.15.67:0/205361418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80102810 0x7f1c801090c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800b0d0 tx=0x7f1c7800b3e0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.392+0000 7f1c88400700 1 -- 172.21.15.67:0/205361418 shutdown_connections 2024-03-30T02:35:22.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.392+0000 7f1c88400700 1 --2- 172.21.15.67:0/205361418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80100c80 0x7f1c801010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.392+0000 7f1c88400700 1 --2- 172.21.15.67:0/205361418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80102810 0x7f1c801090c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.392+0000 7f1c88400700 1 --2- 172.21.15.67:0/205361418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c80101e70 0x7f1c801022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.392+0000 7f1c88400700 1 -- 172.21.15.67:0/205361418 >> 172.21.15.67:0/205361418 conn(0x7f1c800fc1f0 msgr2=0x7f1c800fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:22.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.392+0000 7f1c88400700 1 -- 172.21.15.67:0/205361418 shutdown_connections 2024-03-30T02:35:22.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.392+0000 7f1c88400700 1 -- 172.21.15.67:0/205361418 wait complete. 2024-03-30T02:35:22.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.393+0000 7f1c88400700 1 Processor -- start 2024-03-30T02:35:22.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.393+0000 7f1c88400700 1 -- start start 2024-03-30T02:35:22.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.393+0000 7f1c88400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c80100c80 0x7f1c80198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:22.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c88400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80101e70 0x7f1c80199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:22.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c88400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80102810 0x7f1c801a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:22.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c88400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c801a0a60 con 0x7f1c80102810 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c88400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c801a0ba0 con 0x7f1c80101e70 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c88400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c801a0d30 con 0x7f1c80100c80 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c86400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80102810 0x7f1c801a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c86400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80102810 0x7f1c801a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57570/0 (socket says 172.21.15.67:57570) 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c85000700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80101e70 0x7f1c80199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c86400700 1 -- 172.21.15.67:0/3513910927 learned_addr learned my addr 172.21.15.67:0/3513910927 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c85a00700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c80100c80 0x7f1c80198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c86400700 1 -- 172.21.15.67:0/3513910927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c80100c80 msgr2=0x7f1c80198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c86400700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c80100c80 0x7f1c80198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c86400700 1 -- 172.21.15.67:0/3513910927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80101e70 msgr2=0x7f1c80199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c86400700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80101e70 0x7f1c80199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c86400700 1 -- 172.21.15.67:0/3513910927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c78009d00 con 0x7f1c80102810 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.394+0000 7f1c85000700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80101e70 0x7f1c80199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.395+0000 7f1c86400700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80102810 0x7f1c801a0490 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1c78007b60 tx=0x7f1c78008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.395+0000 7f1c7e000700 1 -- 172.21.15.67:0/3513910927 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7800e040 con 0x7f1c80102810 2024-03-30T02:35:22.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.395+0000 7f1c7e000700 1 -- 172.21.15.67:0/3513910927 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c7801d930 con 0x7f1c80102810 2024-03-30T02:35:22.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.395+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c801a0fb0 con 0x7f1c80102810 2024-03-30T02:35:22.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.395+0000 7f1c7e000700 1 -- 172.21.15.67:0/3513910927 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7801b9f0 con 0x7f1c80102810 2024-03-30T02:35:22.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.395+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c801a1380 con 0x7f1c80102810 2024-03-30T02:35:22.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.396+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c80067a10 con 0x7f1c80102810 2024-03-30T02:35:22.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.398+0000 7f1c7e000700 1 -- 172.21.15.67:0/3513910927 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1c7801daa0 con 0x7f1c80102810 2024-03-30T02:35:22.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.399+0000 7f1c7e000700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1c680a11b0 0x7f1c680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:22.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.399+0000 7f1c7e000700 1 -- 172.21.15.67:0/3513910927 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1c780ce830 con 0x7f1c80102810 2024-03-30T02:35:22.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.400+0000 7f1c85a00700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1c680a11b0 0x7f1c680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:22.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.400+0000 7f1c85a00700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1c680a11b0 0x7f1c680a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1c80101cd0 tx=0x7f1c74009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:22.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.401+0000 7f1c7e000700 1 -- 172.21.15.67:0/3513910927 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1c78097ca0 con 0x7f1c80102810 2024-03-30T02:35:22.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:22 smithi067 ceph-mon[37808]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:22.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:22 smithi067 ceph-mon[37808]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:22.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:22 smithi067 ceph-mon[37808]: pgmap v250: 129 pgs: 129 active+clean; 583 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.5 KiB/s wr, 8 op/s 2024-03-30T02:35:22.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:22 smithi067 ceph-mon[37808]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-30T02:35:22.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:22 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/287330702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:35:22.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:22 smithi170 ceph-mon[78798]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:22.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:22 smithi170 ceph-mon[78798]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:22.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:22 smithi170 ceph-mon[78798]: pgmap v250: 129 pgs: 129 active+clean; 583 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.5 KiB/s wr, 8 op/s 2024-03-30T02:35:22.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:22 smithi170 ceph-mon[78798]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-30T02:35:22.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:22 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/287330702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:35:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:22 smithi082 ceph-mon[79176]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:22 smithi082 ceph-mon[79176]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:22 smithi082 ceph-mon[79176]: pgmap v250: 129 pgs: 129 active+clean; 583 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.5 KiB/s wr, 8 op/s 2024-03-30T02:35:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:22 smithi082 ceph-mon[79176]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-03-30T02:35:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:22 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/287330702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:35:22.500 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:35:22 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: 2024-03-30T02:35:22.113+0000 7f86ad81d700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-30T02:35:22.500 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:35:22 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: 2024-03-30T02:35:22.113+0000 7f86ad81d700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-30T02:35:22.500 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:35:22 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: 2024-03-30T02:35:22.113+0000 7f86ad81d700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-30T02:35:22.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.616+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c80074700 con 0x7f1c680a11b0 2024-03-30T02:35:22.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.632+0000 7f1c7e000700 1 -- 172.21.15.67:0/3513910927 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422438 (secure 0 0 0) 0x7f1c80074700 con 0x7f1c680a11b0 2024-03-30T02:35:22.639 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1c680a11b0 msgr2=0x7f1c680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1c680a11b0 0x7f1c680a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1c80101cd0 tx=0x7f1c74009260 comp rx=0 tx=0).stop 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80102810 msgr2=0x7f1c801a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80102810 0x7f1c801a0490 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1c78007b60 tx=0x7f1c78008e40 comp rx=0 tx=0).stop 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 shutdown_connections 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1c680a11b0 0x7f1c680a3670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c80102810 0x7f1c801a0490 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c80100c80 0x7f1c80198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 --2- 172.21.15.67:0/3513910927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c80101e70 0x7f1c80199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 >> 172.21.15.67:0/3513910927 conn(0x7f1c800fc1f0 msgr2=0x7f1c80072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:22.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 shutdown_connections 2024-03-30T02:35:22.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.637+0000 7f1c88400700 1 -- 172.21.15.67:0/3513910927 wait complete. 2024-03-30T02:35:22.644 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-30T02:35:22.658 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-03-30T02:35:21.005751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596724,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":138,"num_read_kb":121,"num_write":328,"num_write_kb":2987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":263,"ondisk_log_size":263,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359696,"kb_used_data":10908,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124582128,"statfs":{"total":1151940427776,"available":1151572099072,"internally_reserved":0,"allocated":11169792,"data_stored":7153981,"data_compressed":248136,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19130,"internal_metadata":357020998},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1762,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":98,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003611"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076787+0000","last_change":"2024-03-30T02:34:46.755257+0000","last_active":"2024-03-30T02:34:51.076787+0000","last_peered":"2024-03-30T02:34:51.076787+0000","last_clean":"2024-03-30T02:34:51.076787+0000","last_became_active":"2024-03-30T02:34:46.754922+0000","last_became_peered":"2024-03-30T02:34:46.754922+0000","last_unstale":"2024-03-30T02:34:51.076787+0000","last_undegraded":"2024-03-30T02:34:51.076787+0000","last_fullsized":"2024-03-30T02:34:51.076787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:09.980124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.755256+0000","last_change":"2024-03-30T02:34:46.742846+0000","last_active":"2024-03-30T02:34:50.755256+0000","last_peered":"2024-03-30T02:34:50.755256+0000","last_clean":"2024-03-30T02:34:50.755256+0000","last_became_active":"2024-03-30T02:34:46.738096+0000","last_became_peered":"2024-03-30T02:34:46.738096+0000","last_unstale":"2024-03-30T02:34:50.755256+0000","last_undegraded":"2024-03-30T02:34:50.755256+0000","last_fullsized":"2024-03-30T02:34:50.755256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:43:16.982251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.755723+0000","last_change":"2024-03-30T02:34:46.739607+0000","last_active":"2024-03-30T02:34:50.755723+0000","last_peered":"2024-03-30T02:34:50.755723+0000","last_clean":"2024-03-30T02:34:50.755723+0000","last_became_active":"2024-03-30T02:34:46.739230+0000","last_became_peered":"2024-03-30T02:34:46.739230+0000","last_unstale":"2024-03-30T02:34:50.755723+0000","last_undegraded":"2024-03-30T02:34:50.755723+0000","last_fullsized":"2024-03-30T02:34:50.755723+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:17.818874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107220+0000","last_change":"2024-03-30T02:34:46.742118+0000","last_active":"2024-03-30T02:34:51.107220+0000","last_peered":"2024-03-30T02:34:51.107220+0000","last_clean":"2024-03-30T02:34:51.107220+0000","last_became_active":"2024-03-30T02:34:46.740845+0000","last_became_peered":"2024-03-30T02:34:46.740845+0000","last_unstale":"2024-03-30T02:34:51.107220+0000","last_undegraded":"2024-03-30T02:34:51.107220+0000","last_fullsized":"2024-03-30T02:34:51.107220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:33:28.595456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.744963+0000","last_change":"2024-03-30T02:34:46.739557+0000","last_active":"2024-03-30T02:34:50.744963+0000","last_peered":"2024-03-30T02:34:50.744963+0000","last_clean":"2024-03-30T02:34:50.744963+0000","last_became_active":"2024-03-30T02:34:46.739147+0000","last_became_peered":"2024-03-30T02:34:46.739147+0000","last_unstale":"2024-03-30T02:34:50.744963+0000","last_undegraded":"2024-03-30T02:34:50.744963+0000","last_fullsized":"2024-03-30T02:34:50.744963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:12:52.307386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.796263+0000","last_change":"2024-03-30T02:34:46.735157+0000","last_active":"2024-03-30T02:34:50.796263+0000","last_peered":"2024-03-30T02:34:50.796263+0000","last_clean":"2024-03-30T02:34:50.796263+0000","last_became_active":"2024-03-30T02:34:46.734725+0000","last_became_peered":"2024-03-30T02:34:46.734725+0000","last_unstale":"2024-03-30T02:34:50.796263+0000","last_undegraded":"2024-03-30T02:34:50.796263+0000","last_fullsized":"2024-03-30T02:34:50.796263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:00.823203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117652+0000","last_change":"2024-03-30T02:34:46.735466+0000","last_active":"2024-03-30T02:34:51.117652+0000","last_peered":"2024-03-30T02:34:51.117652+0000","last_clean":"2024-03-30T02:34:51.117652+0000","last_became_active":"2024-03-30T02:34:46.734411+0000","last_became_peered":"2024-03-30T02:34:46.734411+0000","last_unstale":"2024-03-30T02:34:51.117652+0000","last_undegraded":"2024-03-30T02:34:51.117652+0000","last_fullsized":"2024-03-30T02:34:51.117652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:57.587768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.749906+0000","last_change":"2024-03-30T02:34:46.747001+0000","last_active":"2024-03-30T02:34:50.749906+0000","last_peered":"2024-03-30T02:34:50.749906+0000","last_clean":"2024-03-30T02:34:50.749906+0000","last_became_active":"2024-03-30T02:34:46.740241+0000","last_became_peered":"2024-03-30T02:34:46.740241+0000","last_unstale":"2024-03-30T02:34:50.749906+0000","last_undegraded":"2024-03-30T02:34:50.749906+0000","last_fullsized":"2024-03-30T02:34:50.749906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:05.902190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743753+0000","last_change":"2024-03-30T02:34:46.745549+0000","last_active":"2024-03-30T02:34:50.743753+0000","last_peered":"2024-03-30T02:34:50.743753+0000","last_clean":"2024-03-30T02:34:50.743753+0000","last_became_active":"2024-03-30T02:34:46.745200+0000","last_became_peered":"2024-03-30T02:34:46.745200+0000","last_unstale":"2024-03-30T02:34:50.743753+0000","last_undegraded":"2024-03-30T02:34:50.743753+0000","last_fullsized":"2024-03-30T02:34:50.743753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:37.931440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.796486+0000","last_change":"2024-03-30T02:34:46.735859+0000","last_active":"2024-03-30T02:34:50.796486+0000","last_peered":"2024-03-30T02:34:50.796486+0000","last_clean":"2024-03-30T02:34:50.796486+0000","last_became_active":"2024-03-30T02:34:46.735481+0000","last_became_peered":"2024-03-30T02:34:46.735481+0000","last_unstale":"2024-03-30T02:34:50.796486+0000","last_undegraded":"2024-03-30T02:34:50.796486+0000","last_fullsized":"2024-03-30T02:34:50.796486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:51.037887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117750+0000","last_change":"2024-03-30T02:34:46.741412+0000","last_active":"2024-03-30T02:34:51.117750+0000","last_peered":"2024-03-30T02:34:51.117750+0000","last_clean":"2024-03-30T02:34:51.117750+0000","last_became_active":"2024-03-30T02:34:46.741106+0000","last_became_peered":"2024-03-30T02:34:46.741106+0000","last_unstale":"2024-03-30T02:34:51.117750+0000","last_undegraded":"2024-03-30T02:34:51.117750+0000","last_fullsized":"2024-03-30T02:34:51.117750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:06:08.553028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107207+0000","last_change":"2024-03-30T02:34:46.746205+0000","last_active":"2024-03-30T02:34:51.107207+0000","last_peered":"2024-03-30T02:34:51.107207+0000","last_clean":"2024-03-30T02:34:51.107207+0000","last_became_active":"2024-03-30T02:34:46.745803+0000","last_became_peered":"2024-03-30T02:34:46.745803+0000","last_unstale":"2024-03-30T02:34:51.107207+0000","last_undegraded":"2024-03-30T02:34:51.107207+0000","last_fullsized":"2024-03-30T02:34:51.107207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:05.127095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747388+0000","last_change":"2024-03-30T02:34:46.722978+0000","last_active":"2024-03-30T02:34:50.747388+0000","last_peered":"2024-03-30T02:34:50.747388+0000","last_clean":"2024-03-30T02:34:50.747388+0000","last_became_active":"2024-03-30T02:34:46.722652+0000","last_became_peered":"2024-03-30T02:34:46.722652+0000","last_unstale":"2024-03-30T02:34:50.747388+0000","last_undegraded":"2024-03-30T02:34:50.747388+0000","last_fullsized":"2024-03-30T02:34:50.747388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:28:12.078216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.752810+0000","last_change":"2024-03-30T02:34:46.738895+0000","last_active":"2024-03-30T02:34:50.752810+0000","last_peered":"2024-03-30T02:34:50.752810+0000","last_clean":"2024-03-30T02:34:50.752810+0000","last_became_active":"2024-03-30T02:34:46.738510+0000","last_became_peered":"2024-03-30T02:34:46.738510+0000","last_unstale":"2024-03-30T02:34:50.752810+0000","last_undegraded":"2024-03-30T02:34:50.752810+0000","last_fullsized":"2024-03-30T02:34:50.752810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:09.859259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.751025+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:34:50.751025+0000","last_peered":"2024-03-30T02:34:50.751025+0000","last_clean":"2024-03-30T02:34:50.751025+0000","last_became_active":"2024-03-30T02:34:46.750301+0000","last_became_peered":"2024-03-30T02:34:46.750301+0000","last_unstale":"2024-03-30T02:34:50.751025+0000","last_undegraded":"2024-03-30T02:34:50.751025+0000","last_fullsized":"2024-03-30T02:34:50.751025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:06.475881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743238+0000","last_change":"2024-03-30T02:34:46.748966+0000","last_active":"2024-03-30T02:34:50.743238+0000","last_peered":"2024-03-30T02:34:50.743238+0000","last_clean":"2024-03-30T02:34:50.743238+0000","last_became_active":"2024-03-30T02:34:46.748650+0000","last_became_peered":"2024-03-30T02:34:46.748650+0000","last_unstale":"2024-03-30T02:34:50.743238+0000","last_undegraded":"2024-03-30T02:34:50.743238+0000","last_fullsized":"2024-03-30T02:34:50.743238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:36.558864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076777+0000","last_change":"2024-03-30T02:34:46.746127+0000","last_active":"2024-03-30T02:34:51.076777+0000","last_peered":"2024-03-30T02:34:51.076777+0000","last_clean":"2024-03-30T02:34:51.076777+0000","last_became_active":"2024-03-30T02:34:46.745806+0000","last_became_peered":"2024-03-30T02:34:46.745806+0000","last_unstale":"2024-03-30T02:34:51.076777+0000","last_undegraded":"2024-03-30T02:34:51.076777+0000","last_fullsized":"2024-03-30T02:34:51.076777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:54.097996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.740772+0000","last_change":"2024-03-30T02:34:46.738391+0000","last_active":"2024-03-30T02:34:50.740772+0000","last_peered":"2024-03-30T02:34:50.740772+0000","last_clean":"2024-03-30T02:34:50.740772+0000","last_became_active":"2024-03-30T02:34:46.737110+0000","last_became_peered":"2024-03-30T02:34:46.737110+0000","last_unstale":"2024-03-30T02:34:50.740772+0000","last_undegraded":"2024-03-30T02:34:50.740772+0000","last_fullsized":"2024-03-30T02:34:50.740772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:20.326107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.752034+0000","last_change":"2024-03-30T02:34:46.744522+0000","last_active":"2024-03-30T02:34:50.752034+0000","last_peered":"2024-03-30T02:34:50.752034+0000","last_clean":"2024-03-30T02:34:50.752034+0000","last_became_active":"2024-03-30T02:34:46.744088+0000","last_became_peered":"2024-03-30T02:34:46.744088+0000","last_unstale":"2024-03-30T02:34:50.752034+0000","last_undegraded":"2024-03-30T02:34:50.752034+0000","last_fullsized":"2024-03-30T02:34:50.752034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:35.698979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.272847+0000","last_change":"2024-03-30T02:34:50.770855+0000","last_active":"2024-03-30T02:34:51.272847+0000","last_peered":"2024-03-30T02:34:51.272847+0000","last_clean":"2024-03-30T02:34:51.272847+0000","last_became_active":"2024-03-30T02:34:50.770420+0000","last_became_peered":"2024-03-30T02:34:50.770420+0000","last_unstale":"2024-03-30T02:34:51.272847+0000","last_undegraded":"2024-03-30T02:34:51.272847+0000","last_fullsized":"2024-03-30T02:34:51.272847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:00:08.847059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:20.128766+0000","last_change":"2024-03-30T02:34:46.207252+0000","last_active":"2024-03-30T02:35:20.128766+0000","last_peered":"2024-03-30T02:35:20.128766+0000","last_clean":"2024-03-30T02:35:20.128766+0000","last_became_active":"2024-03-30T02:34:46.206916+0000","last_became_peered":"2024-03-30T02:34:46.206916+0000","last_unstale":"2024-03-30T02:35:20.128766+0000","last_undegraded":"2024-03-30T02:35:20.128766+0000","last_fullsized":"2024-03-30T02:35:20.128766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:03:50.192245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.754252+0000","last_change":"2024-03-30T02:34:46.745778+0000","last_active":"2024-03-30T02:34:50.754252+0000","last_peered":"2024-03-30T02:34:50.754252+0000","last_clean":"2024-03-30T02:34:50.754252+0000","last_became_active":"2024-03-30T02:34:46.745471+0000","last_became_peered":"2024-03-30T02:34:46.745471+0000","last_unstale":"2024-03-30T02:34:50.754252+0000","last_undegraded":"2024-03-30T02:34:50.754252+0000","last_fullsized":"2024-03-30T02:34:50.754252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:57.314163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107117+0000","last_change":"2024-03-30T02:34:50.769577+0000","last_active":"2024-03-30T02:34:51.107117+0000","last_peered":"2024-03-30T02:34:51.107117+0000","last_clean":"2024-03-30T02:34:51.107117+0000","last_became_active":"2024-03-30T02:34:50.769100+0000","last_became_peered":"2024-03-30T02:34:50.769100+0000","last_unstale":"2024-03-30T02:34:51.107117+0000","last_undegraded":"2024-03-30T02:34:51.107117+0000","last_fullsized":"2024-03-30T02:34:51.107117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:08:43.495671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.274453+0000","last_change":"2024-03-30T02:34:46.218032+0000","last_active":"2024-03-30T02:34:51.274453+0000","last_peered":"2024-03-30T02:34:51.274453+0000","last_clean":"2024-03-30T02:34:51.274453+0000","last_became_active":"2024-03-30T02:34:46.217733+0000","last_became_peered":"2024-03-30T02:34:46.217733+0000","last_unstale":"2024-03-30T02:34:51.274453+0000","last_undegraded":"2024-03-30T02:34:51.274453+0000","last_fullsized":"2024-03-30T02:34:51.274453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:58.413590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.759280+0000","last_change":"2024-03-30T02:34:46.750588+0000","last_active":"2024-03-30T02:34:50.759280+0000","last_peered":"2024-03-30T02:34:50.759280+0000","last_clean":"2024-03-30T02:34:50.759280+0000","last_became_active":"2024-03-30T02:34:46.750016+0000","last_became_peered":"2024-03-30T02:34:46.750016+0000","last_unstale":"2024-03-30T02:34:50.759280+0000","last_undegraded":"2024-03-30T02:34:50.759280+0000","last_fullsized":"2024-03-30T02:34:50.759280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:19.144544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107167+0000","last_change":"2024-03-30T02:34:50.760473+0000","last_active":"2024-03-30T02:34:51.107167+0000","last_peered":"2024-03-30T02:34:51.107167+0000","last_clean":"2024-03-30T02:34:51.107167+0000","last_became_active":"2024-03-30T02:34:50.760068+0000","last_became_peered":"2024-03-30T02:34:50.760068+0000","last_unstale":"2024-03-30T02:34:51.107167+0000","last_undegraded":"2024-03-30T02:34:51.107167+0000","last_fullsized":"2024-03-30T02:34:51.107167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:49:12.902937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.113595+0000","last_change":"2024-03-30T02:34:46.213741+0000","last_active":"2024-03-30T02:35:18.113595+0000","last_peered":"2024-03-30T02:35:18.113595+0000","last_clean":"2024-03-30T02:35:18.113595+0000","last_became_active":"2024-03-30T02:34:46.213395+0000","last_became_peered":"2024-03-30T02:34:46.213395+0000","last_unstale":"2024-03-30T02:35:18.113595+0000","last_undegraded":"2024-03-30T02:35:18.113595+0000","last_fullsized":"2024-03-30T02:35:18.113595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:13.192415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743330+0000","last_change":"2024-03-30T02:34:46.738401+0000","last_active":"2024-03-30T02:34:50.743330+0000","last_peered":"2024-03-30T02:34:50.743330+0000","last_clean":"2024-03-30T02:34:50.743330+0000","last_became_active":"2024-03-30T02:34:46.737804+0000","last_became_peered":"2024-03-30T02:34:46.737804+0000","last_unstale":"2024-03-30T02:34:50.743330+0000","last_undegraded":"2024-03-30T02:34:50.743330+0000","last_fullsized":"2024-03-30T02:34:50.743330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:56:08.194439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107154+0000","last_change":"2024-03-30T02:34:50.766222+0000","last_active":"2024-03-30T02:34:51.107154+0000","last_peered":"2024-03-30T02:34:51.107154+0000","last_clean":"2024-03-30T02:34:51.107154+0000","last_became_active":"2024-03-30T02:34:50.765836+0000","last_became_peered":"2024-03-30T02:34:50.765836+0000","last_unstale":"2024-03-30T02:34:51.107154+0000","last_undegraded":"2024-03-30T02:34:51.107154+0000","last_fullsized":"2024-03-30T02:34:51.107154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:32.529535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.744949+0000","last_change":"2024-03-30T02:34:46.212586+0000","last_active":"2024-03-30T02:34:50.744949+0000","last_peered":"2024-03-30T02:34:50.744949+0000","last_clean":"2024-03-30T02:34:50.744949+0000","last_became_active":"2024-03-30T02:34:46.212110+0000","last_became_peered":"2024-03-30T02:34:46.212110+0000","last_unstale":"2024-03-30T02:34:50.744949+0000","last_undegraded":"2024-03-30T02:34:50.744949+0000","last_fullsized":"2024-03-30T02:34:50.744949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:38.630953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748588+0000","last_change":"2024-03-30T02:34:46.734923+0000","last_active":"2024-03-30T02:34:50.748588+0000","last_peered":"2024-03-30T02:34:50.748588+0000","last_clean":"2024-03-30T02:34:50.748588+0000","last_became_active":"2024-03-30T02:34:46.734572+0000","last_became_peered":"2024-03-30T02:34:46.734572+0000","last_unstale":"2024-03-30T02:34:50.748588+0000","last_undegraded":"2024-03-30T02:34:50.748588+0000","last_fullsized":"2024-03-30T02:34:50.748588+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:28.647788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.273238+0000","last_change":"2024-03-30T02:34:50.757803+0000","last_active":"2024-03-30T02:34:51.273238+0000","last_peered":"2024-03-30T02:34:51.273238+0000","last_clean":"2024-03-30T02:34:51.273238+0000","last_became_active":"2024-03-30T02:34:50.757198+0000","last_became_peered":"2024-03-30T02:34:50.757198+0000","last_unstale":"2024-03-30T02:34:51.273238+0000","last_undegraded":"2024-03-30T02:34:51.273238+0000","last_fullsized":"2024-03-30T02:34:51.273238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:54:49.399741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.102299+0000","last_change":"2024-03-30T02:34:46.214549+0000","last_active":"2024-03-30T02:35:16.102299+0000","last_peered":"2024-03-30T02:35:16.102299+0000","last_clean":"2024-03-30T02:35:16.102299+0000","last_became_active":"2024-03-30T02:34:46.214198+0000","last_became_peered":"2024-03-30T02:34:46.214198+0000","last_unstale":"2024-03-30T02:35:16.102299+0000","last_undegraded":"2024-03-30T02:35:16.102299+0000","last_fullsized":"2024-03-30T02:35:16.102299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:45:25.618982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.746267+0000","last_change":"2024-03-30T02:34:46.737771+0000","last_active":"2024-03-30T02:34:50.746267+0000","last_peered":"2024-03-30T02:34:50.746267+0000","last_clean":"2024-03-30T02:34:50.746267+0000","last_became_active":"2024-03-30T02:34:46.737438+0000","last_became_peered":"2024-03-30T02:34:46.737438+0000","last_unstale":"2024-03-30T02:34:50.746267+0000","last_undegraded":"2024-03-30T02:34:50.746267+0000","last_fullsized":"2024-03-30T02:34:50.746267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:08.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.272971+0000","last_change":"2024-03-30T02:34:50.767304+0000","last_active":"2024-03-30T02:34:51.272971+0000","last_peered":"2024-03-30T02:34:51.272971+0000","last_clean":"2024-03-30T02:34:51.272971+0000","last_became_active":"2024-03-30T02:34:50.766909+0000","last_became_peered":"2024-03-30T02:34:50.766909+0000","last_unstale":"2024-03-30T02:34:51.272971+0000","last_undegraded":"2024-03-30T02:34:51.272971+0000","last_fullsized":"2024-03-30T02:34:51.272971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:24.021991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.114383+0000","last_change":"2024-03-30T02:34:46.214937+0000","last_active":"2024-03-30T02:35:18.114383+0000","last_peered":"2024-03-30T02:35:18.114383+0000","last_clean":"2024-03-30T02:35:18.114383+0000","last_became_active":"2024-03-30T02:34:46.214503+0000","last_became_peered":"2024-03-30T02:34:46.214503+0000","last_unstale":"2024-03-30T02:35:18.114383+0000","last_undegraded":"2024-03-30T02:35:18.114383+0000","last_fullsized":"2024-03-30T02:35:18.114383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:20:11.624919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747888+0000","last_change":"2024-03-30T02:34:46.737374+0000","last_active":"2024-03-30T02:34:50.747888+0000","last_peered":"2024-03-30T02:34:50.747888+0000","last_clean":"2024-03-30T02:34:50.747888+0000","last_became_active":"2024-03-30T02:34:46.735307+0000","last_became_peered":"2024-03-30T02:34:46.735307+0000","last_unstale":"2024-03-30T02:34:50.747888+0000","last_undegraded":"2024-03-30T02:34:50.747888+0000","last_fullsized":"2024-03-30T02:34:50.747888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:30.482728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076765+0000","last_change":"2024-03-30T02:34:50.768016+0000","last_active":"2024-03-30T02:34:51.076765+0000","last_peered":"2024-03-30T02:34:51.076765+0000","last_clean":"2024-03-30T02:34:51.076765+0000","last_became_active":"2024-03-30T02:34:50.767564+0000","last_became_peered":"2024-03-30T02:34:50.767564+0000","last_unstale":"2024-03-30T02:34:51.076765+0000","last_undegraded":"2024-03-30T02:34:51.076765+0000","last_fullsized":"2024-03-30T02:34:51.076765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:16.888754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:17.111130+0000","last_change":"2024-03-30T02:34:46.212982+0000","last_active":"2024-03-30T02:35:17.111130+0000","last_peered":"2024-03-30T02:35:17.111130+0000","last_clean":"2024-03-30T02:35:17.111130+0000","last_became_active":"2024-03-30T02:34:46.212550+0000","last_became_peered":"2024-03-30T02:34:46.212550+0000","last_unstale":"2024-03-30T02:35:17.111130+0000","last_undegraded":"2024-03-30T02:35:17.111130+0000","last_fullsized":"2024-03-30T02:35:17.111130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:44:50.156615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.749854+0000","last_change":"2024-03-30T02:34:46.736394+0000","last_active":"2024-03-30T02:34:50.749854+0000","last_peered":"2024-03-30T02:34:50.749854+0000","last_clean":"2024-03-30T02:34:50.749854+0000","last_became_active":"2024-03-30T02:34:46.735018+0000","last_became_peered":"2024-03-30T02:34:46.735018+0000","last_unstale":"2024-03-30T02:34:50.749854+0000","last_undegraded":"2024-03-30T02:34:50.749854+0000","last_fullsized":"2024-03-30T02:34:50.749854+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:32:15.769536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.651902+0000","last_change":"2024-03-30T02:34:50.773036+0000","last_active":"2024-03-30T02:34:51.651902+0000","last_peered":"2024-03-30T02:34:51.651902+0000","last_clean":"2024-03-30T02:34:51.651902+0000","last_became_active":"2024-03-30T02:34:50.772614+0000","last_became_peered":"2024-03-30T02:34:50.772614+0000","last_unstale":"2024-03-30T02:34:51.651902+0000","last_undegraded":"2024-03-30T02:34:51.651902+0000","last_fullsized":"2024-03-30T02:34:51.651902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:54:28.792793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.652742+0000","last_change":"2024-03-30T02:34:46.209802+0000","last_active":"2024-03-30T02:34:51.652742+0000","last_peered":"2024-03-30T02:34:51.652742+0000","last_clean":"2024-03-30T02:34:51.652742+0000","last_became_active":"2024-03-30T02:34:46.209246+0000","last_became_peered":"2024-03-30T02:34:46.209246+0000","last_unstale":"2024-03-30T02:34:51.652742+0000","last_undegraded":"2024-03-30T02:34:51.652742+0000","last_fullsized":"2024-03-30T02:34:51.652742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:09:57.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.740493+0000","last_change":"2024-03-30T02:34:46.743638+0000","last_active":"2024-03-30T02:34:50.740493+0000","last_peered":"2024-03-30T02:34:50.740493+0000","last_clean":"2024-03-30T02:34:50.740493+0000","last_became_active":"2024-03-30T02:34:46.743246+0000","last_became_peered":"2024-03-30T02:34:46.743246+0000","last_unstale":"2024-03-30T02:34:50.740493+0000","last_undegraded":"2024-03-30T02:34:50.740493+0000","last_fullsized":"2024-03-30T02:34:50.740493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:49:55.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.273748+0000","last_change":"2024-03-30T02:34:50.768586+0000","last_active":"2024-03-30T02:34:51.273748+0000","last_peered":"2024-03-30T02:34:51.273748+0000","last_clean":"2024-03-30T02:34:51.273748+0000","last_became_active":"2024-03-30T02:34:50.766894+0000","last_became_peered":"2024-03-30T02:34:50.766894+0000","last_unstale":"2024-03-30T02:34:51.273748+0000","last_undegraded":"2024-03-30T02:34:51.273748+0000","last_fullsized":"2024-03-30T02:34:51.273748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:36:39.329608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.103474+0000","last_change":"2024-03-30T02:34:46.204994+0000","last_active":"2024-03-30T02:35:16.103474+0000","last_peered":"2024-03-30T02:35:16.103474+0000","last_clean":"2024-03-30T02:35:16.103474+0000","last_became_active":"2024-03-30T02:34:46.204710+0000","last_became_peered":"2024-03-30T02:34:46.204710+0000","last_unstale":"2024-03-30T02:35:16.103474+0000","last_undegraded":"2024-03-30T02:35:16.103474+0000","last_fullsized":"2024-03-30T02:35:16.103474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:39.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743575+0000","last_change":"2024-03-30T02:34:46.733573+0000","last_active":"2024-03-30T02:34:50.743575+0000","last_peered":"2024-03-30T02:34:50.743575+0000","last_clean":"2024-03-30T02:34:50.743575+0000","last_became_active":"2024-03-30T02:34:46.733241+0000","last_became_peered":"2024-03-30T02:34:46.733241+0000","last_unstale":"2024-03-30T02:34:50.743575+0000","last_undegraded":"2024-03-30T02:34:50.743575+0000","last_fullsized":"2024-03-30T02:34:50.743575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:07:10.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.764235+0000","last_change":"2024-03-30T02:34:50.761422+0000","last_active":"2024-03-30T02:34:50.764235+0000","last_peered":"2024-03-30T02:34:50.764235+0000","last_clean":"2024-03-30T02:34:50.764235+0000","last_became_active":"2024-03-30T02:34:50.760976+0000","last_became_peered":"2024-03-30T02:34:50.760976+0000","last_unstale":"2024-03-30T02:34:50.764235+0000","last_undegraded":"2024-03-30T02:34:50.764235+0000","last_fullsized":"2024-03-30T02:34:50.764235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:59:48.745844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.103100+0000","last_change":"2024-03-30T02:34:46.212732+0000","last_active":"2024-03-30T02:35:16.103100+0000","last_peered":"2024-03-30T02:35:16.103100+0000","last_clean":"2024-03-30T02:35:16.103100+0000","last_became_active":"2024-03-30T02:34:46.212306+0000","last_became_peered":"2024-03-30T02:34:46.212306+0000","last_unstale":"2024-03-30T02:35:16.103100+0000","last_undegraded":"2024-03-30T02:35:16.103100+0000","last_fullsized":"2024-03-30T02:35:16.103100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:27:15.766203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.759376+0000","last_change":"2024-03-30T02:34:46.749420+0000","last_active":"2024-03-30T02:34:50.759376+0000","last_peered":"2024-03-30T02:34:50.759376+0000","last_clean":"2024-03-30T02:34:50.759376+0000","last_became_active":"2024-03-30T02:34:46.745286+0000","last_became_peered":"2024-03-30T02:34:46.745286+0000","last_unstale":"2024-03-30T02:34:50.759376+0000","last_undegraded":"2024-03-30T02:34:50.759376+0000","last_fullsized":"2024-03-30T02:34:50.759376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:01.489862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107185+0000","last_change":"2024-03-30T02:34:50.760542+0000","last_active":"2024-03-30T02:34:51.107185+0000","last_peered":"2024-03-30T02:34:51.107185+0000","last_clean":"2024-03-30T02:34:51.107185+0000","last_became_active":"2024-03-30T02:34:50.760087+0000","last_became_peered":"2024-03-30T02:34:50.760087+0000","last_unstale":"2024-03-30T02:34:51.107185+0000","last_undegraded":"2024-03-30T02:34:51.107185+0000","last_fullsized":"2024-03-30T02:34:51.107185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:46.352189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.652842+0000","last_change":"2024-03-30T02:34:46.224058+0000","last_active":"2024-03-30T02:34:51.652842+0000","last_peered":"2024-03-30T02:34:51.652842+0000","last_clean":"2024-03-30T02:34:51.652842+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:34:51.652842+0000","last_undegraded":"2024-03-30T02:34:51.652842+0000","last_fullsized":"2024-03-30T02:34:51.652842+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:31:10.124533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743426+0000","last_change":"2024-03-30T02:34:46.741413+0000","last_active":"2024-03-30T02:34:50.743426+0000","last_peered":"2024-03-30T02:34:50.743426+0000","last_clean":"2024-03-30T02:34:50.743426+0000","last_became_active":"2024-03-30T02:34:46.741112+0000","last_became_peered":"2024-03-30T02:34:46.741112+0000","last_unstale":"2024-03-30T02:34:50.743426+0000","last_undegraded":"2024-03-30T02:34:50.743426+0000","last_fullsized":"2024-03-30T02:34:50.743426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:34.820166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.758467+0000","last_change":"2024-03-30T02:34:50.752315+0000","last_active":"2024-03-30T02:34:50.758467+0000","last_peered":"2024-03-30T02:34:50.758467+0000","last_clean":"2024-03-30T02:34:50.758467+0000","last_became_active":"2024-03-30T02:34:50.750985+0000","last_became_peered":"2024-03-30T02:34:50.750985+0000","last_unstale":"2024-03-30T02:34:50.758467+0000","last_undegraded":"2024-03-30T02:34:50.758467+0000","last_fullsized":"2024-03-30T02:34:50.758467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:37.271063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"83'12","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.101908+0000","last_change":"2024-03-30T02:34:46.214759+0000","last_active":"2024-03-30T02:35:16.101908+0000","last_peered":"2024-03-30T02:35:16.101908+0000","last_clean":"2024-03-30T02:35:16.101908+0000","last_became_active":"2024-03-30T02:34:46.214388+0000","last_became_peered":"2024-03-30T02:34:46.214388+0000","last_unstale":"2024-03-30T02:35:16.101908+0000","last_undegraded":"2024-03-30T02:35:16.101908+0000","last_fullsized":"2024-03-30T02:35:16.101908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:41.100461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.745434+0000","last_change":"2024-03-30T02:34:46.741112+0000","last_active":"2024-03-30T02:34:50.745434+0000","last_peered":"2024-03-30T02:34:50.745434+0000","last_clean":"2024-03-30T02:34:50.745434+0000","last_became_active":"2024-03-30T02:34:46.739664+0000","last_became_peered":"2024-03-30T02:34:46.739664+0000","last_unstale":"2024-03-30T02:34:50.745434+0000","last_undegraded":"2024-03-30T02:34:50.745434+0000","last_fullsized":"2024-03-30T02:34:50.745434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:07.763942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.653103+0000","last_change":"2024-03-30T02:34:50.765645+0000","last_active":"2024-03-30T02:34:51.653103+0000","last_peered":"2024-03-30T02:34:51.653103+0000","last_clean":"2024-03-30T02:34:51.653103+0000","last_became_active":"2024-03-30T02:34:50.765345+0000","last_became_peered":"2024-03-30T02:34:50.765345+0000","last_unstale":"2024-03-30T02:34:51.653103+0000","last_undegraded":"2024-03-30T02:34:51.653103+0000","last_fullsized":"2024-03-30T02:34:51.653103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:25:38.591525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.103173+0000","last_change":"2024-03-30T02:34:46.216643+0000","last_active":"2024-03-30T02:35:16.103173+0000","last_peered":"2024-03-30T02:35:16.103173+0000","last_clean":"2024-03-30T02:35:16.103173+0000","last_became_active":"2024-03-30T02:34:46.216188+0000","last_became_peered":"2024-03-30T02:34:46.216188+0000","last_unstale":"2024-03-30T02:35:16.103173+0000","last_undegraded":"2024-03-30T02:35:16.103173+0000","last_fullsized":"2024-03-30T02:35:16.103173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:49:10.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747685+0000","last_change":"2024-03-30T02:34:46.748313+0000","last_active":"2024-03-30T02:34:50.747685+0000","last_peered":"2024-03-30T02:34:50.747685+0000","last_clean":"2024-03-30T02:34:50.747685+0000","last_became_active":"2024-03-30T02:34:46.742897+0000","last_became_peered":"2024-03-30T02:34:46.742897+0000","last_unstale":"2024-03-30T02:34:50.747685+0000","last_undegraded":"2024-03-30T02:34:50.747685+0000","last_fullsized":"2024-03-30T02:34:50.747685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:00.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076711+0000","last_change":"2024-03-30T02:34:50.760993+0000","last_active":"2024-03-30T02:34:51.076711+0000","last_peered":"2024-03-30T02:34:51.076711+0000","last_clean":"2024-03-30T02:34:51.076711+0000","last_became_active":"2024-03-30T02:34:50.760535+0000","last_became_peered":"2024-03-30T02:34:50.760535+0000","last_unstale":"2024-03-30T02:34:51.076711+0000","last_undegraded":"2024-03-30T02:34:51.076711+0000","last_fullsized":"2024-03-30T02:34:51.076711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:15.569533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.104489+0000","last_change":"2024-03-30T02:34:46.216603+0000","last_active":"2024-03-30T02:35:16.104489+0000","last_peered":"2024-03-30T02:35:16.104489+0000","last_clean":"2024-03-30T02:35:16.104489+0000","last_became_active":"2024-03-30T02:34:46.216231+0000","last_became_peered":"2024-03-30T02:34:46.216231+0000","last_unstale":"2024-03-30T02:35:16.104489+0000","last_undegraded":"2024-03-30T02:35:16.104489+0000","last_fullsized":"2024-03-30T02:35:16.104489+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:51:00.659652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117710+0000","last_change":"2024-03-30T02:34:46.742815+0000","last_active":"2024-03-30T02:34:51.117710+0000","last_peered":"2024-03-30T02:34:51.117710+0000","last_clean":"2024-03-30T02:34:51.117710+0000","last_became_active":"2024-03-30T02:34:46.742487+0000","last_became_peered":"2024-03-30T02:34:46.742487+0000","last_unstale":"2024-03-30T02:34:51.117710+0000","last_undegraded":"2024-03-30T02:34:51.117710+0000","last_fullsized":"2024-03-30T02:34:51.117710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:07.182990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.772972+0000","last_change":"2024-03-30T02:34:50.770398+0000","last_active":"2024-03-30T02:34:50.772972+0000","last_peered":"2024-03-30T02:34:50.772972+0000","last_clean":"2024-03-30T02:34:50.772972+0000","last_became_active":"2024-03-30T02:34:50.770000+0000","last_became_peered":"2024-03-30T02:34:50.770000+0000","last_unstale":"2024-03-30T02:34:50.772972+0000","last_undegraded":"2024-03-30T02:34:50.772972+0000","last_fullsized":"2024-03-30T02:34:50.772972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:14:49.057081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747049+0000","last_change":"2024-03-30T02:34:46.209403+0000","last_active":"2024-03-30T02:34:50.747049+0000","last_peered":"2024-03-30T02:34:50.747049+0000","last_clean":"2024-03-30T02:34:50.747049+0000","last_became_active":"2024-03-30T02:34:46.209065+0000","last_became_peered":"2024-03-30T02:34:46.209065+0000","last_unstale":"2024-03-30T02:34:50.747049+0000","last_undegraded":"2024-03-30T02:34:50.747049+0000","last_fullsized":"2024-03-30T02:34:50.747049+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:36:20.427720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117638+0000","last_change":"2024-03-30T02:34:46.734861+0000","last_active":"2024-03-30T02:34:51.117638+0000","last_peered":"2024-03-30T02:34:51.117638+0000","last_clean":"2024-03-30T02:34:51.117638+0000","last_became_active":"2024-03-30T02:34:46.734376+0000","last_became_peered":"2024-03-30T02:34:46.734376+0000","last_unstale":"2024-03-30T02:34:51.117638+0000","last_undegraded":"2024-03-30T02:34:51.117638+0000","last_fullsized":"2024-03-30T02:34:51.117638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:08:36.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.770978+0000","last_change":"2024-03-30T02:34:50.767469+0000","last_active":"2024-03-30T02:34:50.770978+0000","last_peered":"2024-03-30T02:34:50.770978+0000","last_clean":"2024-03-30T02:34:50.770978+0000","last_became_active":"2024-03-30T02:34:50.765460+0000","last_became_peered":"2024-03-30T02:34:50.765460+0000","last_unstale":"2024-03-30T02:34:50.770978+0000","last_undegraded":"2024-03-30T02:34:50.770978+0000","last_fullsized":"2024-03-30T02:34:50.770978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:54:06.940171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'16","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:17.111894+0000","last_change":"2024-03-30T02:34:46.224053+0000","last_active":"2024-03-30T02:35:17.111894+0000","last_peered":"2024-03-30T02:35:17.111894+0000","last_clean":"2024-03-30T02:35:17.111894+0000","last_became_active":"2024-03-30T02:34:46.223666+0000","last_became_peered":"2024-03-30T02:34:46.223666+0000","last_unstale":"2024-03-30T02:35:17.111894+0000","last_undegraded":"2024-03-30T02:35:17.111894+0000","last_fullsized":"2024-03-30T02:35:17.111894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:18:22.976298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748472+0000","last_change":"2024-03-30T02:34:46.734838+0000","last_active":"2024-03-30T02:34:50.748472+0000","last_peered":"2024-03-30T02:34:50.748472+0000","last_clean":"2024-03-30T02:34:50.748472+0000","last_became_active":"2024-03-30T02:34:46.734428+0000","last_became_peered":"2024-03-30T02:34:46.734428+0000","last_unstale":"2024-03-30T02:34:50.748472+0000","last_undegraded":"2024-03-30T02:34:50.748472+0000","last_fullsized":"2024-03-30T02:34:50.748472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:42:51.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.652925+0000","last_change":"2024-03-30T02:34:50.771150+0000","last_active":"2024-03-30T02:34:51.652925+0000","last_peered":"2024-03-30T02:34:51.652925+0000","last_clean":"2024-03-30T02:34:51.652925+0000","last_became_active":"2024-03-30T02:34:50.770703+0000","last_became_peered":"2024-03-30T02:34:50.770703+0000","last_unstale":"2024-03-30T02:34:51.652925+0000","last_undegraded":"2024-03-30T02:34:51.652925+0000","last_fullsized":"2024-03-30T02:34:51.652925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:04.466664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.745296+0000","last_change":"2024-03-30T02:34:46.203558+0000","last_active":"2024-03-30T02:34:50.745296+0000","last_peered":"2024-03-30T02:34:50.745296+0000","last_clean":"2024-03-30T02:34:50.745296+0000","last_became_active":"2024-03-30T02:34:46.202896+0000","last_became_peered":"2024-03-30T02:34:46.202896+0000","last_unstale":"2024-03-30T02:34:50.745296+0000","last_undegraded":"2024-03-30T02:34:50.745296+0000","last_fullsized":"2024-03-30T02:34:50.745296+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:33.945665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076752+0000","last_change":"2024-03-30T02:34:46.747519+0000","last_active":"2024-03-30T02:34:51.076752+0000","last_peered":"2024-03-30T02:34:51.076752+0000","last_clean":"2024-03-30T02:34:51.076752+0000","last_became_active":"2024-03-30T02:34:46.747220+0000","last_became_peered":"2024-03-30T02:34:46.747220+0000","last_unstale":"2024-03-30T02:34:51.076752+0000","last_undegraded":"2024-03-30T02:34:51.076752+0000","last_fullsized":"2024-03-30T02:34:51.076752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:21:19.419986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.369649+0000","last_change":"2024-03-30T02:34:50.797758+0000","last_active":"2024-03-30T02:34:51.369649+0000","last_peered":"2024-03-30T02:34:51.369649+0000","last_clean":"2024-03-30T02:34:51.369649+0000","last_became_active":"2024-03-30T02:34:50.797311+0000","last_became_peered":"2024-03-30T02:34:50.797311+0000","last_unstale":"2024-03-30T02:34:51.369649+0000","last_undegraded":"2024-03-30T02:34:51.369649+0000","last_fullsized":"2024-03-30T02:34:51.369649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:32.337422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.367367+0000","last_change":"2024-03-30T02:34:46.224397+0000","last_active":"2024-03-30T02:34:51.367367+0000","last_peered":"2024-03-30T02:34:51.367367+0000","last_clean":"2024-03-30T02:34:51.367367+0000","last_became_active":"2024-03-30T02:34:46.223825+0000","last_became_peered":"2024-03-30T02:34:46.223825+0000","last_unstale":"2024-03-30T02:34:51.367367+0000","last_undegraded":"2024-03-30T02:34:51.367367+0000","last_fullsized":"2024-03-30T02:34:51.367367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:53.154665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.796460+0000","last_change":"2024-03-30T02:34:46.741358+0000","last_active":"2024-03-30T02:34:50.796460+0000","last_peered":"2024-03-30T02:34:50.796460+0000","last_clean":"2024-03-30T02:34:50.796460+0000","last_became_active":"2024-03-30T02:34:46.740734+0000","last_became_peered":"2024-03-30T02:34:46.740734+0000","last_unstale":"2024-03-30T02:34:50.796460+0000","last_undegraded":"2024-03-30T02:34:50.796460+0000","last_fullsized":"2024-03-30T02:34:50.796460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:59.678431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.770598+0000","last_change":"2024-03-30T02:34:50.761998+0000","last_active":"2024-03-30T02:34:50.770598+0000","last_peered":"2024-03-30T02:34:50.770598+0000","last_clean":"2024-03-30T02:34:50.770598+0000","last_became_active":"2024-03-30T02:34:50.761560+0000","last_became_peered":"2024-03-30T02:34:50.761560+0000","last_unstale":"2024-03-30T02:34:50.770598+0000","last_undegraded":"2024-03-30T02:34:50.770598+0000","last_fullsized":"2024-03-30T02:34:50.770598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:57:17.213624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747544+0000","last_change":"2024-03-30T02:34:46.213427+0000","last_active":"2024-03-30T02:34:50.747544+0000","last_peered":"2024-03-30T02:34:50.747544+0000","last_clean":"2024-03-30T02:34:50.747544+0000","last_became_active":"2024-03-30T02:34:46.213082+0000","last_became_peered":"2024-03-30T02:34:46.213082+0000","last_unstale":"2024-03-30T02:34:50.747544+0000","last_undegraded":"2024-03-30T02:34:50.747544+0000","last_fullsized":"2024-03-30T02:34:50.747544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:31:35.513723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.796452+0000","last_change":"2024-03-30T02:34:46.747197+0000","last_active":"2024-03-30T02:34:50.796452+0000","last_peered":"2024-03-30T02:34:50.796452+0000","last_clean":"2024-03-30T02:34:50.796452+0000","last_became_active":"2024-03-30T02:34:46.743841+0000","last_became_peered":"2024-03-30T02:34:46.743841+0000","last_unstale":"2024-03-30T02:34:50.796452+0000","last_undegraded":"2024-03-30T02:34:50.796452+0000","last_fullsized":"2024-03-30T02:34:50.796452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:09.234382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.798995+0000","last_change":"2024-03-30T02:34:50.797423+0000","last_active":"2024-03-30T02:34:50.798995+0000","last_peered":"2024-03-30T02:34:50.798995+0000","last_clean":"2024-03-30T02:34:50.798995+0000","last_became_active":"2024-03-30T02:34:50.796746+0000","last_became_peered":"2024-03-30T02:34:50.796746+0000","last_unstale":"2024-03-30T02:34:50.798995+0000","last_undegraded":"2024-03-30T02:34:50.798995+0000","last_fullsized":"2024-03-30T02:34:50.798995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:24.014131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748816+0000","last_change":"2024-03-30T02:34:46.734972+0000","last_active":"2024-03-30T02:34:50.748816+0000","last_peered":"2024-03-30T02:34:50.748816+0000","last_clean":"2024-03-30T02:34:50.748816+0000","last_became_active":"2024-03-30T02:34:46.734520+0000","last_became_peered":"2024-03-30T02:34:46.734520+0000","last_unstale":"2024-03-30T02:34:50.748816+0000","last_undegraded":"2024-03-30T02:34:50.748816+0000","last_fullsized":"2024-03-30T02:34:50.748816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:15.635060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:17.111904+0000","last_change":"2024-03-30T02:34:46.224036+0000","last_active":"2024-03-30T02:35:17.111904+0000","last_peered":"2024-03-30T02:35:17.111904+0000","last_clean":"2024-03-30T02:35:17.111904+0000","last_became_active":"2024-03-30T02:34:46.223706+0000","last_became_peered":"2024-03-30T02:34:46.223706+0000","last_unstale":"2024-03-30T02:35:17.111904+0000","last_undegraded":"2024-03-30T02:35:17.111904+0000","last_fullsized":"2024-03-30T02:35:17.111904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:40.458345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117736+0000","last_change":"2024-03-30T02:34:50.757894+0000","last_active":"2024-03-30T02:34:51.117736+0000","last_peered":"2024-03-30T02:34:51.117736+0000","last_clean":"2024-03-30T02:34:51.117736+0000","last_became_active":"2024-03-30T02:34:50.757486+0000","last_became_peered":"2024-03-30T02:34:50.757486+0000","last_unstale":"2024-03-30T02:34:51.117736+0000","last_undegraded":"2024-03-30T02:34:51.117736+0000","last_fullsized":"2024-03-30T02:34:51.117736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:30.609851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748899+0000","last_change":"2024-03-30T02:34:46.745990+0000","last_active":"2024-03-30T02:34:50.748899+0000","last_peered":"2024-03-30T02:34:50.748899+0000","last_clean":"2024-03-30T02:34:50.748899+0000","last_became_active":"2024-03-30T02:34:46.741937+0000","last_became_peered":"2024-03-30T02:34:46.741937+0000","last_unstale":"2024-03-30T02:34:50.748899+0000","last_undegraded":"2024-03-30T02:34:50.748899+0000","last_fullsized":"2024-03-30T02:34:50.748899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:26:17.833164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.102092+0000","last_change":"2024-03-30T02:34:46.208858+0000","last_active":"2024-03-30T02:35:16.102092+0000","last_peered":"2024-03-30T02:35:16.102092+0000","last_clean":"2024-03-30T02:35:16.102092+0000","last_became_active":"2024-03-30T02:34:46.208479+0000","last_became_peered":"2024-03-30T02:34:46.208479+0000","last_unstale":"2024-03-30T02:35:16.102092+0000","last_undegraded":"2024-03-30T02:35:16.102092+0000","last_fullsized":"2024-03-30T02:35:16.102092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:58:14.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.653940+0000","last_change":"2024-03-30T02:33:56.208237+0000","last_active":"2024-03-30T02:34:51.653940+0000","last_peered":"2024-03-30T02:34:51.653940+0000","last_clean":"2024-03-30T02:34:51.653940+0000","last_became_active":"2024-03-30T02:33:56.207820+0000","last_became_peered":"2024-03-30T02:33:56.207820+0000","last_unstale":"2024-03-30T02:34:51.653940+0000","last_undegraded":"2024-03-30T02:34:51.653940+0000","last_fullsized":"2024-03-30T02:34:51.653940+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_clean_scrub_stamp":"2024-03-30T02:31:23.922434+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:26.198503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.652719+0000","last_change":"2024-03-30T02:34:50.797822+0000","last_active":"2024-03-30T02:34:51.652719+0000","last_peered":"2024-03-30T02:34:51.652719+0000","last_clean":"2024-03-30T02:34:51.652719+0000","last_became_active":"2024-03-30T02:34:50.797357+0000","last_became_peered":"2024-03-30T02:34:50.797357+0000","last_unstale":"2024-03-30T02:34:51.652719+0000","last_undegraded":"2024-03-30T02:34:51.652719+0000","last_fullsized":"2024-03-30T02:34:51.652719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:25:15.806076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747081+0000","last_change":"2024-03-30T02:34:46.739757+0000","last_active":"2024-03-30T02:34:50.747081+0000","last_peered":"2024-03-30T02:34:50.747081+0000","last_clean":"2024-03-30T02:34:50.747081+0000","last_became_active":"2024-03-30T02:34:46.738269+0000","last_became_peered":"2024-03-30T02:34:46.738269+0000","last_unstale":"2024-03-30T02:34:50.747081+0000","last_undegraded":"2024-03-30T02:34:50.747081+0000","last_fullsized":"2024-03-30T02:34:50.747081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:03.411187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.654032+0000","last_change":"2024-03-30T02:34:46.212532+0000","last_active":"2024-03-30T02:34:51.654032+0000","last_peered":"2024-03-30T02:34:51.654032+0000","last_clean":"2024-03-30T02:34:51.654032+0000","last_became_active":"2024-03-30T02:34:46.212191+0000","last_became_peered":"2024-03-30T02:34:46.212191+0000","last_unstale":"2024-03-30T02:34:51.654032+0000","last_undegraded":"2024-03-30T02:34:51.654032+0000","last_fullsized":"2024-03-30T02:34:51.654032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:19.120960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076666+0000","last_change":"2024-03-30T02:34:50.759911+0000","last_active":"2024-03-30T02:34:51.076666+0000","last_peered":"2024-03-30T02:34:51.076666+0000","last_clean":"2024-03-30T02:34:51.076666+0000","last_became_active":"2024-03-30T02:34:50.759487+0000","last_became_peered":"2024-03-30T02:34:50.759487+0000","last_unstale":"2024-03-30T02:34:51.076666+0000","last_undegraded":"2024-03-30T02:34:51.076666+0000","last_fullsized":"2024-03-30T02:34:51.076666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:46.403754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.115192+0000","last_change":"2024-03-30T02:34:46.217092+0000","last_active":"2024-03-30T02:35:18.115192+0000","last_peered":"2024-03-30T02:35:18.115192+0000","last_clean":"2024-03-30T02:35:18.115192+0000","last_became_active":"2024-03-30T02:34:46.216779+0000","last_became_peered":"2024-03-30T02:34:46.216779+0000","last_unstale":"2024-03-30T02:35:18.115192+0000","last_undegraded":"2024-03-30T02:35:18.115192+0000","last_fullsized":"2024-03-30T02:35:18.115192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:10.894577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.757761+0000","last_change":"2024-03-30T02:34:46.735212+0000","last_active":"2024-03-30T02:34:50.757761+0000","last_peered":"2024-03-30T02:34:50.757761+0000","last_clean":"2024-03-30T02:34:50.757761+0000","last_became_active":"2024-03-30T02:34:46.734282+0000","last_became_peered":"2024-03-30T02:34:46.734282+0000","last_unstale":"2024-03-30T02:34:50.757761+0000","last_undegraded":"2024-03-30T02:34:50.757761+0000","last_fullsized":"2024-03-30T02:34:50.757761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:28.596961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.273248+0000","last_change":"2024-03-30T02:34:50.799290+0000","last_active":"2024-03-30T02:34:51.273248+0000","last_peered":"2024-03-30T02:34:51.273248+0000","last_clean":"2024-03-30T02:34:51.273248+0000","last_became_active":"2024-03-30T02:34:50.797524+0000","last_became_peered":"2024-03-30T02:34:50.797524+0000","last_unstale":"2024-03-30T02:34:51.273248+0000","last_undegraded":"2024-03-30T02:34:51.273248+0000","last_fullsized":"2024-03-30T02:34:51.273248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:53:27.931444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.115016+0000","last_change":"2024-03-30T02:34:46.202508+0000","last_active":"2024-03-30T02:35:18.115016+0000","last_peered":"2024-03-30T02:35:18.115016+0000","last_clean":"2024-03-30T02:35:18.115016+0000","last_became_active":"2024-03-30T02:34:46.202145+0000","last_became_peered":"2024-03-30T02:34:46.202145+0000","last_unstale":"2024-03-30T02:35:18.115016+0000","last_undegraded":"2024-03-30T02:35:18.115016+0000","last_fullsized":"2024-03-30T02:35:18.115016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:21.138776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.753707+0000","last_change":"2024-03-30T02:34:46.740056+0000","last_active":"2024-03-30T02:34:50.753707+0000","last_peered":"2024-03-30T02:34:50.753707+0000","last_clean":"2024-03-30T02:34:50.753707+0000","last_became_active":"2024-03-30T02:34:46.739260+0000","last_became_peered":"2024-03-30T02:34:46.739260+0000","last_unstale":"2024-03-30T02:34:50.753707+0000","last_undegraded":"2024-03-30T02:34:50.753707+0000","last_fullsized":"2024-03-30T02:34:50.753707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:53.550978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.272694+0000","last_change":"2024-03-30T02:34:50.756851+0000","last_active":"2024-03-30T02:34:51.272694+0000","last_peered":"2024-03-30T02:34:51.272694+0000","last_clean":"2024-03-30T02:34:51.272694+0000","last_became_active":"2024-03-30T02:34:50.756416+0000","last_became_peered":"2024-03-30T02:34:50.756416+0000","last_unstale":"2024-03-30T02:34:51.272694+0000","last_undegraded":"2024-03-30T02:34:51.272694+0000","last_fullsized":"2024-03-30T02:34:51.272694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:11.469599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.099514+0000","last_change":"2024-03-30T02:34:46.215663+0000","last_active":"2024-03-30T02:35:16.099514+0000","last_peered":"2024-03-30T02:35:16.099514+0000","last_clean":"2024-03-30T02:35:16.099514+0000","last_became_active":"2024-03-30T02:34:46.214224+0000","last_became_peered":"2024-03-30T02:34:46.214224+0000","last_unstale":"2024-03-30T02:35:16.099514+0000","last_undegraded":"2024-03-30T02:35:16.099514+0000","last_fullsized":"2024-03-30T02:35:16.099514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:24:58.734434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.755260+0000","last_change":"2024-03-30T02:34:46.754150+0000","last_active":"2024-03-30T02:34:50.755260+0000","last_peered":"2024-03-30T02:34:50.755260+0000","last_clean":"2024-03-30T02:34:50.755260+0000","last_became_active":"2024-03-30T02:34:46.753786+0000","last_became_peered":"2024-03-30T02:34:46.753786+0000","last_unstale":"2024-03-30T02:34:50.755260+0000","last_undegraded":"2024-03-30T02:34:50.755260+0000","last_fullsized":"2024-03-30T02:34:50.755260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:54:50.165020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107246+0000","last_change":"2024-03-30T02:34:50.797890+0000","last_active":"2024-03-30T02:34:51.107246+0000","last_peered":"2024-03-30T02:34:51.107246+0000","last_clean":"2024-03-30T02:34:51.107246+0000","last_became_active":"2024-03-30T02:34:50.797395+0000","last_became_peered":"2024-03-30T02:34:50.797395+0000","last_unstale":"2024-03-30T02:34:51.107246+0000","last_undegraded":"2024-03-30T02:34:51.107246+0000","last_fullsized":"2024-03-30T02:34:51.107246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:36:14.958804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.102548+0000","last_change":"2024-03-30T02:34:46.220873+0000","last_active":"2024-03-30T02:35:16.102548+0000","last_peered":"2024-03-30T02:35:16.102548+0000","last_clean":"2024-03-30T02:35:16.102548+0000","last_became_active":"2024-03-30T02:34:46.220544+0000","last_became_peered":"2024-03-30T02:34:46.220544+0000","last_unstale":"2024-03-30T02:35:16.102548+0000","last_undegraded":"2024-03-30T02:35:16.102548+0000","last_fullsized":"2024-03-30T02:35:16.102548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:37.691602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.746086+0000","last_change":"2024-03-30T02:34:46.739184+0000","last_active":"2024-03-30T02:34:50.746086+0000","last_peered":"2024-03-30T02:34:50.746086+0000","last_clean":"2024-03-30T02:34:50.746086+0000","last_became_active":"2024-03-30T02:34:46.738843+0000","last_became_peered":"2024-03-30T02:34:46.738843+0000","last_unstale":"2024-03-30T02:34:50.746086+0000","last_undegraded":"2024-03-30T02:34:50.746086+0000","last_fullsized":"2024-03-30T02:34:50.746086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:15:09.894754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.772096+0000","last_change":"2024-03-30T02:34:50.768763+0000","last_active":"2024-03-30T02:34:50.772096+0000","last_peered":"2024-03-30T02:34:50.772096+0000","last_clean":"2024-03-30T02:34:50.772096+0000","last_became_active":"2024-03-30T02:34:50.767363+0000","last_became_peered":"2024-03-30T02:34:50.767363+0000","last_unstale":"2024-03-30T02:34:50.772096+0000","last_undegraded":"2024-03-30T02:34:50.772096+0000","last_fullsized":"2024-03-30T02:34:50.772096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:06.932826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.113271+0000","last_change":"2024-03-30T02:34:46.213006+0000","last_active":"2024-03-30T02:35:18.113271+0000","last_peered":"2024-03-30T02:35:18.113271+0000","last_clean":"2024-03-30T02:35:18.113271+0000","last_became_active":"2024-03-30T02:34:46.206271+0000","last_became_peered":"2024-03-30T02:34:46.206271+0000","last_unstale":"2024-03-30T02:35:18.113271+0000","last_undegraded":"2024-03-30T02:35:18.113271+0000","last_fullsized":"2024-03-30T02:35:18.113271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:00:36.447352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076628+0000","last_change":"2024-03-30T02:34:46.739566+0000","last_active":"2024-03-30T02:34:51.076628+0000","last_peered":"2024-03-30T02:34:51.076628+0000","last_clean":"2024-03-30T02:34:51.076628+0000","last_became_active":"2024-03-30T02:34:46.739205+0000","last_became_peered":"2024-03-30T02:34:46.739205+0000","last_unstale":"2024-03-30T02:34:51.076628+0000","last_undegraded":"2024-03-30T02:34:51.076628+0000","last_fullsized":"2024-03-30T02:34:51.076628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:01.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107196+0000","last_change":"2024-03-30T02:34:50.758016+0000","last_active":"2024-03-30T02:34:51.107196+0000","last_peered":"2024-03-30T02:34:51.107196+0000","last_clean":"2024-03-30T02:34:51.107196+0000","last_became_active":"2024-03-30T02:34:50.757616+0000","last_became_peered":"2024-03-30T02:34:50.757616+0000","last_unstale":"2024-03-30T02:34:51.107196+0000","last_undegraded":"2024-03-30T02:34:51.107196+0000","last_fullsized":"2024-03-30T02:34:51.107196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:35:07.902347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747497+0000","last_change":"2024-03-30T02:34:46.217138+0000","last_active":"2024-03-30T02:34:50.747497+0000","last_peered":"2024-03-30T02:34:50.747497+0000","last_clean":"2024-03-30T02:34:50.747497+0000","last_became_active":"2024-03-30T02:34:46.216799+0000","last_became_peered":"2024-03-30T02:34:46.216799+0000","last_unstale":"2024-03-30T02:34:50.747497+0000","last_undegraded":"2024-03-30T02:34:50.747497+0000","last_fullsized":"2024-03-30T02:34:50.747497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:43:42.562874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748720+0000","last_change":"2024-03-30T02:34:46.754114+0000","last_active":"2024-03-30T02:34:50.748720+0000","last_peered":"2024-03-30T02:34:50.748720+0000","last_clean":"2024-03-30T02:34:50.748720+0000","last_became_active":"2024-03-30T02:34:46.752691+0000","last_became_peered":"2024-03-30T02:34:46.752691+0000","last_unstale":"2024-03-30T02:34:50.748720+0000","last_undegraded":"2024-03-30T02:34:50.748720+0000","last_fullsized":"2024-03-30T02:34:50.748720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:42.908877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107236+0000","last_change":"2024-03-30T02:34:50.798110+0000","last_active":"2024-03-30T02:34:51.107236+0000","last_peered":"2024-03-30T02:34:51.107236+0000","last_clean":"2024-03-30T02:34:51.107236+0000","last_became_active":"2024-03-30T02:34:50.797551+0000","last_became_peered":"2024-03-30T02:34:50.797551+0000","last_unstale":"2024-03-30T02:34:51.107236+0000","last_undegraded":"2024-03-30T02:34:51.107236+0000","last_fullsized":"2024-03-30T02:34:51.107236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:49.362018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.102696+0000","last_change":"2024-03-30T02:34:46.211647+0000","last_active":"2024-03-30T02:35:16.102696+0000","last_peered":"2024-03-30T02:35:16.102696+0000","last_clean":"2024-03-30T02:35:16.102696+0000","last_became_active":"2024-03-30T02:34:46.211311+0000","last_became_peered":"2024-03-30T02:34:46.211311+0000","last_unstale":"2024-03-30T02:35:16.102696+0000","last_undegraded":"2024-03-30T02:35:16.102696+0000","last_fullsized":"2024-03-30T02:35:16.102696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:05:34.135289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117673+0000","last_change":"2024-03-30T02:34:46.740256+0000","last_active":"2024-03-30T02:34:51.117673+0000","last_peered":"2024-03-30T02:34:51.117673+0000","last_clean":"2024-03-30T02:34:51.117673+0000","last_became_active":"2024-03-30T02:34:46.739913+0000","last_became_peered":"2024-03-30T02:34:46.739913+0000","last_unstale":"2024-03-30T02:34:51.117673+0000","last_undegraded":"2024-03-30T02:34:51.117673+0000","last_fullsized":"2024-03-30T02:34:51.117673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:44.152475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.762477+0000","last_change":"2024-03-30T02:34:50.762149+0000","last_active":"2024-03-30T02:34:50.762477+0000","last_peered":"2024-03-30T02:34:50.762477+0000","last_clean":"2024-03-30T02:34:50.762477+0000","last_became_active":"2024-03-30T02:34:50.761717+0000","last_became_peered":"2024-03-30T02:34:50.761717+0000","last_unstale":"2024-03-30T02:34:50.762477+0000","last_undegraded":"2024-03-30T02:34:50.762477+0000","last_fullsized":"2024-03-30T02:34:50.762477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:22.373079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.746982+0000","last_change":"2024-03-30T02:34:46.224098+0000","last_active":"2024-03-30T02:34:50.746982+0000","last_peered":"2024-03-30T02:34:50.746982+0000","last_clean":"2024-03-30T02:34:50.746982+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:34:50.746982+0000","last_undegraded":"2024-03-30T02:34:50.746982+0000","last_fullsized":"2024-03-30T02:34:50.746982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:20:25.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117762+0000","last_change":"2024-03-30T02:34:46.737661+0000","last_active":"2024-03-30T02:34:51.117762+0000","last_peered":"2024-03-30T02:34:51.117762+0000","last_clean":"2024-03-30T02:34:51.117762+0000","last_became_active":"2024-03-30T02:34:46.737303+0000","last_became_peered":"2024-03-30T02:34:46.737303+0000","last_unstale":"2024-03-30T02:34:51.117762+0000","last_undegraded":"2024-03-30T02:34:51.117762+0000","last_fullsized":"2024-03-30T02:34:51.117762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:44.929643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.651797+0000","last_change":"2024-03-30T02:34:50.767119+0000","last_active":"2024-03-30T02:34:51.651797+0000","last_peered":"2024-03-30T02:34:51.651797+0000","last_clean":"2024-03-30T02:34:51.651797+0000","last_became_active":"2024-03-30T02:34:50.766472+0000","last_became_peered":"2024-03-30T02:34:50.766472+0000","last_unstale":"2024-03-30T02:34:51.651797+0000","last_undegraded":"2024-03-30T02:34:51.651797+0000","last_fullsized":"2024-03-30T02:34:51.651797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:46.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748706+0000","last_change":"2024-03-30T02:34:46.216713+0000","last_active":"2024-03-30T02:34:50.748706+0000","last_peered":"2024-03-30T02:34:50.748706+0000","last_clean":"2024-03-30T02:34:50.748706+0000","last_became_active":"2024-03-30T02:34:46.216366+0000","last_became_peered":"2024-03-30T02:34:46.216366+0000","last_unstale":"2024-03-30T02:34:50.748706+0000","last_undegraded":"2024-03-30T02:34:50.748706+0000","last_fullsized":"2024-03-30T02:34:50.748706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:43.803955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117724+0000","last_change":"2024-03-30T02:34:46.736854+0000","last_active":"2024-03-30T02:34:51.117724+0000","last_peered":"2024-03-30T02:34:51.117724+0000","last_clean":"2024-03-30T02:34:51.117724+0000","last_became_active":"2024-03-30T02:34:46.736511+0000","last_became_peered":"2024-03-30T02:34:46.736511+0000","last_unstale":"2024-03-30T02:34:51.117724+0000","last_undegraded":"2024-03-30T02:34:51.117724+0000","last_fullsized":"2024-03-30T02:34:51.117724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:12.701467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.652228+0000","last_change":"2024-03-30T02:34:50.757227+0000","last_active":"2024-03-30T02:34:51.652228+0000","last_peered":"2024-03-30T02:34:51.652228+0000","last_clean":"2024-03-30T02:34:51.652228+0000","last_became_active":"2024-03-30T02:34:50.756742+0000","last_became_peered":"2024-03-30T02:34:50.756742+0000","last_unstale":"2024-03-30T02:34:51.652228+0000","last_undegraded":"2024-03-30T02:34:51.652228+0000","last_fullsized":"2024-03-30T02:34:51.652228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:48.792202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:19.123452+0000","last_change":"2024-03-30T02:34:46.218954+0000","last_active":"2024-03-30T02:35:19.123452+0000","last_peered":"2024-03-30T02:35:19.123452+0000","last_clean":"2024-03-30T02:35:19.123452+0000","last_became_active":"2024-03-30T02:34:46.218019+0000","last_became_peered":"2024-03-30T02:34:46.218019+0000","last_unstale":"2024-03-30T02:35:19.123452+0000","last_undegraded":"2024-03-30T02:35:19.123452+0000","last_fullsized":"2024-03-30T02:35:19.123452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:32.651216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117781+0000","last_change":"2024-03-30T02:34:46.741735+0000","last_active":"2024-03-30T02:34:51.117781+0000","last_peered":"2024-03-30T02:34:51.117781+0000","last_clean":"2024-03-30T02:34:51.117781+0000","last_became_active":"2024-03-30T02:34:46.740663+0000","last_became_peered":"2024-03-30T02:34:46.740663+0000","last_unstale":"2024-03-30T02:34:51.117781+0000","last_undegraded":"2024-03-30T02:34:51.117781+0000","last_fullsized":"2024-03-30T02:34:51.117781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:09.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076652+0000","last_change":"2024-03-30T02:34:46.739082+0000","last_active":"2024-03-30T02:34:51.076652+0000","last_peered":"2024-03-30T02:34:51.076652+0000","last_clean":"2024-03-30T02:34:51.076652+0000","last_became_active":"2024-03-30T02:34:46.738732+0000","last_became_peered":"2024-03-30T02:34:46.738732+0000","last_unstale":"2024-03-30T02:34:51.076652+0000","last_undegraded":"2024-03-30T02:34:51.076652+0000","last_fullsized":"2024-03-30T02:34:51.076652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:53.961520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.750297+0000","last_change":"2024-03-30T02:34:46.744297+0000","last_active":"2024-03-30T02:34:50.750297+0000","last_peered":"2024-03-30T02:34:50.750297+0000","last_clean":"2024-03-30T02:34:50.750297+0000","last_became_active":"2024-03-30T02:34:46.743284+0000","last_became_peered":"2024-03-30T02:34:46.743284+0000","last_unstale":"2024-03-30T02:34:50.750297+0000","last_undegraded":"2024-03-30T02:34:50.750297+0000","last_fullsized":"2024-03-30T02:34:50.750297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:30:20.960739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747399+0000","last_change":"2024-03-30T02:34:46.742590+0000","last_active":"2024-03-30T02:34:50.747399+0000","last_peered":"2024-03-30T02:34:50.747399+0000","last_clean":"2024-03-30T02:34:50.747399+0000","last_became_active":"2024-03-30T02:34:46.740904+0000","last_became_peered":"2024-03-30T02:34:46.740904+0000","last_unstale":"2024-03-30T02:34:50.747399+0000","last_undegraded":"2024-03-30T02:34:50.747399+0000","last_fullsized":"2024-03-30T02:34:50.747399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:22.653881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.756079+0000","last_change":"2024-03-30T02:34:46.753119+0000","last_active":"2024-03-30T02:34:50.756079+0000","last_peered":"2024-03-30T02:34:50.756079+0000","last_clean":"2024-03-30T02:34:50.756079+0000","last_became_active":"2024-03-30T02:34:46.745680+0000","last_became_peered":"2024-03-30T02:34:46.745680+0000","last_unstale":"2024-03-30T02:34:50.756079+0000","last_undegraded":"2024-03-30T02:34:50.756079+0000","last_fullsized":"2024-03-30T02:34:50.756079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:02.352961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.757900+0000","last_change":"2024-03-30T02:34:46.740004+0000","last_active":"2024-03-30T02:34:50.757900+0000","last_peered":"2024-03-30T02:34:50.757900+0000","last_clean":"2024-03-30T02:34:50.757900+0000","last_became_active":"2024-03-30T02:34:46.737934+0000","last_became_peered":"2024-03-30T02:34:46.737934+0000","last_unstale":"2024-03-30T02:34:50.757900+0000","last_undegraded":"2024-03-30T02:34:50.757900+0000","last_fullsized":"2024-03-30T02:34:50.757900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:03:39.976441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076739+0000","last_change":"2024-03-30T02:34:46.737882+0000","last_active":"2024-03-30T02:34:51.076739+0000","last_peered":"2024-03-30T02:34:51.076739+0000","last_clean":"2024-03-30T02:34:51.076739+0000","last_became_active":"2024-03-30T02:34:46.737546+0000","last_became_peered":"2024-03-30T02:34:46.737546+0000","last_unstale":"2024-03-30T02:34:51.076739+0000","last_undegraded":"2024-03-30T02:34:51.076739+0000","last_fullsized":"2024-03-30T02:34:51.076739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:48:41.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.751323+0000","last_change":"2024-03-30T02:34:46.742703+0000","last_active":"2024-03-30T02:34:50.751323+0000","last_peered":"2024-03-30T02:34:50.751323+0000","last_clean":"2024-03-30T02:34:50.751323+0000","last_became_active":"2024-03-30T02:34:46.740689+0000","last_became_peered":"2024-03-30T02:34:46.740689+0000","last_unstale":"2024-03-30T02:34:50.751323+0000","last_undegraded":"2024-03-30T02:34:50.751323+0000","last_fullsized":"2024-03-30T02:34:50.751323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:45.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.753822+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:34:50.753822+0000","last_peered":"2024-03-30T02:34:50.753822+0000","last_clean":"2024-03-30T02:34:50.753822+0000","last_became_active":"2024-03-30T02:34:46.750395+0000","last_became_peered":"2024-03-30T02:34:46.750395+0000","last_unstale":"2024-03-30T02:34:50.753822+0000","last_undegraded":"2024-03-30T02:34:50.753822+0000","last_fullsized":"2024-03-30T02:34:50.753822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:01.282677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748705+0000","last_change":"2024-03-30T02:34:46.734814+0000","last_active":"2024-03-30T02:34:50.748705+0000","last_peered":"2024-03-30T02:34:50.748705+0000","last_clean":"2024-03-30T02:34:50.748705+0000","last_became_active":"2024-03-30T02:34:46.733978+0000","last_became_peered":"2024-03-30T02:34:46.733978+0000","last_unstale":"2024-03-30T02:34:50.748705+0000","last_undegraded":"2024-03-30T02:34:50.748705+0000","last_fullsized":"2024-03-30T02:34:50.748705+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:31:39.467478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748398+0000","last_change":"2024-03-30T02:34:46.748163+0000","last_active":"2024-03-30T02:34:50.748398+0000","last_peered":"2024-03-30T02:34:50.748398+0000","last_clean":"2024-03-30T02:34:50.748398+0000","last_became_active":"2024-03-30T02:34:46.738845+0000","last_became_peered":"2024-03-30T02:34:46.738845+0000","last_unstale":"2024-03-30T02:34:50.748398+0000","last_undegraded":"2024-03-30T02:34:50.748398+0000","last_fullsized":"2024-03-30T02:34:50.748398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:09.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743901+0000","last_change":"2024-03-30T02:34:46.743231+0000","last_active":"2024-03-30T02:34:50.743901+0000","last_peered":"2024-03-30T02:34:50.743901+0000","last_clean":"2024-03-30T02:34:50.743901+0000","last_became_active":"2024-03-30T02:34:46.742619+0000","last_became_peered":"2024-03-30T02:34:46.742619+0000","last_unstale":"2024-03-30T02:34:50.743901+0000","last_undegraded":"2024-03-30T02:34:50.743901+0000","last_fullsized":"2024-03-30T02:34:50.743901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:17:51.789139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747504+0000","last_change":"2024-03-30T02:34:46.745872+0000","last_active":"2024-03-30T02:34:50.747504+0000","last_peered":"2024-03-30T02:34:50.747504+0000","last_clean":"2024-03-30T02:34:50.747504+0000","last_became_active":"2024-03-30T02:34:46.745476+0000","last_became_peered":"2024-03-30T02:34:46.745476+0000","last_unstale":"2024-03-30T02:34:50.747504+0000","last_undegraded":"2024-03-30T02:34:50.747504+0000","last_fullsized":"2024-03-30T02:34:50.747504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:30.774316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076724+0000","last_change":"2024-03-30T02:34:46.746178+0000","last_active":"2024-03-30T02:34:51.076724+0000","last_peered":"2024-03-30T02:34:51.076724+0000","last_clean":"2024-03-30T02:34:51.076724+0000","last_became_active":"2024-03-30T02:34:46.745865+0000","last_became_peered":"2024-03-30T02:34:46.745865+0000","last_unstale":"2024-03-30T02:34:51.076724+0000","last_undegraded":"2024-03-30T02:34:51.076724+0000","last_fullsized":"2024-03-30T02:34:51.076724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:22.613637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248704,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6356,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":147,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":42074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579919,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":444925,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56599999999999995}]},{"osd":1,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64900000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.745},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.76200000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.60899999999999999}]},{"osd":7,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":9,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.70099999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.75600000000000001}]}]},{"osd":10,"up_from":67,"seq":287762808852,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":991232,"data_stored":1040619,"data_compressed":47678,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42199999999999999}]},{"osd":2,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61699999999999999}]},{"osd":3,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64800000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.44800000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56299999999999994}]},{"osd":6,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":8,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.749},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58299999999999996}]},{"osd":9,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63300000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.72799999999999998}]}]},{"osd":9,"up_from":62,"seq":266287972375,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":450143,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.55900000000000005}]},{"osd":3,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.46600000000000003}]},{"osd":4,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.629},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Mar 30 02:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":8,"up_from":55,"seq":236223201307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":445332,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.505},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56200000000000006}]},{"osd":1,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83399999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.621}]},{"osd":2,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.45300000000000001,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.32300000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.66600000000000004}]},{"osd":3,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88200000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89900000000000002,"5min":0.995,"15min":0.995},"last":0.83399999999999996}]},{"osd":5,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.40200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59199999999999997}]},{"osd":6,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.056,"15min":1.056},"last":0.75},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.878,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.443,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.998,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73099999999999998}]},{"osd":9,"last update":"Sat Mar 30 02:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.90000000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.82599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":7,"up_from":48,"seq":206158430238,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":1035602,"data_compressed":47678,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.41999999999999998},"min":{"1min":0.35399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44,"15min":0.435},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.871,"5min":1.034,"15min":1.034},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.51000000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.72799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77400000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.317,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70199999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42299999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.503,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.496,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58599999999999997}]},{"osd":6,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.56000000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.82699999999999996,"5min":0.877,"15min":0.877},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.47899999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88800000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.78600000000000003}]},{"osd":8,"last update":"Sat Mar 30 02:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.753}]},{"osd":9,"last update":"Sat Mar 30 02:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.81100000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.54900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561059,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":913408,"data_stored":445599,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56699999999999995}]},{"osd":1,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.49099999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77400000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72299999999999998,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.95199999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.747,"15min":0.755},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91100000000000003,"5min":1.069,"15min":1.069},"last":0.70699999999999996}]},{"osd":3,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65500000000000003,"15min":0.65900000000000003},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.89500000000000002,"5min":1.236,"15min":1.236},"last":0.68400000000000005}]},{"osd":4,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72599999999999998,"15min":0.72999999999999998},"min":{"1min":0.49099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.054,"5min":1.069,"15min":1.069},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.754,"15min":0.755},"min":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.86099999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.83199999999999996,"15min":0.84399999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97999999999999998,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.87,"15min":0.874},"min":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.262,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.90900000000000003}]},{"osd":7,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":0.84999999999999998},"min":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.875},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.627,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.89100000000000001}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91400000000000003,"15min":0.92200000000000004},"min":{"1min":0.628,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1220000000000001,"5min":1.355,"15min":1.355},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.88600000000000001,"15min":0.89200000000000002},"min":{"1min":0.70099999999999996,"5min":0.623,"15min":0.623},"max":{"1min":1.0609999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Sat Mar 30 02:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.88700000000000001}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":469726,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.66100000000000003,"15min":0.77500000000000002},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81999999999999995,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.378,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70999999999999996}]},{"osd":2,"last update":"Sat Mar 30 02:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.499},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56200000000000006}]},{"osd":3,"last update":"Sat Mar 30 02:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.443},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57499999999999996,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52700000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41399999999999998,"15min":0.39800000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.378},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.41399999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57599999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74099999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Mar 30 02:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89900000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.45400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.95599999999999996,"5min":1.044,"15min":1.044},"last":0.745}]},{"osd":9,"last update":"Sat Mar 30 02:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.73499999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":446403,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 02:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81599999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57299999999999995,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86199999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65800000000000003}]},{"osd":2,"last update":"Sat Mar 30 02:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.45900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.74099999999999999}]},{"osd":3,"last update":"Sat Mar 30 02:33:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54300000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.626,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":1.006,"15min":1.006},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64500000000000002,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65900000000000003,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.85899999999999999}]},{"osd":6,"last update":"Sat Mar 30 02:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92800000000000005,"5min":1.091,"15min":1.091},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93700000000000006,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69399999999999995}]},{"osd":7,"last update":"Sat Mar 30 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.82999999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.90200000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.008},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.92600000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.196}]},{"osd":11,"last update":"Sat Mar 30 02:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.92000000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":448755,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88300000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82199999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.48799999999999999}]},{"osd":1,"last update":"Sat Mar 30 02:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71199999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.39800000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71099999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.77200000000000002}]},{"osd":3,"last update":"Sat Mar 30 02:34:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85499999999999998,"5min":0.878,"15min":0.878},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.61399999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.441,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.65100000000000002}]},{"osd":5,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46100000000000002,"15min":0.45400000000000001},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56100000000000005}]},{"osd":6,"last update":"Sat Mar 30 02:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.874,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68500000000000005}]},{"osd":7,"last update":"Sat Mar 30 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.84699999999999998}]},{"osd":8,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.749}]},{"osd":9,"last update":"Sat Mar 30 02:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.76400000000000001}]},{"osd":10,"last update":"Sat Mar 30 02:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.874}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":913408,"data_stored":445209,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:47 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.42499999999999999,"15min":0.39200000000000002},"min":{"1min":0.35599999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.441,"15min":0.42199999999999999},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.624,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:34:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55000000000000004,"15min":0.53000000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85199999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.46500000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68400000000000005}]},{"osd":2,"last update":"Sat Mar 30 02:34:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.749},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.47299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.66600000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93999999999999995,"5min":0.996,"15min":0.996},"last":0.62}]},{"osd":5,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.55800000000000005,"5min":0.495,"15min":0.495},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.54100000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.86199999999999999}]},{"osd":6,"last update":"Sat Mar 30 02:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.77800000000000002,"15min":0.77200000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1120000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Mar 30 02:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.997,"5min":1.071,"15min":1.071},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.131,"5min":1.25,"15min":1.25},"last":0.89200000000000002}]},{"osd":8,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.004}]},{"osd":9,"last update":"Sat Mar 30 02:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.80200000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.93899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953513,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":901120,"data_stored":444947,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.38,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.747,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.36699999999999999,"15min":0.34699999999999998},"min":{"1min":0.313,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.34999999999999998},"min":{"1min":0.32400000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.748},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53900000000000003}]},{"osd":5,"last update":"Sat Mar 30 02:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.47899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.64100000000000001}]},{"osd":6,"last update":"Sat Mar 30 02:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.78800000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45700000000000002}]},{"osd":7,"last update":"Sat Mar 30 02:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.38900000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.48899999999999999}]},{"osd":8,"last update":"Sat Mar 30 02:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.72499999999999998}]},{"osd":9,"last update":"Sat Mar 30 02:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58199999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.51100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789990,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":974848,"data_stored":1036721,"data_compressed":47678,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.39200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.69399999999999995}]},{"osd":1,"last update":"Sat Mar 30 02:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.41499999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.51300000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.48599999999999999}]},{"osd":2,"last update":"Sat Mar 30 02:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.35599999999999998},"min":{"1min":0.316,"5min":0.25,"15min":0.25},"max":{"1min":0.55100000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.504},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.36299999999999999,"15min":0.35399999999999998},"min":{"1min":0.315,"5min":0.253,"15min":0.253},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.44800000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.41499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75900000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.76100000000000001}]},{"osd":4,"last update":"Sat Mar 30 02:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.57899999999999996,"15min":0.56899999999999995},"min":{"1min":0.441,"5min":0.378,"15min":0.378},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.58399999999999996,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.55200000000000005}]},{"osd":6,"last update":"Sat Mar 30 02:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.623,"15min":0.623},"min":{"1min":0.47199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71899999999999997}]},{"osd":7,"last update":"Sat Mar 30 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.499,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.53900000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.58299999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.621}]},{"osd":9,"last update":"Sat Mar 30 02:34:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.75},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.85899999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.66100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T02:35:22.666 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T02:35:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.899+0000 7fb1f8a00700 1 -- 172.21.15.67:0/831589256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 msgr2=0x7fb1f41022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.899+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/831589256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 0x7fb1f41022d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00099c0 tx=0x7fb1e0009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.899+0000 7fb1f8a00700 1 -- 172.21.15.67:0/831589256 shutdown_connections 2024-03-30T02:35:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.899+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/831589256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f4102810 0x7fb1f41090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.899+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/831589256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f4100c80 0x7fb1f41010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.899+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/831589256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 0x7fb1f41022d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.899+0000 7fb1f8a00700 1 -- 172.21.15.67:0/831589256 >> 172.21.15.67:0/831589256 conn(0x7fb1f40fc1f0 msgr2=0x7fb1f40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.899+0000 7fb1f8a00700 1 -- 172.21.15.67:0/831589256 shutdown_connections 2024-03-30T02:35:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.900+0000 7fb1f8a00700 1 -- 172.21.15.67:0/831589256 wait complete. 2024-03-30T02:35:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.900+0000 7fb1f8a00700 1 Processor -- start 2024-03-30T02:35:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f8a00700 1 -- start start 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f4100c80 0x7fb1f4198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 0x7fb1f4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f8a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f4102810 0x7fb1f41a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f8a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f41a08e0 con 0x7fb1f4101e70 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f8a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f41a0a50 con 0x7fb1f4100c80 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f8a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f41a0c10 con 0x7fb1f4102810 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 0x7fb1f4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 0x7fb1f4199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57592/0 (socket says 172.21.15.67:57592) 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f1400700 1 -- 172.21.15.67:0/2438110040 learned_addr learned my addr 172.21.15.67:0/2438110040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f4100c80 0x7fb1f4198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.901+0000 7fb1f2800700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f4102810 0x7fb1f41a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1f1e00700 1 -- 172.21.15.67:0/2438110040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f4102810 msgr2=0x7fb1f41a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1f1e00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f4102810 0x7fb1f41a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1f1e00700 1 -- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 msgr2=0x7fb1f4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1f1e00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 0x7fb1f4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1f1e00700 1 -- 172.21.15.67:0/2438110040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e00095f0 con 0x7fb1f4100c80 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1f1400700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 0x7fb1f4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1f1e00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f4100c80 0x7fb1f4198df0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400e9c0 tx=0x7fb1e400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1eaa00700 1 -- 172.21.15.67:0/2438110040 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4004220 con 0x7fb1f4100c80 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f41a0ef0 con 0x7fb1f4100c80 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1eaa00700 1 -- 172.21.15.67:0/2438110040 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1e4004380 con 0x7fb1f4100c80 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1eaa00700 1 -- 172.21.15.67:0/2438110040 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4003be0 con 0x7fb1f4100c80 2024-03-30T02:35:22.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.902+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f41a1310 con 0x7fb1f4100c80 2024-03-30T02:35:22.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.903+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f4067a10 con 0x7fb1f4100c80 2024-03-30T02:35:22.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.905+0000 7fb1eaa00700 1 -- 172.21.15.67:0/2438110040 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb1e4003d40 con 0x7fb1f4100c80 2024-03-30T02:35:22.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.907+0000 7fb1eaa00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a11b0 0x7fb1d80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:22.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.907+0000 7fb1eaa00700 1 -- 172.21.15.67:0/2438110040 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb1e4014070 con 0x7fb1f4100c80 2024-03-30T02:35:22.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.907+0000 7fb1f1400700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a11b0 0x7fb1d80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:22.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.908+0000 7fb1f1400700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a11b0 0x7fb1d80a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0004800 tx=0x7fb1e001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:22.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:22.909+0000 7fb1eaa00700 1 -- 172.21.15.67:0/2438110040 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb1e4096470 con 0x7fb1f4100c80 2024-03-30T02:35:23.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.122+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1f4074700 con 0x7fb1d80a11b0 2024-03-30T02:35:23.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.135+0000 7fb1eaa00700 1 -- 172.21.15.67:0/2438110040 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422509 (secure 0 0 0) 0x7fb1f4074700 con 0x7fb1d80a11b0 2024-03-30T02:35:23.141 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a11b0 msgr2=0x7fb1d80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a11b0 0x7fb1d80a3670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0004800 tx=0x7fb1e001a040 comp rx=0 tx=0).stop 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f4100c80 msgr2=0x7fb1f4198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f4100c80 0x7fb1f4198df0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400e9c0 tx=0x7fb1e400ecd0 comp rx=0 tx=0).stop 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 shutdown_connections 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a11b0 0x7fb1d80a3670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f4102810 0x7fb1f41a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f4100c80 0x7fb1f4198df0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 --2- 172.21.15.67:0/2438110040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f4101e70 0x7fb1f4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 >> 172.21.15.67:0/2438110040 conn(0x7fb1f40fc1f0 msgr2=0x7fb1f4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 shutdown_connections 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:23.140+0000 7fb1f8a00700 1 -- 172.21.15.67:0/2438110040 wait complete. 2024-03-30T02:35:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-30T02:35:23.161 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-03-30T02:35:23.006262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600742,"num_objects":92,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":165,"num_read_kb":141,"num_write":374,"num_write_kb":3007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359828,"kb_used_data":11060,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581996,"statfs":{"total":1151940427776,"available":1151571963904,"internally_reserved":0,"allocated":11325440,"data_stored":7217772,"data_compressed":249964,"data_compressed_allocated":335872,"data_compressed_original":2170880,"omap_allocated":19130,"internal_metadata":357020998},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5780,"num_objects":68,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":144,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003494"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076787+0000","last_change":"2024-03-30T02:34:46.755257+0000","last_active":"2024-03-30T02:34:51.076787+0000","last_peered":"2024-03-30T02:34:51.076787+0000","last_clean":"2024-03-30T02:34:51.076787+0000","last_became_active":"2024-03-30T02:34:46.754922+0000","last_became_peered":"2024-03-30T02:34:46.754922+0000","last_unstale":"2024-03-30T02:34:51.076787+0000","last_undegraded":"2024-03-30T02:34:51.076787+0000","last_fullsized":"2024-03-30T02:34:51.076787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:09.980124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.755256+0000","last_change":"2024-03-30T02:34:46.742846+0000","last_active":"2024-03-30T02:34:50.755256+0000","last_peered":"2024-03-30T02:34:50.755256+0000","last_clean":"2024-03-30T02:34:50.755256+0000","last_became_active":"2024-03-30T02:34:46.738096+0000","last_became_peered":"2024-03-30T02:34:46.738096+0000","last_unstale":"2024-03-30T02:34:50.755256+0000","last_undegraded":"2024-03-30T02:34:50.755256+0000","last_fullsized":"2024-03-30T02:34:50.755256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:43:16.982251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.755723+0000","last_change":"2024-03-30T02:34:46.739607+0000","last_active":"2024-03-30T02:34:50.755723+0000","last_peered":"2024-03-30T02:34:50.755723+0000","last_clean":"2024-03-30T02:34:50.755723+0000","last_became_active":"2024-03-30T02:34:46.739230+0000","last_became_peered":"2024-03-30T02:34:46.739230+0000","last_unstale":"2024-03-30T02:34:50.755723+0000","last_undegraded":"2024-03-30T02:34:50.755723+0000","last_fullsized":"2024-03-30T02:34:50.755723+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:17.818874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107220+0000","last_change":"2024-03-30T02:34:46.742118+0000","last_active":"2024-03-30T02:34:51.107220+0000","last_peered":"2024-03-30T02:34:51.107220+0000","last_clean":"2024-03-30T02:34:51.107220+0000","last_became_active":"2024-03-30T02:34:46.740845+0000","last_became_peered":"2024-03-30T02:34:46.740845+0000","last_unstale":"2024-03-30T02:34:51.107220+0000","last_undegraded":"2024-03-30T02:34:51.107220+0000","last_fullsized":"2024-03-30T02:34:51.107220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:33:28.595456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.744963+0000","last_change":"2024-03-30T02:34:46.739557+0000","last_active":"2024-03-30T02:34:50.744963+0000","last_peered":"2024-03-30T02:34:50.744963+0000","last_clean":"2024-03-30T02:34:50.744963+0000","last_became_active":"2024-03-30T02:34:46.739147+0000","last_became_peered":"2024-03-30T02:34:46.739147+0000","last_unstale":"2024-03-30T02:34:50.744963+0000","last_undegraded":"2024-03-30T02:34:50.744963+0000","last_fullsized":"2024-03-30T02:34:50.744963+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:12:52.307386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.796263+0000","last_change":"2024-03-30T02:34:46.735157+0000","last_active":"2024-03-30T02:34:50.796263+0000","last_peered":"2024-03-30T02:34:50.796263+0000","last_clean":"2024-03-30T02:34:50.796263+0000","last_became_active":"2024-03-30T02:34:46.734725+0000","last_became_peered":"2024-03-30T02:34:46.734725+0000","last_unstale":"2024-03-30T02:34:50.796263+0000","last_undegraded":"2024-03-30T02:34:50.796263+0000","last_fullsized":"2024-03-30T02:34:50.796263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:00.823203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117652+0000","last_change":"2024-03-30T02:34:46.735466+0000","last_active":"2024-03-30T02:34:51.117652+0000","last_peered":"2024-03-30T02:34:51.117652+0000","last_clean":"2024-03-30T02:34:51.117652+0000","last_became_active":"2024-03-30T02:34:46.734411+0000","last_became_peered":"2024-03-30T02:34:46.734411+0000","last_unstale":"2024-03-30T02:34:51.117652+0000","last_undegraded":"2024-03-30T02:34:51.117652+0000","last_fullsized":"2024-03-30T02:34:51.117652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:57.587768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.749906+0000","last_change":"2024-03-30T02:34:46.747001+0000","last_active":"2024-03-30T02:34:50.749906+0000","last_peered":"2024-03-30T02:34:50.749906+0000","last_clean":"2024-03-30T02:34:50.749906+0000","last_became_active":"2024-03-30T02:34:46.740241+0000","last_became_peered":"2024-03-30T02:34:46.740241+0000","last_unstale":"2024-03-30T02:34:50.749906+0000","last_undegraded":"2024-03-30T02:34:50.749906+0000","last_fullsized":"2024-03-30T02:34:50.749906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:05.902190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743753+0000","last_change":"2024-03-30T02:34:46.745549+0000","last_active":"2024-03-30T02:34:50.743753+0000","last_peered":"2024-03-30T02:34:50.743753+0000","last_clean":"2024-03-30T02:34:50.743753+0000","last_became_active":"2024-03-30T02:34:46.745200+0000","last_became_peered":"2024-03-30T02:34:46.745200+0000","last_unstale":"2024-03-30T02:34:50.743753+0000","last_undegraded":"2024-03-30T02:34:50.743753+0000","last_fullsized":"2024-03-30T02:34:50.743753+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:37.931440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.796486+0000","last_change":"2024-03-30T02:34:46.735859+0000","last_active":"2024-03-30T02:34:50.796486+0000","last_peered":"2024-03-30T02:34:50.796486+0000","last_clean":"2024-03-30T02:34:50.796486+0000","last_became_active":"2024-03-30T02:34:46.735481+0000","last_became_peered":"2024-03-30T02:34:46.735481+0000","last_unstale":"2024-03-30T02:34:50.796486+0000","last_undegraded":"2024-03-30T02:34:50.796486+0000","last_fullsized":"2024-03-30T02:34:50.796486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:51.037887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117750+0000","last_change":"2024-03-30T02:34:46.741412+0000","last_active":"2024-03-30T02:34:51.117750+0000","last_peered":"2024-03-30T02:34:51.117750+0000","last_clean":"2024-03-30T02:34:51.117750+0000","last_became_active":"2024-03-30T02:34:46.741106+0000","last_became_peered":"2024-03-30T02:34:46.741106+0000","last_unstale":"2024-03-30T02:34:51.117750+0000","last_undegraded":"2024-03-30T02:34:51.117750+0000","last_fullsized":"2024-03-30T02:34:51.117750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:06:08.553028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107207+0000","last_change":"2024-03-30T02:34:46.746205+0000","last_active":"2024-03-30T02:34:51.107207+0000","last_peered":"2024-03-30T02:34:51.107207+0000","last_clean":"2024-03-30T02:34:51.107207+0000","last_became_active":"2024-03-30T02:34:46.745803+0000","last_became_peered":"2024-03-30T02:34:46.745803+0000","last_unstale":"2024-03-30T02:34:51.107207+0000","last_undegraded":"2024-03-30T02:34:51.107207+0000","last_fullsized":"2024-03-30T02:34:51.107207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:05.127095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747388+0000","last_change":"2024-03-30T02:34:46.722978+0000","last_active":"2024-03-30T02:34:50.747388+0000","last_peered":"2024-03-30T02:34:50.747388+0000","last_clean":"2024-03-30T02:34:50.747388+0000","last_became_active":"2024-03-30T02:34:46.722652+0000","last_became_peered":"2024-03-30T02:34:46.722652+0000","last_unstale":"2024-03-30T02:34:50.747388+0000","last_undegraded":"2024-03-30T02:34:50.747388+0000","last_fullsized":"2024-03-30T02:34:50.747388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:28:12.078216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.752810+0000","last_change":"2024-03-30T02:34:46.738895+0000","last_active":"2024-03-30T02:34:50.752810+0000","last_peered":"2024-03-30T02:34:50.752810+0000","last_clean":"2024-03-30T02:34:50.752810+0000","last_became_active":"2024-03-30T02:34:46.738510+0000","last_became_peered":"2024-03-30T02:34:46.738510+0000","last_unstale":"2024-03-30T02:34:50.752810+0000","last_undegraded":"2024-03-30T02:34:50.752810+0000","last_fullsized":"2024-03-30T02:34:50.752810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:09.859259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.751025+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:34:50.751025+0000","last_peered":"2024-03-30T02:34:50.751025+0000","last_clean":"2024-03-30T02:34:50.751025+0000","last_became_active":"2024-03-30T02:34:46.750301+0000","last_became_peered":"2024-03-30T02:34:46.750301+0000","last_unstale":"2024-03-30T02:34:50.751025+0000","last_undegraded":"2024-03-30T02:34:50.751025+0000","last_fullsized":"2024-03-30T02:34:50.751025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:06.475881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743238+0000","last_change":"2024-03-30T02:34:46.748966+0000","last_active":"2024-03-30T02:34:50.743238+0000","last_peered":"2024-03-30T02:34:50.743238+0000","last_clean":"2024-03-30T02:34:50.743238+0000","last_became_active":"2024-03-30T02:34:46.748650+0000","last_became_peered":"2024-03-30T02:34:46.748650+0000","last_unstale":"2024-03-30T02:34:50.743238+0000","last_undegraded":"2024-03-30T02:34:50.743238+0000","last_fullsized":"2024-03-30T02:34:50.743238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:36.558864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076777+0000","last_change":"2024-03-30T02:34:46.746127+0000","last_active":"2024-03-30T02:34:51.076777+0000","last_peered":"2024-03-30T02:34:51.076777+0000","last_clean":"2024-03-30T02:34:51.076777+0000","last_became_active":"2024-03-30T02:34:46.745806+0000","last_became_peered":"2024-03-30T02:34:46.745806+0000","last_unstale":"2024-03-30T02:34:51.076777+0000","last_undegraded":"2024-03-30T02:34:51.076777+0000","last_fullsized":"2024-03-30T02:34:51.076777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:54.097996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.740772+0000","last_change":"2024-03-30T02:34:46.738391+0000","last_active":"2024-03-30T02:34:50.740772+0000","last_peered":"2024-03-30T02:34:50.740772+0000","last_clean":"2024-03-30T02:34:50.740772+0000","last_became_active":"2024-03-30T02:34:46.737110+0000","last_became_peered":"2024-03-30T02:34:46.737110+0000","last_unstale":"2024-03-30T02:34:50.740772+0000","last_undegraded":"2024-03-30T02:34:50.740772+0000","last_fullsized":"2024-03-30T02:34:50.740772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:20.326107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.752034+0000","last_change":"2024-03-30T02:34:46.744522+0000","last_active":"2024-03-30T02:34:50.752034+0000","last_peered":"2024-03-30T02:34:50.752034+0000","last_clean":"2024-03-30T02:34:50.752034+0000","last_became_active":"2024-03-30T02:34:46.744088+0000","last_became_peered":"2024-03-30T02:34:46.744088+0000","last_unstale":"2024-03-30T02:34:50.752034+0000","last_undegraded":"2024-03-30T02:34:50.752034+0000","last_fullsized":"2024-03-30T02:34:50.752034+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:35.698979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.272847+0000","last_change":"2024-03-30T02:34:50.770855+0000","last_active":"2024-03-30T02:34:51.272847+0000","last_peered":"2024-03-30T02:34:51.272847+0000","last_clean":"2024-03-30T02:34:51.272847+0000","last_became_active":"2024-03-30T02:34:50.770420+0000","last_became_peered":"2024-03-30T02:34:50.770420+0000","last_unstale":"2024-03-30T02:34:51.272847+0000","last_undegraded":"2024-03-30T02:34:51.272847+0000","last_fullsized":"2024-03-30T02:34:51.272847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:00:08.847059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:20.128766+0000","last_change":"2024-03-30T02:34:46.207252+0000","last_active":"2024-03-30T02:35:20.128766+0000","last_peered":"2024-03-30T02:35:20.128766+0000","last_clean":"2024-03-30T02:35:20.128766+0000","last_became_active":"2024-03-30T02:34:46.206916+0000","last_became_peered":"2024-03-30T02:34:46.206916+0000","last_unstale":"2024-03-30T02:35:20.128766+0000","last_undegraded":"2024-03-30T02:35:20.128766+0000","last_fullsized":"2024-03-30T02:35:20.128766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:03:50.192245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.754252+0000","last_change":"2024-03-30T02:34:46.745778+0000","last_active":"2024-03-30T02:34:50.754252+0000","last_peered":"2024-03-30T02:34:50.754252+0000","last_clean":"2024-03-30T02:34:50.754252+0000","last_became_active":"2024-03-30T02:34:46.745471+0000","last_became_peered":"2024-03-30T02:34:46.745471+0000","last_unstale":"2024-03-30T02:34:50.754252+0000","last_undegraded":"2024-03-30T02:34:50.754252+0000","last_fullsized":"2024-03-30T02:34:50.754252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:57.314163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107117+0000","last_change":"2024-03-30T02:34:50.769577+0000","last_active":"2024-03-30T02:34:51.107117+0000","last_peered":"2024-03-30T02:34:51.107117+0000","last_clean":"2024-03-30T02:34:51.107117+0000","last_became_active":"2024-03-30T02:34:50.769100+0000","last_became_peered":"2024-03-30T02:34:50.769100+0000","last_unstale":"2024-03-30T02:34:51.107117+0000","last_undegraded":"2024-03-30T02:34:51.107117+0000","last_fullsized":"2024-03-30T02:34:51.107117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:08:43.495671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.274453+0000","last_change":"2024-03-30T02:34:46.218032+0000","last_active":"2024-03-30T02:34:51.274453+0000","last_peered":"2024-03-30T02:34:51.274453+0000","last_clean":"2024-03-30T02:34:51.274453+0000","last_became_active":"2024-03-30T02:34:46.217733+0000","last_became_peered":"2024-03-30T02:34:46.217733+0000","last_unstale":"2024-03-30T02:34:51.274453+0000","last_undegraded":"2024-03-30T02:34:51.274453+0000","last_fullsized":"2024-03-30T02:34:51.274453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:58.413590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.759280+0000","last_change":"2024-03-30T02:34:46.750588+0000","last_active":"2024-03-30T02:34:50.759280+0000","last_peered":"2024-03-30T02:34:50.759280+0000","last_clean":"2024-03-30T02:34:50.759280+0000","last_became_active":"2024-03-30T02:34:46.750016+0000","last_became_peered":"2024-03-30T02:34:46.750016+0000","last_unstale":"2024-03-30T02:34:50.759280+0000","last_undegraded":"2024-03-30T02:34:50.759280+0000","last_fullsized":"2024-03-30T02:34:50.759280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:19.144544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107167+0000","last_change":"2024-03-30T02:34:50.760473+0000","last_active":"2024-03-30T02:34:51.107167+0000","last_peered":"2024-03-30T02:34:51.107167+0000","last_clean":"2024-03-30T02:34:51.107167+0000","last_became_active":"2024-03-30T02:34:50.760068+0000","last_became_peered":"2024-03-30T02:34:50.760068+0000","last_unstale":"2024-03-30T02:34:51.107167+0000","last_undegraded":"2024-03-30T02:34:51.107167+0000","last_fullsized":"2024-03-30T02:34:51.107167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:49:12.902937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.113595+0000","last_change":"2024-03-30T02:34:46.213741+0000","last_active":"2024-03-30T02:35:18.113595+0000","last_peered":"2024-03-30T02:35:18.113595+0000","last_clean":"2024-03-30T02:35:18.113595+0000","last_became_active":"2024-03-30T02:34:46.213395+0000","last_became_peered":"2024-03-30T02:34:46.213395+0000","last_unstale":"2024-03-30T02:35:18.113595+0000","last_undegraded":"2024-03-30T02:35:18.113595+0000","last_fullsized":"2024-03-30T02:35:18.113595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:13.192415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743330+0000","last_change":"2024-03-30T02:34:46.738401+0000","last_active":"2024-03-30T02:34:50.743330+0000","last_peered":"2024-03-30T02:34:50.743330+0000","last_clean":"2024-03-30T02:34:50.743330+0000","last_became_active":"2024-03-30T02:34:46.737804+0000","last_became_peered":"2024-03-30T02:34:46.737804+0000","last_unstale":"2024-03-30T02:34:50.743330+0000","last_undegraded":"2024-03-30T02:34:50.743330+0000","last_fullsized":"2024-03-30T02:34:50.743330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:56:08.194439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107154+0000","last_change":"2024-03-30T02:34:50.766222+0000","last_active":"2024-03-30T02:34:51.107154+0000","last_peered":"2024-03-30T02:34:51.107154+0000","last_clean":"2024-03-30T02:34:51.107154+0000","last_became_active":"2024-03-30T02:34:50.765836+0000","last_became_peered":"2024-03-30T02:34:50.765836+0000","last_unstale":"2024-03-30T02:34:51.107154+0000","last_undegraded":"2024-03-30T02:34:51.107154+0000","last_fullsized":"2024-03-30T02:34:51.107154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:32.529535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.744949+0000","last_change":"2024-03-30T02:34:46.212586+0000","last_active":"2024-03-30T02:34:50.744949+0000","last_peered":"2024-03-30T02:34:50.744949+0000","last_clean":"2024-03-30T02:34:50.744949+0000","last_became_active":"2024-03-30T02:34:46.212110+0000","last_became_peered":"2024-03-30T02:34:46.212110+0000","last_unstale":"2024-03-30T02:34:50.744949+0000","last_undegraded":"2024-03-30T02:34:50.744949+0000","last_fullsized":"2024-03-30T02:34:50.744949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:38.630953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748588+0000","last_change":"2024-03-30T02:34:46.734923+0000","last_active":"2024-03-30T02:34:50.748588+0000","last_peered":"2024-03-30T02:34:50.748588+0000","last_clean":"2024-03-30T02:34:50.748588+0000","last_became_active":"2024-03-30T02:34:46.734572+0000","last_became_peered":"2024-03-30T02:34:46.734572+0000","last_unstale":"2024-03-30T02:34:50.748588+0000","last_undegraded":"2024-03-30T02:34:50.748588+0000","last_fullsized":"2024-03-30T02:34:50.748588+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:28.647788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.273238+0000","last_change":"2024-03-30T02:34:50.757803+0000","last_active":"2024-03-30T02:34:51.273238+0000","last_peered":"2024-03-30T02:34:51.273238+0000","last_clean":"2024-03-30T02:34:51.273238+0000","last_became_active":"2024-03-30T02:34:50.757198+0000","last_became_peered":"2024-03-30T02:34:50.757198+0000","last_unstale":"2024-03-30T02:34:51.273238+0000","last_undegraded":"2024-03-30T02:34:51.273238+0000","last_fullsized":"2024-03-30T02:34:51.273238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:54:49.399741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:21.134416+0000","last_change":"2024-03-30T02:34:46.214549+0000","last_active":"2024-03-30T02:35:21.134416+0000","last_peered":"2024-03-30T02:35:21.134416+0000","last_clean":"2024-03-30T02:35:21.134416+0000","last_became_active":"2024-03-30T02:34:46.214198+0000","last_became_peered":"2024-03-30T02:34:46.214198+0000","last_unstale":"2024-03-30T02:35:21.134416+0000","last_undegraded":"2024-03-30T02:35:21.134416+0000","last_fullsized":"2024-03-30T02:35:21.134416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:45:25.618982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.746267+0000","last_change":"2024-03-30T02:34:46.737771+0000","last_active":"2024-03-30T02:34:50.746267+0000","last_peered":"2024-03-30T02:34:50.746267+0000","last_clean":"2024-03-30T02:34:50.746267+0000","last_became_active":"2024-03-30T02:34:46.737438+0000","last_became_peered":"2024-03-30T02:34:46.737438+0000","last_unstale":"2024-03-30T02:34:50.746267+0000","last_undegraded":"2024-03-30T02:34:50.746267+0000","last_fullsized":"2024-03-30T02:34:50.746267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:08.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.272971+0000","last_change":"2024-03-30T02:34:50.767304+0000","last_active":"2024-03-30T02:34:51.272971+0000","last_peered":"2024-03-30T02:34:51.272971+0000","last_clean":"2024-03-30T02:34:51.272971+0000","last_became_active":"2024-03-30T02:34:50.766909+0000","last_became_peered":"2024-03-30T02:34:50.766909+0000","last_unstale":"2024-03-30T02:34:51.272971+0000","last_undegraded":"2024-03-30T02:34:51.272971+0000","last_fullsized":"2024-03-30T02:34:51.272971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:24.021991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.114383+0000","last_change":"2024-03-30T02:34:46.214937+0000","last_active":"2024-03-30T02:35:18.114383+0000","last_peered":"2024-03-30T02:35:18.114383+0000","last_clean":"2024-03-30T02:35:18.114383+0000","last_became_active":"2024-03-30T02:34:46.214503+0000","last_became_peered":"2024-03-30T02:34:46.214503+0000","last_unstale":"2024-03-30T02:35:18.114383+0000","last_undegraded":"2024-03-30T02:35:18.114383+0000","last_fullsized":"2024-03-30T02:35:18.114383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:20:11.624919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747888+0000","last_change":"2024-03-30T02:34:46.737374+0000","last_active":"2024-03-30T02:34:50.747888+0000","last_peered":"2024-03-30T02:34:50.747888+0000","last_clean":"2024-03-30T02:34:50.747888+0000","last_became_active":"2024-03-30T02:34:46.735307+0000","last_became_peered":"2024-03-30T02:34:46.735307+0000","last_unstale":"2024-03-30T02:34:50.747888+0000","last_undegraded":"2024-03-30T02:34:50.747888+0000","last_fullsized":"2024-03-30T02:34:50.747888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:30.482728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076765+0000","last_change":"2024-03-30T02:34:50.768016+0000","last_active":"2024-03-30T02:34:51.076765+0000","last_peered":"2024-03-30T02:34:51.076765+0000","last_clean":"2024-03-30T02:34:51.076765+0000","last_became_active":"2024-03-30T02:34:50.767564+0000","last_became_peered":"2024-03-30T02:34:50.767564+0000","last_unstale":"2024-03-30T02:34:51.076765+0000","last_undegraded":"2024-03-30T02:34:51.076765+0000","last_fullsized":"2024-03-30T02:34:51.076765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:16.888754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:17.111130+0000","last_change":"2024-03-30T02:34:46.212982+0000","last_active":"2024-03-30T02:35:17.111130+0000","last_peered":"2024-03-30T02:35:17.111130+0000","last_clean":"2024-03-30T02:35:17.111130+0000","last_became_active":"2024-03-30T02:34:46.212550+0000","last_became_peered":"2024-03-30T02:34:46.212550+0000","last_unstale":"2024-03-30T02:35:17.111130+0000","last_undegraded":"2024-03-30T02:35:17.111130+0000","last_fullsized":"2024-03-30T02:35:17.111130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:44:50.156615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.749854+0000","last_change":"2024-03-30T02:34:46.736394+0000","last_active":"2024-03-30T02:34:50.749854+0000","last_peered":"2024-03-30T02:34:50.749854+0000","last_clean":"2024-03-30T02:34:50.749854+0000","last_became_active":"2024-03-30T02:34:46.735018+0000","last_became_peered":"2024-03-30T02:34:46.735018+0000","last_unstale":"2024-03-30T02:34:50.749854+0000","last_undegraded":"2024-03-30T02:34:50.749854+0000","last_fullsized":"2024-03-30T02:34:50.749854+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:32:15.769536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.651902+0000","last_change":"2024-03-30T02:34:50.773036+0000","last_active":"2024-03-30T02:34:51.651902+0000","last_peered":"2024-03-30T02:34:51.651902+0000","last_clean":"2024-03-30T02:34:51.651902+0000","last_became_active":"2024-03-30T02:34:50.772614+0000","last_became_peered":"2024-03-30T02:34:50.772614+0000","last_unstale":"2024-03-30T02:34:51.651902+0000","last_undegraded":"2024-03-30T02:34:51.651902+0000","last_fullsized":"2024-03-30T02:34:51.651902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:54:28.792793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:19.124743+0000","last_change":"2024-03-30T02:34:46.209802+0000","last_active":"2024-03-30T02:35:19.124743+0000","last_peered":"2024-03-30T02:35:19.124743+0000","last_clean":"2024-03-30T02:35:19.124743+0000","last_became_active":"2024-03-30T02:34:46.209246+0000","last_became_peered":"2024-03-30T02:34:46.209246+0000","last_unstale":"2024-03-30T02:35:19.124743+0000","last_undegraded":"2024-03-30T02:35:19.124743+0000","last_fullsized":"2024-03-30T02:35:19.124743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:09:57.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.740493+0000","last_change":"2024-03-30T02:34:46.743638+0000","last_active":"2024-03-30T02:34:50.740493+0000","last_peered":"2024-03-30T02:34:50.740493+0000","last_clean":"2024-03-30T02:34:50.740493+0000","last_became_active":"2024-03-30T02:34:46.743246+0000","last_became_peered":"2024-03-30T02:34:46.743246+0000","last_unstale":"2024-03-30T02:34:50.740493+0000","last_undegraded":"2024-03-30T02:34:50.740493+0000","last_fullsized":"2024-03-30T02:34:50.740493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:49:55.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.273748+0000","last_change":"2024-03-30T02:34:50.768586+0000","last_active":"2024-03-30T02:34:51.273748+0000","last_peered":"2024-03-30T02:34:51.273748+0000","last_clean":"2024-03-30T02:34:51.273748+0000","last_became_active":"2024-03-30T02:34:50.766894+0000","last_became_peered":"2024-03-30T02:34:50.766894+0000","last_unstale":"2024-03-30T02:34:51.273748+0000","last_undegraded":"2024-03-30T02:34:51.273748+0000","last_fullsized":"2024-03-30T02:34:51.273748+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:36:39.329608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.103474+0000","last_change":"2024-03-30T02:34:46.204994+0000","last_active":"2024-03-30T02:35:16.103474+0000","last_peered":"2024-03-30T02:35:16.103474+0000","last_clean":"2024-03-30T02:35:16.103474+0000","last_became_active":"2024-03-30T02:34:46.204710+0000","last_became_peered":"2024-03-30T02:34:46.204710+0000","last_unstale":"2024-03-30T02:35:16.103474+0000","last_undegraded":"2024-03-30T02:35:16.103474+0000","last_fullsized":"2024-03-30T02:35:16.103474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:39.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743575+0000","last_change":"2024-03-30T02:34:46.733573+0000","last_active":"2024-03-30T02:34:50.743575+0000","last_peered":"2024-03-30T02:34:50.743575+0000","last_clean":"2024-03-30T02:34:50.743575+0000","last_became_active":"2024-03-30T02:34:46.733241+0000","last_became_peered":"2024-03-30T02:34:46.733241+0000","last_unstale":"2024-03-30T02:34:50.743575+0000","last_undegraded":"2024-03-30T02:34:50.743575+0000","last_fullsized":"2024-03-30T02:34:50.743575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:07:10.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.764235+0000","last_change":"2024-03-30T02:34:50.761422+0000","last_active":"2024-03-30T02:34:50.764235+0000","last_peered":"2024-03-30T02:34:50.764235+0000","last_clean":"2024-03-30T02:34:50.764235+0000","last_became_active":"2024-03-30T02:34:50.760976+0000","last_became_peered":"2024-03-30T02:34:50.760976+0000","last_unstale":"2024-03-30T02:34:50.764235+0000","last_undegraded":"2024-03-30T02:34:50.764235+0000","last_fullsized":"2024-03-30T02:34:50.764235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:59:48.745844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:19.123513+0000","last_change":"2024-03-30T02:34:46.212732+0000","last_active":"2024-03-30T02:35:19.123513+0000","last_peered":"2024-03-30T02:35:19.123513+0000","last_clean":"2024-03-30T02:35:19.123513+0000","last_became_active":"2024-03-30T02:34:46.212306+0000","last_became_peered":"2024-03-30T02:34:46.212306+0000","last_unstale":"2024-03-30T02:35:19.123513+0000","last_undegraded":"2024-03-30T02:35:19.123513+0000","last_fullsized":"2024-03-30T02:35:19.123513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:27:15.766203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.759376+0000","last_change":"2024-03-30T02:34:46.749420+0000","last_active":"2024-03-30T02:34:50.759376+0000","last_peered":"2024-03-30T02:34:50.759376+0000","last_clean":"2024-03-30T02:34:50.759376+0000","last_became_active":"2024-03-30T02:34:46.745286+0000","last_became_peered":"2024-03-30T02:34:46.745286+0000","last_unstale":"2024-03-30T02:34:50.759376+0000","last_undegraded":"2024-03-30T02:34:50.759376+0000","last_fullsized":"2024-03-30T02:34:50.759376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:01.489862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107185+0000","last_change":"2024-03-30T02:34:50.760542+0000","last_active":"2024-03-30T02:34:51.107185+0000","last_peered":"2024-03-30T02:34:51.107185+0000","last_clean":"2024-03-30T02:34:51.107185+0000","last_became_active":"2024-03-30T02:34:50.760087+0000","last_became_peered":"2024-03-30T02:34:50.760087+0000","last_unstale":"2024-03-30T02:34:51.107185+0000","last_undegraded":"2024-03-30T02:34:51.107185+0000","last_fullsized":"2024-03-30T02:34:51.107185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:46.352189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.115590+0000","last_change":"2024-03-30T02:34:46.224058+0000","last_active":"2024-03-30T02:35:18.115590+0000","last_peered":"2024-03-30T02:35:18.115590+0000","last_clean":"2024-03-30T02:35:18.115590+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:35:18.115590+0000","last_undegraded":"2024-03-30T02:35:18.115590+0000","last_fullsized":"2024-03-30T02:35:18.115590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:31:10.124533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743426+0000","last_change":"2024-03-30T02:34:46.741413+0000","last_active":"2024-03-30T02:34:50.743426+0000","last_peered":"2024-03-30T02:34:50.743426+0000","last_clean":"2024-03-30T02:34:50.743426+0000","last_became_active":"2024-03-30T02:34:46.741112+0000","last_became_peered":"2024-03-30T02:34:46.741112+0000","last_unstale":"2024-03-30T02:34:50.743426+0000","last_undegraded":"2024-03-30T02:34:50.743426+0000","last_fullsized":"2024-03-30T02:34:50.743426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:34.820166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.758467+0000","last_change":"2024-03-30T02:34:50.752315+0000","last_active":"2024-03-30T02:34:50.758467+0000","last_peered":"2024-03-30T02:34:50.758467+0000","last_clean":"2024-03-30T02:34:50.758467+0000","last_became_active":"2024-03-30T02:34:50.750985+0000","last_became_peered":"2024-03-30T02:34:50.750985+0000","last_unstale":"2024-03-30T02:34:50.758467+0000","last_undegraded":"2024-03-30T02:34:50.758467+0000","last_fullsized":"2024-03-30T02:34:50.758467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:37.271063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"83'12","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.101908+0000","last_change":"2024-03-30T02:34:46.214759+0000","last_active":"2024-03-30T02:35:16.101908+0000","last_peered":"2024-03-30T02:35:16.101908+0000","last_clean":"2024-03-30T02:35:16.101908+0000","last_became_active":"2024-03-30T02:34:46.214388+0000","last_became_peered":"2024-03-30T02:34:46.214388+0000","last_unstale":"2024-03-30T02:35:16.101908+0000","last_undegraded":"2024-03-30T02:35:16.101908+0000","last_fullsized":"2024-03-30T02:35:16.101908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:41.100461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.745434+0000","last_change":"2024-03-30T02:34:46.741112+0000","last_active":"2024-03-30T02:34:50.745434+0000","last_peered":"2024-03-30T02:34:50.745434+0000","last_clean":"2024-03-30T02:34:50.745434+0000","last_became_active":"2024-03-30T02:34:46.739664+0000","last_became_peered":"2024-03-30T02:34:46.739664+0000","last_unstale":"2024-03-30T02:34:50.745434+0000","last_undegraded":"2024-03-30T02:34:50.745434+0000","last_fullsized":"2024-03-30T02:34:50.745434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:07.763942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.653103+0000","last_change":"2024-03-30T02:34:50.765645+0000","last_active":"2024-03-30T02:34:51.653103+0000","last_peered":"2024-03-30T02:34:51.653103+0000","last_clean":"2024-03-30T02:34:51.653103+0000","last_became_active":"2024-03-30T02:34:50.765345+0000","last_became_peered":"2024-03-30T02:34:50.765345+0000","last_unstale":"2024-03-30T02:34:51.653103+0000","last_undegraded":"2024-03-30T02:34:51.653103+0000","last_fullsized":"2024-03-30T02:34:51.653103+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:25:38.591525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:20.128025+0000","last_change":"2024-03-30T02:34:46.216643+0000","last_active":"2024-03-30T02:35:20.128025+0000","last_peered":"2024-03-30T02:35:20.128025+0000","last_clean":"2024-03-30T02:35:20.128025+0000","last_became_active":"2024-03-30T02:34:46.216188+0000","last_became_peered":"2024-03-30T02:34:46.216188+0000","last_unstale":"2024-03-30T02:35:20.128025+0000","last_undegraded":"2024-03-30T02:35:20.128025+0000","last_fullsized":"2024-03-30T02:35:20.128025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:49:10.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747685+0000","last_change":"2024-03-30T02:34:46.748313+0000","last_active":"2024-03-30T02:34:50.747685+0000","last_peered":"2024-03-30T02:34:50.747685+0000","last_clean":"2024-03-30T02:34:50.747685+0000","last_became_active":"2024-03-30T02:34:46.742897+0000","last_became_peered":"2024-03-30T02:34:46.742897+0000","last_unstale":"2024-03-30T02:34:50.747685+0000","last_undegraded":"2024-03-30T02:34:50.747685+0000","last_fullsized":"2024-03-30T02:34:50.747685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:00.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076711+0000","last_change":"2024-03-30T02:34:50.760993+0000","last_active":"2024-03-30T02:34:51.076711+0000","last_peered":"2024-03-30T02:34:51.076711+0000","last_clean":"2024-03-30T02:34:51.076711+0000","last_became_active":"2024-03-30T02:34:50.760535+0000","last_became_peered":"2024-03-30T02:34:50.760535+0000","last_unstale":"2024-03-30T02:34:51.076711+0000","last_undegraded":"2024-03-30T02:34:51.076711+0000","last_fullsized":"2024-03-30T02:34:51.076711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:15.569533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:19.124610+0000","last_change":"2024-03-30T02:34:46.216603+0000","last_active":"2024-03-30T02:35:19.124610+0000","last_peered":"2024-03-30T02:35:19.124610+0000","last_clean":"2024-03-30T02:35:19.124610+0000","last_became_active":"2024-03-30T02:34:46.216231+0000","last_became_peered":"2024-03-30T02:34:46.216231+0000","last_unstale":"2024-03-30T02:35:19.124610+0000","last_undegraded":"2024-03-30T02:35:19.124610+0000","last_fullsized":"2024-03-30T02:35:19.124610+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:51:00.659652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117710+0000","last_change":"2024-03-30T02:34:46.742815+0000","last_active":"2024-03-30T02:34:51.117710+0000","last_peered":"2024-03-30T02:34:51.117710+0000","last_clean":"2024-03-30T02:34:51.117710+0000","last_became_active":"2024-03-30T02:34:46.742487+0000","last_became_peered":"2024-03-30T02:34:46.742487+0000","last_unstale":"2024-03-30T02:34:51.117710+0000","last_undegraded":"2024-03-30T02:34:51.117710+0000","last_fullsized":"2024-03-30T02:34:51.117710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:07.182990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.772972+0000","last_change":"2024-03-30T02:34:50.770398+0000","last_active":"2024-03-30T02:34:50.772972+0000","last_peered":"2024-03-30T02:34:50.772972+0000","last_clean":"2024-03-30T02:34:50.772972+0000","last_became_active":"2024-03-30T02:34:50.770000+0000","last_became_peered":"2024-03-30T02:34:50.770000+0000","last_unstale":"2024-03-30T02:34:50.772972+0000","last_undegraded":"2024-03-30T02:34:50.772972+0000","last_fullsized":"2024-03-30T02:34:50.772972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:14:49.057081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:20.131989+0000","last_change":"2024-03-30T02:34:46.209403+0000","last_active":"2024-03-30T02:35:20.131989+0000","last_peered":"2024-03-30T02:35:20.131989+0000","last_clean":"2024-03-30T02:35:20.131989+0000","last_became_active":"2024-03-30T02:34:46.209065+0000","last_became_peered":"2024-03-30T02:34:46.209065+0000","last_unstale":"2024-03-30T02:35:20.131989+0000","last_undegraded":"2024-03-30T02:35:20.131989+0000","last_fullsized":"2024-03-30T02:35:20.131989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:36:20.427720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117638+0000","last_change":"2024-03-30T02:34:46.734861+0000","last_active":"2024-03-30T02:34:51.117638+0000","last_peered":"2024-03-30T02:34:51.117638+0000","last_clean":"2024-03-30T02:34:51.117638+0000","last_became_active":"2024-03-30T02:34:46.734376+0000","last_became_peered":"2024-03-30T02:34:46.734376+0000","last_unstale":"2024-03-30T02:34:51.117638+0000","last_undegraded":"2024-03-30T02:34:51.117638+0000","last_fullsized":"2024-03-30T02:34:51.117638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:08:36.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.770978+0000","last_change":"2024-03-30T02:34:50.767469+0000","last_active":"2024-03-30T02:34:50.770978+0000","last_peered":"2024-03-30T02:34:50.770978+0000","last_clean":"2024-03-30T02:34:50.770978+0000","last_became_active":"2024-03-30T02:34:50.765460+0000","last_became_peered":"2024-03-30T02:34:50.765460+0000","last_unstale":"2024-03-30T02:34:50.770978+0000","last_undegraded":"2024-03-30T02:34:50.770978+0000","last_fullsized":"2024-03-30T02:34:50.770978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:54:06.940171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'16","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:17.111894+0000","last_change":"2024-03-30T02:34:46.224053+0000","last_active":"2024-03-30T02:35:17.111894+0000","last_peered":"2024-03-30T02:35:17.111894+0000","last_clean":"2024-03-30T02:35:17.111894+0000","last_became_active":"2024-03-30T02:34:46.223666+0000","last_became_peered":"2024-03-30T02:34:46.223666+0000","last_unstale":"2024-03-30T02:35:17.111894+0000","last_undegraded":"2024-03-30T02:35:17.111894+0000","last_fullsized":"2024-03-30T02:35:17.111894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:18:22.976298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748472+0000","last_change":"2024-03-30T02:34:46.734838+0000","last_active":"2024-03-30T02:34:50.748472+0000","last_peered":"2024-03-30T02:34:50.748472+0000","last_clean":"2024-03-30T02:34:50.748472+0000","last_became_active":"2024-03-30T02:34:46.734428+0000","last_became_peered":"2024-03-30T02:34:46.734428+0000","last_unstale":"2024-03-30T02:34:50.748472+0000","last_undegraded":"2024-03-30T02:34:50.748472+0000","last_fullsized":"2024-03-30T02:34:50.748472+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:42:51.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.652925+0000","last_change":"2024-03-30T02:34:50.771150+0000","last_active":"2024-03-30T02:34:51.652925+0000","last_peered":"2024-03-30T02:34:51.652925+0000","last_clean":"2024-03-30T02:34:51.652925+0000","last_became_active":"2024-03-30T02:34:50.770703+0000","last_became_peered":"2024-03-30T02:34:50.770703+0000","last_unstale":"2024-03-30T02:34:51.652925+0000","last_undegraded":"2024-03-30T02:34:51.652925+0000","last_fullsized":"2024-03-30T02:34:51.652925+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:04.466664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.745296+0000","last_change":"2024-03-30T02:34:46.203558+0000","last_active":"2024-03-30T02:34:50.745296+0000","last_peered":"2024-03-30T02:34:50.745296+0000","last_clean":"2024-03-30T02:34:50.745296+0000","last_became_active":"2024-03-30T02:34:46.202896+0000","last_became_peered":"2024-03-30T02:34:46.202896+0000","last_unstale":"2024-03-30T02:34:50.745296+0000","last_undegraded":"2024-03-30T02:34:50.745296+0000","last_fullsized":"2024-03-30T02:34:50.745296+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:33.945665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076752+0000","last_change":"2024-03-30T02:34:46.747519+0000","last_active":"2024-03-30T02:34:51.076752+0000","last_peered":"2024-03-30T02:34:51.076752+0000","last_clean":"2024-03-30T02:34:51.076752+0000","last_became_active":"2024-03-30T02:34:46.747220+0000","last_became_peered":"2024-03-30T02:34:46.747220+0000","last_unstale":"2024-03-30T02:34:51.076752+0000","last_undegraded":"2024-03-30T02:34:51.076752+0000","last_fullsized":"2024-03-30T02:34:51.076752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:21:19.419986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.369649+0000","last_change":"2024-03-30T02:34:50.797758+0000","last_active":"2024-03-30T02:34:51.369649+0000","last_peered":"2024-03-30T02:34:51.369649+0000","last_clean":"2024-03-30T02:34:51.369649+0000","last_became_active":"2024-03-30T02:34:50.797311+0000","last_became_peered":"2024-03-30T02:34:50.797311+0000","last_unstale":"2024-03-30T02:34:51.369649+0000","last_undegraded":"2024-03-30T02:34:51.369649+0000","last_fullsized":"2024-03-30T02:34:51.369649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:32.337422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.367367+0000","last_change":"2024-03-30T02:34:46.224397+0000","last_active":"2024-03-30T02:34:51.367367+0000","last_peered":"2024-03-30T02:34:51.367367+0000","last_clean":"2024-03-30T02:34:51.367367+0000","last_became_active":"2024-03-30T02:34:46.223825+0000","last_became_peered":"2024-03-30T02:34:46.223825+0000","last_unstale":"2024-03-30T02:34:51.367367+0000","last_undegraded":"2024-03-30T02:34:51.367367+0000","last_fullsized":"2024-03-30T02:34:51.367367+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:53.154665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.796460+0000","last_change":"2024-03-30T02:34:46.741358+0000","last_active":"2024-03-30T02:34:50.796460+0000","last_peered":"2024-03-30T02:34:50.796460+0000","last_clean":"2024-03-30T02:34:50.796460+0000","last_became_active":"2024-03-30T02:34:46.740734+0000","last_became_peered":"2024-03-30T02:34:46.740734+0000","last_unstale":"2024-03-30T02:34:50.796460+0000","last_undegraded":"2024-03-30T02:34:50.796460+0000","last_fullsized":"2024-03-30T02:34:50.796460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:59.678431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.770598+0000","last_change":"2024-03-30T02:34:50.761998+0000","last_active":"2024-03-30T02:34:50.770598+0000","last_peered":"2024-03-30T02:34:50.770598+0000","last_clean":"2024-03-30T02:34:50.770598+0000","last_became_active":"2024-03-30T02:34:50.761560+0000","last_became_peered":"2024-03-30T02:34:50.761560+0000","last_unstale":"2024-03-30T02:34:50.770598+0000","last_undegraded":"2024-03-30T02:34:50.770598+0000","last_fullsized":"2024-03-30T02:34:50.770598+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:57:17.213624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:20.130143+0000","last_change":"2024-03-30T02:34:46.213427+0000","last_active":"2024-03-30T02:35:20.130143+0000","last_peered":"2024-03-30T02:35:20.130143+0000","last_clean":"2024-03-30T02:35:20.130143+0000","last_became_active":"2024-03-30T02:34:46.213082+0000","last_became_peered":"2024-03-30T02:34:46.213082+0000","last_unstale":"2024-03-30T02:35:20.130143+0000","last_undegraded":"2024-03-30T02:35:20.130143+0000","last_fullsized":"2024-03-30T02:35:20.130143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:31:35.513723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.796452+0000","last_change":"2024-03-30T02:34:46.747197+0000","last_active":"2024-03-30T02:34:50.796452+0000","last_peered":"2024-03-30T02:34:50.796452+0000","last_clean":"2024-03-30T02:34:50.796452+0000","last_became_active":"2024-03-30T02:34:46.743841+0000","last_became_peered":"2024-03-30T02:34:46.743841+0000","last_unstale":"2024-03-30T02:34:50.796452+0000","last_undegraded":"2024-03-30T02:34:50.796452+0000","last_fullsized":"2024-03-30T02:34:50.796452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:09.234382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.798995+0000","last_change":"2024-03-30T02:34:50.797423+0000","last_active":"2024-03-30T02:34:50.798995+0000","last_peered":"2024-03-30T02:34:50.798995+0000","last_clean":"2024-03-30T02:34:50.798995+0000","last_became_active":"2024-03-30T02:34:50.796746+0000","last_became_peered":"2024-03-30T02:34:50.796746+0000","last_unstale":"2024-03-30T02:34:50.798995+0000","last_undegraded":"2024-03-30T02:34:50.798995+0000","last_fullsized":"2024-03-30T02:34:50.798995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:24.014131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748816+0000","last_change":"2024-03-30T02:34:46.734972+0000","last_active":"2024-03-30T02:34:50.748816+0000","last_peered":"2024-03-30T02:34:50.748816+0000","last_clean":"2024-03-30T02:34:50.748816+0000","last_became_active":"2024-03-30T02:34:46.734520+0000","last_became_peered":"2024-03-30T02:34:46.734520+0000","last_unstale":"2024-03-30T02:34:50.748816+0000","last_undegraded":"2024-03-30T02:34:50.748816+0000","last_fullsized":"2024-03-30T02:34:50.748816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:15.635060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:17.111904+0000","last_change":"2024-03-30T02:34:46.224036+0000","last_active":"2024-03-30T02:35:17.111904+0000","last_peered":"2024-03-30T02:35:17.111904+0000","last_clean":"2024-03-30T02:35:17.111904+0000","last_became_active":"2024-03-30T02:34:46.223706+0000","last_became_peered":"2024-03-30T02:34:46.223706+0000","last_unstale":"2024-03-30T02:35:17.111904+0000","last_undegraded":"2024-03-30T02:35:17.111904+0000","last_fullsized":"2024-03-30T02:35:17.111904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:40.458345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.654312+0000","last_change":"2024-03-30T02:34:50.757894+0000","last_active":"2024-03-30T02:34:51.654312+0000","last_peered":"2024-03-30T02:34:51.654312+0000","last_clean":"2024-03-30T02:34:51.654312+0000","last_became_active":"2024-03-30T02:34:50.757486+0000","last_became_peered":"2024-03-30T02:34:50.757486+0000","last_unstale":"2024-03-30T02:34:51.654312+0000","last_undegraded":"2024-03-30T02:34:51.654312+0000","last_fullsized":"2024-03-30T02:34:51.654312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:30.609851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748899+0000","last_change":"2024-03-30T02:34:46.745990+0000","last_active":"2024-03-30T02:34:50.748899+0000","last_peered":"2024-03-30T02:34:50.748899+0000","last_clean":"2024-03-30T02:34:50.748899+0000","last_became_active":"2024-03-30T02:34:46.741937+0000","last_became_peered":"2024-03-30T02:34:46.741937+0000","last_unstale":"2024-03-30T02:34:50.748899+0000","last_undegraded":"2024-03-30T02:34:50.748899+0000","last_fullsized":"2024-03-30T02:34:50.748899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:26:17.833164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:20.129822+0000","last_change":"2024-03-30T02:34:46.208858+0000","last_active":"2024-03-30T02:35:20.129822+0000","last_peered":"2024-03-30T02:35:20.129822+0000","last_clean":"2024-03-30T02:35:20.129822+0000","last_became_active":"2024-03-30T02:34:46.208479+0000","last_became_peered":"2024-03-30T02:34:46.208479+0000","last_unstale":"2024-03-30T02:35:20.129822+0000","last_undegraded":"2024-03-30T02:35:20.129822+0000","last_fullsized":"2024-03-30T02:35:20.129822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:58:14.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.653940+0000","last_change":"2024-03-30T02:33:56.208237+0000","last_active":"2024-03-30T02:34:51.653940+0000","last_peered":"2024-03-30T02:34:51.653940+0000","last_clean":"2024-03-30T02:34:51.653940+0000","last_became_active":"2024-03-30T02:33:56.207820+0000","last_became_peered":"2024-03-30T02:33:56.207820+0000","last_unstale":"2024-03-30T02:34:51.653940+0000","last_undegraded":"2024-03-30T02:34:51.653940+0000","last_fullsized":"2024-03-30T02:34:51.653940+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_clean_scrub_stamp":"2024-03-30T02:31:23.922434+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:26.198503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.652719+0000","last_change":"2024-03-30T02:34:50.797822+0000","last_active":"2024-03-30T02:34:51.652719+0000","last_peered":"2024-03-30T02:34:51.652719+0000","last_clean":"2024-03-30T02:34:51.652719+0000","last_became_active":"2024-03-30T02:34:50.797357+0000","last_became_peered":"2024-03-30T02:34:50.797357+0000","last_unstale":"2024-03-30T02:34:51.652719+0000","last_undegraded":"2024-03-30T02:34:51.652719+0000","last_fullsized":"2024-03-30T02:34:51.652719+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:25:15.806076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747081+0000","last_change":"2024-03-30T02:34:46.739757+0000","last_active":"2024-03-30T02:34:50.747081+0000","last_peered":"2024-03-30T02:34:50.747081+0000","last_clean":"2024-03-30T02:34:50.747081+0000","last_became_active":"2024-03-30T02:34:46.738269+0000","last_became_peered":"2024-03-30T02:34:46.738269+0000","last_unstale":"2024-03-30T02:34:50.747081+0000","last_undegraded":"2024-03-30T02:34:50.747081+0000","last_fullsized":"2024-03-30T02:34:50.747081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:03.411187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:17.108662+0000","last_change":"2024-03-30T02:34:46.212532+0000","last_active":"2024-03-30T02:35:17.108662+0000","last_peered":"2024-03-30T02:35:17.108662+0000","last_clean":"2024-03-30T02:35:17.108662+0000","last_became_active":"2024-03-30T02:34:46.212191+0000","last_became_peered":"2024-03-30T02:34:46.212191+0000","last_unstale":"2024-03-30T02:35:17.108662+0000","last_undegraded":"2024-03-30T02:35:17.108662+0000","last_fullsized":"2024-03-30T02:35:17.108662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:19.120960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076666+0000","last_change":"2024-03-30T02:34:50.759911+0000","last_active":"2024-03-30T02:34:51.076666+0000","last_peered":"2024-03-30T02:34:51.076666+0000","last_clean":"2024-03-30T02:34:51.076666+0000","last_became_active":"2024-03-30T02:34:50.759487+0000","last_became_peered":"2024-03-30T02:34:50.759487+0000","last_unstale":"2024-03-30T02:34:51.076666+0000","last_undegraded":"2024-03-30T02:34:51.076666+0000","last_fullsized":"2024-03-30T02:34:51.076666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:46.403754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'3","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.115192+0000","last_change":"2024-03-30T02:34:46.217092+0000","last_active":"2024-03-30T02:35:18.115192+0000","last_peered":"2024-03-30T02:35:18.115192+0000","last_clean":"2024-03-30T02:35:18.115192+0000","last_became_active":"2024-03-30T02:34:46.216779+0000","last_became_peered":"2024-03-30T02:34:46.216779+0000","last_unstale":"2024-03-30T02:35:18.115192+0000","last_undegraded":"2024-03-30T02:35:18.115192+0000","last_fullsized":"2024-03-30T02:35:18.115192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:10.894577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.757761+0000","last_change":"2024-03-30T02:34:46.735212+0000","last_active":"2024-03-30T02:34:50.757761+0000","last_peered":"2024-03-30T02:34:50.757761+0000","last_clean":"2024-03-30T02:34:50.757761+0000","last_became_active":"2024-03-30T02:34:46.734282+0000","last_became_peered":"2024-03-30T02:34:46.734282+0000","last_unstale":"2024-03-30T02:34:50.757761+0000","last_undegraded":"2024-03-30T02:34:50.757761+0000","last_fullsized":"2024-03-30T02:34:50.757761+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:28.596961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.273248+0000","last_change":"2024-03-30T02:34:50.799290+0000","last_active":"2024-03-30T02:34:51.273248+0000","last_peered":"2024-03-30T02:34:51.273248+0000","last_clean":"2024-03-30T02:34:51.273248+0000","last_became_active":"2024-03-30T02:34:50.797524+0000","last_became_peered":"2024-03-30T02:34:50.797524+0000","last_unstale":"2024-03-30T02:34:51.273248+0000","last_undegraded":"2024-03-30T02:34:51.273248+0000","last_fullsized":"2024-03-30T02:34:51.273248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:53:27.931444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'7","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.115016+0000","last_change":"2024-03-30T02:34:46.202508+0000","last_active":"2024-03-30T02:35:18.115016+0000","last_peered":"2024-03-30T02:35:18.115016+0000","last_clean":"2024-03-30T02:35:18.115016+0000","last_became_active":"2024-03-30T02:34:46.202145+0000","last_became_peered":"2024-03-30T02:34:46.202145+0000","last_unstale":"2024-03-30T02:35:18.115016+0000","last_undegraded":"2024-03-30T02:35:18.115016+0000","last_fullsized":"2024-03-30T02:35:18.115016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:21.138776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.753707+0000","last_change":"2024-03-30T02:34:46.740056+0000","last_active":"2024-03-30T02:34:50.753707+0000","last_peered":"2024-03-30T02:34:50.753707+0000","last_clean":"2024-03-30T02:34:50.753707+0000","last_became_active":"2024-03-30T02:34:46.739260+0000","last_became_peered":"2024-03-30T02:34:46.739260+0000","last_unstale":"2024-03-30T02:34:50.753707+0000","last_undegraded":"2024-03-30T02:34:50.753707+0000","last_fullsized":"2024-03-30T02:34:50.753707+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:53.550978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.272694+0000","last_change":"2024-03-30T02:34:50.756851+0000","last_active":"2024-03-30T02:34:51.272694+0000","last_peered":"2024-03-30T02:34:51.272694+0000","last_clean":"2024-03-30T02:34:51.272694+0000","last_became_active":"2024-03-30T02:34:50.756416+0000","last_became_peered":"2024-03-30T02:34:50.756416+0000","last_unstale":"2024-03-30T02:34:51.272694+0000","last_undegraded":"2024-03-30T02:34:51.272694+0000","last_fullsized":"2024-03-30T02:34:51.272694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:11.469599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:19.123893+0000","last_change":"2024-03-30T02:34:46.215663+0000","last_active":"2024-03-30T02:35:19.123893+0000","last_peered":"2024-03-30T02:35:19.123893+0000","last_clean":"2024-03-30T02:35:19.123893+0000","last_became_active":"2024-03-30T02:34:46.214224+0000","last_became_peered":"2024-03-30T02:34:46.214224+0000","last_unstale":"2024-03-30T02:35:19.123893+0000","last_undegraded":"2024-03-30T02:35:19.123893+0000","last_fullsized":"2024-03-30T02:35:19.123893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:24:58.734434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.755260+0000","last_change":"2024-03-30T02:34:46.754150+0000","last_active":"2024-03-30T02:34:50.755260+0000","last_peered":"2024-03-30T02:34:50.755260+0000","last_clean":"2024-03-30T02:34:50.755260+0000","last_became_active":"2024-03-30T02:34:46.753786+0000","last_became_peered":"2024-03-30T02:34:46.753786+0000","last_unstale":"2024-03-30T02:34:50.755260+0000","last_undegraded":"2024-03-30T02:34:50.755260+0000","last_fullsized":"2024-03-30T02:34:50.755260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:54:50.165020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107246+0000","last_change":"2024-03-30T02:34:50.797890+0000","last_active":"2024-03-30T02:34:51.107246+0000","last_peered":"2024-03-30T02:34:51.107246+0000","last_clean":"2024-03-30T02:34:51.107246+0000","last_became_active":"2024-03-30T02:34:50.797395+0000","last_became_peered":"2024-03-30T02:34:50.797395+0000","last_unstale":"2024-03-30T02:34:51.107246+0000","last_undegraded":"2024-03-30T02:34:51.107246+0000","last_fullsized":"2024-03-30T02:34:51.107246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:36:14.958804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.102548+0000","last_change":"2024-03-30T02:34:46.220873+0000","last_active":"2024-03-30T02:35:16.102548+0000","last_peered":"2024-03-30T02:35:16.102548+0000","last_clean":"2024-03-30T02:35:16.102548+0000","last_became_active":"2024-03-30T02:34:46.220544+0000","last_became_peered":"2024-03-30T02:34:46.220544+0000","last_unstale":"2024-03-30T02:35:16.102548+0000","last_undegraded":"2024-03-30T02:35:16.102548+0000","last_fullsized":"2024-03-30T02:35:16.102548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:37.691602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.746086+0000","last_change":"2024-03-30T02:34:46.739184+0000","last_active":"2024-03-30T02:34:50.746086+0000","last_peered":"2024-03-30T02:34:50.746086+0000","last_clean":"2024-03-30T02:34:50.746086+0000","last_became_active":"2024-03-30T02:34:46.738843+0000","last_became_peered":"2024-03-30T02:34:46.738843+0000","last_unstale":"2024-03-30T02:34:50.746086+0000","last_undegraded":"2024-03-30T02:34:50.746086+0000","last_fullsized":"2024-03-30T02:34:50.746086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:15:09.894754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.772096+0000","last_change":"2024-03-30T02:34:50.768763+0000","last_active":"2024-03-30T02:34:50.772096+0000","last_peered":"2024-03-30T02:34:50.772096+0000","last_clean":"2024-03-30T02:34:50.772096+0000","last_became_active":"2024-03-30T02:34:50.767363+0000","last_became_peered":"2024-03-30T02:34:50.767363+0000","last_unstale":"2024-03-30T02:34:50.772096+0000","last_undegraded":"2024-03-30T02:34:50.772096+0000","last_fullsized":"2024-03-30T02:34:50.772096+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:06.932826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:18.113271+0000","last_change":"2024-03-30T02:34:46.213006+0000","last_active":"2024-03-30T02:35:18.113271+0000","last_peered":"2024-03-30T02:35:18.113271+0000","last_clean":"2024-03-30T02:35:18.113271+0000","last_became_active":"2024-03-30T02:34:46.206271+0000","last_became_peered":"2024-03-30T02:34:46.206271+0000","last_unstale":"2024-03-30T02:35:18.113271+0000","last_undegraded":"2024-03-30T02:35:18.113271+0000","last_fullsized":"2024-03-30T02:35:18.113271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:00:36.447352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076628+0000","last_change":"2024-03-30T02:34:46.739566+0000","last_active":"2024-03-30T02:34:51.076628+0000","last_peered":"2024-03-30T02:34:51.076628+0000","last_clean":"2024-03-30T02:34:51.076628+0000","last_became_active":"2024-03-30T02:34:46.739205+0000","last_became_peered":"2024-03-30T02:34:46.739205+0000","last_unstale":"2024-03-30T02:34:51.076628+0000","last_undegraded":"2024-03-30T02:34:51.076628+0000","last_fullsized":"2024-03-30T02:34:51.076628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:01.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.652397+0000","last_change":"2024-03-30T02:34:50.758016+0000","last_active":"2024-03-30T02:34:51.652397+0000","last_peered":"2024-03-30T02:34:51.652397+0000","last_clean":"2024-03-30T02:34:51.652397+0000","last_became_active":"2024-03-30T02:34:50.757616+0000","last_became_peered":"2024-03-30T02:34:50.757616+0000","last_unstale":"2024-03-30T02:34:51.652397+0000","last_undegraded":"2024-03-30T02:34:51.652397+0000","last_fullsized":"2024-03-30T02:34:51.652397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:35:07.902347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:17.207752+0000","last_change":"2024-03-30T02:34:46.217138+0000","last_active":"2024-03-30T02:35:17.207752+0000","last_peered":"2024-03-30T02:35:17.207752+0000","last_clean":"2024-03-30T02:35:17.207752+0000","last_became_active":"2024-03-30T02:34:46.216799+0000","last_became_peered":"2024-03-30T02:34:46.216799+0000","last_unstale":"2024-03-30T02:35:17.207752+0000","last_undegraded":"2024-03-30T02:35:17.207752+0000","last_fullsized":"2024-03-30T02:35:17.207752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:43:42.562874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748720+0000","last_change":"2024-03-30T02:34:46.754114+0000","last_active":"2024-03-30T02:34:50.748720+0000","last_peered":"2024-03-30T02:34:50.748720+0000","last_clean":"2024-03-30T02:34:50.748720+0000","last_became_active":"2024-03-30T02:34:46.752691+0000","last_became_peered":"2024-03-30T02:34:46.752691+0000","last_unstale":"2024-03-30T02:34:50.748720+0000","last_undegraded":"2024-03-30T02:34:50.748720+0000","last_fullsized":"2024-03-30T02:34:50.748720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:42.908877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.107236+0000","last_change":"2024-03-30T02:34:50.798110+0000","last_active":"2024-03-30T02:34:51.107236+0000","last_peered":"2024-03-30T02:34:51.107236+0000","last_clean":"2024-03-30T02:34:51.107236+0000","last_became_active":"2024-03-30T02:34:50.797551+0000","last_became_peered":"2024-03-30T02:34:50.797551+0000","last_unstale":"2024-03-30T02:34:51.107236+0000","last_undegraded":"2024-03-30T02:34:51.107236+0000","last_fullsized":"2024-03-30T02:34:51.107236+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:49.362018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:16.102696+0000","last_change":"2024-03-30T02:34:46.211647+0000","last_active":"2024-03-30T02:35:16.102696+0000","last_peered":"2024-03-30T02:35:16.102696+0000","last_clean":"2024-03-30T02:35:16.102696+0000","last_became_active":"2024-03-30T02:34:46.211311+0000","last_became_peered":"2024-03-30T02:34:46.211311+0000","last_unstale":"2024-03-30T02:35:16.102696+0000","last_undegraded":"2024-03-30T02:35:16.102696+0000","last_fullsized":"2024-03-30T02:35:16.102696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:05:34.135289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117673+0000","last_change":"2024-03-30T02:34:46.740256+0000","last_active":"2024-03-30T02:34:51.117673+0000","last_peered":"2024-03-30T02:34:51.117673+0000","last_clean":"2024-03-30T02:34:51.117673+0000","last_became_active":"2024-03-30T02:34:46.739913+0000","last_became_peered":"2024-03-30T02:34:46.739913+0000","last_unstale":"2024-03-30T02:34:51.117673+0000","last_undegraded":"2024-03-30T02:34:51.117673+0000","last_fullsized":"2024-03-30T02:34:51.117673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:44.152475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.762477+0000","last_change":"2024-03-30T02:34:50.762149+0000","last_active":"2024-03-30T02:34:50.762477+0000","last_peered":"2024-03-30T02:34:50.762477+0000","last_clean":"2024-03-30T02:34:50.762477+0000","last_became_active":"2024-03-30T02:34:50.761717+0000","last_became_peered":"2024-03-30T02:34:50.761717+0000","last_unstale":"2024-03-30T02:34:50.762477+0000","last_undegraded":"2024-03-30T02:34:50.762477+0000","last_fullsized":"2024-03-30T02:34:50.762477+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:22.373079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:20.133517+0000","last_change":"2024-03-30T02:34:46.224098+0000","last_active":"2024-03-30T02:35:20.133517+0000","last_peered":"2024-03-30T02:35:20.133517+0000","last_clean":"2024-03-30T02:35:20.133517+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:35:20.133517+0000","last_undegraded":"2024-03-30T02:35:20.133517+0000","last_fullsized":"2024-03-30T02:35:20.133517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:20:25.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117762+0000","last_change":"2024-03-30T02:34:46.737661+0000","last_active":"2024-03-30T02:34:51.117762+0000","last_peered":"2024-03-30T02:34:51.117762+0000","last_clean":"2024-03-30T02:34:51.117762+0000","last_became_active":"2024-03-30T02:34:46.737303+0000","last_became_peered":"2024-03-30T02:34:46.737303+0000","last_unstale":"2024-03-30T02:34:51.117762+0000","last_undegraded":"2024-03-30T02:34:51.117762+0000","last_fullsized":"2024-03-30T02:34:51.117762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:44.929643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.651797+0000","last_change":"2024-03-30T02:34:50.767119+0000","last_active":"2024-03-30T02:34:51.651797+0000","last_peered":"2024-03-30T02:34:51.651797+0000","last_clean":"2024-03-30T02:34:51.651797+0000","last_became_active":"2024-03-30T02:34:50.766472+0000","last_became_peered":"2024-03-30T02:34:50.766472+0000","last_unstale":"2024-03-30T02:34:51.651797+0000","last_undegraded":"2024-03-30T02:34:51.651797+0000","last_fullsized":"2024-03-30T02:34:51.651797+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:46.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:19.124243+0000","last_change":"2024-03-30T02:34:46.216713+0000","last_active":"2024-03-30T02:35:19.124243+0000","last_peered":"2024-03-30T02:35:19.124243+0000","last_clean":"2024-03-30T02:35:19.124243+0000","last_became_active":"2024-03-30T02:34:46.216366+0000","last_became_peered":"2024-03-30T02:34:46.216366+0000","last_unstale":"2024-03-30T02:35:19.124243+0000","last_undegraded":"2024-03-30T02:35:19.124243+0000","last_fullsized":"2024-03-30T02:35:19.124243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:43.803955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117724+0000","last_change":"2024-03-30T02:34:46.736854+0000","last_active":"2024-03-30T02:34:51.117724+0000","last_peered":"2024-03-30T02:34:51.117724+0000","last_clean":"2024-03-30T02:34:51.117724+0000","last_became_active":"2024-03-30T02:34:46.736511+0000","last_became_peered":"2024-03-30T02:34:46.736511+0000","last_unstale":"2024-03-30T02:34:51.117724+0000","last_undegraded":"2024-03-30T02:34:51.117724+0000","last_fullsized":"2024-03-30T02:34:51.117724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:12.701467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.652228+0000","last_change":"2024-03-30T02:34:50.757227+0000","last_active":"2024-03-30T02:34:51.652228+0000","last_peered":"2024-03-30T02:34:51.652228+0000","last_clean":"2024-03-30T02:34:51.652228+0000","last_became_active":"2024-03-30T02:34:50.756742+0000","last_became_peered":"2024-03-30T02:34:50.756742+0000","last_unstale":"2024-03-30T02:34:51.652228+0000","last_undegraded":"2024-03-30T02:34:51.652228+0000","last_fullsized":"2024-03-30T02:34:51.652228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:48.792202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-03-30T02:35:19.123452+0000","last_change":"2024-03-30T02:34:46.218954+0000","last_active":"2024-03-30T02:35:19.123452+0000","last_peered":"2024-03-30T02:35:19.123452+0000","last_clean":"2024-03-30T02:35:19.123452+0000","last_became_active":"2024-03-30T02:34:46.218019+0000","last_became_peered":"2024-03-30T02:34:46.218019+0000","last_unstale":"2024-03-30T02:35:19.123452+0000","last_undegraded":"2024-03-30T02:35:19.123452+0000","last_fullsized":"2024-03-30T02:35:19.123452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:32.651216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.117781+0000","last_change":"2024-03-30T02:34:46.741735+0000","last_active":"2024-03-30T02:34:51.117781+0000","last_peered":"2024-03-30T02:34:51.117781+0000","last_clean":"2024-03-30T02:34:51.117781+0000","last_became_active":"2024-03-30T02:34:46.740663+0000","last_became_peered":"2024-03-30T02:34:46.740663+0000","last_unstale":"2024-03-30T02:34:51.117781+0000","last_undegraded":"2024-03-30T02:34:51.117781+0000","last_fullsized":"2024-03-30T02:34:51.117781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:09.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076652+0000","last_change":"2024-03-30T02:34:46.739082+0000","last_active":"2024-03-30T02:34:51.076652+0000","last_peered":"2024-03-30T02:34:51.076652+0000","last_clean":"2024-03-30T02:34:51.076652+0000","last_became_active":"2024-03-30T02:34:46.738732+0000","last_became_peered":"2024-03-30T02:34:46.738732+0000","last_unstale":"2024-03-30T02:34:51.076652+0000","last_undegraded":"2024-03-30T02:34:51.076652+0000","last_fullsized":"2024-03-30T02:34:51.076652+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:53.961520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.750297+0000","last_change":"2024-03-30T02:34:46.744297+0000","last_active":"2024-03-30T02:34:50.750297+0000","last_peered":"2024-03-30T02:34:50.750297+0000","last_clean":"2024-03-30T02:34:50.750297+0000","last_became_active":"2024-03-30T02:34:46.743284+0000","last_became_peered":"2024-03-30T02:34:46.743284+0000","last_unstale":"2024-03-30T02:34:50.750297+0000","last_undegraded":"2024-03-30T02:34:50.750297+0000","last_fullsized":"2024-03-30T02:34:50.750297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:30:20.960739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747399+0000","last_change":"2024-03-30T02:34:46.742590+0000","last_active":"2024-03-30T02:34:50.747399+0000","last_peered":"2024-03-30T02:34:50.747399+0000","last_clean":"2024-03-30T02:34:50.747399+0000","last_became_active":"2024-03-30T02:34:46.740904+0000","last_became_peered":"2024-03-30T02:34:46.740904+0000","last_unstale":"2024-03-30T02:34:50.747399+0000","last_undegraded":"2024-03-30T02:34:50.747399+0000","last_fullsized":"2024-03-30T02:34:50.747399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:22.653881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.756079+0000","last_change":"2024-03-30T02:34:46.753119+0000","last_active":"2024-03-30T02:34:50.756079+0000","last_peered":"2024-03-30T02:34:50.756079+0000","last_clean":"2024-03-30T02:34:50.756079+0000","last_became_active":"2024-03-30T02:34:46.745680+0000","last_became_peered":"2024-03-30T02:34:46.745680+0000","last_unstale":"2024-03-30T02:34:50.756079+0000","last_undegraded":"2024-03-30T02:34:50.756079+0000","last_fullsized":"2024-03-30T02:34:50.756079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:02.352961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.757900+0000","last_change":"2024-03-30T02:34:46.740004+0000","last_active":"2024-03-30T02:34:50.757900+0000","last_peered":"2024-03-30T02:34:50.757900+0000","last_clean":"2024-03-30T02:34:50.757900+0000","last_became_active":"2024-03-30T02:34:46.737934+0000","last_became_peered":"2024-03-30T02:34:46.737934+0000","last_unstale":"2024-03-30T02:34:50.757900+0000","last_undegraded":"2024-03-30T02:34:50.757900+0000","last_fullsized":"2024-03-30T02:34:50.757900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:03:39.976441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076739+0000","last_change":"2024-03-30T02:34:46.737882+0000","last_active":"2024-03-30T02:34:51.076739+0000","last_peered":"2024-03-30T02:34:51.076739+0000","last_clean":"2024-03-30T02:34:51.076739+0000","last_became_active":"2024-03-30T02:34:46.737546+0000","last_became_peered":"2024-03-30T02:34:46.737546+0000","last_unstale":"2024-03-30T02:34:51.076739+0000","last_undegraded":"2024-03-30T02:34:51.076739+0000","last_fullsized":"2024-03-30T02:34:51.076739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:48:41.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.751323+0000","last_change":"2024-03-30T02:34:46.742703+0000","last_active":"2024-03-30T02:34:50.751323+0000","last_peered":"2024-03-30T02:34:50.751323+0000","last_clean":"2024-03-30T02:34:50.751323+0000","last_became_active":"2024-03-30T02:34:46.740689+0000","last_became_peered":"2024-03-30T02:34:46.740689+0000","last_unstale":"2024-03-30T02:34:50.751323+0000","last_undegraded":"2024-03-30T02:34:50.751323+0000","last_fullsized":"2024-03-30T02:34:50.751323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:45.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.753822+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:34:50.753822+0000","last_peered":"2024-03-30T02:34:50.753822+0000","last_clean":"2024-03-30T02:34:50.753822+0000","last_became_active":"2024-03-30T02:34:46.750395+0000","last_became_peered":"2024-03-30T02:34:46.750395+0000","last_unstale":"2024-03-30T02:34:50.753822+0000","last_undegraded":"2024-03-30T02:34:50.753822+0000","last_fullsized":"2024-03-30T02:34:50.753822+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:01.282677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748705+0000","last_change":"2024-03-30T02:34:46.734814+0000","last_active":"2024-03-30T02:34:50.748705+0000","last_peered":"2024-03-30T02:34:50.748705+0000","last_clean":"2024-03-30T02:34:50.748705+0000","last_became_active":"2024-03-30T02:34:46.733978+0000","last_became_peered":"2024-03-30T02:34:46.733978+0000","last_unstale":"2024-03-30T02:34:50.748705+0000","last_undegraded":"2024-03-30T02:34:50.748705+0000","last_fullsized":"2024-03-30T02:34:50.748705+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:31:39.467478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.748398+0000","last_change":"2024-03-30T02:34:46.748163+0000","last_active":"2024-03-30T02:34:50.748398+0000","last_peered":"2024-03-30T02:34:50.748398+0000","last_clean":"2024-03-30T02:34:50.748398+0000","last_became_active":"2024-03-30T02:34:46.738845+0000","last_became_peered":"2024-03-30T02:34:46.738845+0000","last_unstale":"2024-03-30T02:34:50.748398+0000","last_undegraded":"2024-03-30T02:34:50.748398+0000","last_fullsized":"2024-03-30T02:34:50.748398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:09.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.743901+0000","last_change":"2024-03-30T02:34:46.743231+0000","last_active":"2024-03-30T02:34:50.743901+0000","last_peered":"2024-03-30T02:34:50.743901+0000","last_clean":"2024-03-30T02:34:50.743901+0000","last_became_active":"2024-03-30T02:34:46.742619+0000","last_became_peered":"2024-03-30T02:34:46.742619+0000","last_unstale":"2024-03-30T02:34:50.743901+0000","last_undegraded":"2024-03-30T02:34:50.743901+0000","last_fullsized":"2024-03-30T02:34:50.743901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:17:51.789139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:50.747504+0000","last_change":"2024-03-30T02:34:46.745872+0000","last_active":"2024-03-30T02:34:50.747504+0000","last_peered":"2024-03-30T02:34:50.747504+0000","last_clean":"2024-03-30T02:34:50.747504+0000","last_became_active":"2024-03-30T02:34:46.745476+0000","last_became_peered":"2024-03-30T02:34:46.745476+0000","last_unstale":"2024-03-30T02:34:50.747504+0000","last_undegraded":"2024-03-30T02:34:50.747504+0000","last_fullsized":"2024-03-30T02:34:50.747504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:30.774316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-03-30T02:34:51.076724+0000","last_change":"2024-03-30T02:34:46.746178+0000","last_active":"2024-03-30T02:34:51.076724+0000","last_peered":"2024-03-30T02:34:51.076724+0000","last_clean":"2024-03-30T02:34:51.076724+0000","last_became_active":"2024-03-30T02:34:46.745865+0000","last_became_peered":"2024-03-30T02:34:46.745865+0000","last_unstale":"2024-03-30T02:34:51.076724+0000","last_undegraded":"2024-03-30T02:34:51.076724+0000","last_fullsized":"2024-03-30T02:34:51.076724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:22.613637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248704,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10140,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":191,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":58,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":92948,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":169,"ondisk_log_size":169,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579920,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":450165,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63300000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.501},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.56200000000000006}]},{"osd":6,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.44},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.57399999999999995}]},{"osd":7,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.68400000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":9,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.84999999999999998}]},{"osd":10,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.69599999999999995}]}]},{"osd":10,"up_from":67,"seq":287762808852,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":991232,"data_stored":1040619,"data_compressed":47678,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42199999999999999}]},{"osd":2,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61699999999999999}]},{"osd":3,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64800000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.44800000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56299999999999994}]},{"osd":6,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":8,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.749},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58299999999999996}]},{"osd":9,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63300000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.72799999999999998}]}]},{"osd":9,"up_from":62,"seq":266287972375,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":450143,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.55900000000000005}]},{"osd":3,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.46600000000000003}]},{"osd":4,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.629},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.81499999999999995}]},{"osd":10,"last update":"Sat Mar 30 02:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":8,"up_from":55,"seq":236223201307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":445332,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.505},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56200000000000006}]},{"osd":1,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83399999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.621}]},{"osd":2,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.45300000000000001,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.32300000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.66600000000000004}]},{"osd":3,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88200000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89900000000000002,"5min":0.995,"15min":0.995},"last":0.83399999999999996}]},{"osd":5,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.40200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59199999999999997}]},{"osd":6,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.056,"15min":1.056},"last":0.75},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.878,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.443,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.998,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73099999999999998}]},{"osd":9,"last update":"Sat Mar 30 02:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.90000000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.82599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":7,"up_from":48,"seq":206158430239,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":1046996,"data_compressed":47678,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.41999999999999998},"min":{"1min":0.35399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44,"15min":0.435},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.378}]},{"osd":2,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.871,"5min":1.034,"15min":1.034},"last":0.53600000000000003}]},{"osd":3,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.51000000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.72799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.54500000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.317,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70199999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36199999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.503,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.625},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.496,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.56000000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.82699999999999996,"5min":0.877,"15min":0.877},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.47899999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88800000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.505}]},{"osd":8,"last update":"Sat Mar 30 02:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.59799999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90100000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.47999999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70899999999999996}]},{"osd":9,"last update":"Sat Mar 30 02:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68400000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561059,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":913408,"data_stored":445599,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56699999999999995}]},{"osd":1,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.67600000000000005},"min":{"1min":0.49099999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.77400000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72299999999999998,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.95199999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.747,"15min":0.755},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91100000000000003,"5min":1.069,"15min":1.069},"last":0.70699999999999996}]},{"osd":3,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65500000000000003,"15min":0.65900000000000003},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96199999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.73399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47099999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.89500000000000002,"5min":1.236,"15min":1.236},"last":0.68400000000000005}]},{"osd":4,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72599999999999998,"15min":0.72999999999999998},"min":{"1min":0.49099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.054,"5min":1.069,"15min":1.069},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.754,"15min":0.755},"min":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.03,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.86099999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.83199999999999996,"15min":0.84399999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97999999999999998,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.87,"15min":0.874},"min":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.262,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.90900000000000003}]},{"osd":7,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":0.84999999999999998},"min":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.875},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.627,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.89100000000000001}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91400000000000003,"15min":0.92200000000000004},"min":{"1min":0.628,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1220000000000001,"5min":1.355,"15min":1.355},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.88600000000000001,"15min":0.89200000000000002},"min":{"1min":0.70099999999999996,"5min":0.623,"15min":0.623},"max":{"1min":1.0609999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Sat Mar 30 02:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.88700000000000001}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":469726,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.66100000000000003,"15min":0.77500000000000002},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81999999999999995,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.378,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70999999999999996}]},{"osd":2,"last update":"Sat Mar 30 02:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.35399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.499},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56200000000000006}]},{"osd":3,"last update":"Sat Mar 30 02:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.443},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57499999999999996,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52700000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41399999999999998,"15min":0.39800000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.378},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.41399999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.57599999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74099999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Mar 30 02:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89900000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.45400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.95599999999999996,"5min":1.044,"15min":1.044},"last":0.745}]},{"osd":9,"last update":"Sat Mar 30 02:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.73499999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":446403,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 02:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81599999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57299999999999995,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86199999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65800000000000003}]},{"osd":2,"last update":"Sat Mar 30 02:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.45900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.376,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.74099999999999999}]},{"osd":3,"last update":"Sat Mar 30 02:33:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.34699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.38100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54300000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.626,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":1.006,"15min":1.006},"last":0.59899999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:34:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64500000000000002,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65900000000000003,"15min":0.65200000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.85899999999999999}]},{"osd":6,"last update":"Sat Mar 30 02:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92800000000000005,"5min":1.091,"15min":1.091},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93700000000000006,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69399999999999995}]},{"osd":7,"last update":"Sat Mar 30 02:33:57 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.82999999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:34:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.90200000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:34:41 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.008},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.92600000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.196}]},{"osd":11,"last update":"Sat Mar 30 02:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.92000000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":489056,"data_compressed":13506,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88300000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.36899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82199999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71199999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.39800000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.71099999999999997,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67100000000000004}]},{"osd":3,"last update":"Sat Mar 30 02:34:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85499999999999998,"5min":0.878,"15min":0.878},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat Mar 30 02:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.441,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60599999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.82899999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65300000000000002}]},{"osd":6,"last update":"Sat Mar 30 02:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.874,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.624}]},{"osd":7,"last update":"Sat Mar 30 02:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.79900000000000004}]},{"osd":8,"last update":"Sat Mar 30 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.49399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90000000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.751}]},{"osd":9,"last update":"Sat Mar 30 02:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69899999999999995}]},{"osd":10,"last update":"Sat Mar 30 02:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.84599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182445,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":913408,"data_stored":445209,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:47 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.42499999999999999,"15min":0.39200000000000002},"min":{"1min":0.35599999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.441,"15min":0.42199999999999999},"min":{"1min":0.313,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.624,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.45300000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:34:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55000000000000004,"15min":0.53000000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85199999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.46500000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68400000000000005}]},{"osd":2,"last update":"Sat Mar 30 02:34:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.749},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.60199999999999998,"15min":0.58599999999999997},"min":{"1min":0.47299999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.66600000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93999999999999995,"5min":0.996,"15min":0.996},"last":0.62}]},{"osd":5,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.65000000000000002,"15min":0.63800000000000001},"min":{"1min":0.55800000000000005,"5min":0.495,"15min":0.495},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.71099999999999997,"15min":0.69099999999999995},"min":{"1min":0.54100000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.86199999999999999}]},{"osd":6,"last update":"Sat Mar 30 02:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.77800000000000002,"15min":0.77200000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1120000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69899999999999995,"15min":0.70099999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Sat Mar 30 02:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.997,"5min":1.071,"15min":1.071},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.131,"5min":1.25,"15min":1.25},"last":0.89200000000000002}]},{"osd":8,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.004}]},{"osd":9,"last update":"Sat Mar 30 02:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.80200000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.93899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953514,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":917504,"data_stored":445970,"data_compressed":11678,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.38,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.747,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.36699999999999999,"15min":0.34699999999999998},"min":{"1min":0.313,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.34999999999999998},"min":{"1min":0.32400000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.44}]},{"osd":2,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54700000000000004}]},{"osd":5,"last update":"Sat Mar 30 02:34:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.47899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58099999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.44900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.78800000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59599999999999997}]},{"osd":7,"last update":"Sat Mar 30 02:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.501},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.38900000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.623}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.497,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.69099999999999995,"15min":0.67300000000000004},"min":{"1min":0.52900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.63500000000000001}]},{"osd":9,"last update":"Sat Mar 30 02:34:38 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.755}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.73399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":5,"up_from":37,"seq":158913789991,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":995328,"data_stored":1042554,"data_compressed":47678,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.30099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.37,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.48599999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78300000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.376,"15min":0.36099999999999999},"min":{"1min":0.31900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.379,"15min":0.35899999999999999},"min":{"1min":0.308,"5min":0.253,"15min":0.253},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.42099999999999999}]},{"osd":3,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81000000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51700000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.55800000000000005,"5min":0.378,"15min":0.378},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.55200000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69299999999999995}]},{"osd":6,"last update":"Sat Mar 30 02:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.48699999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.623,"15min":0.623},"min":{"1min":0.47199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79500000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56699999999999995}]},{"osd":7,"last update":"Sat Mar 30 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.499,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.53900000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67000000000000004}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.61899999999999999}]},{"osd":9,"last update":"Sat Mar 30 02:34:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.73799999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.754},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":48746,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T02:35:23.168 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T02:35:23.169 INFO:tasks.cephadm:Setting up client nodes... 2024-03-30T02:35:23.169 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-30T02:35:23.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:23 smithi067 ceph-mon[37808]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:23.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:23 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/977433592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:23.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:23 smithi170 ceph-mon[78798]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:23.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:23 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/977433592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:23.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:23 smithi082 ceph-mon[79176]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:35:23.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:23 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/977433592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:23.572 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:24.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:24 smithi067 ceph-mon[37808]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:35:24.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:24 smithi067 ceph-mon[37808]: pgmap v251: 129 pgs: 129 active+clean; 587 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-30T02:35:24.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:24 smithi067 ceph-mon[37808]: from='client.24872 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:35:24.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:24 smithi170 ceph-mon[78798]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:35:24.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:24 smithi170 ceph-mon[78798]: pgmap v251: 129 pgs: 129 active+clean; 587 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-30T02:35:24.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:24 smithi170 ceph-mon[78798]: from='client.24872 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:35:24.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:24 smithi082 ceph-mon[79176]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:35:24.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:24 smithi082 ceph-mon[79176]: pgmap v251: 129 pgs: 129 active+clean; 587 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-30T02:35:24.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:24 smithi082 ceph-mon[79176]: from='client.24872 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:35:24.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.768+0000 7f8d11600700 1 -- 172.21.15.67:0/4195901902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 msgr2=0x7f8d0c102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:24.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.768+0000 7f8d11600700 1 --2- 172.21.15.67:0/4195901902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 0x7f8d0c102420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80099c0 tx=0x7f8cf8009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:24.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.768+0000 7f8d11600700 1 -- 172.21.15.67:0/4195901902 shutdown_connections 2024-03-30T02:35:24.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.768+0000 7f8d11600700 1 --2- 172.21.15.67:0/4195901902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d0c102960 0x7f8d0c109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:24.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.768+0000 7f8d11600700 1 --2- 172.21.15.67:0/4195901902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d0c100dc0 0x7f8d0c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:24.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.768+0000 7f8d11600700 1 --2- 172.21.15.67:0/4195901902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 0x7f8d0c102420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:24.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.768+0000 7f8d11600700 1 -- 172.21.15.67:0/4195901902 >> 172.21.15.67:0/4195901902 conn(0x7f8d0c0fc360 msgr2=0x7f8d0c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:24.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.768+0000 7f8d11600700 1 -- 172.21.15.67:0/4195901902 shutdown_connections 2024-03-30T02:35:24.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.768+0000 7f8d11600700 1 -- 172.21.15.67:0/4195901902 wait complete. 2024-03-30T02:35:24.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.769+0000 7f8d11600700 1 Processor -- start 2024-03-30T02:35:24.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.769+0000 7f8d11600700 1 -- start start 2024-03-30T02:35:24.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.769+0000 7f8d11600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d0c100dc0 0x7f8d0c196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:24.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.769+0000 7f8d11600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 0x7f8d0c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.769+0000 7f8d11600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d0c102960 0x7f8d0c19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.769+0000 7f8d11600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c19e820 con 0x7f8d0c101fc0 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.769+0000 7f8d11600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c19e960 con 0x7f8d0c102960 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.769+0000 7f8d11600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c19eb20 con 0x7f8d0c100dc0 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 0x7f8d0c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 0x7f8d0c197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57614/0 (socket says 172.21.15.67:57614) 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0a000700 1 -- 172.21.15.67:0/482366624 learned_addr learned my addr 172.21.15.67:0/482366624 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d0c102960 0x7f8d0c19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0aa00700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d0c100dc0 0x7f8d0c196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0a000700 1 -- 172.21.15.67:0/482366624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d0c100dc0 msgr2=0x7f8d0c196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0a000700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d0c100dc0 0x7f8d0c196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0a000700 1 -- 172.21.15.67:0/482366624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d0c102960 msgr2=0x7f8d0c19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0a000700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d0c102960 0x7f8d0c19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:24.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0a000700 1 -- 172.21.15.67:0/482366624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf80095f0 con 0x7f8d0c101fc0 2024-03-30T02:35:24.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.770+0000 7f8d0b400700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d0c102960 0x7f8d0c19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:24.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.771+0000 7f8d0a000700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 0x7f8d0c197230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8003730 tx=0x7f8cf800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:24.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.771+0000 7f8d03400700 1 -- 172.21.15.67:0/482366624 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf801d070 con 0x7f8d0c101fc0 2024-03-30T02:35:24.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.771+0000 7f8d03400700 1 -- 172.21.15.67:0/482366624 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cf800bda0 con 0x7f8d0c101fc0 2024-03-30T02:35:24.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.771+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c19eda0 con 0x7f8d0c101fc0 2024-03-30T02:35:24.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.771+0000 7f8d03400700 1 -- 172.21.15.67:0/482366624 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8004bd0 con 0x7f8d0c101fc0 2024-03-30T02:35:24.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.771+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c19f1b0 con 0x7f8d0c101fc0 2024-03-30T02:35:24.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.772+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c1081f0 con 0x7f8d0c101fc0 2024-03-30T02:35:24.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.774+0000 7f8d03400700 1 -- 172.21.15.67:0/482366624 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8cf8004d30 con 0x7f8d0c101fc0 2024-03-30T02:35:24.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.775+0000 7f8d03400700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8cf00a11b0 0x7f8cf00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:24.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.775+0000 7f8d03400700 1 -- 172.21.15.67:0/482366624 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8cf80cdab0 con 0x7f8d0c101fc0 2024-03-30T02:35:24.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.775+0000 7f8d0aa00700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8cf00a11b0 0x7f8cf00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:24.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.776+0000 7f8d0aa00700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8cf00a11b0 0x7f8cf00a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0fdfb0 tx=0x7f8cfc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:24.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:24.779+0000 7f8d03400700 1 -- 172.21.15.67:0/482366624 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8cf8096f20 con 0x7f8d0c101fc0 2024-03-30T02:35:25.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.078+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8d0c10c7a0 con 0x7f8d0c101fc0 2024-03-30T02:35:25.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.083+0000 7f8d03400700 1 -- 172.21.15.67:0/482366624 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8cf809a540 con 0x7f8d0c101fc0 2024-03-30T02:35:25.084 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-03-30T02:35:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQBtegdmYUXMBBAAa6tc3M+tOYj1gD7xMquxNA== 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.085+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8cf00a11b0 msgr2=0x7f8cf00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.085+0000 7f8d11600700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8cf00a11b0 0x7f8cf00a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0fdfb0 tx=0x7f8cfc009260 comp rx=0 tx=0).stop 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.085+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 msgr2=0x7f8d0c197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.085+0000 7f8d11600700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 0x7f8d0c197230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8003730 tx=0x7f8cf800fa00 comp rx=0 tx=0).stop 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.086+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 shutdown_connections 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.086+0000 7f8d11600700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8cf00a11b0 0x7f8cf00a3670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.086+0000 7f8d11600700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d0c102960 0x7f8d0c19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.086+0000 7f8d11600700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d0c100dc0 0x7f8d0c196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.086+0000 7f8d11600700 1 --2- 172.21.15.67:0/482366624 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d0c101fc0 0x7f8d0c197230 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.086+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 >> 172.21.15.67:0/482366624 conn(0x7f8d0c0fc360 msgr2=0x7f8d0c105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.086+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 shutdown_connections 2024-03-30T02:35:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:25.086+0000 7f8d11600700 1 -- 172.21.15.67:0/482366624 wait complete. 2024-03-30T02:35:25.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:25 smithi067 ceph-mon[37808]: pgmap v252: 129 pgs: 129 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-03-30T02:35:25.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:25 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/482366624' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-30T02:35:25.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:25 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/482366624' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-30T02:35:25.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:25 smithi170 ceph-mon[78798]: pgmap v252: 129 pgs: 129 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-03-30T02:35:25.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:25 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/482366624' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-30T02:35:25.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:25 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/482366624' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-30T02:35:25.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:25 smithi082 ceph-mon[79176]: pgmap v252: 129 pgs: 129 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-03-30T02:35:25.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:25 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/482366624' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-30T02:35:25.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:25 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/482366624' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-30T02:35:25.506 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:35:25.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-30T02:35:25.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-30T02:35:25.605 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-30T02:35:25.606 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-30T02:35:25.606 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-03-30T02:35:26.013 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:27.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.176+0000 7ff051600700 1 -- 172.21.15.67:0/4223495908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c106ef0 msgr2=0x7ff04c1072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.176+0000 7ff051600700 1 --2- 172.21.15.67:0/4223495908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c106ef0 0x7ff04c1072d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0099c0 tx=0x7ff03c009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.176+0000 7ff051600700 1 -- 172.21.15.67:0/4223495908 shutdown_connections 2024-03-30T02:35:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.176+0000 7ff051600700 1 --2- 172.21.15.67:0/4223495908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c106ef0 0x7ff04c1072d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.176+0000 7ff051600700 1 --2- 172.21.15.67:0/4223495908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff04c101820 0x7ff04c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.176+0000 7ff051600700 1 --2- 172.21.15.67:0/4223495908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff04c100e60 0x7ff04c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.176+0000 7ff051600700 1 -- 172.21.15.67:0/4223495908 >> 172.21.15.67:0/4223495908 conn(0x7ff04c077fa0 msgr2=0x7ff04c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.176+0000 7ff051600700 1 -- 172.21.15.67:0/4223495908 shutdown_connections 2024-03-30T02:35:27.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.176+0000 7ff051600700 1 -- 172.21.15.67:0/4223495908 wait complete. 2024-03-30T02:35:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.177+0000 7ff051600700 1 Processor -- start 2024-03-30T02:35:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.177+0000 7ff051600700 1 -- start start 2024-03-30T02:35:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff051600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff04c100e60 0x7ff04c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff051600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c101820 0x7ff04c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff051600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff04c106ef0 0x7ff04c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff051600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c1a5580 con 0x7ff04c101820 2024-03-30T02:35:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff051600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c1a56f0 con 0x7ff04c106ef0 2024-03-30T02:35:27.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff051600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c1a58b0 con 0x7ff04c100e60 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff04a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c101820 0x7ff04c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff04a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c101820 0x7ff04c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53752/0 (socket says 172.21.15.67:53752) 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff04a000700 1 -- 172.21.15.67:0/2941286726 learned_addr learned my addr 172.21.15.67:0/2941286726 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff04aa00700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff04c100e60 0x7ff04c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.178+0000 7ff04b400700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff04c106ef0 0x7ff04c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff04a000700 1 -- 172.21.15.67:0/2941286726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff04c100e60 msgr2=0x7ff04c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff04a000700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff04c100e60 0x7ff04c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff04a000700 1 -- 172.21.15.67:0/2941286726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff04c106ef0 msgr2=0x7ff04c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff04a000700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff04c106ef0 0x7ff04c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff04a000700 1 -- 172.21.15.67:0/2941286726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c0095f0 con 0x7ff04c101820 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff04a000700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c101820 0x7ff04c0735f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff03800e9a0 tx=0x7ff03800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff043400700 1 -- 172.21.15.67:0/2941286726 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038009790 con 0x7ff04c101820 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff043400700 1 -- 172.21.15.67:0/2941286726 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff038010430 con 0x7ff04c101820 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff043400700 1 -- 172.21.15.67:0/2941286726 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038018680 con 0x7ff04c101820 2024-03-30T02:35:27.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04c1a5b90 con 0x7ff04c101820 2024-03-30T02:35:27.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.179+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04c1a6090 con 0x7ff04c101820 2024-03-30T02:35:27.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.181+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04c1092d0 con 0x7ff04c101820 2024-03-30T02:35:27.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.182+0000 7ff043400700 1 -- 172.21.15.67:0/2941286726 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff0380105a0 con 0x7ff04c101820 2024-03-30T02:35:27.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.183+0000 7ff043400700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff0300a1230 0x7ff0300a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.184+0000 7ff043400700 1 -- 172.21.15.67:0/2941286726 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff038014070 con 0x7ff04c101820 2024-03-30T02:35:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.184+0000 7ff04aa00700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff0300a1230 0x7ff0300a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:27.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.184+0000 7ff04aa00700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff0300a1230 0x7ff0300a36f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009990 tx=0x7ff03c005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:27.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.188+0000 7ff043400700 1 -- 172.21.15.67:0/2941286726 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff0380529f0 con 0x7ff04c101820 2024-03-30T02:35:27.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.467+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff04c074590 con 0x7ff04c101820 2024-03-30T02:35:27.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.473+0000 7ff043400700 1 -- 172.21.15.67:0/2941286726 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260274 (secure 0 0 0) 0x7ff0380961d0 con 0x7ff04c101820 2024-03-30T02:35:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff0300a1230 msgr2=0x7ff0300a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff0300a1230 0x7ff0300a36f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009990 tx=0x7ff03c005fd0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c101820 msgr2=0x7ff04c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c101820 0x7ff04c0735f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff03800e9a0 tx=0x7ff03800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 shutdown_connections 2024-03-30T02:35:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff0300a1230 0x7ff0300a36f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff04c106ef0 0x7ff04c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff04c100e60 0x7ff04c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 --2- 172.21.15.67:0/2941286726 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff04c101820 0x7ff04c0735f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 >> 172.21.15.67:0/2941286726 conn(0x7ff04c077fa0 msgr2=0x7ff04c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 shutdown_connections 2024-03-30T02:35:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:27.477+0000 7ff051600700 1 -- 172.21.15.67:0/2941286726 wait complete. 2024-03-30T02:35:27.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-30T02:35:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-30T02:35:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-30T02:35:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-30T02:35:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-30T02:35:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-30T02:35:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-03-30T02:35:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-03-30T02:35:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]: dispatch 2024-03-30T02:35:27.500 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:27 smithi067 ceph-mon[37808]: pgmap v253: 129 pgs: 129 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]: dispatch 2024-03-30T02:35:27.501 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:27 smithi082 ceph-mon[79176]: pgmap v253: 129 pgs: 129 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-03-30T02:35:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-30T02:35:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-30T02:35:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-30T02:35:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-03-30T02:35:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-03-30T02:35:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-03-30T02:35:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-03-30T02:35:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-03-30T02:35:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]: dispatch 2024-03-30T02:35:27.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:27 smithi170 ceph-mon[78798]: pgmap v253: 129 pgs: 129 active+clean; 629 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-03-30T02:35:27.885 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"active_gid":14154,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":3801648476},{"type":"v1","addr":"172.21.15.67:6801","nonce":3801648476}]},"active_addr":"172.21.15.67:6801/3801648476","active_change":"2024-03-30T02:29:26.930090+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.67:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":1220244664}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":759150839}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2449045224}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3018284010}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":685078089}]}]} 2024-03-30T02:35:27.889 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-30T02:35:27.889 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-30T02:35:27.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-03-30T02:35:28.302 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:28.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-30T02:35:28.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-30T02:35:28.488 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-30T02:35:28.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-30T02:35:28.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-30T02:35:28.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-30T02:35:28.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-03-30T02:35:28.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-03-30T02:35:28.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-03-30T02:35:28.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: osdmap e84: 12 total, 12 up, 12 in 2024-03-30T02:35:28.489 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:28 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2941286726' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: osdmap e84: 12 total, 12 up, 12 in 2024-03-30T02:35:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:28 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2941286726' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8, 5, 10]}]': finished 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: osdmap e84: 12 total, 12 up, 12 in 2024-03-30T02:35:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:28 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2941286726' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-30T02:35:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.467+0000 7f078ec00700 1 -- 172.21.15.67:0/206199485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 msgr2=0x7f078810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:29.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.467+0000 7f078ec00700 1 --2- 172.21.15.67:0/206199485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 0x7f078810dc10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f078000b0d0 tx=0x7f078000b3e0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.467+0000 7f078ec00700 1 -- 172.21.15.67:0/206199485 shutdown_connections 2024-03-30T02:35:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.467+0000 7f078ec00700 1 --2- 172.21.15.67:0/206199485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 0x7f078810dc10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.467+0000 7f078ec00700 1 --2- 172.21.15.67:0/206199485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0788106ef0 0x7f07881072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.467+0000 7f078ec00700 1 --2- 172.21.15.67:0/206199485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788100e60 0x7f07881012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.467+0000 7f078ec00700 1 -- 172.21.15.67:0/206199485 >> 172.21.15.67:0/206199485 conn(0x7f0788077fa0 msgr2=0x7f07880783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.467+0000 7f078ec00700 1 -- 172.21.15.67:0/206199485 shutdown_connections 2024-03-30T02:35:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.467+0000 7f078ec00700 1 -- 172.21.15.67:0/206199485 wait complete. 2024-03-30T02:35:29.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.468+0000 7f078ec00700 1 Processor -- start 2024-03-30T02:35:29.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.468+0000 7f078ec00700 1 -- start start 2024-03-30T02:35:29.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f078ec00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0788100e60 0x7f07881994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:29.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f078ec00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 0x7f0788199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:29.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f078ec00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788106ef0 0x7f078819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:29.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f078ec00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078819e3e0 con 0x7f0788101820 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f078ec00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078819e550 con 0x7f0788100e60 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f078ec00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078819e710 con 0x7f0788106ef0 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f0787400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 0x7f0788199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f0787400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 0x7f0788199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53770/0 (socket says 172.21.15.67:53770) 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f0787400700 1 -- 172.21.15.67:0/3316293718 learned_addr learned my addr 172.21.15.67:0/3316293718 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f0787e00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0788100e60 0x7f07881994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f078cc00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788106ef0 0x7f078819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f0787400700 1 -- 172.21.15.67:0/3316293718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788106ef0 msgr2=0x7f078819ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f0787400700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788106ef0 0x7f078819ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.469+0000 7f0787400700 1 -- 172.21.15.67:0/3316293718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0788100e60 msgr2=0x7f07881994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.470+0000 7f0787400700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0788100e60 0x7f07881994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.470+0000 7f0787400700 1 -- 172.21.15.67:0/3316293718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780009d00 con 0x7f0788101820 2024-03-30T02:35:29.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.470+0000 7f0787e00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0788100e60 0x7f07881994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:29.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.470+0000 7f0787400700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 0x7f0788199a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f077400e9a0 tx=0x7f077400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:29.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.470+0000 7f0784c00700 1 -- 172.21.15.67:0/3316293718 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774009790 con 0x7f0788101820 2024-03-30T02:35:29.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.470+0000 7f0784c00700 1 -- 172.21.15.67:0/3316293718 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0774010430 con 0x7f0788101820 2024-03-30T02:35:29.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.470+0000 7f0784c00700 1 -- 172.21.15.67:0/3316293718 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774018680 con 0x7f0788101820 2024-03-30T02:35:29.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.470+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078819e9f0 con 0x7f0788101820 2024-03-30T02:35:29.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.470+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07881a6250 con 0x7f0788101820 2024-03-30T02:35:29.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.471+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07881024f0 con 0x7f0788101820 2024-03-30T02:35:29.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.473+0000 7f0784c00700 1 -- 172.21.15.67:0/3316293718 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f07740105a0 con 0x7f0788101820 2024-03-30T02:35:29.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.475+0000 7f0784c00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f076c0a1230 0x7f076c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:29.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.475+0000 7f0784c00700 1 -- 172.21.15.67:0/3316293718 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0774014070 con 0x7f0788101820 2024-03-30T02:35:29.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.476+0000 7f0787e00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f076c0a1230 0x7f076c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:29.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.476+0000 7f0787e00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f076c0a1230 0x7f076c0a36f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0778011490 tx=0x7f0778009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:29.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.480+0000 7f0784c00700 1 -- 172.21.15.67:0/3316293718 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f07740529f0 con 0x7f0788101820 2024-03-30T02:35:29.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:29 smithi082 ceph-mon[79176]: osdmap e85: 12 total, 12 up, 12 in 2024-03-30T02:35:29.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:29 smithi082 ceph-mon[79176]: pgmap v256: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-03-30T02:35:29.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:29 smithi067 ceph-mon[37808]: osdmap e85: 12 total, 12 up, 12 in 2024-03-30T02:35:29.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:29 smithi067 ceph-mon[37808]: pgmap v256: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-03-30T02:35:29.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:29 smithi170 ceph-mon[78798]: osdmap e85: 12 total, 12 up, 12 in 2024-03-30T02:35:29.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:29 smithi170 ceph-mon[78798]: pgmap v256: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-03-30T02:35:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.691+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f078804ea80 con 0x7f0788101820 2024-03-30T02:35:29.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.693+0000 7f0784c00700 1 -- 172.21.15.67:0/3316293718 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24368 (secure 0 0 0) 0x7f07740961d0 con 0x7f0788101820 2024-03-30T02:35:29.693 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:29.693 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:35:28.198711+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:35:29.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f076c0a1230 msgr2=0x7f076c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:29.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f076c0a1230 0x7f076c0a36f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0778011490 tx=0x7f0778009260 comp rx=0 tx=0).stop 2024-03-30T02:35:29.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 msgr2=0x7f0788199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:29.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 0x7f0788199a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f077400e9a0 tx=0x7f077400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 shutdown_connections 2024-03-30T02:35:29.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f076c0a1230 0x7f076c0a36f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788106ef0 0x7f078819ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0788100e60 0x7f07881994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 --2- 172.21.15.67:0/3316293718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0788101820 0x7f0788199a20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:29.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 >> 172.21.15.67:0/3316293718 conn(0x7f0788077fa0 msgr2=0x7f078810c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:29.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.695+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 shutdown_connections 2024-03-30T02:35:29.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:29.696+0000 7f078ec00700 1 -- 172.21.15.67:0/3316293718 wait complete. 2024-03-30T02:35:30.119 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-30T02:35:30.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-03-30T02:35:30.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:30 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3316293718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:30.490 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:30.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:30 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3316293718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:30 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3316293718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:31.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:31 smithi067 ceph-mon[37808]: pgmap v257: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-03-30T02:35:31.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:31 smithi082 ceph-mon[79176]: pgmap v257: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-03-30T02:35:31.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:31 smithi170 ceph-mon[78798]: pgmap v257: 129 pgs: 1 peering, 128 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.9 KiB/s wr, 16 op/s 2024-03-30T02:35:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.692+0000 7f1f20600700 1 -- 172.21.15.67:0/1959037713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f18101820 msgr2=0x7f1f1810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.692+0000 7f1f20600700 1 --2- 172.21.15.67:0/1959037713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f18101820 0x7f1f1810dc10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000b0d0 tx=0x7f1f1000b3e0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.692+0000 7f1f20600700 1 -- 172.21.15.67:0/1959037713 shutdown_connections 2024-03-30T02:35:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.692+0000 7f1f20600700 1 --2- 172.21.15.67:0/1959037713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f18106ef0 0x7f1f181072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.692+0000 7f1f20600700 1 --2- 172.21.15.67:0/1959037713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f18101820 0x7f1f1810dc10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.692+0000 7f1f20600700 1 --2- 172.21.15.67:0/1959037713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f18100e60 0x7f1f181012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.692+0000 7f1f20600700 1 -- 172.21.15.67:0/1959037713 >> 172.21.15.67:0/1959037713 conn(0x7f1f18077fa0 msgr2=0x7f1f180783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.692+0000 7f1f20600700 1 -- 172.21.15.67:0/1959037713 shutdown_connections 2024-03-30T02:35:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.692+0000 7f1f20600700 1 -- 172.21.15.67:0/1959037713 wait complete. 2024-03-30T02:35:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.694+0000 7f1f20600700 1 Processor -- start 2024-03-30T02:35:31.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.694+0000 7f1f20600700 1 -- start start 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.694+0000 7f1f20600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f18100e60 0x7f1f180730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.694+0000 7f1f20600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f18101820 0x7f1f180735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f20600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f18106ef0 0x7f1f18073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f20600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181a5720 con 0x7f1f18106ef0 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f20600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181a5890 con 0x7f1f18101820 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f20600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f181a5a50 con 0x7f1f18100e60 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1e600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f18106ef0 0x7f1f18073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1dc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f18100e60 0x7f1f180730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1e600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f18106ef0 0x7f1f18073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53784/0 (socket says 172.21.15.67:53784) 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1e600700 1 -- 172.21.15.67:0/1744562878 learned_addr learned my addr 172.21.15.67:0/1744562878 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1d200700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f18101820 0x7f1f180735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:31.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1e600700 1 -- 172.21.15.67:0/1744562878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f18100e60 msgr2=0x7f1f180730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1e600700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f18100e60 0x7f1f180730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1e600700 1 -- 172.21.15.67:0/1744562878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f18101820 msgr2=0x7f1f180735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1e600700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f18101820 0x7f1f180735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1e600700 1 -- 172.21.15.67:0/1744562878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f10009d00 con 0x7f1f18106ef0 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.695+0000 7f1f1dc00700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f18100e60 0x7f1f180730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.696+0000 7f1f1e600700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f18106ef0 0x7f1f18073b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000bcd0 tx=0x7f1f1000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.696+0000 7f1f16000700 1 -- 172.21.15.67:0/1744562878 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000e040 con 0x7f1f18106ef0 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.696+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f181a5cd0 con 0x7f1f18106ef0 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.696+0000 7f1f16000700 1 -- 172.21.15.67:0/1744562878 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f10005070 con 0x7f1f18106ef0 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.696+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f181a6140 con 0x7f1f18106ef0 2024-03-30T02:35:31.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.696+0000 7f1f16000700 1 -- 172.21.15.67:0/1744562878 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1001b660 con 0x7f1f18106ef0 2024-03-30T02:35:31.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.697+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f181092f0 con 0x7f1f18106ef0 2024-03-30T02:35:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.699+0000 7f1f16000700 1 -- 172.21.15.67:0/1744562878 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1f100051e0 con 0x7f1f18106ef0 2024-03-30T02:35:31.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.700+0000 7f1f16000700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1f000a11b0 0x7f1f000a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:31.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.700+0000 7f1f16000700 1 -- 172.21.15.67:0/1744562878 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1f100cd750 con 0x7f1f18106ef0 2024-03-30T02:35:31.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.701+0000 7f1f1dc00700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1f000a11b0 0x7f1f000a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:31.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.701+0000 7f1f1dc00700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1f000a11b0 0x7f1f000a3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009fd0 tx=0x7f1f0c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:31.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.705+0000 7f1f16000700 1 -- 172.21.15.67:0/1744562878 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1f10096ab0 con 0x7f1f18106ef0 2024-03-30T02:35:31.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.918+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f181024f0 con 0x7f1f18106ef0 2024-03-30T02:35:31.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.920+0000 7f1f16000700 1 -- 172.21.15.67:0/1744562878 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24368 (secure 0 0 0) 0x7f1f1009a0d0 con 0x7f1f18106ef0 2024-03-30T02:35:31.920 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:31.921 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:35:28.198711+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1f000a11b0 msgr2=0x7f1f000a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1f000a11b0 0x7f1f000a3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009fd0 tx=0x7f1f0c009260 comp rx=0 tx=0).stop 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f18106ef0 msgr2=0x7f1f18073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f18106ef0 0x7f1f18073b30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000bcd0 tx=0x7f1f1000bd00 comp rx=0 tx=0).stop 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 shutdown_connections 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1f000a11b0 0x7f1f000a3670 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1f18106ef0 0x7f1f18073b30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f18100e60 0x7f1f180730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 --2- 172.21.15.67:0/1744562878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f18101820 0x7f1f180735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 >> 172.21.15.67:0/1744562878 conn(0x7f1f18077fa0 msgr2=0x7f1f180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:31.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 shutdown_connections 2024-03-30T02:35:31.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:31.923+0000 7f1f20600700 1 -- 172.21.15.67:0/1744562878 wait complete. 2024-03-30T02:35:32.331 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-03-30T02:35:32.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-03-30T02:35:32.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-03-30T02:35:32.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-03-30T02:35:32.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-03-30T02:35:32.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-03-30T02:35:32.334 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.6 flush_pg_stats 2024-03-30T02:35:32.334 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.7 flush_pg_stats 2024-03-30T02:35:32.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.8 flush_pg_stats 2024-03-30T02:35:32.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.9 flush_pg_stats 2024-03-30T02:35:32.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.10 flush_pg_stats 2024-03-30T02:35:32.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph tell osd.11 flush_pg_stats 2024-03-30T02:35:32.477 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:32 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1744562878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:32.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:32 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1744562878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:32 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1744562878' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:35:33.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:33 smithi082 ceph-mon[79176]: pgmap v258: 129 pgs: 129 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:33.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:33 smithi067 ceph-mon[37808]: pgmap v258: 129 pgs: 129 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:33.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:33 smithi170 ceph-mon[78798]: pgmap v258: 129 pgs: 129 active+clean; 637 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:34.176 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:34.940 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:34.940 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:34.996 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:36.303 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:36.320 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:36 smithi067 ceph-mon[37808]: pgmap v259: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:36.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:36 smithi170 ceph-mon[78798]: pgmap v259: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:36 smithi082 ceph-mon[79176]: pgmap v259: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.2 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:36.858 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.885+0000 7fb1f7a00700 1 -- 172.21.15.67:0/1137287895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f00690f0 msgr2=0x7fb1f010dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.885+0000 7fb1f7a00700 1 --2- 172.21.15.67:0/1137287895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f00690f0 0x7fb1f010dc30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800b0a0 tx=0x7fb1e800b3b0 comp rx=0 tx=0).stop 2024-03-30T02:35:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.886+0000 7fb1f7a00700 1 -- 172.21.15.67:0/1137287895 shutdown_connections 2024-03-30T02:35:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.886+0000 7fb1f7a00700 1 --2- 172.21.15.67:0/1137287895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f0106df0 0x7fb1f01071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.886+0000 7fb1f7a00700 1 --2- 172.21.15.67:0/1137287895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f00690f0 0x7fb1f010dc30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.886+0000 7fb1f7a00700 1 --2- 172.21.15.67:0/1137287895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f0068750 0x7fb1f0068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.886+0000 7fb1f7a00700 1 -- 172.21.15.67:0/1137287895 >> 172.21.15.67:0/1137287895 conn(0x7fb1f0075d50 msgr2=0x7fb1f0076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.886+0000 7fb1f7a00700 1 -- 172.21.15.67:0/1137287895 shutdown_connections 2024-03-30T02:35:36.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.886+0000 7fb1f7a00700 1 -- 172.21.15.67:0/1137287895 wait complete. 2024-03-30T02:35:36.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.887+0000 7fb1f7a00700 1 Processor -- start 2024-03-30T02:35:36.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.887+0000 7fb1f7a00700 1 -- start start 2024-03-30T02:35:36.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f7a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f0068750 0x7fb1f00ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:36.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f7a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f00690f0 0x7fb1f0100340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:36.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f7a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f0106df0 0x7fb1f0100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:36.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f7a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f01a5690 con 0x7fb1f00690f0 2024-03-30T02:35:36.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f7a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f01a5800 con 0x7fb1f0106df0 2024-03-30T02:35:36.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f7a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f01a59c0 con 0x7fb1f0068750 2024-03-30T02:35:36.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1efe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f00690f0 0x7fb1f0100340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:36.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f5a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f0106df0 0x7fb1f0100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:36.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f5a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f0106df0 0x7fb1f0100880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:47952/0 (socket says 172.21.15.67:47952) 2024-03-30T02:35:36.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1efe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f00690f0 0x7fb1f0100340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38196/0 (socket says 172.21.15.67:38196) 2024-03-30T02:35:36.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f5a00700 1 -- 172.21.15.67:0/4071450129 learned_addr learned my addr 172.21.15.67:0/4071450129 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:36.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f5000700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f0068750 0x7fb1f00ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:36.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f5a00700 1 -- 172.21.15.67:0/4071450129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f0068750 msgr2=0x7fb1f00ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f5a00700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1f0068750 0x7fb1f00ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f5a00700 1 -- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f00690f0 msgr2=0x7fb1f0100340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f5a00700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f00690f0 0x7fb1f0100340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1f5a00700 1 -- 172.21.15.67:0/4071450129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e4009520 con 0x7fb1f0106df0 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.888+0000 7fb1efe00700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1f00690f0 0x7fb1f0100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.889+0000 7fb1f5a00700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f0106df0 0x7fb1f0100880 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800b070 tx=0x7fb1e80078e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.889+0000 7fb1ed600700 1 -- 172.21.15.67:0/4071450129 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e800e040 con 0x7fb1f0106df0 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.889+0000 7fb1ed600700 1 -- 172.21.15.67:0/4071450129 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1e801d470 con 0x7fb1f0106df0 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.889+0000 7fb1ed600700 1 -- 172.21.15.67:0/4071450129 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e80052a0 con 0x7fb1f0106df0 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.889+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e8009d00 con 0x7fb1f0106df0 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.889+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f0197490 con 0x7fb1f0106df0 2024-03-30T02:35:36.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.891+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb1f0110f30 con 0x7fb1f0106df0 2024-03-30T02:35:36.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.895+0000 7fb1ed600700 1 -- 172.21.15.67:0/4071450129 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb1e801d5e0 con 0x7fb1f0106df0 2024-03-30T02:35:36.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.897+0000 7fb1ed600700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a1300 0x7fb1d80a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:36.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.897+0000 7fb1ed600700 1 -- 172.21.15.67:0/4071450129 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb1e80ce2c0 con 0x7fb1f0106df0 2024-03-30T02:35:36.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.898+0000 7fb1f5000700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a1300 0x7fb1d80a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:36.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.898+0000 7fb1ed600700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] conn(0x7fb1d80ac2c0 0x7fb1d80ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:36.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.898+0000 7fb1ed600700 1 -- 172.21.15.67:0/4071450129 --> [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1d80aee10 con 0x7fb1d80ac2c0 2024-03-30T02:35:36.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.898+0000 7fb1ed600700 1 -- 172.21.15.67:0/4071450129 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1e80ce6d0 con 0x7fb1f0106df0 2024-03-30T02:35:36.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.898+0000 7fb1efe00700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] conn(0x7fb1d80ac2c0 0x7fb1d80ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:36.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.898+0000 7fb1f5000700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a1300 0x7fb1d80a37c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009d70 tx=0x7fb1e4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:36.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.899+0000 7fb1efe00700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] conn(0x7fb1d80ac2c0 0x7fb1d80ae740 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:36.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.900+0000 7fb1ed600700 1 -- 172.21.15.67:0/4071450129 <== osd.0 v2:172.21.15.67:6802/1668225002 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb1d80aee10 con 0x7fb1d80ac2c0 2024-03-30T02:35:36.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.919+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 --> [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb1f00614b0 con 0x7fb1d80ac2c0 2024-03-30T02:35:36.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.920+0000 7fb1ed600700 1 -- 172.21.15.67:0/4071450129 <== osd.0 v2:172.21.15.67:6802/1668225002 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb1f00614b0 con 0x7fb1d80ac2c0 2024-03-30T02:35:36.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.921+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] conn(0x7fb1d80ac2c0 msgr2=0x7fb1d80ae740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.921+0000 7fb1f7a00700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] conn(0x7fb1d80ac2c0 0x7fb1d80ae740 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.921+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a1300 msgr2=0x7fb1d80a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.921+0000 7fb1f7a00700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb1d80a1300 0x7fb1d80a37c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009d70 tx=0x7fb1e4009260 comp rx=0 tx=0).stop 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.921+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f0106df0 msgr2=0x7fb1f0100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.921+0000 7fb1f7a00700 1 --2- 172.21.15.67:0/4071450129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1f0106df0 0x7fb1f0100880 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800b070 tx=0x7fb1e80078e0 comp rx=0 tx=0).stop 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.921+0000 7fb1f5a00700 1 -- 172.21.15.67:0/4071450129 reap_dead start 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.922+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 shutdown_connections 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.922+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 >> 172.21.15.67:0/4071450129 conn(0x7fb1f0075d50 msgr2=0x7fb1f00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.922+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 shutdown_connections 2024-03-30T02:35:36.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:36.922+0000 7fb1f7a00700 1 -- 172.21.15.67:0/4071450129 wait complete. 2024-03-30T02:35:36.923 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:37.410 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:37.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:37 smithi067 ceph-mon[37808]: pgmap v260: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:37.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:37 smithi082 ceph-mon[79176]: pgmap v260: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:37.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:37 smithi170 ceph-mon[78798]: pgmap v260: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:37.723 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.800+0000 7fa506200700 1 -- 172.21.15.67:0/952621297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50010d810 msgr2=0x7fa50010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.800+0000 7fa506200700 1 --2- 172.21.15.67:0/952621297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50010d810 0x7fa50010fc00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009f90 tx=0x7fa4f800b330 comp rx=0 tx=0).stop 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.800+0000 7fa506200700 1 -- 172.21.15.67:0/952621297 shutdown_connections 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.800+0000 7fa506200700 1 --2- 172.21.15.67:0/952621297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50010d810 0x7fa50010fc00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.800+0000 7fa506200700 1 --2- 172.21.15.67:0/952621297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa500072fe0 0x7fa5000733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.800+0000 7fa506200700 1 --2- 172.21.15.67:0/952621297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa500073900 0x7fa50010d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.800+0000 7fa506200700 1 -- 172.21.15.67:0/952621297 >> 172.21.15.67:0/952621297 conn(0x7fa500077fa0 msgr2=0x7fa5000783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.801+0000 7fa506200700 1 -- 172.21.15.67:0/952621297 shutdown_connections 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.801+0000 7fa506200700 1 -- 172.21.15.67:0/952621297 wait complete. 2024-03-30T02:35:37.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa506200700 1 Processor -- start 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa506200700 1 -- start start 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa506200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa500072fe0 0x7fa500199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa506200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa500073900 0x7fa500199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa506200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50010d810 0x7fa50019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa506200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50019e520 con 0x7fa50010d810 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa506200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50019e690 con 0x7fa500073900 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa506200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50019e850 con 0x7fa500072fe0 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa4ffe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50010d810 0x7fa50019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.802+0000 7fa4ff400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa500072fe0 0x7fa500199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4ffe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50010d810 0x7fa50019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38214/0 (socket says 172.21.15.67:38214) 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4ffe00700 1 -- 172.21.15.67:0/3207228652 learned_addr learned my addr 172.21.15.67:0/3207228652 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4ff400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa500072fe0 0x7fa500199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:53534/0 (socket says 172.21.15.67:53534) 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4fea00700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa500073900 0x7fa500199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:37.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4fea00700 1 -- 172.21.15.67:0/3207228652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa500072fe0 msgr2=0x7fa500199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4fea00700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa500072fe0 0x7fa500199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4fea00700 1 -- 172.21.15.67:0/3207228652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50010d810 msgr2=0x7fa50019df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4fea00700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa50010d810 0x7fa50019df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4fea00700 1 -- 172.21.15.67:0/3207228652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8009c40 con 0x7fa500073900 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4ff400700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa500072fe0 0x7fa500199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.803+0000 7fa4fea00700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa500073900 0x7fa500199b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00e9c0 tx=0x7fa4ec00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.804+0000 7fa4f7e00700 1 -- 172.21.15.67:0/3207228652 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00c9c0 con 0x7fa500073900 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.804+0000 7fa4f7e00700 1 -- 172.21.15.67:0/3207228652 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4ec010430 con 0x7fa500073900 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.804+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50019eb30 con 0x7fa500073900 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.804+0000 7fa4f7e00700 1 -- 172.21.15.67:0/3207228652 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec018680 con 0x7fa500073900 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.804+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5001a6390 con 0x7fa500073900 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.805+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa5001132d0 con 0x7fa500073900 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.807+0000 7fa4f7e00700 1 -- 172.21.15.67:0/3207228652 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa4ec010860 con 0x7fa500073900 2024-03-30T02:35:37.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.809+0000 7fa4f7e00700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa4e40a11b0 0x7fa4e40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.809+0000 7fa4f7e00700 1 -- 172.21.15.67:0/3207228652 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa4ec014070 con 0x7fa500073900 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.809+0000 7fa4ff400700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa4e40a11b0 0x7fa4e40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.809+0000 7fa4f7e00700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] conn(0x7fa4e40ac170 0x7fa4e40ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.809+0000 7fa4f7e00700 1 -- 172.21.15.67:0/3207228652 --> [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4e40aecc0 con 0x7fa4e40ac170 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.809+0000 7fa4f7e00700 1 -- 172.21.15.67:0/3207228652 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4ec0d12a0 con 0x7fa500073900 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.810+0000 7fa4ff400700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa4e40a11b0 0x7fa4e40a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0011490 tx=0x7fa4f0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.810+0000 7fa4ffe00700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] conn(0x7fa4e40ac170 0x7fa4e40ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.810+0000 7fa4ffe00700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] conn(0x7fa4e40ac170 0x7fa4e40ae5f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.811+0000 7fa4f7e00700 1 -- 172.21.15.67:0/3207228652 <== osd.11 v2:172.21.15.170:6824/3333975805 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa4e40aecc0 con 0x7fa4e40ac170 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.829+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 --> [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa500061430 con 0x7fa4e40ac170 2024-03-30T02:35:37.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa4f7e00700 1 -- 172.21.15.67:0/3207228652 <== osd.11 v2:172.21.15.170:6824/3333975805 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa500061430 con 0x7fa4e40ac170 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 >> [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] conn(0x7fa4e40ac170 msgr2=0x7fa4e40ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa506200700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] conn(0x7fa4e40ac170 0x7fa4e40ae5f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa4e40a11b0 msgr2=0x7fa4e40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa506200700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa4e40a11b0 0x7fa4e40a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0011490 tx=0x7fa4f0009260 comp rx=0 tx=0).stop 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa500073900 msgr2=0x7fa500199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa506200700 1 --2- 172.21.15.67:0/3207228652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa500073900 0x7fa500199b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00e9c0 tx=0x7fa4ec00ecd0 comp rx=0 tx=0).stop 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa4ffe00700 1 -- 172.21.15.67:0/3207228652 reap_dead start 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 shutdown_connections 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.830+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 >> 172.21.15.67:0/3207228652 conn(0x7fa500077fa0 msgr2=0x7fa5000fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.831+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 shutdown_connections 2024-03-30T02:35:37.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:37.831+0000 7fa506200700 1 -- 172.21.15.67:0/3207228652 wait complete. 2024-03-30T02:35:38.327 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:38.330 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:38.667 INFO:teuthology.orchestra.run.smithi067.stdout:317827579924 2024-03-30T02:35:38.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.11 2024-03-30T02:35:38.847 INFO:teuthology.orchestra.run.smithi067.stdout:34359738429 2024-03-30T02:35:38.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-03-30T02:35:38.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.941+0000 7f6586800700 1 -- 172.21.15.67:0/2911953268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6580100e60 msgr2=0x7f65801012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:38.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.941+0000 7f6586800700 1 --2- 172.21.15.67:0/2911953268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6580100e60 0x7f65801012e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f656c009920 tx=0x7f656c009c30 comp rx=0 tx=0).stop 2024-03-30T02:35:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.941+0000 7f6586800700 1 -- 172.21.15.67:0/2911953268 shutdown_connections 2024-03-30T02:35:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.941+0000 7f6586800700 1 --2- 172.21.15.67:0/2911953268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6580101820 0x7f658010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.941+0000 7f6586800700 1 --2- 172.21.15.67:0/2911953268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6580106ef0 0x7f65801072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.941+0000 7f6586800700 1 --2- 172.21.15.67:0/2911953268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6580100e60 0x7f65801012e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.941+0000 7f6586800700 1 -- 172.21.15.67:0/2911953268 >> 172.21.15.67:0/2911953268 conn(0x7f6580077fa0 msgr2=0x7f65800783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.941+0000 7f6586800700 1 -- 172.21.15.67:0/2911953268 shutdown_connections 2024-03-30T02:35:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.941+0000 7f6586800700 1 -- 172.21.15.67:0/2911953268 wait complete. 2024-03-30T02:35:38.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.942+0000 7f6586800700 1 Processor -- start 2024-03-30T02:35:38.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.942+0000 7f6586800700 1 -- start start 2024-03-30T02:35:38.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f6586800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6580100e60 0x7f6580195090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:38.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f6586800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6580101820 0x7f65801955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:38.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f6586800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6580106ef0 0x7f6580199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:38.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f6586800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6580199f90 con 0x7f6580106ef0 2024-03-30T02:35:38.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f6586800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658019a100 con 0x7f6580100e60 2024-03-30T02:35:38.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f6586800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658019a2c0 con 0x7f6580101820 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f657f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6580100e60 0x7f6580195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f657fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6580106ef0 0x7f6580199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f657f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6580100e60 0x7f6580195090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:47974/0 (socket says 172.21.15.67:47974) 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f657f400700 1 -- 172.21.15.67:0/4283763201 learned_addr learned my addr 172.21.15.67:0/4283763201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.943+0000 7f657ea00700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6580101820 0x7f65801955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.944+0000 7f657ea00700 1 -- 172.21.15.67:0/4283763201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6580100e60 msgr2=0x7f6580195090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.944+0000 7f657ea00700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6580100e60 0x7f6580195090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.944+0000 7f657ea00700 1 -- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6580106ef0 msgr2=0x7f6580199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.944+0000 7f657ea00700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6580106ef0 0x7f6580199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.944+0000 7f657ea00700 1 -- 172.21.15.67:0/4283763201 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c009520 con 0x7f6580101820 2024-03-30T02:35:38.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.944+0000 7f657fe00700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6580106ef0 0x7f6580199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:38.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.944+0000 7f657ea00700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6580101820 0x7f65801955d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f656c000c00 tx=0x7f656c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:38.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.945+0000 7f6577e00700 1 -- 172.21.15.67:0/4283763201 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c01c070 con 0x7f6580101820 2024-03-30T02:35:38.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.945+0000 7f6577e00700 1 -- 172.21.15.67:0/4283763201 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f656c00ba20 con 0x7f6580101820 2024-03-30T02:35:38.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.945+0000 7f6577e00700 1 -- 172.21.15.67:0/4283763201 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c004930 con 0x7f6580101820 2024-03-30T02:35:38.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.945+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658019a540 con 0x7f6580101820 2024-03-30T02:35:38.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.945+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65801a1c30 con 0x7f6580101820 2024-03-30T02:35:38.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.946+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f658010ef10 con 0x7f6580101820 2024-03-30T02:35:38.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.948+0000 7f6577e00700 1 -- 172.21.15.67:0/4283763201 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f656c020070 con 0x7f6580101820 2024-03-30T02:35:38.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.950+0000 7f6577e00700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f65640a1230 0x7f65640a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:38.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.951+0000 7f6577e00700 1 -- 172.21.15.67:0/4283763201 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f656c0cdeb0 con 0x7f6580101820 2024-03-30T02:35:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.951+0000 7f657f400700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f65640a1230 0x7f65640a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.951+0000 7f6577e00700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] conn(0x7f65640ac1f0 0x7f65640ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.951+0000 7f6577e00700 1 -- 172.21.15.67:0/4283763201 --> [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65640aed40 con 0x7f65640ac1f0 2024-03-30T02:35:38.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.951+0000 7f6577e00700 1 -- 172.21.15.67:0/4283763201 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f656c0ce220 con 0x7f6580101820 2024-03-30T02:35:38.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.951+0000 7f657fe00700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] conn(0x7f65640ac1f0 0x7f65640ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:38.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.951+0000 7f657f400700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f65640a1230 0x7f65640a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6570005950 tx=0x7f657000f040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:38.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.952+0000 7f657fe00700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] conn(0x7f65640ac1f0 0x7f65640ae670 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:38.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.953+0000 7f6577e00700 1 -- 172.21.15.67:0/4283763201 <== osd.6 v2:172.21.15.67:6818/3459663543 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f65640aed40 con 0x7f65640ac1f0 2024-03-30T02:35:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.970+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 --> [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6580061430 con 0x7f65640ac1f0 2024-03-30T02:35:38.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.971+0000 7f6577e00700 1 -- 172.21.15.67:0/4283763201 <== osd.6 v2:172.21.15.67:6818/3459663543 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6580061430 con 0x7f65640ac1f0 2024-03-30T02:35:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.971+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] conn(0x7f65640ac1f0 msgr2=0x7f65640ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:38.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.971+0000 7f6586800700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] conn(0x7f65640ac1f0 0x7f65640ae670 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.971+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f65640a1230 msgr2=0x7f65640a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.971+0000 7f6586800700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f65640a1230 0x7f65640a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6570005950 tx=0x7f657000f040 comp rx=0 tx=0).stop 2024-03-30T02:35:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.971+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6580101820 msgr2=0x7f65801955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.971+0000 7f6586800700 1 --2- 172.21.15.67:0/4283763201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6580101820 0x7f65801955d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f656c000c00 tx=0x7f656c00fa00 comp rx=0 tx=0).stop 2024-03-30T02:35:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.971+0000 7f657fe00700 1 -- 172.21.15.67:0/4283763201 reap_dead start 2024-03-30T02:35:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.972+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 shutdown_connections 2024-03-30T02:35:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.972+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 >> 172.21.15.67:0/4283763201 conn(0x7f6580077fa0 msgr2=0x7f65800ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.972+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 shutdown_connections 2024-03-30T02:35:38.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:38.972+0000 7f6586800700 1 -- 172.21.15.67:0/4283763201 wait complete. 2024-03-30T02:35:40.326 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:40 smithi067 ceph-mon[37808]: pgmap v261: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:40.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:40 smithi170 ceph-mon[78798]: pgmap v261: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:40.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:40 smithi082 ceph-mon[79176]: pgmap v261: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.577+0000 7f0fe5c00700 1 -- 172.21.15.67:0/1771840396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fe0100920 msgr2=0x7f0fe01048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.577+0000 7f0fe5c00700 1 --2- 172.21.15.67:0/1771840396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fe0100920 0x7f0fe01048d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009990 tx=0x7f0fcc009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.577+0000 7f0fe5c00700 1 -- 172.21.15.67:0/1771840396 shutdown_connections 2024-03-30T02:35:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.577+0000 7f0fe5c00700 1 --2- 172.21.15.67:0/1771840396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fe0104fb0 0x7f0fe010cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.577+0000 7f0fe5c00700 1 --2- 172.21.15.67:0/1771840396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fe00fff70 0x7f0fe0100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.577+0000 7f0fe5c00700 1 --2- 172.21.15.67:0/1771840396 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fe0100920 0x7f0fe01048d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.577+0000 7f0fe5c00700 1 -- 172.21.15.67:0/1771840396 >> 172.21.15.67:0/1771840396 conn(0x7f0fe0075d50 msgr2=0x7f0fe0076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.577+0000 7f0fe5c00700 1 -- 172.21.15.67:0/1771840396 shutdown_connections 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.577+0000 7f0fe5c00700 1 -- 172.21.15.67:0/1771840396 wait complete. 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.578+0000 7f0fe5c00700 1 Processor -- start 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.579+0000 7f0fe5c00700 1 -- start start 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.579+0000 7f0fe5c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fe00fff70 0x7f0fe0199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.579+0000 7f0fe5c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fe0100920 0x7f0fe0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.579+0000 7f0fe5c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fe0104fb0 0x7f0fe019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.579+0000 7f0fe5c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe019e350 con 0x7f0fe0104fb0 2024-03-30T02:35:40.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.579+0000 7f0fe5c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe019e4c0 con 0x7f0fe00fff70 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.579+0000 7f0fe5c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe019e680 con 0x7f0fe0100920 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fe0104fb0 0x7f0fe019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fe00fff70 0x7f0fe0199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fe0104fb0 0x7f0fe019dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38242/0 (socket says 172.21.15.67:38242) 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fe00fff70 0x7f0fe0199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:47992/0 (socket says 172.21.15.67:47992) 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdf400700 1 -- 172.21.15.67:0/18292282 learned_addr learned my addr 172.21.15.67:0/18292282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fde000700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fe0100920 0x7f0fe0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdea00700 1 -- 172.21.15.67:0/18292282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fe0100920 msgr2=0x7f0fe0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdea00700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0fe0100920 0x7f0fe0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdea00700 1 -- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fe0104fb0 msgr2=0x7f0fe019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdea00700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0fe0104fb0 0x7f0fe019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdea00700 1 -- 172.21.15.67:0/18292282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0095f0 con 0x7f0fe00fff70 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fdea00700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fe00fff70 0x7f0fe0199450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e9f0 tx=0x7f0fd000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.580+0000 7f0fd7400700 1 -- 172.21.15.67:0/18292282 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000c9c0 con 0x7f0fe00fff70 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.581+0000 7f0fd7400700 1 -- 172.21.15.67:0/18292282 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fd0010430 con 0x7f0fe00fff70 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.581+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe019e960 con 0x7f0fe00fff70 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.581+0000 7f0fd7400700 1 -- 172.21.15.67:0/18292282 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0018620 con 0x7f0fe00fff70 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.581+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe01a61c0 con 0x7f0fe00fff70 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.581+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0fe01013b0 con 0x7f0fe00fff70 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.584+0000 7f0fd7400700 1 -- 172.21.15.67:0/18292282 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f0fd0010860 con 0x7f0fe00fff70 2024-03-30T02:35:40.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.585+0000 7f0fd7400700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0fc40a11b0 0x7f0fc40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.585+0000 7f0fd7400700 1 -- 172.21.15.67:0/18292282 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f0fd0014070 con 0x7f0fe00fff70 2024-03-30T02:35:40.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.585+0000 7f0fde000700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0fc40a11b0 0x7f0fc40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.585+0000 7f0fd7400700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] conn(0x7f0fc40ac170 0x7f0fc40ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.585+0000 7f0fd7400700 1 -- 172.21.15.67:0/18292282 --> [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fc40aecc0 con 0x7f0fc40ac170 2024-03-30T02:35:40.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.585+0000 7f0fd7400700 1 -- 172.21.15.67:0/18292282 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fd00cd4d0 con 0x7f0fe00fff70 2024-03-30T02:35:40.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.586+0000 7f0fdf400700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] conn(0x7f0fc40ac170 0x7f0fc40ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.586+0000 7f0fde000700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0fc40a11b0 0x7f0fc40a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0fe019a9b0 tx=0x7f0fcc00b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:40.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.586+0000 7f0fdf400700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] conn(0x7f0fc40ac170 0x7f0fc40ae5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:40.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.588+0000 7f0fd7400700 1 -- 172.21.15.67:0/18292282 <== osd.3 v2:172.21.15.67:6810/2637526467 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f0fc40aecc0 con 0x7f0fc40ac170 2024-03-30T02:35:40.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.608+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 --> [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0fe010fe40 con 0x7f0fc40ac170 2024-03-30T02:35:40.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.608+0000 7f0fd7400700 1 -- 172.21.15.67:0/18292282 <== osd.3 v2:172.21.15.67:6810/2637526467 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0fe010fe40 con 0x7f0fc40ac170 2024-03-30T02:35:40.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] conn(0x7f0fc40ac170 msgr2=0x7f0fc40ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fe5c00700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] conn(0x7f0fc40ac170 0x7f0fc40ae5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0fc40a11b0 msgr2=0x7f0fc40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fe5c00700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0fc40a11b0 0x7f0fc40a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0fe019a9b0 tx=0x7f0fcc00b260 comp rx=0 tx=0).stop 2024-03-30T02:35:40.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fe00fff70 msgr2=0x7f0fe0199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fe5c00700 1 --2- 172.21.15.67:0/18292282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fe00fff70 0x7f0fe0199450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000e9f0 tx=0x7f0fd000ed00 comp rx=0 tx=0).stop 2024-03-30T02:35:40.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fdf400700 1 -- 172.21.15.67:0/18292282 reap_dead start 2024-03-30T02:35:40.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 shutdown_connections 2024-03-30T02:35:40.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 >> 172.21.15.67:0/18292282 conn(0x7f0fe0075d50 msgr2=0x7f0fe0106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:40.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.609+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 shutdown_connections 2024-03-30T02:35:40.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.610+0000 7f0fe5c00700 1 -- 172.21.15.67:0/18292282 wait complete. 2024-03-30T02:35:40.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.589+0000 7fc7e1200700 1 -- 172.21.15.67:0/973843728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc068750 msgr2=0x7fc7dc068b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.589+0000 7fc7e1200700 1 --2- 172.21.15.67:0/973843728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc068750 0x7fc7dc068b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0098f0 tx=0x7fc7cc009c00 comp rx=0 tx=0).stop 2024-03-30T02:35:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.589+0000 7fc7e1200700 1 -- 172.21.15.67:0/973843728 shutdown_connections 2024-03-30T02:35:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.589+0000 7fc7e1200700 1 --2- 172.21.15.67:0/973843728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc068750 0x7fc7dc068b30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.589+0000 7fc7e1200700 1 --2- 172.21.15.67:0/973843728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7dc10d8c0 0x7fc7dc10fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.589+0000 7fc7e1200700 1 --2- 172.21.15.67:0/973843728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7dc069070 0x7fc7dc10d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.589+0000 7fc7e1200700 1 -- 172.21.15.67:0/973843728 >> 172.21.15.67:0/973843728 conn(0x7fc7dc075d50 msgr2=0x7fc7dc076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.590+0000 7fc7e1200700 1 -- 172.21.15.67:0/973843728 shutdown_connections 2024-03-30T02:35:40.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.590+0000 7fc7e1200700 1 -- 172.21.15.67:0/973843728 wait complete. 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7e1200700 1 Processor -- start 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7e1200700 1 -- start start 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7e1200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7dc068750 0x7fc7dc199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7e1200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc069070 0x7fc7dc199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7e1200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7dc10d8c0 0x7fc7dc19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7e1200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc19e310 con 0x7fc7dc069070 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7e1200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc19e480 con 0x7fc7dc10d8c0 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7e1200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc19e640 con 0x7fc7dc068750 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7d9400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc069070 0x7fc7dc199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7d9e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7dc068750 0x7fc7dc199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7d9400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc069070 0x7fc7dc199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38270/0 (socket says 172.21.15.67:38270) 2024-03-30T02:35:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7d9400700 1 -- 172.21.15.67:0/3379742082 learned_addr learned my addr 172.21.15.67:0/3379742082 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.591+0000 7fc7da800700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7dc10d8c0 0x7fc7dc19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.592+0000 7fc7d9e00700 1 -- 172.21.15.67:0/3379742082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7dc10d8c0 msgr2=0x7fc7dc19dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.592+0000 7fc7d9e00700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7dc10d8c0 0x7fc7dc19dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.592+0000 7fc7d9e00700 1 -- 172.21.15.67:0/3379742082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc069070 msgr2=0x7fc7dc199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.592+0000 7fc7d9e00700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc7dc069070 0x7fc7dc199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.592+0000 7fc7d9e00700 1 -- 172.21.15.67:0/3379742082 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80095f0 con 0x7fc7dc068750 2024-03-30T02:35:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.592+0000 7fc7d9e00700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7dc068750 0x7fc7dc199410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009230 tx=0x7fc7cc00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:40.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.592+0000 7fc7d2a00700 1 -- 172.21.15.67:0/3379742082 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc01c070 con 0x7fc7dc068750 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.593+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7cc009520 con 0x7fc7dc068750 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.593+0000 7fc7d2a00700 1 -- 172.21.15.67:0/3379742082 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7cc00ba20 con 0x7fc7dc068750 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.593+0000 7fc7d2a00700 1 -- 172.21.15.67:0/3379742082 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0038e0 con 0x7fc7dc068750 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.593+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc1a5fb0 con 0x7fc7dc068750 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.594+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7dc19eba0 con 0x7fc7dc068750 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.596+0000 7fc7d2a00700 1 -- 172.21.15.67:0/3379742082 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc7cc020030 con 0x7fc7dc068750 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.597+0000 7fc7d2a00700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc7c00a1230 0x7fc7c00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.597+0000 7fc7d2a00700 1 -- 172.21.15.67:0/3379742082 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fc7cc0cdd50 con 0x7fc7dc068750 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.597+0000 7fc7d9400700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc7c00a1230 0x7fc7c00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.597+0000 7fc7d2a00700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] conn(0x7fc7c00ac1f0 0x7fc7c00ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:40.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.597+0000 7fc7d2a00700 1 -- 172.21.15.67:0/3379742082 --> [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7c00aed40 con 0x7fc7c00ac1f0 2024-03-30T02:35:40.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.597+0000 7fc7d2a00700 1 -- 172.21.15.67:0/3379742082 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7cc0ce0c0 con 0x7fc7dc068750 2024-03-30T02:35:40.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.598+0000 7fc7da800700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] conn(0x7fc7c00ac1f0 0x7fc7c00ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:40.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.598+0000 7fc7d9400700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc7c00a1230 0x7fc7c00a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc19a970 tx=0x7fc7c8009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:40.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.598+0000 7fc7da800700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] conn(0x7fc7c00ac1f0 0x7fc7c00ae670 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:40.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.599+0000 7fc7d2a00700 1 -- 172.21.15.67:0/3379742082 <== osd.2 v2:172.21.15.170:6800/229013363 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc7c00aed40 con 0x7fc7c00ac1f0 2024-03-30T02:35:40.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.619+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 --> [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc7dc1131a0 con 0x7fc7c00ac1f0 2024-03-30T02:35:40.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.620+0000 7fc7d2a00700 1 -- 172.21.15.67:0/3379742082 <== osd.2 v2:172.21.15.170:6800/229013363 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc7dc1131a0 con 0x7fc7c00ac1f0 2024-03-30T02:35:40.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 >> [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] conn(0x7fc7c00ac1f0 msgr2=0x7fc7c00ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] conn(0x7fc7c00ac1f0 0x7fc7c00ae670 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:40.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc7c00a1230 msgr2=0x7fc7c00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc7c00a1230 0x7fc7c00a36f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc19a970 tx=0x7fc7c8009310 comp rx=0 tx=0).stop 2024-03-30T02:35:40.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7dc068750 msgr2=0x7fc7dc199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:40.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 --2- 172.21.15.67:0/3379742082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7dc068750 0x7fc7dc199410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009230 tx=0x7fc7cc00fa00 comp rx=0 tx=0).stop 2024-03-30T02:35:40.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7da800700 1 -- 172.21.15.67:0/3379742082 reap_dead start 2024-03-30T02:35:40.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 shutdown_connections 2024-03-30T02:35:40.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 >> 172.21.15.67:0/3379742082 conn(0x7fc7dc075d50 msgr2=0x7fc7dc10ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:40.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 shutdown_connections 2024-03-30T02:35:40.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:40.621+0000 7fc7e1200700 1 -- 172.21.15.67:0/3379742082 wait complete. 2024-03-30T02:35:41.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.219+0000 7f6f5d000700 1 -- 172.21.15.67:0/3656180271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58106e20 msgr2=0x7f6f58107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.219+0000 7f6f5d000700 1 --2- 172.21.15.67:0/3656180271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58106e20 0x7f6f58107200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009990 tx=0x7f6f48009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.219+0000 7f6f5d000700 1 -- 172.21.15.67:0/3656180271 shutdown_connections 2024-03-30T02:35:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.219+0000 7f6f5d000700 1 --2- 172.21.15.67:0/3656180271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f580690f0 0x7f6f5810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.219+0000 7f6f5d000700 1 --2- 172.21.15.67:0/3656180271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58106e20 0x7f6f58107200 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.219+0000 7f6f5d000700 1 --2- 172.21.15.67:0/3656180271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f58068750 0x7f6f58068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.219+0000 7f6f5d000700 1 -- 172.21.15.67:0/3656180271 >> 172.21.15.67:0/3656180271 conn(0x7f6f58075d50 msgr2=0x7f6f58076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.219+0000 7f6f5d000700 1 -- 172.21.15.67:0/3656180271 shutdown_connections 2024-03-30T02:35:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.219+0000 7f6f5d000700 1 -- 172.21.15.67:0/3656180271 wait complete. 2024-03-30T02:35:41.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.221+0000 7f6f5d000700 1 Processor -- start 2024-03-30T02:35:41.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.221+0000 7f6f5d000700 1 -- start start 2024-03-30T02:35:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f5d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f58068750 0x7f6f580ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f5d000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f580690f0 0x7f6f58100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f5d000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58106e20 0x7f6f58100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f5d000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581a5600 con 0x7f6f580690f0 2024-03-30T02:35:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f5d000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581a5770 con 0x7f6f58068750 2024-03-30T02:35:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f5d000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581a5930 con 0x7f6f58106e20 2024-03-30T02:35:41.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f55e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f58068750 0x7f6f580ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f4fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f580690f0 0x7f6f58100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f56800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58106e20 0x7f6f58100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f4fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f580690f0 0x7f6f58100320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38276/0 (socket says 172.21.15.67:38276) 2024-03-30T02:35:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.222+0000 7f6f4fe00700 1 -- 172.21.15.67:0/428226077 learned_addr learned my addr 172.21.15.67:0/428226077 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.223+0000 7f6f56800700 1 -- 172.21.15.67:0/428226077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f58068750 msgr2=0x7f6f580ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.223+0000 7f6f56800700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f58068750 0x7f6f580ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.223+0000 7f6f56800700 1 -- 172.21.15.67:0/428226077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f580690f0 msgr2=0x7f6f58100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.223+0000 7f6f56800700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6f580690f0 0x7f6f58100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.223+0000 7f6f56800700 1 -- 172.21.15.67:0/428226077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480095f0 con 0x7f6f58106e20 2024-03-30T02:35:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.223+0000 7f6f55e00700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f58068750 0x7f6f580ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:35:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.223+0000 7f6f56800700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58106e20 0x7f6f58100860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000efc0 tx=0x7f6f5000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.223+0000 7f6f4ea00700 1 -- 172.21.15.67:0/428226077 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f50018070 con 0x7f6f58106e20 2024-03-30T02:35:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.224+0000 7f6f4ea00700 1 -- 172.21.15.67:0/428226077 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f5000f040 con 0x7f6f58106e20 2024-03-30T02:35:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.224+0000 7f6f4ea00700 1 -- 172.21.15.67:0/428226077 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5000ce10 con 0x7f6f58106e20 2024-03-30T02:35:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.224+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f581a5bb0 con 0x7f6f58106e20 2024-03-30T02:35:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.224+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f58197250 con 0x7f6f58106e20 2024-03-30T02:35:41.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.225+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f581024d0 con 0x7f6f58106e20 2024-03-30T02:35:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.227+0000 7f6f4ea00700 1 -- 172.21.15.67:0/428226077 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f6f50003a00 con 0x7f6f58106e20 2024-03-30T02:35:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.228+0000 7f6f4ea00700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f3c0a12b0 0x7f6f3c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.228+0000 7f6f4ea00700 1 -- 172.21.15.67:0/428226077 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6f500cc960 con 0x7f6f58106e20 2024-03-30T02:35:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.229+0000 7f6f4ea00700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] conn(0x7f6f3c0ac270 0x7f6f3c0ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.229+0000 7f6f4ea00700 1 -- 172.21.15.67:0/428226077 --> [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f3c0aedc0 con 0x7f6f3c0ac270 2024-03-30T02:35:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.229+0000 7f6f4ea00700 1 -- 172.21.15.67:0/428226077 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f500ccd10 con 0x7f6f58106e20 2024-03-30T02:35:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.229+0000 7f6f4fe00700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] conn(0x7f6f3c0ac270 0x7f6f3c0ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.229+0000 7f6f55e00700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f3c0a12b0 0x7f6f3c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.230+0000 7f6f4fe00700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] conn(0x7f6f3c0ac270 0x7f6f3c0ae6f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.230+0000 7f6f55e00700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f3c0a12b0 0x7f6f3c0a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f48006010 tx=0x7f6f4800b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.230+0000 7f6f4ea00700 1 -- 172.21.15.67:0/428226077 <== osd.8 v2:172.21.15.170:6816/4041482230 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f6f3c0aedc0 con 0x7f6f3c0ac270 2024-03-30T02:35:41.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.249+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 --> [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f58110f70 con 0x7f6f3c0ac270 2024-03-30T02:35:41.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f4ea00700 1 -- 172.21.15.67:0/428226077 <== osd.8 v2:172.21.15.170:6816/4041482230 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6f58110f70 con 0x7f6f3c0ac270 2024-03-30T02:35:41.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 >> [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] conn(0x7f6f3c0ac270 msgr2=0x7f6f3c0ae6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f5d000700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] conn(0x7f6f3c0ac270 0x7f6f3c0ae6f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f3c0a12b0 msgr2=0x7f6f3c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f5d000700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6f3c0a12b0 0x7f6f3c0a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6f48006010 tx=0x7f6f4800b260 comp rx=0 tx=0).stop 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58106e20 msgr2=0x7f6f58100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f5d000700 1 --2- 172.21.15.67:0/428226077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f58106e20 0x7f6f58100860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000efc0 tx=0x7f6f5000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f56800700 1 -- 172.21.15.67:0/428226077 reap_dead start 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 shutdown_connections 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.250+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 >> 172.21.15.67:0/428226077 conn(0x7f6f58075d50 msgr2=0x7f6f580fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.251+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 shutdown_connections 2024-03-30T02:35:41.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:41.251+0000 7f6f5d000700 1 -- 172.21.15.67:0/428226077 wait complete. 2024-03-30T02:35:41.837 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:42.081 INFO:teuthology.orchestra.run.smithi067.stdout:107374182450 2024-03-30T02:35:42.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-03-30T02:35:42.218 INFO:teuthology.orchestra.run.smithi067.stdout:188978561063 2024-03-30T02:35:42.218 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.6 2024-03-30T02:35:42.316 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:42 smithi067 ceph-mon[37808]: pgmap v262: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:42.413 INFO:teuthology.orchestra.run.smithi067.stdout:77309411383 2024-03-30T02:35:42.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-03-30T02:35:42.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:42 smithi170 ceph-mon[78798]: pgmap v262: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:42 smithi082 ceph-mon[79176]: pgmap v262: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:42.925 INFO:teuthology.orchestra.run.smithi067.stdout:236223201312 2024-03-30T02:35:42.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.8 2024-03-30T02:35:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.153+0000 7fb220c00700 1 -- 172.21.15.67:0/3127157303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb21c106df0 msgr2=0x7fb21c1071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.153+0000 7fb220c00700 1 --2- 172.21.15.67:0/3127157303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb21c106df0 0x7fb21c1071d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0098f0 tx=0x7fb20c009c00 comp rx=0 tx=0).stop 2024-03-30T02:35:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.153+0000 7fb220c00700 1 -- 172.21.15.67:0/3127157303 shutdown_connections 2024-03-30T02:35:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.153+0000 7fb220c00700 1 --2- 172.21.15.67:0/3127157303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb21c0690f0 0x7fb21c10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.153+0000 7fb220c00700 1 --2- 172.21.15.67:0/3127157303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb21c106df0 0x7fb21c1071d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.153+0000 7fb220c00700 1 --2- 172.21.15.67:0/3127157303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb21c068750 0x7fb21c068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.153+0000 7fb220c00700 1 -- 172.21.15.67:0/3127157303 >> 172.21.15.67:0/3127157303 conn(0x7fb21c075d50 msgr2=0x7fb21c076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.154+0000 7fb220c00700 1 -- 172.21.15.67:0/3127157303 shutdown_connections 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.154+0000 7fb220c00700 1 -- 172.21.15.67:0/3127157303 wait complete. 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.155+0000 7fb220c00700 1 Processor -- start 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.155+0000 7fb220c00700 1 -- start start 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.155+0000 7fb220c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb21c068750 0x7fb21c0ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.155+0000 7fb220c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb21c0690f0 0x7fb21c100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.155+0000 7fb220c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb21c106df0 0x7fb21c100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.155+0000 7fb220c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1a5650 con 0x7fb21c068750 2024-03-30T02:35:43.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.155+0000 7fb220c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1a57c0 con 0x7fb21c106df0 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.155+0000 7fb220c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1a5980 con 0x7fb21c0690f0 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.155+0000 7fb219e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb21c068750 0x7fb21c0ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb219e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb21c068750 0x7fb21c0ffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38288/0 (socket says 172.21.15.67:38288) 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb219e00700 1 -- 172.21.15.67:0/3050720181 learned_addr learned my addr 172.21.15.67:0/3050720181 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb219400700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb21c0690f0 0x7fb21c100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb21a800700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb21c106df0 0x7fb21c100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb219e00700 1 -- 172.21.15.67:0/3050720181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb21c0690f0 msgr2=0x7fb21c100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb219e00700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb21c0690f0 0x7fb21c100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb219e00700 1 -- 172.21.15.67:0/3050720181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb21c106df0 msgr2=0x7fb21c100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb219e00700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb21c106df0 0x7fb21c100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb219e00700 1 -- 172.21.15.67:0/3050720181 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c009520 con 0x7fb21c068750 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb21a800700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb21c106df0 0x7fb21c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.156+0000 7fb219400700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb21c0690f0 0x7fb21c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.157+0000 7fb219e00700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb21c068750 0x7fb21c0ffde0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb20c005950 tx=0x7fb20c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.157+0000 7fb212a00700 1 -- 172.21.15.67:0/3050720181 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c01c070 con 0x7fb21c068750 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.157+0000 7fb212a00700 1 -- 172.21.15.67:0/3050720181 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20c00bb20 con 0x7fb21c068750 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.157+0000 7fb212a00700 1 -- 172.21.15.67:0/3050720181 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c0038e0 con 0x7fb21c068750 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.157+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21c1a5c00 con 0x7fb21c068750 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.157+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21c1a5fc0 con 0x7fb21c068750 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.158+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb21c1024a0 con 0x7fb21c068750 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.160+0000 7fb212a00700 1 -- 172.21.15.67:0/3050720181 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb20c020030 con 0x7fb21c068750 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.162+0000 7fb212a00700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb2000a11b0 0x7fb2000a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.162+0000 7fb212a00700 1 -- 172.21.15.67:0/3050720181 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb20c0cdfe0 con 0x7fb21c068750 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.162+0000 7fb219400700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb2000a11b0 0x7fb2000a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.162+0000 7fb212a00700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] conn(0x7fb2000ac170 0x7fb2000ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.162+0000 7fb212a00700 1 -- 172.21.15.67:0/3050720181 --> [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2000aecc0 con 0x7fb2000ac170 2024-03-30T02:35:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.162+0000 7fb212a00700 1 -- 172.21.15.67:0/3050720181 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb20c0ce350 con 0x7fb21c068750 2024-03-30T02:35:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.163+0000 7fb21a800700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] conn(0x7fb2000ac170 0x7fb2000ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.163+0000 7fb219400700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb2000a11b0 0x7fb2000a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb21c1014c0 tx=0x7fb208009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.163+0000 7fb21a800700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] conn(0x7fb2000ac170 0x7fb2000ae5f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.164+0000 7fb212a00700 1 -- 172.21.15.67:0/3050720181 <== osd.5 v2:172.21.15.170:6808/2583913458 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb2000aecc0 con 0x7fb2000ac170 2024-03-30T02:35:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.188+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 --> [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb21c110f40 con 0x7fb2000ac170 2024-03-30T02:35:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb212a00700 1 -- 172.21.15.67:0/3050720181 <== osd.5 v2:172.21.15.170:6808/2583913458 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb21c110f40 con 0x7fb2000ac170 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 >> [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] conn(0x7fb2000ac170 msgr2=0x7fb2000ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb220c00700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] conn(0x7fb2000ac170 0x7fb2000ae5f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb2000a11b0 msgr2=0x7fb2000a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb220c00700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb2000a11b0 0x7fb2000a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb21c1014c0 tx=0x7fb208009310 comp rx=0 tx=0).stop 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb21c068750 msgr2=0x7fb21c0ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb220c00700 1 --2- 172.21.15.67:0/3050720181 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb21c068750 0x7fb21c0ffde0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb20c005950 tx=0x7fb20c00fa00 comp rx=0 tx=0).stop 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb21a800700 1 -- 172.21.15.67:0/3050720181 reap_dead start 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 shutdown_connections 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.189+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 >> 172.21.15.67:0/3050720181 conn(0x7fb21c075d50 msgr2=0x7fb21c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.190+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 shutdown_connections 2024-03-30T02:35:43.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.190+0000 7fb220c00700 1 -- 172.21.15.67:0/3050720181 wait complete. 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.212+0000 7f895c000700 1 -- 172.21.15.67:0/3084158034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8954103b90 msgr2=0x7f8954107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.212+0000 7f895c000700 1 --2- 172.21.15.67:0/3084158034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8954103b90 0x7f8954107be0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8940009990 tx=0x7f8940009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.213+0000 7f895c000700 1 -- 172.21.15.67:0/3084158034 shutdown_connections 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.213+0000 7f895c000700 1 --2- 172.21.15.67:0/3084158034 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89541031e0 0x7f89541035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.213+0000 7f895c000700 1 --2- 172.21.15.67:0/3084158034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89541082c0 0x7f895410fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.213+0000 7f895c000700 1 --2- 172.21.15.67:0/3084158034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8954103b90 0x7f8954107be0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.213+0000 7f895c000700 1 -- 172.21.15.67:0/3084158034 >> 172.21.15.67:0/3084158034 conn(0x7f89540fea70 msgr2=0x7f8954100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.213+0000 7f895c000700 1 -- 172.21.15.67:0/3084158034 shutdown_connections 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.213+0000 7f895c000700 1 -- 172.21.15.67:0/3084158034 wait complete. 2024-03-30T02:35:43.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.214+0000 7f895c000700 1 Processor -- start 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.214+0000 7f895c000700 1 -- start start 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.214+0000 7f895c000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89541031e0 0x7f8954199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.214+0000 7f895c000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8954103b90 0x7f8954199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.214+0000 7f895c000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89541082c0 0x7f895419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.214+0000 7f895c000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895419e3c0 con 0x7f89541082c0 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.214+0000 7f895c000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895419e530 con 0x7f8954103b90 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.214+0000 7f895c000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895419e6f0 con 0x7f89541031e0 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8958c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8954103b90 0x7f8954199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8959600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89541031e0 0x7f8954199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8958c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8954103b90 0x7f8954199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48062/0 (socket says 172.21.15.67:48062) 2024-03-30T02:35:43.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8958c00700 1 -- 172.21.15.67:0/3059986004 learned_addr learned my addr 172.21.15.67:0/3059986004 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f895a000700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89541082c0 0x7f895419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8959600700 1 -- 172.21.15.67:0/3059986004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8954103b90 msgr2=0x7f8954199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8959600700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8954103b90 0x7f8954199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8959600700 1 -- 172.21.15.67:0/3059986004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89541082c0 msgr2=0x7f895419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8959600700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89541082c0 0x7f895419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8959600700 1 -- 172.21.15.67:0/3059986004 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400095f0 con 0x7f89541031e0 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.215+0000 7f8958c00700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8954103b90 0x7f8954199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.216+0000 7f8959600700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89541031e0 0x7f8954199510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f894800e9f0 tx=0x7f894800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.216+0000 7f8952000700 1 -- 172.21.15.67:0/3059986004 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894800c9c0 con 0x7f89541031e0 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.216+0000 7f8952000700 1 -- 172.21.15.67:0/3059986004 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8948010430 con 0x7f89541031e0 2024-03-30T02:35:43.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.216+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895419e9d0 con 0x7f89541031e0 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.216+0000 7f8952000700 1 -- 172.21.15.67:0/3059986004 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948018680 con 0x7f89541031e0 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.216+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89541a6230 con 0x7f89541031e0 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.217+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89541131e0 con 0x7f89541031e0 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.219+0000 7f8952000700 1 -- 172.21.15.67:0/3059986004 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8948010860 con 0x7f89541031e0 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.220+0000 7f8952000700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f893c0a11b0 0x7f893c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.221+0000 7f8952000700 1 -- 172.21.15.67:0/3059986004 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8948014070 con 0x7f89541031e0 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.221+0000 7f8958c00700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f893c0a11b0 0x7f893c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.221+0000 7f8952000700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] conn(0x7f893c0ac170 0x7f893c0ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.221+0000 7f8952000700 1 -- 172.21.15.67:0/3059986004 --> [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f893c0aecc0 con 0x7f893c0ac170 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.221+0000 7f8952000700 1 -- 172.21.15.67:0/3059986004 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89480cd4e0 con 0x7f89541031e0 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.221+0000 7f8958c00700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f893c0a11b0 0x7f893c0a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f895419aa20 tx=0x7f894000b280 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.221+0000 7f895a000700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] conn(0x7f893c0ac170 0x7f893c0ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.222+0000 7f895a000700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] conn(0x7f893c0ac170 0x7f893c0ae5f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.223+0000 7f8952000700 1 -- 172.21.15.67:0/3059986004 <== osd.7 v2:172.21.15.82:6816/318348288 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f893c0aecc0 con 0x7f893c0ac170 2024-03-30T02:35:43.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.240+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 --> [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89541046c0 con 0x7f893c0ac170 2024-03-30T02:35:43.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.241+0000 7f8952000700 1 -- 172.21.15.67:0/3059986004 <== osd.7 v2:172.21.15.82:6816/318348288 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f89541046c0 con 0x7f893c0ac170 2024-03-30T02:35:43.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.241+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 >> [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] conn(0x7f893c0ac170 msgr2=0x7f893c0ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.241+0000 7f895c000700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] conn(0x7f893c0ac170 0x7f893c0ae5f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.242+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f893c0a11b0 msgr2=0x7f893c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.242+0000 7f895c000700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f893c0a11b0 0x7f893c0a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f895419aa20 tx=0x7f894000b280 comp rx=0 tx=0).stop 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.242+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89541031e0 msgr2=0x7f8954199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.242+0000 7f895c000700 1 --2- 172.21.15.67:0/3059986004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89541031e0 0x7f8954199510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f894800e9f0 tx=0x7f894800ed00 comp rx=0 tx=0).stop 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.242+0000 7f895a000700 1 -- 172.21.15.67:0/3059986004 reap_dead start 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.242+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 shutdown_connections 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.242+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 >> 172.21.15.67:0/3059986004 conn(0x7f89540fea70 msgr2=0x7f89540ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.242+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 shutdown_connections 2024-03-30T02:35:43.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.242+0000 7f895c000700 1 -- 172.21.15.67:0/3059986004 wait complete. 2024-03-30T02:35:43.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.690+0000 7f1651200700 1 -- 172.21.15.67:0/1998827601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f164c068c90 msgr2=0x7f164c069110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.690+0000 7f1651200700 1 --2- 172.21.15.67:0/1998827601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f164c068c90 0x7f164c069110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1638009990 tx=0x7f1638009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.690+0000 7f1651200700 1 -- 172.21.15.67:0/1998827601 shutdown_connections 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.690+0000 7f1651200700 1 --2- 172.21.15.67:0/1998827601 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f164c105cb0 0x7f164c068750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.690+0000 7f1651200700 1 --2- 172.21.15.67:0/1998827601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f164c103810 0x7f164c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.690+0000 7f1651200700 1 --2- 172.21.15.67:0/1998827601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f164c068c90 0x7f164c069110 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.690+0000 7f1651200700 1 -- 172.21.15.67:0/1998827601 >> 172.21.15.67:0/1998827601 conn(0x7f164c075d50 msgr2=0x7f164c076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.691+0000 7f1651200700 1 -- 172.21.15.67:0/1998827601 shutdown_connections 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.691+0000 7f1651200700 1 -- 172.21.15.67:0/1998827601 wait complete. 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.692+0000 7f1651200700 1 Processor -- start 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.692+0000 7f1651200700 1 -- start start 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f1651200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f164c068c90 0x7f164c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f1651200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f164c103810 0x7f164c19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f1651200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f164c105cb0 0x7f164c1a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f1651200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c1a15e0 con 0x7f164c068c90 2024-03-30T02:35:43.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f1651200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c1a1750 con 0x7f164c103810 2024-03-30T02:35:43.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f1651200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c1a1910 con 0x7f164c105cb0 2024-03-30T02:35:43.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f164a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f164c105cb0 0x7f164c1a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f164a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f164c105cb0 0x7f164c1a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:53644/0 (socket says 172.21.15.67:53644) 2024-03-30T02:35:43.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f164a800700 1 -- 172.21.15.67:0/1546813741 learned_addr learned my addr 172.21.15.67:0/1546813741 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:43.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.693+0000 7f1649e00700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f164c068c90 0x7f164c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.694+0000 7f164a800700 1 -- 172.21.15.67:0/1546813741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f164c103810 msgr2=0x7f164c19dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.694+0000 7f164a800700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f164c103810 0x7f164c19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.694+0000 7f164a800700 1 -- 172.21.15.67:0/1546813741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f164c068c90 msgr2=0x7f164c19d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.694+0000 7f164a800700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f164c068c90 0x7f164c19d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.694+0000 7f164a800700 1 -- 172.21.15.67:0/1546813741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16380095f0 con 0x7f164c105cb0 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.695+0000 7f164a800700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f164c105cb0 0x7f164c1a0fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f164400ef30 tx=0x7f164400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.695+0000 7f1642a00700 1 -- 172.21.15.67:0/1546813741 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644018070 con 0x7f164c105cb0 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.695+0000 7f1642a00700 1 -- 172.21.15.67:0/1546813741 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f164400f040 con 0x7f164c105cb0 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.695+0000 7f1642a00700 1 -- 172.21.15.67:0/1546813741 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164400cd50 con 0x7f164c105cb0 2024-03-30T02:35:43.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.695+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164c1a1b90 con 0x7f164c105cb0 2024-03-30T02:35:43.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.696+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164c06a9a0 con 0x7f164c105cb0 2024-03-30T02:35:43.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.697+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f164c117570 con 0x7f164c105cb0 2024-03-30T02:35:43.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.700+0000 7f1642a00700 1 -- 172.21.15.67:0/1546813741 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1644004ad0 con 0x7f164c105cb0 2024-03-30T02:35:43.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.702+0000 7f1642a00700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16300a1230 0x7f16300a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.702+0000 7f1642a00700 1 -- 172.21.15.67:0/1546813741 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f16440cc950 con 0x7f164c105cb0 2024-03-30T02:35:43.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.702+0000 7f1642a00700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] conn(0x7f16300ac1f0 0x7f16300ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.703+0000 7f1642a00700 1 -- 172.21.15.67:0/1546813741 --> [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16300aed40 con 0x7f16300ac1f0 2024-03-30T02:35:43.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.703+0000 7f1642a00700 1 -- 172.21.15.67:0/1546813741 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16440ccd60 con 0x7f164c105cb0 2024-03-30T02:35:43.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.703+0000 7f1649e00700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16300a1230 0x7f16300a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.703+0000 7f1649400700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] conn(0x7f16300ac1f0 0x7f16300ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.704+0000 7f1649e00700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16300a1230 0x7f16300a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f163c007b60 tx=0x7f163c0073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.704+0000 7f1649400700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] conn(0x7f16300ac1f0 0x7f16300ae670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.705+0000 7f1642a00700 1 -- 172.21.15.67:0/1546813741 <== osd.1 v2:172.21.15.82:6800/1982951943 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f16300aed40 con 0x7f16300ac1f0 2024-03-30T02:35:43.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.738+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 --> [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f164c1a1e70 con 0x7f16300ac1f0 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.739+0000 7f1642a00700 1 -- 172.21.15.67:0/1546813741 <== osd.1 v2:172.21.15.82:6800/1982951943 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f164c1a1e70 con 0x7f16300ac1f0 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.739+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 >> [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] conn(0x7f16300ac1f0 msgr2=0x7f16300ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.739+0000 7f1651200700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] conn(0x7f16300ac1f0 0x7f16300ae670 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.739+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16300a1230 msgr2=0x7f16300a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.739+0000 7f1651200700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16300a1230 0x7f16300a36f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f163c007b60 tx=0x7f163c0073d0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.739+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f164c105cb0 msgr2=0x7f164c1a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.739+0000 7f1651200700 1 --2- 172.21.15.67:0/1546813741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f164c105cb0 0x7f164c1a0fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f164400ef30 tx=0x7f164400c3a0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.740+0000 7f164a800700 1 -- 172.21.15.67:0/1546813741 reap_dead start 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.740+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 shutdown_connections 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.740+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 >> 172.21.15.67:0/1546813741 conn(0x7f164c075d50 msgr2=0x7f164c1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.740+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 shutdown_connections 2024-03-30T02:35:43.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.740+0000 7f1651200700 1 -- 172.21.15.67:0/1546813741 wait complete. 2024-03-30T02:35:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.824+0000 7fcccb600700 1 -- 172.21.15.67:0/1948370089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccc410d840 msgr2=0x7fccc410fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.824+0000 7fcccb600700 1 --2- 172.21.15.67:0/1948370089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccc410d840 0x7fccc410fc30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00b0a0 tx=0x7fccbc00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.825+0000 7fcccb600700 1 -- 172.21.15.67:0/1948370089 shutdown_connections 2024-03-30T02:35:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.825+0000 7fcccb600700 1 --2- 172.21.15.67:0/1948370089 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccc410d840 0x7fccc410fc30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.825+0000 7fcccb600700 1 --2- 172.21.15.67:0/1948370089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccc4072fe0 0x7fccc40733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.825+0000 7fcccb600700 1 --2- 172.21.15.67:0/1948370089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc4073900 0x7fccc410d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.825+0000 7fcccb600700 1 -- 172.21.15.67:0/1948370089 >> 172.21.15.67:0/1948370089 conn(0x7fccc4077e90 msgr2=0x7fccc40782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.825+0000 7fcccb600700 1 -- 172.21.15.67:0/1948370089 shutdown_connections 2024-03-30T02:35:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.825+0000 7fcccb600700 1 -- 172.21.15.67:0/1948370089 wait complete. 2024-03-30T02:35:43.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.827+0000 7fcccb600700 1 Processor -- start 2024-03-30T02:35:43.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.827+0000 7fcccb600700 1 -- start start 2024-03-30T02:35:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.828+0000 7fcccb600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccc4072fe0 0x7fccc4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.828+0000 7fcccb600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccc4073900 0x7fccc4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.828+0000 7fcccb600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc410d840 0x7fccc419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.828+0000 7fcccb600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc419e300 con 0x7fccc4072fe0 2024-03-30T02:35:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.828+0000 7fcccb600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc419e470 con 0x7fccc4073900 2024-03-30T02:35:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.828+0000 7fcccb600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc419e630 con 0x7fccc410d840 2024-03-30T02:35:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.829+0000 7fccc9600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc410d840 0x7fccc419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.829+0000 7fccc9600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc410d840 0x7fccc419dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:53666/0 (socket says 172.21.15.67:53666) 2024-03-30T02:35:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.829+0000 7fccc9600700 1 -- 172.21.15.67:0/1678309403 learned_addr learned my addr 172.21.15.67:0/1678309403 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.829+0000 7fccc8c00700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccc4072fe0 0x7fccc4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc9600700 1 -- 172.21.15.67:0/1678309403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccc4073900 msgr2=0x7fccc4199940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T02:35:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc9600700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccc4073900 0x7fccc4199940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc9600700 1 -- 172.21.15.67:0/1678309403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccc4072fe0 msgr2=0x7fccc4199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc9600700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccc4072fe0 0x7fccc4199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc9600700 1 -- 172.21.15.67:0/1678309403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc009d00 con 0x7fccc410d840 2024-03-30T02:35:43.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc8c00700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccc4072fe0 0x7fccc4199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:35:43.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc9600700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc410d840 0x7fccc419dcf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00b070 tx=0x7fccbc004b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc1600700 1 -- 172.21.15.67:0/1678309403 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00e040 con 0x7fccc410d840 2024-03-30T02:35:43.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc1600700 1 -- 172.21.15.67:0/1678309403 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccbc0079d0 con 0x7fccc410d840 2024-03-30T02:35:43.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.830+0000 7fccc1600700 1 -- 172.21.15.67:0/1678309403 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0052a0 con 0x7fccc410d840 2024-03-30T02:35:43.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.831+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc419e8b0 con 0x7fccc410d840 2024-03-30T02:35:43.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.831+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc4101060 con 0x7fccc410d840 2024-03-30T02:35:43.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.832+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fccc419eb90 con 0x7fccc410d840 2024-03-30T02:35:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.837+0000 7fccc1600700 1 -- 172.21.15.67:0/1678309403 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fccbc005400 con 0x7fccc410d840 2024-03-30T02:35:43.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.838+0000 7fccc1600700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fccac0a1230 0x7fccac0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.838+0000 7fccc1600700 1 -- 172.21.15.67:0/1678309403 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fccbc0cde50 con 0x7fccc410d840 2024-03-30T02:35:43.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.839+0000 7fccc1600700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] conn(0x7fccac0ac1f0 0x7fccac0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:43.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.839+0000 7fccc1600700 1 -- 172.21.15.67:0/1678309403 --> [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccac0aed40 con 0x7fccac0ac1f0 2024-03-30T02:35:43.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.839+0000 7fccc1600700 1 -- 172.21.15.67:0/1678309403 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccbc0d04d0 con 0x7fccc410d840 2024-03-30T02:35:43.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.839+0000 7fccc3e00700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] conn(0x7fccac0ac1f0 0x7fccac0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.839+0000 7fccc8c00700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fccac0a1230 0x7fccac0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.840+0000 7fccc3e00700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] conn(0x7fccac0ac1f0 0x7fccac0ae670 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.840+0000 7fccc8c00700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fccac0a1230 0x7fccac0a36f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009cb0 tx=0x7fccb8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.841+0000 7fccc1600700 1 -- 172.21.15.67:0/1678309403 <== osd.4 v2:172.21.15.82:6808/1426586615 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fccac0aed40 con 0x7fccac0ac1f0 2024-03-30T02:35:43.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.859+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 --> [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fccc40614b0 con 0x7fccac0ac1f0 2024-03-30T02:35:43.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.860+0000 7fccc1600700 1 -- 172.21.15.67:0/1678309403 <== osd.4 v2:172.21.15.82:6808/1426586615 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fccc40614b0 con 0x7fccac0ac1f0 2024-03-30T02:35:43.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.860+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 >> [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] conn(0x7fccac0ac1f0 msgr2=0x7fccac0ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.860+0000 7fcccb600700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] conn(0x7fccac0ac1f0 0x7fccac0ae670 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:43.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.861+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fccac0a1230 msgr2=0x7fccac0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.861+0000 7fcccb600700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fccac0a1230 0x7fccac0a36f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009cb0 tx=0x7fccb8009260 comp rx=0 tx=0).stop 2024-03-30T02:35:43.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.861+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc410d840 msgr2=0x7fccc419dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:43.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.861+0000 7fcccb600700 1 --2- 172.21.15.67:0/1678309403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc410d840 0x7fccc419dcf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00b070 tx=0x7fccbc004b00 comp rx=0 tx=0).stop 2024-03-30T02:35:43.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.861+0000 7fccc9600700 1 -- 172.21.15.67:0/1678309403 reap_dead start 2024-03-30T02:35:43.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.861+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 shutdown_connections 2024-03-30T02:35:43.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.861+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 >> 172.21.15.67:0/1678309403 conn(0x7fccc4077e90 msgr2=0x7fccc40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:43.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.861+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 shutdown_connections 2024-03-30T02:35:43.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:43.861+0000 7fcccb600700 1 -- 172.21.15.67:0/1678309403 wait complete. 2024-03-30T02:35:44.286 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:44 smithi067 ceph-mon[37808]: pgmap v263: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:44 smithi170 ceph-mon[78798]: pgmap v263: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:44 smithi082 ceph-mon[79176]: pgmap v263: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:35:44.644 INFO:teuthology.orchestra.run.smithi067.stdout:55834574907 2024-03-30T02:35:44.644 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-03-30T02:35:44.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.815+0000 7f6dd3800700 1 -- 172.21.15.67:0/3226260860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc101820 msgr2=0x7f6dcc10dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:44.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.815+0000 7f6dd3800700 1 --2- 172.21.15.67:0/3226260860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc101820 0x7f6dcc10dc10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400b0d0 tx=0x7f6dc400b3e0 comp rx=0 tx=0).stop 2024-03-30T02:35:44.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.816+0000 7f6dd3800700 1 -- 172.21.15.67:0/3226260860 shutdown_connections 2024-03-30T02:35:44.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.816+0000 7f6dd3800700 1 --2- 172.21.15.67:0/3226260860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc101820 0x7f6dcc10dc10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:44.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.816+0000 7f6dd3800700 1 --2- 172.21.15.67:0/3226260860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc106ef0 0x7f6dcc1072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:44.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.816+0000 7f6dd3800700 1 --2- 172.21.15.67:0/3226260860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dcc100e60 0x7f6dcc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:44.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.816+0000 7f6dd3800700 1 -- 172.21.15.67:0/3226260860 >> 172.21.15.67:0/3226260860 conn(0x7f6dcc077fa0 msgr2=0x7f6dcc0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.816+0000 7f6dd3800700 1 -- 172.21.15.67:0/3226260860 shutdown_connections 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.816+0000 7f6dd3800700 1 -- 172.21.15.67:0/3226260860 wait complete. 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.817+0000 7f6dd3800700 1 Processor -- start 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.818+0000 7f6dd3800700 1 -- start start 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.818+0000 7f6dd3800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dcc100e60 0x7f6dcc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.818+0000 7f6dd3800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc101820 0x7f6dcc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.818+0000 7f6dd3800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc106ef0 0x7f6dcc073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.818+0000 7f6dd3800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1a56d0 con 0x7f6dcc101820 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.818+0000 7f6dd3800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1a5840 con 0x7f6dcc106ef0 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.818+0000 7f6dd3800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1a5a00 con 0x7f6dcc100e60 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.818+0000 7f6dcbe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc101820 0x7f6dcc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.818+0000 7f6dcbe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc101820 0x7f6dcc0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38376/0 (socket says 172.21.15.67:38376) 2024-03-30T02:35:44.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.819+0000 7f6dcbe00700 1 -- 172.21.15.67:0/2874855732 learned_addr learned my addr 172.21.15.67:0/2874855732 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.819+0000 7f6dd1800700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc106ef0 0x7f6dcc073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.819+0000 7f6dd0e00700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dcc100e60 0x7f6dcc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.819+0000 7f6dcbe00700 1 -- 172.21.15.67:0/2874855732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dcc100e60 msgr2=0x7f6dcc0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.819+0000 7f6dcbe00700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dcc100e60 0x7f6dcc0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.819+0000 7f6dcbe00700 1 -- 172.21.15.67:0/2874855732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc106ef0 msgr2=0x7f6dcc073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.819+0000 7f6dcbe00700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dcc106ef0 0x7f6dcc073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.819+0000 7f6dcbe00700 1 -- 172.21.15.67:0/2874855732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc4009d00 con 0x7f6dcc101820 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.819+0000 7f6dcbe00700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc101820 0x7f6dcc0735f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6db800e9a0 tx=0x7f6db800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.820+0000 7f6dc9600700 1 -- 172.21.15.67:0/2874855732 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8009790 con 0x7f6dcc101820 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.820+0000 7f6dc9600700 1 -- 172.21.15.67:0/2874855732 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db8010430 con 0x7f6dcc101820 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.820+0000 7f6dc9600700 1 -- 172.21.15.67:0/2874855732 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8018680 con 0x7f6dcc101820 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.820+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc1a5ce0 con 0x7f6dcc101820 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.820+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc1a6190 con 0x7f6dcc101820 2024-03-30T02:35:44.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.821+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6dcc110fc0 con 0x7f6dcc101820 2024-03-30T02:35:44.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.824+0000 7f6dc9600700 1 -- 172.21.15.67:0/2874855732 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f6db80105a0 con 0x7f6dcc101820 2024-03-30T02:35:44.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.826+0000 7f6dc9600700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6db40a11b0 0x7f6db40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:44.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.826+0000 7f6dc9600700 1 -- 172.21.15.67:0/2874855732 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f6db8014070 con 0x7f6dcc101820 2024-03-30T02:35:44.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.826+0000 7f6dd0e00700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6db40a11b0 0x7f6db40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:44.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.826+0000 7f6dc9600700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] conn(0x7f6db40ac170 0x7f6db40ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:44.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.826+0000 7f6dc9600700 1 -- 172.21.15.67:0/2874855732 --> [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6db40aecc0 con 0x7f6db40ac170 2024-03-30T02:35:44.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.826+0000 7f6dc9600700 1 -- 172.21.15.67:0/2874855732 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6db80cd640 con 0x7f6dcc101820 2024-03-30T02:35:44.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.827+0000 7f6dd0e00700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6db40a11b0 0x7f6db40a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009be0 tx=0x7f6dc0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.827+0000 7f6dd1800700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] conn(0x7f6db40ac170 0x7f6db40ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.827+0000 7f6dd1800700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] conn(0x7f6db40ac170 0x7f6db40ae5f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.828+0000 7f6dc9600700 1 -- 172.21.15.67:0/2874855732 <== osd.10 v2:172.21.15.82:6824/3088174869 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f6db40aecc0 con 0x7f6db40ac170 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.844+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 --> [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6dcc061430 con 0x7f6db40ac170 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.845+0000 7f6dc9600700 1 -- 172.21.15.67:0/2874855732 <== osd.10 v2:172.21.15.82:6824/3088174869 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6dcc061430 con 0x7f6db40ac170 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 >> [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] conn(0x7f6db40ac170 msgr2=0x7f6db40ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] conn(0x7f6db40ac170 0x7f6db40ae5f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6db40a11b0 msgr2=0x7f6db40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6db40a11b0 0x7f6db40a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009be0 tx=0x7f6dc0009260 comp rx=0 tx=0).stop 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc101820 msgr2=0x7f6dcc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 --2- 172.21.15.67:0/2874855732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6dcc101820 0x7f6dcc0735f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6db800e9a0 tx=0x7f6db800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd1800700 1 -- 172.21.15.67:0/2874855732 reap_dead start 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 shutdown_connections 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 >> 172.21.15.67:0/2874855732 conn(0x7f6dcc077fa0 msgr2=0x7f6dcc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:44.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 shutdown_connections 2024-03-30T02:35:44.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:44.846+0000 7f6dd3800700 1 -- 172.21.15.67:0/2874855732 wait complete. 2024-03-30T02:35:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.012+0000 7ff134c00700 1 -- 172.21.15.67:0/1150637655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1301039d0 msgr2=0x7ff130107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.012+0000 7ff134c00700 1 --2- 172.21.15.67:0/1150637655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1301039d0 0x7ff130107a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009990 tx=0x7ff11c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:45.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.012+0000 7ff134c00700 1 -- 172.21.15.67:0/1150637655 shutdown_connections 2024-03-30T02:35:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.012+0000 7ff134c00700 1 --2- 172.21.15.67:0/1150637655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff130108100 0x7ff13010fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.012+0000 7ff134c00700 1 --2- 172.21.15.67:0/1150637655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff130103020 0x7ff130103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.012+0000 7ff134c00700 1 --2- 172.21.15.67:0/1150637655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1301039d0 0x7ff130107a20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.012+0000 7ff134c00700 1 -- 172.21.15.67:0/1150637655 >> 172.21.15.67:0/1150637655 conn(0x7ff1300fe890 msgr2=0x7ff130100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.012+0000 7ff134c00700 1 -- 172.21.15.67:0/1150637655 shutdown_connections 2024-03-30T02:35:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.013+0000 7ff134c00700 1 -- 172.21.15.67:0/1150637655 wait complete. 2024-03-30T02:35:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.014+0000 7ff134c00700 1 Processor -- start 2024-03-30T02:35:45.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.014+0000 7ff134c00700 1 -- start start 2024-03-30T02:35:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff134c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff130103020 0x7ff130199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff134c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1301039d0 0x7ff1301997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff134c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff130108100 0x7ff13019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff134c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019e190 con 0x7ff1301039d0 2024-03-30T02:35:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff134c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019e300 con 0x7ff130108100 2024-03-30T02:35:45.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff134c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019e4c0 con 0x7ff130103020 2024-03-30T02:35:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff12ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1301039d0 0x7ff1301997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff12f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff130103020 0x7ff130199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:45.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff12ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1301039d0 0x7ff1301997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38406/0 (socket says 172.21.15.67:38406) 2024-03-30T02:35:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.015+0000 7ff12ea00700 1 -- 172.21.15.67:0/3130038975 learned_addr learned my addr 172.21.15.67:0/3130038975 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.016+0000 7ff12fe00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff130108100 0x7ff13019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.016+0000 7ff12f400700 1 -- 172.21.15.67:0/3130038975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff130108100 msgr2=0x7ff13019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.016+0000 7ff12f400700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff130108100 0x7ff13019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.016+0000 7ff12f400700 1 -- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1301039d0 msgr2=0x7ff1301997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.016+0000 7ff12f400700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1301039d0 0x7ff1301997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.016+0000 7ff12f400700 1 -- 172.21.15.67:0/3130038975 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c0095f0 con 0x7ff130103020 2024-03-30T02:35:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.016+0000 7ff12ea00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff1301039d0 0x7ff1301997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:45.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.016+0000 7ff12f400700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff130103020 0x7ff130199290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff12000e9f0 tx=0x7ff12000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.017+0000 7ff127e00700 1 -- 172.21.15.67:0/3130038975 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12000c9c0 con 0x7ff130103020 2024-03-30T02:35:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.017+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13019e7a0 con 0x7ff130103020 2024-03-30T02:35:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.017+0000 7ff127e00700 1 -- 172.21.15.67:0/3130038975 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff120010430 con 0x7ff130103020 2024-03-30T02:35:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.017+0000 7ff127e00700 1 -- 172.21.15.67:0/3130038975 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120018680 con 0x7ff130103020 2024-03-30T02:35:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.017+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1301a6000 con 0x7ff130103020 2024-03-30T02:35:45.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.018+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff130104500 con 0x7ff130103020 2024-03-30T02:35:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.020+0000 7ff127e00700 1 -- 172.21.15.67:0/3130038975 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ff120010860 con 0x7ff130103020 2024-03-30T02:35:45.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.021+0000 7ff127e00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1140a11b0 0x7ff1140a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:45.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.022+0000 7ff127e00700 1 -- 172.21.15.67:0/3130038975 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ff120014070 con 0x7ff130103020 2024-03-30T02:35:45.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.022+0000 7ff12ea00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1140a11b0 0x7ff1140a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:45.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.022+0000 7ff127e00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] conn(0x7ff1140ac170 0x7ff1140ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:45.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.022+0000 7ff127e00700 1 -- 172.21.15.67:0/3130038975 --> [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1140aecc0 con 0x7ff1140ac170 2024-03-30T02:35:45.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.022+0000 7ff127e00700 1 -- 172.21.15.67:0/3130038975 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1200cd510 con 0x7ff130103020 2024-03-30T02:35:45.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.022+0000 7ff12fe00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] conn(0x7ff1140ac170 0x7ff1140ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:45.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.023+0000 7ff12ea00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1140a11b0 0x7ff1140a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff13019a7f0 tx=0x7ff11c00b280 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:45.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.023+0000 7ff12fe00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] conn(0x7ff1140ac170 0x7ff1140ae5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:45.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.024+0000 7ff127e00700 1 -- 172.21.15.67:0/3130038975 <== osd.9 v2:172.21.15.67:6826/2316326417 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff1140aecc0 con 0x7ff1140ac170 2024-03-30T02:35:45.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.043+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 --> [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff130112f80 con 0x7ff1140ac170 2024-03-30T02:35:45.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.043+0000 7ff127e00700 1 -- 172.21.15.67:0/3130038975 <== osd.9 v2:172.21.15.67:6826/2316326417 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff130112f80 con 0x7ff1140ac170 2024-03-30T02:35:45.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] conn(0x7ff1140ac170 msgr2=0x7ff1140ae5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:45.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] conn(0x7ff1140ac170 0x7ff1140ae5f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:45.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1140a11b0 msgr2=0x7ff1140a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:45.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff1140a11b0 0x7ff1140a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff13019a7f0 tx=0x7ff11c00b280 comp rx=0 tx=0).stop 2024-03-30T02:35:45.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff130103020 msgr2=0x7ff130199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:45.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 --2- 172.21.15.67:0/3130038975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff130103020 0x7ff130199290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff12000e9f0 tx=0x7ff12000ed00 comp rx=0 tx=0).stop 2024-03-30T02:35:45.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff12fe00700 1 -- 172.21.15.67:0/3130038975 reap_dead start 2024-03-30T02:35:45.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 shutdown_connections 2024-03-30T02:35:45.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 >> 172.21.15.67:0/3130038975 conn(0x7ff1300fe890 msgr2=0x7ff1300ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:45.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 shutdown_connections 2024-03-30T02:35:45.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:45.044+0000 7ff134c00700 1 -- 172.21.15.67:0/3130038975 wait complete. 2024-03-30T02:35:45.213 INFO:teuthology.orchestra.run.smithi067.stdout:206158430244 2024-03-30T02:35:45.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.7 2024-03-30T02:35:45.411 INFO:teuthology.orchestra.run.smithi067.stdout:158913789996 2024-03-30T02:35:45.411 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-03-30T02:35:45.703 INFO:teuthology.orchestra.run.smithi067.stdout:287762808858 2024-03-30T02:35:45.704 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.10 2024-03-30T02:35:45.806 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:46.087 INFO:teuthology.orchestra.run.smithi067.stdout:266287972380 2024-03-30T02:35:46.087 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.9 2024-03-30T02:35:46.320 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:46 smithi067 ceph-mon[37808]: pgmap v264: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-30T02:35:46.320 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:46 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:35:46.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:46 smithi170 ceph-mon[78798]: pgmap v264: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-30T02:35:46.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:46 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:35:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.497+0000 7f6308200700 1 -- 172.21.15.67:0/1619422779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6300105cb0 msgr2=0x7f6300068750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.497+0000 7f6308200700 1 --2- 172.21.15.67:0/1619422779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6300105cb0 0x7f6300068750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f62f40098f0 tx=0x7f62f4009c00 comp rx=0 tx=0).stop 2024-03-30T02:35:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.497+0000 7f6308200700 1 -- 172.21.15.67:0/1619422779 shutdown_connections 2024-03-30T02:35:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.497+0000 7f6308200700 1 --2- 172.21.15.67:0/1619422779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6300103810 0x7f6300103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.497+0000 7f6308200700 1 --2- 172.21.15.67:0/1619422779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6300105cb0 0x7f6300068750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.497+0000 7f6308200700 1 --2- 172.21.15.67:0/1619422779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6300068c90 0x7f6300069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.497+0000 7f6308200700 1 -- 172.21.15.67:0/1619422779 >> 172.21.15.67:0/1619422779 conn(0x7f6300075d50 msgr2=0x7f6300076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:46.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.498+0000 7f6308200700 1 -- 172.21.15.67:0/1619422779 shutdown_connections 2024-03-30T02:35:46.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.498+0000 7f6308200700 1 -- 172.21.15.67:0/1619422779 wait complete. 2024-03-30T02:35:46.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.499+0000 7f6308200700 1 Processor -- start 2024-03-30T02:35:46.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.499+0000 7f6308200700 1 -- start start 2024-03-30T02:35:46.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.499+0000 7f6308200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6300068c90 0x7f630019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:46.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.499+0000 7f6308200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6300103810 0x7f630019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:46.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.499+0000 7f6308200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6300105cb0 0x7f63001a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:46.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.499+0000 7f6308200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001a1610 con 0x7f6300103810 2024-03-30T02:35:46.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6308200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001a1780 con 0x7f6300105cb0 2024-03-30T02:35:46.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6308200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001a1940 con 0x7f6300068c90 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6304e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6300103810 0x7f630019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6305800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6300068c90 0x7f630019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6304e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6300103810 0x7f630019dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42566/0 (socket says 172.21.15.67:42566) 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6304e00700 1 -- 172.21.15.67:0/2448599238 learned_addr learned my addr 172.21.15.67:0/2448599238 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6306200700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6300105cb0 0x7f63001a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6305800700 1 -- 172.21.15.67:0/2448599238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6300105cb0 msgr2=0x7f63001a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6305800700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6300105cb0 0x7f63001a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6305800700 1 -- 172.21.15.67:0/2448599238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6300103810 msgr2=0x7f630019dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6305800700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6300103810 0x7f630019dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.500+0000 7f6305800700 1 -- 172.21.15.67:0/2448599238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f8009d00 con 0x7f6300068c90 2024-03-30T02:35:46.502 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:46 smithi082 ceph-mon[79176]: pgmap v264: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-30T02:35:46.502 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:46 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:35:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.501+0000 7f6305800700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6300068c90 0x7f630019d7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009fd0 tx=0x7f62f4005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.501+0000 7f62fe000700 1 -- 172.21.15.67:0/2448599238 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4005040 con 0x7f6300068c90 2024-03-30T02:35:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.501+0000 7f62fe000700 1 -- 172.21.15.67:0/2448599238 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62f4005660 con 0x7f6300068c90 2024-03-30T02:35:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.501+0000 7f62fe000700 1 -- 172.21.15.67:0/2448599238 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4020430 con 0x7f6300068c90 2024-03-30T02:35:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.501+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f4009520 con 0x7f6300068c90 2024-03-30T02:35:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.501+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63001a1e50 con 0x7f6300068c90 2024-03-30T02:35:46.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.502+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63001002f0 con 0x7f6300068c90 2024-03-30T02:35:46.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.506+0000 7f62fe000700 1 -- 172.21.15.67:0/2448599238 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f62f4003680 con 0x7f6300068c90 2024-03-30T02:35:46.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.507+0000 7f62fe000700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62e80a1230 0x7f62e80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:46.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.507+0000 7f62fe000700 1 -- 172.21.15.67:0/2448599238 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f62f40cda40 con 0x7f6300068c90 2024-03-30T02:35:46.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.507+0000 7f6304e00700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62e80a1230 0x7f62e80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:46.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.508+0000 7f6304e00700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62e80a1230 0x7f62e80a36f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f630019ed30 tx=0x7f62ec009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:46.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.511+0000 7f62fe000700 1 -- 172.21.15.67:0/2448599238 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f62f4096e00 con 0x7f6300068c90 2024-03-30T02:35:46.595 INFO:teuthology.orchestra.run.smithi067.stdout:137438953519 2024-03-30T02:35:46.595 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-03-30T02:35:46.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.747+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f630004ea80 con 0x7f6300068c90 2024-03-30T02:35:46.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.747+0000 7f62fe000700 1 -- 172.21.15.67:0/2448599238 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f630004ea80 con 0x7f6300068c90 2024-03-30T02:35:46.748 INFO:teuthology.orchestra.run.smithi067.stdout:34359738431 2024-03-30T02:35:46.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62e80a1230 msgr2=0x7f62e80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:46.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62e80a1230 0x7f62e80a36f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f630019ed30 tx=0x7f62ec009260 comp rx=0 tx=0).stop 2024-03-30T02:35:46.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6300068c90 msgr2=0x7f630019d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:46.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6300068c90 0x7f630019d7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009fd0 tx=0x7f62f4005df0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 shutdown_connections 2024-03-30T02:35:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62e80a1230 0x7f62e80a36f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6300105cb0 0x7f63001a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6300068c90 0x7f630019d7d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 --2- 172.21.15.67:0/2448599238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6300103810 0x7f630019dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 >> 172.21.15.67:0/2448599238 conn(0x7f6300075d50 msgr2=0x7f63000fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 shutdown_connections 2024-03-30T02:35:46.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:46.750+0000 7f6308200700 1 -- 172.21.15.67:0/2448599238 wait complete. 2024-03-30T02:35:47.088 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:47.170 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:47.254 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738429 got 34359738431 for osd.0 2024-03-30T02:35:47.254 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:47.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:47 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2448599238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-30T02:35:47.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:47 smithi067 ceph-mon[37808]: pgmap v265: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-30T02:35:47.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:47 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:47 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2448599238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-30T02:35:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:47 smithi082 ceph-mon[79176]: pgmap v265: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-30T02:35:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:47 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:47 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2448599238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-30T02:35:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:47 smithi170 ceph-mon[78798]: pgmap v265: 129 pgs: 129 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-03-30T02:35:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:47 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:47.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.740+0000 7f93c5200700 1 -- 172.21.15.67:0/3577986836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0074d30 msgr2=0x7f93c0072fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:47.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.740+0000 7f93c5200700 1 --2- 172.21.15.67:0/3577986836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0074d30 0x7f93c0072fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f93b00098f0 tx=0x7f93b0009c00 comp rx=0 tx=0).stop 2024-03-30T02:35:47.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.740+0000 7f93c5200700 1 -- 172.21.15.67:0/3577986836 shutdown_connections 2024-03-30T02:35:47.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.740+0000 7f93c5200700 1 --2- 172.21.15.67:0/3577986836 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93c0111b30 0x7f93c0113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.740+0000 7f93c5200700 1 --2- 172.21.15.67:0/3577986836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0074d30 0x7f93c0072fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.740+0000 7f93c5200700 1 --2- 172.21.15.67:0/3577986836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0073520 0x7f93c00739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.740+0000 7f93c5200700 1 -- 172.21.15.67:0/3577986836 >> 172.21.15.67:0/3577986836 conn(0x7f93c0077fa0 msgr2=0x7f93c00783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:47.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.741+0000 7f93c5200700 1 -- 172.21.15.67:0/3577986836 shutdown_connections 2024-03-30T02:35:47.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.741+0000 7f93c5200700 1 -- 172.21.15.67:0/3577986836 wait complete. 2024-03-30T02:35:47.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.742+0000 7f93c5200700 1 Processor -- start 2024-03-30T02:35:47.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.742+0000 7f93c5200700 1 -- start start 2024-03-30T02:35:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.742+0000 7f93c5200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0073520 0x7f93c01a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.742+0000 7f93c5200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0074d30 0x7f93c01a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.742+0000 7f93c5200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93c0111b30 0x7f93c019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.742+0000 7f93c5200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c019de30 con 0x7f93c0111b30 2024-03-30T02:35:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.742+0000 7f93c5200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c019dfa0 con 0x7f93c0074d30 2024-03-30T02:35:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.742+0000 7f93c5200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c019e160 con 0x7f93c0073520 2024-03-30T02:35:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.742+0000 7f93bde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0073520 0x7f93c01a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.743+0000 7f93bde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0073520 0x7f93c01a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48010/0 (socket says 172.21.15.67:48010) 2024-03-30T02:35:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.743+0000 7f93bde00700 1 -- 172.21.15.67:0/4255795538 learned_addr learned my addr 172.21.15.67:0/4255795538 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.743+0000 7f93bde00700 1 -- 172.21.15.67:0/4255795538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0074d30 msgr2=0x7f93c01a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.743+0000 7f93bde00700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0074d30 0x7f93c01a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.743+0000 7f93bde00700 1 -- 172.21.15.67:0/4255795538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93c0111b30 msgr2=0x7f93c019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.743+0000 7f93bde00700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93c0111b30 0x7f93c019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.743+0000 7f93bde00700 1 -- 172.21.15.67:0/4255795538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b0009520 con 0x7f93c0073520 2024-03-30T02:35:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.743+0000 7f93bde00700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0073520 0x7f93c01a3680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f93b0005950 tx=0x7f93b0010ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.743+0000 7f93b6a00700 1 -- 172.21.15.67:0/4255795538 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b001c070 con 0x7f93c0073520 2024-03-30T02:35:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.744+0000 7f93b6a00700 1 -- 172.21.15.67:0/4255795538 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93b000bbe0 con 0x7f93c0073520 2024-03-30T02:35:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.744+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c019e3e0 con 0x7f93c0073520 2024-03-30T02:35:47.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.744+0000 7f93b6a00700 1 -- 172.21.15.67:0/4255795538 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0003970 con 0x7f93c0073520 2024-03-30T02:35:47.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.744+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c01aa670 con 0x7f93c0073520 2024-03-30T02:35:47.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.745+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c0074670 con 0x7f93c0073520 2024-03-30T02:35:47.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.747+0000 7f93b6a00700 1 -- 172.21.15.67:0/4255795538 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f93b0020030 con 0x7f93c0073520 2024-03-30T02:35:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.748+0000 7f93b6a00700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f93a40a1230 0x7f93a40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.748+0000 7f93b6a00700 1 -- 172.21.15.67:0/4255795538 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f93b00cdbd0 con 0x7f93c0073520 2024-03-30T02:35:47.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.748+0000 7f93bd400700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f93a40a1230 0x7f93a40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:47.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.749+0000 7f93bd400700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f93a40a1230 0x7f93a40a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005950 tx=0x7f93ac0058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.752+0000 7f93b6a00700 1 -- 172.21.15.67:0/4255795538 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f93b0096f30 con 0x7f93c0073520 2024-03-30T02:35:47.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.981+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f93c019ec80 con 0x7f93c0073520 2024-03-30T02:35:47.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.982+0000 7f93b6a00700 1 -- 172.21.15.67:0/4255795538 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f93b009a550 con 0x7f93c0073520 2024-03-30T02:35:47.983 INFO:teuthology.orchestra.run.smithi067.stdout:317827579925 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.984+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f93a40a1230 msgr2=0x7f93a40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.984+0000 7f93c5200700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f93a40a1230 0x7f93a40a36f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005950 tx=0x7f93ac0058e0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.984+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0073520 msgr2=0x7f93c01a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.984+0000 7f93c5200700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0073520 0x7f93c01a3680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f93b0005950 tx=0x7f93b0010ff0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.984+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 shutdown_connections 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.985+0000 7f93c5200700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f93a40a1230 0x7f93a40a36f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.985+0000 7f93c5200700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f93c0111b30 0x7f93c019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.985+0000 7f93c5200700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93c0073520 0x7f93c01a3680 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.985+0000 7f93c5200700 1 --2- 172.21.15.67:0/4255795538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c0074d30 0x7f93c01a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.985+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 >> 172.21.15.67:0/4255795538 conn(0x7f93c0077fa0 msgr2=0x7f93c0110860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.985+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 shutdown_connections 2024-03-30T02:35:47.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:47.985+0000 7f93c5200700 1 -- 172.21.15.67:0/4255795538 wait complete. 2024-03-30T02:35:48.330 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:48 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4255795538' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-30T02:35:48.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579924 got 317827579925 for osd.11 2024-03-30T02:35:48.351 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:48 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4255795538' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-30T02:35:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:48 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4255795538' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-30T02:35:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:48.936 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:49.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:49 smithi067 ceph-mon[37808]: pgmap v266: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-30T02:35:49.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:49 smithi082 ceph-mon[79176]: pgmap v266: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-30T02:35:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:49 smithi170 ceph-mon[78798]: pgmap v266: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-30T02:35:49.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.591+0000 7f86d0e00700 1 -- 172.21.15.67:0/4255062191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc073520 msgr2=0x7f86cc0739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.591+0000 7f86d0e00700 1 --2- 172.21.15.67:0/4255062191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc073520 0x7f86cc0739a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009990 tx=0x7f86b8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.591+0000 7f86d0e00700 1 -- 172.21.15.67:0/4255062191 shutdown_connections 2024-03-30T02:35:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.591+0000 7f86d0e00700 1 --2- 172.21.15.67:0/4255062191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86cc111b30 0x7f86cc113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.591+0000 7f86d0e00700 1 --2- 172.21.15.67:0/4255062191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86cc074d30 0x7f86cc072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.591+0000 7f86d0e00700 1 --2- 172.21.15.67:0/4255062191 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc073520 0x7f86cc0739a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.591+0000 7f86d0e00700 1 -- 172.21.15.67:0/4255062191 >> 172.21.15.67:0/4255062191 conn(0x7f86cc077fa0 msgr2=0x7f86cc0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.591+0000 7f86d0e00700 1 -- 172.21.15.67:0/4255062191 shutdown_connections 2024-03-30T02:35:49.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.591+0000 7f86d0e00700 1 -- 172.21.15.67:0/4255062191 wait complete. 2024-03-30T02:35:49.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.592+0000 7f86d0e00700 1 Processor -- start 2024-03-30T02:35:49.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86d0e00700 1 -- start start 2024-03-30T02:35:49.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86d0e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86cc073520 0x7f86cc1a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:49.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86d0e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc074d30 0x7f86cc1a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:49.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86d0e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86cc111b30 0x7f86cc19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:49.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86d0e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc19dd30 con 0x7f86cc074d30 2024-03-30T02:35:49.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86d0e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc19dea0 con 0x7f86cc111b30 2024-03-30T02:35:49.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86d0e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc19e060 con 0x7f86cc073520 2024-03-30T02:35:49.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86c9400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc074d30 0x7f86cc1a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:49.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86c9400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc074d30 0x7f86cc1a3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42616/0 (socket says 172.21.15.67:42616) 2024-03-30T02:35:49.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86c9400700 1 -- 172.21.15.67:0/980066833 learned_addr learned my addr 172.21.15.67:0/980066833 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:49.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86ca800700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86cc111b30 0x7f86cc19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:49.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.593+0000 7f86c9e00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86cc073520 0x7f86cc1a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:49.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86c9400700 1 -- 172.21.15.67:0/980066833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86cc073520 msgr2=0x7f86cc1a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:49.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86c9400700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86cc073520 0x7f86cc1a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86c9400700 1 -- 172.21.15.67:0/980066833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86cc111b30 msgr2=0x7f86cc19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:49.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86c9400700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86cc111b30 0x7f86cc19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86c9400700 1 -- 172.21.15.67:0/980066833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b80095f0 con 0x7f86cc074d30 2024-03-30T02:35:49.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86c9400700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc074d30 0x7f86cc1a3ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86b80095c0 tx=0x7f86b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:49.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86c2a00700 1 -- 172.21.15.67:0/980066833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b801d070 con 0x7f86cc074d30 2024-03-30T02:35:49.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc19e2e0 con 0x7f86cc074d30 2024-03-30T02:35:49.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86c2a00700 1 -- 172.21.15.67:0/980066833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86b8005050 con 0x7f86cc074d30 2024-03-30T02:35:49.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.594+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc1aa580 con 0x7f86cc074d30 2024-03-30T02:35:49.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.595+0000 7f86c2a00700 1 -- 172.21.15.67:0/980066833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8004a30 con 0x7f86cc074d30 2024-03-30T02:35:49.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.596+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86cc074670 con 0x7f86cc074d30 2024-03-30T02:35:49.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.598+0000 7f86c2a00700 1 -- 172.21.15.67:0/980066833 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f86b80051c0 con 0x7f86cc074d30 2024-03-30T02:35:49.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.600+0000 7f86c2a00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f86b00a11b0 0x7f86b00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:49.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.600+0000 7f86c2a00700 1 -- 172.21.15.67:0/980066833 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f86b80ce290 con 0x7f86cc074d30 2024-03-30T02:35:49.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.600+0000 7f86c9e00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f86b00a11b0 0x7f86b00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:49.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.601+0000 7f86c9e00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f86b00a11b0 0x7f86b00a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86bc005950 tx=0x7f86bc00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:49.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.603+0000 7f86c2a00700 1 -- 172.21.15.67:0/980066833 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f86b80975f0 con 0x7f86cc074d30 2024-03-30T02:35:49.965 INFO:teuthology.orchestra.run.smithi067.stdout:107374182452 2024-03-30T02:35:49.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.837+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f86cc19eb90 con 0x7f86cc074d30 2024-03-30T02:35:49.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.838+0000 7f86c2a00700 1 -- 172.21.15.67:0/980066833 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f86b809ac10 con 0x7f86cc074d30 2024-03-30T02:35:49.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f86b00a11b0 msgr2=0x7f86b00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:49.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f86b00a11b0 0x7f86b00a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86bc005950 tx=0x7f86bc00b410 comp rx=0 tx=0).stop 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc074d30 msgr2=0x7f86cc1a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc074d30 0x7f86cc1a3ad0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86b80095c0 tx=0x7f86b800f690 comp rx=0 tx=0).stop 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 shutdown_connections 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f86b00a11b0 0x7f86b00a3670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86cc111b30 0x7f86cc19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86cc073520 0x7f86cc1a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 --2- 172.21.15.67:0/980066833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f86cc074d30 0x7f86cc1a3ad0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 >> 172.21.15.67:0/980066833 conn(0x7f86cc077fa0 msgr2=0x7f86cc110700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.841+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 shutdown_connections 2024-03-30T02:35:49.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:49.842+0000 7f86d0e00700 1 -- 172.21.15.67:0/980066833 wait complete. 2024-03-30T02:35:49.968 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:50.071 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:50.256 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:50.275 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:50.384 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:50.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:50 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/980066833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-30T02:35:50.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:50 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/980066833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-30T02:35:50.686 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:50 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/980066833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-30T02:35:50.944 INFO:teuthology.orchestra.run.smithi067.stdout:188978561065 2024-03-30T02:35:50.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.408+0000 7f064d600700 1 -- 172.21.15.67:0/5687103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648106df0 msgr2=0x7f06481071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:50.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.408+0000 7f064d600700 1 --2- 172.21.15.67:0/5687103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648106df0 0x7f06481071d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0638009990 tx=0x7f0638009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.408+0000 7f064d600700 1 -- 172.21.15.67:0/5687103 shutdown_connections 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.408+0000 7f064d600700 1 --2- 172.21.15.67:0/5687103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06480690f0 0x7f064810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.408+0000 7f064d600700 1 --2- 172.21.15.67:0/5687103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648106df0 0x7f06481071d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.408+0000 7f064d600700 1 --2- 172.21.15.67:0/5687103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0648068750 0x7f0648068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.408+0000 7f064d600700 1 -- 172.21.15.67:0/5687103 >> 172.21.15.67:0/5687103 conn(0x7f0648075d50 msgr2=0x7f0648076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.408+0000 7f064d600700 1 -- 172.21.15.67:0/5687103 shutdown_connections 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.408+0000 7f064d600700 1 -- 172.21.15.67:0/5687103 wait complete. 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.409+0000 7f064d600700 1 Processor -- start 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.410+0000 7f064d600700 1 -- start start 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.410+0000 7f064d600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648068750 0x7f06481095a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.410+0000 7f064d600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06480690f0 0x7f0648109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.410+0000 7f064d600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0648106df0 0x7f06481a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:50.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.410+0000 7f064d600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481a1250 con 0x7f0648106df0 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.410+0000 7f064d600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481a13c0 con 0x7f0648068750 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.410+0000 7f064d600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06481a1580 con 0x7f06480690f0 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648068750 0x7f06481095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648068750 0x7f06481095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37350/0 (socket says 172.21.15.67:37350) 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646a00700 1 -- 172.21.15.67:0/1109486116 learned_addr learned my addr 172.21.15.67:0/1109486116 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646000700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06480690f0 0x7f0648109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646a00700 1 -- 172.21.15.67:0/1109486116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06480690f0 msgr2=0x7f0648109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646a00700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06480690f0 0x7f0648109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0647400700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0648106df0 0x7f06481a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646a00700 1 -- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0648106df0 msgr2=0x7f06481a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646a00700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0648106df0 0x7f06481a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646a00700 1 -- 172.21.15.67:0/1109486116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380095f0 con 0x7f0648068750 2024-03-30T02:35:50.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0646000700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06480690f0 0x7f0648109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.411+0000 7f0647400700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0648106df0 0x7f06481a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.412+0000 7f0646a00700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648068750 0x7f06481095a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0638005910 tx=0x7f0638004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.412+0000 7f063f400700 1 -- 172.21.15.67:0/1109486116 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063801d070 con 0x7f0648068750 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.412+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06481a1800 con 0x7f0648068750 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.412+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06481a1cf0 con 0x7f0648068750 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.412+0000 7f063f400700 1 -- 172.21.15.67:0/1109486116 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0638022470 con 0x7f0648068750 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.412+0000 7f063f400700 1 -- 172.21.15.67:0/1109486116 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638017650 con 0x7f0648068750 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.413+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06280052f0 con 0x7f0648068750 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.415+0000 7f063f400700 1 -- 172.21.15.67:0/1109486116 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f063800fac0 con 0x7f0648068750 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.416+0000 7f063f400700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f062c0a11b0 0x7f062c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.417+0000 7f063f400700 1 -- 172.21.15.67:0/1109486116 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f06380cdfa0 con 0x7f0648068750 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.417+0000 7f0646000700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f062c0a11b0 0x7f062c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.418+0000 7f0646000700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f062c0a11b0 0x7f062c0a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f064810ab00 tx=0x7f0634009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.421+0000 7f063f400700 1 -- 172.21.15.67:0/1109486116 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f063805d9f0 con 0x7f0648068750 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.643+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0628005160 con 0x7f0648068750 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.644+0000 7f063f400700 1 -- 172.21.15.67:0/1109486116 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0638097300 con 0x7f0648068750 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f062c0a11b0 msgr2=0x7f062c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f062c0a11b0 0x7f062c0a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f064810ab00 tx=0x7f0634009260 comp rx=0 tx=0).stop 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648068750 msgr2=0x7f06481095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648068750 0x7f06481095a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0638005910 tx=0x7f0638004730 comp rx=0 tx=0).stop 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 shutdown_connections 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f062c0a11b0 0x7f062c0a3670 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0648106df0 0x7f06481a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0648068750 0x7f06481095a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 --2- 172.21.15.67:0/1109486116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06480690f0 0x7f0648109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 >> 172.21.15.67:0/1109486116 conn(0x7f0648075d50 msgr2=0x7f06480fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 shutdown_connections 2024-03-30T02:35:50.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:50.647+0000 7f064d600700 1 -- 172.21.15.67:0/1109486116 wait complete. 2024-03-30T02:35:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.390+0000 7f2069400700 1 -- 172.21.15.67:0/2374951217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 msgr2=0x7f20641071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.390+0000 7f2069400700 1 --2- 172.21.15.67:0/2374951217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 0x7f20641071d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2054009990 tx=0x7f2054009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.390+0000 7f2069400700 1 -- 172.21.15.67:0/2374951217 shutdown_connections 2024-03-30T02:35:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.390+0000 7f2069400700 1 --2- 172.21.15.67:0/2374951217 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 0x7f20641071d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.390+0000 7f2069400700 1 --2- 172.21.15.67:0/2374951217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20640690f0 0x7f206410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.390+0000 7f2069400700 1 --2- 172.21.15.67:0/2374951217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2064068750 0x7f2064068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.390+0000 7f2069400700 1 -- 172.21.15.67:0/2374951217 >> 172.21.15.67:0/2374951217 conn(0x7f2064075d50 msgr2=0x7f2064076160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.390+0000 7f2069400700 1 -- 172.21.15.67:0/2374951217 shutdown_connections 2024-03-30T02:35:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.390+0000 7f2069400700 1 -- 172.21.15.67:0/2374951217 wait complete. 2024-03-30T02:35:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.391+0000 7f2069400700 1 Processor -- start 2024-03-30T02:35:51.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.391+0000 7f2069400700 1 -- start start 2024-03-30T02:35:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2069400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2064068750 0x7f20640ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2069400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20640690f0 0x7f2064100320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2069400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 0x7f2064100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:51.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2069400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20641a5560 con 0x7f2064106df0 2024-03-30T02:35:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2069400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20641a56d0 con 0x7f2064068750 2024-03-30T02:35:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2069400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20641a5890 con 0x7f20640690f0 2024-03-30T02:35:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2063200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 0x7f2064100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2062800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2064068750 0x7f20640ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2063200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 0x7f2064100860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42640/0 (socket says 172.21.15.67:42640) 2024-03-30T02:35:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2063200700 1 -- 172.21.15.67:0/2115570640 learned_addr learned my addr 172.21.15.67:0/2115570640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2062800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2064068750 0x7f20640ffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37374/0 (socket says 172.21.15.67:37374) 2024-03-30T02:35:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2061e00700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20640690f0 0x7f2064100320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:51.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2063200700 1 -- 172.21.15.67:0/2115570640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20640690f0 msgr2=0x7f2064100320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2063200700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20640690f0 0x7f2064100320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2063200700 1 -- 172.21.15.67:0/2115570640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2064068750 msgr2=0x7f20640ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2063200700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2064068750 0x7f20640ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.392+0000 7f2063200700 1 -- 172.21.15.67:0/2115570640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20540095f0 con 0x7f2064106df0 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.393+0000 7f2063200700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 0x7f2064100860 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f205c00ef30 tx=0x7f205c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.393+0000 7f205b400700 1 -- 172.21.15.67:0/2115570640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c018070 con 0x7f2064106df0 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.393+0000 7f205b400700 1 -- 172.21.15.67:0/2115570640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f205c00f040 con 0x7f2064106df0 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.393+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20641a5b10 con 0x7f2064106df0 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.393+0000 7f205b400700 1 -- 172.21.15.67:0/2115570640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205c00cd00 con 0x7f2064106df0 2024-03-30T02:35:51.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.393+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20641a6010 con 0x7f2064106df0 2024-03-30T02:35:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.395+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2064109310 con 0x7f2064106df0 2024-03-30T02:35:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.399+0000 7f205b400700 1 -- 172.21.15.67:0/2115570640 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f205c003a00 con 0x7f2064106df0 2024-03-30T02:35:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.401+0000 7f205b400700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f20480a1230 0x7f20480a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.401+0000 7f205b400700 1 -- 172.21.15.67:0/2115570640 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f205c0cc3f0 con 0x7f2064106df0 2024-03-30T02:35:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.401+0000 7f2062800700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f20480a1230 0x7f20480a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:51.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.402+0000 7f2062800700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f20480a1230 0x7f20480a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20540098e0 tx=0x7f2054005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:51.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.408+0000 7f205b400700 1 -- 172.21.15.67:0/2115570640 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f205c0957b0 con 0x7f2064106df0 2024-03-30T02:35:51.484 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:51 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1109486116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-30T02:35:51.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:51 smithi067 ceph-mon[37808]: pgmap v267: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-30T02:35:51.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:51 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1109486116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-30T02:35:51.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:51 smithi082 ceph-mon[79176]: pgmap v267: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-30T02:35:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.668+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f20641024a0 con 0x7f2064106df0 2024-03-30T02:35:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.669+0000 7f205b400700 1 -- 172.21.15.67:0/2115570640 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f205c098dd0 con 0x7f2064106df0 2024-03-30T02:35:51.670 INFO:teuthology.orchestra.run.smithi067.stdout:77309411385 2024-03-30T02:35:51.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f20480a1230 msgr2=0x7f20480a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f20480a1230 0x7f20480a36f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20540098e0 tx=0x7f2054005960 comp rx=0 tx=0).stop 2024-03-30T02:35:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 msgr2=0x7f2064100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 0x7f2064100860 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f205c00ef30 tx=0x7f205c00c3a0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 shutdown_connections 2024-03-30T02:35:51.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f20480a1230 0x7f20480a36f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2064106df0 0x7f2064100860 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2064068750 0x7f20640ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 --2- 172.21.15.67:0/2115570640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20640690f0 0x7f2064100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 >> 172.21.15.67:0/2115570640 conn(0x7f2064075d50 msgr2=0x7f20640feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:51.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 shutdown_connections 2024-03-30T02:35:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:51.671+0000 7f2069400700 1 -- 172.21.15.67:0/2115570640 wait complete. 2024-03-30T02:35:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:51 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1109486116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-30T02:35:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:51 smithi170 ceph-mon[78798]: pgmap v267: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-30T02:35:52.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.119+0000 7f31fe600700 1 -- 172.21.15.67:0/1512729216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8111b30 msgr2=0x7f31f8113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:52.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.119+0000 7f31fe600700 1 --2- 172.21.15.67:0/1512729216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8111b30 0x7f31f8113f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009f90 tx=0x7f31f000b330 comp rx=0 tx=0).stop 2024-03-30T02:35:52.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.119+0000 7f31fe600700 1 -- 172.21.15.67:0/1512729216 shutdown_connections 2024-03-30T02:35:52.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.119+0000 7f31fe600700 1 --2- 172.21.15.67:0/1512729216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8111b30 0x7f31f8113f60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:52.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.119+0000 7f31fe600700 1 --2- 172.21.15.67:0/1512729216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31f8074d30 0x7f31f8072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:52.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.119+0000 7f31fe600700 1 --2- 172.21.15.67:0/1512729216 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31f8073520 0x7f31f80739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:52.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.119+0000 7f31fe600700 1 -- 172.21.15.67:0/1512729216 >> 172.21.15.67:0/1512729216 conn(0x7f31f8077fa0 msgr2=0x7f31f80783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:52.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.119+0000 7f31fe600700 1 -- 172.21.15.67:0/1512729216 shutdown_connections 2024-03-30T02:35:52.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.119+0000 7f31fe600700 1 -- 172.21.15.67:0/1512729216 wait complete. 2024-03-30T02:35:52.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.120+0000 7f31fe600700 1 Processor -- start 2024-03-30T02:35:52.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.120+0000 7f31fe600700 1 -- start start 2024-03-30T02:35:52.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31fe600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8073520 0x7f31f81a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:52.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31fe600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31f8074d30 0x7f31f81a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31fe600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31f8111b30 0x7f31f819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31fe600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f819de70 con 0x7f31f8111b30 2024-03-30T02:35:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31fe600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f819dfe0 con 0x7f31f8074d30 2024-03-30T02:35:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31fe600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f819e1a0 con 0x7f31f8073520 2024-03-30T02:35:52.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31f7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31f8111b30 0x7f31f819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31f7400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8073520 0x7f31f81a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31f7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31f8111b30 0x7f31f819d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42670/0 (socket says 172.21.15.67:42670) 2024-03-30T02:35:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31f7e00700 1 -- 172.21.15.67:0/3363155845 learned_addr learned my addr 172.21.15.67:0/3363155845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31f7400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8073520 0x7f31f81a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48102/0 (socket says 172.21.15.67:48102) 2024-03-30T02:35:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.121+0000 7f31f6a00700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31f8074d30 0x7f31f81a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:52.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31f7e00700 1 -- 172.21.15.67:0/3363155845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8073520 msgr2=0x7f31f81a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31f7e00700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8073520 0x7f31f81a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31f7e00700 1 -- 172.21.15.67:0/3363155845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31f8074d30 msgr2=0x7f31f81a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31f7e00700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31f8074d30 0x7f31f81a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31f7e00700 1 -- 172.21.15.67:0/3363155845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40095f0 con 0x7f31f8111b30 2024-03-30T02:35:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31f7400700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8073520 0x7f31f81a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:52.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31f7e00700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31f8111b30 0x7f31f819d800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009f60 tx=0x7f31f0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31efe00700 1 -- 172.21.15.67:0/3363155845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f000e040 con 0x7f31f8111b30 2024-03-30T02:35:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31efe00700 1 -- 172.21.15.67:0/3363155845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31f0007e60 con 0x7f31f8111b30 2024-03-30T02:35:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f0009c40 con 0x7f31f8111b30 2024-03-30T02:35:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.122+0000 7f31efe00700 1 -- 172.21.15.67:0/3363155845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f001be80 con 0x7f31f8111b30 2024-03-30T02:35:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.123+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f81aa490 con 0x7f31f8111b30 2024-03-30T02:35:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.124+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f8074670 con 0x7f31f8111b30 2024-03-30T02:35:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.125+0000 7f31efe00700 1 -- 172.21.15.67:0/3363155845 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f31f000b9f0 con 0x7f31f8111b30 2024-03-30T02:35:52.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.127+0000 7f31efe00700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f31dc0a11b0 0x7f31dc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.128+0000 7f31efe00700 1 -- 172.21.15.67:0/3363155845 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f31f00cdef0 con 0x7f31f8111b30 2024-03-30T02:35:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.128+0000 7f31f7400700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f31dc0a11b0 0x7f31dc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.128+0000 7f31f7400700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f31dc0a11b0 0x7f31dc0a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31e8011490 tx=0x7f31e8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:52.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.132+0000 7f31efe00700 1 -- 172.21.15.67:0/3363155845 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f31f0097250 con 0x7f31f8111b30 2024-03-30T02:35:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.356+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f31f804ea80 con 0x7f31f8111b30 2024-03-30T02:35:52.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.356+0000 7f31efe00700 1 -- 172.21.15.67:0/3363155845 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f31f009a870 con 0x7f31f8111b30 2024-03-30T02:35:52.358 INFO:teuthology.orchestra.run.smithi067.stdout:236223201314 2024-03-30T02:35:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f31dc0a11b0 msgr2=0x7f31dc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f31dc0a11b0 0x7f31dc0a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f31e8011490 tx=0x7f31e8009260 comp rx=0 tx=0).stop 2024-03-30T02:35:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31f8111b30 msgr2=0x7f31f819d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31f8111b30 0x7f31f819d800 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009f60 tx=0x7f31f0004f80 comp rx=0 tx=0).stop 2024-03-30T02:35:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 shutdown_connections 2024-03-30T02:35:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f31dc0a11b0 0x7f31dc0a3670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f31f8111b30 0x7f31f819d800 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31f8073520 0x7f31f81a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 --2- 172.21.15.67:0/3363155845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31f8074d30 0x7f31f81a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 >> 172.21.15.67:0/3363155845 conn(0x7f31f8077fa0 msgr2=0x7f31f81107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 shutdown_connections 2024-03-30T02:35:52.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:52.359+0000 7f31fe600700 1 -- 172.21.15.67:0/3363155845 wait complete. 2024-03-30T02:35:52.482 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:52 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2115570640' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-30T02:35:52.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:52 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2115570640' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-30T02:35:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:52 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2115570640' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-30T02:35:53.050 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182450 got 107374182452 for osd.3 2024-03-30T02:35:53.051 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:53.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:53 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3363155845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-30T02:35:53.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:53 smithi067 ceph-mon[37808]: pgmap v268: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-30T02:35:53.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:53 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3363155845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-30T02:35:53.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:53 smithi082 ceph-mon[79176]: pgmap v268: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-30T02:35:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:53 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3363155845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-30T02:35:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:53 smithi170 ceph-mon[78798]: pgmap v268: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-30T02:35:53.747 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411383 got 77309411385 for osd.2 2024-03-30T02:35:53.747 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:53.896 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201312 got 236223201314 for osd.8 2024-03-30T02:35:53.896 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.674+0000 7ffa1de00700 1 -- 172.21.15.67:0/1911949113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18106ef0 msgr2=0x7ffa181072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.674+0000 7ffa1de00700 1 --2- 172.21.15.67:0/1911949113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18106ef0 0x7ffa181072d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffa080099c0 tx=0x7ffa08009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.674+0000 7ffa1de00700 1 -- 172.21.15.67:0/1911949113 shutdown_connections 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.674+0000 7ffa1de00700 1 --2- 172.21.15.67:0/1911949113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18106ef0 0x7ffa181072d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.674+0000 7ffa1de00700 1 --2- 172.21.15.67:0/1911949113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa18101820 0x7ffa1810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.674+0000 7ffa1de00700 1 --2- 172.21.15.67:0/1911949113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18100e60 0x7ffa181012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.674+0000 7ffa1de00700 1 -- 172.21.15.67:0/1911949113 >> 172.21.15.67:0/1911949113 conn(0x7ffa18077fa0 msgr2=0x7ffa180783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.675+0000 7ffa1de00700 1 -- 172.21.15.67:0/1911949113 shutdown_connections 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.675+0000 7ffa1de00700 1 -- 172.21.15.67:0/1911949113 wait complete. 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.676+0000 7ffa1de00700 1 Processor -- start 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.676+0000 7ffa1de00700 1 -- start start 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa1de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa18100e60 0x7ffa18199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa1de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18101820 0x7ffa181999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa1de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18106ef0 0x7ffa1819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa1de00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1819e370 con 0x7ffa18106ef0 2024-03-30T02:35:54.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa1de00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1819e4e0 con 0x7ffa18101820 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa1de00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1819e6a0 con 0x7ffa18100e60 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa18100e60 0x7ffa18199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa16a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa18100e60 0x7ffa18199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48108/0 (socket says 172.21.15.67:48108) 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa16a00700 1 -- 172.21.15.67:0/2656897806 learned_addr learned my addr 172.21.15.67:0/2656897806 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa17400700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18106ef0 0x7ffa1819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.677+0000 7ffa16000700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18101820 0x7ffa181999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa17400700 1 -- 172.21.15.67:0/2656897806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa18100e60 msgr2=0x7ffa18199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa17400700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa18100e60 0x7ffa18199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa17400700 1 -- 172.21.15.67:0/2656897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18101820 msgr2=0x7ffa181999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa17400700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18101820 0x7ffa181999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa17400700 1 -- 172.21.15.67:0/2656897806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa080095f0 con 0x7ffa18106ef0 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa16000700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18101820 0x7ffa181999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:54.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa17400700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18106ef0 0x7ffa1819dd60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007f10 tx=0x7ffa1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa1ca00700 1 -- 172.21.15.67:0/2656897806 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10004db0 con 0x7ffa18106ef0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa1ca00700 1 -- 172.21.15.67:0/2656897806 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa1000f040 con 0x7ffa18106ef0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1819e980 con 0x7ffa18106ef0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.678+0000 7ffa1ca00700 1 -- 172.21.15.67:0/2656897806 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10005830 con 0x7ffa18106ef0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.679+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa181a61e0 con 0x7ffa18106ef0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.680+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa18104180 con 0x7ffa18106ef0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.682+0000 7ffa1ca00700 1 -- 172.21.15.67:0/2656897806 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7ffa1000db20 con 0x7ffa18106ef0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.684+0000 7ffa1ca00700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9fc0a11b0 0x7ff9fc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.684+0000 7ffa1ca00700 1 -- 172.21.15.67:0/2656897806 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7ffa100cc570 con 0x7ffa18106ef0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.684+0000 7ffa16a00700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9fc0a11b0 0x7ff9fc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.685+0000 7ffa16a00700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9fc0a11b0 0x7ff9fc0a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009f50 tx=0x7ffa08005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.688+0000 7ffa1ca00700 1 -- 172.21.15.67:0/2656897806 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffa10095840 con 0x7ffa18106ef0 2024-03-30T02:35:54.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.929+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7ffa1819a780 con 0x7ffa18106ef0 2024-03-30T02:35:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.929+0000 7ffa1ca00700 1 -- 172.21.15.67:0/2656897806 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa10098e60 con 0x7ffa18106ef0 2024-03-30T02:35:54.015 INFO:teuthology.orchestra.run.smithi067.stdout:266287972382 2024-03-30T02:35:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9fc0a11b0 msgr2=0x7ff9fc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9fc0a11b0 0x7ff9fc0a3670 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009f50 tx=0x7ffa08005fb0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18106ef0 msgr2=0x7ffa1819dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18106ef0 0x7ffa1819dd60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007f10 tx=0x7ffa1000d3b0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 shutdown_connections 2024-03-30T02:35:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff9fc0a11b0 0x7ff9fc0a3670 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffa18106ef0 0x7ffa1819dd60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa18100e60 0x7ffa18199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 --2- 172.21.15.67:0/2656897806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa18101820 0x7ffa181999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 >> 172.21.15.67:0/2656897806 conn(0x7ffa18077fa0 msgr2=0x7ffa1810c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 shutdown_connections 2024-03-30T02:35:54.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:53.932+0000 7ffa1de00700 1 -- 172.21.15.67:0/2656897806 wait complete. 2024-03-30T02:35:54.288 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561063 got 188978561065 for osd.6 2024-03-30T02:35:54.288 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:54.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:54 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2656897806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-30T02:35:54.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.553+0000 7f1805200700 1 -- 172.21.15.67:0/519205024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1800073a70 msgr2=0x7f1800111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.553+0000 7f1805200700 1 --2- 172.21.15.67:0/519205024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1800073a70 0x7f1800111680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009990 tx=0x7f17ec009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.553+0000 7f1805200700 1 -- 172.21.15.67:0/519205024 shutdown_connections 2024-03-30T02:35:54.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.553+0000 7f1805200700 1 --2- 172.21.15.67:0/519205024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800111bc0 0x7f1800113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.553+0000 7f1805200700 1 --2- 172.21.15.67:0/519205024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1800073150 0x7f1800073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.553+0000 7f1805200700 1 --2- 172.21.15.67:0/519205024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1800073a70 0x7f1800111680 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.553+0000 7f1805200700 1 -- 172.21.15.67:0/519205024 >> 172.21.15.67:0/519205024 conn(0x7f18000fc6e0 msgr2=0x7f18000feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:54.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.554+0000 7f1805200700 1 -- 172.21.15.67:0/519205024 shutdown_connections 2024-03-30T02:35:54.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.554+0000 7f1805200700 1 -- 172.21.15.67:0/519205024 wait complete. 2024-03-30T02:35:54.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.555+0000 7f1805200700 1 Processor -- start 2024-03-30T02:35:54.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.555+0000 7f1805200700 1 -- start start 2024-03-30T02:35:54.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.555+0000 7f1805200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1800073150 0x7f180019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.555+0000 7f1805200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1800073a70 0x7f180019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.555+0000 7f1805200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800111bc0 0x7f18001a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.555+0000 7f1805200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18001a2580 con 0x7f1800073a70 2024-03-30T02:35:54.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.555+0000 7f1805200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18001a26f0 con 0x7f1800073150 2024-03-30T02:35:54.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.555+0000 7f1805200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18001a28b0 con 0x7f1800111bc0 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17fea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1800073a70 0x7f180019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17ff400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1800073150 0x7f180019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17ffe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800111bc0 0x7f18001a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17fea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1800073a70 0x7f180019dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42700/0 (socket says 172.21.15.67:42700) 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17fea00700 1 -- 172.21.15.67:0/1335586173 learned_addr learned my addr 172.21.15.67:0/1335586173 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17ff400700 1 -- 172.21.15.67:0/1335586173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800111bc0 msgr2=0x7f18001a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17ff400700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800111bc0 0x7f18001a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17ff400700 1 -- 172.21.15.67:0/1335586173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1800073a70 msgr2=0x7f180019dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17ff400700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1800073a70 0x7f180019dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17ff400700 1 -- 172.21.15.67:0/1335586173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec0095f0 con 0x7f1800073150 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.556+0000 7f17ff400700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1800073150 0x7f180019d6d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9f0 tx=0x7f17f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:54.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.557+0000 7f17f7e00700 1 -- 172.21.15.67:0/1335586173 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f000c9c0 con 0x7f1800073150 2024-03-30T02:35:54.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.557+0000 7f17f7e00700 1 -- 172.21.15.67:0/1335586173 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17f0010430 con 0x7f1800073150 2024-03-30T02:35:54.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.557+0000 7f17f7e00700 1 -- 172.21.15.67:0/1335586173 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0018680 con 0x7f1800073150 2024-03-30T02:35:54.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.557+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18001a2b90 con 0x7f1800073150 2024-03-30T02:35:54.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.557+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18001aa3f0 con 0x7f1800073150 2024-03-30T02:35:54.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.558+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18001040c0 con 0x7f1800073150 2024-03-30T02:35:54.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.561+0000 7f17f7e00700 1 -- 172.21.15.67:0/1335586173 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f17f0010860 con 0x7f1800073150 2024-03-30T02:35:54.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.563+0000 7f17f7e00700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17e40a1230 0x7f17e40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.563+0000 7f17f7e00700 1 -- 172.21.15.67:0/1335586173 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f17f0014070 con 0x7f1800073150 2024-03-30T02:35:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.563+0000 7f17fea00700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17e40a1230 0x7f17e40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.564+0000 7f17fea00700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17e40a1230 0x7f17e40a36f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f180019ebe0 tx=0x7f17ec00b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:54.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.567+0000 7f17f7e00700 1 -- 172.21.15.67:0/1335586173 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f17f00529f0 con 0x7f1800073150 2024-03-30T02:35:54.653 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:54 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2656897806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-30T02:35:54.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.666+0000 7fb4ef600700 1 -- 172.21.15.67:0/982368796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e8074d30 msgr2=0x7fb4e8072fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.666+0000 7fb4ef600700 1 --2- 172.21.15.67:0/982368796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e8074d30 0x7fb4e8072fe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009990 tx=0x7fb4dc009ca0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.666+0000 7fb4ef600700 1 -- 172.21.15.67:0/982368796 shutdown_connections 2024-03-30T02:35:54.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.666+0000 7fb4ef600700 1 --2- 172.21.15.67:0/982368796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e8074d30 0x7fb4e8072fe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.666+0000 7fb4ef600700 1 --2- 172.21.15.67:0/982368796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4e8111b30 0x7fb4e8113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.666+0000 7fb4ef600700 1 --2- 172.21.15.67:0/982368796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e8073520 0x7fb4e80739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.666+0000 7fb4ef600700 1 -- 172.21.15.67:0/982368796 >> 172.21.15.67:0/982368796 conn(0x7fb4e8077fa0 msgr2=0x7fb4e80783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:54.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.666+0000 7fb4ef600700 1 -- 172.21.15.67:0/982368796 shutdown_connections 2024-03-30T02:35:54.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.666+0000 7fb4ef600700 1 -- 172.21.15.67:0/982368796 wait complete. 2024-03-30T02:35:54.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.668+0000 7fb4ef600700 1 Processor -- start 2024-03-30T02:35:54.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.668+0000 7fb4ef600700 1 -- start start 2024-03-30T02:35:54.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.668+0000 7fb4ef600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e8073520 0x7fb4e81a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ef600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e8074d30 0x7fb4e81a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ef600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4e8111b30 0x7fb4e819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ef600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e819dd80 con 0x7fb4e8074d30 2024-03-30T02:35:54.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ef600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e819def0 con 0x7fb4e8073520 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ef600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e819e0b0 con 0x7fb4e8111b30 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ecc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e8073520 0x7fb4e81a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ecc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e8073520 0x7fb4e81a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37448/0 (socket says 172.21.15.67:37448) 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ecc00700 1 -- 172.21.15.67:0/2276061331 learned_addr learned my addr 172.21.15.67:0/2276061331 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ed600700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4e8111b30 0x7fb4e819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4e7e00700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e8074d30 0x7fb4e81a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ecc00700 1 -- 172.21.15.67:0/2276061331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4e8111b30 msgr2=0x7fb4e819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ecc00700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4e8111b30 0x7fb4e819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ecc00700 1 -- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e8074d30 msgr2=0x7fb4e81a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.669+0000 7fb4ecc00700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e8074d30 0x7fb4e81a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.670+0000 7fb4ecc00700 1 -- 172.21.15.67:0/2276061331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d4009520 con 0x7fb4e8073520 2024-03-30T02:35:54.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.670+0000 7fb4ecc00700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e8073520 0x7fb4e81a35e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc000c00 tx=0x7fb4dc005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:54.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.670+0000 7fb4e5600700 1 -- 172.21.15.67:0/2276061331 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc01d070 con 0x7fb4e8073520 2024-03-30T02:35:54.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.670+0000 7fb4e5600700 1 -- 172.21.15.67:0/2276061331 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4dc022470 con 0x7fb4e8073520 2024-03-30T02:35:54.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.670+0000 7fb4e5600700 1 -- 172.21.15.67:0/2276061331 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc017650 con 0x7fb4e8073520 2024-03-30T02:35:54.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.670+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc0095f0 con 0x7fb4e8073520 2024-03-30T02:35:54.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.670+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e819e560 con 0x7fb4e8073520 2024-03-30T02:35:54.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.671+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e8074670 con 0x7fb4e8073520 2024-03-30T02:35:54.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.674+0000 7fb4e5600700 1 -- 172.21.15.67:0/2276061331 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb4dc0225e0 con 0x7fb4e8073520 2024-03-30T02:35:54.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.675+0000 7fb4e5600700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb4d00a11b0 0x7fb4d00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:54.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.676+0000 7fb4e5600700 1 -- 172.21.15.67:0/2276061331 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb4dc0ce1f0 con 0x7fb4e8073520 2024-03-30T02:35:54.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.676+0000 7fb4e7e00700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb4d00a11b0 0x7fb4d00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:54.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.676+0000 7fb4e7e00700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb4d00a11b0 0x7fb4d00a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4e819ee30 tx=0x7fb4d4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:54.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.677+0000 7fb4e5600700 1 -- 172.21.15.67:0/2276061331 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb4dc097670 con 0x7fb4e8073520 2024-03-30T02:35:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:54 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2656897806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-30T02:35:54.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.810+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1800074890 con 0x7f1800073150 2024-03-30T02:35:54.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.811+0000 7f17f7e00700 1 -- 172.21.15.67:0/1335586173 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17f0096160 con 0x7f1800073150 2024-03-30T02:35:54.812 INFO:teuthology.orchestra.run.smithi067.stdout:206158430246 2024-03-30T02:35:54.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17e40a1230 msgr2=0x7f17e40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17e40a1230 0x7f17e40a36f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f180019ebe0 tx=0x7f17ec00b260 comp rx=0 tx=0).stop 2024-03-30T02:35:54.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1800073150 msgr2=0x7f180019d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1800073150 0x7f180019d6d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9f0 tx=0x7f17f000ed00 comp rx=0 tx=0).stop 2024-03-30T02:35:54.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 shutdown_connections 2024-03-30T02:35:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f17e40a1230 0x7f17e40a36f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1800111bc0 0x7f18001a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1800073150 0x7f180019d6d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 --2- 172.21.15.67:0/1335586173 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1800073a70 0x7f180019dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 >> 172.21.15.67:0/1335586173 conn(0x7f18000fc6e0 msgr2=0x7f18000fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 shutdown_connections 2024-03-30T02:35:54.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.813+0000 7f1805200700 1 -- 172.21.15.67:0/1335586173 wait complete. 2024-03-30T02:35:54.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.922+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb4e804ea80 con 0x7fb4e8073520 2024-03-30T02:35:54.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.922+0000 7fb4e5600700 1 -- 172.21.15.67:0/2276061331 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4dc027090 con 0x7fb4e8073520 2024-03-30T02:35:54.923 INFO:teuthology.orchestra.run.smithi067.stdout:158913789998 2024-03-30T02:35:54.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb4d00a11b0 msgr2=0x7fb4d00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb4d00a11b0 0x7fb4d00a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4e819ee30 tx=0x7fb4d4009260 comp rx=0 tx=0).stop 2024-03-30T02:35:54.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e8073520 msgr2=0x7fb4e81a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:54.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e8073520 0x7fb4e81a35e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc000c00 tx=0x7fb4dc005740 comp rx=0 tx=0).stop 2024-03-30T02:35:54.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 shutdown_connections 2024-03-30T02:35:54.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb4d00a11b0 0x7fb4d00a3670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4e8111b30 0x7fb4e819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4e8073520 0x7fb4e81a35e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 --2- 172.21.15.67:0/2276061331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4e8074d30 0x7fb4e81a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:54.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 >> 172.21.15.67:0/2276061331 conn(0x7fb4e8077fa0 msgr2=0x7fb4e81107c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:54.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 shutdown_connections 2024-03-30T02:35:54.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:54.925+0000 7fb4ef600700 1 -- 172.21.15.67:0/2276061331 wait complete. 2024-03-30T02:35:54.950 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972380 got 266287972382 for osd.9 2024-03-30T02:35:54.951 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:55.480 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:55 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1335586173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-30T02:35:55.480 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:55 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2276061331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-30T02:35:55.480 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:55 smithi067 ceph-mon[37808]: pgmap v269: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-30T02:35:55.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:55 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1335586173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-30T02:35:55.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:55 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2276061331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-30T02:35:55.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:55 smithi082 ceph-mon[79176]: pgmap v269: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-30T02:35:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:55 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1335586173' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-30T02:35:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:55 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2276061331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-30T02:35:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:55 smithi170 ceph-mon[78798]: pgmap v269: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-30T02:35:55.771 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430244 got 206158430246 for osd.7 2024-03-30T02:35:55.772 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.233+0000 7f0539000700 1 -- 172.21.15.67:0/1822189270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0534074d30 msgr2=0x7f0534072fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.233+0000 7f0539000700 1 --2- 172.21.15.67:0/1822189270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0534074d30 0x7f0534072fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f05240098f0 tx=0x7f0524009c00 comp rx=0 tx=0).stop 2024-03-30T02:35:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.233+0000 7f0539000700 1 -- 172.21.15.67:0/1822189270 shutdown_connections 2024-03-30T02:35:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.233+0000 7f0539000700 1 --2- 172.21.15.67:0/1822189270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0534111b30 0x7f0534113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.233+0000 7f0539000700 1 --2- 172.21.15.67:0/1822189270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0534074d30 0x7f0534072fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.233+0000 7f0539000700 1 --2- 172.21.15.67:0/1822189270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0534073520 0x7f05340739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.233+0000 7f0539000700 1 -- 172.21.15.67:0/1822189270 >> 172.21.15.67:0/1822189270 conn(0x7f0534077fa0 msgr2=0x7f05340783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.234+0000 7f0539000700 1 -- 172.21.15.67:0/1822189270 shutdown_connections 2024-03-30T02:35:56.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.234+0000 7f0539000700 1 -- 172.21.15.67:0/1822189270 wait complete. 2024-03-30T02:35:56.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.235+0000 7f0539000700 1 Processor -- start 2024-03-30T02:35:56.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.235+0000 7f0539000700 1 -- start start 2024-03-30T02:35:56.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.235+0000 7f0539000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0534073520 0x7f05341a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:56.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.235+0000 7f0539000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0534074d30 0x7f05341a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:56.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0539000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0534111b30 0x7f053419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:56.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0539000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053419de30 con 0x7f0534074d30 2024-03-30T02:35:56.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0539000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053419dfa0 con 0x7f0534073520 2024-03-30T02:35:56.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0539000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053419e160 con 0x7f0534111b30 2024-03-30T02:35:56.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0531e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0534073520 0x7f05341a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:56.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0531e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0534073520 0x7f05341a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35016/0 (socket says 172.21.15.67:35016) 2024-03-30T02:35:56.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0531e00700 1 -- 172.21.15.67:0/1986530744 learned_addr learned my addr 172.21.15.67:0/1986530744 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:56.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0531400700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0534074d30 0x7f05341a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:56.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0532800700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0534111b30 0x7f053419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:56.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0531e00700 1 -- 172.21.15.67:0/1986530744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0534111b30 msgr2=0x7f053419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0531e00700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0534111b30 0x7f053419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0531e00700 1 -- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0534074d30 msgr2=0x7f05341a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0531e00700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0534074d30 0x7f05341a3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.236+0000 7f0531e00700 1 -- 172.21.15.67:0/1986530744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524009520 con 0x7f0534073520 2024-03-30T02:35:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.237+0000 7f0531e00700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0534073520 0x7f05341a3680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0524005950 tx=0x7f052400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.237+0000 7f052aa00700 1 -- 172.21.15.67:0/1986530744 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052401c070 con 0x7f0534073520 2024-03-30T02:35:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.237+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053419e3e0 con 0x7f0534073520 2024-03-30T02:35:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.237+0000 7f052aa00700 1 -- 172.21.15.67:0/1986530744 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052400bb20 con 0x7f0534073520 2024-03-30T02:35:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.237+0000 7f052aa00700 1 -- 172.21.15.67:0/1986530744 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05240038e0 con 0x7f0534073520 2024-03-30T02:35:56.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.237+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05341aa670 con 0x7f0534073520 2024-03-30T02:35:56.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.238+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0534074670 con 0x7f0534073520 2024-03-30T02:35:56.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.240+0000 7f052aa00700 1 -- 172.21.15.67:0/1986530744 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f0524020030 con 0x7f0534073520 2024-03-30T02:35:56.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.242+0000 7f052aa00700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f05180a1230 0x7f05180a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.243+0000 7f052aa00700 1 -- 172.21.15.67:0/1986530744 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f05240cdbd0 con 0x7f0534073520 2024-03-30T02:35:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.243+0000 7f0531400700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f05180a1230 0x7f05180a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:56.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.243+0000 7f0531400700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f05180a1230 0x7f05180a36f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f053419ebe0 tx=0x7f0520009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:56.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.246+0000 7f052aa00700 1 -- 172.21.15.67:0/1986530744 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0524096f30 con 0x7f0534073520 2024-03-30T02:35:56.264 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789996 got 158913789998 for osd.5 2024-03-30T02:35:56.265 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:56.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.549+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f053404ea80 con 0x7f0534073520 2024-03-30T02:35:56.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.550+0000 7f052aa00700 1 -- 172.21.15.67:0/1986530744 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f052409a550 con 0x7f0534073520 2024-03-30T02:35:56.551 INFO:teuthology.orchestra.run.smithi067.stdout:137438953521 2024-03-30T02:35:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f05180a1230 msgr2=0x7f05180a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f05180a1230 0x7f05180a36f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f053419ebe0 tx=0x7f0520009260 comp rx=0 tx=0).stop 2024-03-30T02:35:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0534073520 msgr2=0x7f05341a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0534073520 0x7f05341a3680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0524005950 tx=0x7f052400fa00 comp rx=0 tx=0).stop 2024-03-30T02:35:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 shutdown_connections 2024-03-30T02:35:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f05180a1230 0x7f05180a36f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0534111b30 0x7f053419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0534073520 0x7f05341a3680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 --2- 172.21.15.67:0/1986530744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0534074d30 0x7f05341a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 >> 172.21.15.67:0/1986530744 conn(0x7f0534077fa0 msgr2=0x7f0534110860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:56.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 shutdown_connections 2024-03-30T02:35:56.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.553+0000 7f0539000700 1 -- 172.21.15.67:0/1986530744 wait complete. 2024-03-30T02:35:56.586 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:56.586 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:56 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:56.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:56.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:56 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.719+0000 7f1910c00700 1 -- 172.21.15.67:0/2897904307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 msgr2=0x7f190c1012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.719+0000 7f1910c00700 1 --2- 172.21.15.67:0/2897904307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 0x7f190c1012e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f18f80099c0 tx=0x7f18f8009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.719+0000 7f1910c00700 1 -- 172.21.15.67:0/2897904307 shutdown_connections 2024-03-30T02:35:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.719+0000 7f1910c00700 1 --2- 172.21.15.67:0/2897904307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f190c101820 0x7f190c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.719+0000 7f1910c00700 1 --2- 172.21.15.67:0/2897904307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f190c106ef0 0x7f190c1072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.719+0000 7f1910c00700 1 --2- 172.21.15.67:0/2897904307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 0x7f190c1012e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.719+0000 7f1910c00700 1 -- 172.21.15.67:0/2897904307 >> 172.21.15.67:0/2897904307 conn(0x7f190c077fa0 msgr2=0x7f190c0783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.719+0000 7f1910c00700 1 -- 172.21.15.67:0/2897904307 shutdown_connections 2024-03-30T02:35:56.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.719+0000 7f1910c00700 1 -- 172.21.15.67:0/2897904307 wait complete. 2024-03-30T02:35:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.721+0000 7f1910c00700 1 Processor -- start 2024-03-30T02:35:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.721+0000 7f1910c00700 1 -- start start 2024-03-30T02:35:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.721+0000 7f1910c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 0x7f190c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:56.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f1910c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f190c101820 0x7f190c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:56.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f1910c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f190c106ef0 0x7f190c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:56.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f1910c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c19c160 con 0x7f190c101820 2024-03-30T02:35:56.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f1910c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c19c2d0 con 0x7f190c100e60 2024-03-30T02:35:56.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f1910c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c19c490 con 0x7f190c106ef0 2024-03-30T02:35:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f1909e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 0x7f190c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f1909e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 0x7f190c197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35028/0 (socket says 172.21.15.67:35028) 2024-03-30T02:35:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f1909e00700 1 -- 172.21.15.67:0/530492136 learned_addr learned my addr 172.21.15.67:0/530492136 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:56.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f190a800700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f190c106ef0 0x7f190c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.722+0000 7f1909400700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f190c101820 0x7f190c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1909e00700 1 -- 172.21.15.67:0/530492136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f190c106ef0 msgr2=0x7f190c19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1909e00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f190c106ef0 0x7f190c19bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1909e00700 1 -- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f190c101820 msgr2=0x7f190c1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1909e00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f190c101820 0x7f190c1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1909e00700 1 -- 172.21.15.67:0/530492136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f80095f0 con 0x7f190c100e60 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1909400700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f190c101820 0x7f190c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1909e00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 0x7f190c197260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00e9a0 tx=0x7f18fc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1902a00700 1 -- 172.21.15.67:0/530492136 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc009790 con 0x7f190c100e60 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1902a00700 1 -- 172.21.15.67:0/530492136 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18fc00cbd0 con 0x7f190c100e60 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1902a00700 1 -- 172.21.15.67:0/530492136 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc018680 con 0x7f190c100e60 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.723+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c19c770 con 0x7f190c100e60 2024-03-30T02:35:56.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.724+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c1a3fd0 con 0x7f190c100e60 2024-03-30T02:35:56.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.725+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c1092f0 con 0x7f190c100e60 2024-03-30T02:35:56.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.726+0000 7f1902a00700 1 -- 172.21.15.67:0/530492136 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f18fc00cd40 con 0x7f190c100e60 2024-03-30T02:35:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.728+0000 7f1902a00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18f00a1230 0x7f18f00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.728+0000 7f1902a00700 1 -- 172.21.15.67:0/530492136 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f18fc014070 con 0x7f190c100e60 2024-03-30T02:35:56.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.728+0000 7f1909400700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18f00a1230 0x7f18f00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:56.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.729+0000 7f1909400700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18f00a1230 0x7f18f00a36f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009990 tx=0x7f18f8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:56.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.732+0000 7f1902a00700 1 -- 172.21.15.67:0/530492136 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f18fc0529f0 con 0x7f190c100e60 2024-03-30T02:35:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:56 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:56.954 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953519 got 137438953521 for osd.4 2024-03-30T02:35:56.954 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:56.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.959+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f190c198570 con 0x7f190c100e60 2024-03-30T02:35:56.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.960+0000 7f1902a00700 1 -- 172.21.15.67:0/530492136 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f18fc096210 con 0x7f190c100e60 2024-03-30T02:35:56.961 INFO:teuthology.orchestra.run.smithi067.stdout:287762808860 2024-03-30T02:35:56.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18f00a1230 msgr2=0x7f18f00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18f00a1230 0x7f18f00a36f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009990 tx=0x7f18f8005fb0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 msgr2=0x7f190c197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:56.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 0x7f190c197260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00e9a0 tx=0x7f18fc00ecb0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 shutdown_connections 2024-03-30T02:35:56.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18f00a1230 0x7f18f00a36f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f190c106ef0 0x7f190c19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f190c100e60 0x7f190c197260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 --2- 172.21.15.67:0/530492136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f190c101820 0x7f190c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:56.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.962+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 >> 172.21.15.67:0/530492136 conn(0x7f190c077fa0 msgr2=0x7f190c0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:56.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.963+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 shutdown_connections 2024-03-30T02:35:56.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:56.963+0000 7f1910c00700 1 -- 172.21.15.67:0/530492136 wait complete. 2024-03-30T02:35:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.370+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2156642462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf0101820 msgr2=0x7f3cf010dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.370+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2156642462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf0101820 0x7f3cf010dc10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009fc0 tx=0x7f3ce800b330 comp rx=0 tx=0).stop 2024-03-30T02:35:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.370+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2156642462 shutdown_connections 2024-03-30T02:35:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.370+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2156642462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf0101820 0x7f3cf010dc10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.370+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2156642462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cf0106ef0 0x7f3cf01072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.370+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2156642462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf0100e60 0x7f3cf01012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.370+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2156642462 >> 172.21.15.67:0/2156642462 conn(0x7f3cf0077fa0 msgr2=0x7f3cf00783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.370+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2156642462 shutdown_connections 2024-03-30T02:35:57.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.370+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2156642462 wait complete. 2024-03-30T02:35:57.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.372+0000 7f3cf4a00700 1 Processor -- start 2024-03-30T02:35:57.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.372+0000 7f3cf4a00700 1 -- start start 2024-03-30T02:35:57.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.372+0000 7f3cf4a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf0100e60 0x7f3cf01994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:57.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.372+0000 7f3cf4a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cf0101820 0x7f3cf0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.372+0000 7f3cf4a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf0106ef0 0x7f3cf019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.372+0000 7f3cf4a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf019e3c0 con 0x7f3cf0100e60 2024-03-30T02:35:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.372+0000 7f3cf4a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf019e530 con 0x7f3cf0106ef0 2024-03-30T02:35:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.372+0000 7f3cf4a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf019e6f0 con 0x7f3cf0101820 2024-03-30T02:35:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.372+0000 7f3cede00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf0100e60 0x7f3cf01994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cede00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf0100e60 0x7f3cf01994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49628/0 (socket says 172.21.15.67:49628) 2024-03-30T02:35:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cede00700 1 -- 172.21.15.67:0/2822130952 learned_addr learned my addr 172.21.15.67:0/2822130952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cee800700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf0106ef0 0x7f3cf019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:57.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3ced400700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cf0101820 0x7f3cf0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cede00700 1 -- 172.21.15.67:0/2822130952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cf0101820 msgr2=0x7f3cf0199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cede00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cf0101820 0x7f3cf0199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cede00700 1 -- 172.21.15.67:0/2822130952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf0106ef0 msgr2=0x7f3cf019ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cede00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf0106ef0 0x7f3cf019ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cede00700 1 -- 172.21.15.67:0/2822130952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc0095f0 con 0x7f3cf0100e60 2024-03-30T02:35:57.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cee800700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf0106ef0 0x7f3cf019ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:35:57.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cede00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf0100e60 0x7f3cf01994c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000e9a0 tx=0x7f3ce000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:57.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3ce6a00700 1 -- 172.21.15.67:0/2822130952 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0009790 con 0x7f3cf0100e60 2024-03-30T02:35:57.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.373+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce8009c40 con 0x7f3cf0100e60 2024-03-30T02:35:57.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.374+0000 7f3ce6a00700 1 -- 172.21.15.67:0/2822130952 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ce0010430 con 0x7f3cf0100e60 2024-03-30T02:35:57.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.374+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf01a6060 con 0x7f3cf0100e60 2024-03-30T02:35:57.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.374+0000 7f3ce6a00700 1 -- 172.21.15.67:0/2822130952 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0018620 con 0x7f3cf0100e60 2024-03-30T02:35:57.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.375+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf01024f0 con 0x7f3cf0100e60 2024-03-30T02:35:57.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.378+0000 7f3ce6a00700 1 -- 172.21.15.67:0/2822130952 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3ce00105a0 con 0x7f3cf0100e60 2024-03-30T02:35:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.379+0000 7f3ce6a00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3cd40a11b0 0x7f3cd40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.379+0000 7f3ce6a00700 1 -- 172.21.15.67:0/2822130952 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3ce0014070 con 0x7f3cf0100e60 2024-03-30T02:35:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.379+0000 7f3ced400700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3cd40a11b0 0x7f3cd40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:57.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.380+0000 7f3ced400700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3cd40a11b0 0x7f3cd40a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3cf019aa20 tx=0x7f3cdc009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:57.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.383+0000 7f3ce6a00700 1 -- 172.21.15.67:0/2822130952 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3ce00529f0 con 0x7f3cf0100e60 2024-03-30T02:35:57.562 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808858 got 287762808860 for osd.10 2024-03-30T02:35:57.562 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:57.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.606+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3cf019a870 con 0x7f3cf0100e60 2024-03-30T02:35:57.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.606+0000 7f3ce6a00700 1 -- 172.21.15.67:0/2822130952 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ce0096200 con 0x7f3cf0100e60 2024-03-30T02:35:57.607 INFO:teuthology.orchestra.run.smithi067.stdout:55834574910 2024-03-30T02:35:57.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.608+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3cd40a11b0 msgr2=0x7f3cd40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.608+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3cd40a11b0 0x7f3cd40a3670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3cf019aa20 tx=0x7f3cdc009310 comp rx=0 tx=0).stop 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.608+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf0100e60 msgr2=0x7f3cf01994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.608+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf0100e60 0x7f3cf01994c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000e9a0 tx=0x7f3ce000ecb0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.608+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 shutdown_connections 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.608+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3cd40a11b0 0x7f3cd40a3670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.609+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3cf0100e60 0x7f3cf01994c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.609+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf0106ef0 0x7f3cf019ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.609+0000 7f3cf4a00700 1 --2- 172.21.15.67:0/2822130952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cf0101820 0x7f3cf0199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.609+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 >> 172.21.15.67:0/2822130952 conn(0x7f3cf0077fa0 msgr2=0x7f3cf00ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.609+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 shutdown_connections 2024-03-30T02:35:57.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:57.609+0000 7f3cf4a00700 1 -- 172.21.15.67:0/2822130952 wait complete. 2024-03-30T02:35:57.632 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:57 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1986530744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-30T02:35:57.632 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:35:57.632 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:35:57.632 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:57 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:57.632 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:57 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/530492136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-30T02:35:57.632 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:57 smithi067 ceph-mon[37808]: pgmap v270: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T02:35:57.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:57 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1986530744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-30T02:35:57.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:57 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:35:57.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:57 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:35:57.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:57 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:57.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:57 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/530492136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-30T02:35:57.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:57 smithi170 ceph-mon[78798]: pgmap v270: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T02:35:57.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:57 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1986530744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-30T02:35:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:57 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:35:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:57 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:35:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:57 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' 2024-03-30T02:35:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:57 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/530492136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-30T02:35:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:57 smithi082 ceph-mon[79176]: pgmap v270: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-30T02:35:58.012 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574907 got 55834574910 for osd.1 2024-03-30T02:35:58.012 DEBUG:teuthology.parallel:result is None 2024-03-30T02:35:58.012 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-30T02:35:58.012 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-03-30T02:35:58.386 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:35:58.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:58 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2822130952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-30T02:35:58.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:58 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2822130952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-30T02:35:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:58 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2822130952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-30T02:35:59.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.547+0000 7fdc7c000700 1 -- 172.21.15.67:0/2230605644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74106ef0 msgr2=0x7fdc741072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:59.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.547+0000 7fdc7c000700 1 --2- 172.21.15.67:0/2230605644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74106ef0 0x7fdc741072d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc680099c0 tx=0x7fdc68009cd0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.548+0000 7fdc7c000700 1 -- 172.21.15.67:0/2230605644 shutdown_connections 2024-03-30T02:35:59.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.548+0000 7fdc7c000700 1 --2- 172.21.15.67:0/2230605644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74106ef0 0x7fdc741072d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.548+0000 7fdc7c000700 1 --2- 172.21.15.67:0/2230605644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc74101820 0x7fdc7410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.548+0000 7fdc7c000700 1 --2- 172.21.15.67:0/2230605644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc74100e60 0x7fdc741012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.548+0000 7fdc7c000700 1 -- 172.21.15.67:0/2230605644 >> 172.21.15.67:0/2230605644 conn(0x7fdc74077fa0 msgr2=0x7fdc740783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:59.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.548+0000 7fdc7c000700 1 -- 172.21.15.67:0/2230605644 shutdown_connections 2024-03-30T02:35:59.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.548+0000 7fdc7c000700 1 -- 172.21.15.67:0/2230605644 wait complete. 2024-03-30T02:35:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.549+0000 7fdc7c000700 1 Processor -- start 2024-03-30T02:35:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.549+0000 7fdc7c000700 1 -- start start 2024-03-30T02:35:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.549+0000 7fdc7c000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc74100e60 0x7fdc74199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.549+0000 7fdc7c000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74101820 0x7fdc741998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:59.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.549+0000 7fdc7c000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc74106ef0 0x7fdc7419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.549+0000 7fdc7c000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7419e290 con 0x7fdc74101820 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.549+0000 7fdc7c000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7419e400 con 0x7fdc74106ef0 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.549+0000 7fdc7c000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7419e5c0 con 0x7fdc74100e60 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc78c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74101820 0x7fdc741998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc78c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74101820 0x7fdc741998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49644/0 (socket says 172.21.15.67:49644) 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc78c00700 1 -- 172.21.15.67:0/1650612556 learned_addr learned my addr 172.21.15.67:0/1650612556 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc7a000700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc74106ef0 0x7fdc7419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc79600700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc74100e60 0x7fdc74199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc78c00700 1 -- 172.21.15.67:0/1650612556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc74100e60 msgr2=0x7fdc74199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc78c00700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc74100e60 0x7fdc74199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc78c00700 1 -- 172.21.15.67:0/1650612556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc74106ef0 msgr2=0x7fdc7419dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc78c00700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc74106ef0 0x7fdc7419dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc78c00700 1 -- 172.21.15.67:0/1650612556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680095f0 con 0x7fdc74101820 2024-03-30T02:35:59.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.550+0000 7fdc78c00700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74101820 0x7fdc741998d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e9a0 tx=0x7fdc6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.551+0000 7fdc72000700 1 -- 172.21.15.67:0/1650612556 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60009790 con 0x7fdc74101820 2024-03-30T02:35:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.551+0000 7fdc72000700 1 -- 172.21.15.67:0/1650612556 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc60010430 con 0x7fdc74101820 2024-03-30T02:35:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.551+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7419e8a0 con 0x7fdc74101820 2024-03-30T02:35:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.551+0000 7fdc72000700 1 -- 172.21.15.67:0/1650612556 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60018680 con 0x7fdc74101820 2024-03-30T02:35:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.551+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc741a6100 con 0x7fdc74101820 2024-03-30T02:35:59.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.552+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc74104180 con 0x7fdc74101820 2024-03-30T02:35:59.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.554+0000 7fdc72000700 1 -- 172.21.15.67:0/1650612556 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fdc600105a0 con 0x7fdc74101820 2024-03-30T02:35:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.555+0000 7fdc72000700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc5c0a11b0 0x7fdc5c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:35:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.555+0000 7fdc72000700 1 -- 172.21.15.67:0/1650612556 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fdc60014070 con 0x7fdc74101820 2024-03-30T02:35:59.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.556+0000 7fdc79600700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc5c0a11b0 0x7fdc5c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:35:59.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.556+0000 7fdc79600700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc5c0a11b0 0x7fdc5c0a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009fd0 tx=0x7fdc68019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:35:59.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.559+0000 7fdc72000700 1 -- 172.21.15.67:0/1650612556 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdc600529f0 con 0x7fdc74101820 2024-03-30T02:35:59.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:35:59 smithi067 ceph-mon[37808]: pgmap v271: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-30T02:35:59.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:35:59 smithi170 ceph-mon[78798]: pgmap v271: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-30T02:35:59.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:35:59 smithi082 ceph-mon[79176]: pgmap v271: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-30T02:35:59.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.767+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc7419a6a0 con 0x7fdc5c0a11b0 2024-03-30T02:35:59.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.782+0000 7fdc72000700 1 -- 172.21.15.67:0/1650612556 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427006 (secure 0 0 0) 0x7fdc7419a6a0 con 0x7fdc5c0a11b0 2024-03-30T02:35:59.790 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc5c0a11b0 msgr2=0x7fdc5c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc5c0a11b0 0x7fdc5c0a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009fd0 tx=0x7fdc68019040 comp rx=0 tx=0).stop 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74101820 msgr2=0x7fdc741998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74101820 0x7fdc741998d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000e9a0 tx=0x7fdc6000ecb0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 shutdown_connections 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc5c0a11b0 0x7fdc5c0a3670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc74106ef0 0x7fdc7419dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc74100e60 0x7fdc74199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 --2- 172.21.15.67:0/1650612556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc74101820 0x7fdc741998d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 >> 172.21.15.67:0/1650612556 conn(0x7fdc74077fa0 msgr2=0x7fdc7410c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 shutdown_connections 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:35:59.787+0000 7fdc7c000700 1 -- 172.21.15.67:0/1650612556 wait complete. 2024-03-30T02:35:59.793 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-30T02:36:00.180 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-03-30T02:35:59.016377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660757,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":548,"num_read_kb":403,"num_write":423,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360356,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581468,"statfs":{"total":1151940427776,"available":1151571423232,"internally_reserved":0,"allocated":11796480,"data_stored":7577605,"data_compressed":324744,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19130,"internal_metadata":357020998},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":64,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003297"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206850+0000","last_change":"2024-03-30T02:34:46.755257+0000","last_active":"2024-03-30T02:35:27.206850+0000","last_peered":"2024-03-30T02:35:27.206850+0000","last_clean":"2024-03-30T02:35:27.206850+0000","last_became_active":"2024-03-30T02:34:46.754922+0000","last_became_peered":"2024-03-30T02:34:46.754922+0000","last_unstale":"2024-03-30T02:35:27.206850+0000","last_undegraded":"2024-03-30T02:35:27.206850+0000","last_fullsized":"2024-03-30T02:35:27.206850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:09.980124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207483+0000","last_change":"2024-03-30T02:34:46.742846+0000","last_active":"2024-03-30T02:35:27.207483+0000","last_peered":"2024-03-30T02:35:27.207483+0000","last_clean":"2024-03-30T02:35:27.207483+0000","last_became_active":"2024-03-30T02:34:46.738096+0000","last_became_peered":"2024-03-30T02:34:46.738096+0000","last_unstale":"2024-03-30T02:35:27.207483+0000","last_undegraded":"2024-03-30T02:35:27.207483+0000","last_fullsized":"2024-03-30T02:35:27.207483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:43:16.982251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207053+0000","last_change":"2024-03-30T02:34:46.739607+0000","last_active":"2024-03-30T02:35:27.207053+0000","last_peered":"2024-03-30T02:35:27.207053+0000","last_clean":"2024-03-30T02:35:27.207053+0000","last_became_active":"2024-03-30T02:34:46.739230+0000","last_became_peered":"2024-03-30T02:34:46.739230+0000","last_unstale":"2024-03-30T02:35:27.207053+0000","last_undegraded":"2024-03-30T02:35:27.207053+0000","last_fullsized":"2024-03-30T02:35:27.207053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:17.818874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206175+0000","last_change":"2024-03-30T02:34:46.742118+0000","last_active":"2024-03-30T02:35:27.206175+0000","last_peered":"2024-03-30T02:35:27.206175+0000","last_clean":"2024-03-30T02:35:27.206175+0000","last_became_active":"2024-03-30T02:34:46.740845+0000","last_became_peered":"2024-03-30T02:34:46.740845+0000","last_unstale":"2024-03-30T02:35:27.206175+0000","last_undegraded":"2024-03-30T02:35:27.206175+0000","last_fullsized":"2024-03-30T02:35:27.206175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:33:28.595456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210363+0000","last_change":"2024-03-30T02:34:46.739557+0000","last_active":"2024-03-30T02:35:27.210363+0000","last_peered":"2024-03-30T02:35:27.210363+0000","last_clean":"2024-03-30T02:35:27.210363+0000","last_became_active":"2024-03-30T02:34:46.739147+0000","last_became_peered":"2024-03-30T02:34:46.739147+0000","last_unstale":"2024-03-30T02:35:27.210363+0000","last_undegraded":"2024-03-30T02:35:27.210363+0000","last_fullsized":"2024-03-30T02:35:27.210363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:12:52.307386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.211536+0000","last_change":"2024-03-30T02:35:28.211536+0000","last_active":"2024-03-30T02:35:28.211536+0000","last_peered":"2024-03-30T02:35:28.211536+0000","last_clean":"2024-03-30T02:35:28.211536+0000","last_became_active":"2024-03-30T02:35:28.211085+0000","last_became_peered":"2024-03-30T02:35:28.211085+0000","last_unstale":"2024-03-30T02:35:28.211536+0000","last_undegraded":"2024-03-30T02:35:28.211536+0000","last_fullsized":"2024-03-30T02:35:28.211536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:06:34.034277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202193+0000","last_change":"2024-03-30T02:34:46.735466+0000","last_active":"2024-03-30T02:35:27.202193+0000","last_peered":"2024-03-30T02:35:27.202193+0000","last_clean":"2024-03-30T02:35:27.202193+0000","last_became_active":"2024-03-30T02:34:46.734411+0000","last_became_peered":"2024-03-30T02:34:46.734411+0000","last_unstale":"2024-03-30T02:35:27.202193+0000","last_undegraded":"2024-03-30T02:35:27.202193+0000","last_fullsized":"2024-03-30T02:35:27.202193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:57.587768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210992+0000","last_change":"2024-03-30T02:34:46.747001+0000","last_active":"2024-03-30T02:35:27.210992+0000","last_peered":"2024-03-30T02:35:27.210992+0000","last_clean":"2024-03-30T02:35:27.210992+0000","last_became_active":"2024-03-30T02:34:46.740241+0000","last_became_peered":"2024-03-30T02:34:46.740241+0000","last_unstale":"2024-03-30T02:35:27.210992+0000","last_undegraded":"2024-03-30T02:35:27.210992+0000","last_fullsized":"2024-03-30T02:35:27.210992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:05.902190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.475138+0000","last_change":"2024-03-30T02:34:46.745549+0000","last_active":"2024-03-30T02:35:27.475138+0000","last_peered":"2024-03-30T02:35:27.475138+0000","last_clean":"2024-03-30T02:35:27.475138+0000","last_became_active":"2024-03-30T02:34:46.745200+0000","last_became_peered":"2024-03-30T02:34:46.745200+0000","last_unstale":"2024-03-30T02:35:27.475138+0000","last_undegraded":"2024-03-30T02:35:27.475138+0000","last_fullsized":"2024-03-30T02:35:27.475138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:37.931440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207118+0000","last_change":"2024-03-30T02:34:46.735859+0000","last_active":"2024-03-30T02:35:27.207118+0000","last_peered":"2024-03-30T02:35:27.207118+0000","last_clean":"2024-03-30T02:35:27.207118+0000","last_became_active":"2024-03-30T02:34:46.735481+0000","last_became_peered":"2024-03-30T02:34:46.735481+0000","last_unstale":"2024-03-30T02:35:27.207118+0000","last_undegraded":"2024-03-30T02:35:27.207118+0000","last_fullsized":"2024-03-30T02:35:27.207118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:51.037887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202925+0000","last_change":"2024-03-30T02:34:46.741412+0000","last_active":"2024-03-30T02:35:27.202925+0000","last_peered":"2024-03-30T02:35:27.202925+0000","last_clean":"2024-03-30T02:35:27.202925+0000","last_became_active":"2024-03-30T02:34:46.741106+0000","last_became_peered":"2024-03-30T02:34:46.741106+0000","last_unstale":"2024-03-30T02:35:27.202925+0000","last_undegraded":"2024-03-30T02:35:27.202925+0000","last_fullsized":"2024-03-30T02:35:27.202925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:06:08.553028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205930+0000","last_change":"2024-03-30T02:34:46.746205+0000","last_active":"2024-03-30T02:35:27.205930+0000","last_peered":"2024-03-30T02:35:27.205930+0000","last_clean":"2024-03-30T02:35:27.205930+0000","last_became_active":"2024-03-30T02:34:46.745803+0000","last_became_peered":"2024-03-30T02:34:46.745803+0000","last_unstale":"2024-03-30T02:35:27.205930+0000","last_undegraded":"2024-03-30T02:35:27.205930+0000","last_fullsized":"2024-03-30T02:35:27.205930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:05.127095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201535+0000","last_change":"2024-03-30T02:34:46.722978+0000","last_active":"2024-03-30T02:35:27.201535+0000","last_peered":"2024-03-30T02:35:27.201535+0000","last_clean":"2024-03-30T02:35:27.201535+0000","last_became_active":"2024-03-30T02:34:46.722652+0000","last_became_peered":"2024-03-30T02:34:46.722652+0000","last_unstale":"2024-03-30T02:35:27.201535+0000","last_undegraded":"2024-03-30T02:35:27.201535+0000","last_fullsized":"2024-03-30T02:35:27.201535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:28:12.078216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206658+0000","last_change":"2024-03-30T02:34:46.738895+0000","last_active":"2024-03-30T02:35:27.206658+0000","last_peered":"2024-03-30T02:35:27.206658+0000","last_clean":"2024-03-30T02:35:27.206658+0000","last_became_active":"2024-03-30T02:34:46.738510+0000","last_became_peered":"2024-03-30T02:34:46.738510+0000","last_unstale":"2024-03-30T02:35:27.206658+0000","last_undegraded":"2024-03-30T02:35:27.206658+0000","last_fullsized":"2024-03-30T02:35:27.206658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:09.859259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206363+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:35:27.206363+0000","last_peered":"2024-03-30T02:35:27.206363+0000","last_clean":"2024-03-30T02:35:27.206363+0000","last_became_active":"2024-03-30T02:34:46.750301+0000","last_became_peered":"2024-03-30T02:34:46.750301+0000","last_unstale":"2024-03-30T02:35:27.206363+0000","last_undegraded":"2024-03-30T02:35:27.206363+0000","last_fullsized":"2024-03-30T02:35:27.206363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:06.475881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474462+0000","last_change":"2024-03-30T02:34:46.748966+0000","last_active":"2024-03-30T02:35:27.474462+0000","last_peered":"2024-03-30T02:35:27.474462+0000","last_clean":"2024-03-30T02:35:27.474462+0000","last_became_active":"2024-03-30T02:34:46.748650+0000","last_became_peered":"2024-03-30T02:34:46.748650+0000","last_unstale":"2024-03-30T02:35:27.474462+0000","last_undegraded":"2024-03-30T02:35:27.474462+0000","last_fullsized":"2024-03-30T02:35:27.474462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:36.558864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206625+0000","last_change":"2024-03-30T02:34:46.746127+0000","last_active":"2024-03-30T02:35:27.206625+0000","last_peered":"2024-03-30T02:35:27.206625+0000","last_clean":"2024-03-30T02:35:27.206625+0000","last_became_active":"2024-03-30T02:34:46.745806+0000","last_became_peered":"2024-03-30T02:34:46.745806+0000","last_unstale":"2024-03-30T02:35:27.206625+0000","last_undegraded":"2024-03-30T02:35:27.206625+0000","last_fullsized":"2024-03-30T02:35:27.206625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:54.097996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.212243+0000","last_change":"2024-03-30T02:34:46.738391+0000","last_active":"2024-03-30T02:35:27.212243+0000","last_peered":"2024-03-30T02:35:27.212243+0000","last_clean":"2024-03-30T02:35:27.212243+0000","last_became_active":"2024-03-30T02:34:46.737110+0000","last_became_peered":"2024-03-30T02:34:46.737110+0000","last_unstale":"2024-03-30T02:35:27.212243+0000","last_undegraded":"2024-03-30T02:35:27.212243+0000","last_fullsized":"2024-03-30T02:35:27.212243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:20.326107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206276+0000","last_change":"2024-03-30T02:34:46.744522+0000","last_active":"2024-03-30T02:35:27.206276+0000","last_peered":"2024-03-30T02:35:27.206276+0000","last_clean":"2024-03-30T02:35:27.206276+0000","last_became_active":"2024-03-30T02:34:46.744088+0000","last_became_peered":"2024-03-30T02:34:46.744088+0000","last_unstale":"2024-03-30T02:35:27.206276+0000","last_undegraded":"2024-03-30T02:35:27.206276+0000","last_fullsized":"2024-03-30T02:35:27.206276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:35.698979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207803+0000","last_change":"2024-03-30T02:34:50.770855+0000","last_active":"2024-03-30T02:35:27.207803+0000","last_peered":"2024-03-30T02:35:27.207803+0000","last_clean":"2024-03-30T02:35:27.207803+0000","last_became_active":"2024-03-30T02:34:50.770420+0000","last_became_peered":"2024-03-30T02:34:50.770420+0000","last_unstale":"2024-03-30T02:35:27.207803+0000","last_undegraded":"2024-03-30T02:35:27.207803+0000","last_fullsized":"2024-03-30T02:35:27.207803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:00:08.847059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:54.185922+0000","last_change":"2024-03-30T02:34:46.207252+0000","last_active":"2024-03-30T02:35:54.185922+0000","last_peered":"2024-03-30T02:35:54.185922+0000","last_clean":"2024-03-30T02:35:54.185922+0000","last_became_active":"2024-03-30T02:34:46.206916+0000","last_became_peered":"2024-03-30T02:34:46.206916+0000","last_unstale":"2024-03-30T02:35:54.185922+0000","last_undegraded":"2024-03-30T02:35:54.185922+0000","last_fullsized":"2024-03-30T02:35:54.185922+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:03:50.192245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.209974+0000","last_change":"2024-03-30T02:34:46.745778+0000","last_active":"2024-03-30T02:35:27.209974+0000","last_peered":"2024-03-30T02:35:27.209974+0000","last_clean":"2024-03-30T02:35:27.209974+0000","last_became_active":"2024-03-30T02:34:46.745471+0000","last_became_peered":"2024-03-30T02:34:46.745471+0000","last_unstale":"2024-03-30T02:35:27.209974+0000","last_undegraded":"2024-03-30T02:35:27.209974+0000","last_fullsized":"2024-03-30T02:35:27.209974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:57.314163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205256+0000","last_change":"2024-03-30T02:34:50.769577+0000","last_active":"2024-03-30T02:35:27.205256+0000","last_peered":"2024-03-30T02:35:27.205256+0000","last_clean":"2024-03-30T02:35:27.205256+0000","last_became_active":"2024-03-30T02:34:50.769100+0000","last_became_peered":"2024-03-30T02:34:50.769100+0000","last_unstale":"2024-03-30T02:35:27.205256+0000","last_undegraded":"2024-03-30T02:35:27.205256+0000","last_fullsized":"2024-03-30T02:35:27.205256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:08:43.495671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211228+0000","last_change":"2024-03-30T02:34:46.218032+0000","last_active":"2024-03-30T02:35:27.211228+0000","last_peered":"2024-03-30T02:35:27.211228+0000","last_clean":"2024-03-30T02:35:27.211228+0000","last_became_active":"2024-03-30T02:34:46.217733+0000","last_became_peered":"2024-03-30T02:34:46.217733+0000","last_unstale":"2024-03-30T02:35:27.211228+0000","last_undegraded":"2024-03-30T02:35:27.211228+0000","last_fullsized":"2024-03-30T02:35:27.211228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:58.413590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207675+0000","last_change":"2024-03-30T02:34:46.750588+0000","last_active":"2024-03-30T02:35:27.207675+0000","last_peered":"2024-03-30T02:35:27.207675+0000","last_clean":"2024-03-30T02:35:27.207675+0000","last_became_active":"2024-03-30T02:34:46.750016+0000","last_became_peered":"2024-03-30T02:34:46.750016+0000","last_unstale":"2024-03-30T02:35:27.207675+0000","last_undegraded":"2024-03-30T02:35:27.207675+0000","last_fullsized":"2024-03-30T02:35:27.207675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:19.144544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.213404+0000","last_change":"2024-03-30T02:35:28.213404+0000","last_active":"2024-03-30T02:35:28.213404+0000","last_peered":"2024-03-30T02:35:28.213404+0000","last_clean":"2024-03-30T02:35:28.213404+0000","last_became_active":"2024-03-30T02:35:28.212746+0000","last_became_peered":"2024-03-30T02:35:28.212746+0000","last_unstale":"2024-03-30T02:35:28.213404+0000","last_undegraded":"2024-03-30T02:35:28.213404+0000","last_fullsized":"2024-03-30T02:35:28.213404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:24.807469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211663+0000","last_change":"2024-03-30T02:34:46.213741+0000","last_active":"2024-03-30T02:35:27.211663+0000","last_peered":"2024-03-30T02:35:27.211663+0000","last_clean":"2024-03-30T02:35:27.211663+0000","last_became_active":"2024-03-30T02:34:46.213395+0000","last_became_peered":"2024-03-30T02:34:46.213395+0000","last_unstale":"2024-03-30T02:35:27.211663+0000","last_undegraded":"2024-03-30T02:35:27.211663+0000","last_fullsized":"2024-03-30T02:35:27.211663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:13.192415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474698+0000","last_change":"2024-03-30T02:34:46.738401+0000","last_active":"2024-03-30T02:35:27.474698+0000","last_peered":"2024-03-30T02:35:27.474698+0000","last_clean":"2024-03-30T02:35:27.474698+0000","last_became_active":"2024-03-30T02:34:46.737804+0000","last_became_peered":"2024-03-30T02:34:46.737804+0000","last_unstale":"2024-03-30T02:35:27.474698+0000","last_undegraded":"2024-03-30T02:35:27.474698+0000","last_fullsized":"2024-03-30T02:35:27.474698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:56:08.194439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205124+0000","last_change":"2024-03-30T02:34:50.766222+0000","last_active":"2024-03-30T02:35:27.205124+0000","last_peered":"2024-03-30T02:35:27.205124+0000","last_clean":"2024-03-30T02:35:27.205124+0000","last_became_active":"2024-03-30T02:34:50.765836+0000","last_became_peered":"2024-03-30T02:34:50.765836+0000","last_unstale":"2024-03-30T02:35:27.205124+0000","last_undegraded":"2024-03-30T02:35:27.205124+0000","last_fullsized":"2024-03-30T02:35:27.205124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:32.529535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210488+0000","last_change":"2024-03-30T02:34:46.212586+0000","last_active":"2024-03-30T02:35:27.210488+0000","last_peered":"2024-03-30T02:35:27.210488+0000","last_clean":"2024-03-30T02:35:27.210488+0000","last_became_active":"2024-03-30T02:34:46.212110+0000","last_became_peered":"2024-03-30T02:34:46.212110+0000","last_unstale":"2024-03-30T02:35:27.210488+0000","last_undegraded":"2024-03-30T02:35:27.210488+0000","last_fullsized":"2024-03-30T02:35:27.210488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:38.630953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202351+0000","last_change":"2024-03-30T02:34:46.734923+0000","last_active":"2024-03-30T02:35:27.202351+0000","last_peered":"2024-03-30T02:35:27.202351+0000","last_clean":"2024-03-30T02:35:27.202351+0000","last_became_active":"2024-03-30T02:34:46.734572+0000","last_became_peered":"2024-03-30T02:34:46.734572+0000","last_unstale":"2024-03-30T02:35:27.202351+0000","last_undegraded":"2024-03-30T02:35:27.202351+0000","last_fullsized":"2024-03-30T02:35:27.202351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:28.647788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206133+0000","last_change":"2024-03-30T02:34:50.757803+0000","last_active":"2024-03-30T02:35:27.206133+0000","last_peered":"2024-03-30T02:35:27.206133+0000","last_clean":"2024-03-30T02:35:27.206133+0000","last_became_active":"2024-03-30T02:34:50.757198+0000","last_became_peered":"2024-03-30T02:34:50.757198+0000","last_unstale":"2024-03-30T02:35:27.206133+0000","last_undegraded":"2024-03-30T02:35:27.206133+0000","last_fullsized":"2024-03-30T02:35:27.206133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:54:49.399741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:56.183893+0000","last_change":"2024-03-30T02:34:46.214549+0000","last_active":"2024-03-30T02:35:56.183893+0000","last_peered":"2024-03-30T02:35:56.183893+0000","last_clean":"2024-03-30T02:35:56.183893+0000","last_became_active":"2024-03-30T02:34:46.214198+0000","last_became_peered":"2024-03-30T02:34:46.214198+0000","last_unstale":"2024-03-30T02:35:56.183893+0000","last_undegraded":"2024-03-30T02:35:56.183893+0000","last_fullsized":"2024-03-30T02:35:56.183893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:45:25.618982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210410+0000","last_change":"2024-03-30T02:34:46.737771+0000","last_active":"2024-03-30T02:35:27.210410+0000","last_peered":"2024-03-30T02:35:27.210410+0000","last_clean":"2024-03-30T02:35:27.210410+0000","last_became_active":"2024-03-30T02:34:46.737438+0000","last_became_peered":"2024-03-30T02:34:46.737438+0000","last_unstale":"2024-03-30T02:35:27.210410+0000","last_undegraded":"2024-03-30T02:35:27.210410+0000","last_fullsized":"2024-03-30T02:35:27.210410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:08.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206737+0000","last_change":"2024-03-30T02:34:50.767304+0000","last_active":"2024-03-30T02:35:27.206737+0000","last_peered":"2024-03-30T02:35:27.206737+0000","last_clean":"2024-03-30T02:35:27.206737+0000","last_became_active":"2024-03-30T02:34:50.766909+0000","last_became_peered":"2024-03-30T02:34:50.766909+0000","last_unstale":"2024-03-30T02:35:27.206737+0000","last_undegraded":"2024-03-30T02:35:27.206737+0000","last_fullsized":"2024-03-30T02:35:27.206737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:24.021991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206539+0000","last_change":"2024-03-30T02:34:46.214937+0000","last_active":"2024-03-30T02:35:27.206539+0000","last_peered":"2024-03-30T02:35:27.206539+0000","last_clean":"2024-03-30T02:35:27.206539+0000","last_became_active":"2024-03-30T02:34:46.214503+0000","last_became_peered":"2024-03-30T02:34:46.214503+0000","last_unstale":"2024-03-30T02:35:27.206539+0000","last_undegraded":"2024-03-30T02:35:27.206539+0000","last_fullsized":"2024-03-30T02:35:27.206539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:20:11.624919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.203098+0000","last_change":"2024-03-30T02:34:46.737374+0000","last_active":"2024-03-30T02:35:27.203098+0000","last_peered":"2024-03-30T02:35:27.203098+0000","last_clean":"2024-03-30T02:35:27.203098+0000","last_became_active":"2024-03-30T02:34:46.735307+0000","last_became_peered":"2024-03-30T02:34:46.735307+0000","last_unstale":"2024-03-30T02:35:27.203098+0000","last_undegraded":"2024-03-30T02:35:27.203098+0000","last_fullsized":"2024-03-30T02:35:27.203098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:30.482728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206531+0000","last_change":"2024-03-30T02:34:50.768016+0000","last_active":"2024-03-30T02:35:27.206531+0000","last_peered":"2024-03-30T02:35:27.206531+0000","last_clean":"2024-03-30T02:35:27.206531+0000","last_became_active":"2024-03-30T02:34:50.767564+0000","last_became_peered":"2024-03-30T02:34:50.767564+0000","last_unstale":"2024-03-30T02:35:27.206531+0000","last_undegraded":"2024-03-30T02:35:27.206531+0000","last_fullsized":"2024-03-30T02:35:27.206531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:16.888754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201236+0000","last_change":"2024-03-30T02:34:46.212982+0000","last_active":"2024-03-30T02:35:27.201236+0000","last_peered":"2024-03-30T02:35:27.201236+0000","last_clean":"2024-03-30T02:35:27.201236+0000","last_became_active":"2024-03-30T02:34:46.212550+0000","last_became_peered":"2024-03-30T02:34:46.212550+0000","last_unstale":"2024-03-30T02:35:27.201236+0000","last_undegraded":"2024-03-30T02:35:27.201236+0000","last_fullsized":"2024-03-30T02:35:27.201236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:44:50.156615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.215165+0000","last_change":"2024-03-30T02:35:28.213774+0000","last_active":"2024-03-30T02:35:28.215165+0000","last_peered":"2024-03-30T02:35:28.215165+0000","last_clean":"2024-03-30T02:35:28.215165+0000","last_became_active":"2024-03-30T02:35:28.213085+0000","last_became_peered":"2024-03-30T02:35:28.213085+0000","last_unstale":"2024-03-30T02:35:28.215165+0000","last_undegraded":"2024-03-30T02:35:28.215165+0000","last_fullsized":"2024-03-30T02:35:28.215165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:56:30.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207389+0000","last_change":"2024-03-30T02:34:50.773036+0000","last_active":"2024-03-30T02:35:27.207389+0000","last_peered":"2024-03-30T02:35:27.207389+0000","last_clean":"2024-03-30T02:35:27.207389+0000","last_became_active":"2024-03-30T02:34:50.772614+0000","last_became_peered":"2024-03-30T02:34:50.772614+0000","last_unstale":"2024-03-30T02:35:27.207389+0000","last_undegraded":"2024-03-30T02:35:27.207389+0000","last_fullsized":"2024-03-30T02:35:27.207389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:54:28.792793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201717+0000","last_change":"2024-03-30T02:34:46.209802+0000","last_active":"2024-03-30T02:35:27.201717+0000","last_peered":"2024-03-30T02:35:27.201717+0000","last_clean":"2024-03-30T02:35:27.201717+0000","last_became_active":"2024-03-30T02:34:46.209246+0000","last_became_peered":"2024-03-30T02:34:46.209246+0000","last_unstale":"2024-03-30T02:35:27.201717+0000","last_undegraded":"2024-03-30T02:35:27.201717+0000","last_fullsized":"2024-03-30T02:35:27.201717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:09:57.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.209595+0000","last_change":"2024-03-30T02:34:46.743638+0000","last_active":"2024-03-30T02:35:27.209595+0000","last_peered":"2024-03-30T02:35:27.209595+0000","last_clean":"2024-03-30T02:35:27.209595+0000","last_became_active":"2024-03-30T02:34:46.743246+0000","last_became_peered":"2024-03-30T02:34:46.743246+0000","last_unstale":"2024-03-30T02:35:27.209595+0000","last_undegraded":"2024-03-30T02:35:27.209595+0000","last_fullsized":"2024-03-30T02:35:27.209595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:49:55.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210472+0000","last_change":"2024-03-30T02:34:50.768586+0000","last_active":"2024-03-30T02:35:27.210472+0000","last_peered":"2024-03-30T02:35:27.210472+0000","last_clean":"2024-03-30T02:35:27.210472+0000","last_became_active":"2024-03-30T02:34:50.766894+0000","last_became_peered":"2024-03-30T02:34:50.766894+0000","last_unstale":"2024-03-30T02:35:27.210472+0000","last_undegraded":"2024-03-30T02:35:27.210472+0000","last_fullsized":"2024-03-30T02:35:27.210472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:36:39.329608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":100,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:58.185838+0000","last_change":"2024-03-30T02:34:46.204994+0000","last_active":"2024-03-30T02:35:58.185838+0000","last_peered":"2024-03-30T02:35:58.185838+0000","last_clean":"2024-03-30T02:35:58.185838+0000","last_became_active":"2024-03-30T02:34:46.204710+0000","last_became_peered":"2024-03-30T02:34:46.204710+0000","last_unstale":"2024-03-30T02:35:58.185838+0000","last_undegraded":"2024-03-30T02:35:58.185838+0000","last_fullsized":"2024-03-30T02:35:58.185838+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:39.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.475019+0000","last_change":"2024-03-30T02:34:46.733573+0000","last_active":"2024-03-30T02:35:27.475019+0000","last_peered":"2024-03-30T02:35:27.475019+0000","last_clean":"2024-03-30T02:35:27.475019+0000","last_became_active":"2024-03-30T02:34:46.733241+0000","last_became_peered":"2024-03-30T02:34:46.733241+0000","last_unstale":"2024-03-30T02:35:27.475019+0000","last_undegraded":"2024-03-30T02:35:27.475019+0000","last_fullsized":"2024-03-30T02:35:27.475019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:07:10.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.313882+0000","last_change":"2024-03-30T02:35:28.313493+0000","last_active":"2024-03-30T02:35:28.313882+0000","last_peered":"2024-03-30T02:35:28.313882+0000","last_clean":"2024-03-30T02:35:28.313882+0000","last_became_active":"2024-03-30T02:35:28.313065+0000","last_became_peered":"2024-03-30T02:35:28.313065+0000","last_unstale":"2024-03-30T02:35:28.313882+0000","last_undegraded":"2024-03-30T02:35:28.313882+0000","last_fullsized":"2024-03-30T02:35:28.313882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:44:52.675251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202943+0000","last_change":"2024-03-30T02:34:46.212732+0000","last_active":"2024-03-30T02:35:27.202943+0000","last_peered":"2024-03-30T02:35:27.202943+0000","last_clean":"2024-03-30T02:35:27.202943+0000","last_became_active":"2024-03-30T02:34:46.212306+0000","last_became_peered":"2024-03-30T02:34:46.212306+0000","last_unstale":"2024-03-30T02:35:27.202943+0000","last_undegraded":"2024-03-30T02:35:27.202943+0000","last_fullsized":"2024-03-30T02:35:27.202943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:27:15.766203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207755+0000","last_change":"2024-03-30T02:34:46.749420+0000","last_active":"2024-03-30T02:35:27.207755+0000","last_peered":"2024-03-30T02:35:27.207755+0000","last_clean":"2024-03-30T02:35:27.207755+0000","last_became_active":"2024-03-30T02:34:46.745286+0000","last_became_peered":"2024-03-30T02:34:46.745286+0000","last_unstale":"2024-03-30T02:35:27.207755+0000","last_undegraded":"2024-03-30T02:35:27.207755+0000","last_fullsized":"2024-03-30T02:35:27.207755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:01.489862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206147+0000","last_change":"2024-03-30T02:34:50.760542+0000","last_active":"2024-03-30T02:35:27.206147+0000","last_peered":"2024-03-30T02:35:27.206147+0000","last_clean":"2024-03-30T02:35:27.206147+0000","last_became_active":"2024-03-30T02:34:50.760087+0000","last_became_peered":"2024-03-30T02:34:50.760087+0000","last_unstale":"2024-03-30T02:35:27.206147+0000","last_undegraded":"2024-03-30T02:35:27.206147+0000","last_fullsized":"2024-03-30T02:35:27.206147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:46.352189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202468+0000","last_change":"2024-03-30T02:34:46.224058+0000","last_active":"2024-03-30T02:35:27.202468+0000","last_peered":"2024-03-30T02:35:27.202468+0000","last_clean":"2024-03-30T02:35:27.202468+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:35:27.202468+0000","last_undegraded":"2024-03-30T02:35:27.202468+0000","last_fullsized":"2024-03-30T02:35:27.202468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:31:10.124533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474763+0000","last_change":"2024-03-30T02:34:46.741413+0000","last_active":"2024-03-30T02:35:27.474763+0000","last_peered":"2024-03-30T02:35:27.474763+0000","last_clean":"2024-03-30T02:35:27.474763+0000","last_became_active":"2024-03-30T02:34:46.741112+0000","last_became_peered":"2024-03-30T02:34:46.741112+0000","last_unstale":"2024-03-30T02:35:27.474763+0000","last_undegraded":"2024-03-30T02:35:27.474763+0000","last_fullsized":"2024-03-30T02:35:27.474763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:34.820166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206802+0000","last_change":"2024-03-30T02:34:50.752315+0000","last_active":"2024-03-30T02:35:27.206802+0000","last_peered":"2024-03-30T02:35:27.206802+0000","last_clean":"2024-03-30T02:35:27.206802+0000","last_became_active":"2024-03-30T02:34:50.750985+0000","last_became_peered":"2024-03-30T02:34:50.750985+0000","last_unstale":"2024-03-30T02:35:27.206802+0000","last_undegraded":"2024-03-30T02:35:27.206802+0000","last_fullsized":"2024-03-30T02:35:27.206802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:37.271063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'36","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:58.185998+0000","last_change":"2024-03-30T02:34:46.214759+0000","last_active":"2024-03-30T02:35:58.185998+0000","last_peered":"2024-03-30T02:35:58.185998+0000","last_clean":"2024-03-30T02:35:58.185998+0000","last_became_active":"2024-03-30T02:34:46.214388+0000","last_became_peered":"2024-03-30T02:34:46.214388+0000","last_unstale":"2024-03-30T02:35:58.185998+0000","last_undegraded":"2024-03-30T02:35:58.185998+0000","last_fullsized":"2024-03-30T02:35:58.185998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:41.100461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63139,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":105,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211453+0000","last_change":"2024-03-30T02:34:46.741112+0000","last_active":"2024-03-30T02:35:27.211453+0000","last_peered":"2024-03-30T02:35:27.211453+0000","last_clean":"2024-03-30T02:35:27.211453+0000","last_became_active":"2024-03-30T02:34:46.739664+0000","last_became_peered":"2024-03-30T02:34:46.739664+0000","last_unstale":"2024-03-30T02:35:27.211453+0000","last_undegraded":"2024-03-30T02:35:27.211453+0000","last_fullsized":"2024-03-30T02:35:27.211453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:07.763942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202806+0000","last_change":"2024-03-30T02:34:50.765645+0000","last_active":"2024-03-30T02:35:27.202806+0000","last_peered":"2024-03-30T02:35:27.202806+0000","last_clean":"2024-03-30T02:35:27.202806+0000","last_became_active":"2024-03-30T02:34:50.765345+0000","last_became_peered":"2024-03-30T02:34:50.765345+0000","last_unstale":"2024-03-30T02:35:27.202806+0000","last_undegraded":"2024-03-30T02:35:27.202806+0000","last_fullsized":"2024-03-30T02:35:27.202806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:25:38.591525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206620+0000","last_change":"2024-03-30T02:34:46.216643+0000","last_active":"2024-03-30T02:35:27.206620+0000","last_peered":"2024-03-30T02:35:27.206620+0000","last_clean":"2024-03-30T02:35:27.206620+0000","last_became_active":"2024-03-30T02:34:46.216188+0000","last_became_peered":"2024-03-30T02:34:46.216188+0000","last_unstale":"2024-03-30T02:35:27.206620+0000","last_undegraded":"2024-03-30T02:35:27.206620+0000","last_fullsized":"2024-03-30T02:35:27.206620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:49:10.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201557+0000","last_change":"2024-03-30T02:34:46.748313+0000","last_active":"2024-03-30T02:35:27.201557+0000","last_peered":"2024-03-30T02:35:27.201557+0000","last_clean":"2024-03-30T02:35:27.201557+0000","last_became_active":"2024-03-30T02:34:46.742897+0000","last_became_peered":"2024-03-30T02:34:46.742897+0000","last_unstale":"2024-03-30T02:35:27.201557+0000","last_undegraded":"2024-03-30T02:35:27.201557+0000","last_fullsized":"2024-03-30T02:35:27.201557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:00.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205859+0000","last_change":"2024-03-30T02:34:50.760993+0000","last_active":"2024-03-30T02:35:27.205859+0000","last_peered":"2024-03-30T02:35:27.205859+0000","last_clean":"2024-03-30T02:35:27.205859+0000","last_became_active":"2024-03-30T02:34:50.760535+0000","last_became_peered":"2024-03-30T02:34:50.760535+0000","last_unstale":"2024-03-30T02:35:27.205859+0000","last_undegraded":"2024-03-30T02:35:27.205859+0000","last_fullsized":"2024-03-30T02:35:27.205859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:15.569533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202035+0000","last_change":"2024-03-30T02:34:46.216603+0000","last_active":"2024-03-30T02:35:27.202035+0000","last_peered":"2024-03-30T02:35:27.202035+0000","last_clean":"2024-03-30T02:35:27.202035+0000","last_became_active":"2024-03-30T02:34:46.216231+0000","last_became_peered":"2024-03-30T02:34:46.216231+0000","last_unstale":"2024-03-30T02:35:27.202035+0000","last_undegraded":"2024-03-30T02:35:27.202035+0000","last_fullsized":"2024-03-30T02:35:27.202035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:51:00.659652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202643+0000","last_change":"2024-03-30T02:34:46.742815+0000","last_active":"2024-03-30T02:35:27.202643+0000","last_peered":"2024-03-30T02:35:27.202643+0000","last_clean":"2024-03-30T02:35:27.202643+0000","last_became_active":"2024-03-30T02:34:46.742487+0000","last_became_peered":"2024-03-30T02:34:46.742487+0000","last_unstale":"2024-03-30T02:35:27.202643+0000","last_undegraded":"2024-03-30T02:35:27.202643+0000","last_fullsized":"2024-03-30T02:35:27.202643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:07.182990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211657+0000","last_change":"2024-03-30T02:34:50.770398+0000","last_active":"2024-03-30T02:35:27.211657+0000","last_peered":"2024-03-30T02:35:27.211657+0000","last_clean":"2024-03-30T02:35:27.211657+0000","last_became_active":"2024-03-30T02:34:50.770000+0000","last_became_peered":"2024-03-30T02:34:50.770000+0000","last_unstale":"2024-03-30T02:35:27.211657+0000","last_undegraded":"2024-03-30T02:35:27.211657+0000","last_fullsized":"2024-03-30T02:35:27.211657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:14:49.057081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:54.180972+0000","last_change":"2024-03-30T02:34:46.209403+0000","last_active":"2024-03-30T02:35:54.180972+0000","last_peered":"2024-03-30T02:35:54.180972+0000","last_clean":"2024-03-30T02:35:54.180972+0000","last_became_active":"2024-03-30T02:34:46.209065+0000","last_became_peered":"2024-03-30T02:34:46.209065+0000","last_unstale":"2024-03-30T02:35:54.180972+0000","last_undegraded":"2024-03-30T02:35:54.180972+0000","last_fullsized":"2024-03-30T02:35:54.180972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:36:20.427720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202270+0000","last_change":"2024-03-30T02:34:46.734861+0000","last_active":"2024-03-30T02:35:27.202270+0000","last_peered":"2024-03-30T02:35:27.202270+0000","last_clean":"2024-03-30T02:35:27.202270+0000","last_became_active":"2024-03-30T02:34:46.734376+0000","last_became_peered":"2024-03-30T02:34:46.734376+0000","last_unstale":"2024-03-30T02:35:27.202270+0000","last_undegraded":"2024-03-30T02:35:27.202270+0000","last_fullsized":"2024-03-30T02:35:27.202270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:08:36.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201818+0000","last_change":"2024-03-30T02:34:50.767469+0000","last_active":"2024-03-30T02:35:27.201818+0000","last_peered":"2024-03-30T02:35:27.201818+0000","last_clean":"2024-03-30T02:35:27.201818+0000","last_became_active":"2024-03-30T02:34:50.765460+0000","last_became_peered":"2024-03-30T02:34:50.765460+0000","last_unstale":"2024-03-30T02:35:27.201818+0000","last_undegraded":"2024-03-30T02:35:27.201818+0000","last_fullsized":"2024-03-30T02:35:27.201818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:54:06.940171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210118+0000","last_change":"2024-03-30T02:34:46.224053+0000","last_active":"2024-03-30T02:35:27.210118+0000","last_peered":"2024-03-30T02:35:27.210118+0000","last_clean":"2024-03-30T02:35:27.210118+0000","last_became_active":"2024-03-30T02:34:46.223666+0000","last_became_peered":"2024-03-30T02:34:46.223666+0000","last_unstale":"2024-03-30T02:35:27.210118+0000","last_undegraded":"2024-03-30T02:35:27.210118+0000","last_fullsized":"2024-03-30T02:35:27.210118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:18:22.976298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201712+0000","last_change":"2024-03-30T02:34:46.734838+0000","last_active":"2024-03-30T02:35:27.201712+0000","last_peered":"2024-03-30T02:35:27.201712+0000","last_clean":"2024-03-30T02:35:27.201712+0000","last_became_active":"2024-03-30T02:34:46.734428+0000","last_became_peered":"2024-03-30T02:34:46.734428+0000","last_unstale":"2024-03-30T02:35:27.201712+0000","last_undegraded":"2024-03-30T02:35:27.201712+0000","last_fullsized":"2024-03-30T02:35:27.201712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:42:51.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201629+0000","last_change":"2024-03-30T02:34:50.771150+0000","last_active":"2024-03-30T02:35:27.201629+0000","last_peered":"2024-03-30T02:35:27.201629+0000","last_clean":"2024-03-30T02:35:27.201629+0000","last_became_active":"2024-03-30T02:34:50.770703+0000","last_became_peered":"2024-03-30T02:34:50.770703+0000","last_unstale":"2024-03-30T02:35:27.201629+0000","last_undegraded":"2024-03-30T02:35:27.201629+0000","last_fullsized":"2024-03-30T02:35:27.201629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:04.466664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211399+0000","last_change":"2024-03-30T02:34:46.203558+0000","last_active":"2024-03-30T02:35:27.211399+0000","last_peered":"2024-03-30T02:35:27.211399+0000","last_clean":"2024-03-30T02:35:27.211399+0000","last_became_active":"2024-03-30T02:34:46.202896+0000","last_became_peered":"2024-03-30T02:34:46.202896+0000","last_unstale":"2024-03-30T02:35:27.211399+0000","last_undegraded":"2024-03-30T02:35:27.211399+0000","last_fullsized":"2024-03-30T02:35:27.211399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:33.945665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.212775+0000","last_change":"2024-03-30T02:35:28.212775+0000","last_active":"2024-03-30T02:35:28.212775+0000","last_peered":"2024-03-30T02:35:28.212775+0000","last_clean":"2024-03-30T02:35:28.212775+0000","last_became_active":"2024-03-30T02:35:28.212368+0000","last_became_peered":"2024-03-30T02:35:28.212368+0000","last_unstale":"2024-03-30T02:35:28.212775+0000","last_undegraded":"2024-03-30T02:35:28.212775+0000","last_fullsized":"2024-03-30T02:35:28.212775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:50.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210436+0000","last_change":"2024-03-30T02:34:50.797758+0000","last_active":"2024-03-30T02:35:27.210436+0000","last_peered":"2024-03-30T02:35:27.210436+0000","last_clean":"2024-03-30T02:35:27.210436+0000","last_became_active":"2024-03-30T02:34:50.797311+0000","last_became_peered":"2024-03-30T02:34:50.797311+0000","last_unstale":"2024-03-30T02:35:27.210436+0000","last_undegraded":"2024-03-30T02:35:27.210436+0000","last_fullsized":"2024-03-30T02:35:27.210436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:32.337422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211691+0000","last_change":"2024-03-30T02:34:46.224397+0000","last_active":"2024-03-30T02:35:27.211691+0000","last_peered":"2024-03-30T02:35:27.211691+0000","last_clean":"2024-03-30T02:35:27.211691+0000","last_became_active":"2024-03-30T02:34:46.223825+0000","last_became_peered":"2024-03-30T02:34:46.223825+0000","last_unstale":"2024-03-30T02:35:27.211691+0000","last_undegraded":"2024-03-30T02:35:27.211691+0000","last_fullsized":"2024-03-30T02:35:27.211691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:53.154665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206934+0000","last_change":"2024-03-30T02:34:46.741358+0000","last_active":"2024-03-30T02:35:27.206934+0000","last_peered":"2024-03-30T02:35:27.206934+0000","last_clean":"2024-03-30T02:35:27.206934+0000","last_became_active":"2024-03-30T02:34:46.740734+0000","last_became_peered":"2024-03-30T02:34:46.740734+0000","last_unstale":"2024-03-30T02:35:27.206934+0000","last_undegraded":"2024-03-30T02:35:27.206934+0000","last_fullsized":"2024-03-30T02:35:27.206934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:59.678431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201960+0000","last_change":"2024-03-30T02:34:50.761998+0000","last_active":"2024-03-30T02:35:27.201960+0000","last_peered":"2024-03-30T02:35:27.201960+0000","last_clean":"2024-03-30T02:35:27.201960+0000","last_became_active":"2024-03-30T02:34:50.761560+0000","last_became_peered":"2024-03-30T02:34:50.761560+0000","last_unstale":"2024-03-30T02:35:27.201960+0000","last_undegraded":"2024-03-30T02:35:27.201960+0000","last_fullsized":"2024-03-30T02:35:27.201960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:57:17.213624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:54.186540+0000","last_change":"2024-03-30T02:34:46.213427+0000","last_active":"2024-03-30T02:35:54.186540+0000","last_peered":"2024-03-30T02:35:54.186540+0000","last_clean":"2024-03-30T02:35:54.186540+0000","last_became_active":"2024-03-30T02:34:46.213082+0000","last_became_peered":"2024-03-30T02:34:46.213082+0000","last_unstale":"2024-03-30T02:35:54.186540+0000","last_undegraded":"2024-03-30T02:35:54.186540+0000","last_fullsized":"2024-03-30T02:35:54.186540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:31:35.513723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206780+0000","last_change":"2024-03-30T02:34:46.747197+0000","last_active":"2024-03-30T02:35:27.206780+0000","last_peered":"2024-03-30T02:35:27.206780+0000","last_clean":"2024-03-30T02:35:27.206780+0000","last_became_active":"2024-03-30T02:34:46.743841+0000","last_became_peered":"2024-03-30T02:34:46.743841+0000","last_unstale":"2024-03-30T02:35:27.206780+0000","last_undegraded":"2024-03-30T02:35:27.206780+0000","last_fullsized":"2024-03-30T02:35:27.206780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:09.234382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201581+0000","last_change":"2024-03-30T02:34:50.797423+0000","last_active":"2024-03-30T02:35:27.201581+0000","last_peered":"2024-03-30T02:35:27.201581+0000","last_clean":"2024-03-30T02:35:27.201581+0000","last_became_active":"2024-03-30T02:34:50.796746+0000","last_became_peered":"2024-03-30T02:34:50.796746+0000","last_unstale":"2024-03-30T02:35:27.201581+0000","last_undegraded":"2024-03-30T02:35:27.201581+0000","last_fullsized":"2024-03-30T02:35:27.201581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:24.014131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201844+0000","last_change":"2024-03-30T02:34:46.734972+0000","last_active":"2024-03-30T02:35:27.201844+0000","last_peered":"2024-03-30T02:35:27.201844+0000","last_clean":"2024-03-30T02:35:27.201844+0000","last_became_active":"2024-03-30T02:34:46.734520+0000","last_became_peered":"2024-03-30T02:34:46.734520+0000","last_unstale":"2024-03-30T02:35:27.201844+0000","last_undegraded":"2024-03-30T02:35:27.201844+0000","last_fullsized":"2024-03-30T02:35:27.201844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:15.635060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:53.179217+0000","last_change":"2024-03-30T02:34:46.224036+0000","last_active":"2024-03-30T02:35:53.179217+0000","last_peered":"2024-03-30T02:35:53.179217+0000","last_clean":"2024-03-30T02:35:53.179217+0000","last_became_active":"2024-03-30T02:34:46.223706+0000","last_became_peered":"2024-03-30T02:34:46.223706+0000","last_unstale":"2024-03-30T02:35:53.179217+0000","last_undegraded":"2024-03-30T02:35:53.179217+0000","last_fullsized":"2024-03-30T02:35:53.179217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:40.458345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202738+0000","last_change":"2024-03-30T02:34:50.757894+0000","last_active":"2024-03-30T02:35:27.202738+0000","last_peered":"2024-03-30T02:35:27.202738+0000","last_clean":"2024-03-30T02:35:27.202738+0000","last_became_active":"2024-03-30T02:34:50.757486+0000","last_became_peered":"2024-03-30T02:34:50.757486+0000","last_unstale":"2024-03-30T02:35:27.202738+0000","last_undegraded":"2024-03-30T02:35:27.202738+0000","last_fullsized":"2024-03-30T02:35:27.202738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:30.609851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210726+0000","last_change":"2024-03-30T02:34:46.745990+0000","last_active":"2024-03-30T02:35:27.210726+0000","last_peered":"2024-03-30T02:35:27.210726+0000","last_clean":"2024-03-30T02:35:27.210726+0000","last_became_active":"2024-03-30T02:34:46.741937+0000","last_became_peered":"2024-03-30T02:34:46.741937+0000","last_unstale":"2024-03-30T02:35:27.210726+0000","last_undegraded":"2024-03-30T02:35:27.210726+0000","last_fullsized":"2024-03-30T02:35:27.210726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:26:17.833164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206019+0000","last_change":"2024-03-30T02:34:46.208858+0000","last_active":"2024-03-30T02:35:27.206019+0000","last_peered":"2024-03-30T02:35:27.206019+0000","last_clean":"2024-03-30T02:35:27.206019+0000","last_became_active":"2024-03-30T02:34:46.208479+0000","last_became_peered":"2024-03-30T02:34:46.208479+0000","last_unstale":"2024-03-30T02:35:27.206019+0000","last_undegraded":"2024-03-30T02:35:27.206019+0000","last_fullsized":"2024-03-30T02:35:27.206019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:58:14.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":336,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202124+0000","last_change":"2024-03-30T02:33:56.208237+0000","last_active":"2024-03-30T02:35:27.202124+0000","last_peered":"2024-03-30T02:35:27.202124+0000","last_clean":"2024-03-30T02:35:27.202124+0000","last_became_active":"2024-03-30T02:33:56.207820+0000","last_became_peered":"2024-03-30T02:33:56.207820+0000","last_unstale":"2024-03-30T02:35:27.202124+0000","last_undegraded":"2024-03-30T02:35:27.202124+0000","last_fullsized":"2024-03-30T02:35:27.202124+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_clean_scrub_stamp":"2024-03-30T02:31:23.922434+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:26.198503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.213743+0000","last_change":"2024-03-30T02:35:28.213743+0000","last_active":"2024-03-30T02:35:28.213743+0000","last_peered":"2024-03-30T02:35:28.213743+0000","last_clean":"2024-03-30T02:35:28.213743+0000","last_became_active":"2024-03-30T02:35:28.213318+0000","last_became_peered":"2024-03-30T02:35:28.213318+0000","last_unstale":"2024-03-30T02:35:28.213743+0000","last_undegraded":"2024-03-30T02:35:28.213743+0000","last_fullsized":"2024-03-30T02:35:28.213743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:15:35.803168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201730+0000","last_change":"2024-03-30T02:34:46.739757+0000","last_active":"2024-03-30T02:35:27.201730+0000","last_peered":"2024-03-30T02:35:27.201730+0000","last_clean":"2024-03-30T02:35:27.201730+0000","last_became_active":"2024-03-30T02:34:46.738269+0000","last_became_peered":"2024-03-30T02:34:46.738269+0000","last_unstale":"2024-03-30T02:35:27.201730+0000","last_undegraded":"2024-03-30T02:35:27.201730+0000","last_fullsized":"2024-03-30T02:35:27.201730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:03.411187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202176+0000","last_change":"2024-03-30T02:34:46.212532+0000","last_active":"2024-03-30T02:35:27.202176+0000","last_peered":"2024-03-30T02:35:27.202176+0000","last_clean":"2024-03-30T02:35:27.202176+0000","last_became_active":"2024-03-30T02:34:46.212191+0000","last_became_peered":"2024-03-30T02:34:46.212191+0000","last_unstale":"2024-03-30T02:35:27.202176+0000","last_undegraded":"2024-03-30T02:35:27.202176+0000","last_fullsized":"2024-03-30T02:35:27.202176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:19.120960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205726+0000","last_change":"2024-03-30T02:34:50.759911+0000","last_active":"2024-03-30T02:35:27.205726+0000","last_peered":"2024-03-30T02:35:27.205726+0000","last_clean":"2024-03-30T02:35:27.205726+0000","last_became_active":"2024-03-30T02:34:50.759487+0000","last_became_peered":"2024-03-30T02:34:50.759487+0000","last_unstale":"2024-03-30T02:35:27.205726+0000","last_undegraded":"2024-03-30T02:35:27.205726+0000","last_fullsized":"2024-03-30T02:35:27.205726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:46.403754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211512+0000","last_change":"2024-03-30T02:34:46.217092+0000","last_active":"2024-03-30T02:35:27.211512+0000","last_peered":"2024-03-30T02:35:27.211512+0000","last_clean":"2024-03-30T02:35:27.211512+0000","last_became_active":"2024-03-30T02:34:46.216779+0000","last_became_peered":"2024-03-30T02:34:46.216779+0000","last_unstale":"2024-03-30T02:35:27.211512+0000","last_undegraded":"2024-03-30T02:35:27.211512+0000","last_fullsized":"2024-03-30T02:35:27.211512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:10.894577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202103+0000","last_change":"2024-03-30T02:34:46.735212+0000","last_active":"2024-03-30T02:35:27.202103+0000","last_peered":"2024-03-30T02:35:27.202103+0000","last_clean":"2024-03-30T02:35:27.202103+0000","last_became_active":"2024-03-30T02:34:46.734282+0000","last_became_peered":"2024-03-30T02:34:46.734282+0000","last_unstale":"2024-03-30T02:35:27.202103+0000","last_undegraded":"2024-03-30T02:35:27.202103+0000","last_fullsized":"2024-03-30T02:35:27.202103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:28.596961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210151+0000","last_change":"2024-03-30T02:34:50.799290+0000","last_active":"2024-03-30T02:35:27.210151+0000","last_peered":"2024-03-30T02:35:27.210151+0000","last_clean":"2024-03-30T02:35:27.210151+0000","last_became_active":"2024-03-30T02:34:50.797524+0000","last_became_peered":"2024-03-30T02:34:50.797524+0000","last_unstale":"2024-03-30T02:35:27.210151+0000","last_undegraded":"2024-03-30T02:35:27.210151+0000","last_fullsized":"2024-03-30T02:35:27.210151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:53:27.931444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.206105+0000","last_change":"2024-03-30T02:34:46.202508+0000","last_active":"2024-03-30T02:35:28.206105+0000","last_peered":"2024-03-30T02:35:28.206105+0000","last_clean":"2024-03-30T02:35:28.206105+0000","last_became_active":"2024-03-30T02:34:46.202145+0000","last_became_peered":"2024-03-30T02:34:46.202145+0000","last_unstale":"2024-03-30T02:35:28.206105+0000","last_undegraded":"2024-03-30T02:35:28.206105+0000","last_fullsized":"2024-03-30T02:35:28.206105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:21.138776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211176+0000","last_change":"2024-03-30T02:34:46.740056+0000","last_active":"2024-03-30T02:35:27.211176+0000","last_peered":"2024-03-30T02:35:27.211176+0000","last_clean":"2024-03-30T02:35:27.211176+0000","last_became_active":"2024-03-30T02:34:46.739260+0000","last_became_peered":"2024-03-30T02:34:46.739260+0000","last_unstale":"2024-03-30T02:35:27.211176+0000","last_undegraded":"2024-03-30T02:35:27.211176+0000","last_fullsized":"2024-03-30T02:35:27.211176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:53.550978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207567+0000","last_change":"2024-03-30T02:34:50.756851+0000","last_active":"2024-03-30T02:35:27.207567+0000","last_peered":"2024-03-30T02:35:27.207567+0000","last_clean":"2024-03-30T02:35:27.207567+0000","last_became_active":"2024-03-30T02:34:50.756416+0000","last_became_peered":"2024-03-30T02:34:50.756416+0000","last_unstale":"2024-03-30T02:35:27.207567+0000","last_undegraded":"2024-03-30T02:35:27.207567+0000","last_fullsized":"2024-03-30T02:35:27.207567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:11.469599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206997+0000","last_change":"2024-03-30T02:34:46.215663+0000","last_active":"2024-03-30T02:35:27.206997+0000","last_peered":"2024-03-30T02:35:27.206997+0000","last_clean":"2024-03-30T02:35:27.206997+0000","last_became_active":"2024-03-30T02:34:46.214224+0000","last_became_peered":"2024-03-30T02:34:46.214224+0000","last_unstale":"2024-03-30T02:35:27.206997+0000","last_undegraded":"2024-03-30T02:35:27.206997+0000","last_fullsized":"2024-03-30T02:35:27.206997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:24:58.734434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207567+0000","last_change":"2024-03-30T02:34:46.754150+0000","last_active":"2024-03-30T02:35:27.207567+0000","last_peered":"2024-03-30T02:35:27.207567+0000","last_clean":"2024-03-30T02:35:27.207567+0000","last_became_active":"2024-03-30T02:34:46.753786+0000","last_became_peered":"2024-03-30T02:34:46.753786+0000","last_unstale":"2024-03-30T02:35:27.207567+0000","last_undegraded":"2024-03-30T02:35:27.207567+0000","last_fullsized":"2024-03-30T02:35:27.207567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:54:50.165020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205116+0000","last_change":"2024-03-30T02:34:50.797890+0000","last_active":"2024-03-30T02:35:27.205116+0000","last_peered":"2024-03-30T02:35:27.205116+0000","last_clean":"2024-03-30T02:35:27.205116+0000","last_became_active":"2024-03-30T02:34:50.797395+0000","last_became_peered":"2024-03-30T02:34:50.797395+0000","last_unstale":"2024-03-30T02:35:27.205116+0000","last_undegraded":"2024-03-30T02:35:27.205116+0000","last_fullsized":"2024-03-30T02:35:27.205116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:36:14.958804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474547+0000","last_change":"2024-03-30T02:34:46.220873+0000","last_active":"2024-03-30T02:35:27.474547+0000","last_peered":"2024-03-30T02:35:27.474547+0000","last_clean":"2024-03-30T02:35:27.474547+0000","last_became_active":"2024-03-30T02:34:46.220544+0000","last_became_peered":"2024-03-30T02:34:46.220544+0000","last_unstale":"2024-03-30T02:35:27.474547+0000","last_undegraded":"2024-03-30T02:35:27.474547+0000","last_fullsized":"2024-03-30T02:35:27.474547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:37.691602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210124+0000","last_change":"2024-03-30T02:34:46.739184+0000","last_active":"2024-03-30T02:35:27.210124+0000","last_peered":"2024-03-30T02:35:27.210124+0000","last_clean":"2024-03-30T02:35:27.210124+0000","last_became_active":"2024-03-30T02:34:46.738843+0000","last_became_peered":"2024-03-30T02:34:46.738843+0000","last_unstale":"2024-03-30T02:35:27.210124+0000","last_undegraded":"2024-03-30T02:35:27.210124+0000","last_fullsized":"2024-03-30T02:35:27.210124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:15:09.894754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211824+0000","last_change":"2024-03-30T02:34:50.768763+0000","last_active":"2024-03-30T02:35:27.211824+0000","last_peered":"2024-03-30T02:35:27.211824+0000","last_clean":"2024-03-30T02:35:27.211824+0000","last_became_active":"2024-03-30T02:34:50.767363+0000","last_became_peered":"2024-03-30T02:34:50.767363+0000","last_unstale":"2024-03-30T02:35:27.211824+0000","last_undegraded":"2024-03-30T02:35:27.211824+0000","last_fullsized":"2024-03-30T02:35:27.211824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:06.932826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.209434+0000","last_change":"2024-03-30T02:34:46.213006+0000","last_active":"2024-03-30T02:35:27.209434+0000","last_peered":"2024-03-30T02:35:27.209434+0000","last_clean":"2024-03-30T02:35:27.209434+0000","last_became_active":"2024-03-30T02:34:46.206271+0000","last_became_peered":"2024-03-30T02:34:46.206271+0000","last_unstale":"2024-03-30T02:35:27.209434+0000","last_undegraded":"2024-03-30T02:35:27.209434+0000","last_fullsized":"2024-03-30T02:35:27.209434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:00:36.447352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206241+0000","last_change":"2024-03-30T02:34:46.739566+0000","last_active":"2024-03-30T02:35:27.206241+0000","last_peered":"2024-03-30T02:35:27.206241+0000","last_clean":"2024-03-30T02:35:27.206241+0000","last_became_active":"2024-03-30T02:34:46.739205+0000","last_became_peered":"2024-03-30T02:34:46.739205+0000","last_unstale":"2024-03-30T02:35:27.206241+0000","last_undegraded":"2024-03-30T02:35:27.206241+0000","last_fullsized":"2024-03-30T02:35:27.206241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:01.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205645+0000","last_change":"2024-03-30T02:34:50.758016+0000","last_active":"2024-03-30T02:35:27.205645+0000","last_peered":"2024-03-30T02:35:27.205645+0000","last_clean":"2024-03-30T02:35:27.205645+0000","last_became_active":"2024-03-30T02:34:50.757616+0000","last_became_peered":"2024-03-30T02:34:50.757616+0000","last_unstale":"2024-03-30T02:35:27.205645+0000","last_undegraded":"2024-03-30T02:35:27.205645+0000","last_fullsized":"2024-03-30T02:35:27.205645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:35:07.902347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.217914+0000","last_change":"2024-03-30T02:35:28.217541+0000","last_active":"2024-03-30T02:35:28.217914+0000","last_peered":"2024-03-30T02:35:28.217914+0000","last_clean":"2024-03-30T02:35:28.217914+0000","last_became_active":"2024-03-30T02:35:28.212300+0000","last_became_peered":"2024-03-30T02:35:28.212300+0000","last_unstale":"2024-03-30T02:35:28.217914+0000","last_undegraded":"2024-03-30T02:35:28.217914+0000","last_fullsized":"2024-03-30T02:35:28.217914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:41.638612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.209758+0000","last_change":"2024-03-30T02:34:46.754114+0000","last_active":"2024-03-30T02:35:27.209758+0000","last_peered":"2024-03-30T02:35:27.209758+0000","last_clean":"2024-03-30T02:35:27.209758+0000","last_became_active":"2024-03-30T02:34:46.752691+0000","last_became_peered":"2024-03-30T02:34:46.752691+0000","last_unstale":"2024-03-30T02:35:27.209758+0000","last_undegraded":"2024-03-30T02:35:27.209758+0000","last_fullsized":"2024-03-30T02:35:27.209758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:42.908877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205333+0000","last_change":"2024-03-30T02:34:50.798110+0000","last_active":"2024-03-30T02:35:27.205333+0000","last_peered":"2024-03-30T02:35:27.205333+0000","last_clean":"2024-03-30T02:35:27.205333+0000","last_became_active":"2024-03-30T02:34:50.797551+0000","last_became_peered":"2024-03-30T02:34:50.797551+0000","last_unstale":"2024-03-30T02:35:27.205333+0000","last_undegraded":"2024-03-30T02:35:27.205333+0000","last_fullsized":"2024-03-30T02:35:27.205333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:49.362018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210186+0000","last_change":"2024-03-30T02:34:46.211647+0000","last_active":"2024-03-30T02:35:27.210186+0000","last_peered":"2024-03-30T02:35:27.210186+0000","last_clean":"2024-03-30T02:35:27.210186+0000","last_became_active":"2024-03-30T02:34:46.211311+0000","last_became_peered":"2024-03-30T02:34:46.211311+0000","last_unstale":"2024-03-30T02:35:27.210186+0000","last_undegraded":"2024-03-30T02:35:27.210186+0000","last_fullsized":"2024-03-30T02:35:27.210186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:05:34.135289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202501+0000","last_change":"2024-03-30T02:34:46.740256+0000","last_active":"2024-03-30T02:35:27.202501+0000","last_peered":"2024-03-30T02:35:27.202501+0000","last_clean":"2024-03-30T02:35:27.202501+0000","last_became_active":"2024-03-30T02:34:46.739913+0000","last_became_peered":"2024-03-30T02:34:46.739913+0000","last_unstale":"2024-03-30T02:35:27.202501+0000","last_undegraded":"2024-03-30T02:35:27.202501+0000","last_fullsized":"2024-03-30T02:35:27.202501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:44.152475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474682+0000","last_change":"2024-03-30T02:34:50.762149+0000","last_active":"2024-03-30T02:35:27.474682+0000","last_peered":"2024-03-30T02:35:27.474682+0000","last_clean":"2024-03-30T02:35:27.474682+0000","last_became_active":"2024-03-30T02:34:50.761717+0000","last_became_peered":"2024-03-30T02:34:50.761717+0000","last_unstale":"2024-03-30T02:35:27.474682+0000","last_undegraded":"2024-03-30T02:35:27.474682+0000","last_fullsized":"2024-03-30T02:35:27.474682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:22.373079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:55.182522+0000","last_change":"2024-03-30T02:34:46.224098+0000","last_active":"2024-03-30T02:35:55.182522+0000","last_peered":"2024-03-30T02:35:55.182522+0000","last_clean":"2024-03-30T02:35:55.182522+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:35:55.182522+0000","last_undegraded":"2024-03-30T02:35:55.182522+0000","last_fullsized":"2024-03-30T02:35:55.182522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:20:25.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.203056+0000","last_change":"2024-03-30T02:34:46.737661+0000","last_active":"2024-03-30T02:35:27.203056+0000","last_peered":"2024-03-30T02:35:27.203056+0000","last_clean":"2024-03-30T02:35:27.203056+0000","last_became_active":"2024-03-30T02:34:46.737303+0000","last_became_peered":"2024-03-30T02:34:46.737303+0000","last_unstale":"2024-03-30T02:35:27.203056+0000","last_undegraded":"2024-03-30T02:35:27.203056+0000","last_fullsized":"2024-03-30T02:35:27.203056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:44.929643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206679+0000","last_change":"2024-03-30T02:34:50.767119+0000","last_active":"2024-03-30T02:35:27.206679+0000","last_peered":"2024-03-30T02:35:27.206679+0000","last_clean":"2024-03-30T02:35:27.206679+0000","last_became_active":"2024-03-30T02:34:50.766472+0000","last_became_peered":"2024-03-30T02:34:50.766472+0000","last_unstale":"2024-03-30T02:35:27.206679+0000","last_undegraded":"2024-03-30T02:35:27.206679+0000","last_fullsized":"2024-03-30T02:35:27.206679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:46.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202854+0000","last_change":"2024-03-30T02:34:46.216713+0000","last_active":"2024-03-30T02:35:27.202854+0000","last_peered":"2024-03-30T02:35:27.202854+0000","last_clean":"2024-03-30T02:35:27.202854+0000","last_became_active":"2024-03-30T02:34:46.216366+0000","last_became_peered":"2024-03-30T02:34:46.216366+0000","last_unstale":"2024-03-30T02:35:27.202854+0000","last_undegraded":"2024-03-30T02:35:27.202854+0000","last_fullsized":"2024-03-30T02:35:27.202854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:43.803955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202806+0000","last_change":"2024-03-30T02:34:46.736854+0000","last_active":"2024-03-30T02:35:27.202806+0000","last_peered":"2024-03-30T02:35:27.202806+0000","last_clean":"2024-03-30T02:35:27.202806+0000","last_became_active":"2024-03-30T02:34:46.736511+0000","last_became_peered":"2024-03-30T02:34:46.736511+0000","last_unstale":"2024-03-30T02:35:27.202806+0000","last_undegraded":"2024-03-30T02:35:27.202806+0000","last_fullsized":"2024-03-30T02:35:27.202806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:12.701467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201444+0000","last_change":"2024-03-30T02:34:50.757227+0000","last_active":"2024-03-30T02:35:27.201444+0000","last_peered":"2024-03-30T02:35:27.201444+0000","last_clean":"2024-03-30T02:35:27.201444+0000","last_became_active":"2024-03-30T02:34:50.756742+0000","last_became_peered":"2024-03-30T02:34:50.756742+0000","last_unstale":"2024-03-30T02:35:27.201444+0000","last_undegraded":"2024-03-30T02:35:27.201444+0000","last_fullsized":"2024-03-30T02:35:27.201444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:48.792202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201916+0000","last_change":"2024-03-30T02:34:46.218954+0000","last_active":"2024-03-30T02:35:27.201916+0000","last_peered":"2024-03-30T02:35:27.201916+0000","last_clean":"2024-03-30T02:35:27.201916+0000","last_became_active":"2024-03-30T02:34:46.218019+0000","last_became_peered":"2024-03-30T02:34:46.218019+0000","last_unstale":"2024-03-30T02:35:27.201916+0000","last_undegraded":"2024-03-30T02:35:27.201916+0000","last_fullsized":"2024-03-30T02:35:27.201916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:32.651216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202921+0000","last_change":"2024-03-30T02:34:46.741735+0000","last_active":"2024-03-30T02:35:27.202921+0000","last_peered":"2024-03-30T02:35:27.202921+0000","last_clean":"2024-03-30T02:35:27.202921+0000","last_became_active":"2024-03-30T02:34:46.740663+0000","last_became_peered":"2024-03-30T02:34:46.740663+0000","last_unstale":"2024-03-30T02:35:27.202921+0000","last_undegraded":"2024-03-30T02:35:27.202921+0000","last_fullsized":"2024-03-30T02:35:27.202921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:09.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206349+0000","last_change":"2024-03-30T02:34:46.739082+0000","last_active":"2024-03-30T02:35:27.206349+0000","last_peered":"2024-03-30T02:35:27.206349+0000","last_clean":"2024-03-30T02:35:27.206349+0000","last_became_active":"2024-03-30T02:34:46.738732+0000","last_became_peered":"2024-03-30T02:34:46.738732+0000","last_unstale":"2024-03-30T02:35:27.206349+0000","last_undegraded":"2024-03-30T02:35:27.206349+0000","last_fullsized":"2024-03-30T02:35:27.206349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:53.961520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.211005+0000","last_change":"2024-03-30T02:35:28.211005+0000","last_active":"2024-03-30T02:35:28.211005+0000","last_peered":"2024-03-30T02:35:28.211005+0000","last_clean":"2024-03-30T02:35:28.211005+0000","last_became_active":"2024-03-30T02:35:28.210610+0000","last_became_peered":"2024-03-30T02:35:28.210610+0000","last_unstale":"2024-03-30T02:35:28.211005+0000","last_undegraded":"2024-03-30T02:35:28.211005+0000","last_fullsized":"2024-03-30T02:35:28.211005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:53.054107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205930+0000","last_change":"2024-03-30T02:34:46.742590+0000","last_active":"2024-03-30T02:35:27.205930+0000","last_peered":"2024-03-30T02:35:27.205930+0000","last_clean":"2024-03-30T02:35:27.205930+0000","last_became_active":"2024-03-30T02:34:46.740904+0000","last_became_peered":"2024-03-30T02:34:46.740904+0000","last_unstale":"2024-03-30T02:35:27.205930+0000","last_undegraded":"2024-03-30T02:35:27.205930+0000","last_fullsized":"2024-03-30T02:35:27.205930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:22.653881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206792+0000","last_change":"2024-03-30T02:34:46.753119+0000","last_active":"2024-03-30T02:35:27.206792+0000","last_peered":"2024-03-30T02:35:27.206792+0000","last_clean":"2024-03-30T02:35:27.206792+0000","last_became_active":"2024-03-30T02:34:46.745680+0000","last_became_peered":"2024-03-30T02:34:46.745680+0000","last_unstale":"2024-03-30T02:35:27.206792+0000","last_undegraded":"2024-03-30T02:35:27.206792+0000","last_fullsized":"2024-03-30T02:35:27.206792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:02.352961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.313317+0000","last_change":"2024-03-30T02:35:28.313317+0000","last_active":"2024-03-30T02:35:28.313317+0000","last_peered":"2024-03-30T02:35:28.313317+0000","last_clean":"2024-03-30T02:35:28.313317+0000","last_became_active":"2024-03-30T02:35:28.312929+0000","last_became_peered":"2024-03-30T02:35:28.312929+0000","last_unstale":"2024-03-30T02:35:28.313317+0000","last_undegraded":"2024-03-30T02:35:28.313317+0000","last_fullsized":"2024-03-30T02:35:28.313317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:06.390076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207241+0000","last_change":"2024-03-30T02:34:46.737882+0000","last_active":"2024-03-30T02:35:27.207241+0000","last_peered":"2024-03-30T02:35:27.207241+0000","last_clean":"2024-03-30T02:35:27.207241+0000","last_became_active":"2024-03-30T02:34:46.737546+0000","last_became_peered":"2024-03-30T02:34:46.737546+0000","last_unstale":"2024-03-30T02:35:27.207241+0000","last_undegraded":"2024-03-30T02:35:27.207241+0000","last_fullsized":"2024-03-30T02:35:27.207241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:48:41.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206671+0000","last_change":"2024-03-30T02:34:46.742703+0000","last_active":"2024-03-30T02:35:27.206671+0000","last_peered":"2024-03-30T02:35:27.206671+0000","last_clean":"2024-03-30T02:35:27.206671+0000","last_became_active":"2024-03-30T02:34:46.740689+0000","last_became_peered":"2024-03-30T02:34:46.740689+0000","last_unstale":"2024-03-30T02:35:27.206671+0000","last_undegraded":"2024-03-30T02:35:27.206671+0000","last_fullsized":"2024-03-30T02:35:27.206671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:45.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211295+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:35:27.211295+0000","last_peered":"2024-03-30T02:35:27.211295+0000","last_clean":"2024-03-30T02:35:27.211295+0000","last_became_active":"2024-03-30T02:34:46.750395+0000","last_became_peered":"2024-03-30T02:34:46.750395+0000","last_unstale":"2024-03-30T02:35:27.211295+0000","last_undegraded":"2024-03-30T02:35:27.211295+0000","last_fullsized":"2024-03-30T02:35:27.211295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:01.282677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201720+0000","last_change":"2024-03-30T02:34:46.734814+0000","last_active":"2024-03-30T02:35:27.201720+0000","last_peered":"2024-03-30T02:35:27.201720+0000","last_clean":"2024-03-30T02:35:27.201720+0000","last_became_active":"2024-03-30T02:34:46.733978+0000","last_became_peered":"2024-03-30T02:34:46.733978+0000","last_unstale":"2024-03-30T02:35:27.201720+0000","last_undegraded":"2024-03-30T02:35:27.201720+0000","last_fullsized":"2024-03-30T02:35:27.201720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:31:39.467478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210574+0000","last_change":"2024-03-30T02:34:46.748163+0000","last_active":"2024-03-30T02:35:27.210574+0000","last_peered":"2024-03-30T02:35:27.210574+0000","last_clean":"2024-03-30T02:35:27.210574+0000","last_became_active":"2024-03-30T02:34:46.738845+0000","last_became_peered":"2024-03-30T02:34:46.738845+0000","last_unstale":"2024-03-30T02:35:27.210574+0000","last_undegraded":"2024-03-30T02:35:27.210574+0000","last_fullsized":"2024-03-30T02:35:27.210574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:09.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474944+0000","last_change":"2024-03-30T02:34:46.743231+0000","last_active":"2024-03-30T02:35:27.474944+0000","last_peered":"2024-03-30T02:35:27.474944+0000","last_clean":"2024-03-30T02:35:27.474944+0000","last_became_active":"2024-03-30T02:34:46.742619+0000","last_became_peered":"2024-03-30T02:34:46.742619+0000","last_unstale":"2024-03-30T02:35:27.474944+0000","last_undegraded":"2024-03-30T02:35:27.474944+0000","last_fullsized":"2024-03-30T02:35:27.474944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:17:51.789139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201627+0000","last_change":"2024-03-30T02:34:46.745872+0000","last_active":"2024-03-30T02:35:27.201627+0000","last_peered":"2024-03-30T02:35:27.201627+0000","last_clean":"2024-03-30T02:35:27.201627+0000","last_became_active":"2024-03-30T02:34:46.745476+0000","last_became_peered":"2024-03-30T02:34:46.745476+0000","last_unstale":"2024-03-30T02:35:27.201627+0000","last_undegraded":"2024-03-30T02:35:27.201627+0000","last_fullsized":"2024-03-30T02:35:27.201627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:30.774316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205973+0000","last_change":"2024-03-30T02:34:46.746178+0000","last_active":"2024-03-30T02:35:27.205973+0000","last_peered":"2024-03-30T02:35:27.205973+0000","last_clean":"2024-03-30T02:35:27.205973+0000","last_became_active":"2024-03-30T02:34:46.745865+0000","last_became_peered":"2024-03-30T02:34:46.745865+0000","last_unstale":"2024-03-30T02:35:27.205973+0000","last_undegraded":"2024-03-30T02:35:27.205973+0000","last_fullsized":"2024-03-30T02:35:27.205973+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:22.613637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248704,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70155,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":412,"num_read_kb":284,"num_write":240,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210465,"data_compressed":5484,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579928,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":468896,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.38500000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.66800000000000004}]},{"osd":2,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.503},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.76500000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.40000000000000002}]},{"osd":6,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.67900000000000005}]},{"osd":7,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70799999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56499999999999995}]},{"osd":9,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.72799999999999998}]},{"osd":10,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.81299999999999994}]}]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1085440,"data_stored":1119244,"data_compressed":55433,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.441},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.498,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56799999999999995}]},{"osd":3,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.495},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.35599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.629},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":8,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.59599999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67000000000000004}]},{"osd":9,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.60799999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75800000000000001,"15min":0.76100000000000001},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.755},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81599999999999995}]}]},{"osd":9,"up_from":62,"seq":266287972383,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":472970,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.44}]},{"osd":1,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.77000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":6.774,"5min":6.774,"15min":6.774},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.72399999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.52100000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":2.5310000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.52800000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.278,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78200000000000003},"min":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.038,"5min":1.099,"15min":1.099},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":0.878},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3149999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.751}]},{"osd":6,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64900000000000002}]},{"osd":7,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.91200000000000003,"15min":0.93200000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3879999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":2.4929999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.96199999999999997},"min":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.81899999999999995}]},{"osd":8,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88700000000000001,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":2.5470000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.878}]},{"osd":11,"last update":"Sat Mar 30 02:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.90400000000000003}]}]},{"osd":8,"up_from":55,"seq":236223201316,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":468921,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83399999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53300000000000003}]},{"osd":2,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.45300000000000001,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.32300000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.435}]},{"osd":3,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.621},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65000000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88200000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89900000000000002,"5min":0.995,"15min":0.995},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.40200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45900000000000002}]},{"osd":6,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.056,"15min":1.056},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.878,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68500000000000005}]},{"osd":7,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.443,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.998,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.71499999999999997}]},{"osd":9,"last update":"Sat Mar 30 02:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.55500000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.746},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.751,"15min":0.73799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.73599999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.88700000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.69799999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92100000000000004,"5min":1.091,"15min":1.091},"last":0.79300000000000004}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76600000000000001}]}]},{"osd":7,"up_from":48,"seq":206158430247,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1065727,"data_compressed":53605,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.41999999999999998},"min":{"1min":0.35399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44,"15min":0.435},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.871,"5min":1.034,"15min":1.034},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.51000000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.72799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.375},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.317,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70199999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.441}]},{"osd":5,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.503,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.496,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.78100000000000003}]},{"osd":6,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.56000000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.82699999999999996,"5min":0.877,"15min":0.877},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.47899999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88800000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.49199999999999999}]},{"osd":8,"last update":"Sat Mar 30 02:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.59799999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90100000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.47999999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65100000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89600000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70999999999999996,"15min":0.71099999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.81100000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.623}]},{"osd":10,"last update":"Sat Mar 30 02:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63500000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71599999999999997}]}]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":937984,"data_stored":464330,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97699999999999998,"5min":0.998,"15min":0.998},"last":1.6970000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.093}]},{"osd":1,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":1.6220000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.069,"15min":1.069},"last":0.58199999999999996}]},{"osd":3,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96199999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.9019999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.74099999999999999},"min":{"1min":0.45700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1040000000000001,"5min":1.236,"15min":1.236},"last":1.7689999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":1.069,"15min":1.069},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0329999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.63300000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0289999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.5289999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.871},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96599999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.95299999999999996}]},{"osd":7,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":0.84999999999999998},"min":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.627,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.734}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91400000000000003,"15min":0.92200000000000004},"min":{"1min":0.628,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1220000000000001,"5min":1.355,"15min":1.355},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.88600000000000001,"15min":0.89200000000000002},"min":{"1min":0.70099999999999996,"5min":0.623,"15min":0.623},"max":{"1min":1.0609999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":1.579}]},{"osd":9,"last update":"Sat Mar 30 02:35:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.95299999999999996,"15min":0.96899999999999997},"min":{"1min":0.627,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0800000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.135},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.90900000000000003,"15min":0.92700000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.004,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.9359999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.865},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.038}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1.6599999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.0660000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":528499,"data_compressed":19433,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.66100000000000003,"15min":0.77500000000000002},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81999999999999995,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.378,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60099999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54400000000000004,"15min":0.51400000000000001},"min":{"1min":0.46200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85799999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.54400000000000004,"15min":0.51200000000000001},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.438,"5min":0.378,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":1.028,"15min":1.028},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.56000000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.81100000000000005}]},{"osd":4,"last update":"Sat Mar 30 02:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41399999999999998,"15min":0.39800000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.44800000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61099999999999999,"15min":0.58099999999999996},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.92300000000000004,"5min":1.026,"15min":1.026},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.55300000000000005},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.622}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65700000000000003,"15min":0.63800000000000001},"min":{"1min":0.51600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66600000000000004}]},{"osd":7,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74099999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64000000000000001}]},{"osd":8,"last update":"Sat Mar 30 02:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89900000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.45400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.95599999999999996,"5min":1.044,"15min":1.044},"last":0.78500000000000003}]},{"osd":9,"last update":"Sat Mar 30 02:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69999999999999996,"15min":0.69899999999999995},"min":{"1min":0.58899999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83799999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93799999999999994,"5min":1.056,"15min":1.056},"last":0.70899999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.65600000000000003}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.751}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":465786,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 02:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81599999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57299999999999995,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86199999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57599999999999996}]},{"osd":2,"last update":"Sat Mar 30 02:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83399999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.60899999999999999,"15min":0.58999999999999997},"min":{"1min":0.44800000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73099999999999998}]},{"osd":3,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.496}]},{"osd":4,"last update":"Sat Mar 30 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.626,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":1.006,"15min":1.006},"last":0.65000000000000002}]},{"osd":5,"last update":"Sat Mar 30 02:35:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64500000000000002},"min":{"1min":0.499,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.84399999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90200000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84299999999999997,"5min":1.091,"15min":1.091},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Sat Mar 30 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.54800000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89100000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.071,"5min":1.113,"15min":1.113},"last":0.872}]},{"osd":9,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0629999999999999,"5min":1.212,"15min":1.212},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.879,"15min":0.876},"min":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.77100000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.874,"15min":0.86499999999999999},"min":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.66800000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.90200000000000002}]},{"osd":11,"last update":"Sat Mar 30 02:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.91600000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":529064,"data_compressed":19433,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.51600000000000001},"min":{"1min":0.34899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72799999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.63500000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.65900000000000003}]},{"osd":1,"last update":"Sat Mar 30 02:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56299999999999994,"15min":0.53300000000000003},"min":{"1min":0.44800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81000000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.73999999999999999}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84599999999999997,"5min":0.878,"15min":0.878},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.61399999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.82899999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47299999999999998}]},{"osd":6,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.58099999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85199999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49199999999999999}]},{"osd":7,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.76600000000000001}]},{"osd":8,"last update":"Sat Mar 30 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.49399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90000000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52100000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.58499999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.69999999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79200000000000004}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.72399999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":950272,"data_stored":464086,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47099999999999997,"15min":0.40699999999999997},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.434},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.60799999999999998,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.66100000000000003,"15min":0.58499999999999996},"min":{"1min":0.47299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.51400000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.66700000000000004,"15min":0.623},"min":{"1min":0.56899999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.64800000000000002,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93999999999999995,"5min":0.996,"15min":0.996},"last":0.88200000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0569999999999999,"5min":1.089,"15min":1.089},"last":0.81200000000000006}]},{"osd":6,"last update":"Sat Mar 30 02:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.80000000000000004,"15min":0.78000000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.47799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.0069999999999999}]},{"osd":7,"last update":"Sat Mar 30 02:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.997,"5min":1.071,"15min":1.071},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.131,"5min":1.25,"15min":1.25},"last":0.70699999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.004}]},{"osd":9,"last update":"Sat Mar 30 02:35:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.95299999999999996,"15min":0.94499999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.86599999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.038}]},{"osd":10,"last update":"Sat Mar 30 02:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.73999999999999999}]},{"osd":11,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.071},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.83499999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":946176,"data_stored":468797,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.38,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.747,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.36699999999999999,"15min":0.34699999999999998},"min":{"1min":0.313,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.34999999999999998},"min":{"1min":0.32400000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.39800000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.878,"15min":0.878},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58299999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Mar 30 02:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.501},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.38900000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.53200000000000003}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.497,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.69099999999999995,"15min":0.67300000000000004},"min":{"1min":0.52900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.63400000000000001}]},{"osd":9,"last update":"Sat Mar 30 02:35:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.80700000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.74099999999999999}]},{"osd":11,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.77500000000000002}]}]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":1019904,"data_stored":1061285,"data_compressed":53605,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.30099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64200000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.37,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.48599999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78300000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.66000000000000003}]},{"osd":2,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.376,"15min":0.36099999999999999},"min":{"1min":0.31900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.379,"15min":0.35899999999999999},"min":{"1min":0.308,"5min":0.253,"15min":0.253},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.44700000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81000000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.55800000000000005,"5min":0.378,"15min":0.378},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.55200000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.49199999999999999}]},{"osd":6,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623}]},{"osd":7,"last update":"Sat Mar 30 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.499,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.53900000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54200000000000004}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55500000000000005}]},{"osd":9,"last update":"Sat Mar 30 02:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.69999999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.873,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.86299999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.86599999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.747,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.871,"5min":1.024,"15min":1.024},"last":0.73399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64843,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65408,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65220,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T02:36:00.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-03-30T02:36:00.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:00 smithi067 ceph-mon[37808]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:00.560 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:00.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:00 smithi170 ceph-mon[78798]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:00.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:00 smithi082 ceph-mon[79176]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:01.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:01 smithi067 ceph-mon[37808]: pgmap v272: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-30T02:36:01.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:01 smithi170 ceph-mon[78798]: pgmap v272: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-30T02:36:01.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.696+0000 7f3617c00700 1 -- 172.21.15.67:0/2778231232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3610100e60 msgr2=0x7f36101012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:01.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.696+0000 7f3617c00700 1 --2- 172.21.15.67:0/2778231232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3610100e60 0x7f36101012e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0099c0 tx=0x7f35fc009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.696+0000 7f3617c00700 1 -- 172.21.15.67:0/2778231232 shutdown_connections 2024-03-30T02:36:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.696+0000 7f3617c00700 1 --2- 172.21.15.67:0/2778231232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3610101820 0x7f361010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.696+0000 7f3617c00700 1 --2- 172.21.15.67:0/2778231232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3610106ef0 0x7f36101072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.696+0000 7f3617c00700 1 --2- 172.21.15.67:0/2778231232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3610100e60 0x7f36101012e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.696+0000 7f3617c00700 1 -- 172.21.15.67:0/2778231232 >> 172.21.15.67:0/2778231232 conn(0x7f3610077fa0 msgr2=0x7f36100783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.697+0000 7f3617c00700 1 -- 172.21.15.67:0/2778231232 shutdown_connections 2024-03-30T02:36:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.697+0000 7f3617c00700 1 -- 172.21.15.67:0/2778231232 wait complete. 2024-03-30T02:36:01.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.698+0000 7f3617c00700 1 Processor -- start 2024-03-30T02:36:01.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.698+0000 7f3617c00700 1 -- start start 2024-03-30T02:36:01.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.698+0000 7f3617c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3610100e60 0x7f3610197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:01.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.698+0000 7f3617c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3610101820 0x7f3610197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:01.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.698+0000 7f3617c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3610106ef0 0x7f361019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:01.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.698+0000 7f3617c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361019c0d0 con 0x7f3610106ef0 2024-03-30T02:36:01.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.698+0000 7f3617c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361019c240 con 0x7f3610101820 2024-03-30T02:36:01.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.698+0000 7f3617c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361019c400 con 0x7f3610100e60 2024-03-30T02:36:01.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f360fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3610101820 0x7f3610197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f360fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3610101820 0x7f3610197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35074/0 (socket says 172.21.15.67:35074) 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f360fe00700 1 -- 172.21.15.67:0/1404523413 learned_addr learned my addr 172.21.15.67:0/1404523413 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f3615200700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3610100e60 0x7f3610197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f3615c00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3610106ef0 0x7f361019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f360fe00700 1 -- 172.21.15.67:0/1404523413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3610100e60 msgr2=0x7f3610197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f360fe00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3610100e60 0x7f3610197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f360fe00700 1 -- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3610106ef0 msgr2=0x7f361019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f360fe00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3610106ef0 0x7f361019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f360fe00700 1 -- 172.21.15.67:0/1404523413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0095f0 con 0x7f3610101820 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.699+0000 7f360fe00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3610101820 0x7f3610197760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00b580 tx=0x7f35fc00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.700+0000 7f360d600700 1 -- 172.21.15.67:0/1404523413 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc01c070 con 0x7f3610101820 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.700+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361019c680 con 0x7f3610101820 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.700+0000 7f360d600700 1 -- 172.21.15.67:0/1404523413 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35fc00fd40 con 0x7f3610101820 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.700+0000 7f360d600700 1 -- 172.21.15.67:0/1404523413 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc022b80 con 0x7f3610101820 2024-03-30T02:36:01.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.700+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36101a3f40 con 0x7f3610101820 2024-03-30T02:36:01.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.701+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36101092f0 con 0x7f3610101820 2024-03-30T02:36:01.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.703+0000 7f360d600700 1 -- 172.21.15.67:0/1404523413 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f35fc020070 con 0x7f3610101820 2024-03-30T02:36:01.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.704+0000 7f360d600700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35f80a11b0 0x7f35f80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:01.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.704+0000 7f360d600700 1 -- 172.21.15.67:0/1404523413 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f35fc0cdbb0 con 0x7f3610101820 2024-03-30T02:36:01.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.704+0000 7f3615200700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35f80a11b0 0x7f35f80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:01.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.705+0000 7f3615200700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35f80a11b0 0x7f35f80a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3604009c60 tx=0x7f3604009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:01.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.707+0000 7f360d600700 1 -- 172.21.15.67:0/1404523413 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f35fc096f10 con 0x7f3610101820 2024-03-30T02:36:01.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:01 smithi082 ceph-mon[79176]: pgmap v272: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-30T02:36:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.912+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f361004ea80 con 0x7f35f80a11b0 2024-03-30T02:36:01.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.925+0000 7f360d600700 1 -- 172.21.15.67:0/1404523413 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427210 (secure 0 0 0) 0x7f361004ea80 con 0x7f35f80a11b0 2024-03-30T02:36:01.932 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35f80a11b0 msgr2=0x7f35f80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35f80a11b0 0x7f35f80a3670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3604009c60 tx=0x7f3604009260 comp rx=0 tx=0).stop 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3610101820 msgr2=0x7f3610197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3610101820 0x7f3610197760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00b580 tx=0x7f35fc00bfd0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 shutdown_connections 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35f80a11b0 0x7f35f80a3670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3610106ef0 0x7f361019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3610100e60 0x7f3610197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 --2- 172.21.15.67:0/1404523413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3610101820 0x7f3610197760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 >> 172.21.15.67:0/1404523413 conn(0x7f3610077fa0 msgr2=0x7f36100ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 shutdown_connections 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:01.931+0000 7f3617c00700 1 -- 172.21.15.67:0/1404523413 wait complete. 2024-03-30T02:36:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-30T02:36:02.315 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-03-30T02:36:01.016866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660757,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":558,"num_read_kb":413,"num_write":423,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360356,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581468,"statfs":{"total":1151940427776,"available":1151571423232,"internally_reserved":0,"allocated":11796480,"data_stored":7577605,"data_compressed":324744,"data_compressed_allocated":442368,"data_compressed_original":2383872,"omap_allocated":19130,"internal_metadata":357020998},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003185"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206850+0000","last_change":"2024-03-30T02:34:46.755257+0000","last_active":"2024-03-30T02:35:27.206850+0000","last_peered":"2024-03-30T02:35:27.206850+0000","last_clean":"2024-03-30T02:35:27.206850+0000","last_became_active":"2024-03-30T02:34:46.754922+0000","last_became_peered":"2024-03-30T02:34:46.754922+0000","last_unstale":"2024-03-30T02:35:27.206850+0000","last_undegraded":"2024-03-30T02:35:27.206850+0000","last_fullsized":"2024-03-30T02:35:27.206850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:09.980124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207483+0000","last_change":"2024-03-30T02:34:46.742846+0000","last_active":"2024-03-30T02:35:27.207483+0000","last_peered":"2024-03-30T02:35:27.207483+0000","last_clean":"2024-03-30T02:35:27.207483+0000","last_became_active":"2024-03-30T02:34:46.738096+0000","last_became_peered":"2024-03-30T02:34:46.738096+0000","last_unstale":"2024-03-30T02:35:27.207483+0000","last_undegraded":"2024-03-30T02:35:27.207483+0000","last_fullsized":"2024-03-30T02:35:27.207483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:43:16.982251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207053+0000","last_change":"2024-03-30T02:34:46.739607+0000","last_active":"2024-03-30T02:35:27.207053+0000","last_peered":"2024-03-30T02:35:27.207053+0000","last_clean":"2024-03-30T02:35:27.207053+0000","last_became_active":"2024-03-30T02:34:46.739230+0000","last_became_peered":"2024-03-30T02:34:46.739230+0000","last_unstale":"2024-03-30T02:35:27.207053+0000","last_undegraded":"2024-03-30T02:35:27.207053+0000","last_fullsized":"2024-03-30T02:35:27.207053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:17.818874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206175+0000","last_change":"2024-03-30T02:34:46.742118+0000","last_active":"2024-03-30T02:35:27.206175+0000","last_peered":"2024-03-30T02:35:27.206175+0000","last_clean":"2024-03-30T02:35:27.206175+0000","last_became_active":"2024-03-30T02:34:46.740845+0000","last_became_peered":"2024-03-30T02:34:46.740845+0000","last_unstale":"2024-03-30T02:35:27.206175+0000","last_undegraded":"2024-03-30T02:35:27.206175+0000","last_fullsized":"2024-03-30T02:35:27.206175+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:33:28.595456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210363+0000","last_change":"2024-03-30T02:34:46.739557+0000","last_active":"2024-03-30T02:35:27.210363+0000","last_peered":"2024-03-30T02:35:27.210363+0000","last_clean":"2024-03-30T02:35:27.210363+0000","last_became_active":"2024-03-30T02:34:46.739147+0000","last_became_peered":"2024-03-30T02:34:46.739147+0000","last_unstale":"2024-03-30T02:35:27.210363+0000","last_undegraded":"2024-03-30T02:35:27.210363+0000","last_fullsized":"2024-03-30T02:35:27.210363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:12:52.307386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.211536+0000","last_change":"2024-03-30T02:35:28.211536+0000","last_active":"2024-03-30T02:35:28.211536+0000","last_peered":"2024-03-30T02:35:28.211536+0000","last_clean":"2024-03-30T02:35:28.211536+0000","last_became_active":"2024-03-30T02:35:28.211085+0000","last_became_peered":"2024-03-30T02:35:28.211085+0000","last_unstale":"2024-03-30T02:35:28.211536+0000","last_undegraded":"2024-03-30T02:35:28.211536+0000","last_fullsized":"2024-03-30T02:35:28.211536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:06:34.034277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202193+0000","last_change":"2024-03-30T02:34:46.735466+0000","last_active":"2024-03-30T02:35:27.202193+0000","last_peered":"2024-03-30T02:35:27.202193+0000","last_clean":"2024-03-30T02:35:27.202193+0000","last_became_active":"2024-03-30T02:34:46.734411+0000","last_became_peered":"2024-03-30T02:34:46.734411+0000","last_unstale":"2024-03-30T02:35:27.202193+0000","last_undegraded":"2024-03-30T02:35:27.202193+0000","last_fullsized":"2024-03-30T02:35:27.202193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:57.587768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210992+0000","last_change":"2024-03-30T02:34:46.747001+0000","last_active":"2024-03-30T02:35:27.210992+0000","last_peered":"2024-03-30T02:35:27.210992+0000","last_clean":"2024-03-30T02:35:27.210992+0000","last_became_active":"2024-03-30T02:34:46.740241+0000","last_became_peered":"2024-03-30T02:34:46.740241+0000","last_unstale":"2024-03-30T02:35:27.210992+0000","last_undegraded":"2024-03-30T02:35:27.210992+0000","last_fullsized":"2024-03-30T02:35:27.210992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:05.902190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.475138+0000","last_change":"2024-03-30T02:34:46.745549+0000","last_active":"2024-03-30T02:35:27.475138+0000","last_peered":"2024-03-30T02:35:27.475138+0000","last_clean":"2024-03-30T02:35:27.475138+0000","last_became_active":"2024-03-30T02:34:46.745200+0000","last_became_peered":"2024-03-30T02:34:46.745200+0000","last_unstale":"2024-03-30T02:35:27.475138+0000","last_undegraded":"2024-03-30T02:35:27.475138+0000","last_fullsized":"2024-03-30T02:35:27.475138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:37.931440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207118+0000","last_change":"2024-03-30T02:34:46.735859+0000","last_active":"2024-03-30T02:35:27.207118+0000","last_peered":"2024-03-30T02:35:27.207118+0000","last_clean":"2024-03-30T02:35:27.207118+0000","last_became_active":"2024-03-30T02:34:46.735481+0000","last_became_peered":"2024-03-30T02:34:46.735481+0000","last_unstale":"2024-03-30T02:35:27.207118+0000","last_undegraded":"2024-03-30T02:35:27.207118+0000","last_fullsized":"2024-03-30T02:35:27.207118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:51.037887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202925+0000","last_change":"2024-03-30T02:34:46.741412+0000","last_active":"2024-03-30T02:35:27.202925+0000","last_peered":"2024-03-30T02:35:27.202925+0000","last_clean":"2024-03-30T02:35:27.202925+0000","last_became_active":"2024-03-30T02:34:46.741106+0000","last_became_peered":"2024-03-30T02:34:46.741106+0000","last_unstale":"2024-03-30T02:35:27.202925+0000","last_undegraded":"2024-03-30T02:35:27.202925+0000","last_fullsized":"2024-03-30T02:35:27.202925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:06:08.553028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205930+0000","last_change":"2024-03-30T02:34:46.746205+0000","last_active":"2024-03-30T02:35:27.205930+0000","last_peered":"2024-03-30T02:35:27.205930+0000","last_clean":"2024-03-30T02:35:27.205930+0000","last_became_active":"2024-03-30T02:34:46.745803+0000","last_became_peered":"2024-03-30T02:34:46.745803+0000","last_unstale":"2024-03-30T02:35:27.205930+0000","last_undegraded":"2024-03-30T02:35:27.205930+0000","last_fullsized":"2024-03-30T02:35:27.205930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:05.127095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201535+0000","last_change":"2024-03-30T02:34:46.722978+0000","last_active":"2024-03-30T02:35:27.201535+0000","last_peered":"2024-03-30T02:35:27.201535+0000","last_clean":"2024-03-30T02:35:27.201535+0000","last_became_active":"2024-03-30T02:34:46.722652+0000","last_became_peered":"2024-03-30T02:34:46.722652+0000","last_unstale":"2024-03-30T02:35:27.201535+0000","last_undegraded":"2024-03-30T02:35:27.201535+0000","last_fullsized":"2024-03-30T02:35:27.201535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:28:12.078216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206658+0000","last_change":"2024-03-30T02:34:46.738895+0000","last_active":"2024-03-30T02:35:27.206658+0000","last_peered":"2024-03-30T02:35:27.206658+0000","last_clean":"2024-03-30T02:35:27.206658+0000","last_became_active":"2024-03-30T02:34:46.738510+0000","last_became_peered":"2024-03-30T02:34:46.738510+0000","last_unstale":"2024-03-30T02:35:27.206658+0000","last_undegraded":"2024-03-30T02:35:27.206658+0000","last_fullsized":"2024-03-30T02:35:27.206658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:09.859259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206363+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:35:27.206363+0000","last_peered":"2024-03-30T02:35:27.206363+0000","last_clean":"2024-03-30T02:35:27.206363+0000","last_became_active":"2024-03-30T02:34:46.750301+0000","last_became_peered":"2024-03-30T02:34:46.750301+0000","last_unstale":"2024-03-30T02:35:27.206363+0000","last_undegraded":"2024-03-30T02:35:27.206363+0000","last_fullsized":"2024-03-30T02:35:27.206363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:06.475881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474462+0000","last_change":"2024-03-30T02:34:46.748966+0000","last_active":"2024-03-30T02:35:27.474462+0000","last_peered":"2024-03-30T02:35:27.474462+0000","last_clean":"2024-03-30T02:35:27.474462+0000","last_became_active":"2024-03-30T02:34:46.748650+0000","last_became_peered":"2024-03-30T02:34:46.748650+0000","last_unstale":"2024-03-30T02:35:27.474462+0000","last_undegraded":"2024-03-30T02:35:27.474462+0000","last_fullsized":"2024-03-30T02:35:27.474462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:36.558864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206625+0000","last_change":"2024-03-30T02:34:46.746127+0000","last_active":"2024-03-30T02:35:27.206625+0000","last_peered":"2024-03-30T02:35:27.206625+0000","last_clean":"2024-03-30T02:35:27.206625+0000","last_became_active":"2024-03-30T02:34:46.745806+0000","last_became_peered":"2024-03-30T02:34:46.745806+0000","last_unstale":"2024-03-30T02:35:27.206625+0000","last_undegraded":"2024-03-30T02:35:27.206625+0000","last_fullsized":"2024-03-30T02:35:27.206625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:54.097996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.212243+0000","last_change":"2024-03-30T02:34:46.738391+0000","last_active":"2024-03-30T02:35:27.212243+0000","last_peered":"2024-03-30T02:35:27.212243+0000","last_clean":"2024-03-30T02:35:27.212243+0000","last_became_active":"2024-03-30T02:34:46.737110+0000","last_became_peered":"2024-03-30T02:34:46.737110+0000","last_unstale":"2024-03-30T02:35:27.212243+0000","last_undegraded":"2024-03-30T02:35:27.212243+0000","last_fullsized":"2024-03-30T02:35:27.212243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:20.326107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206276+0000","last_change":"2024-03-30T02:34:46.744522+0000","last_active":"2024-03-30T02:35:27.206276+0000","last_peered":"2024-03-30T02:35:27.206276+0000","last_clean":"2024-03-30T02:35:27.206276+0000","last_became_active":"2024-03-30T02:34:46.744088+0000","last_became_peered":"2024-03-30T02:34:46.744088+0000","last_unstale":"2024-03-30T02:35:27.206276+0000","last_undegraded":"2024-03-30T02:35:27.206276+0000","last_fullsized":"2024-03-30T02:35:27.206276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:35.698979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207803+0000","last_change":"2024-03-30T02:34:50.770855+0000","last_active":"2024-03-30T02:35:27.207803+0000","last_peered":"2024-03-30T02:35:27.207803+0000","last_clean":"2024-03-30T02:35:27.207803+0000","last_became_active":"2024-03-30T02:34:50.770420+0000","last_became_peered":"2024-03-30T02:34:50.770420+0000","last_unstale":"2024-03-30T02:35:27.207803+0000","last_undegraded":"2024-03-30T02:35:27.207803+0000","last_fullsized":"2024-03-30T02:35:27.207803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:00:08.847059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:59.192537+0000","last_change":"2024-03-30T02:34:46.207252+0000","last_active":"2024-03-30T02:35:59.192537+0000","last_peered":"2024-03-30T02:35:59.192537+0000","last_clean":"2024-03-30T02:35:59.192537+0000","last_became_active":"2024-03-30T02:34:46.206916+0000","last_became_peered":"2024-03-30T02:34:46.206916+0000","last_unstale":"2024-03-30T02:35:59.192537+0000","last_undegraded":"2024-03-30T02:35:59.192537+0000","last_fullsized":"2024-03-30T02:35:59.192537+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:03:50.192245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.209974+0000","last_change":"2024-03-30T02:34:46.745778+0000","last_active":"2024-03-30T02:35:27.209974+0000","last_peered":"2024-03-30T02:35:27.209974+0000","last_clean":"2024-03-30T02:35:27.209974+0000","last_became_active":"2024-03-30T02:34:46.745471+0000","last_became_peered":"2024-03-30T02:34:46.745471+0000","last_unstale":"2024-03-30T02:35:27.209974+0000","last_undegraded":"2024-03-30T02:35:27.209974+0000","last_fullsized":"2024-03-30T02:35:27.209974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:57.314163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205256+0000","last_change":"2024-03-30T02:34:50.769577+0000","last_active":"2024-03-30T02:35:27.205256+0000","last_peered":"2024-03-30T02:35:27.205256+0000","last_clean":"2024-03-30T02:35:27.205256+0000","last_became_active":"2024-03-30T02:34:50.769100+0000","last_became_peered":"2024-03-30T02:34:50.769100+0000","last_unstale":"2024-03-30T02:35:27.205256+0000","last_undegraded":"2024-03-30T02:35:27.205256+0000","last_fullsized":"2024-03-30T02:35:27.205256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:08:43.495671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211228+0000","last_change":"2024-03-30T02:34:46.218032+0000","last_active":"2024-03-30T02:35:27.211228+0000","last_peered":"2024-03-30T02:35:27.211228+0000","last_clean":"2024-03-30T02:35:27.211228+0000","last_became_active":"2024-03-30T02:34:46.217733+0000","last_became_peered":"2024-03-30T02:34:46.217733+0000","last_unstale":"2024-03-30T02:35:27.211228+0000","last_undegraded":"2024-03-30T02:35:27.211228+0000","last_fullsized":"2024-03-30T02:35:27.211228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:58.413590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207675+0000","last_change":"2024-03-30T02:34:46.750588+0000","last_active":"2024-03-30T02:35:27.207675+0000","last_peered":"2024-03-30T02:35:27.207675+0000","last_clean":"2024-03-30T02:35:27.207675+0000","last_became_active":"2024-03-30T02:34:46.750016+0000","last_became_peered":"2024-03-30T02:34:46.750016+0000","last_unstale":"2024-03-30T02:35:27.207675+0000","last_undegraded":"2024-03-30T02:35:27.207675+0000","last_fullsized":"2024-03-30T02:35:27.207675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:19.144544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.213404+0000","last_change":"2024-03-30T02:35:28.213404+0000","last_active":"2024-03-30T02:35:28.213404+0000","last_peered":"2024-03-30T02:35:28.213404+0000","last_clean":"2024-03-30T02:35:28.213404+0000","last_became_active":"2024-03-30T02:35:28.212746+0000","last_became_peered":"2024-03-30T02:35:28.212746+0000","last_unstale":"2024-03-30T02:35:28.213404+0000","last_undegraded":"2024-03-30T02:35:28.213404+0000","last_fullsized":"2024-03-30T02:35:28.213404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:24.807469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211663+0000","last_change":"2024-03-30T02:34:46.213741+0000","last_active":"2024-03-30T02:35:27.211663+0000","last_peered":"2024-03-30T02:35:27.211663+0000","last_clean":"2024-03-30T02:35:27.211663+0000","last_became_active":"2024-03-30T02:34:46.213395+0000","last_became_peered":"2024-03-30T02:34:46.213395+0000","last_unstale":"2024-03-30T02:35:27.211663+0000","last_undegraded":"2024-03-30T02:35:27.211663+0000","last_fullsized":"2024-03-30T02:35:27.211663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:13.192415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474698+0000","last_change":"2024-03-30T02:34:46.738401+0000","last_active":"2024-03-30T02:35:27.474698+0000","last_peered":"2024-03-30T02:35:27.474698+0000","last_clean":"2024-03-30T02:35:27.474698+0000","last_became_active":"2024-03-30T02:34:46.737804+0000","last_became_peered":"2024-03-30T02:34:46.737804+0000","last_unstale":"2024-03-30T02:35:27.474698+0000","last_undegraded":"2024-03-30T02:35:27.474698+0000","last_fullsized":"2024-03-30T02:35:27.474698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:56:08.194439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205124+0000","last_change":"2024-03-30T02:34:50.766222+0000","last_active":"2024-03-30T02:35:27.205124+0000","last_peered":"2024-03-30T02:35:27.205124+0000","last_clean":"2024-03-30T02:35:27.205124+0000","last_became_active":"2024-03-30T02:34:50.765836+0000","last_became_peered":"2024-03-30T02:34:50.765836+0000","last_unstale":"2024-03-30T02:35:27.205124+0000","last_undegraded":"2024-03-30T02:35:27.205124+0000","last_fullsized":"2024-03-30T02:35:27.205124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:32.529535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210488+0000","last_change":"2024-03-30T02:34:46.212586+0000","last_active":"2024-03-30T02:35:27.210488+0000","last_peered":"2024-03-30T02:35:27.210488+0000","last_clean":"2024-03-30T02:35:27.210488+0000","last_became_active":"2024-03-30T02:34:46.212110+0000","last_became_peered":"2024-03-30T02:34:46.212110+0000","last_unstale":"2024-03-30T02:35:27.210488+0000","last_undegraded":"2024-03-30T02:35:27.210488+0000","last_fullsized":"2024-03-30T02:35:27.210488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:38.630953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202351+0000","last_change":"2024-03-30T02:34:46.734923+0000","last_active":"2024-03-30T02:35:27.202351+0000","last_peered":"2024-03-30T02:35:27.202351+0000","last_clean":"2024-03-30T02:35:27.202351+0000","last_became_active":"2024-03-30T02:34:46.734572+0000","last_became_peered":"2024-03-30T02:34:46.734572+0000","last_unstale":"2024-03-30T02:35:27.202351+0000","last_undegraded":"2024-03-30T02:35:27.202351+0000","last_fullsized":"2024-03-30T02:35:27.202351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:28.647788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206133+0000","last_change":"2024-03-30T02:34:50.757803+0000","last_active":"2024-03-30T02:35:27.206133+0000","last_peered":"2024-03-30T02:35:27.206133+0000","last_clean":"2024-03-30T02:35:27.206133+0000","last_became_active":"2024-03-30T02:34:50.757198+0000","last_became_peered":"2024-03-30T02:34:50.757198+0000","last_unstale":"2024-03-30T02:35:27.206133+0000","last_undegraded":"2024-03-30T02:35:27.206133+0000","last_fullsized":"2024-03-30T02:35:27.206133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:54:49.399741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:56.183893+0000","last_change":"2024-03-30T02:34:46.214549+0000","last_active":"2024-03-30T02:35:56.183893+0000","last_peered":"2024-03-30T02:35:56.183893+0000","last_clean":"2024-03-30T02:35:56.183893+0000","last_became_active":"2024-03-30T02:34:46.214198+0000","last_became_peered":"2024-03-30T02:34:46.214198+0000","last_unstale":"2024-03-30T02:35:56.183893+0000","last_undegraded":"2024-03-30T02:35:56.183893+0000","last_fullsized":"2024-03-30T02:35:56.183893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:45:25.618982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210410+0000","last_change":"2024-03-30T02:34:46.737771+0000","last_active":"2024-03-30T02:35:27.210410+0000","last_peered":"2024-03-30T02:35:27.210410+0000","last_clean":"2024-03-30T02:35:27.210410+0000","last_became_active":"2024-03-30T02:34:46.737438+0000","last_became_peered":"2024-03-30T02:34:46.737438+0000","last_unstale":"2024-03-30T02:35:27.210410+0000","last_undegraded":"2024-03-30T02:35:27.210410+0000","last_fullsized":"2024-03-30T02:35:27.210410+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:08.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206737+0000","last_change":"2024-03-30T02:34:50.767304+0000","last_active":"2024-03-30T02:35:27.206737+0000","last_peered":"2024-03-30T02:35:27.206737+0000","last_clean":"2024-03-30T02:35:27.206737+0000","last_became_active":"2024-03-30T02:34:50.766909+0000","last_became_peered":"2024-03-30T02:34:50.766909+0000","last_unstale":"2024-03-30T02:35:27.206737+0000","last_undegraded":"2024-03-30T02:35:27.206737+0000","last_fullsized":"2024-03-30T02:35:27.206737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:24.021991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206539+0000","last_change":"2024-03-30T02:34:46.214937+0000","last_active":"2024-03-30T02:35:27.206539+0000","last_peered":"2024-03-30T02:35:27.206539+0000","last_clean":"2024-03-30T02:35:27.206539+0000","last_became_active":"2024-03-30T02:34:46.214503+0000","last_became_peered":"2024-03-30T02:34:46.214503+0000","last_unstale":"2024-03-30T02:35:27.206539+0000","last_undegraded":"2024-03-30T02:35:27.206539+0000","last_fullsized":"2024-03-30T02:35:27.206539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:20:11.624919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.203098+0000","last_change":"2024-03-30T02:34:46.737374+0000","last_active":"2024-03-30T02:35:27.203098+0000","last_peered":"2024-03-30T02:35:27.203098+0000","last_clean":"2024-03-30T02:35:27.203098+0000","last_became_active":"2024-03-30T02:34:46.735307+0000","last_became_peered":"2024-03-30T02:34:46.735307+0000","last_unstale":"2024-03-30T02:35:27.203098+0000","last_undegraded":"2024-03-30T02:35:27.203098+0000","last_fullsized":"2024-03-30T02:35:27.203098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:30.482728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206531+0000","last_change":"2024-03-30T02:34:50.768016+0000","last_active":"2024-03-30T02:35:27.206531+0000","last_peered":"2024-03-30T02:35:27.206531+0000","last_clean":"2024-03-30T02:35:27.206531+0000","last_became_active":"2024-03-30T02:34:50.767564+0000","last_became_peered":"2024-03-30T02:34:50.767564+0000","last_unstale":"2024-03-30T02:35:27.206531+0000","last_undegraded":"2024-03-30T02:35:27.206531+0000","last_fullsized":"2024-03-30T02:35:27.206531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:16.888754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201236+0000","last_change":"2024-03-30T02:34:46.212982+0000","last_active":"2024-03-30T02:35:27.201236+0000","last_peered":"2024-03-30T02:35:27.201236+0000","last_clean":"2024-03-30T02:35:27.201236+0000","last_became_active":"2024-03-30T02:34:46.212550+0000","last_became_peered":"2024-03-30T02:34:46.212550+0000","last_unstale":"2024-03-30T02:35:27.201236+0000","last_undegraded":"2024-03-30T02:35:27.201236+0000","last_fullsized":"2024-03-30T02:35:27.201236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:44:50.156615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.215165+0000","last_change":"2024-03-30T02:35:28.213774+0000","last_active":"2024-03-30T02:35:28.215165+0000","last_peered":"2024-03-30T02:35:28.215165+0000","last_clean":"2024-03-30T02:35:28.215165+0000","last_became_active":"2024-03-30T02:35:28.213085+0000","last_became_peered":"2024-03-30T02:35:28.213085+0000","last_unstale":"2024-03-30T02:35:28.215165+0000","last_undegraded":"2024-03-30T02:35:28.215165+0000","last_fullsized":"2024-03-30T02:35:28.215165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:56:30.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207389+0000","last_change":"2024-03-30T02:34:50.773036+0000","last_active":"2024-03-30T02:35:27.207389+0000","last_peered":"2024-03-30T02:35:27.207389+0000","last_clean":"2024-03-30T02:35:27.207389+0000","last_became_active":"2024-03-30T02:34:50.772614+0000","last_became_peered":"2024-03-30T02:34:50.772614+0000","last_unstale":"2024-03-30T02:35:27.207389+0000","last_undegraded":"2024-03-30T02:35:27.207389+0000","last_fullsized":"2024-03-30T02:35:27.207389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:54:28.792793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201717+0000","last_change":"2024-03-30T02:34:46.209802+0000","last_active":"2024-03-30T02:35:27.201717+0000","last_peered":"2024-03-30T02:35:27.201717+0000","last_clean":"2024-03-30T02:35:27.201717+0000","last_became_active":"2024-03-30T02:34:46.209246+0000","last_became_peered":"2024-03-30T02:34:46.209246+0000","last_unstale":"2024-03-30T02:35:27.201717+0000","last_undegraded":"2024-03-30T02:35:27.201717+0000","last_fullsized":"2024-03-30T02:35:27.201717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:09:57.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.209595+0000","last_change":"2024-03-30T02:34:46.743638+0000","last_active":"2024-03-30T02:35:27.209595+0000","last_peered":"2024-03-30T02:35:27.209595+0000","last_clean":"2024-03-30T02:35:27.209595+0000","last_became_active":"2024-03-30T02:34:46.743246+0000","last_became_peered":"2024-03-30T02:34:46.743246+0000","last_unstale":"2024-03-30T02:35:27.209595+0000","last_undegraded":"2024-03-30T02:35:27.209595+0000","last_fullsized":"2024-03-30T02:35:27.209595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:49:55.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210472+0000","last_change":"2024-03-30T02:34:50.768586+0000","last_active":"2024-03-30T02:35:27.210472+0000","last_peered":"2024-03-30T02:35:27.210472+0000","last_clean":"2024-03-30T02:35:27.210472+0000","last_became_active":"2024-03-30T02:34:50.766894+0000","last_became_peered":"2024-03-30T02:34:50.766894+0000","last_unstale":"2024-03-30T02:35:27.210472+0000","last_undegraded":"2024-03-30T02:35:27.210472+0000","last_fullsized":"2024-03-30T02:35:27.210472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:36:39.329608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":100,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:58.185838+0000","last_change":"2024-03-30T02:34:46.204994+0000","last_active":"2024-03-30T02:35:58.185838+0000","last_peered":"2024-03-30T02:35:58.185838+0000","last_clean":"2024-03-30T02:35:58.185838+0000","last_became_active":"2024-03-30T02:34:46.204710+0000","last_became_peered":"2024-03-30T02:34:46.204710+0000","last_unstale":"2024-03-30T02:35:58.185838+0000","last_undegraded":"2024-03-30T02:35:58.185838+0000","last_fullsized":"2024-03-30T02:35:58.185838+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:39.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.475019+0000","last_change":"2024-03-30T02:34:46.733573+0000","last_active":"2024-03-30T02:35:27.475019+0000","last_peered":"2024-03-30T02:35:27.475019+0000","last_clean":"2024-03-30T02:35:27.475019+0000","last_became_active":"2024-03-30T02:34:46.733241+0000","last_became_peered":"2024-03-30T02:34:46.733241+0000","last_unstale":"2024-03-30T02:35:27.475019+0000","last_undegraded":"2024-03-30T02:35:27.475019+0000","last_fullsized":"2024-03-30T02:35:27.475019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:07:10.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.313882+0000","last_change":"2024-03-30T02:35:28.313493+0000","last_active":"2024-03-30T02:35:28.313882+0000","last_peered":"2024-03-30T02:35:28.313882+0000","last_clean":"2024-03-30T02:35:28.313882+0000","last_became_active":"2024-03-30T02:35:28.313065+0000","last_became_peered":"2024-03-30T02:35:28.313065+0000","last_unstale":"2024-03-30T02:35:28.313882+0000","last_undegraded":"2024-03-30T02:35:28.313882+0000","last_fullsized":"2024-03-30T02:35:28.313882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:44:52.675251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202943+0000","last_change":"2024-03-30T02:34:46.212732+0000","last_active":"2024-03-30T02:35:27.202943+0000","last_peered":"2024-03-30T02:35:27.202943+0000","last_clean":"2024-03-30T02:35:27.202943+0000","last_became_active":"2024-03-30T02:34:46.212306+0000","last_became_peered":"2024-03-30T02:34:46.212306+0000","last_unstale":"2024-03-30T02:35:27.202943+0000","last_undegraded":"2024-03-30T02:35:27.202943+0000","last_fullsized":"2024-03-30T02:35:27.202943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:27:15.766203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207755+0000","last_change":"2024-03-30T02:34:46.749420+0000","last_active":"2024-03-30T02:35:27.207755+0000","last_peered":"2024-03-30T02:35:27.207755+0000","last_clean":"2024-03-30T02:35:27.207755+0000","last_became_active":"2024-03-30T02:34:46.745286+0000","last_became_peered":"2024-03-30T02:34:46.745286+0000","last_unstale":"2024-03-30T02:35:27.207755+0000","last_undegraded":"2024-03-30T02:35:27.207755+0000","last_fullsized":"2024-03-30T02:35:27.207755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:01.489862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206147+0000","last_change":"2024-03-30T02:34:50.760542+0000","last_active":"2024-03-30T02:35:27.206147+0000","last_peered":"2024-03-30T02:35:27.206147+0000","last_clean":"2024-03-30T02:35:27.206147+0000","last_became_active":"2024-03-30T02:34:50.760087+0000","last_became_peered":"2024-03-30T02:34:50.760087+0000","last_unstale":"2024-03-30T02:35:27.206147+0000","last_undegraded":"2024-03-30T02:35:27.206147+0000","last_fullsized":"2024-03-30T02:35:27.206147+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:46.352189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202468+0000","last_change":"2024-03-30T02:34:46.224058+0000","last_active":"2024-03-30T02:35:27.202468+0000","last_peered":"2024-03-30T02:35:27.202468+0000","last_clean":"2024-03-30T02:35:27.202468+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:35:27.202468+0000","last_undegraded":"2024-03-30T02:35:27.202468+0000","last_fullsized":"2024-03-30T02:35:27.202468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:31:10.124533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474763+0000","last_change":"2024-03-30T02:34:46.741413+0000","last_active":"2024-03-30T02:35:27.474763+0000","last_peered":"2024-03-30T02:35:27.474763+0000","last_clean":"2024-03-30T02:35:27.474763+0000","last_became_active":"2024-03-30T02:34:46.741112+0000","last_became_peered":"2024-03-30T02:34:46.741112+0000","last_unstale":"2024-03-30T02:35:27.474763+0000","last_undegraded":"2024-03-30T02:35:27.474763+0000","last_fullsized":"2024-03-30T02:35:27.474763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:34.820166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206802+0000","last_change":"2024-03-30T02:34:50.752315+0000","last_active":"2024-03-30T02:35:27.206802+0000","last_peered":"2024-03-30T02:35:27.206802+0000","last_clean":"2024-03-30T02:35:27.206802+0000","last_became_active":"2024-03-30T02:34:50.750985+0000","last_became_peered":"2024-03-30T02:34:50.750985+0000","last_unstale":"2024-03-30T02:35:27.206802+0000","last_undegraded":"2024-03-30T02:35:27.206802+0000","last_fullsized":"2024-03-30T02:35:27.206802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:37.271063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'36","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:58.185998+0000","last_change":"2024-03-30T02:34:46.214759+0000","last_active":"2024-03-30T02:35:58.185998+0000","last_peered":"2024-03-30T02:35:58.185998+0000","last_clean":"2024-03-30T02:35:58.185998+0000","last_became_active":"2024-03-30T02:34:46.214388+0000","last_became_peered":"2024-03-30T02:34:46.214388+0000","last_unstale":"2024-03-30T02:35:58.185998+0000","last_undegraded":"2024-03-30T02:35:58.185998+0000","last_fullsized":"2024-03-30T02:35:58.185998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:41.100461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63139,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":86,"num_read_kb":105,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211453+0000","last_change":"2024-03-30T02:34:46.741112+0000","last_active":"2024-03-30T02:35:27.211453+0000","last_peered":"2024-03-30T02:35:27.211453+0000","last_clean":"2024-03-30T02:35:27.211453+0000","last_became_active":"2024-03-30T02:34:46.739664+0000","last_became_peered":"2024-03-30T02:34:46.739664+0000","last_unstale":"2024-03-30T02:35:27.211453+0000","last_undegraded":"2024-03-30T02:35:27.211453+0000","last_fullsized":"2024-03-30T02:35:27.211453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:07.763942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202806+0000","last_change":"2024-03-30T02:34:50.765645+0000","last_active":"2024-03-30T02:35:27.202806+0000","last_peered":"2024-03-30T02:35:27.202806+0000","last_clean":"2024-03-30T02:35:27.202806+0000","last_became_active":"2024-03-30T02:34:50.765345+0000","last_became_peered":"2024-03-30T02:34:50.765345+0000","last_unstale":"2024-03-30T02:35:27.202806+0000","last_undegraded":"2024-03-30T02:35:27.202806+0000","last_fullsized":"2024-03-30T02:35:27.202806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:25:38.591525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206620+0000","last_change":"2024-03-30T02:34:46.216643+0000","last_active":"2024-03-30T02:35:27.206620+0000","last_peered":"2024-03-30T02:35:27.206620+0000","last_clean":"2024-03-30T02:35:27.206620+0000","last_became_active":"2024-03-30T02:34:46.216188+0000","last_became_peered":"2024-03-30T02:34:46.216188+0000","last_unstale":"2024-03-30T02:35:27.206620+0000","last_undegraded":"2024-03-30T02:35:27.206620+0000","last_fullsized":"2024-03-30T02:35:27.206620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:49:10.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201557+0000","last_change":"2024-03-30T02:34:46.748313+0000","last_active":"2024-03-30T02:35:27.201557+0000","last_peered":"2024-03-30T02:35:27.201557+0000","last_clean":"2024-03-30T02:35:27.201557+0000","last_became_active":"2024-03-30T02:34:46.742897+0000","last_became_peered":"2024-03-30T02:34:46.742897+0000","last_unstale":"2024-03-30T02:35:27.201557+0000","last_undegraded":"2024-03-30T02:35:27.201557+0000","last_fullsized":"2024-03-30T02:35:27.201557+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:00.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205859+0000","last_change":"2024-03-30T02:34:50.760993+0000","last_active":"2024-03-30T02:35:27.205859+0000","last_peered":"2024-03-30T02:35:27.205859+0000","last_clean":"2024-03-30T02:35:27.205859+0000","last_became_active":"2024-03-30T02:34:50.760535+0000","last_became_peered":"2024-03-30T02:34:50.760535+0000","last_unstale":"2024-03-30T02:35:27.205859+0000","last_undegraded":"2024-03-30T02:35:27.205859+0000","last_fullsized":"2024-03-30T02:35:27.205859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:15.569533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202035+0000","last_change":"2024-03-30T02:34:46.216603+0000","last_active":"2024-03-30T02:35:27.202035+0000","last_peered":"2024-03-30T02:35:27.202035+0000","last_clean":"2024-03-30T02:35:27.202035+0000","last_became_active":"2024-03-30T02:34:46.216231+0000","last_became_peered":"2024-03-30T02:34:46.216231+0000","last_unstale":"2024-03-30T02:35:27.202035+0000","last_undegraded":"2024-03-30T02:35:27.202035+0000","last_fullsized":"2024-03-30T02:35:27.202035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:51:00.659652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202643+0000","last_change":"2024-03-30T02:34:46.742815+0000","last_active":"2024-03-30T02:35:27.202643+0000","last_peered":"2024-03-30T02:35:27.202643+0000","last_clean":"2024-03-30T02:35:27.202643+0000","last_became_active":"2024-03-30T02:34:46.742487+0000","last_became_peered":"2024-03-30T02:34:46.742487+0000","last_unstale":"2024-03-30T02:35:27.202643+0000","last_undegraded":"2024-03-30T02:35:27.202643+0000","last_fullsized":"2024-03-30T02:35:27.202643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:07.182990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211657+0000","last_change":"2024-03-30T02:34:50.770398+0000","last_active":"2024-03-30T02:35:27.211657+0000","last_peered":"2024-03-30T02:35:27.211657+0000","last_clean":"2024-03-30T02:35:27.211657+0000","last_became_active":"2024-03-30T02:34:50.770000+0000","last_became_peered":"2024-03-30T02:34:50.770000+0000","last_unstale":"2024-03-30T02:35:27.211657+0000","last_undegraded":"2024-03-30T02:35:27.211657+0000","last_fullsized":"2024-03-30T02:35:27.211657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:14:49.057081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:54.180972+0000","last_change":"2024-03-30T02:34:46.209403+0000","last_active":"2024-03-30T02:35:54.180972+0000","last_peered":"2024-03-30T02:35:54.180972+0000","last_clean":"2024-03-30T02:35:54.180972+0000","last_became_active":"2024-03-30T02:34:46.209065+0000","last_became_peered":"2024-03-30T02:34:46.209065+0000","last_unstale":"2024-03-30T02:35:54.180972+0000","last_undegraded":"2024-03-30T02:35:54.180972+0000","last_fullsized":"2024-03-30T02:35:54.180972+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:36:20.427720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":933,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202270+0000","last_change":"2024-03-30T02:34:46.734861+0000","last_active":"2024-03-30T02:35:27.202270+0000","last_peered":"2024-03-30T02:35:27.202270+0000","last_clean":"2024-03-30T02:35:27.202270+0000","last_became_active":"2024-03-30T02:34:46.734376+0000","last_became_peered":"2024-03-30T02:34:46.734376+0000","last_unstale":"2024-03-30T02:35:27.202270+0000","last_undegraded":"2024-03-30T02:35:27.202270+0000","last_fullsized":"2024-03-30T02:35:27.202270+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:08:36.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201818+0000","last_change":"2024-03-30T02:34:50.767469+0000","last_active":"2024-03-30T02:35:27.201818+0000","last_peered":"2024-03-30T02:35:27.201818+0000","last_clean":"2024-03-30T02:35:27.201818+0000","last_became_active":"2024-03-30T02:34:50.765460+0000","last_became_peered":"2024-03-30T02:34:50.765460+0000","last_unstale":"2024-03-30T02:35:27.201818+0000","last_undegraded":"2024-03-30T02:35:27.201818+0000","last_fullsized":"2024-03-30T02:35:27.201818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:54:06.940171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210118+0000","last_change":"2024-03-30T02:34:46.224053+0000","last_active":"2024-03-30T02:35:27.210118+0000","last_peered":"2024-03-30T02:35:27.210118+0000","last_clean":"2024-03-30T02:35:27.210118+0000","last_became_active":"2024-03-30T02:34:46.223666+0000","last_became_peered":"2024-03-30T02:34:46.223666+0000","last_unstale":"2024-03-30T02:35:27.210118+0000","last_undegraded":"2024-03-30T02:35:27.210118+0000","last_fullsized":"2024-03-30T02:35:27.210118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:18:22.976298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201712+0000","last_change":"2024-03-30T02:34:46.734838+0000","last_active":"2024-03-30T02:35:27.201712+0000","last_peered":"2024-03-30T02:35:27.201712+0000","last_clean":"2024-03-30T02:35:27.201712+0000","last_became_active":"2024-03-30T02:34:46.734428+0000","last_became_peered":"2024-03-30T02:34:46.734428+0000","last_unstale":"2024-03-30T02:35:27.201712+0000","last_undegraded":"2024-03-30T02:35:27.201712+0000","last_fullsized":"2024-03-30T02:35:27.201712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:42:51.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201629+0000","last_change":"2024-03-30T02:34:50.771150+0000","last_active":"2024-03-30T02:35:27.201629+0000","last_peered":"2024-03-30T02:35:27.201629+0000","last_clean":"2024-03-30T02:35:27.201629+0000","last_became_active":"2024-03-30T02:34:50.770703+0000","last_became_peered":"2024-03-30T02:34:50.770703+0000","last_unstale":"2024-03-30T02:35:27.201629+0000","last_undegraded":"2024-03-30T02:35:27.201629+0000","last_fullsized":"2024-03-30T02:35:27.201629+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:04.466664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211399+0000","last_change":"2024-03-30T02:34:46.203558+0000","last_active":"2024-03-30T02:35:27.211399+0000","last_peered":"2024-03-30T02:35:27.211399+0000","last_clean":"2024-03-30T02:35:27.211399+0000","last_became_active":"2024-03-30T02:34:46.202896+0000","last_became_peered":"2024-03-30T02:34:46.202896+0000","last_unstale":"2024-03-30T02:35:27.211399+0000","last_undegraded":"2024-03-30T02:35:27.211399+0000","last_fullsized":"2024-03-30T02:35:27.211399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:33.945665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.212775+0000","last_change":"2024-03-30T02:35:28.212775+0000","last_active":"2024-03-30T02:35:28.212775+0000","last_peered":"2024-03-30T02:35:28.212775+0000","last_clean":"2024-03-30T02:35:28.212775+0000","last_became_active":"2024-03-30T02:35:28.212368+0000","last_became_peered":"2024-03-30T02:35:28.212368+0000","last_unstale":"2024-03-30T02:35:28.212775+0000","last_undegraded":"2024-03-30T02:35:28.212775+0000","last_fullsized":"2024-03-30T02:35:28.212775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:50.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210436+0000","last_change":"2024-03-30T02:34:50.797758+0000","last_active":"2024-03-30T02:35:27.210436+0000","last_peered":"2024-03-30T02:35:27.210436+0000","last_clean":"2024-03-30T02:35:27.210436+0000","last_became_active":"2024-03-30T02:34:50.797311+0000","last_became_peered":"2024-03-30T02:34:50.797311+0000","last_unstale":"2024-03-30T02:35:27.210436+0000","last_undegraded":"2024-03-30T02:35:27.210436+0000","last_fullsized":"2024-03-30T02:35:27.210436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:32.337422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211691+0000","last_change":"2024-03-30T02:34:46.224397+0000","last_active":"2024-03-30T02:35:27.211691+0000","last_peered":"2024-03-30T02:35:27.211691+0000","last_clean":"2024-03-30T02:35:27.211691+0000","last_became_active":"2024-03-30T02:34:46.223825+0000","last_became_peered":"2024-03-30T02:34:46.223825+0000","last_unstale":"2024-03-30T02:35:27.211691+0000","last_undegraded":"2024-03-30T02:35:27.211691+0000","last_fullsized":"2024-03-30T02:35:27.211691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:53.154665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206934+0000","last_change":"2024-03-30T02:34:46.741358+0000","last_active":"2024-03-30T02:35:27.206934+0000","last_peered":"2024-03-30T02:35:27.206934+0000","last_clean":"2024-03-30T02:35:27.206934+0000","last_became_active":"2024-03-30T02:34:46.740734+0000","last_became_peered":"2024-03-30T02:34:46.740734+0000","last_unstale":"2024-03-30T02:35:27.206934+0000","last_undegraded":"2024-03-30T02:35:27.206934+0000","last_fullsized":"2024-03-30T02:35:27.206934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:59.678431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201960+0000","last_change":"2024-03-30T02:34:50.761998+0000","last_active":"2024-03-30T02:35:27.201960+0000","last_peered":"2024-03-30T02:35:27.201960+0000","last_clean":"2024-03-30T02:35:27.201960+0000","last_became_active":"2024-03-30T02:34:50.761560+0000","last_became_peered":"2024-03-30T02:34:50.761560+0000","last_unstale":"2024-03-30T02:35:27.201960+0000","last_undegraded":"2024-03-30T02:35:27.201960+0000","last_fullsized":"2024-03-30T02:35:27.201960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:57:17.213624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":60,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:54.186540+0000","last_change":"2024-03-30T02:34:46.213427+0000","last_active":"2024-03-30T02:35:54.186540+0000","last_peered":"2024-03-30T02:35:54.186540+0000","last_clean":"2024-03-30T02:35:54.186540+0000","last_became_active":"2024-03-30T02:34:46.213082+0000","last_became_peered":"2024-03-30T02:34:46.213082+0000","last_unstale":"2024-03-30T02:35:54.186540+0000","last_undegraded":"2024-03-30T02:35:54.186540+0000","last_fullsized":"2024-03-30T02:35:54.186540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:31:35.513723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206780+0000","last_change":"2024-03-30T02:34:46.747197+0000","last_active":"2024-03-30T02:35:27.206780+0000","last_peered":"2024-03-30T02:35:27.206780+0000","last_clean":"2024-03-30T02:35:27.206780+0000","last_became_active":"2024-03-30T02:34:46.743841+0000","last_became_peered":"2024-03-30T02:34:46.743841+0000","last_unstale":"2024-03-30T02:35:27.206780+0000","last_undegraded":"2024-03-30T02:35:27.206780+0000","last_fullsized":"2024-03-30T02:35:27.206780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:09.234382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201581+0000","last_change":"2024-03-30T02:34:50.797423+0000","last_active":"2024-03-30T02:35:27.201581+0000","last_peered":"2024-03-30T02:35:27.201581+0000","last_clean":"2024-03-30T02:35:27.201581+0000","last_became_active":"2024-03-30T02:34:50.796746+0000","last_became_peered":"2024-03-30T02:34:50.796746+0000","last_unstale":"2024-03-30T02:35:27.201581+0000","last_undegraded":"2024-03-30T02:35:27.201581+0000","last_fullsized":"2024-03-30T02:35:27.201581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:24.014131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201844+0000","last_change":"2024-03-30T02:34:46.734972+0000","last_active":"2024-03-30T02:35:27.201844+0000","last_peered":"2024-03-30T02:35:27.201844+0000","last_clean":"2024-03-30T02:35:27.201844+0000","last_became_active":"2024-03-30T02:34:46.734520+0000","last_became_peered":"2024-03-30T02:34:46.734520+0000","last_unstale":"2024-03-30T02:35:27.201844+0000","last_undegraded":"2024-03-30T02:35:27.201844+0000","last_fullsized":"2024-03-30T02:35:27.201844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:15.635060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:58.185804+0000","last_change":"2024-03-30T02:34:46.224036+0000","last_active":"2024-03-30T02:35:58.185804+0000","last_peered":"2024-03-30T02:35:58.185804+0000","last_clean":"2024-03-30T02:35:58.185804+0000","last_became_active":"2024-03-30T02:34:46.223706+0000","last_became_peered":"2024-03-30T02:34:46.223706+0000","last_unstale":"2024-03-30T02:35:58.185804+0000","last_undegraded":"2024-03-30T02:35:58.185804+0000","last_fullsized":"2024-03-30T02:35:58.185804+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:40.458345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202738+0000","last_change":"2024-03-30T02:34:50.757894+0000","last_active":"2024-03-30T02:35:27.202738+0000","last_peered":"2024-03-30T02:35:27.202738+0000","last_clean":"2024-03-30T02:35:27.202738+0000","last_became_active":"2024-03-30T02:34:50.757486+0000","last_became_peered":"2024-03-30T02:34:50.757486+0000","last_unstale":"2024-03-30T02:35:27.202738+0000","last_undegraded":"2024-03-30T02:35:27.202738+0000","last_fullsized":"2024-03-30T02:35:27.202738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:30.609851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210726+0000","last_change":"2024-03-30T02:34:46.745990+0000","last_active":"2024-03-30T02:35:27.210726+0000","last_peered":"2024-03-30T02:35:27.210726+0000","last_clean":"2024-03-30T02:35:27.210726+0000","last_became_active":"2024-03-30T02:34:46.741937+0000","last_became_peered":"2024-03-30T02:34:46.741937+0000","last_unstale":"2024-03-30T02:35:27.210726+0000","last_undegraded":"2024-03-30T02:35:27.210726+0000","last_fullsized":"2024-03-30T02:35:27.210726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:26:17.833164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206019+0000","last_change":"2024-03-30T02:34:46.208858+0000","last_active":"2024-03-30T02:35:27.206019+0000","last_peered":"2024-03-30T02:35:27.206019+0000","last_clean":"2024-03-30T02:35:27.206019+0000","last_became_active":"2024-03-30T02:34:46.208479+0000","last_became_peered":"2024-03-30T02:34:46.208479+0000","last_unstale":"2024-03-30T02:35:27.206019+0000","last_undegraded":"2024-03-30T02:35:27.206019+0000","last_fullsized":"2024-03-30T02:35:27.206019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:58:14.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":336,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202124+0000","last_change":"2024-03-30T02:33:56.208237+0000","last_active":"2024-03-30T02:35:27.202124+0000","last_peered":"2024-03-30T02:35:27.202124+0000","last_clean":"2024-03-30T02:35:27.202124+0000","last_became_active":"2024-03-30T02:33:56.207820+0000","last_became_peered":"2024-03-30T02:33:56.207820+0000","last_unstale":"2024-03-30T02:35:27.202124+0000","last_undegraded":"2024-03-30T02:35:27.202124+0000","last_fullsized":"2024-03-30T02:35:27.202124+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_clean_scrub_stamp":"2024-03-30T02:31:23.922434+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:26.198503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.213743+0000","last_change":"2024-03-30T02:35:28.213743+0000","last_active":"2024-03-30T02:35:28.213743+0000","last_peered":"2024-03-30T02:35:28.213743+0000","last_clean":"2024-03-30T02:35:28.213743+0000","last_became_active":"2024-03-30T02:35:28.213318+0000","last_became_peered":"2024-03-30T02:35:28.213318+0000","last_unstale":"2024-03-30T02:35:28.213743+0000","last_undegraded":"2024-03-30T02:35:28.213743+0000","last_fullsized":"2024-03-30T02:35:28.213743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:15:35.803168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201730+0000","last_change":"2024-03-30T02:34:46.739757+0000","last_active":"2024-03-30T02:35:27.201730+0000","last_peered":"2024-03-30T02:35:27.201730+0000","last_clean":"2024-03-30T02:35:27.201730+0000","last_became_active":"2024-03-30T02:34:46.738269+0000","last_became_peered":"2024-03-30T02:34:46.738269+0000","last_unstale":"2024-03-30T02:35:27.201730+0000","last_undegraded":"2024-03-30T02:35:27.201730+0000","last_fullsized":"2024-03-30T02:35:27.201730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:03.411187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202176+0000","last_change":"2024-03-30T02:34:46.212532+0000","last_active":"2024-03-30T02:35:27.202176+0000","last_peered":"2024-03-30T02:35:27.202176+0000","last_clean":"2024-03-30T02:35:27.202176+0000","last_became_active":"2024-03-30T02:34:46.212191+0000","last_became_peered":"2024-03-30T02:34:46.212191+0000","last_unstale":"2024-03-30T02:35:27.202176+0000","last_undegraded":"2024-03-30T02:35:27.202176+0000","last_fullsized":"2024-03-30T02:35:27.202176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:19.120960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205726+0000","last_change":"2024-03-30T02:34:50.759911+0000","last_active":"2024-03-30T02:35:27.205726+0000","last_peered":"2024-03-30T02:35:27.205726+0000","last_clean":"2024-03-30T02:35:27.205726+0000","last_became_active":"2024-03-30T02:34:50.759487+0000","last_became_peered":"2024-03-30T02:34:50.759487+0000","last_unstale":"2024-03-30T02:35:27.205726+0000","last_undegraded":"2024-03-30T02:35:27.205726+0000","last_fullsized":"2024-03-30T02:35:27.205726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:46.403754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211512+0000","last_change":"2024-03-30T02:34:46.217092+0000","last_active":"2024-03-30T02:35:27.211512+0000","last_peered":"2024-03-30T02:35:27.211512+0000","last_clean":"2024-03-30T02:35:27.211512+0000","last_became_active":"2024-03-30T02:34:46.216779+0000","last_became_peered":"2024-03-30T02:34:46.216779+0000","last_unstale":"2024-03-30T02:35:27.211512+0000","last_undegraded":"2024-03-30T02:35:27.211512+0000","last_fullsized":"2024-03-30T02:35:27.211512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:10.894577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202103+0000","last_change":"2024-03-30T02:34:46.735212+0000","last_active":"2024-03-30T02:35:27.202103+0000","last_peered":"2024-03-30T02:35:27.202103+0000","last_clean":"2024-03-30T02:35:27.202103+0000","last_became_active":"2024-03-30T02:34:46.734282+0000","last_became_peered":"2024-03-30T02:34:46.734282+0000","last_unstale":"2024-03-30T02:35:27.202103+0000","last_undegraded":"2024-03-30T02:35:27.202103+0000","last_fullsized":"2024-03-30T02:35:27.202103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:28.596961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210151+0000","last_change":"2024-03-30T02:34:50.799290+0000","last_active":"2024-03-30T02:35:27.210151+0000","last_peered":"2024-03-30T02:35:27.210151+0000","last_clean":"2024-03-30T02:35:27.210151+0000","last_became_active":"2024-03-30T02:34:50.797524+0000","last_became_peered":"2024-03-30T02:34:50.797524+0000","last_unstale":"2024-03-30T02:35:27.210151+0000","last_undegraded":"2024-03-30T02:35:27.210151+0000","last_fullsized":"2024-03-30T02:35:27.210151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:53:27.931444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.206105+0000","last_change":"2024-03-30T02:34:46.202508+0000","last_active":"2024-03-30T02:35:28.206105+0000","last_peered":"2024-03-30T02:35:28.206105+0000","last_clean":"2024-03-30T02:35:28.206105+0000","last_became_active":"2024-03-30T02:34:46.202145+0000","last_became_peered":"2024-03-30T02:34:46.202145+0000","last_unstale":"2024-03-30T02:35:28.206105+0000","last_undegraded":"2024-03-30T02:35:28.206105+0000","last_fullsized":"2024-03-30T02:35:28.206105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:19:21.138776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211176+0000","last_change":"2024-03-30T02:34:46.740056+0000","last_active":"2024-03-30T02:35:27.211176+0000","last_peered":"2024-03-30T02:35:27.211176+0000","last_clean":"2024-03-30T02:35:27.211176+0000","last_became_active":"2024-03-30T02:34:46.739260+0000","last_became_peered":"2024-03-30T02:34:46.739260+0000","last_unstale":"2024-03-30T02:35:27.211176+0000","last_undegraded":"2024-03-30T02:35:27.211176+0000","last_fullsized":"2024-03-30T02:35:27.211176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:53.550978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207567+0000","last_change":"2024-03-30T02:34:50.756851+0000","last_active":"2024-03-30T02:35:27.207567+0000","last_peered":"2024-03-30T02:35:27.207567+0000","last_clean":"2024-03-30T02:35:27.207567+0000","last_became_active":"2024-03-30T02:34:50.756416+0000","last_became_peered":"2024-03-30T02:34:50.756416+0000","last_unstale":"2024-03-30T02:35:27.207567+0000","last_undegraded":"2024-03-30T02:35:27.207567+0000","last_fullsized":"2024-03-30T02:35:27.207567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:11.469599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206997+0000","last_change":"2024-03-30T02:34:46.215663+0000","last_active":"2024-03-30T02:35:27.206997+0000","last_peered":"2024-03-30T02:35:27.206997+0000","last_clean":"2024-03-30T02:35:27.206997+0000","last_became_active":"2024-03-30T02:34:46.214224+0000","last_became_peered":"2024-03-30T02:34:46.214224+0000","last_unstale":"2024-03-30T02:35:27.206997+0000","last_undegraded":"2024-03-30T02:35:27.206997+0000","last_fullsized":"2024-03-30T02:35:27.206997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:24:58.734434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207567+0000","last_change":"2024-03-30T02:34:46.754150+0000","last_active":"2024-03-30T02:35:27.207567+0000","last_peered":"2024-03-30T02:35:27.207567+0000","last_clean":"2024-03-30T02:35:27.207567+0000","last_became_active":"2024-03-30T02:34:46.753786+0000","last_became_peered":"2024-03-30T02:34:46.753786+0000","last_unstale":"2024-03-30T02:35:27.207567+0000","last_undegraded":"2024-03-30T02:35:27.207567+0000","last_fullsized":"2024-03-30T02:35:27.207567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:54:50.165020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205116+0000","last_change":"2024-03-30T02:34:50.797890+0000","last_active":"2024-03-30T02:35:27.205116+0000","last_peered":"2024-03-30T02:35:27.205116+0000","last_clean":"2024-03-30T02:35:27.205116+0000","last_became_active":"2024-03-30T02:34:50.797395+0000","last_became_peered":"2024-03-30T02:34:50.797395+0000","last_unstale":"2024-03-30T02:35:27.205116+0000","last_undegraded":"2024-03-30T02:35:27.205116+0000","last_fullsized":"2024-03-30T02:35:27.205116+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:36:14.958804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474547+0000","last_change":"2024-03-30T02:34:46.220873+0000","last_active":"2024-03-30T02:35:27.474547+0000","last_peered":"2024-03-30T02:35:27.474547+0000","last_clean":"2024-03-30T02:35:27.474547+0000","last_became_active":"2024-03-30T02:34:46.220544+0000","last_became_peered":"2024-03-30T02:34:46.220544+0000","last_unstale":"2024-03-30T02:35:27.474547+0000","last_undegraded":"2024-03-30T02:35:27.474547+0000","last_fullsized":"2024-03-30T02:35:27.474547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:37.691602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210124+0000","last_change":"2024-03-30T02:34:46.739184+0000","last_active":"2024-03-30T02:35:27.210124+0000","last_peered":"2024-03-30T02:35:27.210124+0000","last_clean":"2024-03-30T02:35:27.210124+0000","last_became_active":"2024-03-30T02:34:46.738843+0000","last_became_peered":"2024-03-30T02:34:46.738843+0000","last_unstale":"2024-03-30T02:35:27.210124+0000","last_undegraded":"2024-03-30T02:35:27.210124+0000","last_fullsized":"2024-03-30T02:35:27.210124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:15:09.894754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211824+0000","last_change":"2024-03-30T02:34:50.768763+0000","last_active":"2024-03-30T02:35:27.211824+0000","last_peered":"2024-03-30T02:35:27.211824+0000","last_clean":"2024-03-30T02:35:27.211824+0000","last_became_active":"2024-03-30T02:34:50.767363+0000","last_became_peered":"2024-03-30T02:34:50.767363+0000","last_unstale":"2024-03-30T02:35:27.211824+0000","last_undegraded":"2024-03-30T02:35:27.211824+0000","last_fullsized":"2024-03-30T02:35:27.211824+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:06.932826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.209434+0000","last_change":"2024-03-30T02:34:46.213006+0000","last_active":"2024-03-30T02:35:27.209434+0000","last_peered":"2024-03-30T02:35:27.209434+0000","last_clean":"2024-03-30T02:35:27.209434+0000","last_became_active":"2024-03-30T02:34:46.206271+0000","last_became_peered":"2024-03-30T02:34:46.206271+0000","last_unstale":"2024-03-30T02:35:27.209434+0000","last_undegraded":"2024-03-30T02:35:27.209434+0000","last_fullsized":"2024-03-30T02:35:27.209434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:00:36.447352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206241+0000","last_change":"2024-03-30T02:34:46.739566+0000","last_active":"2024-03-30T02:35:27.206241+0000","last_peered":"2024-03-30T02:35:27.206241+0000","last_clean":"2024-03-30T02:35:27.206241+0000","last_became_active":"2024-03-30T02:34:46.739205+0000","last_became_peered":"2024-03-30T02:34:46.739205+0000","last_unstale":"2024-03-30T02:35:27.206241+0000","last_undegraded":"2024-03-30T02:35:27.206241+0000","last_fullsized":"2024-03-30T02:35:27.206241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:01.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205645+0000","last_change":"2024-03-30T02:34:50.758016+0000","last_active":"2024-03-30T02:35:27.205645+0000","last_peered":"2024-03-30T02:35:27.205645+0000","last_clean":"2024-03-30T02:35:27.205645+0000","last_became_active":"2024-03-30T02:34:50.757616+0000","last_became_peered":"2024-03-30T02:34:50.757616+0000","last_unstale":"2024-03-30T02:35:27.205645+0000","last_undegraded":"2024-03-30T02:35:27.205645+0000","last_fullsized":"2024-03-30T02:35:27.205645+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:35:07.902347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.217914+0000","last_change":"2024-03-30T02:35:28.217541+0000","last_active":"2024-03-30T02:35:28.217914+0000","last_peered":"2024-03-30T02:35:28.217914+0000","last_clean":"2024-03-30T02:35:28.217914+0000","last_became_active":"2024-03-30T02:35:28.212300+0000","last_became_peered":"2024-03-30T02:35:28.212300+0000","last_unstale":"2024-03-30T02:35:28.217914+0000","last_undegraded":"2024-03-30T02:35:28.217914+0000","last_fullsized":"2024-03-30T02:35:28.217914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:42:41.638612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.209758+0000","last_change":"2024-03-30T02:34:46.754114+0000","last_active":"2024-03-30T02:35:27.209758+0000","last_peered":"2024-03-30T02:35:27.209758+0000","last_clean":"2024-03-30T02:35:27.209758+0000","last_became_active":"2024-03-30T02:34:46.752691+0000","last_became_peered":"2024-03-30T02:34:46.752691+0000","last_unstale":"2024-03-30T02:35:27.209758+0000","last_undegraded":"2024-03-30T02:35:27.209758+0000","last_fullsized":"2024-03-30T02:35:27.209758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:42.908877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205333+0000","last_change":"2024-03-30T02:34:50.798110+0000","last_active":"2024-03-30T02:35:27.205333+0000","last_peered":"2024-03-30T02:35:27.205333+0000","last_clean":"2024-03-30T02:35:27.205333+0000","last_became_active":"2024-03-30T02:34:50.797551+0000","last_became_peered":"2024-03-30T02:34:50.797551+0000","last_unstale":"2024-03-30T02:35:27.205333+0000","last_undegraded":"2024-03-30T02:35:27.205333+0000","last_fullsized":"2024-03-30T02:35:27.205333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:49.362018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210186+0000","last_change":"2024-03-30T02:34:46.211647+0000","last_active":"2024-03-30T02:35:27.210186+0000","last_peered":"2024-03-30T02:35:27.210186+0000","last_clean":"2024-03-30T02:35:27.210186+0000","last_became_active":"2024-03-30T02:34:46.211311+0000","last_became_peered":"2024-03-30T02:34:46.211311+0000","last_unstale":"2024-03-30T02:35:27.210186+0000","last_undegraded":"2024-03-30T02:35:27.210186+0000","last_fullsized":"2024-03-30T02:35:27.210186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:05:34.135289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202501+0000","last_change":"2024-03-30T02:34:46.740256+0000","last_active":"2024-03-30T02:35:27.202501+0000","last_peered":"2024-03-30T02:35:27.202501+0000","last_clean":"2024-03-30T02:35:27.202501+0000","last_became_active":"2024-03-30T02:34:46.739913+0000","last_became_peered":"2024-03-30T02:34:46.739913+0000","last_unstale":"2024-03-30T02:35:27.202501+0000","last_undegraded":"2024-03-30T02:35:27.202501+0000","last_fullsized":"2024-03-30T02:35:27.202501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:44.152475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474682+0000","last_change":"2024-03-30T02:34:50.762149+0000","last_active":"2024-03-30T02:35:27.474682+0000","last_peered":"2024-03-30T02:35:27.474682+0000","last_clean":"2024-03-30T02:35:27.474682+0000","last_became_active":"2024-03-30T02:34:50.761717+0000","last_became_peered":"2024-03-30T02:34:50.761717+0000","last_unstale":"2024-03-30T02:35:27.474682+0000","last_undegraded":"2024-03-30T02:35:27.474682+0000","last_fullsized":"2024-03-30T02:35:27.474682+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:22.373079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:55.182522+0000","last_change":"2024-03-30T02:34:46.224098+0000","last_active":"2024-03-30T02:35:55.182522+0000","last_peered":"2024-03-30T02:35:55.182522+0000","last_clean":"2024-03-30T02:35:55.182522+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:35:55.182522+0000","last_undegraded":"2024-03-30T02:35:55.182522+0000","last_fullsized":"2024-03-30T02:35:55.182522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:20:25.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.203056+0000","last_change":"2024-03-30T02:34:46.737661+0000","last_active":"2024-03-30T02:35:27.203056+0000","last_peered":"2024-03-30T02:35:27.203056+0000","last_clean":"2024-03-30T02:35:27.203056+0000","last_became_active":"2024-03-30T02:34:46.737303+0000","last_became_peered":"2024-03-30T02:34:46.737303+0000","last_unstale":"2024-03-30T02:35:27.203056+0000","last_undegraded":"2024-03-30T02:35:27.203056+0000","last_fullsized":"2024-03-30T02:35:27.203056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:44.929643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206679+0000","last_change":"2024-03-30T02:34:50.767119+0000","last_active":"2024-03-30T02:35:27.206679+0000","last_peered":"2024-03-30T02:35:27.206679+0000","last_clean":"2024-03-30T02:35:27.206679+0000","last_became_active":"2024-03-30T02:34:50.766472+0000","last_became_peered":"2024-03-30T02:34:50.766472+0000","last_unstale":"2024-03-30T02:35:27.206679+0000","last_undegraded":"2024-03-30T02:35:27.206679+0000","last_fullsized":"2024-03-30T02:35:27.206679+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:46.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202854+0000","last_change":"2024-03-30T02:34:46.216713+0000","last_active":"2024-03-30T02:35:27.202854+0000","last_peered":"2024-03-30T02:35:27.202854+0000","last_clean":"2024-03-30T02:35:27.202854+0000","last_became_active":"2024-03-30T02:34:46.216366+0000","last_became_peered":"2024-03-30T02:34:46.216366+0000","last_unstale":"2024-03-30T02:35:27.202854+0000","last_undegraded":"2024-03-30T02:35:27.202854+0000","last_fullsized":"2024-03-30T02:35:27.202854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:43.803955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202806+0000","last_change":"2024-03-30T02:34:46.736854+0000","last_active":"2024-03-30T02:35:27.202806+0000","last_peered":"2024-03-30T02:35:27.202806+0000","last_clean":"2024-03-30T02:35:27.202806+0000","last_became_active":"2024-03-30T02:34:46.736511+0000","last_became_peered":"2024-03-30T02:34:46.736511+0000","last_unstale":"2024-03-30T02:35:27.202806+0000","last_undegraded":"2024-03-30T02:35:27.202806+0000","last_fullsized":"2024-03-30T02:35:27.202806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:12.701467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201444+0000","last_change":"2024-03-30T02:34:50.757227+0000","last_active":"2024-03-30T02:35:27.201444+0000","last_peered":"2024-03-30T02:35:27.201444+0000","last_clean":"2024-03-30T02:35:27.201444+0000","last_became_active":"2024-03-30T02:34:50.756742+0000","last_became_peered":"2024-03-30T02:34:50.756742+0000","last_unstale":"2024-03-30T02:35:27.201444+0000","last_undegraded":"2024-03-30T02:35:27.201444+0000","last_fullsized":"2024-03-30T02:35:27.201444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:48.792202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201916+0000","last_change":"2024-03-30T02:34:46.218954+0000","last_active":"2024-03-30T02:35:27.201916+0000","last_peered":"2024-03-30T02:35:27.201916+0000","last_clean":"2024-03-30T02:35:27.201916+0000","last_became_active":"2024-03-30T02:34:46.218019+0000","last_became_peered":"2024-03-30T02:34:46.218019+0000","last_unstale":"2024-03-30T02:35:27.201916+0000","last_undegraded":"2024-03-30T02:35:27.201916+0000","last_fullsized":"2024-03-30T02:35:27.201916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:03:32.651216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.202921+0000","last_change":"2024-03-30T02:34:46.741735+0000","last_active":"2024-03-30T02:35:27.202921+0000","last_peered":"2024-03-30T02:35:27.202921+0000","last_clean":"2024-03-30T02:35:27.202921+0000","last_became_active":"2024-03-30T02:34:46.740663+0000","last_became_peered":"2024-03-30T02:34:46.740663+0000","last_unstale":"2024-03-30T02:35:27.202921+0000","last_undegraded":"2024-03-30T02:35:27.202921+0000","last_fullsized":"2024-03-30T02:35:27.202921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:09.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206349+0000","last_change":"2024-03-30T02:34:46.739082+0000","last_active":"2024-03-30T02:35:27.206349+0000","last_peered":"2024-03-30T02:35:27.206349+0000","last_clean":"2024-03-30T02:35:27.206349+0000","last_became_active":"2024-03-30T02:34:46.738732+0000","last_became_peered":"2024-03-30T02:34:46.738732+0000","last_unstale":"2024-03-30T02:35:27.206349+0000","last_undegraded":"2024-03-30T02:35:27.206349+0000","last_fullsized":"2024-03-30T02:35:27.206349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:53.961520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.211005+0000","last_change":"2024-03-30T02:35:28.211005+0000","last_active":"2024-03-30T02:35:28.211005+0000","last_peered":"2024-03-30T02:35:28.211005+0000","last_clean":"2024-03-30T02:35:28.211005+0000","last_became_active":"2024-03-30T02:35:28.210610+0000","last_became_peered":"2024-03-30T02:35:28.210610+0000","last_unstale":"2024-03-30T02:35:28.211005+0000","last_undegraded":"2024-03-30T02:35:28.211005+0000","last_fullsized":"2024-03-30T02:35:28.211005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:53.054107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205930+0000","last_change":"2024-03-30T02:34:46.742590+0000","last_active":"2024-03-30T02:35:27.205930+0000","last_peered":"2024-03-30T02:35:27.205930+0000","last_clean":"2024-03-30T02:35:27.205930+0000","last_became_active":"2024-03-30T02:34:46.740904+0000","last_became_peered":"2024-03-30T02:34:46.740904+0000","last_unstale":"2024-03-30T02:35:27.205930+0000","last_undegraded":"2024-03-30T02:35:27.205930+0000","last_fullsized":"2024-03-30T02:35:27.205930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:22.653881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206792+0000","last_change":"2024-03-30T02:34:46.753119+0000","last_active":"2024-03-30T02:35:27.206792+0000","last_peered":"2024-03-30T02:35:27.206792+0000","last_clean":"2024-03-30T02:35:27.206792+0000","last_became_active":"2024-03-30T02:34:46.745680+0000","last_became_peered":"2024-03-30T02:34:46.745680+0000","last_unstale":"2024-03-30T02:35:27.206792+0000","last_undegraded":"2024-03-30T02:35:27.206792+0000","last_fullsized":"2024-03-30T02:35:27.206792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:02.352961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-30T02:35:28.313317+0000","last_change":"2024-03-30T02:35:28.313317+0000","last_active":"2024-03-30T02:35:28.313317+0000","last_peered":"2024-03-30T02:35:28.313317+0000","last_clean":"2024-03-30T02:35:28.313317+0000","last_became_active":"2024-03-30T02:35:28.312929+0000","last_became_peered":"2024-03-30T02:35:28.312929+0000","last_unstale":"2024-03-30T02:35:28.313317+0000","last_undegraded":"2024-03-30T02:35:28.313317+0000","last_fullsized":"2024-03-30T02:35:28.313317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:06.390076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.207241+0000","last_change":"2024-03-30T02:34:46.737882+0000","last_active":"2024-03-30T02:35:27.207241+0000","last_peered":"2024-03-30T02:35:27.207241+0000","last_clean":"2024-03-30T02:35:27.207241+0000","last_became_active":"2024-03-30T02:34:46.737546+0000","last_became_peered":"2024-03-30T02:34:46.737546+0000","last_unstale":"2024-03-30T02:35:27.207241+0000","last_undegraded":"2024-03-30T02:35:27.207241+0000","last_fullsized":"2024-03-30T02:35:27.207241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:48:41.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.206671+0000","last_change":"2024-03-30T02:34:46.742703+0000","last_active":"2024-03-30T02:35:27.206671+0000","last_peered":"2024-03-30T02:35:27.206671+0000","last_clean":"2024-03-30T02:35:27.206671+0000","last_became_active":"2024-03-30T02:34:46.740689+0000","last_became_peered":"2024-03-30T02:34:46.740689+0000","last_unstale":"2024-03-30T02:35:27.206671+0000","last_undegraded":"2024-03-30T02:35:27.206671+0000","last_fullsized":"2024-03-30T02:35:27.206671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:45.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.211295+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:35:27.211295+0000","last_peered":"2024-03-30T02:35:27.211295+0000","last_clean":"2024-03-30T02:35:27.211295+0000","last_became_active":"2024-03-30T02:34:46.750395+0000","last_became_peered":"2024-03-30T02:34:46.750395+0000","last_unstale":"2024-03-30T02:35:27.211295+0000","last_undegraded":"2024-03-30T02:35:27.211295+0000","last_fullsized":"2024-03-30T02:35:27.211295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:01.282677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201720+0000","last_change":"2024-03-30T02:34:46.734814+0000","last_active":"2024-03-30T02:35:27.201720+0000","last_peered":"2024-03-30T02:35:27.201720+0000","last_clean":"2024-03-30T02:35:27.201720+0000","last_became_active":"2024-03-30T02:34:46.733978+0000","last_became_peered":"2024-03-30T02:34:46.733978+0000","last_unstale":"2024-03-30T02:35:27.201720+0000","last_undegraded":"2024-03-30T02:35:27.201720+0000","last_fullsized":"2024-03-30T02:35:27.201720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:31:39.467478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.210574+0000","last_change":"2024-03-30T02:34:46.748163+0000","last_active":"2024-03-30T02:35:27.210574+0000","last_peered":"2024-03-30T02:35:27.210574+0000","last_clean":"2024-03-30T02:35:27.210574+0000","last_became_active":"2024-03-30T02:34:46.738845+0000","last_became_peered":"2024-03-30T02:34:46.738845+0000","last_unstale":"2024-03-30T02:35:27.210574+0000","last_undegraded":"2024-03-30T02:35:27.210574+0000","last_fullsized":"2024-03-30T02:35:27.210574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:09.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.474944+0000","last_change":"2024-03-30T02:34:46.743231+0000","last_active":"2024-03-30T02:35:27.474944+0000","last_peered":"2024-03-30T02:35:27.474944+0000","last_clean":"2024-03-30T02:35:27.474944+0000","last_became_active":"2024-03-30T02:34:46.742619+0000","last_became_peered":"2024-03-30T02:34:46.742619+0000","last_unstale":"2024-03-30T02:35:27.474944+0000","last_undegraded":"2024-03-30T02:35:27.474944+0000","last_fullsized":"2024-03-30T02:35:27.474944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:17:51.789139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.201627+0000","last_change":"2024-03-30T02:34:46.745872+0000","last_active":"2024-03-30T02:35:27.201627+0000","last_peered":"2024-03-30T02:35:27.201627+0000","last_clean":"2024-03-30T02:35:27.201627+0000","last_became_active":"2024-03-30T02:34:46.745476+0000","last_became_peered":"2024-03-30T02:34:46.745476+0000","last_unstale":"2024-03-30T02:35:27.201627+0000","last_undegraded":"2024-03-30T02:35:27.201627+0000","last_fullsized":"2024-03-30T02:35:27.201627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:30.774316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-30T02:35:27.205973+0000","last_change":"2024-03-30T02:34:46.746178+0000","last_active":"2024-03-30T02:35:27.205973+0000","last_peered":"2024-03-30T02:35:27.205973+0000","last_clean":"2024-03-30T02:35:27.205973+0000","last_became_active":"2024-03-30T02:34:46.745865+0000","last_became_peered":"2024-03-30T02:34:46.745865+0000","last_unstale":"2024-03-30T02:35:27.205973+0000","last_undegraded":"2024-03-30T02:35:27.205973+0000","last_fullsized":"2024-03-30T02:35:27.205973+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:22.613637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248704,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70155,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":422,"num_read_kb":294,"num_write":240,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210465,"data_compressed":5484,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579928,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":468896,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.38500000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.66800000000000004}]},{"osd":2,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.503},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":4,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.76500000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.40000000000000002}]},{"osd":6,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.67900000000000005}]},{"osd":7,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70799999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56499999999999995}]},{"osd":9,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.72799999999999998}]},{"osd":10,"last update":"Sat Mar 30 02:35:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.81299999999999994}]}]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1085440,"data_stored":1119244,"data_compressed":55433,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.441},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.498,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56799999999999995}]},{"osd":3,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.495},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61199999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.35599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.629},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":8,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.59599999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67000000000000004}]},{"osd":9,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.60799999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75800000000000001,"15min":0.76100000000000001},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76400000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.755},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81599999999999995}]}]},{"osd":9,"up_from":62,"seq":266287972384,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":472970,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56499999999999995}]},{"osd":1,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.77000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":6.774,"5min":6.774,"15min":6.774},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.85199999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.52100000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.66000000000000003}]},{"osd":3,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.76500000000000001}]},{"osd":4,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.278,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.78300000000000003}]},{"osd":5,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78200000000000003},"min":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.038,"5min":1.099,"15min":1.099},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":0.878},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3149999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.73799999999999999}]},{"osd":6,"last update":"Sat Mar 30 02:34:40 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.84199999999999997}]},{"osd":7,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.91200000000000003,"15min":0.93200000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3879999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.96199999999999997},"min":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88700000000000001,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.71399999999999997}]},{"osd":10,"last update":"Sat Mar 30 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.94499999999999995,"15min":0.95299999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84599999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76100000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.88},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.95499999999999996}]}]},{"osd":8,"up_from":55,"seq":236223201316,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":468921,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.33700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83399999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53300000000000003}]},{"osd":2,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.45300000000000001,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.32300000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.435}]},{"osd":3,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.621},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65000000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88200000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89900000000000002,"5min":0.995,"15min":0.995},"last":0.60699999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.50600000000000001,"15min":0.49099999999999999},"min":{"1min":0.40200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45900000000000002}]},{"osd":6,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.67400000000000004,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0009999999999999,"5min":1.056,"15min":1.056},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.878,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68500000000000005}]},{"osd":7,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64000000000000001,"15min":0.629},"min":{"1min":0.443,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.998,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.71499999999999997}]},{"osd":9,"last update":"Sat Mar 30 02:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.55500000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.746},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.751,"15min":0.73799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.73599999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.88700000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.69799999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92100000000000004,"5min":1.091,"15min":1.091},"last":0.79300000000000004}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76600000000000001}]}]},{"osd":7,"up_from":48,"seq":206158430247,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1065727,"data_compressed":53605,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42699999999999999,"15min":0.41999999999999998},"min":{"1min":0.35399999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.44,"15min":0.435},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.46500000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.495,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59099999999999997,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.871,"5min":1.034,"15min":1.034},"last":0.68000000000000005}]},{"osd":3,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.441,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.51000000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.72799999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52000000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.501,"15min":0.495},"min":{"1min":0.379,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.375},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.317,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70199999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.441}]},{"osd":5,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.503,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.496,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.78100000000000003}]},{"osd":6,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.56000000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.82699999999999996,"5min":0.877,"15min":0.877},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.47899999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88800000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.49199999999999999}]},{"osd":8,"last update":"Sat Mar 30 02:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67300000000000004,"15min":0.66600000000000004},"min":{"1min":0.59799999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90100000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.47999999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91400000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65100000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:35:39 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89600000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70999999999999996,"15min":0.71099999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.81100000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.623}]},{"osd":10,"last update":"Sat Mar 30 02:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63500000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.71599999999999997}]}]},{"osd":6,"up_from":44,"seq":188978561068,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":937984,"data_stored":464330,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97699999999999998,"5min":0.998,"15min":0.998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33600000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.61299999999999999}]},{"osd":2,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.069,"15min":1.069},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96199999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.74099999999999999},"min":{"1min":0.45700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1040000000000001,"5min":1.236,"15min":1.236},"last":0.441}]},{"osd":4,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":1.069,"15min":1.069},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0329999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0289999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.871},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96599999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":0.84999999999999998},"min":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.627,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.69299999999999995}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91400000000000003,"15min":0.92200000000000004},"min":{"1min":0.628,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1220000000000001,"5min":1.355,"15min":1.355},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.88600000000000001,"15min":0.89200000000000002},"min":{"1min":0.70099999999999996,"5min":0.623,"15min":0.623},"max":{"1min":1.0609999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.76000000000000001}]},{"osd":9,"last update":"Sat Mar 30 02:35:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.95299999999999996,"15min":0.96899999999999997},"min":{"1min":0.627,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0800000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.90900000000000003,"15min":0.92700000000000005},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.004,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.746}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.876,"15min":0.872},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93200000000000005,"15min":0.93100000000000005},"min":{"1min":0.745,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.24,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.80700000000000005}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.84699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":528499,"data_compressed":19433,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.66100000000000003,"15min":0.77500000000000002},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81999999999999995,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.378,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70099999999999996}]},{"osd":2,"last update":"Sat Mar 30 02:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54400000000000004,"15min":0.51400000000000001},"min":{"1min":0.46200000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85799999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.54400000000000004,"15min":0.51200000000000001},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76500000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.438,"5min":0.378,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":1.028,"15min":1.028},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.56000000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73799999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41399999999999998,"15min":0.39800000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57099999999999995,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.58899999999999997,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.60499999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.61099999999999999,"15min":0.58099999999999996},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.92300000000000004,"5min":1.026,"15min":1.026},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.55300000000000005},"min":{"1min":0.40000000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79600000000000004}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65700000000000003,"15min":0.63800000000000001},"min":{"1min":0.51600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.81100000000000005}]},{"osd":7,"last update":"Sat Mar 30 02:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74099999999999999,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.751},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.91200000000000003}]},{"osd":8,"last update":"Sat Mar 30 02:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89900000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.45400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.95599999999999996,"5min":1.044,"15min":1.044},"last":0.92600000000000005}]},{"osd":9,"last update":"Sat Mar 30 02:35:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69999999999999996,"15min":0.69899999999999995},"min":{"1min":0.58899999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.83799999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.93799999999999994,"5min":1.056,"15min":1.056},"last":0.94899999999999995}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.58499999999999996,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.872},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.48799999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.874,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.88300000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.97099999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":465786,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 02:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81599999999999995,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57299999999999995,"15min":0.54100000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86199999999999999,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57599999999999996}]},{"osd":2,"last update":"Sat Mar 30 02:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.432,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83399999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.60899999999999999,"15min":0.58999999999999997},"min":{"1min":0.44800000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.73099999999999998}]},{"osd":3,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.496}]},{"osd":4,"last update":"Sat Mar 30 02:35:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.626,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.85999999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":1.006,"15min":1.006},"last":0.65000000000000002}]},{"osd":5,"last update":"Sat Mar 30 02:35:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.65400000000000003,"15min":0.64500000000000002},"min":{"1min":0.499,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.84399999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.90200000000000002,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84299999999999997,"5min":1.091,"15min":1.091},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Sat Mar 30 02:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.54800000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.89100000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.071,"5min":1.113,"15min":1.113},"last":0.872}]},{"osd":9,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0629999999999999,"5min":1.212,"15min":1.212},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.879,"15min":0.876},"min":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.77100000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.874,"15min":0.86499999999999999},"min":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.66800000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.90200000000000002}]},{"osd":11,"last update":"Sat Mar 30 02:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.91600000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":529064,"data_compressed":19433,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.51600000000000001},"min":{"1min":0.34899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72799999999999998,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.63500000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.65900000000000003}]},{"osd":1,"last update":"Sat Mar 30 02:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.54800000000000004},"min":{"1min":0.379,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.56299999999999994,"15min":0.53300000000000003},"min":{"1min":0.44800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81000000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.73999999999999999}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84599999999999997,"5min":0.878,"15min":0.878},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.61399999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.82899999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47299999999999998}]},{"osd":6,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.58099999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85199999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49199999999999999}]},{"osd":7,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.76600000000000001}]},{"osd":8,"last update":"Sat Mar 30 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.60699999999999998,"15min":0.59399999999999997},"min":{"1min":0.49399999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90000000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.502,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52100000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.57399999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.58499999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.69999999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79200000000000004}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.72399999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":950272,"data_stored":464086,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47099999999999997,"15min":0.40699999999999997},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.434},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.76500000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.60799999999999998,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.66100000000000003,"15min":0.58499999999999996},"min":{"1min":0.47299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.51400000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.66700000000000004,"15min":0.623},"min":{"1min":0.56899999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.64800000000000002,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:35:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93999999999999995,"5min":0.996,"15min":0.996},"last":0.88200000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.55100000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0569999999999999,"5min":1.089,"15min":1.089},"last":0.81200000000000006}]},{"osd":6,"last update":"Sat Mar 30 02:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.80000000000000004,"15min":0.78000000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.877,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.47799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":1.0069999999999999}]},{"osd":7,"last update":"Sat Mar 30 02:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.997,"5min":1.071,"15min":1.071},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.89200000000000002,"15min":0.89800000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.131,"5min":1.25,"15min":1.25},"last":0.70699999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.004}]},{"osd":9,"last update":"Sat Mar 30 02:35:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.95299999999999996,"15min":0.94499999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.872,"15min":0.86599999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":1.038}]},{"osd":10,"last update":"Sat Mar 30 02:34:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.73999999999999999}]},{"osd":11,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.071},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.83499999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":946176,"data_stored":468797,"data_compressed":17605,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.38,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.747,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.36699999999999999,"15min":0.34699999999999998},"min":{"1min":0.313,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.34999999999999998},"min":{"1min":0.32400000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.39800000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.52100000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Sat Mar 30 02:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.39600000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61099999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:35:25 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59799999999999998,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78900000000000003,"5min":0.878,"15min":0.878},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58299999999999996,"15min":0.56100000000000005},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58299999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.69599999999999995}]},{"osd":7,"last update":"Sat Mar 30 02:34:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52500000000000002,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.501},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.50800000000000001,"15min":0.501},"min":{"1min":0.38900000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.53200000000000003}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.66200000000000003,"15min":0.64600000000000002},"min":{"1min":0.497,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.69099999999999995,"15min":0.67300000000000004},"min":{"1min":0.52900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.63400000000000001}]},{"osd":9,"last update":"Sat Mar 30 02:35:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.80700000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.74099999999999999}]},{"osd":11,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.77500000000000002}]}]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":1019904,"data_stored":1061285,"data_compressed":53605,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.30099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70999999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64200000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.37,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.53400000000000003,"15min":0.51800000000000002},"min":{"1min":0.48599999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78300000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.66000000000000003}]},{"osd":2,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.376,"15min":0.36099999999999999},"min":{"1min":0.31900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.379,"15min":0.35899999999999999},"min":{"1min":0.308,"5min":0.253,"15min":0.253},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.44700000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.34200000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81000000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.443,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76100000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57899999999999996}]},{"osd":4,"last update":"Sat Mar 30 02:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.55800000000000005,"5min":0.378,"15min":0.378},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60699999999999998,"15min":0.58499999999999996},"min":{"1min":0.55200000000000005,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.49199999999999999}]},{"osd":6,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623}]},{"osd":7,"last update":"Sat Mar 30 02:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.499,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.53900000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54200000000000004}]},{"osd":8,"last update":"Sat Mar 30 02:35:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.433,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55500000000000005}]},{"osd":9,"last update":"Sat Mar 30 02:35:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.69999999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.873,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.86299999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.86599999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.747,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.871,"5min":1.024,"15min":1.024},"last":0.73399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":64843,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65408,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":65220,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T02:36:02.322 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-30T02:36:02.322 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-30T02:36:02.323 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-30T02:36:02.323 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-03-30T02:36:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:02 smithi067 ceph-mon[37808]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:02 smithi170 ceph-mon[78798]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:02.693 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:02 smithi082 ceph-mon[79176]: from='client.25043 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:03.628 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:03 smithi067 ceph-mon[37808]: pgmap v273: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:36:03.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:03 smithi170 ceph-mon[78798]: pgmap v273: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:36:03.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:03 smithi082 ceph-mon[79176]: pgmap v273: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:36:03.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.860+0000 7f7f9c600700 1 -- 172.21.15.67:0/745043192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 msgr2=0x7f7f941012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:03.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.860+0000 7f7f9c600700 1 --2- 172.21.15.67:0/745043192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 0x7f7f941012e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f800099c0 tx=0x7f7f80009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:03.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.860+0000 7f7f9c600700 1 -- 172.21.15.67:0/745043192 shutdown_connections 2024-03-30T02:36:03.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.860+0000 7f7f9c600700 1 --2- 172.21.15.67:0/745043192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f94101820 0x7f7f9410dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:03.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.860+0000 7f7f9c600700 1 --2- 172.21.15.67:0/745043192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f94106ef0 0x7f7f941072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:03.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.860+0000 7f7f9c600700 1 --2- 172.21.15.67:0/745043192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 0x7f7f941012e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:03.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.860+0000 7f7f9c600700 1 -- 172.21.15.67:0/745043192 >> 172.21.15.67:0/745043192 conn(0x7f7f94077fa0 msgr2=0x7f7f940783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:03.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.860+0000 7f7f9c600700 1 -- 172.21.15.67:0/745043192 shutdown_connections 2024-03-30T02:36:03.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.860+0000 7f7f9c600700 1 -- 172.21.15.67:0/745043192 wait complete. 2024-03-30T02:36:03.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.861+0000 7f7f9c600700 1 Processor -- start 2024-03-30T02:36:03.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f9c600700 1 -- start start 2024-03-30T02:36:03.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f9c600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 0x7f7f940730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:03.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f9c600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f94101820 0x7f7f940735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:03.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f9c600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f94106ef0 0x7f7f94073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f9c600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f941a5580 con 0x7f7f94100e60 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f9c600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f941a56f0 con 0x7f7f94101820 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f9c600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f941a58b0 con 0x7f7f94106ef0 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f99c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 0x7f7f940730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f99c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 0x7f7f940730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49676/0 (socket says 172.21.15.67:49676) 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f99c00700 1 -- 172.21.15.67:0/729990051 learned_addr learned my addr 172.21.15.67:0/729990051 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f9a600700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f94106ef0 0x7f7f94073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.862+0000 7f7f99200700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f94101820 0x7f7f940735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f99c00700 1 -- 172.21.15.67:0/729990051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f94106ef0 msgr2=0x7f7f94073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f99c00700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f94106ef0 0x7f7f94073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f99c00700 1 -- 172.21.15.67:0/729990051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f94101820 msgr2=0x7f7f940735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f99c00700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f94101820 0x7f7f940735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:03.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f99c00700 1 -- 172.21.15.67:0/729990051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f800095f0 con 0x7f7f94100e60 2024-03-30T02:36:03.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f99c00700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 0x7f7f940730b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9a0 tx=0x7f7f8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:03.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f92000700 1 -- 172.21.15.67:0/729990051 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88009790 con 0x7f7f94100e60 2024-03-30T02:36:03.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f941a5b90 con 0x7f7f94100e60 2024-03-30T02:36:03.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f92000700 1 -- 172.21.15.67:0/729990051 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f88010430 con 0x7f7f94100e60 2024-03-30T02:36:03.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f92000700 1 -- 172.21.15.67:0/729990051 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88018680 con 0x7f7f94100e60 2024-03-30T02:36:03.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.863+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f941a60b0 con 0x7f7f94100e60 2024-03-30T02:36:03.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.864+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f941092f0 con 0x7f7f94100e60 2024-03-30T02:36:03.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.866+0000 7f7f92000700 1 -- 172.21.15.67:0/729990051 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7f880105a0 con 0x7f7f94100e60 2024-03-30T02:36:03.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.867+0000 7f7f92000700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f7c0a11b0 0x7f7f7c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:03.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.868+0000 7f7f92000700 1 -- 172.21.15.67:0/729990051 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f7f88014070 con 0x7f7f94100e60 2024-03-30T02:36:03.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.868+0000 7f7f99200700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f7c0a11b0 0x7f7f7c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:03.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.868+0000 7f7f99200700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f7c0a11b0 0x7f7f7c0a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009990 tx=0x7f7f80005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:03.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:03.872+0000 7f7f92000700 1 -- 172.21.15.67:0/729990051 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7f880529f0 con 0x7f7f94100e60 2024-03-30T02:36:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.132+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7f941024f0 con 0x7f7f94100e60 2024-03-30T02:36:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.132+0000 7f7f92000700 1 -- 172.21.15.67:0/729990051 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7f88096260 con 0x7f7f94100e60 2024-03-30T02:36:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:04.134 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f7c0a11b0 msgr2=0x7f7f7c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f7c0a11b0 0x7f7f7c0a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009990 tx=0x7f7f80005fd0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 msgr2=0x7f7f940730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 0x7f7f940730b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800e9a0 tx=0x7f7f8800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 shutdown_connections 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7f7c0a11b0 0x7f7f7c0a3670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f94100e60 0x7f7f940730b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f94106ef0 0x7f7f94073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 --2- 172.21.15.67:0/729990051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f94101820 0x7f7f940735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 >> 172.21.15.67:0/729990051 conn(0x7f7f94077fa0 msgr2=0x7f7f940ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 shutdown_connections 2024-03-30T02:36:04.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.135+0000 7f7f9c600700 1 -- 172.21.15.67:0/729990051 wait complete. 2024-03-30T02:36:04.496 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-30T02:36:04.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:04.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:04 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/729990051' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-30T02:36:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:04 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/729990051' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-30T02:36:04.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.699+0000 7fa6c8a00700 1 -- 172.21.15.67:0/4149524135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c4074df0 msgr2=0x7fa6c4073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:04.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.699+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/4149524135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c4074df0 0x7fa6c4073240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4009990 tx=0x7fa6b4009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.700+0000 7fa6c8a00700 1 -- 172.21.15.67:0/4149524135 shutdown_connections 2024-03-30T02:36:04.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.700+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/4149524135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c4074df0 0x7fa6c4073240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.700+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/4149524135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c410d250 0x7fa6c410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.700+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/4149524135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6c4073810 0x7fa6c4073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.700+0000 7fa6c8a00700 1 -- 172.21.15.67:0/4149524135 >> 172.21.15.67:0/4149524135 conn(0x7fa6c40fc1f0 msgr2=0x7fa6c40fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:04.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.700+0000 7fa6c8a00700 1 -- 172.21.15.67:0/4149524135 shutdown_connections 2024-03-30T02:36:04.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.700+0000 7fa6c8a00700 1 -- 172.21.15.67:0/4149524135 wait complete. 2024-03-30T02:36:04.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.701+0000 7fa6c8a00700 1 Processor -- start 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.701+0000 7fa6c8a00700 1 -- start start 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.701+0000 7fa6c8a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c4073810 0x7fa6c419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.701+0000 7fa6c8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6c4074df0 0x7fa6c419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.701+0000 7fa6c8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c410d250 0x7fa6c41a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.701+0000 7fa6c8a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41a4d70 con 0x7fa6c410d250 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.701+0000 7fa6c8a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41a4eb0 con 0x7fa6c4074df0 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.701+0000 7fa6c8a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c41a5040 con 0x7fa6c4073810 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.701+0000 7fa6c2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c410d250 0x7fa6c41a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c410d250 0x7fa6c41a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49690/0 (socket says 172.21.15.67:49690) 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c2800700 1 -- 172.21.15.67:0/1392990452 learned_addr learned my addr 172.21.15.67:0/1392990452 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c1400700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6c4074df0 0x7fa6c419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:04.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c1e00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c4073810 0x7fa6c419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c2800700 1 -- 172.21.15.67:0/1392990452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c4073810 msgr2=0x7fa6c419d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c2800700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c4073810 0x7fa6c419d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c2800700 1 -- 172.21.15.67:0/1392990452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6c4074df0 msgr2=0x7fa6c419d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c2800700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6c4074df0 0x7fa6c419d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c2800700 1 -- 172.21.15.67:0/1392990452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b40095f0 con 0x7fa6c410d250 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c2800700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c410d250 0x7fa6c41a47a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc007ee0 tx=0x7fa6bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6baa00700 1 -- 172.21.15.67:0/1392990452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc017070 con 0x7fa6c410d250 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6baa00700 1 -- 172.21.15.67:0/1392990452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6bc00f040 con 0x7fa6c410d250 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c41a5320 con 0x7fa6c410d250 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.702+0000 7fa6baa00700 1 -- 172.21.15.67:0/1392990452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc005580 con 0x7fa6c410d250 2024-03-30T02:36:04.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.703+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c41a5880 con 0x7fa6c410d250 2024-03-30T02:36:04.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.703+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c41034b0 con 0x7fa6c410d250 2024-03-30T02:36:04.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.705+0000 7fa6baa00700 1 -- 172.21.15.67:0/1392990452 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa6bc00db20 con 0x7fa6c410d250 2024-03-30T02:36:04.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.707+0000 7fa6baa00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6a80a1230 0x7fa6a80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:04.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.707+0000 7fa6baa00700 1 -- 172.21.15.67:0/1392990452 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fa6bc0cc420 con 0x7fa6c410d250 2024-03-30T02:36:04.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.707+0000 7fa6c1e00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6a80a1230 0x7fa6a80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:04.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.708+0000 7fa6c1e00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6a80a1230 0x7fa6a80a36f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b2e0 tx=0x7fa6b4005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:04.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.711+0000 7fa6baa00700 1 -- 172.21.15.67:0/1392990452 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa6bc095750 con 0x7fa6c410d250 2024-03-30T02:36:04.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:04 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/729990051' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-30T02:36:04.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.970+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c4067a10 con 0x7fa6c410d250 2024-03-30T02:36:04.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.971+0000 7fa6baa00700 1 -- 172.21.15.67:0/1392990452 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24368 (secure 0 0 0) 0x7fa6bc098d70 con 0x7fa6c410d250 2024-03-30T02:36:04.973 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:04.973 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:35:28.198711+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:04.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6a80a1230 msgr2=0x7fa6a80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:04.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6a80a1230 0x7fa6a80a36f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b2e0 tx=0x7fa6b4005fd0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c410d250 msgr2=0x7fa6c41a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:04.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c410d250 0x7fa6c41a47a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc007ee0 tx=0x7fa6bc00d3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 shutdown_connections 2024-03-30T02:36:04.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6a80a1230 0x7fa6a80a36f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa6c410d250 0x7fa6c41a47a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6c4073810 0x7fa6c419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 --2- 172.21.15.67:0/1392990452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6c4074df0 0x7fa6c419d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:04.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 >> 172.21.15.67:0/1392990452 conn(0x7fa6c40fc1f0 msgr2=0x7fa6c41005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:04.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 shutdown_connections 2024-03-30T02:36:04.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:04.974+0000 7fa6c8a00700 1 -- 172.21.15.67:0/1392990452 wait complete. 2024-03-30T02:36:04.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:36:04.995 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:36:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.197+0000 7f22c8e00700 1 -- 172.21.15.67:0/1485792984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22c4104b20 msgr2=0x7f22c410b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.197+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1485792984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22c4104b20 0x7f22c410b3d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00b0a0 tx=0x7f22bc00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.197+0000 7f22c8e00700 1 -- 172.21.15.67:0/1485792984 shutdown_connections 2024-03-30T02:36:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.197+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1485792984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c4102f90 0x7f22c41033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.197+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1485792984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22c4104b20 0x7f22c410b3d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.197+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1485792984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c4104180 0x7f22c41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.197+0000 7f22c8e00700 1 -- 172.21.15.67:0/1485792984 >> 172.21.15.67:0/1485792984 conn(0x7f22c40fe500 msgr2=0x7f22c4100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.197+0000 7f22c8e00700 1 -- 172.21.15.67:0/1485792984 shutdown_connections 2024-03-30T02:36:05.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.197+0000 7f22c8e00700 1 -- 172.21.15.67:0/1485792984 wait complete. 2024-03-30T02:36:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.198+0000 7f22c8e00700 1 Processor -- start 2024-03-30T02:36:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.198+0000 7f22c8e00700 1 -- start start 2024-03-30T02:36:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c8e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c4102f90 0x7f22c4198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c8e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22c4104180 0x7f22c41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c8e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c4104b20 0x7f22c41a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c8e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41a0ad0 con 0x7f22c4104b20 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c8e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41a0c10 con 0x7f22c4102f90 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c8e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c41a0da0 con 0x7f22c4104180 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c4104b20 0x7f22c41a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c4104b20 0x7f22c41a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49706/0 (socket says 172.21.15.67:49706) 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c2800700 1 -- 172.21.15.67:0/1363296770 learned_addr learned my addr 172.21.15.67:0/1363296770 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c1e00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c4102f90 0x7f22c4198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c1400700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22c4104180 0x7f22c41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c2800700 1 -- 172.21.15.67:0/1363296770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22c4104180 msgr2=0x7f22c41994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c2800700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22c4104180 0x7f22c41994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c2800700 1 -- 172.21.15.67:0/1363296770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c4102f90 msgr2=0x7f22c4198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c2800700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c4102f90 0x7f22c4198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.199+0000 7f22c2800700 1 -- 172.21.15.67:0/1363296770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0009520 con 0x7f22c4104b20 2024-03-30T02:36:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.200+0000 7f22c2800700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c4104b20 0x7f22c41a0500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f22bc007a90 tx=0x7f22bc00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.200+0000 7f22baa00700 1 -- 172.21.15.67:0/1363296770 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00e040 con 0x7f22c4104b20 2024-03-30T02:36:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.200+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc009d00 con 0x7f22c4104b20 2024-03-30T02:36:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.200+0000 7f22baa00700 1 -- 172.21.15.67:0/1363296770 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22bc004e60 con 0x7f22c4104b20 2024-03-30T02:36:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.200+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c41a1310 con 0x7f22c4104b20 2024-03-30T02:36:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.200+0000 7f22baa00700 1 -- 172.21.15.67:0/1363296770 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc005240 con 0x7f22c4104b20 2024-03-30T02:36:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.201+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c410a340 con 0x7f22c4104b20 2024-03-30T02:36:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.203+0000 7f22baa00700 1 -- 172.21.15.67:0/1363296770 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f22bc01d470 con 0x7f22c4104b20 2024-03-30T02:36:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.204+0000 7f22baa00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f22a80a11b0 0x7f22a80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:05.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.205+0000 7f22baa00700 1 -- 172.21.15.67:0/1363296770 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f22bc0ce130 con 0x7f22c4104b20 2024-03-30T02:36:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.205+0000 7f22c1e00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f22a80a11b0 0x7f22a80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.205+0000 7f22c1e00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f22a80a11b0 0x7f22a80a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f22c4103fe0 tx=0x7f22b4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:05.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.209+0000 7f22baa00700 1 -- 172.21.15.67:0/1363296770 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f22bc097490 con 0x7f22c4104b20 2024-03-30T02:36:05.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.430+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22c410e800 con 0x7f22c4104b20 2024-03-30T02:36:05.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.431+0000 7f22baa00700 1 -- 172.21.15.67:0/1363296770 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22bc09aab0 con 0x7f22c4104b20 2024-03-30T02:36:05.432 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:36:05.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f22a80a11b0 msgr2=0x7f22a80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f22a80a11b0 0x7f22a80a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f22c4103fe0 tx=0x7f22b4009260 comp rx=0 tx=0).stop 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c4104b20 msgr2=0x7f22c41a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c4104b20 0x7f22c41a0500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f22bc007a90 tx=0x7f22bc00bce0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 shutdown_connections 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f22a80a11b0 0x7f22a80a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c4104b20 0x7f22c41a0500 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c4102f90 0x7f22c4198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 --2- 172.21.15.67:0/1363296770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22c4104180 0x7f22c41994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.433+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 >> 172.21.15.67:0/1363296770 conn(0x7f22c40fe500 msgr2=0x7f22c4107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.434+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 shutdown_connections 2024-03-30T02:36:05.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.434+0000 7f22c8e00700 1 -- 172.21.15.67:0/1363296770 wait complete. 2024-03-30T02:36:05.453 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:36:05.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.654+0000 7f53b2400700 1 -- 172.21.15.67:0/3803675031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53ac100d80 msgr2=0x7f53ac1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.654+0000 7f53b2400700 1 --2- 172.21.15.67:0/3803675031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53ac100d80 0x7f53ac1011a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f539c009920 tx=0x7f539c009c30 comp rx=0 tx=0).stop 2024-03-30T02:36:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.654+0000 7f53b2400700 1 -- 172.21.15.67:0/3803675031 shutdown_connections 2024-03-30T02:36:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.654+0000 7f53b2400700 1 --2- 172.21.15.67:0/3803675031 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ac102910 0x7f53ac1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.654+0000 7f53b2400700 1 --2- 172.21.15.67:0/3803675031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53ac100d80 0x7f53ac1011a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.654+0000 7f53b2400700 1 --2- 172.21.15.67:0/3803675031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53ac101f70 0x7f53ac1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.654+0000 7f53b2400700 1 -- 172.21.15.67:0/3803675031 >> 172.21.15.67:0/3803675031 conn(0x7f53ac0fc2f0 msgr2=0x7f53ac0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.654+0000 7f53b2400700 1 -- 172.21.15.67:0/3803675031 shutdown_connections 2024-03-30T02:36:05.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.654+0000 7f53b2400700 1 -- 172.21.15.67:0/3803675031 wait complete. 2024-03-30T02:36:05.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.655+0000 7f53b2400700 1 Processor -- start 2024-03-30T02:36:05.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53b2400700 1 -- start start 2024-03-30T02:36:05.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53b2400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ac100d80 0x7f53ac196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53b2400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53ac101f70 0x7f53ac197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53b2400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53ac102910 0x7f53ac19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53b2400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac19e8f0 con 0x7f53ac100d80 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53b2400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac19ea60 con 0x7f53ac101f70 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53b2400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac19ec20 con 0x7f53ac102910 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53ab400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ac100d80 0x7f53ac196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53ab400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ac100d80 0x7f53ac196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49716/0 (socket says 172.21.15.67:49716) 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53ab400700 1 -- 172.21.15.67:0/4273230824 learned_addr learned my addr 172.21.15.67:0/4273230824 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53aaa00700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53ac101f70 0x7f53ac197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.656+0000 7f53abe00700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53ac102910 0x7f53ac19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53ab400700 1 -- 172.21.15.67:0/4273230824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53ac102910 msgr2=0x7f53ac19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53ab400700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53ac102910 0x7f53ac19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53ab400700 1 -- 172.21.15.67:0/4273230824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53ac101f70 msgr2=0x7f53ac197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53ab400700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53ac101f70 0x7f53ac197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53ab400700 1 -- 172.21.15.67:0/4273230824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c009520 con 0x7f53ac100d80 2024-03-30T02:36:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53ab400700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ac100d80 0x7f53ac196e50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f539c009230 tx=0x7f539c00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53a3e00700 1 -- 172.21.15.67:0/4273230824 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c01c070 con 0x7f53ac100d80 2024-03-30T02:36:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53a3e00700 1 -- 172.21.15.67:0/4273230824 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f539c00bb20 con 0x7f53ac100d80 2024-03-30T02:36:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac19eea0 con 0x7f53ac100d80 2024-03-30T02:36:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53a3e00700 1 -- 172.21.15.67:0/4273230824 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c0049c0 con 0x7f53ac100d80 2024-03-30T02:36:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.657+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac19f210 con 0x7f53ac100d80 2024-03-30T02:36:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.658+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac1082c0 con 0x7f53ac100d80 2024-03-30T02:36:05.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.660+0000 7f53a3e00700 1 -- 172.21.15.67:0/4273230824 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f539c020070 con 0x7f53ac100d80 2024-03-30T02:36:05.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.662+0000 7f53a3e00700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f53900a1230 0x7f53900a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:05.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.662+0000 7f53a3e00700 1 -- 172.21.15.67:0/4273230824 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f539c0cdd40 con 0x7f53ac100d80 2024-03-30T02:36:05.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.662+0000 7f53aaa00700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f53900a1230 0x7f53900a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:05.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.662+0000 7f53aaa00700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f53900a1230 0x7f53900a36f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5398009c60 tx=0x7f5398009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.664+0000 7f53a3e00700 1 -- 172.21.15.67:0/4273230824 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f539c0970a0 con 0x7f53ac100d80 2024-03-30T02:36:05.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:05 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1392990452' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:05.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:05 smithi170 ceph-mon[78798]: pgmap v274: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-30T02:36:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:05 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1392990452' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:05 smithi082 ceph-mon[79176]: pgmap v274: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-30T02:36:05.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.881+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53ac10c6f0 con 0x7f53ac100d80 2024-03-30T02:36:05.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.882+0000 7f53a3e00700 1 -- 172.21.15.67:0/4273230824 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f539c09a6c0 con 0x7f53ac100d80 2024-03-30T02:36:05.883 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:05.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f53900a1230 msgr2=0x7f53900a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f53900a1230 0x7f53900a36f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5398009c60 tx=0x7f5398009260 comp rx=0 tx=0).stop 2024-03-30T02:36:05.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ac100d80 msgr2=0x7f53ac196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:05.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ac100d80 0x7f53ac196e50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f539c009230 tx=0x7f539c00bef0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 shutdown_connections 2024-03-30T02:36:05.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f53900a1230 0x7f53900a36f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ac100d80 0x7f53ac196e50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53ac102910 0x7f53ac19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 --2- 172.21.15.67:0/4273230824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53ac101f70 0x7f53ac197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:05.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.884+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 >> 172.21.15.67:0/4273230824 conn(0x7f53ac0fc2f0 msgr2=0x7f53ac105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:05.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.885+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 shutdown_connections 2024-03-30T02:36:05.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:05.885+0000 7f53b2400700 1 -- 172.21.15.67:0/4273230824 wait complete. 2024-03-30T02:36:05.903 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:05 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1392990452' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:05.903 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:05 smithi067 ceph-mon[37808]: pgmap v274: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-30T02:36:05.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:36:06.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.107+0000 7f43de600700 1 -- 172.21.15.67:0/305469585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d8100d80 msgr2=0x7f43d81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.107+0000 7f43de600700 1 --2- 172.21.15.67:0/305469585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d8100d80 0x7f43d81011a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f43c80099c0 tx=0x7f43c8009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.107+0000 7f43de600700 1 -- 172.21.15.67:0/305469585 shutdown_connections 2024-03-30T02:36:06.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.107+0000 7f43de600700 1 --2- 172.21.15.67:0/305469585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d8100d80 0x7f43d81011a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.107+0000 7f43de600700 1 --2- 172.21.15.67:0/305469585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d8102910 0x7f43d81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.107+0000 7f43de600700 1 --2- 172.21.15.67:0/305469585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d8101f70 0x7f43d81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.107+0000 7f43de600700 1 -- 172.21.15.67:0/305469585 >> 172.21.15.67:0/305469585 conn(0x7f43d80fc2f0 msgr2=0x7f43d80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:06.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.107+0000 7f43de600700 1 -- 172.21.15.67:0/305469585 shutdown_connections 2024-03-30T02:36:06.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.107+0000 7f43de600700 1 -- 172.21.15.67:0/305469585 wait complete. 2024-03-30T02:36:06.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.108+0000 7f43de600700 1 Processor -- start 2024-03-30T02:36:06.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43de600700 1 -- start start 2024-03-30T02:36:06.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43de600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d8100d80 0x7f43d8198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:06.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43de600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d8101f70 0x7f43d81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:06.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43de600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d8102910 0x7f43d81a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:06.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43de600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d81a0a20 con 0x7f43d8101f70 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43de600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d81a0b60 con 0x7f43d8102910 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43de600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d81a0cf0 con 0x7f43d8100d80 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43d7400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d8100d80 0x7f43d8198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43d6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d8101f70 0x7f43d81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43d7400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d8100d80 0x7f43d8198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48362/0 (socket says 172.21.15.67:48362) 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43d7400700 1 -- 172.21.15.67:0/894816671 learned_addr learned my addr 172.21.15.67:0/894816671 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43d7e00700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d8102910 0x7f43d81a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43d7400700 1 -- 172.21.15.67:0/894816671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d8102910 msgr2=0x7f43d81a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.109+0000 7f43d7400700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d8102910 0x7f43d81a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43d7400700 1 -- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d8101f70 msgr2=0x7f43d81994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43d7400700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d8101f70 0x7f43d81994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43d7400700 1 -- 172.21.15.67:0/894816671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4009520 con 0x7f43d8100d80 2024-03-30T02:36:06.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43d6a00700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d8101f70 0x7f43d81994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:06.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43d7400700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d8100d80 0x7f43d8198f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43c8006010 tx=0x7f43c800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:06.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43cfe00700 1 -- 172.21.15.67:0/894816671 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c801d070 con 0x7f43d8100d80 2024-03-30T02:36:06.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c80095f0 con 0x7f43d8100d80 2024-03-30T02:36:06.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43cfe00700 1 -- 172.21.15.67:0/894816671 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c8005020 con 0x7f43d8100d80 2024-03-30T02:36:06.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43cfe00700 1 -- 172.21.15.67:0/894816671 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8004b10 con 0x7f43d8100d80 2024-03-30T02:36:06.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.110+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d81a11b0 con 0x7f43d8100d80 2024-03-30T02:36:06.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.111+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d8067af0 con 0x7f43d8100d80 2024-03-30T02:36:06.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.113+0000 7f43cfe00700 1 -- 172.21.15.67:0/894816671 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f43c8005190 con 0x7f43d8100d80 2024-03-30T02:36:06.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.114+0000 7f43cfe00700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f43bc0a11b0 0x7f43bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:06.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.115+0000 7f43cfe00700 1 -- 172.21.15.67:0/894816671 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f43c80cd9c0 con 0x7f43d8100d80 2024-03-30T02:36:06.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.115+0000 7f43d6a00700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f43bc0a11b0 0x7f43bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:06.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.115+0000 7f43d6a00700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f43bc0a11b0 0x7f43bc0a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009fd0 tx=0x7f43c4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:06.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.117+0000 7f43cfe00700 1 -- 172.21.15.67:0/894816671 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f43c8096d20 con 0x7f43d8100d80 2024-03-30T02:36:06.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.339+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43d810c5e0 con 0x7f43d8100d80 2024-03-30T02:36:06.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.340+0000 7f43cfe00700 1 -- 172.21.15.67:0/894816671 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43c809a340 con 0x7f43d8100d80 2024-03-30T02:36:06.341 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f43bc0a11b0 msgr2=0x7f43bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f43bc0a11b0 0x7f43bc0a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009fd0 tx=0x7f43c4009260 comp rx=0 tx=0).stop 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d8100d80 msgr2=0x7f43d8198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d8100d80 0x7f43d8198f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43c8006010 tx=0x7f43c800f690 comp rx=0 tx=0).stop 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 shutdown_connections 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f43bc0a11b0 0x7f43bc0a3670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d8102910 0x7f43d81a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d8100d80 0x7f43d8198f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 --2- 172.21.15.67:0/894816671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d8101f70 0x7f43d81994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.342+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 >> 172.21.15.67:0/894816671 conn(0x7f43d80fc2f0 msgr2=0x7f43d8072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:06.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.343+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 shutdown_connections 2024-03-30T02:36:06.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.343+0000 7f43de600700 1 -- 172.21.15.67:0/894816671 wait complete. 2024-03-30T02:36:06.363 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:36:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.564+0000 7f01f5a00700 1 -- 172.21.15.67:0/1243100533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f0100d80 msgr2=0x7f01f01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.564+0000 7f01f5a00700 1 --2- 172.21.15.67:0/1243100533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f0100d80 0x7f01f01011a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009920 tx=0x7f01e0009c30 comp rx=0 tx=0).stop 2024-03-30T02:36:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.564+0000 7f01f5a00700 1 -- 172.21.15.67:0/1243100533 shutdown_connections 2024-03-30T02:36:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.564+0000 7f01f5a00700 1 --2- 172.21.15.67:0/1243100533 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f0102910 0x7f01f01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.564+0000 7f01f5a00700 1 --2- 172.21.15.67:0/1243100533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f0100d80 0x7f01f01011a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.564+0000 7f01f5a00700 1 --2- 172.21.15.67:0/1243100533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01f0101f70 0x7f01f01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.564+0000 7f01f5a00700 1 -- 172.21.15.67:0/1243100533 >> 172.21.15.67:0/1243100533 conn(0x7f01f00fc2f0 msgr2=0x7f01f00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.564+0000 7f01f5a00700 1 -- 172.21.15.67:0/1243100533 shutdown_connections 2024-03-30T02:36:06.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.564+0000 7f01f5a00700 1 -- 172.21.15.67:0/1243100533 wait complete. 2024-03-30T02:36:06.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.565+0000 7f01f5a00700 1 Processor -- start 2024-03-30T02:36:06.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01f5a00700 1 -- start start 2024-03-30T02:36:06.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01f5a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f0100d80 0x7f01f01990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01f5a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f0101f70 0x7f01f0199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01f5a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01f0102910 0x7f01f01a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01f5a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01a0bb0 con 0x7f01f0100d80 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01f5a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01a0cf0 con 0x7f01f0101f70 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01f5a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f01a0e80 con 0x7f01f0102910 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01eea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f0100d80 0x7f01f01990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01ee000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f0101f70 0x7f01f0199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01eea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f0100d80 0x7f01f01990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57044/0 (socket says 172.21.15.67:57044) 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01eea00700 1 -- 172.21.15.67:0/3106601366 learned_addr learned my addr 172.21.15.67:0/3106601366 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.566+0000 7f01ef400700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01f0102910 0x7f01f01a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01eea00700 1 -- 172.21.15.67:0/3106601366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01f0102910 msgr2=0x7f01f01a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01eea00700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01f0102910 0x7f01f01a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01eea00700 1 -- 172.21.15.67:0/3106601366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f0101f70 msgr2=0x7f01f0199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01eea00700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f0101f70 0x7f01f0199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01eea00700 1 -- 172.21.15.67:0/3106601366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e8009d00 con 0x7f01f0100d80 2024-03-30T02:36:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01eea00700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f0100d80 0x7f01f01990d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009230 tx=0x7f01e000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01e7400700 1 -- 172.21.15.67:0/3106601366 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e001c070 con 0x7f01f0100d80 2024-03-30T02:36:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01e7400700 1 -- 172.21.15.67:0/3106601366 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01e000bb20 con 0x7f01f0100d80 2024-03-30T02:36:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e0009520 con 0x7f01f0100d80 2024-03-30T02:36:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01e7400700 1 -- 172.21.15.67:0/3106601366 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e00049c0 con 0x7f01f0100d80 2024-03-30T02:36:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.567+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f01a1390 con 0x7f01f0100d80 2024-03-30T02:36:06.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.568+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f0067af0 con 0x7f01f0100d80 2024-03-30T02:36:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.570+0000 7f01e7400700 1 -- 172.21.15.67:0/3106601366 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f01e0020070 con 0x7f01f0100d80 2024-03-30T02:36:06.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.571+0000 7f01e7400700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f01d40a1230 0x7f01d40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:06.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.571+0000 7f01e7400700 1 -- 172.21.15.67:0/3106601366 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f01e00cda00 con 0x7f01f0100d80 2024-03-30T02:36:06.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.572+0000 7f01ee000700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f01d40a1230 0x7f01d40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:06.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.572+0000 7f01ee000700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f01d40a1230 0x7f01d40a36f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009c60 tx=0x7f01dc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:06.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.576+0000 7f01e7400700 1 -- 172.21.15.67:0/3106601366 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f01e0096d60 con 0x7f01f0100d80 2024-03-30T02:36:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:06 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1363296770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:06 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4273230824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:06 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/894816671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:06 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1363296770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:06 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4273230824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:06 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/894816671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:06 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1363296770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:06 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4273230824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:06 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/894816671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:06.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.798+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f01f01a1670 con 0x7f01f0100d80 2024-03-30T02:36:06.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.799+0000 7f01e7400700 1 -- 172.21.15.67:0/3106601366 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7f01e009a380 con 0x7f01f0100d80 2024-03-30T02:36:06.800 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f01d40a1230 msgr2=0x7f01d40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f01d40a1230 0x7f01d40a36f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009c60 tx=0x7f01dc009260 comp rx=0 tx=0).stop 2024-03-30T02:36:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f0100d80 msgr2=0x7f01f01990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f0100d80 0x7f01f01990d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009230 tx=0x7f01e000bef0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 shutdown_connections 2024-03-30T02:36:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f01d40a1230 0x7f01d40a36f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01f0100d80 0x7f01f01990d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01f0102910 0x7f01f01a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 --2- 172.21.15.67:0/3106601366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f0101f70 0x7f01f0199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:06.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.801+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 >> 172.21.15.67:0/3106601366 conn(0x7f01f00fc2f0 msgr2=0x7f01f0107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:06.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.802+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 shutdown_connections 2024-03-30T02:36:06.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:06.802+0000 7f01f5a00700 1 -- 172.21.15.67:0/3106601366 wait complete. 2024-03-30T02:36:06.821 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:36:07.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.020+0000 7f3766e00700 1 -- 172.21.15.67:0/1987781772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3760101e70 msgr2=0x7f37601022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.020+0000 7f3766e00700 1 --2- 172.21.15.67:0/1987781772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3760101e70 0x7f37601022d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f374c0099c0 tx=0x7f374c009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.020+0000 7f3766e00700 1 -- 172.21.15.67:0/1987781772 shutdown_connections 2024-03-30T02:36:07.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.020+0000 7f3766e00700 1 --2- 172.21.15.67:0/1987781772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3760102810 0x7f37601090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.020+0000 7f3766e00700 1 --2- 172.21.15.67:0/1987781772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3760100c80 0x7f37601010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.020+0000 7f3766e00700 1 --2- 172.21.15.67:0/1987781772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3760101e70 0x7f37601022d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.020+0000 7f3766e00700 1 -- 172.21.15.67:0/1987781772 >> 172.21.15.67:0/1987781772 conn(0x7f37600fc1f0 msgr2=0x7f37600fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:07.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.020+0000 7f3766e00700 1 -- 172.21.15.67:0/1987781772 shutdown_connections 2024-03-30T02:36:07.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.020+0000 7f3766e00700 1 -- 172.21.15.67:0/1987781772 wait complete. 2024-03-30T02:36:07.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.021+0000 7f3766e00700 1 Processor -- start 2024-03-30T02:36:07.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.021+0000 7f3766e00700 1 -- start start 2024-03-30T02:36:07.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3766e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3760100c80 0x7f3760074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:07.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3766e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3760101e70 0x7f37600731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:07.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3766e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3760102810 0x7f37600736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3766e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3760073e80 con 0x7f3760101e70 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3766e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3760073ff0 con 0x7f3760102810 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3766e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37601a0c40 con 0x7f3760100c80 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3764e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3760102810 0x7f37600736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f375fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3760100c80 0x7f3760074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f375f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3760101e70 0x7f37600731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3764e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3760102810 0x7f37600736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38336/0 (socket says 172.21.15.67:38336) 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f375fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3760100c80 0x7f3760074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48388/0 (socket says 172.21.15.67:48388) 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3764e00700 1 -- 172.21.15.67:0/4286885944 learned_addr learned my addr 172.21.15.67:0/4286885944 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3764e00700 1 -- 172.21.15.67:0/4286885944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3760100c80 msgr2=0x7f3760074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3764e00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3760100c80 0x7f3760074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3764e00700 1 -- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3760101e70 msgr2=0x7f37600731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3764e00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3760101e70 0x7f37600731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f3764e00700 1 -- 172.21.15.67:0/4286885944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0095f0 con 0x7f3760102810 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.022+0000 7f375f400700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3760101e70 0x7f37600731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.023+0000 7f3764e00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3760102810 0x7f37600736e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3758007ee0 tx=0x7f375800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.023+0000 7f375cc00700 1 -- 172.21.15.67:0/4286885944 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758017070 con 0x7f3760102810 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.023+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37601a0ed0 con 0x7f3760102810 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.023+0000 7f375cc00700 1 -- 172.21.15.67:0/4286885944 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f375800f040 con 0x7f3760102810 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.023+0000 7f375cc00700 1 -- 172.21.15.67:0/4286885944 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37580054c0 con 0x7f3760102810 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.023+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37601a13a0 con 0x7f3760102810 2024-03-30T02:36:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.024+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37601080b0 con 0x7f3760102810 2024-03-30T02:36:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.026+0000 7f375cc00700 1 -- 172.21.15.67:0/4286885944 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3758004ad0 con 0x7f3760102810 2024-03-30T02:36:07.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.027+0000 7f375cc00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f37440a11b0 0x7f37440a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:07.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.028+0000 7f375cc00700 1 -- 172.21.15.67:0/4286885944 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f37580ccb40 con 0x7f3760102810 2024-03-30T02:36:07.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.028+0000 7f375fe00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f37440a11b0 0x7f37440a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.028+0000 7f375fe00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f37440a11b0 0x7f37440a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3760101cd0 tx=0x7f3750009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.030+0000 7f375cc00700 1 -- 172.21.15.67:0/4286885944 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3758095e10 con 0x7f3760102810 2024-03-30T02:36:07.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.296+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3760067a10 con 0x7f3760102810 2024-03-30T02:36:07.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.297+0000 7f375cc00700 1 -- 172.21.15.67:0/4286885944 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f3758099430 con 0x7f3760102810 2024-03-30T02:36:07.298 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:07.298 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:35:20.119112+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.299+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f37440a11b0 msgr2=0x7f37440a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f37440a11b0 0x7f37440a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3760101cd0 tx=0x7f3750009260 comp rx=0 tx=0).stop 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3760102810 msgr2=0x7f37600736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3760102810 0x7f37600736e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3758007ee0 tx=0x7f375800d3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 shutdown_connections 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f37440a11b0 0x7f37440a3670 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3760102810 0x7f37600736e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3760100c80 0x7f3760074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 --2- 172.21.15.67:0/4286885944 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3760101e70 0x7f37600731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 >> 172.21.15.67:0/4286885944 conn(0x7f37600fc1f0 msgr2=0x7f37601051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 shutdown_connections 2024-03-30T02:36:07.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.300+0000 7f3766e00700 1 -- 172.21.15.67:0/4286885944 wait complete. 2024-03-30T02:36:07.302 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2024-03-30T02:36:07.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:07.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.519+0000 7fb45d600700 1 -- 172.21.15.67:0/2338966919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 msgr2=0x7fb458103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.519+0000 7fb45d600700 1 --2- 172.21.15.67:0/2338966919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 0x7fb458103b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb448009990 tx=0x7fb448009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.519+0000 7fb45d600700 1 -- 172.21.15.67:0/2338966919 shutdown_connections 2024-03-30T02:36:07.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.519+0000 7fb45d600700 1 --2- 172.21.15.67:0/2338966919 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 0x7fb458103b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.519+0000 7fb45d600700 1 --2- 172.21.15.67:0/2338966919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb458106a00 0x7fb458108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.519+0000 7fb45d600700 1 --2- 172.21.15.67:0/2338966919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4581040d0 0x7fb4581064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.519+0000 7fb45d600700 1 -- 172.21.15.67:0/2338966919 >> 172.21.15.67:0/2338966919 conn(0x7fb4580fb0d0 msgr2=0x7fb4580fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:07.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.519+0000 7fb45d600700 1 -- 172.21.15.67:0/2338966919 shutdown_connections 2024-03-30T02:36:07.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.519+0000 7fb45d600700 1 -- 172.21.15.67:0/2338966919 wait complete. 2024-03-30T02:36:07.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.520+0000 7fb45d600700 1 Processor -- start 2024-03-30T02:36:07.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.520+0000 7fb45d600700 1 -- start start 2024-03-30T02:36:07.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb45d600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 0x7fb458198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:07.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb45d600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4581040d0 0x7fb458199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:07.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb45d600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb458106a00 0x7fb4581a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:07.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb45d600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a0980 con 0x7fb4581017a0 2024-03-30T02:36:07.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb45d600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a0ac0 con 0x7fb458106a00 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb45d600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a0c50 con 0x7fb4581040d0 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb456a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 0x7fb458198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb456a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 0x7fb458198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57072/0 (socket says 172.21.15.67:57072) 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb456a00700 1 -- 172.21.15.67:0/215691866 learned_addr learned my addr 172.21.15.67:0/215691866 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb457400700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb458106a00 0x7fb4581a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb456000700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4581040d0 0x7fb458199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb456a00700 1 -- 172.21.15.67:0/215691866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4581040d0 msgr2=0x7fb458199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb456a00700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4581040d0 0x7fb458199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb456a00700 1 -- 172.21.15.67:0/215691866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb458106a00 msgr2=0x7fb4581a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb456a00700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb458106a00 0x7fb4581a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.521+0000 7fb456a00700 1 -- 172.21.15.67:0/215691866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4480095f0 con 0x7fb4581017a0 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.522+0000 7fb456a00700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 0x7fb458198e50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb44800b680 tx=0x7fb448009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.522+0000 7fb44f400700 1 -- 172.21.15.67:0/215691866 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44801d070 con 0x7fb4581017a0 2024-03-30T02:36:07.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.522+0000 7fb44f400700 1 -- 172.21.15.67:0/215691866 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb448022470 con 0x7fb4581017a0 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.522+0000 7fb44f400700 1 -- 172.21.15.67:0/215691866 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448017650 con 0x7fb4581017a0 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.522+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4581a0ed0 con 0x7fb4581017a0 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.522+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4581a1240 con 0x7fb4581017a0 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.523+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb458108580 con 0x7fb4581017a0 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.525+0000 7fb44f400700 1 -- 172.21.15.67:0/215691866 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb4480225e0 con 0x7fb4581017a0 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.526+0000 7fb44f400700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb43c0a1230 0x7fb43c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.526+0000 7fb44f400700 1 -- 172.21.15.67:0/215691866 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb4480cdff0 con 0x7fb4581017a0 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.527+0000 7fb456000700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb43c0a1230 0x7fb43c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.527+0000 7fb456000700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb43c0a1230 0x7fb43c0a36f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb444009c60 tx=0x7fb444009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:07.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.530+0000 7fb44f400700 1 -- 172.21.15.67:0/215691866 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb448097370 con 0x7fb4581017a0 2024-03-30T02:36:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:07 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3106601366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:07 smithi067 ceph-mon[37808]: pgmap v275: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-30T02:36:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:07 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4286885944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:07 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3106601366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:07 smithi170 ceph-mon[78798]: pgmap v275: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-30T02:36:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:07 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4286885944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:07 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3106601366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:07 smithi082 ceph-mon[79176]: pgmap v275: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-30T02:36:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:07 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4286885944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:07.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.787+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb45810c490 con 0x7fb4581017a0 2024-03-30T02:36:07.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.788+0000 7fb44f400700 1 -- 172.21.15.67:0/215691866 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24368 (secure 0 0 0) 0x7fb44809a990 con 0x7fb4581017a0 2024-03-30T02:36:07.790 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:07.790 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:35:28.198711+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:07.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb43c0a1230 msgr2=0x7fb43c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb43c0a1230 0x7fb43c0a36f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb444009c60 tx=0x7fb444009260 comp rx=0 tx=0).stop 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 msgr2=0x7fb458198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 0x7fb458198e50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb44800b680 tx=0x7fb448009580 comp rx=0 tx=0).stop 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 shutdown_connections 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb43c0a1230 0x7fb43c0a36f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4581017a0 0x7fb458198e50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb458106a00 0x7fb4581a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 --2- 172.21.15.67:0/215691866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4581040d0 0x7fb458199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 >> 172.21.15.67:0/215691866 conn(0x7fb4580fb0d0 msgr2=0x7fb458101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.791+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 shutdown_connections 2024-03-30T02:36:07.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:07.792+0000 7fb45d600700 1 -- 172.21.15.67:0/215691866 wait complete. 2024-03-30T02:36:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:36:07.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:36:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.015+0000 7f8fb3600700 1 -- 172.21.15.67:0/4118114026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac106ae0 msgr2=0x7f8fac108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.015+0000 7f8fb3600700 1 --2- 172.21.15.67:0/4118114026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac106ae0 0x7f8fac108ed0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400b0a0 tx=0x7f8fa400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.015+0000 7f8fb3600700 1 -- 172.21.15.67:0/4118114026 shutdown_connections 2024-03-30T02:36:08.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.015+0000 7f8fb3600700 1 --2- 172.21.15.67:0/4118114026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac106ae0 0x7f8fac108ed0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.015+0000 7f8fb3600700 1 --2- 172.21.15.67:0/4118114026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fac101880 0x7f8fac103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.015+0000 7f8fb3600700 1 --2- 172.21.15.67:0/4118114026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fac1041b0 0x7f8fac1065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.015+0000 7f8fb3600700 1 -- 172.21.15.67:0/4118114026 >> 172.21.15.67:0/4118114026 conn(0x7f8fac0fb1d0 msgr2=0x7f8fac0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:08.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.015+0000 7f8fb3600700 1 -- 172.21.15.67:0/4118114026 shutdown_connections 2024-03-30T02:36:08.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.015+0000 7f8fb3600700 1 -- 172.21.15.67:0/4118114026 wait complete. 2024-03-30T02:36:08.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.016+0000 7f8fb3600700 1 Processor -- start 2024-03-30T02:36:08.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.016+0000 7f8fb3600700 1 -- start start 2024-03-30T02:36:08.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fb3600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fac101880 0x7f8fac199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fb3600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac1041b0 0x7f8fac199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fb3600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fac106ae0 0x7f8fac1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fb3600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1a0b40 con 0x7f8fac1041b0 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fb3600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1a0c80 con 0x7f8fac101880 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fb3600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1a0e10 con 0x7f8fac106ae0 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fabe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac1041b0 0x7f8fac199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fabe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac1041b0 0x7f8fac199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57094/0 (socket says 172.21.15.67:57094) 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fabe00700 1 -- 172.21.15.67:0/300555152 learned_addr learned my addr 172.21.15.67:0/300555152 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fb0c00700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fac101880 0x7f8fac199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.017+0000 7f8fb1600700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fac106ae0 0x7f8fac1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fabe00700 1 -- 172.21.15.67:0/300555152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fac106ae0 msgr2=0x7f8fac1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fabe00700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fac106ae0 0x7f8fac1a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fabe00700 1 -- 172.21.15.67:0/300555152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fac101880 msgr2=0x7f8fac199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fabe00700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fac101880 0x7f8fac199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fabe00700 1 -- 172.21.15.67:0/300555152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0009520 con 0x7f8fac1041b0 2024-03-30T02:36:08.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fb0c00700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fac101880 0x7f8fac199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:08.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fabe00700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac1041b0 0x7f8fac199550 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800e9a0 tx=0x7f8f9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:08.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fa9600700 1 -- 172.21.15.67:0/300555152 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98009790 con 0x7f8fac1041b0 2024-03-30T02:36:08.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fa9600700 1 -- 172.21.15.67:0/300555152 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f98010430 con 0x7f8fac1041b0 2024-03-30T02:36:08.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa4009d00 con 0x7f8fac1041b0 2024-03-30T02:36:08.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fa9600700 1 -- 172.21.15.67:0/300555152 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98018680 con 0x7f8fac1041b0 2024-03-30T02:36:08.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.018+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac1a14c0 con 0x7f8fac1041b0 2024-03-30T02:36:08.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.019+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac108660 con 0x7f8fac1041b0 2024-03-30T02:36:08.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.021+0000 7f8fa9600700 1 -- 172.21.15.67:0/300555152 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8f980105a0 con 0x7f8fac1041b0 2024-03-30T02:36:08.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.023+0000 7f8fa9600700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f940a1230 0x7f8f940a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.023+0000 7f8fa9600700 1 -- 172.21.15.67:0/300555152 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f8f98014070 con 0x7f8fac1041b0 2024-03-30T02:36:08.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.023+0000 7f8fb0c00700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f940a1230 0x7f8f940a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.024+0000 7f8fb0c00700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f940a1230 0x7f8f940a36f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009fd0 tx=0x7f8fa0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:08.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.026+0000 7f8fa9600700 1 -- 172.21.15.67:0/300555152 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8f980529f0 con 0x7f8fac1041b0 2024-03-30T02:36:08.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.246+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fac10c6a0 con 0x7f8fac1041b0 2024-03-30T02:36:08.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.247+0000 7f8fa9600700 1 -- 172.21.15.67:0/300555152 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f980961d0 con 0x7f8fac1041b0 2024-03-30T02:36:08.248 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:36:08.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f940a1230 msgr2=0x7f8f940a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f940a1230 0x7f8f940a36f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009fd0 tx=0x7f8fa0009260 comp rx=0 tx=0).stop 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac1041b0 msgr2=0x7f8fac199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac1041b0 0x7f8fac199550 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800e9a0 tx=0x7f8f9800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 shutdown_connections 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8f940a1230 0x7f8f940a36f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fac106ae0 0x7f8fac1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8fac101880 0x7f8fac199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 --2- 172.21.15.67:0/300555152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fac1041b0 0x7f8fac199550 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 >> 172.21.15.67:0/300555152 conn(0x7f8fac0fb1d0 msgr2=0x7f8fac1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 shutdown_connections 2024-03-30T02:36:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.250+0000 7f8fb3600700 1 -- 172.21.15.67:0/300555152 wait complete. 2024-03-30T02:36:08.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:36:08.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.472+0000 7fd34d200700 1 -- 172.21.15.67:0/2704504215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 msgr2=0x7fd348073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.472+0000 7fd34d200700 1 --2- 172.21.15.67:0/2704504215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 0x7fd348073240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd338009990 tx=0x7fd338009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.472+0000 7fd34d200700 1 -- 172.21.15.67:0/2704504215 shutdown_connections 2024-03-30T02:36:08.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.472+0000 7fd34d200700 1 --2- 172.21.15.67:0/2704504215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 0x7fd348073240 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.472+0000 7fd34d200700 1 --2- 172.21.15.67:0/2704504215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348108c40 0x7fd34810b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.472+0000 7fd34d200700 1 --2- 172.21.15.67:0/2704504215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd348073810 0x7fd348073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.472+0000 7fd34d200700 1 -- 172.21.15.67:0/2704504215 >> 172.21.15.67:0/2704504215 conn(0x7fd3480fc2f0 msgr2=0x7fd3480fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:08.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.472+0000 7fd34d200700 1 -- 172.21.15.67:0/2704504215 shutdown_connections 2024-03-30T02:36:08.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.472+0000 7fd34d200700 1 -- 172.21.15.67:0/2704504215 wait complete. 2024-03-30T02:36:08.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.473+0000 7fd34d200700 1 Processor -- start 2024-03-30T02:36:08.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.473+0000 7fd34d200700 1 -- start start 2024-03-30T02:36:08.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.473+0000 7fd34d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd348073810 0x7fd348198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.473+0000 7fd34d200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 0x7fd3481994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.473+0000 7fd34d200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348108c40 0x7fd3481a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.473+0000 7fd34d200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481a0a60 con 0x7fd348074df0 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.473+0000 7fd34d200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481a0ba0 con 0x7fd348073810 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.473+0000 7fd34d200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481a0d30 con 0x7fd348108c40 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 0x7fd3481994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 0x7fd3481994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57122/0 (socket says 172.21.15.67:57122) 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd346800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348108c40 0x7fd3481a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345400700 1 -- 172.21.15.67:0/1043858410 learned_addr learned my addr 172.21.15.67:0/1043858410 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345e00700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd348073810 0x7fd348198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345400700 1 -- 172.21.15.67:0/1043858410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348108c40 msgr2=0x7fd3481a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345400700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348108c40 0x7fd3481a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345400700 1 -- 172.21.15.67:0/1043858410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd348073810 msgr2=0x7fd348198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345400700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd348073810 0x7fd348198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345400700 1 -- 172.21.15.67:0/1043858410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380095f0 con 0x7fd348074df0 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd345400700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 0x7fd3481994c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd33400e970 tx=0x7fd33400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:08.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd33ea00700 1 -- 172.21.15.67:0/1043858410 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334009790 con 0x7fd348074df0 2024-03-30T02:36:08.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd33ea00700 1 -- 172.21.15.67:0/1043858410 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33400c7e0 con 0x7fd348074df0 2024-03-30T02:36:08.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.474+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3481a0fb0 con 0x7fd348074df0 2024-03-30T02:36:08.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.475+0000 7fd33ea00700 1 -- 172.21.15.67:0/1043858410 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334003c70 con 0x7fd348074df0 2024-03-30T02:36:08.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.475+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3481a1490 con 0x7fd348074df0 2024-03-30T02:36:08.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.476+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd348107a00 con 0x7fd348074df0 2024-03-30T02:36:08.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.478+0000 7fd33ea00700 1 -- 172.21.15.67:0/1043858410 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fd33400cc50 con 0x7fd348074df0 2024-03-30T02:36:08.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.479+0000 7fd33ea00700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd32c0a11b0 0x7fd32c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.479+0000 7fd33ea00700 1 -- 172.21.15.67:0/1043858410 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fd334014070 con 0x7fd348074df0 2024-03-30T02:36:08.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.479+0000 7fd345e00700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd32c0a11b0 0x7fd32c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.480+0000 7fd345e00700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd32c0a11b0 0x7fd32c0a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd33800b2e0 tx=0x7fd338005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:08.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.483+0000 7fd33ea00700 1 -- 172.21.15.67:0/1043858410 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd334096260 con 0x7fd348074df0 2024-03-30T02:36:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:08 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/215691866' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:08 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/300555152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:08 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/215691866' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:08 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/300555152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:08.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.699+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd34810e950 con 0x7fd348074df0 2024-03-30T02:36:08.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.699+0000 7fd33ea00700 1 -- 172.21.15.67:0/1043858410 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd334003dd0 con 0x7fd348074df0 2024-03-30T02:36:08.700 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.701+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd32c0a11b0 msgr2=0x7fd32c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.701+0000 7fd34d200700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd32c0a11b0 0x7fd32c0a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd33800b2e0 tx=0x7fd338005fb0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.701+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 msgr2=0x7fd3481994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.701+0000 7fd34d200700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 0x7fd3481994c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd33400e970 tx=0x7fd33400ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.702+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 shutdown_connections 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.702+0000 7fd34d200700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd32c0a11b0 0x7fd32c0a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.702+0000 7fd34d200700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348108c40 0x7fd3481a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.702+0000 7fd34d200700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd348073810 0x7fd348198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.702+0000 7fd34d200700 1 --2- 172.21.15.67:0/1043858410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd348074df0 0x7fd3481994c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.702+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 >> 172.21.15.67:0/1043858410 conn(0x7fd3480fc2f0 msgr2=0x7fd348104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.702+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 shutdown_connections 2024-03-30T02:36:08.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.702+0000 7fd34d200700 1 -- 172.21.15.67:0/1043858410 wait complete. 2024-03-30T02:36:08.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:36:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:08 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/215691866' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:08 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/300555152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:08.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.923+0000 7f91a2200700 1 -- 172.21.15.67:0/3991526872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f919c102910 msgr2=0x7f919c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.923+0000 7f91a2200700 1 --2- 172.21.15.67:0/3991526872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f919c102910 0x7f919c1091c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9194009f90 tx=0x7f919400b330 comp rx=0 tx=0).stop 2024-03-30T02:36:08.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.923+0000 7f91a2200700 1 -- 172.21.15.67:0/3991526872 shutdown_connections 2024-03-30T02:36:08.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.923+0000 7f91a2200700 1 --2- 172.21.15.67:0/3991526872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f919c102910 0x7f919c1091c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.923+0000 7f91a2200700 1 --2- 172.21.15.67:0/3991526872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f919c100d80 0x7f919c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.923+0000 7f91a2200700 1 --2- 172.21.15.67:0/3991526872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f919c101f70 0x7f919c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.923+0000 7f91a2200700 1 -- 172.21.15.67:0/3991526872 >> 172.21.15.67:0/3991526872 conn(0x7f919c0fc2f0 msgr2=0x7f919c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:08.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.923+0000 7f91a2200700 1 -- 172.21.15.67:0/3991526872 shutdown_connections 2024-03-30T02:36:08.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.923+0000 7f91a2200700 1 -- 172.21.15.67:0/3991526872 wait complete. 2024-03-30T02:36:08.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.924+0000 7f91a2200700 1 Processor -- start 2024-03-30T02:36:08.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f91a2200700 1 -- start start 2024-03-30T02:36:08.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f91a2200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f919c100d80 0x7f919c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f91a2200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f919c101f70 0x7f919c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f91a2200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f919c102910 0x7f919c1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f91a2200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1a0b50 con 0x7f919c101f70 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f91a2200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1a0c90 con 0x7f919c102910 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f91a2200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1a0e20 con 0x7f919c100d80 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f919aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f919c101f70 0x7f919c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f919be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f919c102910 0x7f919c1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f919b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f919c100d80 0x7f919c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f919be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f919c102910 0x7f919c1a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38396/0 (socket says 172.21.15.67:38396) 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.925+0000 7f919be00700 1 -- 172.21.15.67:0/2984442398 learned_addr learned my addr 172.21.15.67:0/2984442398 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f919be00700 1 -- 172.21.15.67:0/2984442398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f919c100d80 msgr2=0x7f919c199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f919be00700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f919c100d80 0x7f919c199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f919be00700 1 -- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f919c101f70 msgr2=0x7f919c199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f919be00700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f919c101f70 0x7f919c199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:08.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f919be00700 1 -- 172.21.15.67:0/2984442398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880095f0 con 0x7f919c102910 2024-03-30T02:36:08.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f919aa00700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f919c101f70 0x7f919c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:08.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f919be00700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f919c102910 0x7f919c1a0580 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9194009f60 tx=0x7f9194004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:08.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f9193e00700 1 -- 172.21.15.67:0/2984442398 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919400e040 con 0x7f919c102910 2024-03-30T02:36:08.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f9193e00700 1 -- 172.21.15.67:0/2984442398 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9194004e70 con 0x7f919c102910 2024-03-30T02:36:08.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9194009c40 con 0x7f919c102910 2024-03-30T02:36:08.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f9193e00700 1 -- 172.21.15.67:0/2984442398 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91940052a0 con 0x7f919c102910 2024-03-30T02:36:08.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.926+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c1a1340 con 0x7f919c102910 2024-03-30T02:36:08.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.927+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919c067af0 con 0x7f919c102910 2024-03-30T02:36:08.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.929+0000 7f9193e00700 1 -- 172.21.15.67:0/2984442398 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f919401d470 con 0x7f919c102910 2024-03-30T02:36:08.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.930+0000 7f9193e00700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91800a11b0 0x7f91800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:08.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.931+0000 7f9193e00700 1 -- 172.21.15.67:0/2984442398 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f91940ce710 con 0x7f919c102910 2024-03-30T02:36:08.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.931+0000 7f919b400700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91800a11b0 0x7f91800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:08.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.931+0000 7f919b400700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91800a11b0 0x7f91800a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f919c101dd0 tx=0x7f918c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:08.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:08.933+0000 7f9193e00700 1 -- 172.21.15.67:0/2984442398 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9194097b90 con 0x7f919c102910 2024-03-30T02:36:09.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.149+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f919c1a15a0 con 0x7f919c102910 2024-03-30T02:36:09.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.150+0000 7f9193e00700 1 -- 172.21.15.67:0/2984442398 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f919401de60 con 0x7f919c102910 2024-03-30T02:36:09.151 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:36:09.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91800a11b0 msgr2=0x7f91800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91800a11b0 0x7f91800a3670 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f919c101dd0 tx=0x7f918c009260 comp rx=0 tx=0).stop 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f919c102910 msgr2=0x7f919c1a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f919c102910 0x7f919c1a0580 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9194009f60 tx=0x7f9194004c80 comp rx=0 tx=0).stop 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 shutdown_connections 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f91800a11b0 0x7f91800a3670 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f919c102910 0x7f919c1a0580 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f919c100d80 0x7f919c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 --2- 172.21.15.67:0/2984442398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f919c101f70 0x7f919c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.152+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 >> 172.21.15.67:0/2984442398 conn(0x7f919c0fc2f0 msgr2=0x7f919c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.153+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 shutdown_connections 2024-03-30T02:36:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.153+0000 7f91a2200700 1 -- 172.21.15.67:0/2984442398 wait complete. 2024-03-30T02:36:09.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:36:09.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.372+0000 7f9a27000700 1 -- 172.21.15.67:0/1197016495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20074df0 msgr2=0x7f9a20073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.372+0000 7f9a27000700 1 --2- 172.21.15.67:0/1197016495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20074df0 0x7f9a20073240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009990 tx=0x7f9a10009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.372+0000 7f9a27000700 1 -- 172.21.15.67:0/1197016495 shutdown_connections 2024-03-30T02:36:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.372+0000 7f9a27000700 1 --2- 172.21.15.67:0/1197016495 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20074df0 0x7f9a20073240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.372+0000 7f9a27000700 1 --2- 172.21.15.67:0/1197016495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a20108c40 0x7f9a2010b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.372+0000 7f9a27000700 1 --2- 172.21.15.67:0/1197016495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a20073810 0x7f9a20073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.372+0000 7f9a27000700 1 -- 172.21.15.67:0/1197016495 >> 172.21.15.67:0/1197016495 conn(0x7f9a200fc2f0 msgr2=0x7f9a200fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.372+0000 7f9a27000700 1 -- 172.21.15.67:0/1197016495 shutdown_connections 2024-03-30T02:36:09.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.372+0000 7f9a27000700 1 -- 172.21.15.67:0/1197016495 wait complete. 2024-03-30T02:36:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.373+0000 7f9a27000700 1 Processor -- start 2024-03-30T02:36:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.373+0000 7f9a27000700 1 -- start start 2024-03-30T02:36:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.373+0000 7f9a27000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a20073810 0x7f9a20198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.373+0000 7f9a27000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a20074df0 0x7f9a201994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a27000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20108c40 0x7f9a201a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a27000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a201a0a50 con 0x7f9a20108c40 2024-03-30T02:36:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a27000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a201a0b90 con 0x7f9a20073810 2024-03-30T02:36:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a27000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a201a0d20 con 0x7f9a20074df0 2024-03-30T02:36:09.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a25000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20108c40 0x7f9a201a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a1fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a20073810 0x7f9a20198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a25000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20108c40 0x7f9a201a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57170/0 (socket says 172.21.15.67:57170) 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a25000700 1 -- 172.21.15.67:0/1811643949 learned_addr learned my addr 172.21.15.67:0/1811643949 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a1f400700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a20074df0 0x7f9a201994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a25000700 1 -- 172.21.15.67:0/1811643949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a20074df0 msgr2=0x7f9a201994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a25000700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a20074df0 0x7f9a201994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a25000700 1 -- 172.21.15.67:0/1811643949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a20073810 msgr2=0x7f9a20198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a25000700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a20073810 0x7f9a20198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a25000700 1 -- 172.21.15.67:0/1811643949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c009520 con 0x7f9a20108c40 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.374+0000 7f9a25000700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20108c40 0x7f9a201a0480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800ef30 tx=0x7f9a1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.375+0000 7f9a1cc00700 1 -- 172.21.15.67:0/1811643949 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18018070 con 0x7f9a20108c40 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.375+0000 7f9a1cc00700 1 -- 172.21.15.67:0/1811643949 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1800f040 con 0x7f9a20108c40 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.375+0000 7f9a1cc00700 1 -- 172.21.15.67:0/1811643949 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800cd00 con 0x7f9a20108c40 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.375+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a100095f0 con 0x7f9a20108c40 2024-03-30T02:36:09.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.375+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a201a1240 con 0x7f9a20108c40 2024-03-30T02:36:09.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.376+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a20107a00 con 0x7f9a20108c40 2024-03-30T02:36:09.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.378+0000 7f9a1cc00700 1 -- 172.21.15.67:0/1811643949 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9a18003a00 con 0x7f9a20108c40 2024-03-30T02:36:09.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.379+0000 7f9a1cc00700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9a040a1230 0x7f9a040a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:09.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.379+0000 7f9a1cc00700 1 -- 172.21.15.67:0/1811643949 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9a180cc840 con 0x7f9a20108c40 2024-03-30T02:36:09.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.380+0000 7f9a1fe00700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9a040a1230 0x7f9a040a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:09.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.380+0000 7f9a1fe00700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9a040a1230 0x7f9a040a36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b2e0 tx=0x7f9a10005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:09.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.384+0000 7f9a1cc00700 1 -- 172.21.15.67:0/1811643949 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9a18095ba0 con 0x7f9a20108c40 2024-03-30T02:36:09.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.600+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9a2010e7c0 con 0x7f9a20108c40 2024-03-30T02:36:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.600+0000 7f9a1cc00700 1 -- 172.21.15.67:0/1811643949 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9a180991c0 con 0x7f9a20108c40 2024-03-30T02:36:09.601 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:09.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.602+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9a040a1230 msgr2=0x7f9a040a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.602+0000 7f9a27000700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9a040a1230 0x7f9a040a36f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b2e0 tx=0x7f9a10005fd0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.602+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20108c40 msgr2=0x7f9a201a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.602+0000 7f9a27000700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20108c40 0x7f9a201a0480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800ef30 tx=0x7f9a1800c3a0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.602+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 shutdown_connections 2024-03-30T02:36:09.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.603+0000 7f9a27000700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9a040a1230 0x7f9a040a36f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.603+0000 7f9a27000700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a20108c40 0x7f9a201a0480 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.603+0000 7f9a27000700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a20073810 0x7f9a20198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.603+0000 7f9a27000700 1 --2- 172.21.15.67:0/1811643949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a20074df0 0x7f9a201994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.603+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 >> 172.21.15.67:0/1811643949 conn(0x7f9a200fc2f0 msgr2=0x7f9a20104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:09.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.603+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 shutdown_connections 2024-03-30T02:36:09.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.603+0000 7f9a27000700 1 -- 172.21.15.67:0/1811643949 wait complete. 2024-03-30T02:36:09.622 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:36:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:09 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1043858410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:09 smithi170 ceph-mon[78798]: pgmap v276: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-30T02:36:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:09 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2984442398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:09 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1043858410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:09 smithi082 ceph-mon[79176]: pgmap v276: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-30T02:36:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:09 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2984442398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:09.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.819+0000 7f66d4e00700 1 -- 172.21.15.67:0/1391726642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d0100c80 msgr2=0x7f66d01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.819+0000 7f66d4e00700 1 --2- 172.21.15.67:0/1391726642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d0100c80 0x7f66d01010a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f66c00099c0 tx=0x7f66c0009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.819+0000 7f66d4e00700 1 -- 172.21.15.67:0/1391726642 shutdown_connections 2024-03-30T02:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.819+0000 7f66d4e00700 1 --2- 172.21.15.67:0/1391726642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d0100c80 0x7f66d01010a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.819+0000 7f66d4e00700 1 --2- 172.21.15.67:0/1391726642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d0102810 0x7f66d01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.819+0000 7f66d4e00700 1 --2- 172.21.15.67:0/1391726642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d0101e70 0x7f66d01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.819+0000 7f66d4e00700 1 -- 172.21.15.67:0/1391726642 >> 172.21.15.67:0/1391726642 conn(0x7f66d00fc1f0 msgr2=0x7f66d00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.819+0000 7f66d4e00700 1 -- 172.21.15.67:0/1391726642 shutdown_connections 2024-03-30T02:36:09.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.819+0000 7f66d4e00700 1 -- 172.21.15.67:0/1391726642 wait complete. 2024-03-30T02:36:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.820+0000 7f66d4e00700 1 Processor -- start 2024-03-30T02:36:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66d4e00700 1 -- start start 2024-03-30T02:36:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66d4e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d0100c80 0x7f66d0198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66d4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d0101e70 0x7f66d0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66d4e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d0102810 0x7f66d01a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:09.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66d4e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d01a0980 con 0x7f66d0102810 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66d4e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d01a0ac0 con 0x7f66d0101e70 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66d4e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d01a0c50 con 0x7f66d0100c80 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66cde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d0100c80 0x7f66d0198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66ce800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d0102810 0x7f66d01a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66cde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d0100c80 0x7f66d0198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48508/0 (socket says 172.21.15.67:48508) 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66cde00700 1 -- 172.21.15.67:0/4108034171 learned_addr learned my addr 172.21.15.67:0/4108034171 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.821+0000 7f66cd400700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d0101e70 0x7f66d0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66cde00700 1 -- 172.21.15.67:0/4108034171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d0101e70 msgr2=0x7f66d0199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66cde00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d0101e70 0x7f66d0199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66cde00700 1 -- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d0102810 msgr2=0x7f66d01a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66cde00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d0102810 0x7f66d01a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66cde00700 1 -- 172.21.15.67:0/4108034171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00095f0 con 0x7f66d0100c80 2024-03-30T02:36:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66ce800700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d0102810 0x7f66d01a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66cde00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d0100c80 0x7f66d0198e50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f66c0003730 tx=0x7f66c000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66c6a00700 1 -- 172.21.15.67:0/4108034171 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c001d070 con 0x7f66d0100c80 2024-03-30T02:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66c6a00700 1 -- 172.21.15.67:0/4108034171 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66c000bda0 con 0x7f66d0100c80 2024-03-30T02:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d01a0ed0 con 0x7f66d0100c80 2024-03-30T02:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66c6a00700 1 -- 172.21.15.67:0/4108034171 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0004b30 con 0x7f66d0100c80 2024-03-30T02:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.822+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d01a1240 con 0x7f66d0100c80 2024-03-30T02:36:09.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.823+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d0067a80 con 0x7f66d0100c80 2024-03-30T02:36:09.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.825+0000 7f66c6a00700 1 -- 172.21.15.67:0/4108034171 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f66c0004c90 con 0x7f66d0100c80 2024-03-30T02:36:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.827+0000 7f66c6a00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f66b40a11b0 0x7f66b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:09.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.827+0000 7f66c6a00700 1 -- 172.21.15.67:0/4108034171 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f66c00cde10 con 0x7f66d0100c80 2024-03-30T02:36:09.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.827+0000 7f66cd400700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f66b40a11b0 0x7f66b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:09.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.827+0000 7f66cd400700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f66b40a11b0 0x7f66b40a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009c60 tx=0x7f66bc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:09.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:09.829+0000 7f66c6a00700 1 -- 172.21.15.67:0/4108034171 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f66c0097170 con 0x7f66d0100c80 2024-03-30T02:36:09.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:09 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1043858410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:09.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:09 smithi067 ceph-mon[37808]: pgmap v276: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-30T02:36:09.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:09 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2984442398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:10.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.095+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66d0067a10 con 0x7f66d0100c80 2024-03-30T02:36:10.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.096+0000 7f66c6a00700 1 -- 172.21.15.67:0/4108034171 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f66c009a790 con 0x7f66d0100c80 2024-03-30T02:36:10.098 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:10.098 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:35:20.119112+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:36:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f66b40a11b0 msgr2=0x7f66b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f66b40a11b0 0x7f66b40a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009c60 tx=0x7f66bc009260 comp rx=0 tx=0).stop 2024-03-30T02:36:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d0100c80 msgr2=0x7f66d0198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d0100c80 0x7f66d0198e50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f66c0003730 tx=0x7f66c000fa00 comp rx=0 tx=0).stop 2024-03-30T02:36:10.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 shutdown_connections 2024-03-30T02:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f66b40a11b0 0x7f66b40a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66d0102810 0x7f66d01a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66d0100c80 0x7f66d0198e50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 --2- 172.21.15.67:0/4108034171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d0101e70 0x7f66d0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 >> 172.21.15.67:0/4108034171 conn(0x7f66d00fc1f0 msgr2=0x7f66d0107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 shutdown_connections 2024-03-30T02:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.099+0000 7f66d4e00700 1 -- 172.21.15.67:0/4108034171 wait complete. 2024-03-30T02:36:10.101 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2024-03-30T02:36:10.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.318+0000 7f1659000700 1 -- 172.21.15.67:0/4138858775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 msgr2=0x7f16541022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.318+0000 7f1659000700 1 --2- 172.21.15.67:0/4138858775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 0x7f16541022d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f16400099c0 tx=0x7f1640009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.318+0000 7f1659000700 1 -- 172.21.15.67:0/4138858775 shutdown_connections 2024-03-30T02:36:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.318+0000 7f1659000700 1 --2- 172.21.15.67:0/4138858775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1654100c80 0x7f16541010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.318+0000 7f1659000700 1 --2- 172.21.15.67:0/4138858775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1654102810 0x7f16541090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.318+0000 7f1659000700 1 --2- 172.21.15.67:0/4138858775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 0x7f16541022d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.318+0000 7f1659000700 1 -- 172.21.15.67:0/4138858775 >> 172.21.15.67:0/4138858775 conn(0x7f16540fc1f0 msgr2=0x7f16540fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.318+0000 7f1659000700 1 -- 172.21.15.67:0/4138858775 shutdown_connections 2024-03-30T02:36:10.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.318+0000 7f1659000700 1 -- 172.21.15.67:0/4138858775 wait complete. 2024-03-30T02:36:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.319+0000 7f1659000700 1 Processor -- start 2024-03-30T02:36:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.319+0000 7f1659000700 1 -- start start 2024-03-30T02:36:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1659000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1654100c80 0x7f1654196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1659000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 0x7f1654197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1659000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1654102810 0x7f165419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1659000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165419e730 con 0x7f1654100c80 2024-03-30T02:36:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1659000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165419e870 con 0x7f1654101e70 2024-03-30T02:36:10.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1659000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165419ea00 con 0x7f1654102810 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1654100c80 0x7f1654196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 0x7f1654197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 0x7f1654197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38444/0 (socket says 172.21.15.67:38444) 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651400700 1 -- 172.21.15.67:0/2011761753 learned_addr learned my addr 172.21.15.67:0/2011761753 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1654100c80 0x7f1654196c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57216/0 (socket says 172.21.15.67:57216) 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1652800700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1654102810 0x7f165419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651400700 1 -- 172.21.15.67:0/2011761753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1654102810 msgr2=0x7f165419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651400700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1654102810 0x7f165419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651400700 1 -- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1654100c80 msgr2=0x7f1654196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651400700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1654100c80 0x7f1654196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651400700 1 -- 172.21.15.67:0/2011761753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16400095f0 con 0x7f1654101e70 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.320+0000 7f1651e00700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1654100c80 0x7f1654196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.321+0000 7f1651400700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 0x7f1654197140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1640006010 tx=0x7f164000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.321+0000 7f164aa00700 1 -- 172.21.15.67:0/2011761753 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164001d070 con 0x7f1654101e70 2024-03-30T02:36:10.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.321+0000 7f164aa00700 1 -- 172.21.15.67:0/2011761753 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1640005030 con 0x7f1654101e70 2024-03-30T02:36:10.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.321+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165419ec80 con 0x7f1654101e70 2024-03-30T02:36:10.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.321+0000 7f164aa00700 1 -- 172.21.15.67:0/2011761753 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640004b10 con 0x7f1654101e70 2024-03-30T02:36:10.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.321+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165419f0a0 con 0x7f1654101e70 2024-03-30T02:36:10.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.322+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1654067a80 con 0x7f1654101e70 2024-03-30T02:36:10.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.324+0000 7f164aa00700 1 -- 172.21.15.67:0/2011761753 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f16400051a0 con 0x7f1654101e70 2024-03-30T02:36:10.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.325+0000 7f164aa00700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16380a11b0 0x7f16380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:10.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.325+0000 7f164aa00700 1 -- 172.21.15.67:0/2011761753 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f16400cdc30 con 0x7f1654101e70 2024-03-30T02:36:10.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.326+0000 7f1651e00700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16380a11b0 0x7f16380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:10.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.326+0000 7f1651e00700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16380a11b0 0x7f16380a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1654101cd0 tx=0x7f1644009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:10.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.328+0000 7f164aa00700 1 -- 172.21.15.67:0/2011761753 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1640096f90 con 0x7f1654101e70 2024-03-30T02:36:10.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.581+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f165410c4e0 con 0x7f1654101e70 2024-03-30T02:36:10.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.583+0000 7f164aa00700 1 -- 172.21.15.67:0/2011761753 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+24368 (secure 0 0 0) 0x7f164009a5b0 con 0x7f1654101e70 2024-03-30T02:36:10.584 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:10.585 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":85,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:35:28.198711+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16380a11b0 msgr2=0x7f16380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16380a11b0 0x7f16380a3670 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1654101cd0 tx=0x7f1644009260 comp rx=0 tx=0).stop 2024-03-30T02:36:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 msgr2=0x7f1654197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 0x7f1654197140 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1640006010 tx=0x7f164000f690 comp rx=0 tx=0).stop 2024-03-30T02:36:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 shutdown_connections 2024-03-30T02:36:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f16380a11b0 0x7f16380a3670 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1654100c80 0x7f1654196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1654102810 0x7f165419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 --2- 172.21.15.67:0/2011761753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1654101e70 0x7f1654197140 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 >> 172.21.15.67:0/2011761753 conn(0x7f16540fc1f0 msgr2=0x7f1654105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 shutdown_connections 2024-03-30T02:36:10.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.586+0000 7f1659000700 1 -- 172.21.15.67:0/2011761753 wait complete. 2024-03-30T02:36:10.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:36:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:10 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1811643949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:10 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4108034171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:10 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1811643949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:10 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4108034171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:10.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.807+0000 7f1857800700 1 -- 172.21.15.67:0/1210291744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1850104010 msgr2=0x7f1850106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.807+0000 7f1857800700 1 --2- 172.21.15.67:0/1210291744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1850104010 0x7f1850106400 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f183c0098f0 tx=0x7f183c009c00 comp rx=0 tx=0).stop 2024-03-30T02:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.807+0000 7f1857800700 1 -- 172.21.15.67:0/1210291744 shutdown_connections 2024-03-30T02:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.807+0000 7f1857800700 1 --2- 172.21.15.67:0/1210291744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1850106940 0x7f1850108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.807+0000 7f1857800700 1 --2- 172.21.15.67:0/1210291744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18501016e0 0x7f1850103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.807+0000 7f1857800700 1 --2- 172.21.15.67:0/1210291744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1850104010 0x7f1850106400 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.807+0000 7f1857800700 1 -- 172.21.15.67:0/1210291744 >> 172.21.15.67:0/1210291744 conn(0x7f18500fb0d0 msgr2=0x7f18500fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.807+0000 7f1857800700 1 -- 172.21.15.67:0/1210291744 shutdown_connections 2024-03-30T02:36:10.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.807+0000 7f1857800700 1 -- 172.21.15.67:0/1210291744 wait complete. 2024-03-30T02:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.808+0000 7f1857800700 1 Processor -- start 2024-03-30T02:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.808+0000 7f1857800700 1 -- start start 2024-03-30T02:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1857800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18501016e0 0x7f1850198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1857800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1850104010 0x7f1850199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1857800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1850106940 0x7f18501a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1857800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18501a0970 con 0x7f1850106940 2024-03-30T02:36:10.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1857800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18501a0ab0 con 0x7f18501016e0 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1857800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18501a0c40 con 0x7f1850104010 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1854e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18501016e0 0x7f1850198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1854e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18501016e0 0x7f1850198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38464/0 (socket says 172.21.15.67:38464) 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1854e00700 1 -- 172.21.15.67:0/4268472185 learned_addr learned my addr 172.21.15.67:0/4268472185 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f184fe00700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1850104010 0x7f1850199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1855800700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1850106940 0x7f18501a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1854e00700 1 -- 172.21.15.67:0/4268472185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1850104010 msgr2=0x7f1850199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1854e00700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1850104010 0x7f1850199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1854e00700 1 -- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1850106940 msgr2=0x7f18501a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1854e00700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1850106940 0x7f18501a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1854e00700 1 -- 172.21.15.67:0/4268472185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c009520 con 0x7f18501016e0 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.809+0000 7f1855800700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1850106940 0x7f18501a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.810+0000 7f1854e00700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18501016e0 0x7f1850198e40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9a0 tx=0x7f184400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:10.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.810+0000 7f184d600700 1 -- 172.21.15.67:0/4268472185 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844009790 con 0x7f18501016e0 2024-03-30T02:36:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.810+0000 7f184d600700 1 -- 172.21.15.67:0/4268472185 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f184400cbd0 con 0x7f18501016e0 2024-03-30T02:36:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.810+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18501a0f20 con 0x7f18501016e0 2024-03-30T02:36:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.810+0000 7f184d600700 1 -- 172.21.15.67:0/4268472185 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844018680 con 0x7f18501016e0 2024-03-30T02:36:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.810+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18501a1340 con 0x7f18501016e0 2024-03-30T02:36:10.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.811+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1850108510 con 0x7f18501016e0 2024-03-30T02:36:10.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.813+0000 7f184d600700 1 -- 172.21.15.67:0/4268472185 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f184400cd40 con 0x7f18501016e0 2024-03-30T02:36:10.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.814+0000 7f184d600700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18380a11b0 0x7f18380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:10.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.814+0000 7f184d600700 1 -- 172.21.15.67:0/4268472185 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1844014070 con 0x7f18501016e0 2024-03-30T02:36:10.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.814+0000 7f184fe00700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18380a11b0 0x7f18380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:10.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.815+0000 7f184fe00700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18380a11b0 0x7f18380a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f183c00fa40 tx=0x7f183c019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:10.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:10.817+0000 7f184d600700 1 -- 172.21.15.67:0/4268472185 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f18440965e0 con 0x7f18501016e0 2024-03-30T02:36:10.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:10 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1811643949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:10.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:10 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4108034171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:11.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.083+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f185010c780 con 0x7f18501016e0 2024-03-30T02:36:11.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.084+0000 7f184d600700 1 -- 172.21.15.67:0/4268472185 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f18440b1ab0 con 0x7f18501016e0 2024-03-30T02:36:11.085 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:11.085 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:35:20.119112+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:36:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18380a11b0 msgr2=0x7f18380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18380a11b0 0x7f18380a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f183c00fa40 tx=0x7f183c019040 comp rx=0 tx=0).stop 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18501016e0 msgr2=0x7f1850198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18501016e0 0x7f1850198e40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9a0 tx=0x7f184400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 shutdown_connections 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f18380a11b0 0x7f18380a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1850106940 0x7f18501a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18501016e0 0x7f1850198e40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 --2- 172.21.15.67:0/4268472185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1850104010 0x7f1850199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 >> 172.21.15.67:0/4268472185 conn(0x7f18500fb0d0 msgr2=0x7f1850100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 shutdown_connections 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.086+0000 7f1857800700 1 -- 172.21.15.67:0/4268472185 wait complete. 2024-03-30T02:36:11.088 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2024-03-30T02:36:11.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-30T02:34:46.713132+0000', 'modified': '2024-03-30T02:35:20.119112+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24578, 'mds_1': 24560, 'mds_2': 24541, 'mds_3': 24539, 'mds_4': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14526': {'gid': 14526, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6841/853027854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6840', 'nonce': 853027854}, {'type': 'v1', 'addr': '172.21.15.67:6841', 'nonce': 853027854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24466': {'gid': 24466, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/3749225376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 3749225376}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 3749225376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24476': {'gid': 24476, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6833/1365991767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6832', 'nonce': 1365991767}, {'type': 'v1', 'addr': '172.21.15.82:6833', 'nonce': 1365991767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/3401372629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 3401372629}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 3401372629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/1685909437', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 1685909437}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 1685909437}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.170:6837/500476759', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 500476759}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 500476759}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.82:6837/1183051781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 1183051781}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 1183051781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.170:6839/2570108618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 2570108618}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 2570108618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24560': {'gid': 24560, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.67:6839/2974132199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6838', 'nonce': 2974132199}, {'type': 'v1', 'addr': '172.21.15.67:6839', 'nonce': 2974132199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6839/604601206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 604601206}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 604601206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T02:36:11.107 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-30T02:36:11.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T02:36:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.306+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2658299461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 msgr2=0x7f3dc41091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.306+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2658299461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 0x7f3dc41091c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00b0d0 tx=0x7f3dbc00b3e0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.306+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2658299461 shutdown_connections 2024-03-30T02:36:11.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.306+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2658299461 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 0x7f3dc41091c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.306+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2658299461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4100d80 0x7f3dc41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.306+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2658299461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc4101f70 0x7f3dc41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.306+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2658299461 >> 172.21.15.67:0/2658299461 conn(0x7f3dc40fc2f0 msgr2=0x7f3dc40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.306+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2658299461 shutdown_connections 2024-03-30T02:36:11.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.306+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2658299461 wait complete. 2024-03-30T02:36:11.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.307+0000 7f3dc8e00700 1 Processor -- start 2024-03-30T02:36:11.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc8e00700 1 -- start start 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc8e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4100d80 0x7f3dc4199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc8e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc4101f70 0x7f3dc41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc8e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 0x7f3dc41a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc8e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc41a0b50 con 0x7f3dc4102910 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc8e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc41a0c90 con 0x7f3dc4100d80 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc8e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc41a0e20 con 0x7f3dc4101f70 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 0x7f3dc41a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4100d80 0x7f3dc4199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 0x7f3dc41a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57248/0 (socket says 172.21.15.67:57248) 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc2800700 1 -- 172.21.15.67:0/2499101769 learned_addr learned my addr 172.21.15.67:0/2499101769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.308+0000 7f3dc1400700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc4101f70 0x7f3dc41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dc2800700 1 -- 172.21.15.67:0/2499101769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc4101f70 msgr2=0x7f3dc41995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dc2800700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc4101f70 0x7f3dc41995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dc2800700 1 -- 172.21.15.67:0/2499101769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4100d80 msgr2=0x7f3dc4199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dc2800700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4100d80 0x7f3dc4199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dc2800700 1 -- 172.21.15.67:0/2499101769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc009d00 con 0x7f3dc4102910 2024-03-30T02:36:11.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dc1e00700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4100d80 0x7f3dc4199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:11.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dc2800700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 0x7f3dc41a0580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc007b60 tx=0x7f3dbc008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dbaa00700 1 -- 172.21.15.67:0/2499101769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00e040 con 0x7f3dc4102910 2024-03-30T02:36:11.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dbaa00700 1 -- 172.21.15.67:0/2499101769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dbc01d930 con 0x7f3dc4102910 2024-03-30T02:36:11.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2499101769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc41a10a0 con 0x7f3dc4102910 2024-03-30T02:36:11.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dbaa00700 1 -- 172.21.15.67:0/2499101769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc01b9f0 con 0x7f3dc4102910 2024-03-30T02:36:11.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.309+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2499101769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc41a14d0 con 0x7f3dc4102910 2024-03-30T02:36:11.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.314+0000 7f3dbaa00700 1 -- 172.21.15.67:0/2499101769 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3dbc01daa0 con 0x7f3dc4102910 2024-03-30T02:36:11.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3ddc985b80 1 Processor -- start 2024-03-30T02:36:11.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3ddc985b80 1 -- start start 2024-03-30T02:36:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3dbaa00700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3da80a11b0 0x7f3da80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x55560147fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 0x5556014809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601481400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556013d7940 con 0x555601480f80 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556013d5300 con 0x5556014800e0 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556013d0bf0 con 0x55560147f780 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3dbaa00700 1 -- 172.21.15.67:0/2499101769 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3dbc0ce2c0 con 0x7f3dc4102910 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3dc1400700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3da80a11b0 0x7f3da80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3dc2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601481400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3dc1e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x55560147fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3dc1e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x55560147fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48566/0 (socket says 172.21.15.67:48566) 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.315+0000 7f3dc1e00700 1 -- 172.21.15.67:0/3278659588 learned_addr learned my addr 172.21.15.67:0/3278659588 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3dc1400700 1 --2- 172.21.15.67:0/3278659588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 0x5556014809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3dc2800700 1 -- 172.21.15.67:0/3278659588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 msgr2=0x55560147fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3dc2800700 1 --2- 172.21.15.67:0/3278659588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x55560147fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3dc2800700 1 -- 172.21.15.67:0/3278659588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 msgr2=0x5556014809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3dc2800700 1 --2- 172.21.15.67:0/3278659588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 0x5556014809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3dc2800700 1 -- 172.21.15.67:0/3278659588 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555601252360 con 0x555601480f80 2024-03-30T02:36:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3dc2800700 1 --2- 172.21.15.67:0/3278659588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601481400 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0cb970 tx=0x7f3dbc09b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d9afde4becbd64 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3dc1400700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3da80a11b0 0x7f3da80a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009fd0 tx=0x7f3db0009b40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3daf400700 1 -- 172.21.15.67:0/3278659588 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc09aca0 con 0x555601480f80 2024-03-30T02:36:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3daf400700 1 -- 172.21.15.67:0/3278659588 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dbc001d00 con 0x555601480f80 2024-03-30T02:36:11.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.316+0000 7f3daf400700 1 -- 172.21.15.67:0/3278659588 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0a3730 con 0x555601480f80 2024-03-30T02:36:11.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.317+0000 7f3ddc985b80 1 -- 172.21.15.67:0/3278659588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 msgr2=0x555601481400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.317+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/3278659588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601481400 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0cb970 tx=0x7f3dbc09b110 comp rx=0 tx=0).stop 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.317+0000 7f3ddc985b80 1 -- 172.21.15.67:0/3278659588 shutdown_connections 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.317+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/3278659588 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601481400 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.317+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/3278659588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x55560147fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.317+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/3278659588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 0x5556014809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.317+0000 7f3ddc985b80 1 -- 172.21.15.67:0/3278659588 >> 172.21.15.67:0/3278659588 conn(0x5556013de650 msgr2=0x55560147a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.317+0000 7f3ddc985b80 1 -- 172.21.15.67:0/3278659588 shutdown_connections 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.317+0000 7f3ddc985b80 1 -- 172.21.15.67:0/3278659588 wait complete. 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.318+0000 7f3ddc985b80 1 -- start start 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.318+0000 7f3ddc985b80 1 Processor -- start 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.318+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x555601489410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.318+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 0x555601489950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.318+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601490990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.318+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560136cd30 con 0x555601480f80 2024-03-30T02:36:11.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.318+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560136c9d0 con 0x5556014800e0 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.318+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555601254650 con 0x55560147f780 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3ddc985b80 10 client.0 _ll_register_callbacks cb 0x555601376420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601490990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601490990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57268/0 (socket says 172.21.15.67:57268) 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc2800700 1 -- 172.21.15.67:0/148901456 learned_addr learned my addr 172.21.15.67:0/148901456 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc1e00700 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x555601489410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc1400700 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 0x555601489950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc2800700 1 -- 172.21.15.67:0/148901456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 msgr2=0x555601489410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc2800700 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x555601489410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc2800700 1 -- 172.21.15.67:0/148901456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 msgr2=0x555601489950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc2800700 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 0x555601489950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc2800700 1 -- 172.21.15.67:0/148901456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc41a14d0 con 0x555601480f80 2024-03-30T02:36:11.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3dc2800700 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601490990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc09b040 tx=0x7f3dbc0cbdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98297556b309d7c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.319+0000 7f3da3e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3da3e00700 1 -- 172.21.15.67:0/148901456 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc09aca0 con 0x555601480f80 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3da3e00700 1 -- 172.21.15.67:0/148901456 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3dbc001b20 con 0x555601480f80 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3ddc985b80 1 -- 172.21.15.67:0/148901456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555601391330 con 0x555601480f80 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3da3e00700 1 -- 172.21.15.67:0/148901456 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dbc09ba40 con 0x555601480f80 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3ddc985b80 10 client.15096 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3ddc985b80 1 -- 172.21.15.67:0/148901456 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5556011feb10 con 0x555601480f80 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3da3e00700 1 -- 172.21.15.67:0/148901456 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dbc0b7080 con 0x555601480f80 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3da3e00700 1 -- 172.21.15.67:0/148901456 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3dbc09b5a0 con 0x555601480f80 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3ddc985b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3ddc985b80 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3ddc985b80 20 client.15096 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555601494070 0x5556014927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3ddc985b80 4 client.15096 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.320+0000 7f3ddc985b80 1 -- 172.21.15.67:0/148901456 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556013c96b0 con 0x555601494070 2024-03-30T02:36:11.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.321+0000 7f3dc1e00700 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555601494070 0x5556014927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.321+0000 7f3dc1e00700 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555601494070 0x5556014927e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c88de4c29527b12 server_cookie=afd31319d5cc950f in_seq=0 out_seq=0 2024-03-30T02:36:11.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.321+0000 7f3da3e00700 10 client.15096 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.322+0000 7f3da3e00700 1 -- 172.21.15.67:0/148901456 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5556013c96b0 con 0x555601494070 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.322+0000 7f3da3e00700 10 client.15096 handle_command_reply: tid=0 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.322+0000 7f3ddc985b80 1 client.15096 shutdown 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.322+0000 7f3ddc985b80 1 -- 172.21.15.67:0/148901456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 msgr2=0x555601490990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.322+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601490990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc09b040 tx=0x7f3dbc0cbdf0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3ddc985b80 1 -- 172.21.15.67:0/148901456 shutdown_connections 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x555601490990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555601494070 0x5556014927e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x555601489410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/148901456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556014800e0 0x555601489950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3ddc985b80 1 -- 172.21.15.67:0/148901456 >> 172.21.15.67:0/148901456 conn(0x5556013de650 msgr2=0x55560147e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3da3e00700 0 client.15096 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:36:11.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3ddc985b80 1 -- 172.21.15.67:0/148901456 shutdown_connections 2024-03-30T02:36:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3ddc985b80 1 -- 172.21.15.67:0/148901456 wait complete. 2024-03-30T02:36:11.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.323+0000 7f3ddc985b80 20 client.15096 trim_cache size 0 max 16384 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.341+0000 7f3ddc985b80 1 Processor -- start 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3ddc985b80 1 -- start start 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x55560147fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556014800e0 0x555601488920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555601480f80 0x555601490950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556013c96b0 con 0x5556014800e0 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55560147f420 con 0x555601480f80 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555601491b30 con 0x55560147f780 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3dc1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556014800e0 0x555601488920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3dc1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556014800e0 0x555601488920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57278/0 (socket says 172.21.15.67:57278) 2024-03-30T02:36:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3dc1400700 1 -- 172.21.15.67:0/220240848 learned_addr learned my addr 172.21.15.67:0/220240848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3dc1e00700 1 --2- 172.21.15.67:0/220240848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x55560147fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.342+0000 7f3dc2800700 1 --2- 172.21.15.67:0/220240848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555601480f80 0x555601490950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.343+0000 7f3dc1400700 1 -- 172.21.15.67:0/220240848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 msgr2=0x55560147fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.343+0000 7f3dc1400700 1 --2- 172.21.15.67:0/220240848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x55560147fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.343+0000 7f3dc1400700 1 -- 172.21.15.67:0/220240848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555601480f80 msgr2=0x555601490950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.343+0000 7f3dc1400700 1 --2- 172.21.15.67:0/220240848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555601480f80 0x555601490950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.343+0000 7f3dc1400700 1 -- 172.21.15.67:0/220240848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555601396e50 con 0x5556014800e0 2024-03-30T02:36:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.343+0000 7f3dc1400700 1 --2- 172.21.15.67:0/220240848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556014800e0 0x555601488920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3db0013210 tx=0x7f3db0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3090f6963b6baa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.343+0000 7f3dafe00700 1 -- 172.21.15.67:0/220240848 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0026070 con 0x5556014800e0 2024-03-30T02:36:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.343+0000 7f3dafe00700 1 -- 172.21.15.67:0/220240848 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3db0013240 con 0x5556014800e0 2024-03-30T02:36:11.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.343+0000 7f3dafe00700 1 -- 172.21.15.67:0/220240848 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0003b30 con 0x5556014800e0 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 -- 172.21.15.67:0/220240848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556014800e0 msgr2=0x555601488920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/220240848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556014800e0 0x555601488920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3db0013210 tx=0x7f3db0009260 comp rx=0 tx=0).stop 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 -- 172.21.15.67:0/220240848 shutdown_connections 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/220240848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555601480f80 0x555601490950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/220240848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55560147f780 0x55560147fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/220240848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5556014800e0 0x555601488920 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 -- 172.21.15.67:0/220240848 >> 172.21.15.67:0/220240848 conn(0x5556013de650 msgr2=0x555601473550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 -- 172.21.15.67:0/220240848 shutdown_connections 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 -- 172.21.15.67:0/220240848 wait complete. 2024-03-30T02:36:11.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.344+0000 7f3ddc985b80 1 -- start start 2024-03-30T02:36:11.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3ddc985b80 1 Processor -- start 2024-03-30T02:36:11.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55560147f780 0x55560148cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556014800e0 0x55560148d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3ddc985b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x55560148fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555601490210 con 0x555601480f80 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555601490380 con 0x55560147f780 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3ddc985b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555601490540 con 0x5556014800e0 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3ddc985b80 10 client.0 _ll_register_callbacks cb 0x555601376420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3dc1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556014800e0 0x55560148d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.345+0000 7f3dc2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x55560148fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556014800e0 0x55560148d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48598/0 (socket says 172.21.15.67:48598) 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc1400700 1 -- 172.21.15.67:0/2867047082 learned_addr learned my addr 172.21.15.67:0/2867047082 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc1e00700 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55560147f780 0x55560148cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc1400700 1 -- 172.21.15.67:0/2867047082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55560147f780 msgr2=0x55560148cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc1400700 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55560147f780 0x55560148cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc1400700 1 -- 172.21.15.67:0/2867047082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 msgr2=0x55560148fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc1400700 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x55560148fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc1400700 1 -- 172.21.15.67:0/2867047082 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db0013e70 con 0x5556014800e0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc2800700 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x55560148fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3da3e00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3dc1400700 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556014800e0 0x55560148d0c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009920 tx=0x7f3db0003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc9f0c112951a3e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3da3e00700 1 -- 172.21.15.67:0/2867047082 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0026040 con 0x5556014800e0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3ddc985b80 1 -- 172.21.15.67:0/2867047082 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dbc001d00 con 0x5556014800e0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.346+0000 7f3da3e00700 1 -- 172.21.15.67:0/2867047082 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3db0011a60 con 0x5556014800e0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3da3e00700 1 -- 172.21.15.67:0/2867047082 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3db0003e80 con 0x5556014800e0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3ddc985b80 10 client.25144 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3ddc985b80 1 -- 172.21.15.67:0/2867047082 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555601256c80 con 0x5556014800e0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3da3e00700 1 -- 172.21.15.67:0/2867047082 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dbc001d00 con 0x5556014800e0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3da3e00700 1 -- 172.21.15.67:0/2867047082 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3db0032020 con 0x5556014800e0 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3ddc985b80 10 client.25144 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3ddc985b80 10 client.25144 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T02:36:11.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3ddc985b80 20 client.25144 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:11.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5556014ed8c0 0x5556014ea280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3ddc985b80 4 client.25144 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:11.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.347+0000 7f3ddc985b80 1 -- 172.21.15.67:0/2867047082 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55560121fca0 con 0x5556014ed8c0 2024-03-30T02:36:11.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.348+0000 7f3dc1e00700 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5556014ed8c0 0x5556014ea280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.348+0000 7f3dc1e00700 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5556014ed8c0 0x5556014ea280 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=14a9ddde65a354aa server_cookie=a0ff7f81481e1a9 in_seq=0 out_seq=0 2024-03-30T02:36:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.348+0000 7f3da3e00700 10 client.25144 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:36:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.348+0000 7f3da3e00700 1 -- 172.21.15.67:0/2867047082 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55560121fca0 con 0x5556014ed8c0 2024-03-30T02:36:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.348+0000 7f3da3e00700 10 client.25144 handle_command_reply: tid=0 2024-03-30T02:36:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.348+0000 7f3ddc985b80 1 client.25144 shutdown 2024-03-30T02:36:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 -- 172.21.15.67:0/2867047082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556014800e0 msgr2=0x55560148d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556014800e0 0x55560148d0c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009920 tx=0x7f3db0003c60 comp rx=0 tx=0).stop 2024-03-30T02:36:11.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 -- 172.21.15.67:0/2867047082 shutdown_connections 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555601480f80 0x55560148fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5556014ed8c0 0x5556014ea280 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55560147f780 0x55560148cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 --2- 172.21.15.67:0/2867047082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5556014800e0 0x55560148d0c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3da3e00700 0 client.25144 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 -- 172.21.15.67:0/2867047082 >> 172.21.15.67:0/2867047082 conn(0x5556013de650 msgr2=0x555601487480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 -- 172.21.15.67:0/2867047082 shutdown_connections 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 1 -- 172.21.15.67:0/2867047082 wait complete. 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.349+0000 7f3ddc985b80 20 client.25144 trim_cache size 0 max 16384 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 4, 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24520, 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 52.236217267999997, 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 85, 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 100.07551234500001 2024-03-30T02:36:11.354 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3da80a11b0 msgr2=0x7f3da80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3da80a11b0 0x7f3da80a3670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009fd0 tx=0x7f3db0009b40 comp rx=0 tx=0).stop 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 msgr2=0x7f3dc41a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 0x7f3dc41a0580 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc007b60 tx=0x7f3dbc008e40 comp rx=0 tx=0).stop 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2499101769 shutdown_connections 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3da80a11b0 0x7f3da80a3670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3dc4102910 0x7f3dc41a0580 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3dc4100d80 0x7f3dc4199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 --2- 172.21.15.67:0/2499101769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dc4101f70 0x7f3dc41995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2499101769 >> 172.21.15.67:0/2499101769 conn(0x7f3dc40fc2f0 msgr2=0x7f3dc4072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2499101769 shutdown_connections 2024-03-30T02:36:11.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.350+0000 7f3dc8e00700 1 -- 172.21.15.67:0/2499101769 wait complete. 2024-03-30T02:36:11.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-30T02:36:11.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.569+0000 7fcf17200700 1 -- 172.21.15.67:0/1678877308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 msgr2=0x7fcf18100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.569+0000 7fcf17200700 1 --2- 172.21.15.67:0/1678877308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 0x7fcf18100fe0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf080099c0 tx=0x7fcf08009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.569+0000 7fcf17200700 1 -- 172.21.15.67:0/1678877308 shutdown_connections 2024-03-30T02:36:11.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.569+0000 7fcf17200700 1 --2- 172.21.15.67:0/1678877308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 0x7fcf18100fe0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.569+0000 7fcf17200700 1 --2- 172.21.15.67:0/1678877308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf18102750 0x7fcf18109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.569+0000 7fcf17200700 1 --2- 172.21.15.67:0/1678877308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf18101db0 0x7fcf18102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.569+0000 7fcf17200700 1 -- 172.21.15.67:0/1678877308 >> 172.21.15.67:0/1678877308 conn(0x7fcf180fc110 msgr2=0x7fcf180fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.569+0000 7fcf17200700 1 -- 172.21.15.67:0/1678877308 shutdown_connections 2024-03-30T02:36:11.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.569+0000 7fcf17200700 1 -- 172.21.15.67:0/1678877308 wait complete. 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.570+0000 7fcf17200700 1 Processor -- start 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.570+0000 7fcf17200700 1 -- start start 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.570+0000 7fcf17200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 0x7fcf18198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.570+0000 7fcf17200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf18101db0 0x7fcf181992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf17200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf18102750 0x7fcf181a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf17200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181a08c0 con 0x7fcf18100bc0 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf17200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181a0a00 con 0x7fcf18102750 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf17200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181a0b90 con 0x7fcf18101db0 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf15e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 0x7fcf18198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf15e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 0x7fcf18198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57304/0 (socket says 172.21.15.67:57304) 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf15e00700 1 -- 172.21.15.67:0/4087069844 learned_addr learned my addr 172.21.15.67:0/4087069844 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf15400700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf18101db0 0x7fcf181992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf16800700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf18102750 0x7fcf181a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf15e00700 1 -- 172.21.15.67:0/4087069844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf18101db0 msgr2=0x7fcf181992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf15e00700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf18101db0 0x7fcf181992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf15e00700 1 -- 172.21.15.67:0/4087069844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf18102750 msgr2=0x7fcf181a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf15e00700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf18102750 0x7fcf181a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.571+0000 7fcf15e00700 1 -- 172.21.15.67:0/4087069844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10009c40 con 0x7fcf18100bc0 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.572+0000 7fcf15e00700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 0x7fcf18198d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcf08003730 tx=0x7fcf0800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.572+0000 7fcf0ea00700 1 -- 172.21.15.67:0/4087069844 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0801d070 con 0x7fcf18100bc0 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.572+0000 7fcf0ea00700 1 -- 172.21.15.67:0/4087069844 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf08005050 con 0x7fcf18100bc0 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.572+0000 7fcf17200700 1 -- 172.21.15.67:0/4087069844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf080095f0 con 0x7fcf18100bc0 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.572+0000 7fcf0ea00700 1 -- 172.21.15.67:0/4087069844 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08004a30 con 0x7fcf18100bc0 2024-03-30T02:36:11.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.572+0000 7fcf17200700 1 -- 172.21.15.67:0/4087069844 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf181a1100 con 0x7fcf18100bc0 2024-03-30T02:36:11.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.575+0000 7fcf0ea00700 1 -- 172.21.15.67:0/4087069844 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fcf080051c0 con 0x7fcf18100bc0 2024-03-30T02:36:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.576+0000 7fcf0ea00700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcefc0a1230 0x7fcefc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.576+0000 7fcf30b89b80 1 Processor -- start 2024-03-30T02:36:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.576+0000 7fcf0ea00700 1 -- 172.21.15.67:0/4087069844 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcf080cda90 con 0x7fcf18100bc0 2024-03-30T02:36:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.576+0000 7fcf15400700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcefc0a1230 0x7fcefc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf30b89b80 1 -- start start 2024-03-30T02:36:11.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd623db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd624b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd6255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd57b940 con 0x563afd6242f0 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd579300 con 0x563afd623990 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd574bf0 con 0x563afd625140 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf15400700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcefc0a1230 0x7fcefc0a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009c60 tx=0x7fcf04009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf15400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd624b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf16800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd6255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf15e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd623db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf15400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd624b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57312/0 (socket says 172.21.15.67:57312) 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf16800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd6255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48622/0 (socket says 172.21.15.67:48622) 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.577+0000 7fcf15400700 1 -- 172.21.15.67:0/934217838 learned_addr learned my addr 172.21.15.67:0/934217838 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf15e00700 1 -- 172.21.15.67:0/934217838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 msgr2=0x563afd6255c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf15e00700 1 --2- 172.21.15.67:0/934217838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd6255c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf15e00700 1 -- 172.21.15.67:0/934217838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 msgr2=0x563afd624b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf15e00700 1 --2- 172.21.15.67:0/934217838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd624b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf15e00700 1 -- 172.21.15.67:0/934217838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563afd3f6360 con 0x563afd623990 2024-03-30T02:36:11.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf15400700 1 --2- 172.21.15.67:0/934217838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd624b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:11.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf16800700 1 --2- 172.21.15.67:0/934217838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd6255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf15e00700 1 --2- 172.21.15.67:0/934217838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd623db0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf080cb0e0 tx=0x7fcf0809b1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fb2ed7bb9418933 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf03400700 1 -- 172.21.15.67:0/934217838 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf080a2ea0 con 0x563afd623990 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf03400700 1 -- 172.21.15.67:0/934217838 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf080cb330 con 0x563afd623990 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.578+0000 7fcf03400700 1 -- 172.21.15.67:0/934217838 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0809a200 con 0x563afd623990 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.579+0000 7fcf30b89b80 1 -- 172.21.15.67:0/934217838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 msgr2=0x563afd623db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.579+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/934217838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd623db0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf080cb0e0 tx=0x7fcf0809b1b0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.579+0000 7fcf30b89b80 1 -- 172.21.15.67:0/934217838 shutdown_connections 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.579+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/934217838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd6255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.579+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/934217838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd623db0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.579+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/934217838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd624b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.579+0000 7fcf30b89b80 1 -- 172.21.15.67:0/934217838 >> 172.21.15.67:0/934217838 conn(0x563afd582650 msgr2=0x563afd61ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.579+0000 7fcf30b89b80 1 -- 172.21.15.67:0/934217838 shutdown_connections 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.579+0000 7fcf30b89b80 1 -- 172.21.15.67:0/934217838 wait complete. 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.580+0000 7fcf30b89b80 1 -- start start 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.580+0000 7fcf30b89b80 1 Processor -- start 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.580+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd62d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.580+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd6242f0 0x563afd62da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd625140 0x563afd634a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd510d30 con 0x563afd625140 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd5109d0 con 0x563afd623990 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd3f8650 con 0x563afd6242f0 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf30b89b80 10 client.0 _ll_register_callbacks cb 0x563afd51a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd62d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd62d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38564/0 (socket says 172.21.15.67:38564) 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15e00700 1 -- 172.21.15.67:0/2514013610 learned_addr learned my addr 172.21.15.67:0/2514013610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15400700 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd6242f0 0x563afd62da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf16800700 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd625140 0x563afd634a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15e00700 1 -- 172.21.15.67:0/2514013610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd6242f0 msgr2=0x563afd62da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15e00700 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd6242f0 0x563afd62da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15e00700 1 -- 172.21.15.67:0/2514013610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd625140 msgr2=0x563afd634a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15e00700 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd625140 0x563afd634a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15e00700 1 -- 172.21.15.67:0/2514013610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1000e330 con 0x563afd623990 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf16800700 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd625140 0x563afd634a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.581+0000 7fcf15400700 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd6242f0 0x563afd62da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.582+0000 7fcf15e00700 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd62d4c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009990 tx=0x7fcf080a2cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2589a20647ef99e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.582+0000 7fcef7e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.582+0000 7fcef7e00700 1 -- 172.21.15.67:0/2514013610 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0809ad10 con 0x563afd623990 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.582+0000 7fcef7e00700 1 -- 172.21.15.67:0/2514013610 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcf080a2ea0 con 0x563afd623990 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.582+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2514013610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563afd535330 con 0x563afd623990 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.582+0000 7fcef7e00700 1 -- 172.21.15.67:0/2514013610 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf080b3380 con 0x563afd623990 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.582+0000 7fcf30b89b80 10 client.25163 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:11.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.582+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2514013610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fcf181a1100 con 0x563afd623990 2024-03-30T02:36:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.582+0000 7fcef7e00700 1 -- 172.21.15.67:0/2514013610 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563afd535330 con 0x563afd623990 2024-03-30T02:36:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.583+0000 7fcef7e00700 1 -- 172.21.15.67:0/2514013610 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf080968a0 con 0x563afd623990 2024-03-30T02:36:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.583+0000 7fcf30b89b80 10 client.25163 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.583+0000 7fcf30b89b80 10 client.25163 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-30T02:36:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.583+0000 7fcf30b89b80 20 client.25163 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.583+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563afd6381f0 0x563afd63d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.583+0000 7fcf30b89b80 4 client.25163 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.583+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2514013610 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563afd56d6b0 con 0x563afd6381f0 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.583+0000 7fcf15400700 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563afd6381f0 0x563afd63d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.584+0000 7fcf15400700 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563afd6381f0 0x563afd63d4b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2692f36fd90d1798 server_cookie=643cde6d9f6c0337 in_seq=0 out_seq=0 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.584+0000 7fcef7e00700 10 client.25163 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.584+0000 7fcef7e00700 1 -- 172.21.15.67:0/2514013610 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563afd56d6b0 con 0x563afd6381f0 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.584+0000 7fcef7e00700 10 client.25163 handle_command_reply: tid=0 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.584+0000 7fcf30b89b80 1 client.25163 shutdown 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2514013610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 msgr2=0x563afd62d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd62d4c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009990 tx=0x7fcf080a2cc0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2514013610 shutdown_connections 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563afd6381f0 0x563afd63d4b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd625140 0x563afd634a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd623990 0x563afd62d4c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcef7e00700 0 client.25163 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2514013610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd6242f0 0x563afd62da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2514013610 >> 172.21.15.67:0/2514013610 conn(0x563afd582650 msgr2=0x563afd622370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2514013610 shutdown_connections 2024-03-30T02:36:11.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2514013610 wait complete. 2024-03-30T02:36:11.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.585+0000 7fcf30b89b80 20 client.25163 trim_cache size 0 max 16384 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf30b89b80 1 Processor -- start 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf30b89b80 1 -- start start 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd623990 0x563afd625520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd634a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd625140 0x563afd617fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd639a60 con 0x563afd6242f0 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd62e3c0 con 0x563afd625140 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd56d6b0 con 0x563afd623990 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf15400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd634a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf15400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd634a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57318/0 (socket says 172.21.15.67:57318) 2024-03-30T02:36:11.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf16800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd625140 0x563afd617fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf15400700 1 -- 172.21.15.67:0/2691047573 learned_addr learned my addr 172.21.15.67:0/2691047573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.604+0000 7fcf15e00700 1 --2- 172.21.15.67:0/2691047573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd623990 0x563afd625520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf15400700 1 -- 172.21.15.67:0/2691047573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd623990 msgr2=0x563afd625520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf15400700 1 --2- 172.21.15.67:0/2691047573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd623990 0x563afd625520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf15400700 1 -- 172.21.15.67:0/2691047573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd625140 msgr2=0x563afd617fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf15400700 1 --2- 172.21.15.67:0/2691047573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd625140 0x563afd617fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf15400700 1 -- 172.21.15.67:0/2691047573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563afd3a2b10 con 0x563afd6242f0 2024-03-30T02:36:11.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf16800700 1 --2- 172.21.15.67:0/2691047573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd625140 0x563afd617fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf15400700 1 --2- 172.21.15.67:0/2691047573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd634a00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400a010 tx=0x7fcf04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13132f0254cf071 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf03e00700 1 -- 172.21.15.67:0/2691047573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0402ae40 con 0x563afd6242f0 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf03e00700 1 -- 172.21.15.67:0/2691047573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf04023040 con 0x563afd6242f0 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.605+0000 7fcf03e00700 1 -- 172.21.15.67:0/2691047573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf040038e0 con 0x563afd6242f0 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2691047573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 msgr2=0x563afd634a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2691047573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd634a00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400a010 tx=0x7fcf04014000 comp rx=0 tx=0).stop 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2691047573 shutdown_connections 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2691047573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd625140 0x563afd617fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2691047573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd623990 0x563afd625520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/2691047573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd6242f0 0x563afd634a00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2691047573 >> 172.21.15.67:0/2691047573 conn(0x563afd582650 msgr2=0x563afd62c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2691047573 shutdown_connections 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 -- 172.21.15.67:0/2691047573 wait complete. 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.606+0000 7fcf30b89b80 1 -- start start 2024-03-30T02:36:11.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf30b89b80 1 Processor -- start 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd623990 0x563afd633c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd6242f0 0x563afd6341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf30b89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd634710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd68caa0 con 0x563afd623990 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd68cbe0 con 0x563afd6242f0 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf30b89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563afd68cd20 con 0x563afd625140 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf30b89b80 10 client.0 _ll_register_callbacks cb 0x563afd51a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf15e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd623990 0x563afd633c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf15e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd623990 0x563afd633c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57332/0 (socket says 172.21.15.67:57332) 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf15e00700 1 -- 172.21.15.67:0/321241959 learned_addr learned my addr 172.21.15.67:0/321241959 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf16800700 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd634710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.607+0000 7fcf15400700 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd6242f0 0x563afd6341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf15e00700 1 -- 172.21.15.67:0/321241959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 msgr2=0x563afd634710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf15e00700 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd634710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf15e00700 1 -- 172.21.15.67:0/321241959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd6242f0 msgr2=0x563afd6341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf15e00700 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd6242f0 0x563afd6341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf15e00700 1 -- 172.21.15.67:0/321241959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf100091c0 con 0x563afd623990 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf15400700 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd6242f0 0x563afd6341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf15e00700 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd623990 0x563afd633c90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcf08000c00 tx=0x7fcf08005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2f472f771aa00a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcef7e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcef7e00700 1 -- 172.21.15.67:0/321241959 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08099310 con 0x563afd623990 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcef7e00700 1 -- 172.21.15.67:0/321241959 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fcf0809ad10 con 0x563afd623990 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf30b89b80 1 -- 172.21.15.67:0/321241959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf080cb330 con 0x563afd623990 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcef7e00700 1 -- 172.21.15.67:0/321241959 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf08084100 con 0x563afd623990 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf30b89b80 10 client.15117 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.608+0000 7fcf30b89b80 1 -- 172.21.15.67:0/321241959 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563afd53ae50 con 0x563afd623990 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.609+0000 7fcef7e00700 1 -- 172.21.15.67:0/321241959 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf080cb330 con 0x563afd623990 2024-03-30T02:36:11.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.609+0000 7fcef7e00700 1 -- 172.21.15.67:0/321241959 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf0809bb10 con 0x563afd623990 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.609+0000 7fcf30b89b80 10 client.15117 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.609+0000 7fcf30b89b80 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.609+0000 7fcf30b89b80 20 client.15117 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.609+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563afd691540 0x563afd68dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.609+0000 7fcf30b89b80 4 client.15117 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.609+0000 7fcf30b89b80 1 -- 172.21.15.67:0/321241959 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563afd3c3ca0 con 0x563afd691540 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.609+0000 7fcf15400700 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563afd691540 0x563afd68dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf15400700 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563afd691540 0x563afd68dfa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c24e2b6e2ae829a server_cookie=ead94a1f13c10cfe in_seq=0 out_seq=0 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcef7e00700 10 client.15117 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcef7e00700 1 -- 172.21.15.67:0/321241959 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563afd3c3ca0 con 0x563afd691540 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcef7e00700 10 client.15117 handle_command_reply: tid=0 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 client.15117 shutdown 2024-03-30T02:36:11.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 -- 172.21.15.67:0/321241959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd623990 msgr2=0x563afd633c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd623990 0x563afd633c90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcf08000c00 tx=0x7fcf08005c00 comp rx=0 tx=0).stop 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 -- 172.21.15.67:0/321241959 shutdown_connections 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563afd691540 0x563afd68dfa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563afd623990 0x563afd633c90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcef7e00700 0 client.15117 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563afd625140 0x563afd634710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 --2- 172.21.15.67:0/321241959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563afd6242f0 0x563afd6341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 -- 172.21.15.67:0/321241959 >> 172.21.15.67:0/321241959 conn(0x563afd582650 msgr2=0x563afd6414d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 -- 172.21.15.67:0/321241959 shutdown_connections 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.610+0000 7fcf30b89b80 1 -- 172.21.15.67:0/321241959 wait complete. 2024-03-30T02:36:11.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf30b89b80 20 client.15117 trim_cache size 0 max 16384 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 3, 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24539, 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 53.507319965999997, 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 85, 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 98.603310918000005 2024-03-30T02:36:11.616 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:11.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 -- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcefc0a1230 msgr2=0x7fcefc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcefc0a1230 0x7fcefc0a36f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009c60 tx=0x7fcf04009260 comp rx=0 tx=0).stop 2024-03-30T02:36:11.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 -- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 msgr2=0x7fcf18198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 0x7fcf18198d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcf08003730 tx=0x7fcf0800f690 comp rx=0 tx=0).stop 2024-03-30T02:36:11.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 -- 172.21.15.67:0/4087069844 shutdown_connections 2024-03-30T02:36:11.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcefc0a1230 0x7fcefc0a36f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf18100bc0 0x7fcf18198d90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf18102750 0x7fcf181a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 --2- 172.21.15.67:0/4087069844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf18101db0 0x7fcf181992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 -- 172.21.15.67:0/4087069844 >> 172.21.15.67:0/4087069844 conn(0x7fcf180fc110 msgr2=0x7fcf18072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 -- 172.21.15.67:0/4087069844 shutdown_connections 2024-03-30T02:36:11.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.611+0000 7fcf17200700 1 -- 172.21.15.67:0/4087069844 wait complete. 2024-03-30T02:36:11.628 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:11 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2011761753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:11.628 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:11 smithi067 ceph-mon[37808]: pgmap v277: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-30T02:36:11.628 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:11 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4268472185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:11.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-30T02:36:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:11 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2011761753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:11 smithi170 ceph-mon[78798]: pgmap v277: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-30T02:36:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:11 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4268472185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:11.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:11 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2011761753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:11 smithi082 ceph-mon[79176]: pgmap v277: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-30T02:36:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:11 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4268472185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:11.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.828+0000 7efe18400700 1 -- 172.21.15.67:0/4140485742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 msgr2=0x7efe101023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.828+0000 7efe18400700 1 --2- 172.21.15.67:0/4140485742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 0x7efe101023d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0099c0 tx=0x7efdfc009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.828+0000 7efe18400700 1 -- 172.21.15.67:0/4140485742 shutdown_connections 2024-03-30T02:36:11.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.828+0000 7efe18400700 1 --2- 172.21.15.67:0/4140485742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe10102910 0x7efe101091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.828+0000 7efe18400700 1 --2- 172.21.15.67:0/4140485742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe10100d80 0x7efe101011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.828+0000 7efe18400700 1 --2- 172.21.15.67:0/4140485742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 0x7efe101023d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.828+0000 7efe18400700 1 -- 172.21.15.67:0/4140485742 >> 172.21.15.67:0/4140485742 conn(0x7efe100fc2f0 msgr2=0x7efe100fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.828+0000 7efe18400700 1 -- 172.21.15.67:0/4140485742 shutdown_connections 2024-03-30T02:36:11.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.828+0000 7efe18400700 1 -- 172.21.15.67:0/4140485742 wait complete. 2024-03-30T02:36:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.829+0000 7efe18400700 1 Processor -- start 2024-03-30T02:36:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.829+0000 7efe18400700 1 -- start start 2024-03-30T02:36:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.829+0000 7efe18400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe10100d80 0x7efe10196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.829+0000 7efe18400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 0x7efe101972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.829+0000 7efe18400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe10102910 0x7efe1019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.829+0000 7efe18400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1019e870 con 0x7efe10101f70 2024-03-30T02:36:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.829+0000 7efe18400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1019e9e0 con 0x7efe10100d80 2024-03-30T02:36:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.829+0000 7efe18400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1019eba0 con 0x7efe10102910 2024-03-30T02:36:11.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 0x7efe101972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe10100d80 0x7efe10196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe10100d80 0x7efe10196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38602/0 (socket says 172.21.15.67:38602) 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15a00700 1 -- 172.21.15.67:0/1849059138 learned_addr learned my addr 172.21.15.67:0/1849059138 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 0x7efe101972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57340/0 (socket says 172.21.15.67:57340) 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe16400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe10102910 0x7efe1019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15a00700 1 -- 172.21.15.67:0/1849059138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe10102910 msgr2=0x7efe1019e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15a00700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe10102910 0x7efe1019e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15a00700 1 -- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 msgr2=0x7efe101972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15a00700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 0x7efe101972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15a00700 1 -- 172.21.15.67:0/1849059138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0095f0 con 0x7efe10100d80 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe16400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe10102910 0x7efe1019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15000700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 0x7efe101972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.830+0000 7efe15a00700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe10100d80 0x7efe10196d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe0400e970 tx=0x7efe0400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.831+0000 7efe0e000700 1 -- 172.21.15.67:0/1849059138 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04009790 con 0x7efe10100d80 2024-03-30T02:36:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.831+0000 7efe0e000700 1 -- 172.21.15.67:0/1849059138 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe0400eec0 con 0x7efe10100d80 2024-03-30T02:36:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.831+0000 7efe18400700 1 -- 172.21.15.67:0/1849059138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1019ee80 con 0x7efe10100d80 2024-03-30T02:36:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.831+0000 7efe0e000700 1 -- 172.21.15.67:0/1849059138 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04003cf0 con 0x7efe10100d80 2024-03-30T02:36:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.831+0000 7efe18400700 1 -- 172.21.15.67:0/1849059138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1019f2a0 con 0x7efe10100d80 2024-03-30T02:36:11.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.834+0000 7efe0e000700 1 -- 172.21.15.67:0/1849059138 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7efe0400ccd0 con 0x7efe10100d80 2024-03-30T02:36:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.835+0000 7efe2be64b80 1 Processor -- start 2024-03-30T02:36:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.835+0000 7efe0e000700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efdf80a11b0 0x7efdf80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.835+0000 7efe2be64b80 1 -- start start 2024-03-30T02:36:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.835+0000 7efe0e000700 1 -- 172.21.15.67:0/1849059138 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efe04014070 con 0x7efe10100d80 2024-03-30T02:36:11.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.835+0000 7efe16400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efdf80a11b0 0x7efdf80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596cb90 0x55e9f596cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596d4f0 0x55e9f596ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f596e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe2be64b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f58c0940 con 0x55e9f596e390 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe2be64b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f58be300 con 0x55e9f596cb90 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe2be64b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f58b9bf0 con 0x55e9f596d4f0 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596cb90 0x55e9f596cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596cb90 0x55e9f596cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38610/0 (socket says 172.21.15.67:38610) 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15a00700 1 -- 172.21.15.67:0/2784253337 learned_addr learned my addr 172.21.15.67:0/2784253337 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15000700 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596d4f0 0x55e9f596ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe16400700 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f596e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15a00700 1 -- 172.21.15.67:0/2784253337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596d4f0 msgr2=0x55e9f596ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe16400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efdf80a11b0 0x7efdf80a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe08009d50 tx=0x7efe08009040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15a00700 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596d4f0 0x55e9f596ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15a00700 1 -- 172.21.15.67:0/2784253337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 msgr2=0x55e9f596e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15a00700 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f596e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15a00700 1 -- 172.21.15.67:0/2784253337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f573b360 con 0x55e9f596cb90 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.836+0000 7efe15000700 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596d4f0 0x55e9f596ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.837+0000 7efe15a00700 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596cb90 0x55e9f596cfb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efe04096a10 tx=0x7efe04099980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763b3d857d9cda9c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.837+0000 7efe02a00700 1 -- 172.21.15.67:0/2784253337 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040b1070 con 0x55e9f596cb90 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.837+0000 7efe02a00700 1 -- 172.21.15.67:0/2784253337 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe0409ddd0 con 0x55e9f596cb90 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.837+0000 7efe02a00700 1 -- 172.21.15.67:0/2784253337 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0409a440 con 0x55e9f596cb90 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.837+0000 7efe2be64b80 1 -- 172.21.15.67:0/2784253337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596cb90 msgr2=0x55e9f596cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.837+0000 7efe2be64b80 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596cb90 0x55e9f596cfb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efe04096a10 tx=0x7efe04099980 comp rx=0 tx=0).stop 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.838+0000 7efe2be64b80 1 -- 172.21.15.67:0/2784253337 shutdown_connections 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.838+0000 7efe2be64b80 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f596e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.838+0000 7efe2be64b80 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596cb90 0x55e9f596cfb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.838+0000 7efe2be64b80 1 --2- 172.21.15.67:0/2784253337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596d4f0 0x55e9f596ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.838+0000 7efe2be64b80 1 -- 172.21.15.67:0/2784253337 >> 172.21.15.67:0/2784253337 conn(0x55e9f58c7650 msgr2=0x55e9f5963950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.838+0000 7efe2be64b80 1 -- 172.21.15.67:0/2784253337 shutdown_connections 2024-03-30T02:36:11.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.838+0000 7efe2be64b80 1 -- 172.21.15.67:0/2784253337 wait complete. 2024-03-30T02:36:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.838+0000 7efe2be64b80 1 -- start start 2024-03-30T02:36:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.838+0000 7efe2be64b80 1 Processor -- start 2024-03-30T02:36:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596cb90 0x55e9f5976a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 0x55e9f5976fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596e390 0x55e9f597e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe2be64b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f5855d30 con 0x55e9f596cb90 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe2be64b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f58559d0 con 0x55e9f596d4f0 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe2be64b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f573d650 con 0x55e9f596e390 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe2be64b80 10 client.0 _ll_register_callbacks cb 0x55e9f585f420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe15a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596cb90 0x55e9f5976a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe15a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596cb90 0x55e9f5976a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57364/0 (socket says 172.21.15.67:57364) 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe15000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 0x55e9f5976fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe15a00700 1 -- 172.21.15.67:0/3755226899 learned_addr learned my addr 172.21.15.67:0/3755226899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.839+0000 7efe16400700 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596e390 0x55e9f597e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe15a00700 1 -- 172.21.15.67:0/3755226899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596e390 msgr2=0x55e9f597e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe15a00700 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596e390 0x55e9f597e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe15a00700 1 -- 172.21.15.67:0/3755226899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 msgr2=0x55e9f5976fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe15a00700 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 0x55e9f5976fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe15a00700 1 -- 172.21.15.67:0/3755226899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1019f2a0 con 0x55e9f596cb90 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe15000700 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 0x55e9f5976fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe15a00700 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596cb90 0x55e9f5976a90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe0400c5b0 tx=0x7efe04095bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf609afac13e0f0d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efdf7400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efdf7400700 1 -- 172.21.15.67:0/3755226899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040b6070 con 0x55e9f596cb90 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efdf7400700 1 -- 172.21.15.67:0/3755226899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efe040a2420 con 0x55e9f596cb90 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe2be64b80 1 -- 172.21.15.67:0/3755226899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9f587a330 con 0x55e9f596cb90 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efdf7400700 1 -- 172.21.15.67:0/3755226899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe040b1070 con 0x55e9f596cb90 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe2be64b80 10 client.15129 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.840+0000 7efe2be64b80 1 -- 172.21.15.67:0/3755226899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e9f56e7b10 con 0x55e9f596cb90 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.841+0000 7efdf7400700 1 -- 172.21.15.67:0/3755226899 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9f587a330 con 0x55e9f596cb90 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.841+0000 7efdf7400700 1 -- 172.21.15.67:0/3755226899 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe040a20b0 con 0x55e9f596cb90 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.841+0000 7efe2be64b80 10 client.15129 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.841+0000 7efe2be64b80 10 client.15129 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.841+0000 7efe2be64b80 20 client.15129 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.841+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55e9f5981890 0x55e9f5986b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.841+0000 7efe2be64b80 4 client.15129 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.841+0000 7efe2be64b80 1 -- 172.21.15.67:0/3755226899 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9f58b26b0 con 0x55e9f5981890 2024-03-30T02:36:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.841+0000 7efe15000700 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55e9f5981890 0x55e9f5986b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.842+0000 7efe15000700 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55e9f5981890 0x55e9f5986b50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=616c6d9d2570b28f server_cookie=d959a05f30fca98b in_seq=0 out_seq=0 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.842+0000 7efdf7400700 10 client.15129 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efdf7400700 1 -- 172.21.15.67:0/3755226899 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9f58b26b0 con 0x55e9f5981890 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efdf7400700 10 client.15129 handle_command_reply: tid=0 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 client.15129 shutdown 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 -- 172.21.15.67:0/3755226899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596cb90 msgr2=0x55e9f5976a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596cb90 0x55e9f5976a90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe0400c5b0 tx=0x7efe04095bc0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 -- 172.21.15.67:0/3755226899 shutdown_connections 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55e9f5981890 0x55e9f5986b50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596cb90 0x55e9f5976a90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596e390 0x55e9f597e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efdf7400700 0 client.15129 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3755226899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 0x55e9f5976fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 -- 172.21.15.67:0/3755226899 >> 172.21.15.67:0/3755226899 conn(0x55e9f58c7650 msgr2=0x55e9f5967120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 -- 172.21.15.67:0/3755226899 shutdown_connections 2024-03-30T02:36:11.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 1 -- 172.21.15.67:0/3755226899 wait complete. 2024-03-30T02:36:11.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.843+0000 7efe2be64b80 20 client.15129 trim_cache size 0 max 16384 2024-03-30T02:36:11.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.861+0000 7efe2be64b80 1 Processor -- start 2024-03-30T02:36:11.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe2be64b80 1 -- start start 2024-03-30T02:36:11.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 0x55e9f596e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 0x55e9f597dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f5962ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe2be64b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f5877ad0 con 0x55e9f596e390 2024-03-30T02:36:11.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe2be64b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f58b2990 con 0x55e9f596d4f0 2024-03-30T02:36:11.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe2be64b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f58b6850 con 0x55e9f596cb90 2024-03-30T02:36:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe16400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f5962ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe16400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f5962ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57378/0 (socket says 172.21.15.67:57378) 2024-03-30T02:36:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe16400700 1 -- 172.21.15.67:0/3961388663 learned_addr learned my addr 172.21.15.67:0/3961388663 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe15000700 1 --2- 172.21.15.67:0/3961388663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 0x55e9f597dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.862+0000 7efe15a00700 1 --2- 172.21.15.67:0/3961388663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 0x55e9f596e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.863+0000 7efe16400700 1 -- 172.21.15.67:0/3961388663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 msgr2=0x55e9f596e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.863+0000 7efe16400700 1 --2- 172.21.15.67:0/3961388663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 0x55e9f596e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.863+0000 7efe16400700 1 -- 172.21.15.67:0/3961388663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 msgr2=0x55e9f597dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.863+0000 7efe16400700 1 --2- 172.21.15.67:0/3961388663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 0x55e9f597dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.863+0000 7efe16400700 1 -- 172.21.15.67:0/3961388663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f587fe50 con 0x55e9f596e390 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.863+0000 7efe16400700 1 --2- 172.21.15.67:0/3961388663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f5962ec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efe080167f0 tx=0x7efe08012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce4d3891264f1f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.863+0000 7efe03400700 1 -- 172.21.15.67:0/3961388663 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08023870 con 0x55e9f596e390 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.863+0000 7efe03400700 1 -- 172.21.15.67:0/3961388663 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe0801c040 con 0x55e9f596e390 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.863+0000 7efe03400700 1 -- 172.21.15.67:0/3961388663 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08014ca0 con 0x55e9f596e390 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 -- 172.21.15.67:0/3961388663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 msgr2=0x55e9f5962ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3961388663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f5962ec0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efe080167f0 tx=0x7efe08012000 comp rx=0 tx=0).stop 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 -- 172.21.15.67:0/3961388663 shutdown_connections 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3961388663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596e390 0x55e9f5962ec0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3961388663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 0x55e9f596e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3961388663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596d4f0 0x55e9f597dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 -- 172.21.15.67:0/3961388663 >> 172.21.15.67:0/3961388663 conn(0x55e9f58c7650 msgr2=0x55e9f59715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 -- 172.21.15.67:0/3961388663 shutdown_connections 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 -- 172.21.15.67:0/3961388663 wait complete. 2024-03-30T02:36:11.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 -- start start 2024-03-30T02:36:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.864+0000 7efe2be64b80 1 Processor -- start 2024-03-30T02:36:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 0x55e9f597d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596d4f0 0x55e9f597d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe2be64b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596e390 0x55e9f597da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe2be64b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f597e0e0 con 0x55e9f596d4f0 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe2be64b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f597e250 con 0x55e9f596e390 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe2be64b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f59d6140 con 0x55e9f596cb90 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe2be64b80 10 client.0 _ll_register_callbacks cb 0x55e9f585f420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596d4f0 0x55e9f597d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 0x55e9f597d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596d4f0 0x55e9f597d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57392/0 (socket says 172.21.15.67:57392) 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15000700 1 -- 172.21.15.67:0/3820482793 learned_addr learned my addr 172.21.15.67:0/3820482793 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe16400700 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596e390 0x55e9f597da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15000700 1 -- 172.21.15.67:0/3820482793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 msgr2=0x55e9f597d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15000700 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 0x55e9f597d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15000700 1 -- 172.21.15.67:0/3820482793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596e390 msgr2=0x55e9f597da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15000700 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596e390 0x55e9f597da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15000700 1 -- 172.21.15.67:0/3820482793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe08016500 con 0x55e9f596d4f0 2024-03-30T02:36:11.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.865+0000 7efe15a00700 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 0x55e9f597d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.866+0000 7efe15000700 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596d4f0 0x55e9f597d550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0095c0 tx=0x7efdfc005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d156ab2bdcc8b5b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.866+0000 7efdf7400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.866+0000 7efdf7400700 1 -- 172.21.15.67:0/3820482793 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc005630 con 0x55e9f596d4f0 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.866+0000 7efdf7400700 1 -- 172.21.15.67:0/3820482793 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7efdfc027a70 con 0x55e9f596d4f0 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.866+0000 7efe2be64b80 1 -- 172.21.15.67:0/3820482793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe0409ddd0 con 0x55e9f596d4f0 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.866+0000 7efdf7400700 1 -- 172.21.15.67:0/3820482793 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdfc027c30 con 0x55e9f596d4f0 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.866+0000 7efe2be64b80 10 client.15135 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.866+0000 7efe2be64b80 1 -- 172.21.15.67:0/3820482793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e9f573fc80 con 0x55e9f596d4f0 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.867+0000 7efdf7400700 1 -- 172.21.15.67:0/3820482793 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe0409ddd0 con 0x55e9f596d4f0 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.867+0000 7efdf7400700 1 -- 172.21.15.67:0/3820482793 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efdfc00f6c0 con 0x55e9f596d4f0 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.867+0000 7efe2be64b80 10 client.15135 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.867+0000 7efe2be64b80 10 client.15135 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.867+0000 7efe2be64b80 20 client.15135 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:11.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.867+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55e9f59dae40 0x55e9f59d7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.867+0000 7efe2be64b80 4 client.15135 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.867+0000 7efe2be64b80 1 -- 172.21.15.67:0/3820482793 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e9f5708ca0 con 0x55e9f59dae40 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.867+0000 7efe15a00700 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55e9f59dae40 0x55e9f59d7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe15a00700 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55e9f59dae40 0x55e9f59d7790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ecae1ee3843c246 server_cookie=b0cf171c14b2c4a5 in_seq=0 out_seq=0 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efdf7400700 10 client.15135 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efdf7400700 1 -- 172.21.15.67:0/3820482793 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e9f5708ca0 con 0x55e9f59dae40 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efdf7400700 10 client.15135 handle_command_reply: tid=0 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 client.15135 shutdown 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 -- 172.21.15.67:0/3820482793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596d4f0 msgr2=0x55e9f597d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596d4f0 0x55e9f597d550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0095c0 tx=0x7efdfc005f00 comp rx=0 tx=0).stop 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 -- 172.21.15.67:0/3820482793 shutdown_connections 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55e9f59dae40 0x55e9f59d7790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f596e390 0x55e9f597da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f596cb90 0x55e9f597d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efdf7400700 0 client.15135 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:36:11.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 --2- 172.21.15.67:0/3820482793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f596d4f0 0x55e9f597d550 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 -- 172.21.15.67:0/3820482793 >> 172.21.15.67:0/3820482793 conn(0x55e9f58c7650 msgr2=0x55e9f5975c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 -- 172.21.15.67:0/3820482793 shutdown_connections 2024-03-30T02:36:11.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 1 -- 172.21.15.67:0/3820482793 wait complete. 2024-03-30T02:36:11.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.868+0000 7efe2be64b80 20 client.15135 trim_cache size 0 max 16384 2024-03-30T02:36:11.874 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:11.874 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:11.874 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24541, 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 54.769405896000002, 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 85, 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 97.104684270000007 2024-03-30T02:36:11.875 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 -- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efdf80a11b0 msgr2=0x7efdf80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efdf80a11b0 0x7efdf80a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe08009d50 tx=0x7efe08009040 comp rx=0 tx=0).stop 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 -- 172.21.15.67:0/1849059138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe10100d80 msgr2=0x7efe10196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe10100d80 0x7efe10196d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe0400e970 tx=0x7efe0400ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 -- 172.21.15.67:0/1849059138 shutdown_connections 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efdf80a11b0 0x7efdf80a3670 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe10102910 0x7efe1019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe10100d80 0x7efe10196d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 --2- 172.21.15.67:0/1849059138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe10101f70 0x7efe101972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 -- 172.21.15.67:0/1849059138 >> 172.21.15.67:0/1849059138 conn(0x7efe100fc2f0 msgr2=0x7efe10105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:11.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 -- 172.21.15.67:0/1849059138 shutdown_connections 2024-03-30T02:36:11.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:11.869+0000 7efe18400700 1 -- 172.21.15.67:0/1849059138 wait complete. 2024-03-30T02:36:11.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-03-30T02:36:12.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.086+0000 7f1422200700 1 -- 172.21.15.67:0/776699454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 msgr2=0x7f141c103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.086+0000 7f1422200700 1 --2- 172.21.15.67:0/776699454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 0x7f141c103bd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f140c009990 tx=0x7f140c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.086+0000 7f1422200700 1 -- 172.21.15.67:0/776699454 shutdown_connections 2024-03-30T02:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.086+0000 7f1422200700 1 --2- 172.21.15.67:0/776699454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 0x7f141c103bd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.086+0000 7f1422200700 1 --2- 172.21.15.67:0/776699454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c106a40 0x7f141c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.086+0000 7f1422200700 1 --2- 172.21.15.67:0/776699454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f141c104110 0x7f141c106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.086+0000 7f1422200700 1 -- 172.21.15.67:0/776699454 >> 172.21.15.67:0/776699454 conn(0x7f141c0fb1d0 msgr2=0x7f141c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.086+0000 7f1422200700 1 -- 172.21.15.67:0/776699454 shutdown_connections 2024-03-30T02:36:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.086+0000 7f1422200700 1 -- 172.21.15.67:0/776699454 wait complete. 2024-03-30T02:36:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.087+0000 7f1422200700 1 Processor -- start 2024-03-30T02:36:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.087+0000 7f1422200700 1 -- start start 2024-03-30T02:36:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f1422200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 0x7f141c101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f1422200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f141c104110 0x7f141c0ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f1422200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c106a40 0x7f141c0ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f1422200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c100470 con 0x7f141c1017e0 2024-03-30T02:36:12.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f1422200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c1005e0 con 0x7f141c104110 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f1422200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c1a0d20 con 0x7f141c106a40 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 0x7f141c101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 0x7f141c101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57414/0 (socket says 172.21.15.67:57414) 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f141c104110 0x7f141c0ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141b400700 1 -- 172.21.15.67:0/1454202369 learned_addr learned my addr 172.21.15.67:0/1454202369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141be00700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c106a40 0x7f141c0ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141b400700 1 -- 172.21.15.67:0/1454202369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c106a40 msgr2=0x7f141c0ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141b400700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c106a40 0x7f141c0ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141b400700 1 -- 172.21.15.67:0/1454202369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f141c104110 msgr2=0x7f141c0ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141b400700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f141c104110 0x7f141c0ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141b400700 1 -- 172.21.15.67:0/1454202369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c0095f0 con 0x7f141c1017e0 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.088+0000 7f141aa00700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f141c104110 0x7f141c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.089+0000 7f141b400700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 0x7f141c101100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f140c00b680 tx=0x7f140c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.089+0000 7f1413e00700 1 -- 172.21.15.67:0/1454202369 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c01d070 con 0x7f141c1017e0 2024-03-30T02:36:12.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.089+0000 7f1413e00700 1 -- 172.21.15.67:0/1454202369 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f140c022470 con 0x7f141c1017e0 2024-03-30T02:36:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.089+0000 7f1422200700 1 -- 172.21.15.67:0/1454202369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c1a0f50 con 0x7f141c1017e0 2024-03-30T02:36:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.089+0000 7f1413e00700 1 -- 172.21.15.67:0/1454202369 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c017650 con 0x7f141c1017e0 2024-03-30T02:36:12.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.089+0000 7f1422200700 1 -- 172.21.15.67:0/1454202369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c1a12c0 con 0x7f141c1017e0 2024-03-30T02:36:12.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.092+0000 7f1413e00700 1 -- 172.21.15.67:0/1454202369 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f140c0225e0 con 0x7f141c1017e0 2024-03-30T02:36:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.093+0000 7f1435ccdb80 1 Processor -- start 2024-03-30T02:36:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f1435ccdb80 1 -- start start 2024-03-30T02:36:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f1413e00700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14000a11b0 0x7f14000a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d91ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f1413e00700 1 -- 172.21.15.67:0/1454202369 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f140c0ce050 con 0x7f141c1017e0 2024-03-30T02:36:12.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 0x55cea1d93400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1ce9940 con 0x55cea1d92f80 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1ce7300 con 0x55cea1d91780 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1ce2bf0 con 0x55cea1d920e0 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f141be00700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14000a11b0 0x7f14000a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f141be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 0x55cea1d93400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f141b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d91ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f141aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f141aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d929a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48748/0 (socket says 172.21.15.67:48748) 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f141b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d91ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38654/0 (socket says 172.21.15.67:38654) 2024-03-30T02:36:12.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.094+0000 7f141aa00700 1 -- 172.21.15.67:0/387874692 learned_addr learned my addr 172.21.15.67:0/387874692 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f141aa00700 1 -- 172.21.15.67:0/387874692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 msgr2=0x55cea1d91ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f141aa00700 1 --2- 172.21.15.67:0/387874692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d91ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f141aa00700 1 -- 172.21.15.67:0/387874692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 msgr2=0x55cea1d93400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f141aa00700 1 --2- 172.21.15.67:0/387874692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 0x55cea1d93400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f141aa00700 1 -- 172.21.15.67:0/387874692 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cea1b64360 con 0x55cea1d920e0 2024-03-30T02:36:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f141b400700 1 --2- 172.21.15.67:0/387874692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d91ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f141be00700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14000a11b0 0x7f14000a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1414009ec0 tx=0x7f14140092f0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f141aa00700 1 --2- 172.21.15.67:0/387874692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d929a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f14080115f0 tx=0x7f1408011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15686df2a0c7d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f1407e00700 1 -- 172.21.15.67:0/387874692 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408019630 con 0x55cea1d920e0 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f1407e00700 1 -- 172.21.15.67:0/387874692 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1408019c50 con 0x55cea1d920e0 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.095+0000 7f1407e00700 1 -- 172.21.15.67:0/387874692 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140800eb00 con 0x55cea1d920e0 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.096+0000 7f1435ccdb80 1 -- 172.21.15.67:0/387874692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 msgr2=0x55cea1d929a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.096+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/387874692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d929a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f14080115f0 tx=0x7f1408011900 comp rx=0 tx=0).stop 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.096+0000 7f1435ccdb80 1 -- 172.21.15.67:0/387874692 shutdown_connections 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.096+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/387874692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 0x55cea1d93400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.096+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/387874692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d91ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.096+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/387874692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d929a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.096+0000 7f1435ccdb80 1 -- 172.21.15.67:0/387874692 >> 172.21.15.67:0/387874692 conn(0x55cea1cf0650 msgr2=0x55cea1d8c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.096+0000 7f1435ccdb80 1 -- 172.21.15.67:0/387874692 shutdown_connections 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.096+0000 7f1435ccdb80 1 -- 172.21.15.67:0/387874692 wait complete. 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.097+0000 7f1435ccdb80 1 -- start start 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.097+0000 7f1435ccdb80 1 Processor -- start 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.097+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d9b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.097+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d9bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.097+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 0x55cea1da2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.097+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1c7ed30 con 0x55cea1d92f80 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.097+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1c7e9d0 con 0x55cea1d91780 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.097+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1b66650 con 0x55cea1d920e0 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f1435ccdb80 10 client.0 _ll_register_callbacks cb 0x55cea1c88420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 0x55cea1da2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d9b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d9b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38668/0 (socket says 172.21.15.67:38668) 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141b400700 1 -- 172.21.15.67:0/695851011 learned_addr learned my addr 172.21.15.67:0/695851011 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141aa00700 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d9bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141be00700 1 -- 172.21.15.67:0/695851011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 msgr2=0x55cea1d9bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141be00700 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d9bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141be00700 1 -- 172.21.15.67:0/695851011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 msgr2=0x55cea1d9b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141be00700 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d9b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.098+0000 7f141be00700 1 -- 172.21.15.67:0/695851011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c1a12c0 con 0x55cea1d92f80 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.099+0000 7f141be00700 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 0x55cea1da2ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55cea1d92760 tx=0x7f141400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d58bf43e5887c01 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.099+0000 7f1404c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.099+0000 7f1404c00700 1 -- 172.21.15.67:0/695851011 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141400e070 con 0x55cea1d92f80 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.099+0000 7f1435ccdb80 1 -- 172.21.15.67:0/695851011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cea1ca3330 con 0x55cea1d92f80 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.099+0000 7f1404c00700 1 -- 172.21.15.67:0/695851011 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1414021040 con 0x55cea1d92f80 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.099+0000 7f1404c00700 1 -- 172.21.15.67:0/695851011 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f141401ae80 con 0x55cea1d92f80 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.099+0000 7f1435ccdb80 10 client.15147 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.099+0000 7f1435ccdb80 1 -- 172.21.15.67:0/695851011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f1408009dd0 con 0x55cea1d92f80 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.099+0000 7f1404c00700 1 -- 172.21.15.67:0/695851011 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cea1ca3330 con 0x55cea1d92f80 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.100+0000 7f1404c00700 1 -- 172.21.15.67:0/695851011 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1414027ce0 con 0x55cea1d92f80 2024-03-30T02:36:12.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.100+0000 7f1435ccdb80 10 client.15147 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.100+0000 7f1435ccdb80 10 client.15147 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.100+0000 7f1435ccdb80 20 client.15147 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.100+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55cea1da6880 0x55cea1dabba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.100+0000 7f1435ccdb80 4 client.15147 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.100+0000 7f1435ccdb80 1 -- 172.21.15.67:0/695851011 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cea1cdb6b0 con 0x55cea1da6880 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.100+0000 7f141b400700 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55cea1da6880 0x55cea1dabba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.101+0000 7f1404c00700 10 client.15147 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.101+0000 7f141b400700 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55cea1da6880 0x55cea1dabba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ba8b73f548ad4f server_cookie=ef2086c8c4639886 in_seq=0 out_seq=0 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.101+0000 7f1404c00700 1 -- 172.21.15.67:0/695851011 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cea1cdb6b0 con 0x55cea1da6880 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.101+0000 7f1404c00700 10 client.15147 handle_command_reply: tid=0 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.101+0000 7f1435ccdb80 1 client.15147 shutdown 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 -- 172.21.15.67:0/695851011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 msgr2=0x55cea1da2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 0x55cea1da2ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55cea1d92760 tx=0x7f141400b040 comp rx=0 tx=0).stop 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 -- 172.21.15.67:0/695851011 shutdown_connections 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55cea1da6880 0x55cea1dabba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d92f80 0x55cea1da2ba0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d91780 0x55cea1d9b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/695851011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d9bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1404c00700 0 client.15147 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 -- 172.21.15.67:0/695851011 >> 172.21.15.67:0/695851011 conn(0x55cea1cf0650 msgr2=0x55cea1d90120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 -- 172.21.15.67:0/695851011 shutdown_connections 2024-03-30T02:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 1 -- 172.21.15.67:0/695851011 wait complete. 2024-03-30T02:36:12.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.102+0000 7f1435ccdb80 20 client.15147 trim_cache size 0 max 16384 2024-03-30T02:36:12.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f1435ccdb80 1 Processor -- start 2024-03-30T02:36:12.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f1435ccdb80 1 -- start start 2024-03-30T02:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1d91b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d9ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d92f80 0x55cea1d93360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1cdb990 con 0x55cea1d91780 2024-03-30T02:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1cdf850 con 0x55cea1d92f80 2024-03-30T02:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1c81de0 con 0x55cea1d920e0 2024-03-30T02:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f141b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1d91b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f141b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1d91b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57438/0 (socket says 172.21.15.67:57438) 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f141aa00700 1 --2- 172.21.15.67:0/3566686736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d9ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.121+0000 7f141b400700 1 -- 172.21.15.67:0/3566686736 learned_addr learned my addr 172.21.15.67:0/3566686736 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f141be00700 1 --2- 172.21.15.67:0/3566686736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d92f80 0x55cea1d93360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f141b400700 1 -- 172.21.15.67:0/3566686736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 msgr2=0x55cea1d9ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f141b400700 1 --2- 172.21.15.67:0/3566686736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d9ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f141b400700 1 -- 172.21.15.67:0/3566686736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d92f80 msgr2=0x55cea1d93360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f141b400700 1 --2- 172.21.15.67:0/3566686736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d92f80 0x55cea1d93360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f141b400700 1 -- 172.21.15.67:0/3566686736 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cea1b10b10 con 0x55cea1d91780 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f141b400700 1 --2- 172.21.15.67:0/3566686736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1d91b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f140c022890 tx=0x7f140c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdeddea41fb35910 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f1411200700 1 -- 172.21.15.67:0/3566686736 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c0b9070 con 0x55cea1d91780 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f1411200700 1 -- 172.21.15.67:0/3566686736 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f140c09b7d0 con 0x55cea1d91780 2024-03-30T02:36:12.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.122+0000 7f1411200700 1 -- 172.21.15.67:0/3566686736 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c0977e0 con 0x55cea1d91780 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 -- 172.21.15.67:0/3566686736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 msgr2=0x55cea1d91b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/3566686736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1d91b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f140c022890 tx=0x7f140c01a040 comp rx=0 tx=0).stop 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 -- 172.21.15.67:0/3566686736 shutdown_connections 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/3566686736 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1d91b60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/3566686736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d92f80 0x55cea1d93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/3566686736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d920e0 0x55cea1d9ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 -- 172.21.15.67:0/3566686736 >> 172.21.15.67:0/3566686736 conn(0x55cea1cf0650 msgr2=0x55cea1d99550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 -- 172.21.15.67:0/3566686736 shutdown_connections 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 -- 172.21.15.67:0/3566686736 wait complete. 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.123+0000 7f1435ccdb80 1 -- start start 2024-03-30T02:36:12.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.124+0000 7f1435ccdb80 1 Processor -- start 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.124+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1da2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.124+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d920e0 0x55cea1d9f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.124+0000 7f1435ccdb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d92f80 0x55cea1d9fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.124+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1da0320 con 0x55cea1d91780 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.124+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1dfb190 con 0x55cea1d920e0 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.124+0000 7f1435ccdb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea1dfb350 con 0x55cea1d92f80 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f1435ccdb80 10 client.0 _ll_register_callbacks cb 0x55cea1c88420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1da2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d920e0 0x55cea1d9f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1da2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57440/0 (socket says 172.21.15.67:57440) 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141b400700 1 -- 172.21.15.67:0/4281639093 learned_addr learned my addr 172.21.15.67:0/4281639093 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141be00700 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d92f80 0x55cea1d9fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141b400700 1 -- 172.21.15.67:0/4281639093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d92f80 msgr2=0x55cea1d9fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141b400700 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d92f80 0x55cea1d9fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141b400700 1 -- 172.21.15.67:0/4281639093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d920e0 msgr2=0x55cea1d9f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141b400700 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d920e0 0x55cea1d9f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141b400700 1 -- 172.21.15.67:0/4281639093 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c1a0f50 con 0x55cea1d91780 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f141b400700 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1da2b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f140c0a2f90 tx=0x7f140c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178103ac502caf20 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f1404c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f1404c00700 1 -- 172.21.15.67:0/4281639093 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c0b9070 con 0x55cea1d91780 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.125+0000 7f1404c00700 1 -- 172.21.15.67:0/4281639093 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f140c09ef00 con 0x55cea1d91780 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1435ccdb80 1 -- 172.21.15.67:0/4281639093 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1408019c50 con 0x55cea1d91780 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1404c00700 1 -- 172.21.15.67:0/4281639093 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f140c0a34c0 con 0x55cea1d91780 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1435ccdb80 10 client.15153 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1435ccdb80 1 -- 172.21.15.67:0/4281639093 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cea1ca8e50 con 0x55cea1d91780 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1404c00700 1 -- 172.21.15.67:0/4281639093 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1408019c50 con 0x55cea1d91780 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1404c00700 1 -- 172.21.15.67:0/4281639093 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f140c09a5c0 con 0x55cea1d91780 2024-03-30T02:36:12.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1435ccdb80 10 client.15153 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1435ccdb80 10 client.15153 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1435ccdb80 20 client.15153 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55cea1dffc30 0x55cea1dfc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1435ccdb80 4 client.15153 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f1435ccdb80 1 -- 172.21.15.67:0/4281639093 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cea1b31ca0 con 0x55cea1dffc30 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.126+0000 7f141aa00700 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55cea1dffc30 0x55cea1dfc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.127+0000 7f1404c00700 10 client.15153 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.127+0000 7f141aa00700 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55cea1dffc30 0x55cea1dfc640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=206256f7aeee6e8f server_cookie=15cebb4217953f19 in_seq=0 out_seq=0 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.127+0000 7f1404c00700 1 -- 172.21.15.67:0/4281639093 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55cea1b31ca0 con 0x55cea1dffc30 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.127+0000 7f1404c00700 10 client.15153 handle_command_reply: tid=0 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.127+0000 7f1435ccdb80 1 client.15153 shutdown 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.127+0000 7f1435ccdb80 1 -- 172.21.15.67:0/4281639093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 msgr2=0x55cea1da2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.127+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1da2b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f140c0a2f90 tx=0x7f140c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1435ccdb80 1 -- 172.21.15.67:0/4281639093 shutdown_connections 2024-03-30T02:36:12.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55cea1dffc30 0x55cea1dfc640 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cea1d91780 0x55cea1da2b40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cea1d92f80 0x55cea1d9fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1435ccdb80 1 --2- 172.21.15.67:0/4281639093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea1d920e0 0x55cea1d9f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1404c00700 0 client.15153 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1435ccdb80 1 -- 172.21.15.67:0/4281639093 >> 172.21.15.67:0/4281639093 conn(0x55cea1cf0650 msgr2=0x55cea1dafbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1435ccdb80 1 -- 172.21.15.67:0/4281639093 shutdown_connections 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1435ccdb80 1 -- 172.21.15.67:0/4281639093 wait complete. 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1435ccdb80 20 client.15153 trim_cache size 0 max 16384 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:12.133 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24560, 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 56.037762635999997, 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 85, 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 94.926975317 2024-03-30T02:36:12.134 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1422200700 1 -- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14000a11b0 msgr2=0x7f14000a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1422200700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14000a11b0 0x7f14000a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1414009ec0 tx=0x7f14140092f0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1422200700 1 -- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 msgr2=0x7f141c101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.128+0000 7f1422200700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 0x7f141c101100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f140c00b680 tx=0x7f140c009580 comp rx=0 tx=0).stop 2024-03-30T02:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.129+0000 7f1422200700 1 -- 172.21.15.67:0/1454202369 shutdown_connections 2024-03-30T02:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.129+0000 7f1422200700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f14000a11b0 0x7f14000a3670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.129+0000 7f1422200700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f141c1017e0 0x7f141c101100 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.129+0000 7f1422200700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f141c106a40 0x7f141c0ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.129+0000 7f1422200700 1 --2- 172.21.15.67:0/1454202369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f141c104110 0x7f141c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.129+0000 7f1422200700 1 -- 172.21.15.67:0/1454202369 >> 172.21.15.67:0/1454202369 conn(0x7f141c0fb1d0 msgr2=0x7f141c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.129+0000 7f1422200700 1 -- 172.21.15.67:0/1454202369 shutdown_connections 2024-03-30T02:36:12.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.129+0000 7f1422200700 1 -- 172.21.15.67:0/1454202369 wait complete. 2024-03-30T02:36:12.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-30T02:36:12.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.346+0000 7fb625e00700 1 -- 172.21.15.67:0/1996239143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb620074d90 msgr2=0x7fb6200731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.346+0000 7fb625e00700 1 --2- 172.21.15.67:0/1996239143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb620074d90 0x7fb6200731e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb610009990 tx=0x7fb610009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.346+0000 7fb625e00700 1 -- 172.21.15.67:0/1996239143 shutdown_connections 2024-03-30T02:36:12.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.346+0000 7fb625e00700 1 --2- 172.21.15.67:0/1996239143 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb620074d90 0x7fb6200731e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.346+0000 7fb625e00700 1 --2- 172.21.15.67:0/1996239143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb620108c50 0x7fb62010b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.346+0000 7fb625e00700 1 --2- 172.21.15.67:0/1996239143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200737b0 0x7fb620073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.346+0000 7fb625e00700 1 -- 172.21.15.67:0/1996239143 >> 172.21.15.67:0/1996239143 conn(0x7fb6200fc2f0 msgr2=0x7fb6200fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.346+0000 7fb625e00700 1 -- 172.21.15.67:0/1996239143 shutdown_connections 2024-03-30T02:36:12.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.346+0000 7fb625e00700 1 -- 172.21.15.67:0/1996239143 wait complete. 2024-03-30T02:36:12.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.347+0000 7fb625e00700 1 Processor -- start 2024-03-30T02:36:12.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.347+0000 7fb625e00700 1 -- start start 2024-03-30T02:36:12.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.347+0000 7fb625e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6200737b0 0x7fb620198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb625e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb620074d90 0x7fb620199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb625e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb620108c50 0x7fb6201a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb625e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6201a0a70 con 0x7fb620108c50 2024-03-30T02:36:12.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb625e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6201a0bb0 con 0x7fb620074d90 2024-03-30T02:36:12.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb625e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6201a0d40 con 0x7fb6200737b0 2024-03-30T02:36:12.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6200737b0 0x7fb620198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61e000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb620074d90 0x7fb620199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6200737b0 0x7fb620198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48808/0 (socket says 172.21.15.67:48808) 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61ea00700 1 -- 172.21.15.67:0/1053925231 learned_addr learned my addr 172.21.15.67:0/1053925231 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61f400700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb620108c50 0x7fb6201a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61e000700 1 -- 172.21.15.67:0/1053925231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6200737b0 msgr2=0x7fb620198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61e000700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6200737b0 0x7fb620198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61e000700 1 -- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb620108c50 msgr2=0x7fb6201a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61e000700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb620108c50 0x7fb6201a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.348+0000 7fb61e000700 1 -- 172.21.15.67:0/1053925231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6100095f0 con 0x7fb620074d90 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.349+0000 7fb61e000700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb620074d90 0x7fb620199480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00e9f0 tx=0x7fb60c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.349+0000 7fb624a00700 1 -- 172.21.15.67:0/1053925231 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00c9c0 con 0x7fb620074d90 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.349+0000 7fb624a00700 1 -- 172.21.15.67:0/1053925231 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb60c010430 con 0x7fb620074d90 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.349+0000 7fb625e00700 1 -- 172.21.15.67:0/1053925231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6201a0fc0 con 0x7fb620074d90 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.349+0000 7fb624a00700 1 -- 172.21.15.67:0/1053925231 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c018680 con 0x7fb620074d90 2024-03-30T02:36:12.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.349+0000 7fb625e00700 1 -- 172.21.15.67:0/1053925231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6201a14a0 con 0x7fb620074d90 2024-03-30T02:36:12.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.352+0000 7fb624a00700 1 -- 172.21.15.67:0/1053925231 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fb60c010860 con 0x7fb620074d90 2024-03-30T02:36:12.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.353+0000 7fb639995b80 1 Processor -- start 2024-03-30T02:36:12.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb639995b80 1 -- start start 2024-03-30T02:36:12.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb639995b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a687780 0x55e68a687ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb639995b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a6889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb639995b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 0x55e68a689400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb639995b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a5df940 con 0x55e68a687780 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb639995b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a5dd300 con 0x55e68a6880e0 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb639995b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a5d8bf0 con 0x55e68a688f80 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb624a00700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb6040a11b0 0x7fb6040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb61f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a687780 0x55e68a687ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb61f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a687780 0x55e68a687ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57482/0 (socket says 172.21.15.67:57482) 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.354+0000 7fb61f400700 1 -- 172.21.15.67:0/3683229918 learned_addr learned my addr 172.21.15.67:0/3683229918 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb61ea00700 1 --2- 172.21.15.67:0/3683229918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a6889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb624a00700 1 -- 172.21.15.67:0/1053925231 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb60c014070 con 0x7fb620074d90 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb61e000700 1 --2- 172.21.15.67:0/3683229918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 0x55e68a689400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb61ea00700 1 -- 172.21.15.67:0/3683229918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 msgr2=0x55e68a689400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb61ea00700 1 --2- 172.21.15.67:0/3683229918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 0x55e68a689400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb61ea00700 1 -- 172.21.15.67:0/3683229918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a687780 msgr2=0x55e68a687ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb61ea00700 1 --2- 172.21.15.67:0/3683229918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a687780 0x55e68a687ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb61ea00700 1 -- 172.21.15.67:0/3683229918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e68a45a360 con 0x55e68a6880e0 2024-03-30T02:36:12.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb61f400700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb6040a11b0 0x7fb6040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb61ea00700 1 --2- 172.21.15.67:0/3683229918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a6889a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb610005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8feda512362c4c61 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.355+0000 7fb60be00700 1 -- 172.21.15.67:0/3683229918 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61001d070 con 0x55e68a6880e0 2024-03-30T02:36:12.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb60be00700 1 -- 172.21.15.67:0/3683229918 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb61001e040 con 0x55e68a6880e0 2024-03-30T02:36:12.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb60be00700 1 -- 172.21.15.67:0/3683229918 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6100048a0 con 0x55e68a6880e0 2024-03-30T02:36:12.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb639995b80 1 -- 172.21.15.67:0/3683229918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 msgr2=0x55e68a6889a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb61f400700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb6040a11b0 0x7fb6040a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb61800fb80 tx=0x7fb618004b40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb639995b80 1 --2- 172.21.15.67:0/3683229918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a6889a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb610005ec0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb639995b80 1 -- 172.21.15.67:0/3683229918 shutdown_connections 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb639995b80 1 --2- 172.21.15.67:0/3683229918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a687780 0x55e68a687ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb639995b80 1 --2- 172.21.15.67:0/3683229918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 0x55e68a689400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb639995b80 1 --2- 172.21.15.67:0/3683229918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a6889a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb639995b80 1 -- 172.21.15.67:0/3683229918 >> 172.21.15.67:0/3683229918 conn(0x55e68a5e6650 msgr2=0x55e68a682950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb639995b80 1 -- 172.21.15.67:0/3683229918 shutdown_connections 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.356+0000 7fb639995b80 1 -- 172.21.15.67:0/3683229918 wait complete. 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.357+0000 7fb639995b80 1 -- start start 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.357+0000 7fb639995b80 1 Processor -- start 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.357+0000 7fb639995b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a691570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.357+0000 7fb639995b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a6880e0 0x55e68a691ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb639995b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a698af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb639995b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a574d30 con 0x55e68a688f80 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb639995b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a5749d0 con 0x55e68a687780 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb639995b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a45c650 con 0x55e68a6880e0 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb639995b80 10 client.0 _ll_register_callbacks cb 0x55e68a57e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a698af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a691570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a698af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57494/0 (socket says 172.21.15.67:57494) 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61e000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a6880e0 0x55e68a691ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61f400700 1 -- 172.21.15.67:0/1282299580 learned_addr learned my addr 172.21.15.67:0/1282299580 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61ea00700 1 -- 172.21.15.67:0/1282299580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a6880e0 msgr2=0x55e68a691ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61ea00700 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a6880e0 0x55e68a691ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61ea00700 1 -- 172.21.15.67:0/1282299580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 msgr2=0x55e68a698af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61ea00700 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a698af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.358+0000 7fb61ea00700 1 -- 172.21.15.67:0/1282299580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62010bce0 con 0x55e68a687780 2024-03-30T02:36:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.359+0000 7fb61ea00700 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a691570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb61000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6841f90ae584515e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.359+0000 7fb608c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.359+0000 7fb608c00700 1 -- 172.21.15.67:0/1282299580 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61001d040 con 0x55e68a687780 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.359+0000 7fb639995b80 1 -- 172.21.15.67:0/1282299580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e68a599330 con 0x55e68a687780 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.359+0000 7fb608c00700 1 -- 172.21.15.67:0/1282299580 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb61000fd10 con 0x55e68a687780 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.359+0000 7fb608c00700 1 -- 172.21.15.67:0/1282299580 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb610005030 con 0x55e68a687780 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.359+0000 7fb639995b80 10 client.25217 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.359+0000 7fb639995b80 1 -- 172.21.15.67:0/1282299580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e68a406b10 con 0x55e68a687780 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.359+0000 7fb608c00700 1 -- 172.21.15.67:0/1282299580 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e68a599330 con 0x55e68a687780 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb608c00700 1 -- 172.21.15.67:0/1282299580 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb610004c20 con 0x55e68a687780 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb639995b80 10 client.25217 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb639995b80 10 client.25217 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb639995b80 20 client.25217 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb639995b80 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e68a69c200 0x55e68a6a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb639995b80 4 client.25217 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:12.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb639995b80 1 -- 172.21.15.67:0/1282299580 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e68a5d16b0 con 0x55e68a69c200 2024-03-30T02:36:12.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb61e000700 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e68a69c200 0x55e68a6a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb608c00700 10 client.25217 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:12.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.360+0000 7fb61e000700 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e68a69c200 0x55e68a6a1520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9772bc63805e37 server_cookie=aeec646d87bf264 in_seq=0 out_seq=0 2024-03-30T02:36:12.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb608c00700 1 -- 172.21.15.67:0/1282299580 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e68a5d16b0 con 0x55e68a69c200 2024-03-30T02:36:12.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb608c00700 10 client.25217 handle_command_reply: tid=0 2024-03-30T02:36:12.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb639995b80 1 client.25217 shutdown 2024-03-30T02:36:12.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb639995b80 1 -- 172.21.15.67:0/1282299580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 msgr2=0x55e68a691570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb639995b80 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a691570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb610006010 tx=0x7fb61000f690 comp rx=0 tx=0).stop 2024-03-30T02:36:12.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb639995b80 1 -- 172.21.15.67:0/1282299580 shutdown_connections 2024-03-30T02:36:12.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb639995b80 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e68a69c200 0x55e68a6a1520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb639995b80 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a698af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb608c00700 0 client.25217 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:12.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb639995b80 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a691570 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb639995b80 1 --2- 172.21.15.67:0/1282299580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a6880e0 0x55e68a691ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.361+0000 7fb639995b80 1 -- 172.21.15.67:0/1282299580 >> 172.21.15.67:0/1282299580 conn(0x55e68a5e6650 msgr2=0x55e68a686120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.362+0000 7fb639995b80 1 -- 172.21.15.67:0/1282299580 shutdown_connections 2024-03-30T02:36:12.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.362+0000 7fb639995b80 1 -- 172.21.15.67:0/1282299580 wait complete. 2024-03-30T02:36:12.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.362+0000 7fb639995b80 20 client.25217 trim_cache size 0 max 16384 2024-03-30T02:36:12.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.380+0000 7fb639995b80 1 Processor -- start 2024-03-30T02:36:12.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.380+0000 7fb639995b80 1 -- start start 2024-03-30T02:36:12.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.380+0000 7fb639995b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a681ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.380+0000 7fb639995b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a6880e0 0x55e68a698a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.380+0000 7fb639995b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 0x55e68a690a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.380+0000 7fb639995b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a5d16e0 con 0x55e68a6880e0 2024-03-30T02:36:12.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.380+0000 7fb639995b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a599360 con 0x55e68a687780 2024-03-30T02:36:12.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.380+0000 7fb639995b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a5d1990 con 0x55e68a688f80 2024-03-30T02:36:12.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a6880e0 0x55e68a698a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a681ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a681ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38732/0 (socket says 172.21.15.67:38732) 2024-03-30T02:36:12.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 0x55e68a690a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61ea00700 1 -- 172.21.15.67:0/1065168825 learned_addr learned my addr 172.21.15.67:0/1065168825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61ea00700 1 -- 172.21.15.67:0/1065168825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 msgr2=0x55e68a690a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61ea00700 1 --2- 172.21.15.67:0/1065168825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 0x55e68a690a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61ea00700 1 -- 172.21.15.67:0/1065168825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a6880e0 msgr2=0x55e68a698a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61ea00700 1 --2- 172.21.15.67:0/1065168825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a6880e0 0x55e68a698a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61ea00700 1 -- 172.21.15.67:0/1065168825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e68a59ee50 con 0x55e68a687780 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.381+0000 7fb61ea00700 1 --2- 172.21.15.67:0/1065168825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a681ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb610005910 tx=0x7fb610009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48c5154f62e128b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb615200700 1 -- 172.21.15.67:0/1065168825 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6100225a0 con 0x55e68a687780 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb615200700 1 -- 172.21.15.67:0/1065168825 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb610016b20 con 0x55e68a687780 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb615200700 1 -- 172.21.15.67:0/1065168825 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610005240 con 0x55e68a687780 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb639995b80 1 -- 172.21.15.67:0/1065168825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 msgr2=0x55e68a681ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb639995b80 1 --2- 172.21.15.67:0/1065168825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a681ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb610005910 tx=0x7fb610009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb639995b80 1 -- 172.21.15.67:0/1065168825 shutdown_connections 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb639995b80 1 --2- 172.21.15.67:0/1065168825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a688f80 0x55e68a690a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb639995b80 1 --2- 172.21.15.67:0/1065168825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a687780 0x55e68a681ec0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb639995b80 1 --2- 172.21.15.67:0/1065168825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a6880e0 0x55e68a698a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb639995b80 1 -- 172.21.15.67:0/1065168825 >> 172.21.15.67:0/1065168825 conn(0x55e68a5e6650 msgr2=0x55e68a67b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb639995b80 1 -- 172.21.15.67:0/1065168825 shutdown_connections 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.382+0000 7fb639995b80 1 -- 172.21.15.67:0/1065168825 wait complete. 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.383+0000 7fb639995b80 1 -- start start 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.383+0000 7fb639995b80 1 Processor -- start 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.383+0000 7fb639995b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a687780 0x55e68a68ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.383+0000 7fb639995b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a68e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.383+0000 7fb639995b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a68e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.383+0000 7fb639995b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a698f10 con 0x55e68a688f80 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.383+0000 7fb639995b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a6f0b10 con 0x55e68a6880e0 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.383+0000 7fb639995b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68a6f0c50 con 0x55e68a687780 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.383+0000 7fb639995b80 10 client.0 _ll_register_callbacks cb 0x55e68a57e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a687780 0x55e68a68ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a68e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61e000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a68e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a687780 0x55e68a68ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48836/0 (socket says 172.21.15.67:48836) 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61ea00700 1 -- 172.21.15.67:0/3181864044 learned_addr learned my addr 172.21.15.67:0/3181864044 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61e000700 1 -- 172.21.15.67:0/3181864044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a687780 msgr2=0x55e68a68ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61e000700 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a687780 0x55e68a68ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61e000700 1 -- 172.21.15.67:0/3181864044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 msgr2=0x55e68a68e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61e000700 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a68e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61e000700 1 -- 172.21.15.67:0/3181864044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e68a406b10 con 0x55e68a6880e0 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61f400700 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a68e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61ea00700 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a687780 0x55e68a68ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb61e000700 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a68e330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0ca520 tx=0x7fb60c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ab8837344ad3fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.384+0000 7fb608c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb608c00700 1 -- 172.21.15.67:0/3181864044 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c09dc90 con 0x55e68a6880e0 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb608c00700 1 -- 172.21.15.67:0/3181864044 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7fb60c0a22e0 con 0x55e68a6880e0 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb639995b80 1 -- 172.21.15.67:0/3181864044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb61001e040 con 0x55e68a6880e0 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb608c00700 1 -- 172.21.15.67:0/3181864044 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb60c09a0c0 con 0x55e68a6880e0 2024-03-30T02:36:12.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb639995b80 10 client.25223 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb639995b80 1 -- 172.21.15.67:0/3181864044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e68a45ec80 con 0x55e68a6880e0 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb608c00700 1 -- 172.21.15.67:0/3181864044 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb61001e040 con 0x55e68a6880e0 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb608c00700 1 -- 172.21.15.67:0/3181864044 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb60c0a1e50 con 0x55e68a6880e0 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb639995b80 10 client.25223 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb639995b80 10 client.25223 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb639995b80 20 client.25223 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb639995b80 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e68a6f5a90 0x55e68a6f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb639995b80 4 client.25223 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.385+0000 7fb639995b80 1 -- 172.21.15.67:0/3181864044 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e68a427ca0 con 0x55e68a6f5a90 2024-03-30T02:36:12.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.386+0000 7fb61ea00700 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e68a6f5a90 0x55e68a6f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.386+0000 7fb61ea00700 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e68a6f5a90 0x55e68a6f23a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459924f3fc2ed885 server_cookie=86fd616fae1c8ebd in_seq=0 out_seq=0 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.386+0000 7fb608c00700 10 client.25223 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.386+0000 7fb608c00700 1 -- 172.21.15.67:0/3181864044 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e68a427ca0 con 0x55e68a6f5a90 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.386+0000 7fb608c00700 10 client.25223 handle_command_reply: tid=0 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.386+0000 7fb639995b80 1 client.25223 shutdown 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.386+0000 7fb639995b80 1 -- 172.21.15.67:0/3181864044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 msgr2=0x55e68a68e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a68e330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0ca520 tx=0x7fb60c0cc040 comp rx=0 tx=0).stop 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 1 -- 172.21.15.67:0/3181864044 shutdown_connections 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e68a6f5a90 0x55e68a6f23a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e68a688f80 0x55e68a68e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e68a687780 0x55e68a68ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 1 --2- 172.21.15.67:0/3181864044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68a6880e0 0x55e68a68e330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb608c00700 0 client.25223 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:12.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 1 -- 172.21.15.67:0/3181864044 >> 172.21.15.67:0/3181864044 conn(0x55e68a5e6650 msgr2=0x55e68a68f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 1 -- 172.21.15.67:0/3181864044 shutdown_connections 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 1 -- 172.21.15.67:0/3181864044 wait complete. 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb639995b80 20 client.25223 trim_cache size 0 max 16384 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24578, 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 85.670357847999995, 2024-03-30T02:36:12.396 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:12.397 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 85, 2024-03-30T02:36:12.397 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:12.397 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 91.024828196000001 2024-03-30T02:36:12.397 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:12.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb625e00700 1 -- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb6040a11b0 msgr2=0x7fb6040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb625e00700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb6040a11b0 0x7fb6040a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb61800fb80 tx=0x7fb618004b40 comp rx=0 tx=0).stop 2024-03-30T02:36:12.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb625e00700 1 -- 172.21.15.67:0/1053925231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb620074d90 msgr2=0x7fb620199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.387+0000 7fb625e00700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb620074d90 0x7fb620199480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00e9f0 tx=0x7fb60c00ed00 comp rx=0 tx=0).stop 2024-03-30T02:36:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.388+0000 7fb625e00700 1 -- 172.21.15.67:0/1053925231 shutdown_connections 2024-03-30T02:36:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.388+0000 7fb625e00700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fb6040a11b0 0x7fb6040a3670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.388+0000 7fb625e00700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb620108c50 0x7fb6201a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.388+0000 7fb625e00700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6200737b0 0x7fb620198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.388+0000 7fb625e00700 1 --2- 172.21.15.67:0/1053925231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb620074d90 0x7fb620199480 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.388+0000 7fb625e00700 1 -- 172.21.15.67:0/1053925231 >> 172.21.15.67:0/1053925231 conn(0x7fb6200fc2f0 msgr2=0x7fb620104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.388+0000 7fb625e00700 1 -- 172.21.15.67:0/1053925231 shutdown_connections 2024-03-30T02:36:12.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:12.388+0000 7fb625e00700 1 -- 172.21.15.67:0/1053925231 wait complete. 2024-03-30T02:36:12.407 INFO:tasks.cephadm:Setup complete, yielding 2024-03-30T02:36:12.407 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-30T02:36:12.416 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:36:12.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-03-30T02:36:12.781 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.953+0000 7f1bb8e00700 1 -- 172.21.15.67:0/2026930192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 msgr2=0x7f1bb41023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.953+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/2026930192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 0x7f1bb41023e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009990 tx=0x7f1ba0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:13.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.953+0000 7f1bb8e00700 1 -- 172.21.15.67:0/2026930192 shutdown_connections 2024-03-30T02:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.953+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/2026930192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb4102920 0x7f1bb4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.953+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/2026930192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bb4100e00 0x7f1bb4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.953+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/2026930192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 0x7f1bb41023e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.953+0000 7f1bb8e00700 1 -- 172.21.15.67:0/2026930192 >> 172.21.15.67:0/2026930192 conn(0x7f1bb40fc3c0 msgr2=0x7f1bb40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.953+0000 7f1bb8e00700 1 -- 172.21.15.67:0/2026930192 shutdown_connections 2024-03-30T02:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.954+0000 7f1bb8e00700 1 -- 172.21.15.67:0/2026930192 wait complete. 2024-03-30T02:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.954+0000 7f1bb8e00700 1 Processor -- start 2024-03-30T02:36:13.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb8e00700 1 -- start start 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb8e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bb4100e00 0x7f1bb4198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb8e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 0x7f1bb4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb8e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb4102920 0x7f1bb41a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb8e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41a0a40 con 0x7f1bb4101f60 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb8e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41a0b80 con 0x7f1bb4102920 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb8e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41a0d10 con 0x7f1bb4100e00 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 0x7f1bb4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb1e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bb4100e00 0x7f1bb4198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 0x7f1bb4199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57538/0 (socket says 172.21.15.67:57538) 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb1400700 1 -- 172.21.15.67:0/508840540 learned_addr learned my addr 172.21.15.67:0/508840540 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:13.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.955+0000 7f1bb2800700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb4102920 0x7f1bb41a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1bb1e00700 1 -- 172.21.15.67:0/508840540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb4102920 msgr2=0x7f1bb41a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1bb1e00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb4102920 0x7f1bb41a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1bb1e00700 1 -- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 msgr2=0x7f1bb4199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1bb1e00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 0x7f1bb4199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1bb1e00700 1 -- 172.21.15.67:0/508840540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00095f0 con 0x7f1bb4100e00 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1bb1400700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 0x7f1bb4199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1bb1e00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bb4100e00 0x7f1bb4198f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400ea20 tx=0x7f1ba400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1baaa00700 1 -- 172.21.15.67:0/508840540 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba400cbd0 con 0x7f1bb4100e00 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb41a0ff0 con 0x7f1bb4100e00 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1baaa00700 1 -- 172.21.15.67:0/508840540 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ba400ee60 con 0x7f1bb4100e00 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.956+0000 7f1baaa00700 1 -- 172.21.15.67:0/508840540 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4018680 con 0x7f1bb4100e00 2024-03-30T02:36:13.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.957+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb41a1400 con 0x7f1bb4100e00 2024-03-30T02:36:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.958+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb404ea80 con 0x7f1bb4100e00 2024-03-30T02:36:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.960+0000 7f1baaa00700 1 -- 172.21.15.67:0/508840540 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1ba4010430 con 0x7f1bb4100e00 2024-03-30T02:36:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.961+0000 7f1baaa00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1b980a11b0 0x7f1b980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.961+0000 7f1baaa00700 1 -- 172.21.15.67:0/508840540 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f1ba4010e40 con 0x7f1bb4100e00 2024-03-30T02:36:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.961+0000 7f1bb1400700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1b980a11b0 0x7f1b980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.962+0000 7f1bb1400700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1b980a11b0 0x7f1b980a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0006010 tx=0x7f1ba0005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:13.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:13.963+0000 7f1baaa00700 1 -- 172.21.15.67:0/508840540 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1ba4096510 con 0x7f1bb4100e00 2024-03-30T02:36:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.192+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bb4108ed0 con 0x7f1b980a11b0 2024-03-30T02:36:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.194+0000 7f1baaa00700 1 -- 172.21.15.67:0/508840540 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1bb4108ed0 con 0x7f1b980a11b0 2024-03-30T02:36:14.195 INFO:teuthology.orchestra.run.smithi067.stdout:Backend: cephadm 2024-03-30T02:36:14.195 INFO:teuthology.orchestra.run.smithi067.stdout:Available: Yes 2024-03-30T02:36:14.195 INFO:teuthology.orchestra.run.smithi067.stdout:Paused: No 2024-03-30T02:36:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1b980a11b0 msgr2=0x7f1b980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1b980a11b0 0x7f1b980a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0006010 tx=0x7f1ba0005960 comp rx=0 tx=0).stop 2024-03-30T02:36:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bb4100e00 msgr2=0x7f1bb4198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bb4100e00 0x7f1bb4198f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400ea20 tx=0x7f1ba400ed30 comp rx=0 tx=0).stop 2024-03-30T02:36:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 shutdown_connections 2024-03-30T02:36:14.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1b980a11b0 0x7f1b980a3670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:14.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb4102920 0x7f1bb41a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:14.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1bb4100e00 0x7f1bb4198f00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:14.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 --2- 172.21.15.67:0/508840540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1bb4101f60 0x7f1bb4199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:14.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 >> 172.21.15.67:0/508840540 conn(0x7f1bb40fc3c0 msgr2=0x7f1bb41077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:14.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 shutdown_connections 2024-03-30T02:36:14.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:14.196+0000 7f1bb8e00700 1 -- 172.21.15.67:0/508840540 wait complete. 2024-03-30T02:36:14.402 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:14 smithi067 ceph-mon[37808]: pgmap v278: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:36:14.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:14 smithi170 ceph-mon[78798]: pgmap v278: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:36:14.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:14 smithi082 ceph-mon[79176]: pgmap v278: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:36:14.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-03-30T02:36:14.967 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:16.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.121+0000 7f3bcde00700 1 -- 172.21.15.67:0/733356046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bc8108cd0 msgr2=0x7f3bc810b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:16.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.121+0000 7f3bcde00700 1 --2- 172.21.15.67:0/733356046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bc8108cd0 0x7f3bc810b160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000b0a0 tx=0x7f3bc000b3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.121+0000 7f3bcde00700 1 -- 172.21.15.67:0/733356046 shutdown_connections 2024-03-30T02:36:16.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.121+0000 7f3bcde00700 1 --2- 172.21.15.67:0/733356046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bc8108cd0 0x7f3bc810b160 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.121+0000 7f3bcde00700 1 --2- 172.21.15.67:0/733356046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc8074de0 0x7f3bc8073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.121+0000 7f3bcde00700 1 --2- 172.21.15.67:0/733356046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc8073810 0x7f3bc8073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.121+0000 7f3bcde00700 1 -- 172.21.15.67:0/733356046 >> 172.21.15.67:0/733356046 conn(0x7f3bc80fc3c0 msgr2=0x7f3bc80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:16.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.121+0000 7f3bcde00700 1 -- 172.21.15.67:0/733356046 shutdown_connections 2024-03-30T02:36:16.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.121+0000 7f3bcde00700 1 -- 172.21.15.67:0/733356046 wait complete. 2024-03-30T02:36:16.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.122+0000 7f3bcde00700 1 Processor -- start 2024-03-30T02:36:16.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.122+0000 7f3bcde00700 1 -- start start 2024-03-30T02:36:16.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bcde00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc8073810 0x7f3bc8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:16.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bcde00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bc8074de0 0x7f3bc81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bcde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc8108cd0 0x7f3bc81a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bcde00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc81a0bc0 con 0x7f3bc8074de0 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bcde00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc81a0d00 con 0x7f3bc8073810 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bcde00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc81a0e90 con 0x7f3bc8108cd0 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc6a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc8073810 0x7f3bc8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc6a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc8073810 0x7f3bc8199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:35794/0 (socket says 172.21.15.67:35794) 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc6a00700 1 -- 172.21.15.67:0/1906694201 learned_addr learned my addr 172.21.15.67:0/1906694201 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc6000700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bc8074de0 0x7f3bc81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc7400700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc8108cd0 0x7f3bc81a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc6a00700 1 -- 172.21.15.67:0/1906694201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc8108cd0 msgr2=0x7f3bc81a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc6a00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc8108cd0 0x7f3bc81a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc6a00700 1 -- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bc8074de0 msgr2=0x7f3bc81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc6a00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bc8074de0 0x7f3bc81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc6a00700 1 -- 172.21.15.67:0/1906694201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc0009d00 con 0x7f3bc8073810 2024-03-30T02:36:16.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.123+0000 7f3bc7400700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc8108cd0 0x7f3bc81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:36:16.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.124+0000 7f3bc6a00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc8073810 0x7f3bc8199080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800e970 tx=0x7f3bb800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:16.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.124+0000 7f3bcca00700 1 -- 172.21.15.67:0/1906694201 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8009790 con 0x7f3bc8073810 2024-03-30T02:36:16.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.124+0000 7f3bcca00700 1 -- 172.21.15.67:0/1906694201 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bb800eec0 con 0x7f3bc8073810 2024-03-30T02:36:16.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.124+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc81a1170 con 0x7f3bc8073810 2024-03-30T02:36:16.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.124+0000 7f3bcca00700 1 -- 172.21.15.67:0/1906694201 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8005580 con 0x7f3bc8073810 2024-03-30T02:36:16.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.124+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc81a16c0 con 0x7f3bc8073810 2024-03-30T02:36:16.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.125+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc8107a90 con 0x7f3bc8073810 2024-03-30T02:36:16.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.127+0000 7f3bcca00700 1 -- 172.21.15.67:0/1906694201 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3bb800cbd0 con 0x7f3bc8073810 2024-03-30T02:36:16.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.128+0000 7f3bcca00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3bac0a1230 0x7f3bac0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:16.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.129+0000 7f3bcca00700 1 -- 172.21.15.67:0/1906694201 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3bb8014070 con 0x7f3bc8073810 2024-03-30T02:36:16.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.129+0000 7f3bc6000700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3bac0a1230 0x7f3bac0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:16.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.129+0000 7f3bc6000700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3bac0a1230 0x7f3bac0a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8074b10 tx=0x7f3bb4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:16.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.131+0000 7f3bcca00700 1 -- 172.21.15.67:0/1906694201 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3bb8096600 con 0x7f3bc8073810 2024-03-30T02:36:16.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:16 smithi067 ceph-mon[37808]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:16.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:16 smithi067 ceph-mon[37808]: pgmap v279: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-30T02:36:16.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.354+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bc802cd50 con 0x7f3bac0a1230 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.368+0000 7f3bcca00700 1 -- 172.21.15.67:0/1906694201 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f3bc802cd50 con 0x7f3bac0a1230 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.a smithi067 running (97s) 20s ago 96s - - 18.2.2-1018-g470100a4 e7c444a08cf7 0596a55c05a0 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.b smithi082 running (95s) 29s ago 94s 29.9M - 18.2.2-1018-g470100a4 e7c444a08cf7 0aa26d63945d 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.c smithi170 running (105s) 29s ago 105s 19.7M - 18.2.2-1018-g470100a4 e7c444a08cf7 be0e85205633 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.d smithi067 running (110s) 20s ago 110s - - 18.2.2-1018-g470100a4 e7c444a08cf7 5a961c55d2b2 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.e smithi082 running (108s) 29s ago 108s 19.8M - 18.2.2-1018-g470100a4 e7c444a08cf7 a4cfa7ba5473 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.f smithi170 running (107s) 29s ago 106s 18.4M - 18.2.2-1018-g470100a4 e7c444a08cf7 77c74bb582fa 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.g smithi067 running (99s) 20s ago 99s - - 18.2.2-1018-g470100a4 e7c444a08cf7 f8df9238ab21 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.h smithi082 running (103s) 29s ago 103s 19.6M - 18.2.2-1018-g470100a4 e7c444a08cf7 e38f3d29778b 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.i smithi170 running (101s) 29s ago 101s 20.8M - 18.2.2-1018-g470100a4 e7c444a08cf7 74390ca9f25c 2024-03-30T02:36:16.368 INFO:teuthology.orchestra.run.smithi067.stdout:mds.j smithi067 running (112s) 20s ago 112s - - 18.2.2-1018-g470100a4 e7c444a08cf7 5208eb05e573 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:mds.k smithi082 running (115s) 29s ago 114s 19.4M - 18.2.2-1018-g470100a4 e7c444a08cf7 8455de682c82 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:mds.l smithi170 running (116s) 29s ago 116s 16.9M - 18.2.2-1018-g470100a4 e7c444a08cf7 5c5992bdfbf5 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.x smithi067 *:9283,8765 running (7m) 20s ago 7m - - 18.2.2-1018-g470100a4 e7c444a08cf7 e4f3bcb9efcd 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.y smithi082 *:8443,8765 running (5m) 29s ago 5m 457M - 18.2.2-1018-g470100a4 e7c444a08cf7 31212d440ea0 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.z smithi170 *:8443,8765 running (5m) 29s ago 5m 459M - 18.2.2-1018-g470100a4 e7c444a08cf7 51a960dd8f63 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:mon.a smithi067 running (7m) 20s ago 7m - 2048M 18.2.2-1018-g470100a4 e7c444a08cf7 e3ad484b6d89 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:mon.b smithi082 running (6m) 29s ago 6m 61.1M 2048M 18.2.2-1018-g470100a4 e7c444a08cf7 263ba4bef7c2 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:mon.c smithi170 running (6m) 29s ago 6m 61.0M 2048M 18.2.2-1018-g470100a4 e7c444a08cf7 0c9961cc03bf 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 smithi067 running (5m) 20s ago 5m - 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 0292ed5547d3 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 smithi082 running (5m) 29s ago 5m 75.7M 4278M 18.2.2-1018-g470100a4 e7c444a08cf7 9d5817b26467 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 smithi170 running (5m) 29s ago 5m 74.9M 4278M 18.2.2-1018-g470100a4 e7c444a08cf7 d546e52b139d 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 smithi067 running (4m) 20s ago 4m - 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 ae4693f7f4bb 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 smithi082 running (4m) 29s ago 4m 135M 4278M 18.2.2-1018-g470100a4 e7c444a08cf7 20602e042191 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 smithi170 running (4m) 29s ago 4m 140M 4278M 18.2.2-1018-g470100a4 e7c444a08cf7 e1a07afcdb84 2024-03-30T02:36:16.369 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 smithi067 running (3m) 20s ago 3m - 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 c4a1a05636ec 2024-03-30T02:36:16.370 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 smithi082 running (3m) 29s ago 3m 64.4M 4278M 18.2.2-1018-g470100a4 e7c444a08cf7 634cdcfaf555 2024-03-30T02:36:16.370 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 smithi170 running (3m) 29s ago 3m 61.2M 4278M 18.2.2-1018-g470100a4 e7c444a08cf7 753049d18898 2024-03-30T02:36:16.370 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 smithi067 running (2m) 20s ago 2m - 4317M 18.2.2-1018-g470100a4 e7c444a08cf7 c59f6683a043 2024-03-30T02:36:16.370 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 smithi082 running (2m) 29s ago 2m 132M 4278M 18.2.2-1018-g470100a4 e7c444a08cf7 39ed6fbb079a 2024-03-30T02:36:16.370 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 smithi170 running (2m) 29s ago 2m 57.4M 4278M 18.2.2-1018-g470100a4 e7c444a08cf7 06bb7577e477 2024-03-30T02:36:16.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3bac0a1230 msgr2=0x7f3bac0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3bac0a1230 0x7f3bac0a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8074b10 tx=0x7f3bb4009260 comp rx=0 tx=0).stop 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc8073810 msgr2=0x7f3bc8199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc8073810 0x7f3bc8199080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800e970 tx=0x7f3bb800ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 shutdown_connections 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3bac0a1230 0x7f3bac0a36f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bc8108cd0 0x7f3bc81a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc8073810 0x7f3bc8199080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 --2- 172.21.15.67:0/1906694201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3bc8074de0 0x7f3bc81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 >> 172.21.15.67:0/1906694201 conn(0x7f3bc80fc3c0 msgr2=0x7f3bc8104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 shutdown_connections 2024-03-30T02:36:16.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:16.370+0000 7f3bcde00700 1 -- 172.21.15.67:0/1906694201 wait complete. 2024-03-30T02:36:16.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:16 smithi170 ceph-mon[78798]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:16.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:16 smithi170 ceph-mon[78798]: pgmap v279: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-30T02:36:16.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:16 smithi082 ceph-mon[79176]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:16.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:16 smithi082 ceph-mon[79176]: pgmap v279: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-30T02:36:16.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-03-30T02:36:17.118 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:17 smithi082 ceph-mon[79176]: from='client.25235 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:17.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:17 smithi082 ceph-mon[79176]: pgmap v280: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-30T02:36:17.651 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:17 smithi067 ceph-mon[37808]: from='client.25235 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:17.651 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:17 smithi067 ceph-mon[37808]: pgmap v280: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-30T02:36:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:17 smithi170 ceph-mon[78798]: from='client.25235 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:17 smithi170 ceph-mon[78798]: pgmap v280: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-30T02:36:18.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.306+0000 7f6983000700 1 -- 172.21.15.67:0/1802083492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 msgr2=0x7f697c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:18.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.306+0000 7f6983000700 1 --2- 172.21.15.67:0/1802083492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 0x7f697c101220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f696c009990 tx=0x7f696c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.307+0000 7f6983000700 1 -- 172.21.15.67:0/1802083492 shutdown_connections 2024-03-30T02:36:18.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.307+0000 7f6983000700 1 --2- 172.21.15.67:0/1802083492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 0x7f697c101220 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.307+0000 7f6983000700 1 --2- 172.21.15.67:0/1802083492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f697c102920 0x7f697c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.307+0000 7f6983000700 1 --2- 172.21.15.67:0/1802083492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697c101f60 0x7f697c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.307+0000 7f6983000700 1 -- 172.21.15.67:0/1802083492 >> 172.21.15.67:0/1802083492 conn(0x7f697c0fc3c0 msgr2=0x7f697c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:18.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.307+0000 7f6983000700 1 -- 172.21.15.67:0/1802083492 shutdown_connections 2024-03-30T02:36:18.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.307+0000 7f6983000700 1 -- 172.21.15.67:0/1802083492 wait complete. 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.308+0000 7f6983000700 1 Processor -- start 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.308+0000 7f6983000700 1 -- start start 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.308+0000 7f6983000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 0x7f697c196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.308+0000 7f6983000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f697c101f60 0x7f697c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.308+0000 7f6983000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697c102920 0x7f697c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.308+0000 7f6983000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c19e890 con 0x7f697c100e00 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.308+0000 7f6983000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c19e9d0 con 0x7f697c102920 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.308+0000 7f6983000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c19eb90 con 0x7f697c101f60 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.308+0000 7f697be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 0x7f697c196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f697b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f697c101f60 0x7f697c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f697be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 0x7f697c196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60698/0 (socket says 172.21.15.67:60698) 2024-03-30T02:36:18.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f697be00700 1 -- 172.21.15.67:0/656643222 learned_addr learned my addr 172.21.15.67:0/656643222 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:18.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f6981000700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697c102920 0x7f697c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:18.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f697be00700 1 -- 172.21.15.67:0/656643222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f697c101f60 msgr2=0x7f697c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:18.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f697be00700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f697c101f60 0x7f697c1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f697be00700 1 -- 172.21.15.67:0/656643222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697c102920 msgr2=0x7f697c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:18.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f697be00700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697c102920 0x7f697c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f697be00700 1 -- 172.21.15.67:0/656643222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c0095f0 con 0x7f697c100e00 2024-03-30T02:36:18.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f697be00700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 0x7f697c196d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f696c000c00 tx=0x7f696c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:18.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.309+0000 7f6978c00700 1 -- 172.21.15.67:0/656643222 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c01d070 con 0x7f697c100e00 2024-03-30T02:36:18.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.310+0000 7f6978c00700 1 -- 172.21.15.67:0/656643222 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f696c021410 con 0x7f697c100e00 2024-03-30T02:36:18.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.310+0000 7f6978c00700 1 -- 172.21.15.67:0/656643222 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c017650 con 0x7f697c100e00 2024-03-30T02:36:18.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.310+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c19ee10 con 0x7f697c100e00 2024-03-30T02:36:18.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.310+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c19f170 con 0x7f697c100e00 2024-03-30T02:36:18.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.311+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c1082d0 con 0x7f697c100e00 2024-03-30T02:36:18.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.313+0000 7f6978c00700 1 -- 172.21.15.67:0/656643222 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f696c021580 con 0x7f697c100e00 2024-03-30T02:36:18.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.314+0000 7f6978c00700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f69600a11b0 0x7f69600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:18.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.314+0000 7f6978c00700 1 -- 172.21.15.67:0/656643222 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f696c0ce1a0 con 0x7f697c100e00 2024-03-30T02:36:18.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.314+0000 7f697b400700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f69600a11b0 0x7f69600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:18.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.315+0000 7f697b400700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f69600a11b0 0x7f69600a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6968009c60 tx=0x7f6968009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:18.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.318+0000 7f6978c00700 1 -- 172.21.15.67:0/656643222 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f696c097500 con 0x7f697c100e00 2024-03-30T02:36:18.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.542+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f697c02cd70 con 0x7f69600a11b0 2024-03-30T02:36:18.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.548+0000 7f6978c00700 1 -- 172.21.15.67:0/656643222 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f697c02cd70 con 0x7f69600a11b0 2024-03-30T02:36:18.548 INFO:teuthology.orchestra.run.smithi067.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-30T02:36:18.548 INFO:teuthology.orchestra.run.smithi067.stdout:mds.all 12/12 31s ago 2m smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi082=b;smithi082=e;smithi082=h;smithi082=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-03-30T02:36:18.548 INFO:teuthology.orchestra.run.smithi067.stdout:mgr 3/3 31s ago 5m smithi067=x;smithi082=y;smithi170=z;count:3 2024-03-30T02:36:18.548 INFO:teuthology.orchestra.run.smithi067.stdout:mon 3/3 31s ago 6m smithi067:172.21.15.67=a;smithi082:172.21.15.82=b;smithi170:172.21.15.170=c;count:3 2024-03-30T02:36:18.548 INFO:teuthology.orchestra.run.smithi067.stdout:osd 12 31s ago - 2024-03-30T02:36:18.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f69600a11b0 msgr2=0x7f69600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:18.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f69600a11b0 0x7f69600a3670 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6968009c60 tx=0x7f6968009260 comp rx=0 tx=0).stop 2024-03-30T02:36:18.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 msgr2=0x7f697c196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:18.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 0x7f697c196d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f696c000c00 tx=0x7f696c005740 comp rx=0 tx=0).stop 2024-03-30T02:36:18.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 shutdown_connections 2024-03-30T02:36:18.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f69600a11b0 0x7f69600a3670 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f697c100e00 0x7f697c196d90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f697c102920 0x7f697c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 --2- 172.21.15.67:0/656643222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f697c101f60 0x7f697c1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:18.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 >> 172.21.15.67:0/656643222 conn(0x7f697c0fc3c0 msgr2=0x7f697c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:18.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 shutdown_connections 2024-03-30T02:36:18.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:18.550+0000 7f6983000700 1 -- 172.21.15.67:0/656643222 wait complete. 2024-03-30T02:36:18.954 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-03-30T02:36:19.324 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:20.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:20 smithi067 ceph-mon[37808]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:20.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:20 smithi067 ceph-mon[37808]: pgmap v281: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-30T02:36:20.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:20 smithi170 ceph-mon[78798]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:20.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:20 smithi170 ceph-mon[78798]: pgmap v281: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-30T02:36:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:20 smithi082 ceph-mon[79176]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:20.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:20 smithi082 ceph-mon[79176]: pgmap v281: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-30T02:36:20.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.555+0000 7f9ee9600700 1 -- 172.21.15.67:0/1814253306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 msgr2=0x7f9ee4108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:20.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.555+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1814253306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 0x7f9ee4108f50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00b0a0 tx=0x7f9edc00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.556+0000 7f9ee9600700 1 -- 172.21.15.67:0/1814253306 shutdown_connections 2024-03-30T02:36:20.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.556+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1814253306 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 0x7f9ee4108f50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.556+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1814253306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ee41018e0 0x7f9ee4103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.556+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1814253306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee4104210 0x7f9ee4106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.556+0000 7f9ee9600700 1 -- 172.21.15.67:0/1814253306 >> 172.21.15.67:0/1814253306 conn(0x7f9ee40fb2b0 msgr2=0x7f9ee40fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:20.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.556+0000 7f9ee9600700 1 -- 172.21.15.67:0/1814253306 shutdown_connections 2024-03-30T02:36:20.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.556+0000 7f9ee9600700 1 -- 172.21.15.67:0/1814253306 wait complete. 2024-03-30T02:36:20.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.557+0000 7f9ee9600700 1 Processor -- start 2024-03-30T02:36:20.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.557+0000 7f9ee9600700 1 -- start start 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee9600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee41018e0 0x7f9ee4196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee9600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ee4104210 0x7f9ee4197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee9600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 0x7f9ee419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee9600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee419e940 con 0x7f9ee4106b40 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee9600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee419ea80 con 0x7f9ee41018e0 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee9600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee419ec10 con 0x7f9ee4104210 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 0x7f9ee419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee2a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee41018e0 0x7f9ee4196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee2000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ee4104210 0x7f9ee4197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 0x7f9ee419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60718/0 (socket says 172.21.15.67:60718) 2024-03-30T02:36:20.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee2000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ee4104210 0x7f9ee4197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48278/0 (socket says 172.21.15.67:48278) 2024-03-30T02:36:20.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.558+0000 7f9ee3400700 1 -- 172.21.15.67:0/1195543332 learned_addr learned my addr 172.21.15.67:0/1195543332 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:20.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9ee3400700 1 -- 172.21.15.67:0/1195543332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ee4104210 msgr2=0x7f9ee4197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:20.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9ee3400700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ee4104210 0x7f9ee4197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9ee3400700 1 -- 172.21.15.67:0/1195543332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee41018e0 msgr2=0x7f9ee4196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:20.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9ee3400700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee41018e0 0x7f9ee4196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9ee3400700 1 -- 172.21.15.67:0/1195543332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc009d00 con 0x7f9ee4106b40 2024-03-30T02:36:20.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9ee3400700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 0x7f9ee419e370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9edc007960 tx=0x7f9edc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:20.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9edb400700 1 -- 172.21.15.67:0/1195543332 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00e040 con 0x7f9ee4106b40 2024-03-30T02:36:20.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee40ff6d0 con 0x7f9ee4106b40 2024-03-30T02:36:20.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9edb400700 1 -- 172.21.15.67:0/1195543332 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9edc004c50 con 0x7f9ee4106b40 2024-03-30T02:36:20.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9edb400700 1 -- 172.21.15.67:0/1195543332 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0052a0 con 0x7f9ee4106b40 2024-03-30T02:36:20.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.559+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee40ffbd0 con 0x7f9ee4106b40 2024-03-30T02:36:20.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.560+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee40fce30 con 0x7f9ee4106b40 2024-03-30T02:36:20.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.562+0000 7f9edb400700 1 -- 172.21.15.67:0/1195543332 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f9edc01d470 con 0x7f9ee4106b40 2024-03-30T02:36:20.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.564+0000 7f9edb400700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9ec80a1230 0x7f9ec80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:20.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.564+0000 7f9edb400700 1 -- 172.21.15.67:0/1195543332 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f9edc0ce300 con 0x7f9ee4106b40 2024-03-30T02:36:20.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.564+0000 7f9ee2a00700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9ec80a1230 0x7f9ec80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:20.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.565+0000 7f9ee2a00700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9ec80a1230 0x7f9ec80a36f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005950 tx=0x7f9ed400f820 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:20.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.567+0000 7f9edb400700 1 -- 172.21.15.67:0/1195543332 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9edc097660 con 0x7f9ee4106b40 2024-03-30T02:36:20.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.786+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ee40611f0 con 0x7f9ec80a1230 2024-03-30T02:36:20.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.788+0000 7f9edb400700 1 -- 172.21.15.67:0/1195543332 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f9ee40611f0 con 0x7f9ec80a1230 2024-03-30T02:36:20.788 INFO:teuthology.orchestra.run.smithi067.stdout:HOST ADDR LABELS STATUS 2024-03-30T02:36:20.788 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 172.21.15.67 2024-03-30T02:36:20.788 INFO:teuthology.orchestra.run.smithi067.stdout:smithi082 172.21.15.82 2024-03-30T02:36:20.788 INFO:teuthology.orchestra.run.smithi067.stdout:smithi170 172.21.15.170 2024-03-30T02:36:20.788 INFO:teuthology.orchestra.run.smithi067.stdout:3 hosts in cluster 2024-03-30T02:36:20.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9ec80a1230 msgr2=0x7f9ec80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:20.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9ec80a1230 0x7f9ec80a36f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005950 tx=0x7f9ed400f820 comp rx=0 tx=0).stop 2024-03-30T02:36:20.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 msgr2=0x7f9ee419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:20.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 0x7f9ee419e370 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9edc007960 tx=0x7f9edc00bab0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 shutdown_connections 2024-03-30T02:36:20.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f9ec80a1230 0x7f9ec80a36f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9ee4106b40 0x7f9ee419e370 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee41018e0 0x7f9ee4196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 --2- 172.21.15.67:0/1195543332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ee4104210 0x7f9ee4197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:20.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 >> 172.21.15.67:0/1195543332 conn(0x7f9ee40fb2b0 msgr2=0x7f9ee4102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:20.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 shutdown_connections 2024-03-30T02:36:20.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:20.790+0000 7f9ee9600700 1 -- 172.21.15.67:0/1195543332 wait complete. 2024-03-30T02:36:21.220 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-03-30T02:36:21.591 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:22 smithi067 ceph-mon[37808]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:22.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:22 smithi067 ceph-mon[37808]: pgmap v282: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-30T02:36:22.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:22 smithi170 ceph-mon[78798]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:22.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:22 smithi170 ceph-mon[78798]: pgmap v282: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-30T02:36:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:22 smithi082 ceph-mon[79176]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:22.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:22 smithi082 ceph-mon[79176]: pgmap v282: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-30T02:36:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.767+0000 7f354f400700 1 -- 172.21.15.67:0/799020542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35481041b0 msgr2=0x7f35481065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.767+0000 7f354f400700 1 --2- 172.21.15.67:0/799020542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35481041b0 0x7f35481065a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3534009990 tx=0x7f3534009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.767+0000 7f354f400700 1 -- 172.21.15.67:0/799020542 shutdown_connections 2024-03-30T02:36:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.767+0000 7f354f400700 1 --2- 172.21.15.67:0/799020542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3548106ae0 0x7f3548108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.767+0000 7f354f400700 1 --2- 172.21.15.67:0/799020542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3548101880 0x7f3548103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.767+0000 7f354f400700 1 --2- 172.21.15.67:0/799020542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35481041b0 0x7f35481065a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.767+0000 7f354f400700 1 -- 172.21.15.67:0/799020542 >> 172.21.15.67:0/799020542 conn(0x7f35480fb2b0 msgr2=0x7f35480fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.768+0000 7f354f400700 1 -- 172.21.15.67:0/799020542 shutdown_connections 2024-03-30T02:36:22.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.768+0000 7f354f400700 1 -- 172.21.15.67:0/799020542 wait complete. 2024-03-30T02:36:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354f400700 1 Processor -- start 2024-03-30T02:36:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354f400700 1 -- start start 2024-03-30T02:36:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3548101880 0x7f3548101190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35481041b0 0x7f35480ff830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3548106ae0 0x7f35480ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354f400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3548100510 con 0x7f3548101880 2024-03-30T02:36:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354f400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3548100680 con 0x7f35481041b0 2024-03-30T02:36:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354f400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35481a0dc0 con 0x7f3548106ae0 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354ca00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3548101880 0x7f3548101190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354d400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3548106ae0 0x7f35480ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354ca00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3548101880 0x7f3548101190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60740/0 (socket says 172.21.15.67:60740) 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.769+0000 7f354ca00700 1 -- 172.21.15.67:0/548662518 learned_addr learned my addr 172.21.15.67:0/548662518 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.770+0000 7f3547e00700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35481041b0 0x7f35480ff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.770+0000 7f354d400700 1 -- 172.21.15.67:0/548662518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35481041b0 msgr2=0x7f35480ff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.770+0000 7f354d400700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35481041b0 0x7f35480ff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.770+0000 7f354d400700 1 -- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3548101880 msgr2=0x7f3548101190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.770+0000 7f354d400700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3548101880 0x7f3548101190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.770+0000 7f354d400700 1 -- 172.21.15.67:0/548662518 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340095f0 con 0x7f3548106ae0 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.770+0000 7f354ca00700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3548101880 0x7f3548101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:22.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.770+0000 7f354d400700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3548106ae0 0x7f35480ffd70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f354000ef90 tx=0x7f354000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.770+0000 7f3545600700 1 -- 172.21.15.67:0/548662518 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540018070 con 0x7f3548106ae0 2024-03-30T02:36:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.771+0000 7f3545600700 1 -- 172.21.15.67:0/548662518 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f354000f040 con 0x7f3548106ae0 2024-03-30T02:36:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.771+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35481a0ff0 con 0x7f3548106ae0 2024-03-30T02:36:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.771+0000 7f3545600700 1 -- 172.21.15.67:0/548662518 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354000cc30 con 0x7f3548106ae0 2024-03-30T02:36:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.771+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35481a1460 con 0x7f3548106ae0 2024-03-30T02:36:22.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.772+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35480fce30 con 0x7f3548106ae0 2024-03-30T02:36:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.774+0000 7f3545600700 1 -- 172.21.15.67:0/548662518 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3540003a00 con 0x7f3548106ae0 2024-03-30T02:36:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.775+0000 7f3545600700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35300a1230 0x7f35300a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.775+0000 7f3545600700 1 -- 172.21.15.67:0/548662518 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f35400cd210 con 0x7f3548106ae0 2024-03-30T02:36:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.775+0000 7f354ca00700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35300a1230 0x7f35300a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.776+0000 7f354ca00700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35300a1230 0x7f35300a36f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f353c011550 tx=0x7f353c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.777+0000 7f3545600700 1 -- 172.21.15.67:0/548662518 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3540096570 con 0x7f3548106ae0 2024-03-30T02:36:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.996+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f354802cd20 con 0x7f35300a1230 2024-03-30T02:36:22.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:22.998+0000 7f3545600700 1 -- 172.21.15.67:0/548662518 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f354802cd20 con 0x7f35300a1230 2024-03-30T02:36:22.998 INFO:teuthology.orchestra.run.smithi067.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-30T02:36:22.999 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440042400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-30T02:36:22.999 INFO:teuthology.orchestra.run.smithi067.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-30T02:36:22.999 INFO:teuthology.orchestra.run.smithi067.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35300a1230 msgr2=0x7f35300a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35300a1230 0x7f35300a36f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f353c011550 tx=0x7f353c009260 comp rx=0 tx=0).stop 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3548106ae0 msgr2=0x7f35480ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3548106ae0 0x7f35480ffd70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f354000ef90 tx=0x7f354000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 shutdown_connections 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f35300a1230 0x7f35300a36f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3548101880 0x7f3548101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3548106ae0 0x7f35480ffd70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 --2- 172.21.15.67:0/548662518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35481041b0 0x7f35480ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 >> 172.21.15.67:0/548662518 conn(0x7f35480fb2b0 msgr2=0x7f3548103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:23.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 shutdown_connections 2024-03-30T02:36:23.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:23.001+0000 7f354f400700 1 -- 172.21.15.67:0/548662518 wait complete. 2024-03-30T02:36:23.441 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-30T02:36:23.452 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:36:23.453 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-03-30T02:36:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:24.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:24 smithi067 ceph-mon[37808]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:24.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:24 smithi067 ceph-mon[37808]: pgmap v283: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:36:24.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:24 smithi170 ceph-mon[78798]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:24.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:24 smithi170 ceph-mon[78798]: pgmap v283: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:36:24.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:24 smithi082 ceph-mon[79176]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:24.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:24 smithi082 ceph-mon[79176]: pgmap v283: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:36:25.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.007+0000 7f3c75e00700 1 -- 172.21.15.67:0/3812413610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70101f60 msgr2=0x7f3c701023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:25.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.007+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3812413610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70101f60 0x7f3c701023e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009990 tx=0x7f3c5c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.007+0000 7f3c75e00700 1 -- 172.21.15.67:0/3812413610 shutdown_connections 2024-03-30T02:36:25.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.007+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3812413610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70102920 0x7f3c70109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.007+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3812413610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c70100e00 0x7f3c70101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.007+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3812413610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70101f60 0x7f3c701023e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.007+0000 7f3c75e00700 1 -- 172.21.15.67:0/3812413610 >> 172.21.15.67:0/3812413610 conn(0x7f3c700fc3c0 msgr2=0x7f3c700fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:25.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.007+0000 7f3c75e00700 1 -- 172.21.15.67:0/3812413610 shutdown_connections 2024-03-30T02:36:25.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.007+0000 7f3c75e00700 1 -- 172.21.15.67:0/3812413610 wait complete. 2024-03-30T02:36:25.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.008+0000 7f3c75e00700 1 Processor -- start 2024-03-30T02:36:25.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.008+0000 7f3c75e00700 1 -- start start 2024-03-30T02:36:25.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c75e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70100e00 0x7f3c70198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:25.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c75e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c70101f60 0x7f3c701994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c75e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102920 0x7f3c701a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c75e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c701a0a70 con 0x7f3c70100e00 2024-03-30T02:36:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c75e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c701a0bb0 con 0x7f3c70101f60 2024-03-30T02:36:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c75e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c701a0d40 con 0x7f3c70102920 2024-03-30T02:36:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70100e00 0x7f3c70198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70100e00 0x7f3c70198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60758/0 (socket says 172.21.15.67:60758) 2024-03-30T02:36:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6ea00700 1 -- 172.21.15.67:0/3893648145 learned_addr learned my addr 172.21.15.67:0/3893648145 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6e000700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c70101f60 0x7f3c701994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6f400700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102920 0x7f3c701a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6ea00700 1 -- 172.21.15.67:0/3893648145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102920 msgr2=0x7f3c701a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6ea00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102920 0x7f3c701a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6ea00700 1 -- 172.21.15.67:0/3893648145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c70101f60 msgr2=0x7f3c701994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6ea00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c70101f60 0x7f3c701994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6ea00700 1 -- 172.21.15.67:0/3893648145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c0095f0 con 0x7f3c70100e00 2024-03-30T02:36:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.009+0000 7f3c6e000700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c70101f60 0x7f3c701994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.010+0000 7f3c6ea00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70100e00 0x7f3c70198f80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000e9d0 tx=0x7f3c6000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.010+0000 7f3c74a00700 1 -- 172.21.15.67:0/3893648145 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60009790 con 0x7f3c70100e00 2024-03-30T02:36:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.010+0000 7f3c74a00700 1 -- 172.21.15.67:0/3893648145 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c6000ee70 con 0x7f3c70100e00 2024-03-30T02:36:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.010+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c701a0fc0 con 0x7f3c70100e00 2024-03-30T02:36:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.010+0000 7f3c74a00700 1 -- 172.21.15.67:0/3893648145 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60010ea0 con 0x7f3c70100e00 2024-03-30T02:36:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.010+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c701a1480 con 0x7f3c70100e00 2024-03-30T02:36:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.011+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7004ea80 con 0x7f3c70100e00 2024-03-30T02:36:25.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.013+0000 7f3c74a00700 1 -- 172.21.15.67:0/3893648145 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f3c6000cbd0 con 0x7f3c70100e00 2024-03-30T02:36:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.014+0000 7f3c74a00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3c540a1230 0x7f3c540a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.014+0000 7f3c74a00700 1 -- 172.21.15.67:0/3893648145 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8999+0+0 (secure 0 0 0) 0x7f3c60014070 con 0x7f3c70100e00 2024-03-30T02:36:25.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.015+0000 7f3c6e000700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3c540a1230 0x7f3c540a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:25.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.015+0000 7f3c6e000700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3c540a1230 0x7f3c540a36f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00b2e0 tx=0x7f3c5c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:25.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.019+0000 7f3c74a00700 1 -- 172.21.15.67:0/3893648145 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3c600529f0 con 0x7f3c70100e00 2024-03-30T02:36:25.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.277+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3c7010c870 con 0x7f3c70100e00 2024-03-30T02:36:25.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.278+0000 7f3c74a00700 1 -- 172.21.15.67:0/3893648145 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+2923 (secure 0 0 0) 0x7f3c60096190 con 0x7f3c70100e00 2024-03-30T02:36:25.278 INFO:teuthology.orchestra.run.smithi067.stdout:e28 2024-03-30T02:36:25.278 INFO:teuthology.orchestra.run.smithi067.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-30T02:36:25.278 INFO:teuthology.orchestra.run.smithi067.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-30T02:36:25.278 INFO:teuthology.orchestra.run.smithi067.stdout:legacy client fscid: 1 2024-03-30T02:36:25.278 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:25.278 INFO:teuthology.orchestra.run.smithi067.stdout:Filesystem 'cephfs' (1) 2024-03-30T02:36:25.278 INFO:teuthology.orchestra.run.smithi067.stdout:fs_name cephfs 2024-03-30T02:36:25.278 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 28 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-03-30T02:34:46.713132+0000 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-03-30T02:35:20.119112+0000 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:tableserver 0 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:root 0 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:session_timeout 300 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:session_autoclose 300 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:max_file_size 1099511627776 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:max_xattr_size 65536 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:required_client_features {} 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure 0 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure_osd_epoch 0 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:max_mds 5 2024-03-30T02:36:25.279 INFO:teuthology.orchestra.run.smithi067.stdout:in 0,1,2,3,4 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:up {0=24578,1=24560,2=24541,3=24539,4=24520} 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:failed 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:damaged 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:stopped 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:data_pools [3,4] 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:metadata_pool 2 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:inline_data disabled 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:balancer 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:bal_rank_mask -1 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:standby_count_wanted 0 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b{0:24578} state up:active seq 3 addr [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.f{0:24514} state up:standby-replay seq 1 addr [v2:172.21.15.170:6834/1685909437,v1:172.21.15.170:6835/1685909437] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.g{1:24560} state up:active seq 11 addr [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.e{1:24512} state up:standby-replay seq 1 addr [v2:172.21.15.82:6834/3401372629,v1:172.21.15.82:6835/3401372629] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.280 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.i{2:24541} state up:active seq 12 addr [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.k{2:24476} state up:standby-replay seq 1 addr [v2:172.21.15.82:6832/1365991767,v1:172.21.15.82:6833/1365991767] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.h{3:24539} state up:active seq 13 addr [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.l{3:24466} state up:standby-replay seq 1 addr [v2:172.21.15.170:6832/3749225376,v1:172.21.15.170:6833/3749225376] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c{4:24520} state up:active seq 13 addr [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a{4:14526} state up:standby-replay seq 1 addr [v2:172.21.15.67:6840/853027854,v1:172.21.15.67:6841/853027854] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout:Standby daemons: 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.j{-1:14463} state up:standby seq 1 addr [v2:172.21.15.67:6834/2327446464,v1:172.21.15.67:6835/2327446464] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.281 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d{-1:14475} state up:standby seq 1 addr [v2:172.21.15.67:6836/437498289,v1:172.21.15.67:6837/437498289] compat {c=[1],r=[1],i=[7ff]}] 2024-03-30T02:36:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3c540a1230 msgr2=0x7f3c540a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3c540a1230 0x7f3c540a36f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00b2e0 tx=0x7f3c5c005fb0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70100e00 msgr2=0x7f3c70198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70100e00 0x7f3c70198f80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000e9d0 tx=0x7f3c6000ece0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 shutdown_connections 2024-03-30T02:36:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f3c540a1230 0x7f3c540a36f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c70100e00 0x7f3c70198f80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c70102920 0x7f3c701a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 --2- 172.21.15.67:0/3893648145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c70101f60 0x7f3c701994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 >> 172.21.15.67:0/3893648145 conn(0x7f3c700fc3c0 msgr2=0x7f3c70072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 shutdown_connections 2024-03-30T02:36:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:25.280+0000 7f3c75e00700 1 -- 172.21.15.67:0/3893648145 wait complete. 2024-03-30T02:36:25.283 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2024-03-30T02:36:25.664 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- bash -c 'ceph osd dump' 2024-03-30T02:36:26.031 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:26.361 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:26 smithi067 ceph-mon[37808]: pgmap v284: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-30T02:36:26.361 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:26 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3893648145' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-30T02:36:26.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:26 smithi170 ceph-mon[78798]: pgmap v284: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-30T02:36:26.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:26 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3893648145' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-30T02:36:26.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:26 smithi082 ceph-mon[79176]: pgmap v284: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-30T02:36:26.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:26 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3893648145' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-30T02:36:27.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.323+0000 7febcba00700 1 -- 172.21.15.67:0/3483114991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4073810 msgr2=0x7febc4073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:27.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.323+0000 7febcba00700 1 --2- 172.21.15.67:0/3483114991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4073810 0x7febc4073c90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7febb0009990 tx=0x7febb0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.323+0000 7febcba00700 1 -- 172.21.15.67:0/3483114991 shutdown_connections 2024-03-30T02:36:27.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.323+0000 7febcba00700 1 --2- 172.21.15.67:0/3483114991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc4108cd0 0x7febc410b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.323+0000 7febcba00700 1 --2- 172.21.15.67:0/3483114991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febc4074de0 0x7febc4073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.323+0000 7febcba00700 1 --2- 172.21.15.67:0/3483114991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4073810 0x7febc4073c90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.323+0000 7febcba00700 1 -- 172.21.15.67:0/3483114991 >> 172.21.15.67:0/3483114991 conn(0x7febc40fc3c0 msgr2=0x7febc40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.323+0000 7febcba00700 1 -- 172.21.15.67:0/3483114991 shutdown_connections 2024-03-30T02:36:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.323+0000 7febcba00700 1 -- 172.21.15.67:0/3483114991 wait complete. 2024-03-30T02:36:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.324+0000 7febcba00700 1 Processor -- start 2024-03-30T02:36:27.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.325+0000 7febcba00700 1 -- start start 2024-03-30T02:36:27.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.325+0000 7febcba00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febc4073810 0x7febc4198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:27.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.325+0000 7febcba00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4074de0 0x7febc41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:27.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.325+0000 7febcba00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc4108cd0 0x7febc41a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.325+0000 7febcba00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41a0a90 con 0x7febc4074de0 2024-03-30T02:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.325+0000 7febcba00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41a0bd0 con 0x7febc4108cd0 2024-03-30T02:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.325+0000 7febcba00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc41a0d60 con 0x7febc4073810 2024-03-30T02:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.325+0000 7febc3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4074de0 0x7febc41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4074de0 0x7febc41994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50110/0 (socket says 172.21.15.67:50110) 2024-03-30T02:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc3e00700 1 -- 172.21.15.67:0/4186823012 learned_addr learned my addr 172.21.15.67:0/4186823012 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc9000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febc4073810 0x7febc4198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:27.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc9a00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc4108cd0 0x7febc41a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc9000700 1 -- 172.21.15.67:0/4186823012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc4108cd0 msgr2=0x7febc41a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc9000700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc4108cd0 0x7febc41a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc9000700 1 -- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4074de0 msgr2=0x7febc41994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc9000700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4074de0 0x7febc41994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc9000700 1 -- 172.21.15.67:0/4186823012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb00095f0 con 0x7febc4073810 2024-03-30T02:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc3e00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4074de0 0x7febc41994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:27.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc9a00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc4108cd0 0x7febc41a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc9000700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febc4073810 0x7febc4198fa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7febb800e9f0 tx=0x7febb800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc1600700 1 -- 172.21.15.67:0/4186823012 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb800eee0 con 0x7febc4073810 2024-03-30T02:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.326+0000 7febc1600700 1 -- 172.21.15.67:0/4186823012 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febb800cbd0 con 0x7febc4073810 2024-03-30T02:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.327+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc41a0fe0 con 0x7febc4073810 2024-03-30T02:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.327+0000 7febc1600700 1 -- 172.21.15.67:0/4186823012 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8018680 con 0x7febc4073810 2024-03-30T02:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.327+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc41a1450 con 0x7febc4073810 2024-03-30T02:36:27.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.328+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc4107a90 con 0x7febc4073810 2024-03-30T02:36:27.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.329+0000 7febc1600700 1 -- 172.21.15.67:0/4186823012 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7febb800cd40 con 0x7febc4073810 2024-03-30T02:36:27.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.331+0000 7febc1600700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7febac0a1120 0x7febac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.331+0000 7febc1600700 1 -- 172.21.15.67:0/4186823012 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9065+0+0 (secure 0 0 0) 0x7febb8014070 con 0x7febc4073810 2024-03-30T02:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.331+0000 7febc3e00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7febac0a1120 0x7febac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:27.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.331+0000 7febc3e00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7febac0a1120 0x7febac0a35e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7febb000b2e0 tx=0x7febb0005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:27.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.335+0000 7febc1600700 1 -- 172.21.15.67:0/4186823012 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7febb80529f0 con 0x7febc4073810 2024-03-30T02:36:27.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:27 smithi067 ceph-mon[37808]: pgmap v285: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-30T02:36:27.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-30T02:36:27.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-03-30T02:36:27.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:27 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-30T02:36:27.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:27 smithi082 ceph-mon[79176]: pgmap v285: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-30T02:36:27.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-30T02:36:27.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-03-30T02:36:27.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:27 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-30T02:36:27.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.545+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7febc402cd50 con 0x7febc4073810 2024-03-30T02:36:27.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.547+0000 7febc1600700 1 -- 172.21.15.67:0/4186823012 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) v1 ==== 56+0+5624 (secure 0 0 0) 0x7febb80961d0 con 0x7febc4073810 2024-03-30T02:36:27.547 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 86 2024-03-30T02:36:27.547 INFO:teuthology.orchestra.run.smithi067.stdout:fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:36:27.547 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-03-30T02:28:23.927384+0000 2024-03-30T02:36:27.547 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-03-30T02:36:27.216829+0000 2024-03-30T02:36:27.547 INFO:teuthology.orchestra.run.smithi067.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-30T02:36:27.547 INFO:teuthology.orchestra.run.smithi067.stdout:crush_version 27 2024-03-30T02:36:27.547 INFO:teuthology.orchestra.run.smithi067.stdout:full_ratio 0.95 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:backfillfull_ratio 0.9 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:nearfull_ratio 0.85 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:require_min_compat_client luminous 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:min_compat_client luminous 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:require_osd_release reef 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:stretch_mode_enabled false 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:max_osd 12 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6802/1668225002,v1:172.21.15.67:6803/1668225002] [v2:172.21.15.67:6804/1668225002,v1:172.21.15.67:6805/1668225002] exists,up 5709d485-60d6-4959-b2dd-a5aeb11363af 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6800/1982951943,v1:172.21.15.82:6801/1982951943] [v2:172.21.15.82:6802/1982951943,v1:172.21.15.82:6803/1982951943] exists,up 1deb576f-c68d-421e-b78a-cfd6e72aafc1 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/229013363,v1:172.21.15.170:6801/229013363] [v2:172.21.15.170:6802/229013363,v1:172.21.15.170:6803/229013363] exists,up 0bcf7865-4fd5-4a95-9cb1-294899b88e2d 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6810/2637526467,v1:172.21.15.67:6811/2637526467] [v2:172.21.15.67:6812/2637526467,v1:172.21.15.67:6813/2637526467] exists,up eea2b812-7830-4563-aa7e-b340ae8a35bb 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 up in weight 1 up_from 32 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6808/1426586615,v1:172.21.15.82:6809/1426586615] [v2:172.21.15.82:6810/1426586615,v1:172.21.15.82:6811/1426586615] exists,up 6a35b5bb-587d-489e-941d-5b9ec9c1c2e4 2024-03-30T02:36:27.548 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 up in weight 1 up_from 37 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/2583913458,v1:172.21.15.170:6809/2583913458] [v2:172.21.15.170:6810/2583913458,v1:172.21.15.170:6811/2583913458] exists,up 787b667a-b309-4292-8a66-d7716b659d95 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 up in weight 1 up_from 44 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6818/3459663543,v1:172.21.15.67:6819/3459663543] [v2:172.21.15.67:6820/3459663543,v1:172.21.15.67:6821/3459663543] exists,up 986d28e3-52e9-462b-b728-d76c2457f15a 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 up in weight 1 up_from 48 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6816/318348288,v1:172.21.15.82:6817/318348288] [v2:172.21.15.82:6818/318348288,v1:172.21.15.82:6819/318348288] exists,up fa6e309c-75b6-4011-a6d1-bb0b1c438ef4 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 up in weight 1 up_from 55 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/4041482230,v1:172.21.15.170:6817/4041482230] [v2:172.21.15.170:6818/4041482230,v1:172.21.15.170:6819/4041482230] exists,up 90ecfccd-eb50-4be0-927c-ae2c8a415c83 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 up in weight 1 up_from 62 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6826/2316326417,v1:172.21.15.67:6827/2316326417] [v2:172.21.15.67:6828/2316326417,v1:172.21.15.67:6829/2316326417] exists,up 1ec7fee7-63f1-4193-aa2c-769a1f5e54d5 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 up in weight 1 up_from 67 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6824/3088174869,v1:172.21.15.82:6825/3088174869] [v2:172.21.15.82:6826/3088174869,v1:172.21.15.82:6827/3088174869] exists,up 2a7ade41-1c90-42e9-a53f-6f75074c9f5b 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 up in weight 1 up_from 74 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/3333975805,v1:172.21.15.170:6825/3333975805] [v2:172.21.15.170:6826/3333975805,v1:172.21.15.170:6827/3333975805] exists,up a7b717af-28b4-4aff-98f9-b652333a9808 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.6 [8,11] 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.a [2,11,8,1] 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.e [2,4] 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.e [4,6] 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.18 [3,9] 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.21 [5,9] 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.24 [4,9] 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.3a [4,8] 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.6 [9,11] 2024-03-30T02:36:27.549 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.10 [9,8] 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.1b [9,8,5,10] 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/2119993048 expires 2024-03-31T02:29:26.929980+0000 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/2682757440 expires 2024-03-31T02:29:26.929980+0000 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/2682757440 expires 2024-03-31T02:29:26.929980+0000 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/598232861 expires 2024-03-31T02:29:26.929980+0000 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/4186481961 expires 2024-03-31T02:29:01.908985+0000 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3533289986 expires 2024-03-31T02:29:26.929980+0000 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/4186481961 expires 2024-03-31T02:29:01.908985+0000 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/1250300809 expires 2024-03-31T02:29:01.908985+0000 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/2693891742 expires 2024-03-31T02:29:01.908985+0000 2024-03-30T02:36:27.550 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3435746414 expires 2024-03-31T02:29:01.908985+0000 2024-03-30T02:36:27.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7febac0a1120 msgr2=0x7febac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:27.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7febac0a1120 0x7febac0a35e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7febb000b2e0 tx=0x7febb0005fd0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febc4073810 msgr2=0x7febc4198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febc4073810 0x7febc4198fa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7febb800e9f0 tx=0x7febb800ed00 comp rx=0 tx=0).stop 2024-03-30T02:36:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 shutdown_connections 2024-03-30T02:36:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7febac0a1120 0x7febac0a35e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febc4108cd0 0x7febc41a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febc4073810 0x7febc4198fa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 --2- 172.21.15.67:0/4186823012 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7febc4074de0 0x7febc41994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 >> 172.21.15.67:0/4186823012 conn(0x7febc40fc3c0 msgr2=0x7febc4104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 shutdown_connections 2024-03-30T02:36:27.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:27.549+0000 7febcba00700 1 -- 172.21.15.67:0/4186823012 wait complete. 2024-03-30T02:36:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:27 smithi170 ceph-mon[78798]: pgmap v285: 129 pgs: 129 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-30T02:36:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-03-30T02:36:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-03-30T02:36:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:27 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-30T02:36:27.919 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-30T02:36:27.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:28.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.133+0000 7ff377200700 1 -- 172.21.15.67:0/4066621783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff370108b40 msgr2=0x7ff37010afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.133+0000 7ff377200700 1 --2- 172.21.15.67:0/4066621783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff370108b40 0x7ff37010afd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff36800b0a0 tx=0x7ff36800b3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.133+0000 7ff377200700 1 -- 172.21.15.67:0/4066621783 shutdown_connections 2024-03-30T02:36:28.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.133+0000 7ff377200700 1 --2- 172.21.15.67:0/4066621783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff370108b40 0x7ff37010afd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.133+0000 7ff377200700 1 --2- 172.21.15.67:0/4066621783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff370074df0 0x7ff370073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.133+0000 7ff377200700 1 --2- 172.21.15.67:0/4066621783 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff370073810 0x7ff370073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.133+0000 7ff377200700 1 -- 172.21.15.67:0/4066621783 >> 172.21.15.67:0/4066621783 conn(0x7ff3700fc1f0 msgr2=0x7ff3700fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.133+0000 7ff377200700 1 -- 172.21.15.67:0/4066621783 shutdown_connections 2024-03-30T02:36:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.133+0000 7ff377200700 1 -- 172.21.15.67:0/4066621783 wait complete. 2024-03-30T02:36:28.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.134+0000 7ff377200700 1 Processor -- start 2024-03-30T02:36:28.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.134+0000 7ff377200700 1 -- start start 2024-03-30T02:36:28.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.134+0000 7ff377200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff370073810 0x7ff370198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.134+0000 7ff377200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff370074df0 0x7ff370199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.134+0000 7ff377200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff370108b40 0x7ff3701a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.134+0000 7ff377200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3701a0a80 con 0x7ff370073810 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.134+0000 7ff377200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3701a0bc0 con 0x7ff370108b40 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.134+0000 7ff377200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3701a0d50 con 0x7ff370074df0 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff370073810 0x7ff370198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff370073810 0x7ff370198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50118/0 (socket says 172.21.15.67:50118) 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36fe00700 1 -- 172.21.15.67:0/643854772 learned_addr learned my addr 172.21.15.67:0/643854772 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36f400700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff370074df0 0x7ff370199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff375200700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff370108b40 0x7ff3701a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36fe00700 1 -- 172.21.15.67:0/643854772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff370074df0 msgr2=0x7ff370199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36fe00700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff370074df0 0x7ff370199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36fe00700 1 -- 172.21.15.67:0/643854772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff370108b40 msgr2=0x7ff3701a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36fe00700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff370108b40 0x7ff3701a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36fe00700 1 -- 172.21.15.67:0/643854772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff368009d00 con 0x7ff370073810 2024-03-30T02:36:28.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36f400700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff370074df0 0x7ff370199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:28.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff375200700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff370108b40 0x7ff3701a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:28.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.135+0000 7ff36fe00700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff370073810 0x7ff370198f50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff36000e9a0 tx=0x7ff36000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:28.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.136+0000 7ff36cc00700 1 -- 172.21.15.67:0/643854772 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360009790 con 0x7ff370073810 2024-03-30T02:36:28.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.136+0000 7ff36cc00700 1 -- 172.21.15.67:0/643854772 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff360010430 con 0x7ff370073810 2024-03-30T02:36:28.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.136+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3701a0fd0 con 0x7ff370073810 2024-03-30T02:36:28.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.136+0000 7ff36cc00700 1 -- 172.21.15.67:0/643854772 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360018680 con 0x7ff370073810 2024-03-30T02:36:28.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.136+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3701a14a0 con 0x7ff370073810 2024-03-30T02:36:28.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.137+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff370107900 con 0x7ff370073810 2024-03-30T02:36:28.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.139+0000 7ff36cc00700 1 -- 172.21.15.67:0/643854772 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7ff3600105a0 con 0x7ff370073810 2024-03-30T02:36:28.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.141+0000 7ff36cc00700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff3540a11a0 0x7ff3540a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:28.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.141+0000 7ff36cc00700 1 -- 172.21.15.67:0/643854772 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9065+0+0 (secure 0 0 0) 0x7ff360014070 con 0x7ff370073810 2024-03-30T02:36:28.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.141+0000 7ff36f400700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff3540a11a0 0x7ff3540a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:28.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.142+0000 7ff36f400700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff3540a11a0 0x7ff3540a3660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff370074b10 tx=0x7ff35c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:28.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.145+0000 7ff36cc00700 1 -- 172.21.15.67:0/643854772 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ff3600529f0 con 0x7ff370073810 2024-03-30T02:36:28.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.415+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff370067a10 con 0x7ff370073810 2024-03-30T02:36:28.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.417+0000 7ff36cc00700 1 -- 172.21.15.67:0/643854772 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24479 (secure 0 0 0) 0x7ff3600961a0 con 0x7ff370073810 2024-03-30T02:36:28.417 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:28.417 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:36:28.218474+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:28.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff3540a11a0 msgr2=0x7ff3540a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff3540a11a0 0x7ff3540a3660 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff370074b10 tx=0x7ff35c009260 comp rx=0 tx=0).stop 2024-03-30T02:36:28.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff370073810 msgr2=0x7ff370198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff370073810 0x7ff370198f50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff36000e9a0 tx=0x7ff36000ecb0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 shutdown_connections 2024-03-30T02:36:28.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7ff3540a11a0 0x7ff3540a3660 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff370073810 0x7ff370198f50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff370108b40 0x7ff3701a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 --2- 172.21.15.67:0/643854772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff370074df0 0x7ff370199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 >> 172.21.15.67:0/643854772 conn(0x7ff3700fc1f0 msgr2=0x7ff370104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:28.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.419+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 shutdown_connections 2024-03-30T02:36:28.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.420+0000 7ff377200700 1 -- 172.21.15.67:0/643854772 wait complete. 2024-03-30T02:36:28.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:36:28.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:36:28.496 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-30T02:36:28.496 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-03-30T02:36:28.496 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:28 smithi067 ceph-mon[37808]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-30T02:36:28.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:28 smithi067 ceph-mon[37808]: osdmap e86: 12 total, 12 up, 12 in 2024-03-30T02:36:28.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:28 smithi067 ceph-mon[37808]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-03-30T02:36:28.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:28 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4186823012' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-30T02:36:28.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:28 smithi067 ceph-mon[37808]: osdmap e87: 12 total, 12 up, 12 in 2024-03-30T02:36:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-30T02:36:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-03-30T02:36:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:28 smithi082 ceph-mon[79176]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-30T02:36:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:28 smithi082 ceph-mon[79176]: osdmap e86: 12 total, 12 up, 12 in 2024-03-30T02:36:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:28 smithi082 ceph-mon[79176]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-03-30T02:36:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:28 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4186823012' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-30T02:36:28.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:28 smithi082 ceph-mon[79176]: osdmap e87: 12 total, 12 up, 12 in 2024-03-30T02:36:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.641+0000 7fbfc9800700 1 -- 172.21.15.67:0/2918972446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc40ff5e0 msgr2=0x7fbfc40ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.641+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2918972446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc40ff5e0 0x7fbfc40ffa00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009990 tx=0x7fbfb4009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.641+0000 7fbfc9800700 1 -- 172.21.15.67:0/2918972446 shutdown_connections 2024-03-30T02:36:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.641+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2918972446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc40ff5e0 0x7fbfc40ffa00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.641+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2918972446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfc4100900 0x7fbfc4100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.641+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2918972446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc40fff40 0x7fbfc41003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.641+0000 7fbfc9800700 1 -- 172.21.15.67:0/2918972446 >> 172.21.15.67:0/2918972446 conn(0x7fbfc40fb200 msgr2=0x7fbfc40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.641+0000 7fbfc9800700 1 -- 172.21.15.67:0/2918972446 shutdown_connections 2024-03-30T02:36:28.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.641+0000 7fbfc9800700 1 -- 172.21.15.67:0/2918972446 wait complete. 2024-03-30T02:36:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.642+0000 7fbfc9800700 1 Processor -- start 2024-03-30T02:36:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.642+0000 7fbfc9800700 1 -- start start 2024-03-30T02:36:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.642+0000 7fbfc9800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc40ff5e0 0x7fbfc4198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.642+0000 7fbfc9800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc40fff40 0x7fbfc4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.642+0000 7fbfc9800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfc4100900 0x7fbfc41a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.642+0000 7fbfc9800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc41a0a40 con 0x7fbfc40fff40 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.642+0000 7fbfc9800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc41a0b80 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.642+0000 7fbfc9800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc41a0d10 con 0x7fbfc4100900 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc40ff5e0 0x7fbfc4198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc40fff40 0x7fbfc4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc40ff5e0 0x7fbfc4198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39490/0 (socket says 172.21.15.67:39490) 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2a00700 1 -- 172.21.15.67:0/2742814845 learned_addr learned my addr 172.21.15.67:0/2742814845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc3400700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfc4100900 0x7fbfc41a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2a00700 1 -- 172.21.15.67:0/2742814845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfc4100900 msgr2=0x7fbfc41a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2a00700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfc4100900 0x7fbfc41a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2a00700 1 -- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc40fff40 msgr2=0x7fbfc4199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2a00700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc40fff40 0x7fbfc4199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2a00700 1 -- 172.21.15.67:0/2742814845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb40095f0 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2000700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc40fff40 0x7fbfc4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:28.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.643+0000 7fbfc2a00700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc40ff5e0 0x7fbfc4198f10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4000c00 tx=0x7fbfb4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:28.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.644+0000 7fbfbb400700 1 -- 172.21.15.67:0/2742814845 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb401d070 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.644+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc41a0f90 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.644+0000 7fbfbb400700 1 -- 172.21.15.67:0/2742814845 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb4021410 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.644+0000 7fbfbb400700 1 -- 172.21.15.67:0/2742814845 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4017650 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.644+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc41a1350 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.645+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc410a640 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.647+0000 7fbfbb400700 1 -- 172.21.15.67:0/2742814845 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fbfb40217f0 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.648+0000 7fbfbb400700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfa80a1120 0x7fbfa80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:28.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.649+0000 7fbfbb400700 1 -- 172.21.15.67:0/2742814845 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fbfb40ce750 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.649+0000 7fbfc2000700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfa80a1120 0x7fbfa80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:28.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.649+0000 7fbfc2000700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfa80a1120 0x7fbfa80a35e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0011490 tx=0x7fbfb0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:28.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.651+0000 7fbfbb400700 1 -- 172.21.15.67:0/2742814845 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbfb4097bd0 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-03-30T02:36:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-03-30T02:36:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:28 smithi170 ceph-mon[78798]: from='mgr.14154 172.21.15.67:0/646752967' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-30T02:36:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:28 smithi170 ceph-mon[78798]: osdmap e86: 12 total, 12 up, 12 in 2024-03-30T02:36:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:28 smithi170 ceph-mon[78798]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-03-30T02:36:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:28 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4186823012' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-30T02:36:28.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:28 smithi170 ceph-mon[78798]: osdmap e87: 12 total, 12 up, 12 in 2024-03-30T02:36:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.866+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbfc410e8b0 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.866+0000 7fbfbb400700 1 -- 172.21.15.67:0/2742814845 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfb409b1f0 con 0x7fbfc40ff5e0 2024-03-30T02:36:28.867 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:36:28.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfa80a1120 msgr2=0x7fbfa80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfa80a1120 0x7fbfa80a35e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0011490 tx=0x7fbfb0009260 comp rx=0 tx=0).stop 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc40ff5e0 msgr2=0x7fbfc4198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc40ff5e0 0x7fbfc4198f10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4000c00 tx=0x7fbfb4005740 comp rx=0 tx=0).stop 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 shutdown_connections 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfa80a1120 0x7fbfa80a35e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfc4100900 0x7fbfc41a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfc40ff5e0 0x7fbfc4198f10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 --2- 172.21.15.67:0/2742814845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc40fff40 0x7fbfc4199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 >> 172.21.15.67:0/2742814845 conn(0x7fbfc40fb200 msgr2=0x7fbfc40fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 shutdown_connections 2024-03-30T02:36:28.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:28.869+0000 7fbfc9800700 1 -- 172.21.15.67:0/2742814845 wait complete. 2024-03-30T02:36:28.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:36:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.089+0000 7f0c53800700 1 -- 172.21.15.67:0/1540669587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c100d80 msgr2=0x7f0c4c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.089+0000 7f0c53800700 1 --2- 172.21.15.67:0/1540669587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c100d80 0x7f0c4c1011a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009990 tx=0x7f0c40009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.090+0000 7f0c53800700 1 -- 172.21.15.67:0/1540669587 shutdown_connections 2024-03-30T02:36:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.090+0000 7f0c53800700 1 --2- 172.21.15.67:0/1540669587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c4c102910 0x7f0c4c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.090+0000 7f0c53800700 1 --2- 172.21.15.67:0/1540669587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c100d80 0x7f0c4c1011a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.090+0000 7f0c53800700 1 --2- 172.21.15.67:0/1540669587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c4c101f70 0x7f0c4c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.090+0000 7f0c53800700 1 -- 172.21.15.67:0/1540669587 >> 172.21.15.67:0/1540669587 conn(0x7f0c4c0fc2f0 msgr2=0x7f0c4c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.090+0000 7f0c53800700 1 -- 172.21.15.67:0/1540669587 shutdown_connections 2024-03-30T02:36:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.090+0000 7f0c53800700 1 -- 172.21.15.67:0/1540669587 wait complete. 2024-03-30T02:36:29.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c53800700 1 Processor -- start 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c53800700 1 -- start start 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c53800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c4c100d80 0x7f0c4c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c53800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c101f70 0x7f0c4c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c53800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c4c102910 0x7f0c4c1a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c53800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c1a0a60 con 0x7f0c4c102910 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c53800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c1a0ba0 con 0x7f0c4c100d80 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c53800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c1a0d30 con 0x7f0c4c101f70 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c51800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c4c102910 0x7f0c4c1a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c51800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c4c102910 0x7f0c4c1a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50146/0 (socket says 172.21.15.67:50146) 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c51800700 1 -- 172.21.15.67:0/745485943 learned_addr learned my addr 172.21.15.67:0/745485943 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c4be00700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c101f70 0x7f0c4c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.091+0000 7f0c50e00700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c4c100d80 0x7f0c4c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:29.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c51800700 1 -- 172.21.15.67:0/745485943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c101f70 msgr2=0x7f0c4c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c51800700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c101f70 0x7f0c4c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c51800700 1 -- 172.21.15.67:0/745485943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c4c100d80 msgr2=0x7f0c4c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c51800700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c4c100d80 0x7f0c4c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c51800700 1 -- 172.21.15.67:0/745485943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c400095f0 con 0x7f0c4c102910 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c51800700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c4c102910 0x7f0c4c1a0490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400ef30 tx=0x7f0c4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c49600700 1 -- 172.21.15.67:0/745485943 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44018070 con 0x7f0c4c102910 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c1a0fb0 con 0x7f0c4c102910 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c49600700 1 -- 172.21.15.67:0/745485943 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c4400f040 con 0x7f0c4c102910 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c49600700 1 -- 172.21.15.67:0/745485943 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400cd00 con 0x7f0c4c102910 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.092+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c1a1430 con 0x7f0c4c102910 2024-03-30T02:36:29.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.093+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c067af0 con 0x7f0c4c102910 2024-03-30T02:36:29.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.095+0000 7f0c49600700 1 -- 172.21.15.67:0/745485943 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0c44003a00 con 0x7f0c4c102910 2024-03-30T02:36:29.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.098+0000 7f0c49600700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0c340a1120 0x7f0c340a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:29.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.098+0000 7f0c49600700 1 -- 172.21.15.67:0/745485943 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f0c440cc770 con 0x7f0c4c102910 2024-03-30T02:36:29.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.098+0000 7f0c50e00700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0c340a1120 0x7f0c340a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:29.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.098+0000 7f0c50e00700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0c340a1120 0x7f0c340a35e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000b680 tx=0x7f0c4000b750 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:29.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.102+0000 7f0c49600700 1 -- 172.21.15.67:0/745485943 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0c44095bf0 con 0x7f0c4c102910 2024-03-30T02:36:29.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.321+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c4c1a1770 con 0x7f0c4c102910 2024-03-30T02:36:29.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.321+0000 7f0c49600700 1 -- 172.21.15.67:0/745485943 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c44099210 con 0x7f0c4c102910 2024-03-30T02:36:29.321 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0c340a1120 msgr2=0x7f0c340a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0c340a1120 0x7f0c340a35e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000b680 tx=0x7f0c4000b750 comp rx=0 tx=0).stop 2024-03-30T02:36:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c4c102910 msgr2=0x7f0c4c1a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c4c102910 0x7f0c4c1a0490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400ef30 tx=0x7f0c4400c3a0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 shutdown_connections 2024-03-30T02:36:29.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0c340a1120 0x7f0c340a35e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0c4c102910 0x7f0c4c1a0490 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c4c100d80 0x7f0c4c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 --2- 172.21.15.67:0/745485943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c4c101f70 0x7f0c4c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 >> 172.21.15.67:0/745485943 conn(0x7f0c4c0fc2f0 msgr2=0x7f0c4c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:29.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 shutdown_connections 2024-03-30T02:36:29.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.324+0000 7f0c53800700 1 -- 172.21.15.67:0/745485943 wait complete. 2024-03-30T02:36:29.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:36:29.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:29 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/643854772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:29.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:29 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2742814845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:29.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:29 smithi082 ceph-mon[79176]: pgmap v288: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 639 B/s wr, 12 op/s 2024-03-30T02:36:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.544+0000 7f944de00700 1 -- 172.21.15.67:0/4199964347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448101f70 msgr2=0x7f94481023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.544+0000 7f944de00700 1 --2- 172.21.15.67:0/4199964347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448101f70 0x7f94481023d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f94340099c0 tx=0x7f9434009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.545+0000 7f944de00700 1 -- 172.21.15.67:0/4199964347 shutdown_connections 2024-03-30T02:36:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.545+0000 7f944de00700 1 --2- 172.21.15.67:0/4199964347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9448102910 0x7f94481091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.545+0000 7f944de00700 1 --2- 172.21.15.67:0/4199964347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448100d80 0x7f94481011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.545+0000 7f944de00700 1 --2- 172.21.15.67:0/4199964347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448101f70 0x7f94481023d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.545+0000 7f944de00700 1 -- 172.21.15.67:0/4199964347 >> 172.21.15.67:0/4199964347 conn(0x7f94480fc2f0 msgr2=0x7f94480fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.545+0000 7f944de00700 1 -- 172.21.15.67:0/4199964347 shutdown_connections 2024-03-30T02:36:29.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.545+0000 7f944de00700 1 -- 172.21.15.67:0/4199964347 wait complete. 2024-03-30T02:36:29.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.546+0000 7f944de00700 1 Processor -- start 2024-03-30T02:36:29.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.546+0000 7f944de00700 1 -- start start 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f944de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448100d80 0x7f9448198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f944de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448101f70 0x7f94481994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f944de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9448102910 0x7f94481a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f944de00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94481a0a60 con 0x7f9448100d80 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f944de00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94481a0ba0 con 0x7f9448102910 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f944de00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94481a0d30 con 0x7f9448101f70 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f9446a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448100d80 0x7f9448198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f9446a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448100d80 0x7f9448198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50168/0 (socket says 172.21.15.67:50168) 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f9446a00700 1 -- 172.21.15.67:0/3436330423 learned_addr learned my addr 172.21.15.67:0/3436330423 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f9446000700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448101f70 0x7f94481994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.547+0000 7f9447400700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9448102910 0x7f94481a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f9446a00700 1 -- 172.21.15.67:0/3436330423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448101f70 msgr2=0x7f94481994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f9446a00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448101f70 0x7f94481994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f9446a00700 1 -- 172.21.15.67:0/3436330423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9448102910 msgr2=0x7f94481a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f9446a00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9448102910 0x7f94481a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f9446a00700 1 -- 172.21.15.67:0/3436330423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94340095f0 con 0x7f9448100d80 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f9446a00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448100d80 0x7f9448198f80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f943800d980 tx=0x7f943800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f944ca00700 1 -- 172.21.15.67:0/3436330423 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438004020 con 0x7f9448100d80 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f944ca00700 1 -- 172.21.15.67:0/3436330423 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f943800f690 con 0x7f9448100d80 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94481a0fb0 con 0x7f9448100d80 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f944ca00700 1 -- 172.21.15.67:0/3436330423 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438003a60 con 0x7f9448100d80 2024-03-30T02:36:29.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.548+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94481a1480 con 0x7f9448100d80 2024-03-30T02:36:29.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.549+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9448067af0 con 0x7f9448100d80 2024-03-30T02:36:29.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.551+0000 7f944ca00700 1 -- 172.21.15.67:0/3436330423 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f943800fb00 con 0x7f9448100d80 2024-03-30T02:36:29.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.552+0000 7f944ca00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f942c0a11a0 0x7f942c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:29.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.553+0000 7f944ca00700 1 -- 172.21.15.67:0/3436330423 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f94380cc740 con 0x7f9448100d80 2024-03-30T02:36:29.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.553+0000 7f9446000700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f942c0a11a0 0x7f942c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:29.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.553+0000 7f9446000700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f942c0a11a0 0x7f942c0a3660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9434006010 tx=0x7f943400b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:29.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.557+0000 7f944ca00700 1 -- 172.21.15.67:0/3436330423 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9438095b30 con 0x7f9448100d80 2024-03-30T02:36:29.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:29 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/643854772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:29.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:29 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2742814845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:29.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:29 smithi067 ceph-mon[37808]: pgmap v288: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 639 B/s wr, 12 op/s 2024-03-30T02:36:29.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:29 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/643854772' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:29.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:29 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2742814845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:29.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:29 smithi170 ceph-mon[78798]: pgmap v288: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 639 B/s wr, 12 op/s 2024-03-30T02:36:29.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.777+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94481a17f0 con 0x7f9448100d80 2024-03-30T02:36:29.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.778+0000 7f944ca00700 1 -- 172.21.15.67:0/3436330423 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9438099150 con 0x7f9448100d80 2024-03-30T02:36:29.778 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:36:29.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f942c0a11a0 msgr2=0x7f942c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f942c0a11a0 0x7f942c0a3660 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9434006010 tx=0x7f943400b540 comp rx=0 tx=0).stop 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448100d80 msgr2=0x7f9448198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448100d80 0x7f9448198f80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f943800d980 tx=0x7f943800dc90 comp rx=0 tx=0).stop 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 shutdown_connections 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f942c0a11a0 0x7f942c0a3660 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9448100d80 0x7f9448198f80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9448102910 0x7f94481a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 --2- 172.21.15.67:0/3436330423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9448101f70 0x7f94481994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 >> 172.21.15.67:0/3436330423 conn(0x7f94480fc2f0 msgr2=0x7f9448072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 shutdown_connections 2024-03-30T02:36:29.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:29.780+0000 7f944de00700 1 -- 172.21.15.67:0/3436330423 wait complete. 2024-03-30T02:36:29.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:36:30.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.002+0000 7f0b58000700 1 -- 172.21.15.67:0/1586771711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50100d80 msgr2=0x7f0b501011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.002+0000 7f0b58000700 1 --2- 172.21.15.67:0/1586771711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50100d80 0x7f0b501011a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b440099c0 tx=0x7f0b44009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.002+0000 7f0b58000700 1 -- 172.21.15.67:0/1586771711 shutdown_connections 2024-03-30T02:36:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.002+0000 7f0b58000700 1 --2- 172.21.15.67:0/1586771711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50100d80 0x7f0b501011a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.002+0000 7f0b58000700 1 --2- 172.21.15.67:0/1586771711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50102910 0x7f0b501091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.002+0000 7f0b58000700 1 --2- 172.21.15.67:0/1586771711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50101f70 0x7f0b501023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.002+0000 7f0b58000700 1 -- 172.21.15.67:0/1586771711 >> 172.21.15.67:0/1586771711 conn(0x7f0b500fc2f0 msgr2=0x7f0b500fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.002+0000 7f0b58000700 1 -- 172.21.15.67:0/1586771711 shutdown_connections 2024-03-30T02:36:30.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.002+0000 7f0b58000700 1 -- 172.21.15.67:0/1586771711 wait complete. 2024-03-30T02:36:30.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.003+0000 7f0b58000700 1 Processor -- start 2024-03-30T02:36:30.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.003+0000 7f0b58000700 1 -- start start 2024-03-30T02:36:30.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b58000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100d80 0x7f0b50196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b58000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50101f70 0x7f0b501972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b58000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50102910 0x7f0b5019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b58000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019e870 con 0x7f0b50101f70 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b58000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019e9e0 con 0x7f0b50102910 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b58000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5019eba0 con 0x7f0b50100d80 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b54c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50101f70 0x7f0b501972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b54c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50101f70 0x7f0b501972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50180/0 (socket says 172.21.15.67:50180) 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b54c00700 1 -- 172.21.15.67:0/1301459529 learned_addr learned my addr 172.21.15.67:0/1301459529 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b56000700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50102910 0x7f0b5019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b55600700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100d80 0x7f0b50196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b54c00700 1 -- 172.21.15.67:0/1301459529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100d80 msgr2=0x7f0b50196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b54c00700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100d80 0x7f0b50196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b54c00700 1 -- 172.21.15.67:0/1301459529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50102910 msgr2=0x7f0b5019e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b54c00700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50102910 0x7f0b5019e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b54c00700 1 -- 172.21.15.67:0/1301459529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b48009c40 con 0x7f0b50101f70 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.004+0000 7f0b55600700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100d80 0x7f0b50196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.005+0000 7f0b54c00700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50101f70 0x7f0b501972c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00d980 tx=0x7f0b3c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:30.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.005+0000 7f0b4e000700 1 -- 172.21.15.67:0/1301459529 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00bb60 con 0x7f0b50101f70 2024-03-30T02:36:30.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.005+0000 7f0b4e000700 1 -- 172.21.15.67:0/1301459529 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b3c007640 con 0x7f0b50101f70 2024-03-30T02:36:30.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.005+0000 7f0b4e000700 1 -- 172.21.15.67:0/1301459529 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c004cd0 con 0x7f0b50101f70 2024-03-30T02:36:30.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.005+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b440095f0 con 0x7f0b50101f70 2024-03-30T02:36:30.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.005+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5019f1b0 con 0x7f0b50101f70 2024-03-30T02:36:30.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.006+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b50108220 con 0x7f0b50101f70 2024-03-30T02:36:30.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.008+0000 7f0b4e000700 1 -- 172.21.15.67:0/1301459529 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f0b3c004020 con 0x7f0b50101f70 2024-03-30T02:36:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.009+0000 7f0b4e000700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b380a11a0 0x7f0b380a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.010+0000 7f0b4e000700 1 -- 172.21.15.67:0/1301459529 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f0b3c0ccf40 con 0x7f0b50101f70 2024-03-30T02:36:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.010+0000 7f0b55600700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b380a11a0 0x7f0b380a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.010+0000 7f0b55600700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b380a11a0 0x7f0b380a3660 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b50198200 tx=0x7f0b4401a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:30.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.012+0000 7f0b4e000700 1 -- 172.21.15.67:0/1301459529 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0b3c096390 con 0x7f0b50101f70 2024-03-30T02:36:30.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.233+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0b5010c5e0 con 0x7f0b50101f70 2024-03-30T02:36:30.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.234+0000 7f0b4e000700 1 -- 172.21.15.67:0/1301459529 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0b3c016070 con 0x7f0b50101f70 2024-03-30T02:36:30.234 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:30.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b380a11a0 msgr2=0x7f0b380a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b380a11a0 0x7f0b380a3660 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b50198200 tx=0x7f0b4401a040 comp rx=0 tx=0).stop 2024-03-30T02:36:30.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50101f70 msgr2=0x7f0b501972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50101f70 0x7f0b501972c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00d980 tx=0x7f0b3c00dd40 comp rx=0 tx=0).stop 2024-03-30T02:36:30.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 shutdown_connections 2024-03-30T02:36:30.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f0b380a11a0 0x7f0b380a3660 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b50102910 0x7f0b5019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b50100d80 0x7f0b50196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 --2- 172.21.15.67:0/1301459529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b50101f70 0x7f0b501972c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.236+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 >> 172.21.15.67:0/1301459529 conn(0x7f0b500fc2f0 msgr2=0x7f0b50105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:30.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.237+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 shutdown_connections 2024-03-30T02:36:30.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.237+0000 7f0b58000700 1 -- 172.21.15.67:0/1301459529 wait complete. 2024-03-30T02:36:30.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:36:30.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.454+0000 7f071ae00700 1 -- 172.21.15.67:0/2974733974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0714102810 msgr2=0x7f07141090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.454+0000 7f071ae00700 1 --2- 172.21.15.67:0/2974733974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0714102810 0x7f07141090c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f070c00b0d0 tx=0x7f070c00b3e0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.454+0000 7f071ae00700 1 -- 172.21.15.67:0/2974733974 shutdown_connections 2024-03-30T02:36:30.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.454+0000 7f071ae00700 1 --2- 172.21.15.67:0/2974733974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0714102810 0x7f07141090c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.454+0000 7f071ae00700 1 --2- 172.21.15.67:0/2974733974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0714100c80 0x7f07141010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.454+0000 7f071ae00700 1 --2- 172.21.15.67:0/2974733974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0714101e70 0x7f07141022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.454+0000 7f071ae00700 1 -- 172.21.15.67:0/2974733974 >> 172.21.15.67:0/2974733974 conn(0x7f07140fc1f0 msgr2=0x7f07140fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:30.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.454+0000 7f071ae00700 1 -- 172.21.15.67:0/2974733974 shutdown_connections 2024-03-30T02:36:30.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.454+0000 7f071ae00700 1 -- 172.21.15.67:0/2974733974 wait complete. 2024-03-30T02:36:30.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.455+0000 7f071ae00700 1 Processor -- start 2024-03-30T02:36:30.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.455+0000 7f071ae00700 1 -- start start 2024-03-30T02:36:30.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f071ae00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0714100c80 0x7f0714198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f071ae00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0714101e70 0x7f0714199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f071ae00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0714102810 0x7f07141a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f071ae00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07141a0a10 con 0x7f0714102810 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f071ae00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07141a0b50 con 0x7f0714100c80 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f071ae00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07141a0ce0 con 0x7f0714101e70 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0713e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0714100c80 0x7f0714198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0713e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0714100c80 0x7f0714198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39564/0 (socket says 172.21.15.67:39564) 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0713e00700 1 -- 172.21.15.67:0/1292711349 learned_addr learned my addr 172.21.15.67:0/1292711349 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0718e00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0714102810 0x7f07141a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0713400700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0714101e70 0x7f0714199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0713e00700 1 -- 172.21.15.67:0/1292711349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0714101e70 msgr2=0x7f0714199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0713e00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0714101e70 0x7f0714199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0713e00700 1 -- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0714102810 msgr2=0x7f07141a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0713e00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0714102810 0x7f07141a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.456+0000 7f0713e00700 1 -- 172.21.15.67:0/1292711349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c009d00 con 0x7f0714100c80 2024-03-30T02:36:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.457+0000 7f0713e00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0714100c80 0x7f0714198f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f070400e970 tx=0x7f070400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.457+0000 7f0710c00700 1 -- 172.21.15.67:0/1292711349 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704009790 con 0x7f0714100c80 2024-03-30T02:36:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.457+0000 7f0710c00700 1 -- 172.21.15.67:0/1292711349 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f070400c7e0 con 0x7f0714100c80 2024-03-30T02:36:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.457+0000 7f0710c00700 1 -- 172.21.15.67:0/1292711349 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704003c70 con 0x7f0714100c80 2024-03-30T02:36:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.457+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07141a0fc0 con 0x7f0714100c80 2024-03-30T02:36:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.457+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07141a1390 con 0x7f0714100c80 2024-03-30T02:36:30.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.458+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0714067a80 con 0x7f0714100c80 2024-03-30T02:36:30.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.460+0000 7f0710c00700 1 -- 172.21.15.67:0/1292711349 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f070400cc50 con 0x7f0714100c80 2024-03-30T02:36:30.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.461+0000 7f0710c00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f06f80a11a0 0x7f06f80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.461+0000 7f0710c00700 1 -- 172.21.15.67:0/1292711349 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f0704014070 con 0x7f0714100c80 2024-03-30T02:36:30.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.462+0000 7f0713400700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f06f80a11a0 0x7f06f80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.462+0000 7f0713400700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f06f80a11a0 0x7f06f80a3660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0700009c60 tx=0x7f0700009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:30.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.466+0000 7f0710c00700 1 -- 172.21.15.67:0/1292711349 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0714067a80 con 0x7f0714100c80 2024-03-30T02:36:30.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:30 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/745485943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:30.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:30 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3436330423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:30.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:30 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1301459529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:30.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:30 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/745485943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:30.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:30 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3436330423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:30.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:30 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1301459529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:30 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/745485943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:30 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3436330423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:30 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1301459529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:30.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.748+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0714067a10 con 0x7f0714100c80 2024-03-30T02:36:30.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.748+0000 7f0710c00700 1 -- 172.21.15.67:0/1292711349 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f07040961c0 con 0x7f0714100c80 2024-03-30T02:36:30.748 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:30.748 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:35:20.119112+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:36:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f06f80a11a0 msgr2=0x7f06f80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f06f80a11a0 0x7f06f80a3660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0700009c60 tx=0x7f0700009260 comp rx=0 tx=0).stop 2024-03-30T02:36:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0714100c80 msgr2=0x7f0714198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0714100c80 0x7f0714198f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f070400e970 tx=0x7f070400ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 shutdown_connections 2024-03-30T02:36:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f06f80a11a0 0x7f06f80a3660 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0714102810 0x7f07141a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0714100c80 0x7f0714198f30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 --2- 172.21.15.67:0/1292711349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0714101e70 0x7f0714199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 >> 172.21.15.67:0/1292711349 conn(0x7f07140fc1f0 msgr2=0x7f0714072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:30.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 shutdown_connections 2024-03-30T02:36:30.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.751+0000 7f071ae00700 1 -- 172.21.15.67:0/1292711349 wait complete. 2024-03-30T02:36:30.752 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2024-03-30T02:36:30.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:30.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.968+0000 7f6277e00700 1 -- 172.21.15.67:0/113967503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6270101e70 msgr2=0x7f62701022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.968+0000 7f6277e00700 1 --2- 172.21.15.67:0/113967503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6270101e70 0x7f62701022d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f625c0099c0 tx=0x7f625c009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.968+0000 7f6277e00700 1 -- 172.21.15.67:0/113967503 shutdown_connections 2024-03-30T02:36:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.968+0000 7f6277e00700 1 --2- 172.21.15.67:0/113967503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6270102810 0x7f62701090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.968+0000 7f6277e00700 1 --2- 172.21.15.67:0/113967503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270100c80 0x7f62701010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.968+0000 7f6277e00700 1 --2- 172.21.15.67:0/113967503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6270101e70 0x7f62701022d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.968+0000 7f6277e00700 1 -- 172.21.15.67:0/113967503 >> 172.21.15.67:0/113967503 conn(0x7f62700fc1f0 msgr2=0x7f62700fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.968+0000 7f6277e00700 1 -- 172.21.15.67:0/113967503 shutdown_connections 2024-03-30T02:36:30.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.968+0000 7f6277e00700 1 -- 172.21.15.67:0/113967503 wait complete. 2024-03-30T02:36:30.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.969+0000 7f6277e00700 1 Processor -- start 2024-03-30T02:36:30.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6277e00700 1 -- start start 2024-03-30T02:36:30.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6277e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270100c80 0x7f6270074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6277e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6270101e70 0x7f62700731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6277e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6270102810 0x7f62700736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6277e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6270073e80 con 0x7f6270102810 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6277e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6270073ff0 con 0x7f6270100c80 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6277e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62701a0c20 con 0x7f6270101e70 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6275400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270100c80 0x7f6270074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6275e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6270102810 0x7f62700736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6275400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270100c80 0x7f6270074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39592/0 (socket says 172.21.15.67:39592) 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6275400700 1 -- 172.21.15.67:0/2246379159 learned_addr learned my addr 172.21.15.67:0/2246379159 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.970+0000 7f6274a00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6270101e70 0x7f62700731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6275400700 1 -- 172.21.15.67:0/2246379159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6270101e70 msgr2=0x7f62700731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6275400700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6270101e70 0x7f62700731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6275400700 1 -- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6270102810 msgr2=0x7f62700736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6275400700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6270102810 0x7f62700736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6275400700 1 -- 172.21.15.67:0/2246379159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c0095f0 con 0x7f6270100c80 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6274a00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6270101e70 0x7f62700731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:30.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6275e00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6270102810 0x7f62700736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6275400700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270100c80 0x7f6270074b10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f626400e970 tx=0x7f626400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f626e000700 1 -- 172.21.15.67:0/2246379159 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264009790 con 0x7f6270100c80 2024-03-30T02:36:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f626e000700 1 -- 172.21.15.67:0/2246379159 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626400eec0 con 0x7f6270100c80 2024-03-30T02:36:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62701a0eb0 con 0x7f6270100c80 2024-03-30T02:36:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f626e000700 1 -- 172.21.15.67:0/2246379159 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264003c90 con 0x7f6270100c80 2024-03-30T02:36:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.971+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62701a12d0 con 0x7f6270100c80 2024-03-30T02:36:30.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.972+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6270108030 con 0x7f6270100c80 2024-03-30T02:36:30.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.974+0000 7f626e000700 1 -- 172.21.15.67:0/2246379159 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f626400cc70 con 0x7f6270100c80 2024-03-30T02:36:30.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.976+0000 7f626e000700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62580a1120 0x7f62580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.976+0000 7f626e000700 1 -- 172.21.15.67:0/2246379159 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f6264014070 con 0x7f6270100c80 2024-03-30T02:36:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.976+0000 7f6274a00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62580a1120 0x7f62580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.976+0000 7f6274a00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62580a1120 0x7f62580a35e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f625c00b580 tx=0x7f625c01a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:30.978+0000 7f626e000700 1 -- 172.21.15.67:0/2246379159 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f62640963a0 con 0x7f6270100c80 2024-03-30T02:36:31.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.238+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f627010c560 con 0x7f6270100c80 2024-03-30T02:36:31.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.245+0000 7f626e000700 1 -- 172.21.15.67:0/2246379159 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24479 (secure 0 0 0) 0x7f62640999c0 con 0x7f6270100c80 2024-03-30T02:36:31.245 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:31.245 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:36:28.218474+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.248+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62580a1120 msgr2=0x7f62580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.248+0000 7f6277e00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62580a1120 0x7f62580a35e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f625c00b580 tx=0x7f625c01a040 comp rx=0 tx=0).stop 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.248+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270100c80 msgr2=0x7f6270074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.248+0000 7f6277e00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270100c80 0x7f6270074b10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f626400e970 tx=0x7f626400ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.249+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 shutdown_connections 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.249+0000 7f6277e00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f62580a1120 0x7f62580a35e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.249+0000 7f6277e00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6270102810 0x7f62700736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.249+0000 7f6277e00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6270100c80 0x7f6270074b10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.249+0000 7f6277e00700 1 --2- 172.21.15.67:0/2246379159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6270101e70 0x7f62700731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.249+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 >> 172.21.15.67:0/2246379159 conn(0x7f62700fc1f0 msgr2=0x7f6270105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.249+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 shutdown_connections 2024-03-30T02:36:31.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.249+0000 7f6277e00700 1 -- 172.21.15.67:0/2246379159 wait complete. 2024-03-30T02:36:31.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:36:31.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:36:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.469+0000 7f7b07e00700 1 -- 172.21.15.67:0/703843186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00100d80 msgr2=0x7f7b001011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.469+0000 7f7b07e00700 1 --2- 172.21.15.67:0/703843186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00100d80 0x7f7b001011a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7af40099c0 tx=0x7f7af4009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.469+0000 7f7b07e00700 1 -- 172.21.15.67:0/703843186 shutdown_connections 2024-03-30T02:36:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.469+0000 7f7b07e00700 1 --2- 172.21.15.67:0/703843186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00100d80 0x7f7b001011a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.470+0000 7f7b07e00700 1 --2- 172.21.15.67:0/703843186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b00102910 0x7f7b001091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.470+0000 7f7b07e00700 1 --2- 172.21.15.67:0/703843186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b00101f70 0x7f7b001023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.470+0000 7f7b07e00700 1 -- 172.21.15.67:0/703843186 >> 172.21.15.67:0/703843186 conn(0x7f7b000fc2f0 msgr2=0x7f7b000fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.470+0000 7f7b07e00700 1 -- 172.21.15.67:0/703843186 shutdown_connections 2024-03-30T02:36:31.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.470+0000 7f7b07e00700 1 -- 172.21.15.67:0/703843186 wait complete. 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b07e00700 1 Processor -- start 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b07e00700 1 -- start start 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b07e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b00100d80 0x7f7b00198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b07e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b00101f70 0x7f7b00199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b07e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00102910 0x7f7b001a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b07e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b001a0ab0 con 0x7f7b00102910 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b07e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b001a0bf0 con 0x7f7b00100d80 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b07e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b001a0d80 con 0x7f7b00101f70 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b05e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00102910 0x7f7b001a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b05400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b00100d80 0x7f7b00198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:31.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b05e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00102910 0x7f7b001a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50258/0 (socket says 172.21.15.67:50258) 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b04a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b00101f70 0x7f7b00199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b05e00700 1 -- 172.21.15.67:0/1715145303 learned_addr learned my addr 172.21.15.67:0/1715145303 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.471+0000 7f7b05400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b00100d80 0x7f7b00198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39610/0 (socket says 172.21.15.67:39610) 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7b04a00700 1 -- 172.21.15.67:0/1715145303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b00100d80 msgr2=0x7f7b00198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7b04a00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b00100d80 0x7f7b00198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7b04a00700 1 -- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00102910 msgr2=0x7f7b001a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7b04a00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00102910 0x7f7b001a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7b04a00700 1 -- 172.21.15.67:0/1715145303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af8009c40 con 0x7f7b00101f70 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7b05e00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00102910 0x7f7b001a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7b04a00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b00101f70 0x7f7b00199510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00e9f0 tx=0x7f7aec00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7afe000700 1 -- 172.21.15.67:0/1715145303 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec004220 con 0x7f7b00101f70 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7afe000700 1 -- 172.21.15.67:0/1715145303 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aec004cf0 con 0x7f7b00101f70 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af40095f0 con 0x7f7b00101f70 2024-03-30T02:36:31.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.472+0000 7f7afe000700 1 -- 172.21.15.67:0/1715145303 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec00ce70 con 0x7f7b00101f70 2024-03-30T02:36:31.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.473+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b001a1390 con 0x7f7b00101f70 2024-03-30T02:36:31.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.473+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b00067af0 con 0x7f7b00101f70 2024-03-30T02:36:31.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.475+0000 7f7afe000700 1 -- 172.21.15.67:0/1715145303 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f7aec004750 con 0x7f7b00101f70 2024-03-30T02:36:31.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.477+0000 7f7afe000700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7ae80a1120 0x7f7ae80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:31.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.477+0000 7f7afe000700 1 -- 172.21.15.67:0/1715145303 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f7aec014070 con 0x7f7b00101f70 2024-03-30T02:36:31.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.477+0000 7f7b05400700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7ae80a1120 0x7f7ae80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:31.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.477+0000 7f7b05400700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7ae80a1120 0x7f7ae80a35e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009990 tx=0x7f7af4000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:31.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.479+0000 7f7afe000700 1 -- 172.21.15.67:0/1715145303 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7aec096610 con 0x7f7b00101f70 2024-03-30T02:36:31.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:31 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1292711349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:31.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:31 smithi082 ceph-mon[79176]: pgmap v289: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:31 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1292711349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:31 smithi067 ceph-mon[37808]: pgmap v289: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:31.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:31 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1292711349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:31.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:31 smithi170 ceph-mon[78798]: pgmap v289: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:31.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.695+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b001a1610 con 0x7f7b00101f70 2024-03-30T02:36:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.696+0000 7f7afe000700 1 -- 172.21.15.67:0/1715145303 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7aec004e60 con 0x7f7b00101f70 2024-03-30T02:36:31.696 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:36:31.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7ae80a1120 msgr2=0x7f7ae80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7ae80a1120 0x7f7ae80a35e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009990 tx=0x7f7af4000bc0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b00101f70 msgr2=0x7f7b00199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b00101f70 0x7f7b00199510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00e9f0 tx=0x7f7aec00ed00 comp rx=0 tx=0).stop 2024-03-30T02:36:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 shutdown_connections 2024-03-30T02:36:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7ae80a1120 0x7f7ae80a35e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b00102910 0x7f7b001a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b00100d80 0x7f7b00198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 --2- 172.21.15.67:0/1715145303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b00101f70 0x7f7b00199510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 >> 172.21.15.67:0/1715145303 conn(0x7f7b000fc2f0 msgr2=0x7f7b001077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 shutdown_connections 2024-03-30T02:36:31.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.698+0000 7f7b07e00700 1 -- 172.21.15.67:0/1715145303 wait complete. 2024-03-30T02:36:31.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:36:31.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.920+0000 7fef45200700 1 -- 172.21.15.67:0/2323655552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 msgr2=0x7fef401011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.920+0000 7fef45200700 1 --2- 172.21.15.67:0/2323655552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 0x7fef401011a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fef300099c0 tx=0x7fef30009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.920+0000 7fef45200700 1 -- 172.21.15.67:0/2323655552 shutdown_connections 2024-03-30T02:36:31.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.920+0000 7fef45200700 1 --2- 172.21.15.67:0/2323655552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 0x7fef401011a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.920+0000 7fef45200700 1 --2- 172.21.15.67:0/2323655552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef40102910 0x7fef401091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.920+0000 7fef45200700 1 --2- 172.21.15.67:0/2323655552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40101f70 0x7fef401023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.920+0000 7fef45200700 1 -- 172.21.15.67:0/2323655552 >> 172.21.15.67:0/2323655552 conn(0x7fef400fc2f0 msgr2=0x7fef400fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:31.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.920+0000 7fef45200700 1 -- 172.21.15.67:0/2323655552 shutdown_connections 2024-03-30T02:36:31.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.920+0000 7fef45200700 1 -- 172.21.15.67:0/2323655552 wait complete. 2024-03-30T02:36:31.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.921+0000 7fef45200700 1 Processor -- start 2024-03-30T02:36:31.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.921+0000 7fef45200700 1 -- start start 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef45200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 0x7fef40198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef45200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40101f70 0x7fef401994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef45200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef40102910 0x7fef401a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef45200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef401a0a20 con 0x7fef40100d80 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef45200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef401a0b60 con 0x7fef40102910 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef45200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef401a0cf0 con 0x7fef40101f70 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef3de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 0x7fef40198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef3de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 0x7fef40198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50276/0 (socket says 172.21.15.67:50276) 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef3de00700 1 -- 172.21.15.67:0/3136965259 learned_addr learned my addr 172.21.15.67:0/3136965259 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef3d400700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40101f70 0x7fef401994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef3e800700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef40102910 0x7fef401a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.922+0000 7fef3de00700 1 -- 172.21.15.67:0/3136965259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40101f70 msgr2=0x7fef401994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef3de00700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40101f70 0x7fef401994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef3de00700 1 -- 172.21.15.67:0/3136965259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef40102910 msgr2=0x7fef401a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef3de00700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef40102910 0x7fef401a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef3de00700 1 -- 172.21.15.67:0/3136965259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef300095f0 con 0x7fef40100d80 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef3de00700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 0x7fef40198f90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fef30003730 tx=0x7fef3000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef36a00700 1 -- 172.21.15.67:0/3136965259 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3001d070 con 0x7fef40100d80 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef36a00700 1 -- 172.21.15.67:0/3136965259 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef3000bda0 con 0x7fef40100d80 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef36a00700 1 -- 172.21.15.67:0/3136965259 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30004b30 con 0x7fef40100d80 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef401a0f70 con 0x7fef40100d80 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.923+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef401a12e0 con 0x7fef40100d80 2024-03-30T02:36:31.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.924+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef40067af0 con 0x7fef40100d80 2024-03-30T02:36:31.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.926+0000 7fef36a00700 1 -- 172.21.15.67:0/3136965259 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fef30004c90 con 0x7fef40100d80 2024-03-30T02:36:31.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.927+0000 7fef36a00700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef240a11a0 0x7fef240a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:31.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.927+0000 7fef36a00700 1 -- 172.21.15.67:0/3136965259 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fef300cddc0 con 0x7fef40100d80 2024-03-30T02:36:31.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.928+0000 7fef3d400700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef240a11a0 0x7fef240a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:31.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.928+0000 7fef3d400700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef240a11a0 0x7fef240a3660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef2c007c60 tx=0x7fef2c0073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:31.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:31.930+0000 7fef36a00700 1 -- 172.21.15.67:0/3136965259 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fef30097240 con 0x7fef40100d80 2024-03-30T02:36:32.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.148+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef4010c770 con 0x7fef40100d80 2024-03-30T02:36:32.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.149+0000 7fef36a00700 1 -- 172.21.15.67:0/3136965259 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef3009a860 con 0x7fef40100d80 2024-03-30T02:36:32.149 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef240a11a0 msgr2=0x7fef240a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef240a11a0 0x7fef240a3660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef2c007c60 tx=0x7fef2c0073d0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 msgr2=0x7fef40198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 0x7fef40198f90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fef30003730 tx=0x7fef3000fa00 comp rx=0 tx=0).stop 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 shutdown_connections 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fef240a11a0 0x7fef240a3660 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef40100d80 0x7fef40198f90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef40102910 0x7fef401a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 --2- 172.21.15.67:0/3136965259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef40101f70 0x7fef401994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 >> 172.21.15.67:0/3136965259 conn(0x7fef400fc2f0 msgr2=0x7fef40072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 shutdown_connections 2024-03-30T02:36:32.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.152+0000 7fef45200700 1 -- 172.21.15.67:0/3136965259 wait complete. 2024-03-30T02:36:32.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:36:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.374+0000 7fcb5fc00700 1 -- 172.21.15.67:0/3317066859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb58104110 msgr2=0x7fcb58106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.374+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/3317066859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb58104110 0x7fcb58106500 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009990 tx=0x7fcb44009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.374+0000 7fcb5fc00700 1 -- 172.21.15.67:0/3317066859 shutdown_connections 2024-03-30T02:36:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.374+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/3317066859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb58106a40 0x7fcb58108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.374+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/3317066859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb581017e0 0x7fcb58103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.374+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/3317066859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb58104110 0x7fcb58106500 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.374+0000 7fcb5fc00700 1 -- 172.21.15.67:0/3317066859 >> 172.21.15.67:0/3317066859 conn(0x7fcb580fb1d0 msgr2=0x7fcb580fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.374+0000 7fcb5fc00700 1 -- 172.21.15.67:0/3317066859 shutdown_connections 2024-03-30T02:36:32.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.374+0000 7fcb5fc00700 1 -- 172.21.15.67:0/3317066859 wait complete. 2024-03-30T02:36:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.375+0000 7fcb5fc00700 1 Processor -- start 2024-03-30T02:36:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.375+0000 7fcb5fc00700 1 -- start start 2024-03-30T02:36:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5fc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb581017e0 0x7fcb58198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:32.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5fc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb58104110 0x7fcb58199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5fc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb58106a40 0x7fcb581a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5fc00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581a0a40 con 0x7fcb58104110 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5fc00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581a0b80 con 0x7fcb581017e0 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5fc00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581a0d10 con 0x7fcb58106a40 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb57e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb58104110 0x7fcb58199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb581017e0 0x7fcb58198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb581017e0 0x7fcb58198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39626/0 (socket says 172.21.15.67:39626) 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5d200700 1 -- 172.21.15.67:0/961910748 learned_addr learned my addr 172.21.15.67:0/961910748 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb57e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb58104110 0x7fcb58199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50300/0 (socket says 172.21.15.67:50300) 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5dc00700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb58106a40 0x7fcb581a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5d200700 1 -- 172.21.15.67:0/961910748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb58106a40 msgr2=0x7fcb581a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5d200700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb58106a40 0x7fcb581a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5d200700 1 -- 172.21.15.67:0/961910748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb58104110 msgr2=0x7fcb58199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5d200700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb58104110 0x7fcb58199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.376+0000 7fcb5d200700 1 -- 172.21.15.67:0/961910748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440095f0 con 0x7fcb581017e0 2024-03-30T02:36:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.377+0000 7fcb5d200700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb581017e0 0x7fcb58198f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9a0 tx=0x7fcb4c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.377+0000 7fcb55600700 1 -- 172.21.15.67:0/961910748 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c009790 con 0x7fcb581017e0 2024-03-30T02:36:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.377+0000 7fcb55600700 1 -- 172.21.15.67:0/961910748 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb4c010430 con 0x7fcb581017e0 2024-03-30T02:36:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.377+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb581a0ff0 con 0x7fcb581017e0 2024-03-30T02:36:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.377+0000 7fcb55600700 1 -- 172.21.15.67:0/961910748 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c018680 con 0x7fcb581017e0 2024-03-30T02:36:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.377+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581a1410 con 0x7fcb581017e0 2024-03-30T02:36:32.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.378+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb58108610 con 0x7fcb581017e0 2024-03-30T02:36:32.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.380+0000 7fcb55600700 1 -- 172.21.15.67:0/961910748 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fcb4c0105a0 con 0x7fcb581017e0 2024-03-30T02:36:32.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.381+0000 7fcb55600700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcb400a1120 0x7fcb400a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:32.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.381+0000 7fcb55600700 1 -- 172.21.15.67:0/961910748 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fcb4c014070 con 0x7fcb581017e0 2024-03-30T02:36:32.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.382+0000 7fcb57e00700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcb400a1120 0x7fcb400a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:32.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.382+0000 7fcb57e00700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcb400a1120 0x7fcb400a35e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400b2e0 tx=0x7fcb44005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.384+0000 7fcb55600700 1 -- 172.21.15.67:0/961910748 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcb4c0961b0 con 0x7fcb581017e0 2024-03-30T02:36:32.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:32 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2246379159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:32.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:32 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1715145303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:32.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:32 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3136965259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:32.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.600+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb5810c7d0 con 0x7fcb581017e0 2024-03-30T02:36:32.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.601+0000 7fcb55600700 1 -- 172.21.15.67:0/961910748 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb4c0997d0 con 0x7fcb581017e0 2024-03-30T02:36:32.601 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcb400a1120 msgr2=0x7fcb400a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcb400a1120 0x7fcb400a35e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400b2e0 tx=0x7fcb44005fb0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb581017e0 msgr2=0x7fcb58198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb581017e0 0x7fcb58198f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e9a0 tx=0x7fcb4c00ecb0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 shutdown_connections 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcb400a1120 0x7fcb400a35e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb58106a40 0x7fcb581a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb581017e0 0x7fcb58198f10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 --2- 172.21.15.67:0/961910748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcb58104110 0x7fcb58199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.603+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 >> 172.21.15.67:0/961910748 conn(0x7fcb580fb1d0 msgr2=0x7fcb581010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.604+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 shutdown_connections 2024-03-30T02:36:32.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.604+0000 7fcb5fc00700 1 -- 172.21.15.67:0/961910748 wait complete. 2024-03-30T02:36:32.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:32 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2246379159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:32.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:32 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1715145303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:32.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:32 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3136965259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:32.623 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:36:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:32 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2246379159' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:32 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1715145303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:32 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3136965259' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.823+0000 7f74f6a00700 1 -- 172.21.15.67:0/2532155099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0101f70 msgr2=0x7f74f01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.823+0000 7f74f6a00700 1 --2- 172.21.15.67:0/2532155099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0101f70 0x7f74f01023d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009990 tx=0x7f74dc009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.824+0000 7f74f6a00700 1 -- 172.21.15.67:0/2532155099 shutdown_connections 2024-03-30T02:36:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.824+0000 7f74f6a00700 1 --2- 172.21.15.67:0/2532155099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74f0102910 0x7f74f01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.824+0000 7f74f6a00700 1 --2- 172.21.15.67:0/2532155099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74f0100d80 0x7f74f01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.824+0000 7f74f6a00700 1 --2- 172.21.15.67:0/2532155099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0101f70 0x7f74f01023d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.824+0000 7f74f6a00700 1 -- 172.21.15.67:0/2532155099 >> 172.21.15.67:0/2532155099 conn(0x7f74f00fc2f0 msgr2=0x7f74f00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.824+0000 7f74f6a00700 1 -- 172.21.15.67:0/2532155099 shutdown_connections 2024-03-30T02:36:32.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.824+0000 7f74f6a00700 1 -- 172.21.15.67:0/2532155099 wait complete. 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.825+0000 7f74f6a00700 1 Processor -- start 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.825+0000 7f74f6a00700 1 -- start start 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.825+0000 7f74f6a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74f0100d80 0x7f74f0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.825+0000 7f74f6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74f0101f70 0x7f74f00731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.825+0000 7f74f6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0102910 0x7f74f00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.825+0000 7f74f6a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f0073e80 con 0x7f74f0101f70 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.825+0000 7f74f6a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f0073ff0 con 0x7f74f0100d80 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.825+0000 7f74f6a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f01a0db0 con 0x7f74f0102910 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.825+0000 7f74ef400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74f0101f70 0x7f74f00731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74ef400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74f0101f70 0x7f74f00731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50318/0 (socket says 172.21.15.67:50318) 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74ef400700 1 -- 172.21.15.67:0/201029007 learned_addr learned my addr 172.21.15.67:0/201029007 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:32.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74efe00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74f0100d80 0x7f74f0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74f4a00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0102910 0x7f74f00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74ef400700 1 -- 172.21.15.67:0/201029007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0102910 msgr2=0x7f74f00736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74ef400700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0102910 0x7f74f00736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74ef400700 1 -- 172.21.15.67:0/201029007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74f0100d80 msgr2=0x7f74f0074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74ef400700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74f0100d80 0x7f74f0074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74ef400700 1 -- 172.21.15.67:0/201029007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc0095f0 con 0x7f74f0101f70 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74efe00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74f0100d80 0x7f74f0074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74ef400700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74f0101f70 0x7f74f00731a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f74dc005910 tx=0x7f74dc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.826+0000 7f74ecc00700 1 -- 172.21.15.67:0/201029007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc01d070 con 0x7f74f0101f70 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.827+0000 7f74ecc00700 1 -- 172.21.15.67:0/201029007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74dc021410 con 0x7f74f0101f70 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.827+0000 7f74ecc00700 1 -- 172.21.15.67:0/201029007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc005720 con 0x7f74f0101f70 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.827+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74f01a0fe0 con 0x7f74f0101f70 2024-03-30T02:36:32.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.827+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f01a1350 con 0x7f74f0101f70 2024-03-30T02:36:32.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.828+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f0108250 con 0x7f74f0101f70 2024-03-30T02:36:32.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.830+0000 7f74ecc00700 1 -- 172.21.15.67:0/201029007 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f74dc021580 con 0x7f74f0101f70 2024-03-30T02:36:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.832+0000 7f74ecc00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f74d40a1120 0x7f74d40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.832+0000 7f74ecc00700 1 -- 172.21.15.67:0/201029007 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f74dc0cdfc0 con 0x7f74f0101f70 2024-03-30T02:36:32.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.832+0000 7f74efe00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f74d40a1120 0x7f74d40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:32.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.833+0000 7f74efe00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f74d40a1120 0x7f74d40a35e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f74f0101dd0 tx=0x7f74e0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:32.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:32.836+0000 7f74ecc00700 1 -- 172.21.15.67:0/201029007 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f74dc0974a0 con 0x7f74f0101f70 2024-03-30T02:36:33.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.052+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f74f002cd00 con 0x7f74f0101f70 2024-03-30T02:36:33.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.052+0000 7f74ecc00700 1 -- 172.21.15.67:0/201029007 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f74dc09aac0 con 0x7f74f0101f70 2024-03-30T02:36:33.053 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.054+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f74d40a1120 msgr2=0x7f74d40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f74d40a1120 0x7f74d40a35e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f74f0101dd0 tx=0x7f74e0009260 comp rx=0 tx=0).stop 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74f0101f70 msgr2=0x7f74f00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74f0101f70 0x7f74f00731a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f74dc005910 tx=0x7f74dc009580 comp rx=0 tx=0).stop 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 shutdown_connections 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f74d40a1120 0x7f74d40a35e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74f0102910 0x7f74f00736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74f0100d80 0x7f74f0074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 --2- 172.21.15.67:0/201029007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74f0101f70 0x7f74f00731a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 >> 172.21.15.67:0/201029007 conn(0x7f74f00fc2f0 msgr2=0x7f74f0105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:33.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 shutdown_connections 2024-03-30T02:36:33.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.055+0000 7f74f6a00700 1 -- 172.21.15.67:0/201029007 wait complete. 2024-03-30T02:36:33.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:36:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.273+0000 7fa047200700 1 -- 172.21.15.67:0/3364680542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048101e40 msgr2=0x7fa0481022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.273+0000 7fa047200700 1 --2- 172.21.15.67:0/3364680542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048101e40 0x7fa0481022a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa034009990 tx=0x7fa034009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.273+0000 7fa047200700 1 -- 172.21.15.67:0/3364680542 shutdown_connections 2024-03-30T02:36:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.273+0000 7fa047200700 1 --2- 172.21.15.67:0/3364680542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa048100c50 0x7fa048101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.273+0000 7fa047200700 1 --2- 172.21.15.67:0/3364680542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0481027e0 0x7fa048109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.273+0000 7fa047200700 1 --2- 172.21.15.67:0/3364680542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa048101e40 0x7fa0481022a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.273+0000 7fa047200700 1 -- 172.21.15.67:0/3364680542 >> 172.21.15.67:0/3364680542 conn(0x7fa0480fc1c0 msgr2=0x7fa0480fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.273+0000 7fa047200700 1 -- 172.21.15.67:0/3364680542 shutdown_connections 2024-03-30T02:36:33.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.273+0000 7fa047200700 1 -- 172.21.15.67:0/3364680542 wait complete. 2024-03-30T02:36:33.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.274+0000 7fa047200700 1 Processor -- start 2024-03-30T02:36:33.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.274+0000 7fa047200700 1 -- start start 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa047200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa048100c50 0x7fa048198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa047200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa048101e40 0x7fa048199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa047200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0481027e0 0x7fa0481a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa047200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481a0930 con 0x7fa048101e40 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa047200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481a0a70 con 0x7fa048100c50 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa047200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481a0c00 con 0x7fa0481027e0 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa045400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa048101e40 0x7fa048199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa045e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa048100c50 0x7fa048198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa045400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa048101e40 0x7fa048199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50340/0 (socket says 172.21.15.67:50340) 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa045400700 1 -- 172.21.15.67:0/2048710499 learned_addr learned my addr 172.21.15.67:0/2048710499 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa045e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa048100c50 0x7fa048198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39662/0 (socket says 172.21.15.67:39662) 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa046800700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0481027e0 0x7fa0481a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa045400700 1 -- 172.21.15.67:0/2048710499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0481027e0 msgr2=0x7fa0481a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.275+0000 7fa045400700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0481027e0 0x7fa0481a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa045400700 1 -- 172.21.15.67:0/2048710499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa048100c50 msgr2=0x7fa048198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa045400700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa048100c50 0x7fa048198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa045400700 1 -- 172.21.15.67:0/2048710499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0340095f0 con 0x7fa048101e40 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa046800700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0481027e0 0x7fa0481a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa045400700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa048101e40 0x7fa048199390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa034000c00 tx=0x7fa034005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa03ea00700 1 -- 172.21.15.67:0/2048710499 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03401d070 con 0x7fa048101e40 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa03ea00700 1 -- 172.21.15.67:0/2048710499 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa034022470 con 0x7fa048101e40 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0481a0e80 con 0x7fa048101e40 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa03ea00700 1 -- 172.21.15.67:0/2048710499 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa034017650 con 0x7fa048101e40 2024-03-30T02:36:33.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.276+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0481a1240 con 0x7fa048101e40 2024-03-30T02:36:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.277+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0480679a0 con 0x7fa048101e40 2024-03-30T02:36:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.279+0000 7fa03ea00700 1 -- 172.21.15.67:0/2048710499 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa034022850 con 0x7fa048101e40 2024-03-30T02:36:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.280+0000 7fa03ea00700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa02c0a1120 0x7fa02c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.281+0000 7fa03ea00700 1 -- 172.21.15.67:0/2048710499 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fa0340ce1a0 con 0x7fa048101e40 2024-03-30T02:36:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.281+0000 7fa045e00700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa02c0a1120 0x7fa02c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.281+0000 7fa045e00700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa02c0a1120 0x7fa02c0a35e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa048101ca0 tx=0x7fa038009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.284+0000 7fa03ea00700 1 -- 172.21.15.67:0/2048710499 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa034097740 con 0x7fa048101e40 2024-03-30T02:36:33.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:33 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/961910748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:33.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:33 smithi067 ceph-mon[37808]: pgmap v290: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:33.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:33 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/201029007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:33.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.551+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa04810c4b0 con 0x7fa048101e40 2024-03-30T02:36:33.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.552+0000 7fa03ea00700 1 -- 172.21.15.67:0/2048710499 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10909 (secure 0 0 0) 0x7fa034027020 con 0x7fa048101e40 2024-03-30T02:36:33.552 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:33.553 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:35:20.119112+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:36:33.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa02c0a1120 msgr2=0x7fa02c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa02c0a1120 0x7fa02c0a35e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa048101ca0 tx=0x7fa038009260 comp rx=0 tx=0).stop 2024-03-30T02:36:33.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa048101e40 msgr2=0x7fa048199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa048101e40 0x7fa048199390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa034000c00 tx=0x7fa034005740 comp rx=0 tx=0).stop 2024-03-30T02:36:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 shutdown_connections 2024-03-30T02:36:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa02c0a1120 0x7fa02c0a35e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0481027e0 0x7fa0481a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa048100c50 0x7fa048198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 --2- 172.21.15.67:0/2048710499 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa048101e40 0x7fa048199390 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 >> 172.21.15.67:0/2048710499 conn(0x7fa0480fc1c0 msgr2=0x7fa048107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 shutdown_connections 2024-03-30T02:36:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.554+0000 7fa047200700 1 -- 172.21.15.67:0/2048710499 wait complete. 2024-03-30T02:36:33.556 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 28 2024-03-30T02:36:33.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:33.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:33 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/961910748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:33.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:33 smithi170 ceph-mon[78798]: pgmap v290: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:33.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:33 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/201029007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:33 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/961910748' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:33 smithi082 ceph-mon[79176]: pgmap v290: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:33 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/201029007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:33.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.772+0000 7fab22800700 1 -- 172.21.15.67:0/2101804490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1c101e70 msgr2=0x7fab1c1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.772+0000 7fab22800700 1 --2- 172.21.15.67:0/2101804490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1c101e70 0x7fab1c1022d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab080098f0 tx=0x7fab08009c00 comp rx=0 tx=0).stop 2024-03-30T02:36:33.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.772+0000 7fab22800700 1 -- 172.21.15.67:0/2101804490 shutdown_connections 2024-03-30T02:36:33.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.772+0000 7fab22800700 1 --2- 172.21.15.67:0/2101804490 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c102810 0x7fab1c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.772+0000 7fab22800700 1 --2- 172.21.15.67:0/2101804490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1c100c80 0x7fab1c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.772+0000 7fab22800700 1 --2- 172.21.15.67:0/2101804490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1c101e70 0x7fab1c1022d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.772+0000 7fab22800700 1 -- 172.21.15.67:0/2101804490 >> 172.21.15.67:0/2101804490 conn(0x7fab1c0fc1f0 msgr2=0x7fab1c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:33.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.772+0000 7fab22800700 1 -- 172.21.15.67:0/2101804490 shutdown_connections 2024-03-30T02:36:33.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.772+0000 7fab22800700 1 -- 172.21.15.67:0/2101804490 wait complete. 2024-03-30T02:36:33.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.773+0000 7fab22800700 1 Processor -- start 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.773+0000 7fab22800700 1 -- start start 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.773+0000 7fab22800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c100c80 0x7fab1c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.773+0000 7fab22800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1c101e70 0x7fab1c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.773+0000 7fab22800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1c102810 0x7fab1c1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.773+0000 7fab22800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1a0940 con 0x7fab1c100c80 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.773+0000 7fab22800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1a0a80 con 0x7fab1c101e70 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.773+0000 7fab22800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1a0c10 con 0x7fab1c102810 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.773+0000 7fab1b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c100c80 0x7fab1c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c100c80 0x7fab1c198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50354/0 (socket says 172.21.15.67:50354) 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1b400700 1 -- 172.21.15.67:0/2842807855 learned_addr learned my addr 172.21.15.67:0/2842807855 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1aa00700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1c101e70 0x7fab1c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1be00700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1c102810 0x7fab1c1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:33.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1b400700 1 -- 172.21.15.67:0/2842807855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1c102810 msgr2=0x7fab1c1a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1b400700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1c102810 0x7fab1c1a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1b400700 1 -- 172.21.15.67:0/2842807855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1c101e70 msgr2=0x7fab1c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1b400700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1c101e70 0x7fab1c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1b400700 1 -- 172.21.15.67:0/2842807855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c0095f0 con 0x7fab1c100c80 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.774+0000 7fab1b400700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c100c80 0x7fab1c198e10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00ebf0 tx=0x7fab0c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.775+0000 7fab13e00700 1 -- 172.21.15.67:0/2842807855 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c009a10 con 0x7fab1c100c80 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.775+0000 7fab13e00700 1 -- 172.21.15.67:0/2842807855 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab0c00cc90 con 0x7fab1c100c80 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.775+0000 7fab13e00700 1 -- 172.21.15.67:0/2842807855 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c018750 con 0x7fab1c100c80 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.775+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab08009520 con 0x7fab1c100c80 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.775+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1a11c0 con 0x7fab1c100c80 2024-03-30T02:36:33.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.776+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c067a80 con 0x7fab1c100c80 2024-03-30T02:36:33.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.778+0000 7fab13e00700 1 -- 172.21.15.67:0/2842807855 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fab0c0188b0 con 0x7fab1c100c80 2024-03-30T02:36:33.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.779+0000 7fab13e00700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fab000a11a0 0x7fab000a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:33.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.779+0000 7fab13e00700 1 -- 172.21.15.67:0/2842807855 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fab0c014070 con 0x7fab1c100c80 2024-03-30T02:36:33.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.779+0000 7fab1aa00700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fab000a11a0 0x7fab000a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:33.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.780+0000 7fab1aa00700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fab000a11a0 0x7fab000a3660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fab08005e90 tx=0x7fab08019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:33.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:33.783+0000 7fab13e00700 1 -- 172.21.15.67:0/2842807855 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fab1c067a80 con 0x7fab1c100c80 2024-03-30T02:36:34.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.034+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab1c10c490 con 0x7fab1c100c80 2024-03-30T02:36:34.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.036+0000 7fab13e00700 1 -- 172.21.15.67:0/2842807855 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24479 (secure 0 0 0) 0x7fab0c0965f0 con 0x7fab1c100c80 2024-03-30T02:36:34.037 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:34.037 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:36:28.218474+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:34.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.038+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fab000a11a0 msgr2=0x7fab000a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.038+0000 7fab22800700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fab000a11a0 0x7fab000a3660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fab08005e90 tx=0x7fab08019040 comp rx=0 tx=0).stop 2024-03-30T02:36:34.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.038+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c100c80 msgr2=0x7fab1c198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.038+0000 7fab22800700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c100c80 0x7fab1c198e10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00ebf0 tx=0x7fab0c00ef00 comp rx=0 tx=0).stop 2024-03-30T02:36:34.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.039+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 shutdown_connections 2024-03-30T02:36:34.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.039+0000 7fab22800700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fab000a11a0 0x7fab000a3660 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.039+0000 7fab22800700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab1c100c80 0x7fab1c198e10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.039+0000 7fab22800700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab1c102810 0x7fab1c1a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.039+0000 7fab22800700 1 --2- 172.21.15.67:0/2842807855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab1c101e70 0x7fab1c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.039+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 >> 172.21.15.67:0/2842807855 conn(0x7fab1c0fc1f0 msgr2=0x7fab1c107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.039+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 shutdown_connections 2024-03-30T02:36:34.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.039+0000 7fab22800700 1 -- 172.21.15.67:0/2842807855 wait complete. 2024-03-30T02:36:34.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-30T02:34:46.713132+0000', 'modified': '2024-03-30T02:35:20.119112+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24578, 'mds_1': 24560, 'mds_2': 24541, 'mds_3': 24539, 'mds_4': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14526': {'gid': 14526, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6841/853027854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6840', 'nonce': 853027854}, {'type': 'v1', 'addr': '172.21.15.67:6841', 'nonce': 853027854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24466': {'gid': 24466, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/3749225376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 3749225376}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 3749225376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24476': {'gid': 24476, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6833/1365991767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6832', 'nonce': 1365991767}, {'type': 'v1', 'addr': '172.21.15.82:6833', 'nonce': 1365991767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24512': {'gid': 24512, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/3401372629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 3401372629}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 3401372629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/1685909437', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 1685909437}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 1685909437}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.170:6837/500476759', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 500476759}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 500476759}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.82:6837/1183051781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 1183051781}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 1183051781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24541': {'gid': 24541, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.170:6839/2570108618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 2570108618}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 2570108618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24560': {'gid': 24560, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.67:6839/2974132199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6838', 'nonce': 2974132199}, {'type': 'v1', 'addr': '172.21.15.67:6839', 'nonce': 2974132199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6839/604601206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 604601206}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 604601206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-30T02:36:34.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-30T02:36:34.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T02:36:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.259+0000 7f16be000700 1 -- 172.21.15.67:0/339576363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b8104110 msgr2=0x7f16b8106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.259+0000 7f16be000700 1 --2- 172.21.15.67:0/339576363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b8104110 0x7f16b8106500 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009990 tx=0x7f16a4009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.259+0000 7f16be000700 1 -- 172.21.15.67:0/339576363 shutdown_connections 2024-03-30T02:36:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.259+0000 7f16be000700 1 --2- 172.21.15.67:0/339576363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16b81017e0 0x7f16b8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.259+0000 7f16be000700 1 --2- 172.21.15.67:0/339576363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b8106a40 0x7f16b8108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.259+0000 7f16be000700 1 --2- 172.21.15.67:0/339576363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b8104110 0x7f16b8106500 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.259+0000 7f16be000700 1 -- 172.21.15.67:0/339576363 >> 172.21.15.67:0/339576363 conn(0x7f16b80fb1d0 msgr2=0x7f16b80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.259+0000 7f16be000700 1 -- 172.21.15.67:0/339576363 shutdown_connections 2024-03-30T02:36:34.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.259+0000 7f16be000700 1 -- 172.21.15.67:0/339576363 wait complete. 2024-03-30T02:36:34.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16be000700 1 Processor -- start 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16be000700 1 -- start start 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16be000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b81017e0 0x7f16b8198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16be000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b8104110 0x7f16b8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16be000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16b8106a40 0x7f16b81a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16be000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b81a0a40 con 0x7f16b8106a40 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16be000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b81a0b80 con 0x7f16b81017e0 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16be000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b81a0d10 con 0x7f16b8104110 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16b7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b81017e0 0x7f16b8198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16b7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b81017e0 0x7f16b8198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39714/0 (socket says 172.21.15.67:39714) 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16b7400700 1 -- 172.21.15.67:0/2902460253 learned_addr learned my addr 172.21.15.67:0/2902460253 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.260+0000 7f16b7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16b8106a40 0x7f16b81a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16b6a00700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b8104110 0x7f16b8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16b7400700 1 -- 172.21.15.67:0/2902460253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b8104110 msgr2=0x7f16b8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16b7400700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b8104110 0x7f16b8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16b7400700 1 -- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16b8106a40 msgr2=0x7f16b81a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16b7400700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16b8106a40 0x7f16b81a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16b7400700 1 -- 172.21.15.67:0/2902460253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8009520 con 0x7f16b81017e0 2024-03-30T02:36:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16b7e00700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16b8106a40 0x7f16b81a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16b7400700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b81017e0 0x7f16b8198f10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f16a800eb30 tx=0x7f16a800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.261+0000 7f16afe00700 1 -- 172.21.15.67:0/2902460253 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8009880 con 0x7f16b81017e0 2024-03-30T02:36:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.262+0000 7f16be000700 1 -- 172.21.15.67:0/2902460253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a40095f0 con 0x7f16b81017e0 2024-03-30T02:36:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.262+0000 7f16afe00700 1 -- 172.21.15.67:0/2902460253 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a80044e0 con 0x7f16b81017e0 2024-03-30T02:36:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.262+0000 7f16afe00700 1 -- 172.21.15.67:0/2902460253 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8003d80 con 0x7f16b81017e0 2024-03-30T02:36:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.262+0000 7f16be000700 1 -- 172.21.15.67:0/2902460253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b81a1290 con 0x7f16b81017e0 2024-03-30T02:36:34.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.265+0000 7f16afe00700 1 -- 172.21.15.67:0/2902460253 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f16a8003f20 con 0x7f16b81017e0 2024-03-30T02:36:34.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.266+0000 7f16afe00700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f169c0a1120 0x7f169c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.266+0000 7f16afe00700 1 -- 172.21.15.67:0/2902460253 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f16a8014070 con 0x7f16b81017e0 2024-03-30T02:36:34.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.266+0000 7f16d1b96b80 1 Processor -- start 2024-03-30T02:36:34.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.266+0000 7f16b6a00700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f169c0a1120 0x7f169c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.266+0000 7f16d1b96b80 1 -- start start 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f03ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f049a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 0x55b419f05400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419e5b940 con 0x55b419f04f80 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419e59300 con 0x55b419f03780 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419e54bf0 con 0x55b419f040e0 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b6a00700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f169c0a1120 0x7f169c0a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f16a400b2e0 tx=0x7f16a4005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f03ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f03ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39722/0 (socket says 172.21.15.67:39722) 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b7400700 1 -- 172.21.15.67:0/3056715714 learned_addr learned my addr 172.21.15.67:0/3056715714 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b7e00700 1 --2- 172.21.15.67:0/3056715714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 0x55b419f05400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b6a00700 1 --2- 172.21.15.67:0/3056715714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b7400700 1 -- 172.21.15.67:0/3056715714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 msgr2=0x55b419f049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b7400700 1 --2- 172.21.15.67:0/3056715714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b7400700 1 -- 172.21.15.67:0/3056715714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 msgr2=0x55b419f05400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b7400700 1 --2- 172.21.15.67:0/3056715714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 0x55b419f05400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.267+0000 7f16b7400700 1 -- 172.21.15.67:0/3056715714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b419cd6360 con 0x55b419f03780 2024-03-30T02:36:34.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.268+0000 7f16b7400700 1 --2- 172.21.15.67:0/3056715714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f03ba0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16a8096760 tx=0x7f16a8099ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70696dbc71a2e98a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.268+0000 7f16a3e00700 1 -- 172.21.15.67:0/3056715714 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80af070 con 0x55b419f03780 2024-03-30T02:36:34.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.268+0000 7f16a3e00700 1 -- 172.21.15.67:0/3056715714 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a809a260 con 0x55b419f03780 2024-03-30T02:36:34.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.268+0000 7f16a3e00700 1 -- 172.21.15.67:0/3056715714 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a801fde0 con 0x55b419f03780 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 -- 172.21.15.67:0/3056715714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 msgr2=0x55b419f03ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/3056715714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f03ba0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f16a8096760 tx=0x7f16a8099ea0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 -- 172.21.15.67:0/3056715714 shutdown_connections 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/3056715714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 0x55b419f05400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/3056715714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f03ba0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/3056715714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 -- 172.21.15.67:0/3056715714 >> 172.21.15.67:0/3056715714 conn(0x55b419e62650 msgr2=0x55b419efe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 -- 172.21.15.67:0/3056715714 shutdown_connections 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 -- 172.21.15.67:0/3056715714 wait complete. 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 -- start start 2024-03-30T02:36:34.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.269+0000 7f16d1b96b80 1 Processor -- start 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f03780 0x55b419f0d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f0db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f04f80 0x55b419f14b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419df0d30 con 0x55b419f03780 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419df09d0 con 0x55b419f04f80 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419cd8650 con 0x55b419f040e0 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16d1b96b80 10 client.0 _ll_register_callbacks cb 0x55b419dfa420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16b7400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f03780 0x55b419f0d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16b6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f0db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16b6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f0db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40810/0 (socket says 172.21.15.67:40810) 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16b6a00700 1 -- 172.21.15.67:0/1291735491 learned_addr learned my addr 172.21.15.67:0/1291735491 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16b7400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f03780 0x55b419f0d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50394/0 (socket says 172.21.15.67:50394) 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16b7e00700 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f04f80 0x55b419f14b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16b6a00700 1 -- 172.21.15.67:0/1291735491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f04f80 msgr2=0x55b419f14b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.270+0000 7f16b6a00700 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f04f80 0x55b419f14b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.271+0000 7f16b6a00700 1 -- 172.21.15.67:0/1291735491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f03780 msgr2=0x55b419f0d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.271+0000 7f16b6a00700 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f03780 0x55b419f0d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.271+0000 7f16b6a00700 1 -- 172.21.15.67:0/1291735491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b81a1290 con 0x55b419f040e0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.271+0000 7f16b7400700 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f03780 0x55b419f0d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.271+0000 7f16b6a00700 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f0db40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009990 tx=0x7f16a401a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3bc868d4d5bc0e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.271+0000 7f16a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.271+0000 7f16a0c00700 1 -- 172.21.15.67:0/1291735491 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4028030 con 0x55b419f040e0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.271+0000 7f16d1b96b80 1 -- 172.21.15.67:0/1291735491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b419e15330 con 0x55b419f040e0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.271+0000 7f16a0c00700 1 -- 172.21.15.67:0/1291735491 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f16a40109c0 con 0x55b419f040e0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16a0c00700 1 -- 172.21.15.67:0/1291735491 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a4010b80 con 0x55b419f040e0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16d1b96b80 10 client.25345 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16d1b96b80 1 -- 172.21.15.67:0/1291735491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b419c82b10 con 0x55b419f040e0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16a0c00700 1 -- 172.21.15.67:0/1291735491 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b419e15330 con 0x55b419f040e0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16a0c00700 1 -- 172.21.15.67:0/1291735491 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f16a4030cf0 con 0x55b419f040e0 2024-03-30T02:36:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16d1b96b80 10 client.25345 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:34.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16d1b96b80 10 client.25345 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T02:36:34.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16d1b96b80 20 client.25345 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:34.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55b419f17d20 0x55b419f1db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16d1b96b80 4 client.25345 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:34.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.272+0000 7f16d1b96b80 1 -- 172.21.15.67:0/1291735491 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b419e4d6b0 con 0x55b419f17d20 2024-03-30T02:36:34.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.273+0000 7f16b7400700 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55b419f17d20 0x55b419f1db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.273+0000 7f16a0c00700 10 client.25345 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:36:34.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.273+0000 7f16b7400700 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55b419f17d20 0x55b419f1db30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8e978474c0cda91 server_cookie=fc6212022394513e in_seq=0 out_seq=0 2024-03-30T02:36:34.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16a0c00700 1 -- 172.21.15.67:0/1291735491 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b419e4d6b0 con 0x55b419f17d20 2024-03-30T02:36:34.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16a0c00700 10 client.25345 handle_command_reply: tid=0 2024-03-30T02:36:34.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16d1b96b80 1 client.25345 shutdown 2024-03-30T02:36:34.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16d1b96b80 1 -- 172.21.15.67:0/1291735491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 msgr2=0x55b419f0db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f0db40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009990 tx=0x7f16a401a050 comp rx=0 tx=0).stop 2024-03-30T02:36:34.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16d1b96b80 1 -- 172.21.15.67:0/1291735491 shutdown_connections 2024-03-30T02:36:34.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f03780 0x55b419f0d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55b419f17d20 0x55b419f1db30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f04f80 0x55b419f14b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/1291735491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f040e0 0x55b419f0db40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16d1b96b80 1 -- 172.21.15.67:0/1291735491 >> 172.21.15.67:0/1291735491 conn(0x55b419e62650 msgr2=0x55b419f02120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.274+0000 7f16a0c00700 0 client.25345 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:36:34.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.275+0000 7f16d1b96b80 1 -- 172.21.15.67:0/1291735491 shutdown_connections 2024-03-30T02:36:34.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.275+0000 7f16d1b96b80 1 -- 172.21.15.67:0/1291735491 wait complete. 2024-03-30T02:36:34.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.275+0000 7f16d1b96b80 20 client.25345 trim_cache size 0 max 16384 2024-03-30T02:36:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.293+0000 7f16d1b96b80 1 Processor -- start 2024-03-30T02:36:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.293+0000 7f16d1b96b80 1 -- start start 2024-03-30T02:36:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.293+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f03780 0x55b419efdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.293+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f040e0 0x55b419f14b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 0x55b419f21b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419e4d990 con 0x55b419f04f80 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419e51850 con 0x55b419f040e0 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419e12ad0 con 0x55b419f03780 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 0x55b419f21b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b6a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f040e0 0x55b419f14b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b6a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f040e0 0x55b419f14b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39742/0 (socket says 172.21.15.67:39742) 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b6a00700 1 -- 172.21.15.67:0/621352776 learned_addr learned my addr 172.21.15.67:0/621352776 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b7400700 1 --2- 172.21.15.67:0/621352776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f03780 0x55b419efdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b6a00700 1 -- 172.21.15.67:0/621352776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f03780 msgr2=0x55b419efdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b6a00700 1 --2- 172.21.15.67:0/621352776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f03780 0x55b419efdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b6a00700 1 -- 172.21.15.67:0/621352776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 msgr2=0x55b419f21b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b6a00700 1 --2- 172.21.15.67:0/621352776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 0x55b419f21b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b6a00700 1 -- 172.21.15.67:0/621352776 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b419e1ae50 con 0x55b419f040e0 2024-03-30T02:36:34.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b7e00700 1 --2- 172.21.15.67:0/621352776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 0x55b419f21b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.294+0000 7f16b6a00700 1 --2- 172.21.15.67:0/621352776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f040e0 0x55b419f14b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f16a4000c00 tx=0x7f16a400b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66bdd1c2d536db5a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16ad200700 1 -- 172.21.15.67:0/621352776 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4028070 con 0x55b419f040e0 2024-03-30T02:36:34.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16ad200700 1 -- 172.21.15.67:0/621352776 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a401e040 con 0x55b419f040e0 2024-03-30T02:36:34.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16ad200700 1 -- 172.21.15.67:0/621352776 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4010bc0 con 0x55b419f040e0 2024-03-30T02:36:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16d1b96b80 1 -- 172.21.15.67:0/621352776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f040e0 msgr2=0x55b419f14b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/621352776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f040e0 0x55b419f14b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f16a4000c00 tx=0x7f16a400b850 comp rx=0 tx=0).stop 2024-03-30T02:36:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16d1b96b80 1 -- 172.21.15.67:0/621352776 shutdown_connections 2024-03-30T02:36:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/621352776 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f04f80 0x55b419f21b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/621352776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f03780 0x55b419efdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/621352776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f040e0 0x55b419f14b40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16d1b96b80 1 -- 172.21.15.67:0/621352776 >> 172.21.15.67:0/621352776 conn(0x55b419e62650 msgr2=0x55b419f0b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16d1b96b80 1 -- 172.21.15.67:0/621352776 shutdown_connections 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.295+0000 7f16d1b96b80 1 -- 172.21.15.67:0/621352776 wait complete. 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.296+0000 7f16d1b96b80 1 -- start start 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.296+0000 7f16d1b96b80 1 Processor -- start 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f14950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f040e0 0x55b419f114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16d1b96b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f04f80 0x55b419f119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419f12080 con 0x55b419f040e0 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419f6d120 con 0x55b419f03780 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16d1b96b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b419f6d290 con 0x55b419f04f80 2024-03-30T02:36:34.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16d1b96b80 10 client.0 _ll_register_callbacks cb 0x55b419dfa420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16b7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f14950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16b7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f14950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39758/0 (socket says 172.21.15.67:39758) 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16b6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f040e0 0x55b419f114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16b7400700 1 -- 172.21.15.67:0/4284332251 learned_addr learned my addr 172.21.15.67:0/4284332251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16b7e00700 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f04f80 0x55b419f119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16b6a00700 1 -- 172.21.15.67:0/4284332251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f04f80 msgr2=0x55b419f119e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.297+0000 7f16b6a00700 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f04f80 0x55b419f119e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16b6a00700 1 -- 172.21.15.67:0/4284332251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 msgr2=0x55b419f14950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16b6a00700 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f14950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16b6a00700 1 -- 172.21.15.67:0/4284332251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b0003bf0 con 0x55b419f040e0 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16b7e00700 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f04f80 0x55b419f119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16b7400700 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f14950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16b6a00700 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f040e0 0x55b419f114a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f16a400b680 tx=0x7f16a400b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760aaa49943a9712 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16a0c00700 1 -- 172.21.15.67:0/4284332251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4018980 con 0x55b419f040e0 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16d1b96b80 1 -- 172.21.15.67:0/4284332251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16a809a260 con 0x55b419f040e0 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16a0c00700 1 -- 172.21.15.67:0/4284332251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f16a4018ae0 con 0x55b419f040e0 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16a0c00700 1 -- 172.21.15.67:0/4284332251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a4018ca0 con 0x55b419f040e0 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16d1b96b80 10 client.15309 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.298+0000 7f16d1b96b80 1 -- 172.21.15.67:0/4284332251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b419c82b10 con 0x55b419f040e0 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.299+0000 7f16a0c00700 1 -- 172.21.15.67:0/4284332251 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16a809a260 con 0x55b419f040e0 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.299+0000 7f16a0c00700 1 -- 172.21.15.67:0/4284332251 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f16a4048030 con 0x55b419f040e0 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.299+0000 7f16d1b96b80 10 client.15309 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.299+0000 7f16d1b96b80 10 client.15309 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.299+0000 7f16d1b96b80 20 client.15309 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.299+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55b419f71b60 0x55b419f6e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.299+0000 7f16d1b96b80 4 client.15309 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.299+0000 7f16d1b96b80 1 -- 172.21.15.67:0/4284332251 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b419ca3ca0 con 0x55b419f71b60 2024-03-30T02:36:34.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.299+0000 7f16b7400700 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55b419f71b60 0x55b419f6e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16b7400700 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55b419f71b60 0x55b419f6e540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe72573c8fb545c5 server_cookie=35aa5f53320802fc in_seq=0 out_seq=0 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16a0c00700 10 client.15309 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16a0c00700 1 -- 172.21.15.67:0/4284332251 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b419ca3ca0 con 0x55b419f71b60 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16a0c00700 10 client.15309 handle_command_reply: tid=0 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 client.15309 shutdown 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 -- 172.21.15.67:0/4284332251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f040e0 msgr2=0x55b419f114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f040e0 0x55b419f114a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f16a400b680 tx=0x7f16a400b400 comp rx=0 tx=0).stop 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 -- 172.21.15.67:0/4284332251 shutdown_connections 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55b419f71b60 0x55b419f6e540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b419f04f80 0x55b419f119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b419f03780 0x55b419f14950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 --2- 172.21.15.67:0/4284332251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b419f040e0 0x55b419f114a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16a0c00700 0 client.15309 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 -- 172.21.15.67:0/4284332251 >> 172.21.15.67:0/4284332251 conn(0x55b419e62650 msgr2=0x55b419f21b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 -- 172.21.15.67:0/4284332251 shutdown_connections 2024-03-30T02:36:34.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 1 -- 172.21.15.67:0/4284332251 wait complete. 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.300+0000 7f16d1b96b80 20 client.15309 trim_cache size 0 max 16384 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 4, 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24520, 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 75.187576669999999, 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:34.306 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 87, 2024-03-30T02:36:34.307 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:34.307 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 123.026872684 2024-03-30T02:36:34.307 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 -- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f169c0a1120 msgr2=0x7f169c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f169c0a1120 0x7f169c0a35e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f16a400b2e0 tx=0x7f16a4005fd0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 -- 172.21.15.67:0/2902460253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b81017e0 msgr2=0x7f16b8198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b81017e0 0x7f16b8198f10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f16a800eb30 tx=0x7f16a800ee40 comp rx=0 tx=0).stop 2024-03-30T02:36:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 -- 172.21.15.67:0/2902460253 shutdown_connections 2024-03-30T02:36:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f169c0a1120 0x7f169c0a35e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16b8106a40 0x7f16b81a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16b81017e0 0x7f16b8198f10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 --2- 172.21.15.67:0/2902460253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b8104110 0x7f16b8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 -- 172.21.15.67:0/2902460253 >> 172.21.15.67:0/2902460253 conn(0x7f16b80fb1d0 msgr2=0x7f16b8101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 -- 172.21.15.67:0/2902460253 shutdown_connections 2024-03-30T02:36:34.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.301+0000 7f16be000700 1 -- 172.21.15.67:0/2902460253 wait complete. 2024-03-30T02:36:34.318 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:34 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2048710499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:34.319 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:34 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2842807855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:34.321 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-30T02:36:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.519+0000 7f5892c00700 1 -- 172.21.15.67:0/1957208053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f588c100d80 msgr2=0x7f588c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.519+0000 7f5892c00700 1 --2- 172.21.15.67:0/1957208053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f588c100d80 0x7f588c1011a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f587c0098f0 tx=0x7f587c009c00 comp rx=0 tx=0).stop 2024-03-30T02:36:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.519+0000 7f5892c00700 1 -- 172.21.15.67:0/1957208053 shutdown_connections 2024-03-30T02:36:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.519+0000 7f5892c00700 1 --2- 172.21.15.67:0/1957208053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f588c102910 0x7f588c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.519+0000 7f5892c00700 1 --2- 172.21.15.67:0/1957208053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f588c100d80 0x7f588c1011a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.519+0000 7f5892c00700 1 --2- 172.21.15.67:0/1957208053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588c101f70 0x7f588c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.519+0000 7f5892c00700 1 -- 172.21.15.67:0/1957208053 >> 172.21.15.67:0/1957208053 conn(0x7f588c0fc2f0 msgr2=0x7f588c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.519+0000 7f5892c00700 1 -- 172.21.15.67:0/1957208053 shutdown_connections 2024-03-30T02:36:34.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.519+0000 7f5892c00700 1 -- 172.21.15.67:0/1957208053 wait complete. 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.520+0000 7f5892c00700 1 Processor -- start 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.520+0000 7f5892c00700 1 -- start start 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.520+0000 7f5892c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f588c100d80 0x7f588c196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.520+0000 7f5892c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f588c101f70 0x7f588c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.520+0000 7f5892c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588c102910 0x7f588c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.520+0000 7f5892c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c19e8f0 con 0x7f588c101f70 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.520+0000 7f5892c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c19ea60 con 0x7f588c102910 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.520+0000 7f5892c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c19ec20 con 0x7f588c100d80 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f588c101f70 0x7f588c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f588c101f70 0x7f588c197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50426/0 (socket says 172.21.15.67:50426) 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588b400700 1 -- 172.21.15.67:0/2383681310 learned_addr learned my addr 172.21.15.67:0/2383681310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f5890c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588c102910 0x7f588c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588be00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f588c100d80 0x7f588c196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588b400700 1 -- 172.21.15.67:0/2383681310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f588c100d80 msgr2=0x7f588c196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588b400700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f588c100d80 0x7f588c196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588b400700 1 -- 172.21.15.67:0/2383681310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588c102910 msgr2=0x7f588c19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588b400700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588c102910 0x7f588c19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588b400700 1 -- 172.21.15.67:0/2383681310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009520 con 0x7f588c101f70 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f5890c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588c102910 0x7f588c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.521+0000 7f588b400700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f588c101f70 0x7f588c197340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f587800e9c0 tx=0x7f587800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.522+0000 7f5888c00700 1 -- 172.21.15.67:0/2383681310 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878004a30 con 0x7f588c101f70 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.522+0000 7f5888c00700 1 -- 172.21.15.67:0/2383681310 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587800cc80 con 0x7f588c101f70 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.522+0000 7f5892c00700 1 -- 172.21.15.67:0/2383681310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c19ef00 con 0x7f588c101f70 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.522+0000 7f5888c00700 1 -- 172.21.15.67:0/2383681310 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780054e0 con 0x7f588c101f70 2024-03-30T02:36:34.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.522+0000 7f5892c00700 1 -- 172.21.15.67:0/2383681310 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c19f320 con 0x7f588c101f70 2024-03-30T02:36:34.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.525+0000 7f5888c00700 1 -- 172.21.15.67:0/2383681310 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f5878004b90 con 0x7f588c101f70 2024-03-30T02:36:34.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.526+0000 7f58a6662b80 1 Processor -- start 2024-03-30T02:36:34.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.526+0000 7f5888c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f58700a1120 0x7f58700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.526+0000 7f58a6662b80 1 -- start start 2024-03-30T02:36:34.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.526+0000 7f5888c00700 1 -- 172.21.15.67:0/2383681310 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f5878014070 con 0x7f588c101f70 2024-03-30T02:36:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.526+0000 7f5890c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f58700a1120 0x7f58700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a7320ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a7321009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 0x55a732101400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f58a6662b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a732057940 con 0x55a7320ff780 2024-03-30T02:36:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f58a6662b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a732055300 con 0x55a732100f80 2024-03-30T02:36:34.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f58a6662b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a732050bf0 con 0x55a7321000e0 2024-03-30T02:36:34.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a7320ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a7321009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a7321009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40856/0 (socket says 172.21.15.67:40856) 2024-03-30T02:36:34.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588b400700 1 -- 172.21.15.67:0/2709609446 learned_addr learned my addr 172.21.15.67:0/2709609446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a7320ffba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50430/0 (socket says 172.21.15.67:50430) 2024-03-30T02:36:34.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f5890c00700 1 --2- 172.21.15.67:0/2709609446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 0x55a732101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588b400700 1 -- 172.21.15.67:0/2709609446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 msgr2=0x55a732101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588b400700 1 --2- 172.21.15.67:0/2709609446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 0x55a732101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f5890c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f58700a1120 0x7f58700a35e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f58840076f0 tx=0x7f5884009130 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588b400700 1 -- 172.21.15.67:0/2709609446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 msgr2=0x55a7320ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588b400700 1 --2- 172.21.15.67:0/2709609446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a7320ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.527+0000 7f588b400700 1 -- 172.21.15.67:0/2709609446 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a731ed2360 con 0x55a7321000e0 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.528+0000 7f588b400700 1 --2- 172.21.15.67:0/2709609446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a7321009a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58780a1d10 tx=0x7f58780997f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d78d5aa5a456943 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.528+0000 7f5881200700 1 -- 172.21.15.67:0/2709609446 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780b1070 con 0x55a7321000e0 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.528+0000 7f5881200700 1 -- 172.21.15.67:0/2709609446 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5878099dc0 con 0x55a7321000e0 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.528+0000 7f5881200700 1 -- 172.21.15.67:0/2709609446 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878098240 con 0x55a7321000e0 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 -- 172.21.15.67:0/2709609446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 msgr2=0x55a7321009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 --2- 172.21.15.67:0/2709609446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a7321009a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58780a1d10 tx=0x7f58780997f0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 -- 172.21.15.67:0/2709609446 shutdown_connections 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 --2- 172.21.15.67:0/2709609446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a7320ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 --2- 172.21.15.67:0/2709609446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 0x55a732101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 --2- 172.21.15.67:0/2709609446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a7321009a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 -- 172.21.15.67:0/2709609446 >> 172.21.15.67:0/2709609446 conn(0x55a73205e650 msgr2=0x55a7320fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 -- 172.21.15.67:0/2709609446 shutdown_connections 2024-03-30T02:36:34.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 -- 172.21.15.67:0/2709609446 wait complete. 2024-03-30T02:36:34.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 -- start start 2024-03-30T02:36:34.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.529+0000 7f58a6662b80 1 Processor -- start 2024-03-30T02:36:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7320ff780 0x55a732109610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a732109b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a732100f80 0x55a732110b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f58a6662b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a731fecd30 con 0x55a732100f80 2024-03-30T02:36:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f58a6662b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a731fec9d0 con 0x55a7320ff780 2024-03-30T02:36:34.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f58a6662b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a731ed4650 con 0x55a7321000e0 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f58a6662b80 10 client.0 _ll_register_callbacks cb 0x55a731ff6420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f588be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7320ff780 0x55a732109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f588b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a732109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f5890c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a732100f80 0x55a732110b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f588be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7320ff780 0x55a732109610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39786/0 (socket says 172.21.15.67:39786) 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.530+0000 7f588be00700 1 -- 172.21.15.67:0/286593843 learned_addr learned my addr 172.21.15.67:0/286593843 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f588b400700 1 -- 172.21.15.67:0/286593843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7320ff780 msgr2=0x55a732109610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f588b400700 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7320ff780 0x55a732109610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f588b400700 1 -- 172.21.15.67:0/286593843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a732100f80 msgr2=0x55a732110b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f588b400700 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a732100f80 0x55a732110b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f588b400700 1 -- 172.21.15.67:0/286593843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c19f320 con 0x55a7321000e0 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f588be00700 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7320ff780 0x55a732109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f588b400700 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a732109b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5878099d50 tx=0x7f5878005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20de43eebd035374 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f5875600700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:36:34.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f5875600700 1 -- 172.21.15.67:0/286593843 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780b6070 con 0x55a7321000e0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f58a6662b80 1 -- 172.21.15.67:0/286593843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a732011330 con 0x55a7321000e0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f5875600700 1 -- 172.21.15.67:0/286593843 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f58780a2240 con 0x55a7321000e0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f5875600700 1 -- 172.21.15.67:0/286593843 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58780b1070 con 0x55a7321000e0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f58a6662b80 10 client.25363 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.531+0000 7f58a6662b80 1 -- 172.21.15.67:0/286593843 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a731e7eb10 con 0x55a7321000e0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.532+0000 7f5875600700 1 -- 172.21.15.67:0/286593843 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a732011330 con 0x55a7321000e0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.532+0000 7f5875600700 1 -- 172.21.15.67:0/286593843 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5878098e20 con 0x55a7321000e0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.532+0000 7f58a6662b80 10 client.25363 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.532+0000 7f58a6662b80 10 client.25363 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.532+0000 7f58a6662b80 20 client.25363 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.532+0000 7f58a6662b80 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a7321143b0 0x55a732119690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.532+0000 7f58a6662b80 4 client.25363 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.532+0000 7f58a6662b80 1 -- 172.21.15.67:0/286593843 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7320496b0 con 0x55a7321143b0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.532+0000 7f588be00700 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a7321143b0 0x55a732119690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.533+0000 7f588be00700 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a7321143b0 0x55a732119690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c77efbe925a06979 server_cookie=f18bd64fe11aad07 in_seq=0 out_seq=0 2024-03-30T02:36:34.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.533+0000 7f5875600700 10 client.25363 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f5875600700 1 -- 172.21.15.67:0/286593843 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a7320496b0 con 0x55a7321143b0 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f5875600700 10 client.25363 handle_command_reply: tid=0 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 client.25363 shutdown 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 -- 172.21.15.67:0/286593843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 msgr2=0x55a732109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a732109b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5878099d50 tx=0x7f5878005870 comp rx=0 tx=0).stop 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 -- 172.21.15.67:0/286593843 shutdown_connections 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a7321143b0 0x55a732119690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a732100f80 0x55a732110b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7320ff780 0x55a732109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f5875600700 0 client.25363 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 --2- 172.21.15.67:0/286593843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7321000e0 0x55a732109b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 -- 172.21.15.67:0/286593843 >> 172.21.15.67:0/286593843 conn(0x55a73205e650 msgr2=0x55a7320fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 -- 172.21.15.67:0/286593843 shutdown_connections 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 1 -- 172.21.15.67:0/286593843 wait complete. 2024-03-30T02:36:34.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.534+0000 7f58a6662b80 20 client.25363 trim_cache size 0 max 16384 2024-03-30T02:36:34.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f58a6662b80 1 Processor -- start 2024-03-30T02:36:34.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f58a6662b80 1 -- start start 2024-03-30T02:36:34.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7320ff780 0x55a7320ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7321000e0 0x55a732101360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 0x55a7320f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f58a6662b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7321128a0 con 0x55a7321000e0 2024-03-30T02:36:34.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f58a6662b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7320496b0 con 0x55a732100f80 2024-03-30T02:36:34.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f58a6662b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a732011330 con 0x55a7320ff780 2024-03-30T02:36:34.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f588b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7321000e0 0x55a732101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f588b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7321000e0 0x55a732101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50440/0 (socket says 172.21.15.67:50440) 2024-03-30T02:36:34.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f588b400700 1 -- 172.21.15.67:0/4144575035 learned_addr learned my addr 172.21.15.67:0/4144575035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f588be00700 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7320ff780 0x55a7320ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.553+0000 7f5890c00700 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 0x55a7320f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f588b400700 1 -- 172.21.15.67:0/4144575035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7320ff780 msgr2=0x55a7320ffb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f588b400700 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7320ff780 0x55a7320ffb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f588b400700 1 -- 172.21.15.67:0/4144575035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 msgr2=0x55a7320f9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f588b400700 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 0x55a7320f9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f588b400700 1 -- 172.21.15.67:0/4144575035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a732016e50 con 0x55a7321000e0 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f5890c00700 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 0x55a7320f9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f588b400700 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7321000e0 0x55a732101360 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5878099ac0 tx=0x7f5878005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e67915da968fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f5881c00700 1 -- 172.21.15.67:0/4144575035 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780b1070 con 0x55a7321000e0 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f5881c00700 1 -- 172.21.15.67:0/4144575035 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587800cdf0 con 0x55a7321000e0 2024-03-30T02:36:34.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.554+0000 7f5881c00700 1 -- 172.21.15.67:0/4144575035 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780a1db0 con 0x55a7321000e0 2024-03-30T02:36:34.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 -- 172.21.15.67:0/4144575035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7321000e0 msgr2=0x55a732101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7321000e0 0x55a732101360 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5878099ac0 tx=0x7f5878005c00 comp rx=0 tx=0).stop 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 -- 172.21.15.67:0/4144575035 shutdown_connections 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a732100f80 0x55a7320f9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a7320ff780 0x55a7320ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 --2- 172.21.15.67:0/4144575035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7321000e0 0x55a732101360 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 -- 172.21.15.67:0/4144575035 >> 172.21.15.67:0/4144575035 conn(0x55a73205e650 msgr2=0x55a732108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 -- 172.21.15.67:0/4144575035 shutdown_connections 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 -- 172.21.15.67:0/4144575035 wait complete. 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 -- start start 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.555+0000 7f58a6662b80 1 Processor -- start 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a732109b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7321000e0 0x55a732169cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f58a6662b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a732100f80 0x55a73216c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f58a6662b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73216c950 con 0x55a7320ff780 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f58a6662b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73216cac0 con 0x55a7321000e0 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f58a6662b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73216cc80 con 0x55a732100f80 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f58a6662b80 10 client.0 _ll_register_callbacks cb 0x55a731ff6420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f588be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a732109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f588b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7321000e0 0x55a732169cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f588be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a732109b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50452/0 (socket says 172.21.15.67:50452) 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f588be00700 1 -- 172.21.15.67:0/3662612695 learned_addr learned my addr 172.21.15.67:0/3662612695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.556+0000 7f5890c00700 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a732100f80 0x55a73216c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f588be00700 1 -- 172.21.15.67:0/3662612695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a732100f80 msgr2=0x55a73216c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f588be00700 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a732100f80 0x55a73216c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f588be00700 1 -- 172.21.15.67:0/3662612695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7321000e0 msgr2=0x55a732169cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f588be00700 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7321000e0 0x55a732169cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f588be00700 1 -- 172.21.15.67:0/3662612695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a731e7eb10 con 0x55a7320ff780 2024-03-30T02:36:34.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f5890c00700 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a732100f80 0x55a73216c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f588b400700 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7321000e0 0x55a732169cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f588be00700 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a732109b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f587c00f9d0 tx=0x7f587c027be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e64b9f394465fd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f5875600700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f5875600700 1 -- 172.21.15.67:0/3662612695 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00b910 con 0x55a7320ff780 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f58a6662b80 1 -- 172.21.15.67:0/3662612695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5878099dc0 con 0x55a7320ff780 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f5875600700 1 -- 172.21.15.67:0/3662612695 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f587c00ba70 con 0x55a7320ff780 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f5875600700 1 -- 172.21.15.67:0/3662612695 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f587c0270d0 con 0x55a7320ff780 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f58a6662b80 10 client.15327 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.557+0000 7f58a6662b80 1 -- 172.21.15.67:0/3662612695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a731ed6c80 con 0x55a7320ff780 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.558+0000 7f5875600700 1 -- 172.21.15.67:0/3662612695 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5878099dc0 con 0x55a7320ff780 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.558+0000 7f5875600700 1 -- 172.21.15.67:0/3662612695 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f587c009920 con 0x55a7320ff780 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.558+0000 7f58a6662b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.558+0000 7f58a6662b80 10 client.15327 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.558+0000 7f58a6662b80 20 client.15327 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.558+0000 7f58a6662b80 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a732171b80 0x55a73216e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.558+0000 7f58a6662b80 4 client.15327 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.558+0000 7f58a6662b80 1 -- 172.21.15.67:0/3662612695 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a731e9fca0 con 0x55a732171b80 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.558+0000 7f588b400700 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a732171b80 0x55a73216e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f5875600700 10 client.15327 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f588b400700 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a732171b80 0x55a73216e4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5082c20baf195301 server_cookie=1c47919bcc3349c1 in_seq=0 out_seq=0 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f5875600700 1 -- 172.21.15.67:0/3662612695 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55a731e9fca0 con 0x55a732171b80 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f5875600700 10 client.15327 handle_command_reply: tid=0 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 client.15327 shutdown 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 -- 172.21.15.67:0/3662612695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 msgr2=0x55a732109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a732109b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f587c00f9d0 tx=0x7f587c027be0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 -- 172.21.15.67:0/3662612695 shutdown_connections 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a732171b80 0x55a73216e4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a7320ff780 0x55a732109b90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f5875600700 0 client.15327 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a732100f80 0x55a73216c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 --2- 172.21.15.67:0/3662612695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7321000e0 0x55a732169cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 -- 172.21.15.67:0/3662612695 >> 172.21.15.67:0/3662612695 conn(0x55a73205e650 msgr2=0x55a732107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 -- 172.21.15.67:0/3662612695 shutdown_connections 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.559+0000 7f58a6662b80 1 -- 172.21.15.67:0/3662612695 wait complete. 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f58a6662b80 20 client.15327 trim_cache size 0 max 16384 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 3, 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24539, 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:34.565 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 76.456270687, 2024-03-30T02:36:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 87, 2024-03-30T02:36:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:34.566 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 121.552259517 2024-03-30T02:36:34.566 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:34.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 -- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f58700a1120 msgr2=0x7f58700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f58700a1120 0x7f58700a35e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f58840076f0 tx=0x7f5884009130 comp rx=0 tx=0).stop 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 -- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f588c101f70 msgr2=0x7f588c197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f588c101f70 0x7f588c197340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f587800e9c0 tx=0x7f587800ecd0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 -- 172.21.15.67:0/2383681310 shutdown_connections 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f58700a1120 0x7f58700a35e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588c102910 0x7f588c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f588c100d80 0x7f588c196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 --2- 172.21.15.67:0/2383681310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f588c101f70 0x7f588c197340 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 -- 172.21.15.67:0/2383681310 >> 172.21.15.67:0/2383681310 conn(0x7f588c0fc2f0 msgr2=0x7f588c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 -- 172.21.15.67:0/2383681310 shutdown_connections 2024-03-30T02:36:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.560+0000 7f5892c00700 1 -- 172.21.15.67:0/2383681310 wait complete. 2024-03-30T02:36:34.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-30T02:36:34.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:34 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2048710499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:34.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:34 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2842807855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:34 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2048710499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:34 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2842807855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.781+0000 7fdc1fe00700 1 -- 172.21.15.67:0/4252280079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc20101630 msgr2=0x7fdc20103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.781+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/4252280079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc20101630 0x7fdc20103a20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009990 tx=0x7fdc10009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.781+0000 7fdc1fe00700 1 -- 172.21.15.67:0/4252280079 shutdown_connections 2024-03-30T02:36:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.781+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/4252280079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc20106890 0x7fdc20108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.781+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/4252280079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc20101630 0x7fdc20103a20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.781+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/4252280079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc20103f60 0x7fdc20106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.781+0000 7fdc1fe00700 1 -- 172.21.15.67:0/4252280079 >> 172.21.15.67:0/4252280079 conn(0x7fdc200fb000 msgr2=0x7fdc200fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.781+0000 7fdc1fe00700 1 -- 172.21.15.67:0/4252280079 shutdown_connections 2024-03-30T02:36:34.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.781+0000 7fdc1fe00700 1 -- 172.21.15.67:0/4252280079 wait complete. 2024-03-30T02:36:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.783+0000 7fdc1fe00700 1 Processor -- start 2024-03-30T02:36:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.783+0000 7fdc1fe00700 1 -- start start 2024-03-30T02:36:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.783+0000 7fdc1fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc20101630 0x7fdc20198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.783+0000 7fdc1fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc20103f60 0x7fdc201992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc20106890 0x7fdc201a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1fe00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc201a0880 con 0x7fdc20106890 2024-03-30T02:36:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1fe00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc201a09f0 con 0x7fdc20101630 2024-03-30T02:36:34.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1fe00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc201a0bb0 con 0x7fdc20103f60 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1e000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc20103f60 0x7fdc201992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc20106890 0x7fdc201a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc20101630 0x7fdc20198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc20106890 0x7fdc201a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50480/0 (socket says 172.21.15.67:50480) 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1f400700 1 -- 172.21.15.67:0/239388105 learned_addr learned my addr 172.21.15.67:0/239388105 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1f400700 1 -- 172.21.15.67:0/239388105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc20103f60 msgr2=0x7fdc201992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1f400700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc20103f60 0x7fdc201992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1f400700 1 -- 172.21.15.67:0/239388105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc20101630 msgr2=0x7fdc20198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1f400700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc20101630 0x7fdc20198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.784+0000 7fdc1f400700 1 -- 172.21.15.67:0/239388105 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100095f0 con 0x7fdc20106890 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.785+0000 7fdc1f400700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc20106890 0x7fdc201a02b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdc18007ee0 tx=0x7fdc1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.785+0000 7fdc17400700 1 -- 172.21.15.67:0/239388105 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18017070 con 0x7fdc20106890 2024-03-30T02:36:34.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.785+0000 7fdc1fe00700 1 -- 172.21.15.67:0/239388105 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc201a0e90 con 0x7fdc20106890 2024-03-30T02:36:34.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.785+0000 7fdc17400700 1 -- 172.21.15.67:0/239388105 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1800f040 con 0x7fdc20106890 2024-03-30T02:36:34.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.785+0000 7fdc17400700 1 -- 172.21.15.67:0/239388105 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18005550 con 0x7fdc20106890 2024-03-30T02:36:34.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.785+0000 7fdc1fe00700 1 -- 172.21.15.67:0/239388105 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc201a1310 con 0x7fdc20106890 2024-03-30T02:36:34.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.788+0000 7fdc17400700 1 -- 172.21.15.67:0/239388105 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fdc1800db20 con 0x7fdc20106890 2024-03-30T02:36:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.789+0000 7fdc17400700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc040a11a0 0x7fdc040a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.789+0000 7fdc3930db80 1 Processor -- start 2024-03-30T02:36:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.789+0000 7fdc17400700 1 -- 172.21.15.67:0/239388105 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fdc180cc2a0 con 0x7fdc20106890 2024-03-30T02:36:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc3930db80 1 -- start start 2024-03-30T02:36:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc040a11a0 0x7fdc040a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df630b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df6315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc3930db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df587940 con 0x55c5df631140 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc3930db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df585300 con 0x55c5df62f990 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc3930db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df580bf0 con 0x55c5df6302f0 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc1e000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df630b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc1f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df6315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc1ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc1e000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df630b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40906/0 (socket says 172.21.15.67:40906) 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc1f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df6315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50486/0 (socket says 172.21.15.67:50486) 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.790+0000 7fdc1e000700 1 -- 172.21.15.67:0/3094469020 learned_addr learned my addr 172.21.15.67:0/3094469020 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc040a11a0 0x7fdc040a3660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000b680 tx=0x7fdc10005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc1e000700 1 -- 172.21.15.67:0/3094469020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 msgr2=0x55c5df62fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc1e000700 1 --2- 172.21.15.67:0/3094469020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc1e000700 1 -- 172.21.15.67:0/3094469020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 msgr2=0x55c5df6315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc1e000700 1 --2- 172.21.15.67:0/3094469020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df6315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc1e000700 1 -- 172.21.15.67:0/3094469020 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5df402360 con 0x55c5df6302f0 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc1f400700 1 --2- 172.21.15.67:0/3094469020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df6315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc1e000700 1 --2- 172.21.15.67:0/3094469020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df630b60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009d80 tx=0x7fdc0c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ea6a9554511f20 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc0be00700 1 -- 172.21.15.67:0/3094469020 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c004230 con 0x55c5df6302f0 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc0be00700 1 -- 172.21.15.67:0/3094469020 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc0c004390 con 0x55c5df6302f0 2024-03-30T02:36:34.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.791+0000 7fdc0be00700 1 -- 172.21.15.67:0/3094469020 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c019860 con 0x55c5df6302f0 2024-03-30T02:36:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.792+0000 7fdc3930db80 1 -- 172.21.15.67:0/3094469020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 msgr2=0x55c5df630b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.792+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3094469020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df630b60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009d80 tx=0x7fdc0c011720 comp rx=0 tx=0).stop 2024-03-30T02:36:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.792+0000 7fdc3930db80 1 -- 172.21.15.67:0/3094469020 shutdown_connections 2024-03-30T02:36:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.792+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3094469020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df6315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.792+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3094469020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.792+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3094469020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df630b60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.792+0000 7fdc3930db80 1 -- 172.21.15.67:0/3094469020 >> 172.21.15.67:0/3094469020 conn(0x55c5df58e650 msgr2=0x55c5df62aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.792+0000 7fdc3930db80 1 -- 172.21.15.67:0/3094469020 shutdown_connections 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.792+0000 7fdc3930db80 1 -- 172.21.15.67:0/3094469020 wait complete. 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.793+0000 7fdc3930db80 1 -- start start 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.793+0000 7fdc3930db80 1 Processor -- start 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df639830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df639d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 0x55c5df640db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc3930db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df51cd30 con 0x55c5df6302f0 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc3930db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df51c9d0 con 0x55c5df62f990 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc3930db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df404650 con 0x55c5df631140 2024-03-30T02:36:34.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc3930db80 10 client.0 _ll_register_callbacks cb 0x55c5df526420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df639d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df639830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df639d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50498/0 (socket says 172.21.15.67:50498) 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1e000700 1 -- 172.21.15.67:0/3892066838 learned_addr learned my addr 172.21.15.67:0/3892066838 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1f400700 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 0x55c5df640db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1e000700 1 -- 172.21.15.67:0/3892066838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 msgr2=0x55c5df640db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1e000700 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 0x55c5df640db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1e000700 1 -- 172.21.15.67:0/3892066838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 msgr2=0x55c5df639830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1e000700 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df639830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1e000700 1 -- 172.21.15.67:0/3892066838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc201a1310 con 0x55c5df6302f0 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.794+0000 7fdc1f400700 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 0x55c5df640db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.795+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df639830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.795+0000 7fdc1e000700 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df639d70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0038b0 tx=0x7fdc0c0038e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909b8b0fe521c280 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.795+0000 7fdc08c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.795+0000 7fdc08c00700 1 -- 172.21.15.67:0/3892066838 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c005ce0 con 0x55c5df6302f0 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.795+0000 7fdc08c00700 1 -- 172.21.15.67:0/3892066838 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fdc0c005040 con 0x55c5df6302f0 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.795+0000 7fdc3930db80 1 -- 172.21.15.67:0/3892066838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5df541330 con 0x55c5df6302f0 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.795+0000 7fdc08c00700 1 -- 172.21.15.67:0/3892066838 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc0c005200 con 0x55c5df6302f0 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.795+0000 7fdc3930db80 10 client.15339 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.795+0000 7fdc3930db80 1 -- 172.21.15.67:0/3892066838 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fdc0c0099e0 con 0x55c5df6302f0 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.796+0000 7fdc08c00700 1 -- 172.21.15.67:0/3892066838 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5df541330 con 0x55c5df6302f0 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.796+0000 7fdc08c00700 1 -- 172.21.15.67:0/3892066838 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc0c021040 con 0x55c5df6302f0 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.796+0000 7fdc3930db80 10 client.15339 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:34.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.796+0000 7fdc3930db80 10 client.15339 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-30T02:36:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.796+0000 7fdc3930db80 20 client.15339 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.796+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c5df644940 0x55c5df649c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.796+0000 7fdc3930db80 4 client.15339 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.796+0000 7fdc3930db80 1 -- 172.21.15.67:0/3892066838 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5df5796b0 con 0x55c5df644940 2024-03-30T02:36:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.796+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c5df644940 0x55c5df649c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.797+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c5df644940 0x55c5df649c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1cb6476c933dcb9d server_cookie=95f0b24d82d3150 in_seq=0 out_seq=0 2024-03-30T02:36:34.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.797+0000 7fdc08c00700 10 client.15339 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:36:34.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc08c00700 1 -- 172.21.15.67:0/3892066838 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c5df5796b0 con 0x55c5df644940 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc08c00700 10 client.15339 handle_command_reply: tid=0 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 client.15339 shutdown 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 -- 172.21.15.67:0/3892066838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 msgr2=0x55c5df639d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df639d70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0038b0 tx=0x7fdc0c0038e0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 -- 172.21.15.67:0/3892066838 shutdown_connections 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c5df644940 0x55c5df649c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 0x55c5df640db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df639830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 --2- 172.21.15.67:0/3892066838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df639d70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 -- 172.21.15.67:0/3892066838 >> 172.21.15.67:0/3892066838 conn(0x55c5df58e650 msgr2=0x55c5df62e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc08c00700 0 client.15339 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:36:34.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 -- 172.21.15.67:0/3892066838 shutdown_connections 2024-03-30T02:36:34.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.798+0000 7fdc3930db80 1 -- 172.21.15.67:0/3892066838 wait complete. 2024-03-30T02:36:34.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.799+0000 7fdc3930db80 20 client.15339 trim_cache size 0 max 16384 2024-03-30T02:36:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.817+0000 7fdc3930db80 1 Processor -- start 2024-03-30T02:36:34.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc3930db80 1 -- start start 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df638d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df631520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc3930db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df53ead0 con 0x55c5df631140 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc3930db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df579990 con 0x55c5df62f990 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc3930db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df57d850 con 0x55c5df6302f0 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc1f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df631520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc1ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc1ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39830/0 (socket says 172.21.15.67:39830) 2024-03-30T02:36:34.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc1f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df631520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50508/0 (socket says 172.21.15.67:50508) 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc1ea00700 1 -- 172.21.15.67:0/2381930685 learned_addr learned my addr 172.21.15.67:0/2381930685 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.818+0000 7fdc1e000700 1 --2- 172.21.15.67:0/2381930685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df638d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.819+0000 7fdc1ea00700 1 -- 172.21.15.67:0/2381930685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 msgr2=0x55c5df638d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.819+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/2381930685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df638d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.819+0000 7fdc1ea00700 1 -- 172.21.15.67:0/2381930685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 msgr2=0x55c5df631520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.819+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/2381930685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df631520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.819+0000 7fdc1ea00700 1 -- 172.21.15.67:0/2381930685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5df3aeb10 con 0x55c5df62f990 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.819+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/2381930685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fd70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc10005f40 tx=0x7fdc10028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d41575c3c97ac0b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.819+0000 7fdc14c00700 1 -- 172.21.15.67:0/2381930685 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004df0 con 0x55c5df62f990 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.819+0000 7fdc14c00700 1 -- 172.21.15.67:0/2381930685 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1001e040 con 0x55c5df62f990 2024-03-30T02:36:34.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.819+0000 7fdc14c00700 1 -- 172.21.15.67:0/2381930685 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000f6c0 con 0x55c5df62f990 2024-03-30T02:36:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 -- 172.21.15.67:0/2381930685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 msgr2=0x55c5df62fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2381930685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fd70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc10005f40 tx=0x7fdc10028000 comp rx=0 tx=0).stop 2024-03-30T02:36:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 -- 172.21.15.67:0/2381930685 shutdown_connections 2024-03-30T02:36:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2381930685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df631140 0x55c5df631520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2381930685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df62fd70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2381930685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df6302f0 0x55c5df638d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 -- 172.21.15.67:0/2381930685 >> 172.21.15.67:0/2381930685 conn(0x55c5df58e650 msgr2=0x55c5df623fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 -- 172.21.15.67:0/2381930685 shutdown_connections 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 -- 172.21.15.67:0/2381930685 wait complete. 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.820+0000 7fdc3930db80 1 -- start start 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.821+0000 7fdc3930db80 1 Processor -- start 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.821+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df63df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.821+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df63e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.821+0000 7fdc3930db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 0x55c5df699230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.821+0000 7fdc3930db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df640fd0 con 0x55c5df6302f0 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.821+0000 7fdc3930db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df641140 con 0x55c5df62f990 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.821+0000 7fdc3930db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5df699770 con 0x55c5df631140 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc3930db80 10 client.0 _ll_register_callbacks cb 0x55c5df526420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df63e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df63e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50516/0 (socket says 172.21.15.67:50516) 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1e000700 1 -- 172.21.15.67:0/2377665317 learned_addr learned my addr 172.21.15.67:0/2377665317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df63df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1f400700 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 0x55c5df699230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1e000700 1 -- 172.21.15.67:0/2377665317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 msgr2=0x55c5df699230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1e000700 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 0x55c5df699230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1e000700 1 -- 172.21.15.67:0/2377665317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 msgr2=0x55c5df63df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1e000700 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df63df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1e000700 1 -- 172.21.15.67:0/2377665317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc201098f0 con 0x55c5df6302f0 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc1e000700 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df63e490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c003f10 tx=0x7fdc0c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d616e2c798ad1042 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.822+0000 7fdc08c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc08c00700 1 -- 172.21.15.67:0/2377665317 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c019c70 con 0x55c5df6302f0 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc3930db80 1 -- 172.21.15.67:0/2377665317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc0c004390 con 0x55c5df6302f0 2024-03-30T02:36:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc08c00700 1 -- 172.21.15.67:0/2377665317 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fdc0c000e40 con 0x55c5df6302f0 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc08c00700 1 -- 172.21.15.67:0/2377665317 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc0c003950 con 0x55c5df6302f0 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc3930db80 10 client.15345 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc3930db80 1 -- 172.21.15.67:0/2377665317 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c5df546e50 con 0x55c5df6302f0 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc08c00700 1 -- 172.21.15.67:0/2377665317 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc0c004390 con 0x55c5df6302f0 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc08c00700 1 -- 172.21.15.67:0/2377665317 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdc0c011790 con 0x55c5df6302f0 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc3930db80 10 client.15345 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc3930db80 10 client.15345 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.823+0000 7fdc3930db80 20 client.15345 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.824+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c5df69dff0 0x55c5df69a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.824+0000 7fdc3930db80 4 client.15345 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:34.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.824+0000 7fdc3930db80 1 -- 172.21.15.67:0/2377665317 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c5df3cfca0 con 0x55c5df69dff0 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.824+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c5df69dff0 0x55c5df69a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.824+0000 7fdc1ea00700 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c5df69dff0 0x55c5df69a990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4373c46bbd33c1b server_cookie=2e4e3d85ec1a0d88 in_seq=0 out_seq=0 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.824+0000 7fdc08c00700 10 client.15345 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc08c00700 1 -- 172.21.15.67:0/2377665317 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c5df3cfca0 con 0x55c5df69dff0 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc08c00700 10 client.15345 handle_command_reply: tid=0 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 client.15345 shutdown 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 -- 172.21.15.67:0/2377665317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 msgr2=0x55c5df63e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df63e490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c003f10 tx=0x7fdc0c005e50 comp rx=0 tx=0).stop 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 -- 172.21.15.67:0/2377665317 shutdown_connections 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c5df69dff0 0x55c5df69a990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5df631140 0x55c5df699230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5df62f990 0x55c5df63df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 --2- 172.21.15.67:0/2377665317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5df6302f0 0x55c5df63e490 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc08c00700 0 client.15345 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:36:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 -- 172.21.15.67:0/2377665317 >> 172.21.15.67:0/2377665317 conn(0x55c5df58e650 msgr2=0x55c5df6372a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 -- 172.21.15.67:0/2377665317 shutdown_connections 2024-03-30T02:36:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 1 -- 172.21.15.67:0/2377665317 wait complete. 2024-03-30T02:36:34.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.825+0000 7fdc3930db80 20 client.15345 trim_cache size 0 max 16384 2024-03-30T02:36:34.832 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:34.832 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:34.832 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-03-30T02:36:34.832 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24541, 2024-03-30T02:36:34.833 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:34.833 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:34.833 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:34.833 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 77.726117349000006, 2024-03-30T02:36:34.833 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:34.833 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 87, 2024-03-30T02:36:34.833 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:34.833 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 120.06139563399999 2024-03-30T02:36:34.833 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 -- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc040a11a0 msgr2=0x7fdc040a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc040a11a0 0x7fdc040a3660 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000b680 tx=0x7fdc10005fd0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 -- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc20106890 msgr2=0x7fdc201a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc20106890 0x7fdc201a02b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdc18007ee0 tx=0x7fdc1800d3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 -- 172.21.15.67:0/239388105 shutdown_connections 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fdc040a11a0 0x7fdc040a3660 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc20106890 0x7fdc201a02b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc20101630 0x7fdc20198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/239388105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc20103f60 0x7fdc201992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 -- 172.21.15.67:0/239388105 >> 172.21.15.67:0/239388105 conn(0x7fdc200fb000 msgr2=0x7fdc200ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 -- 172.21.15.67:0/239388105 shutdown_connections 2024-03-30T02:36:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:34.826+0000 7fdc1fe00700 1 -- 172.21.15.67:0/239388105 wait complete. 2024-03-30T02:36:34.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-03-30T02:36:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.043+0000 7fc8a5c00700 1 -- 172.21.15.67:0/2924925203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a0106a40 msgr2=0x7fc8a0108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.043+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/2924925203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a0106a40 0x7fc8a0108eb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc898009f90 tx=0x7fc89800b330 comp rx=0 tx=0).stop 2024-03-30T02:36:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.043+0000 7fc8a5c00700 1 -- 172.21.15.67:0/2924925203 shutdown_connections 2024-03-30T02:36:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.043+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/2924925203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a0106a40 0x7fc8a0108eb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.043+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/2924925203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8a01017e0 0x7fc8a0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.043+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/2924925203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a0104110 0x7fc8a0106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.043+0000 7fc8a5c00700 1 -- 172.21.15.67:0/2924925203 >> 172.21.15.67:0/2924925203 conn(0x7fc8a00fb1d0 msgr2=0x7fc8a00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.043+0000 7fc8a5c00700 1 -- 172.21.15.67:0/2924925203 shutdown_connections 2024-03-30T02:36:35.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.044+0000 7fc8a5c00700 1 -- 172.21.15.67:0/2924925203 wait complete. 2024-03-30T02:36:35.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc8a5c00700 1 Processor -- start 2024-03-30T02:36:35.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc8a5c00700 1 -- start start 2024-03-30T02:36:35.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc8a5c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8a01017e0 0x7fc8a0101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc8a5c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a0104110 0x7fc8a00ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc8a5c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a0106a40 0x7fc8a00ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc8a5c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a0100470 con 0x7fc8a0104110 2024-03-30T02:36:35.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc8a5c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a01005e0 con 0x7fc8a01017e0 2024-03-30T02:36:35.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc8a5c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a01a0e20 con 0x7fc8a0106a40 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc89e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a0104110 0x7fc8a00ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc89ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8a01017e0 0x7fc8a0101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc89ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8a01017e0 0x7fc8a0101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39868/0 (socket says 172.21.15.67:39868) 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.045+0000 7fc89ea00700 1 -- 172.21.15.67:0/3931427218 learned_addr learned my addr 172.21.15.67:0/3931427218 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc89f400700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a0106a40 0x7fc8a00ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc89ea00700 1 -- 172.21.15.67:0/3931427218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a0106a40 msgr2=0x7fc8a00ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc89ea00700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a0106a40 0x7fc8a00ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc89ea00700 1 -- 172.21.15.67:0/3931427218 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a0104110 msgr2=0x7fc8a00ff790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc89ea00700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a0104110 0x7fc8a00ff790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc89ea00700 1 -- 172.21.15.67:0/3931427218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc898009c40 con 0x7fc8a01017e0 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc89ea00700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8a01017e0 0x7fc8a0101100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9a0 tx=0x7fc89000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc897400700 1 -- 172.21.15.67:0/3931427218 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89000eee0 con 0x7fc8a01017e0 2024-03-30T02:36:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc897400700 1 -- 172.21.15.67:0/3931427218 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc89000cb70 con 0x7fc8a01017e0 2024-03-30T02:36:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc8a5c00700 1 -- 172.21.15.67:0/3931427218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a01a10b0 con 0x7fc8a01017e0 2024-03-30T02:36:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.046+0000 7fc897400700 1 -- 172.21.15.67:0/3931427218 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890009790 con 0x7fc8a01017e0 2024-03-30T02:36:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.047+0000 7fc8a5c00700 1 -- 172.21.15.67:0/3931427218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a01a15c0 con 0x7fc8a01017e0 2024-03-30T02:36:35.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.050+0000 7fc897400700 1 -- 172.21.15.67:0/3931427218 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc890010430 con 0x7fc8a01017e0 2024-03-30T02:36:35.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc8b97bcb80 1 Processor -- start 2024-03-30T02:36:35.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc897400700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc8840a1120 0x7fc8840a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc897400700 1 -- 172.21.15.67:0/3931427218 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fc890014070 con 0x7fc8a01017e0 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc8b97bcb80 1 -- start start 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc89f400700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc8840a1120 0x7fc8840a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 0x56023e216ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e2179a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e218400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e16e940 con 0x56023e2170e0 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.051+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e16c300 con 0x56023e217f80 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e167bf0 con 0x56023e216780 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e2179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e2179a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50540/0 (socket says 172.21.15.67:50540) 2024-03-30T02:36:35.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89e000700 1 -- 172.21.15.67:0/302000347 learned_addr learned my addr 172.21.15.67:0/302000347 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89ea00700 1 --2- 172.21.15.67:0/302000347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 0x56023e216ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89f400700 1 --2- 172.21.15.67:0/302000347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e218400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89f400700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc8840a1120 0x7fc8840a35e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc898009550 tx=0x7fc898009320 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89ea00700 1 -- 172.21.15.67:0/302000347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 msgr2=0x56023e218400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89ea00700 1 --2- 172.21.15.67:0/302000347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e218400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89ea00700 1 -- 172.21.15.67:0/302000347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 msgr2=0x56023e2179a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89ea00700 1 --2- 172.21.15.67:0/302000347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e2179a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89ea00700 1 -- 172.21.15.67:0/302000347 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56023dfe9360 con 0x56023e216780 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.052+0000 7fc89ea00700 1 --2- 172.21.15.67:0/302000347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 0x56023e216ba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e970 tx=0x7fc890099bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae41fc8e2386bb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc88be00700 1 -- 172.21.15.67:0/302000347 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890099f80 con 0x56023e216780 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc88be00700 1 -- 172.21.15.67:0/302000347 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc89009ddb0 con 0x56023e216780 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc88be00700 1 -- 172.21.15.67:0/302000347 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900a21d0 con 0x56023e216780 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/302000347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 msgr2=0x56023e216ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/302000347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 0x56023e216ba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e970 tx=0x7fc890099bf0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/302000347 shutdown_connections 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/302000347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e218400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/302000347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 0x56023e216ba0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/302000347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e2179a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/302000347 >> 172.21.15.67:0/302000347 conn(0x56023e175650 msgr2=0x56023e211950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.053+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/302000347 shutdown_connections 2024-03-30T02:36:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.054+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/302000347 wait complete. 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.054+0000 7fc8b97bcb80 1 -- start start 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.054+0000 7fc8b97bcb80 1 Processor -- start 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e216780 0x56023e220550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 0x56023e220a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e227ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e103d30 con 0x56023e216780 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e1039d0 con 0x56023e217f80 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023dfeb650 con 0x56023e2170e0 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc8b97bcb80 10 client.0 _ll_register_callbacks cb 0x56023e10d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e216780 0x56023e220550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e216780 0x56023e220550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50542/0 (socket says 172.21.15.67:50542) 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89ea00700 1 -- 172.21.15.67:0/2022701144 learned_addr learned my addr 172.21.15.67:0/2022701144 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89f400700 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e227ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89e000700 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 0x56023e220a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89ea00700 1 -- 172.21.15.67:0/2022701144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 msgr2=0x56023e220a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89ea00700 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 0x56023e220a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89ea00700 1 -- 172.21.15.67:0/2022701144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 msgr2=0x56023e227ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89ea00700 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e227ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89ea00700 1 -- 172.21.15.67:0/2022701144 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a01a15c0 con 0x56023e216780 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.055+0000 7fc89f400700 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e227ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc89ea00700 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e216780 0x56023e220550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc890099ec0 tx=0x7fc890099490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788f095d0b250fca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc888c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:35.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc888c00700 1 -- 172.21.15.67:0/2022701144 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900b8380 con 0x56023e216780 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc888c00700 1 -- 172.21.15.67:0/2022701144 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fc890099f80 con 0x56023e216780 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2022701144 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56023e128330 con 0x56023e216780 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc888c00700 1 -- 172.21.15.67:0/2022701144 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc890094b90 con 0x56023e216780 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc8b97bcb80 10 client.15357 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2022701144 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56023df95b10 con 0x56023e216780 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc888c00700 1 -- 172.21.15.67:0/2022701144 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56023e128330 con 0x56023e216780 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc888c00700 1 -- 172.21.15.67:0/2022701144 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc890095d70 con 0x56023e216780 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.056+0000 7fc8b97bcb80 10 client.15357 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.057+0000 7fc8b97bcb80 10 client.15357 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.057+0000 7fc8b97bcb80 20 client.15357 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.057+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x56023e22b420 0x56023e2306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.057+0000 7fc8b97bcb80 4 client.15357 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:35.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.057+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2022701144 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56023e1606b0 con 0x56023e22b420 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.057+0000 7fc89e000700 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x56023e22b420 0x56023e2306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.057+0000 7fc888c00700 10 client.15357 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.057+0000 7fc89e000700 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x56023e22b420 0x56023e2306e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98c839621f40663c server_cookie=b053703faa91b2e9 in_seq=0 out_seq=0 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc888c00700 1 -- 172.21.15.67:0/2022701144 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56023e1606b0 con 0x56023e22b420 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc888c00700 10 client.15357 handle_command_reply: tid=0 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc8b97bcb80 1 client.15357 shutdown 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2022701144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e216780 msgr2=0x56023e220550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e216780 0x56023e220550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc890099ec0 tx=0x7fc890099490 comp rx=0 tx=0).stop 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2022701144 shutdown_connections 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x56023e22b420 0x56023e2306e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e216780 0x56023e220550 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e227ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc888c00700 0 client.15357 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:36:35.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2022701144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 0x56023e220a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.058+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2022701144 >> 172.21.15.67:0/2022701144 conn(0x56023e175650 msgr2=0x56023e215120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.059+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2022701144 shutdown_connections 2024-03-30T02:36:35.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.059+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2022701144 wait complete. 2024-03-30T02:36:35.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.059+0000 7fc8b97bcb80 20 client.15357 trim_cache size 0 max 16384 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.077+0000 7fc8b97bcb80 1 Processor -- start 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.077+0000 7fc8b97bcb80 1 -- start start 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 0x56023e228180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e227a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e218360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e1606e0 con 0x56023e2170e0 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e128360 con 0x56023e217f80 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e125ad0 con 0x56023e216780 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e227a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e227a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50554/0 (socket says 172.21.15.67:50554) 2024-03-30T02:36:35.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89e000700 1 -- 172.21.15.67:0/2277039361 learned_addr learned my addr 172.21.15.67:0/2277039361 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89e000700 1 -- 172.21.15.67:0/2277039361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 msgr2=0x56023e228180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89e000700 1 --2- 172.21.15.67:0/2277039361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 0x56023e228180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89f400700 1 --2- 172.21.15.67:0/2277039361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e218360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89e000700 1 -- 172.21.15.67:0/2277039361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 msgr2=0x56023e218360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89e000700 1 --2- 172.21.15.67:0/2277039361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e218360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89e000700 1 -- 172.21.15.67:0/2277039361 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56023e12de50 con 0x56023e2170e0 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.078+0000 7fc89f400700 1 --2- 172.21.15.67:0/2277039361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e218360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.079+0000 7fc89e000700 1 --2- 172.21.15.67:0/2277039361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e227a90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009840 tx=0x7fc88c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf25aebf0953531e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.079+0000 7fc894c00700 1 -- 172.21.15.67:0/2277039361 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c01f070 con 0x56023e2170e0 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.079+0000 7fc894c00700 1 -- 172.21.15.67:0/2277039361 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc88c018040 con 0x56023e2170e0 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.079+0000 7fc894c00700 1 -- 172.21.15.67:0/2277039361 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c011aa0 con 0x56023e2170e0 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.079+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2277039361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 msgr2=0x56023e227a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.079+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2277039361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e227a90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009840 tx=0x7fc88c009310 comp rx=0 tx=0).stop 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.080+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2277039361 shutdown_connections 2024-03-30T02:36:35.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.080+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2277039361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e217f80 0x56023e218360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.080+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2277039361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e216780 0x56023e228180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.080+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/2277039361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e2170e0 0x56023e227a90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.080+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2277039361 >> 172.21.15.67:0/2277039361 conn(0x56023e175650 msgr2=0x56023e21e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.080+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2277039361 shutdown_connections 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.080+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/2277039361 wait complete. 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.080+0000 7fc8b97bcb80 1 -- start start 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.080+0000 7fc8b97bcb80 1 Processor -- start 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e216780 0x56023e227690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 0x56023e224240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc8b97bcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e217f80 0x56023e224780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e227eb0 con 0x56023e217f80 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e224cc0 con 0x56023e216780 2024-03-30T02:36:35.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc8b97bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023e224e80 con 0x56023e2170e0 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc8b97bcb80 10 client.0 _ll_register_callbacks cb 0x56023e10d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e217f80 0x56023e224780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e216780 0x56023e227690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e217f80 0x56023e224780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50564/0 (socket says 172.21.15.67:50564) 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89f400700 1 -- 172.21.15.67:0/3190184980 learned_addr learned my addr 172.21.15.67:0/3190184980 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89e000700 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 0x56023e224240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89f400700 1 -- 172.21.15.67:0/3190184980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 msgr2=0x56023e224240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89f400700 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 0x56023e224240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89f400700 1 -- 172.21.15.67:0/3190184980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e216780 msgr2=0x56023e227690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89f400700 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e216780 0x56023e227690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89f400700 1 -- 172.21.15.67:0/3190184980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c009b20 con 0x56023e217f80 2024-03-30T02:36:35.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.081+0000 7fc89e000700 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 0x56023e224240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.082+0000 7fc89f400700 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e217f80 0x56023e224780 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc898007840 tx=0x7fc898009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451c0323cfb5603f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.082+0000 7fc888c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.082+0000 7fc888c00700 1 -- 172.21.15.67:0/3190184980 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898005510 con 0x56023e217f80 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.082+0000 7fc888c00700 1 -- 172.21.15.67:0/3190184980 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fc898005670 con 0x56023e217f80 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.082+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/3190184980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc89009ddb0 con 0x56023e217f80 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.082+0000 7fc888c00700 1 -- 172.21.15.67:0/3190184980 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc898005830 con 0x56023e217f80 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.082+0000 7fc8b97bcb80 10 client.15363 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.082+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/3190184980 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56023dfedc80 con 0x56023e217f80 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.082+0000 7fc888c00700 1 -- 172.21.15.67:0/3190184980 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc89009ddb0 con 0x56023e217f80 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc888c00700 1 -- 172.21.15.67:0/3190184980 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc898028400 con 0x56023e217f80 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc8b97bcb80 10 client.15363 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc8b97bcb80 10 client.15363 resolve_mds: resolved name 'g' to daemon mds.g 2024-03-30T02:36:35.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc8b97bcb80 20 client.15363 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:35.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x56023e2849b0 0x56023e281320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc8b97bcb80 4 client.15363 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:35.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/3190184980 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56023dfb6ca0 con 0x56023e2849b0 2024-03-30T02:36:35.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc89ea00700 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x56023e2849b0 0x56023e281320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc89ea00700 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x56023e2849b0 0x56023e281320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=825d6468f963129e server_cookie=5ba39ccdb68a2225 in_seq=0 out_seq=0 2024-03-30T02:36:35.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.083+0000 7fc888c00700 10 client.15363 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:36:35.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc888c00700 1 -- 172.21.15.67:0/3190184980 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56023dfb6ca0 con 0x56023e2849b0 2024-03-30T02:36:35.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc888c00700 10 client.15363 handle_command_reply: tid=0 2024-03-30T02:36:35.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 client.15363 shutdown 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/3190184980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e217f80 msgr2=0x56023e224780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e217f80 0x56023e224780 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc898007840 tx=0x7fc898009920 comp rx=0 tx=0).stop 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/3190184980 shutdown_connections 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x56023e2849b0 0x56023e281320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56023e217f80 0x56023e224780 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56023e216780 0x56023e227690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc888c00700 0 client.15363 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 --2- 172.21.15.67:0/3190184980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56023e2170e0 0x56023e224240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/3190184980 >> 172.21.15.67:0/3190184980 conn(0x56023e175650 msgr2=0x56023e234700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/3190184980 shutdown_connections 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.084+0000 7fc8b97bcb80 1 -- 172.21.15.67:0/3190184980 wait complete. 2024-03-30T02:36:35.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8b97bcb80 20 client.15363 trim_cache size 0 max 16384 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24560, 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 78.994300714000005, 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 87, 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:35.091 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 117.883513228 2024-03-30T02:36:35.092 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:35.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 -- 172.21.15.67:0/3931427218 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc8840a1120 msgr2=0x7fc8840a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc8840a1120 0x7fc8840a35e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc898009550 tx=0x7fc898009320 comp rx=0 tx=0).stop 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 -- 172.21.15.67:0/3931427218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8a01017e0 msgr2=0x7fc8a0101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8a01017e0 0x7fc8a0101100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9a0 tx=0x7fc89000ed60 comp rx=0 tx=0).stop 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 -- 172.21.15.67:0/3931427218 shutdown_connections 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc8840a1120 0x7fc8840a35e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8a0106a40 0x7fc8a00ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8a01017e0 0x7fc8a0101100 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 --2- 172.21.15.67:0/3931427218 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc8a0104110 0x7fc8a00ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 -- 172.21.15.67:0/3931427218 >> 172.21.15.67:0/3931427218 conn(0x7fc8a00fb1d0 msgr2=0x7fc8a01022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 -- 172.21.15.67:0/3931427218 shutdown_connections 2024-03-30T02:36:35.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.085+0000 7fc8a5c00700 1 -- 172.21.15.67:0/3931427218 wait complete. 2024-03-30T02:36:35.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-30T02:36:35.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.304+0000 7fc047200700 1 -- 172.21.15.67:0/1829649401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc040101f70 msgr2=0x7fc0401023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.304+0000 7fc047200700 1 --2- 172.21.15.67:0/1829649401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc040101f70 0x7fc0401023d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0099c0 tx=0x7fc02c009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.304+0000 7fc047200700 1 -- 172.21.15.67:0/1829649401 shutdown_connections 2024-03-30T02:36:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.304+0000 7fc047200700 1 --2- 172.21.15.67:0/1829649401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc040102910 0x7fc0401091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.304+0000 7fc047200700 1 --2- 172.21.15.67:0/1829649401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc040100d80 0x7fc0401011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.304+0000 7fc047200700 1 --2- 172.21.15.67:0/1829649401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc040101f70 0x7fc0401023d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.304+0000 7fc047200700 1 -- 172.21.15.67:0/1829649401 >> 172.21.15.67:0/1829649401 conn(0x7fc0400fc2f0 msgr2=0x7fc0400fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.304+0000 7fc047200700 1 -- 172.21.15.67:0/1829649401 shutdown_connections 2024-03-30T02:36:35.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.304+0000 7fc047200700 1 -- 172.21.15.67:0/1829649401 wait complete. 2024-03-30T02:36:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.305+0000 7fc047200700 1 Processor -- start 2024-03-30T02:36:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.305+0000 7fc047200700 1 -- start start 2024-03-30T02:36:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc047200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc040100d80 0x7fc040196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc047200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc040101f70 0x7fc040197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc047200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc040102910 0x7fc04019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc047200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04019e830 con 0x7fc040102910 2024-03-30T02:36:35.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc047200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04019e9a0 con 0x7fc040100d80 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc047200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04019eb60 con 0x7fc040101f70 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc040100d80 0x7fc040196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc040100d80 0x7fc040196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39912/0 (socket says 172.21.15.67:39912) 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc03fe00700 1 -- 172.21.15.67:0/1744776170 learned_addr learned my addr 172.21.15.67:0/1744776170 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc045200700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc040102910 0x7fc04019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc03f400700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc040101f70 0x7fc040197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc03fe00700 1 -- 172.21.15.67:0/1744776170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc040101f70 msgr2=0x7fc040197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc03fe00700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc040101f70 0x7fc040197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc03fe00700 1 -- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc040102910 msgr2=0x7fc04019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc03fe00700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc040102910 0x7fc04019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.306+0000 7fc03fe00700 1 -- 172.21.15.67:0/1744776170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0095f0 con 0x7fc040100d80 2024-03-30T02:36:35.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.307+0000 7fc03fe00700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc040100d80 0x7fc040196d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc03000e970 tx=0x7fc03000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.307+0000 7fc03cc00700 1 -- 172.21.15.67:0/1744776170 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030009790 con 0x7fc040100d80 2024-03-30T02:36:35.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.307+0000 7fc047200700 1 -- 172.21.15.67:0/1744776170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04019ee40 con 0x7fc040100d80 2024-03-30T02:36:35.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.307+0000 7fc03cc00700 1 -- 172.21.15.67:0/1744776170 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc03000eec0 con 0x7fc040100d80 2024-03-30T02:36:35.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.307+0000 7fc03cc00700 1 -- 172.21.15.67:0/1744776170 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030003cb0 con 0x7fc040100d80 2024-03-30T02:36:35.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.307+0000 7fc047200700 1 -- 172.21.15.67:0/1744776170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04019f270 con 0x7fc040100d80 2024-03-30T02:36:35.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.310+0000 7fc03cc00700 1 -- 172.21.15.67:0/1744776170 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fc03000cbd0 con 0x7fc040100d80 2024-03-30T02:36:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.311+0000 7fc05ac98b80 1 Processor -- start 2024-03-30T02:36:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc03cc00700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc0240a11a0 0x7fc0240a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc05ac98b80 1 -- start start 2024-03-30T02:36:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc03cc00700 1 -- 172.21.15.67:0/1744776170 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fc030014070 con 0x7fc040100d80 2024-03-30T02:36:35.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc045200700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc0240a11a0 0x7fc0240a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abcec9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abceca4f0 0x560abcecadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcecb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abce1d940 con 0x560abcec9b90 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abce1b300 con 0x560abcecb390 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abce16bf0 con 0x560abceca4f0 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abcec9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abcec9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50596/0 (socket says 172.21.15.67:50596) 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc03fe00700 1 -- 172.21.15.67:0/3335247777 learned_addr learned my addr 172.21.15.67:0/3335247777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.312+0000 7fc03f400700 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abceca4f0 0x560abcecadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc045200700 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcecb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc03fe00700 1 -- 172.21.15.67:0/3335247777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abceca4f0 msgr2=0x560abcecadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc03fe00700 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abceca4f0 0x560abcecadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc03fe00700 1 -- 172.21.15.67:0/3335247777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 msgr2=0x560abcecb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc03fe00700 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcecb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc03fe00700 1 -- 172.21.15.67:0/3335247777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560abcc98360 con 0x560abcec9b90 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc045200700 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcecb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc045200700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc0240a11a0 0x7fc0240a3660 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc03800f590 tx=0x7fc03800b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc03fe00700 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abcec9fb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc030099c60 tx=0x7fc03009a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947f48878eeab188 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc035600700 1 -- 172.21.15.67:0/3335247777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0300b1070 con 0x560abcec9b90 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc035600700 1 -- 172.21.15.67:0/3335247777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc03009dd90 con 0x560abcec9b90 2024-03-30T02:36:35.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.313+0000 7fc035600700 1 -- 172.21.15.67:0/3335247777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0300a21b0 con 0x560abcec9b90 2024-03-30T02:36:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.314+0000 7fc05ac98b80 1 -- 172.21.15.67:0/3335247777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 msgr2=0x560abcec9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.314+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abcec9fb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc030099c60 tx=0x7fc03009a660 comp rx=0 tx=0).stop 2024-03-30T02:36:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.314+0000 7fc05ac98b80 1 -- 172.21.15.67:0/3335247777 shutdown_connections 2024-03-30T02:36:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.314+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abcec9fb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.314+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcecb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.314+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/3335247777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abceca4f0 0x560abcecadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.314+0000 7fc05ac98b80 1 -- 172.21.15.67:0/3335247777 >> 172.21.15.67:0/3335247777 conn(0x560abce24650 msgr2=0x560abcec0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.314+0000 7fc05ac98b80 1 -- 172.21.15.67:0/3335247777 shutdown_connections 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.314+0000 7fc05ac98b80 1 -- 172.21.15.67:0/3335247777 wait complete. 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.315+0000 7fc05ac98b80 1 -- start start 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.315+0000 7fc05ac98b80 1 Processor -- start 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcec9b90 0x560abced3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcedb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abcdb2d30 con 0x560abceca4f0 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abcdb29d0 con 0x560abcecb390 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abcc9a650 con 0x560abcec9b90 2024-03-30T02:36:35.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc05ac98b80 10 client.0 _ll_register_callbacks cb 0x560abcdbc420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc03f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc03f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50604/0 (socket says 172.21.15.67:50604) 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc03f400700 1 -- 172.21.15.67:0/1206625594 learned_addr learned my addr 172.21.15.67:0/1206625594 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc03fe00700 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcec9b90 0x560abced3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc045200700 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcedb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc03f400700 1 -- 172.21.15.67:0/1206625594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcec9b90 msgr2=0x560abced3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc03f400700 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcec9b90 0x560abced3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc03f400700 1 -- 172.21.15.67:0/1206625594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 msgr2=0x560abcedb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc03f400700 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcedb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc03f400700 1 -- 172.21.15.67:0/1206625594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04019f270 con 0x560abceca4f0 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.316+0000 7fc045200700 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcedb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.317+0000 7fc03f400700 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced4010 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009fd0 tx=0x7fc02c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2a943c3a2ced34 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.317+0000 7fc02a000700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.317+0000 7fc02a000700 1 -- 172.21.15.67:0/1206625594 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c01d040 con 0x560abceca4f0 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.317+0000 7fc05ac98b80 1 -- 172.21.15.67:0/1206625594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560abcdd7330 con 0x560abceca4f0 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.317+0000 7fc02a000700 1 -- 172.21.15.67:0/1206625594 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fc02c0054e0 con 0x560abceca4f0 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.317+0000 7fc02a000700 1 -- 172.21.15.67:0/1206625594 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc02c0056a0 con 0x560abceca4f0 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.317+0000 7fc05ac98b80 10 client.15375 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.317+0000 7fc05ac98b80 1 -- 172.21.15.67:0/1206625594 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560abcc44b10 con 0x560abceca4f0 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.318+0000 7fc02a000700 1 -- 172.21.15.67:0/1206625594 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560abcdd7330 con 0x560abceca4f0 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.318+0000 7fc02a000700 1 -- 172.21.15.67:0/1206625594 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc02c0238d0 con 0x560abceca4f0 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.318+0000 7fc05ac98b80 10 client.15375 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.318+0000 7fc05ac98b80 10 client.15375 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.318+0000 7fc05ac98b80 20 client.15375 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:35.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.318+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560abcedec00 0x560abcee3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.318+0000 7fc05ac98b80 4 client.15375 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.318+0000 7fc05ac98b80 1 -- 172.21.15.67:0/1206625594 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560abce0f6b0 con 0x560abcedec00 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.318+0000 7fc03fe00700 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560abcedec00 0x560abcee3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.319+0000 7fc03fe00700 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560abcedec00 0x560abcee3f00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2efcd63d76b819c6 server_cookie=1710527104e800f9 in_seq=0 out_seq=0 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.319+0000 7fc02a000700 10 client.15375 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc02a000700 1 -- 172.21.15.67:0/1206625594 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560abce0f6b0 con 0x560abcedec00 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc02a000700 10 client.15375 handle_command_reply: tid=0 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 client.15375 shutdown 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 -- 172.21.15.67:0/1206625594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 msgr2=0x560abced4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced4010 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009fd0 tx=0x7fc02c00bea0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 -- 172.21.15.67:0/1206625594 shutdown_connections 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcecb390 0x560abcedb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcec9b90 0x560abced3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560abcedec00 0x560abcee3f00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/1206625594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced4010 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc02a000700 0 client.15375 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 -- 172.21.15.67:0/1206625594 >> 172.21.15.67:0/1206625594 conn(0x560abce24650 msgr2=0x560abcec4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 -- 172.21.15.67:0/1206625594 shutdown_connections 2024-03-30T02:36:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 1 -- 172.21.15.67:0/1206625594 wait complete. 2024-03-30T02:36:35.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.320+0000 7fc05ac98b80 20 client.15375 trim_cache size 0 max 16384 2024-03-30T02:36:35.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.338+0000 7fc05ac98b80 1 Processor -- start 2024-03-30T02:36:35.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc05ac98b80 1 -- start start 2024-03-30T02:36:35.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abced2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abceca4f0 0x560abcedafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 0x560abcecb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abcec1610 con 0x560abcec9b90 2024-03-30T02:36:35.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abce0f6b0 con 0x560abceca4f0 2024-03-30T02:36:35.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abcdd7330 con 0x560abcecb390 2024-03-30T02:36:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abced2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc03f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abceca4f0 0x560abcedafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc03fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abced2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50610/0 (socket says 172.21.15.67:50610) 2024-03-30T02:36:35.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc03fe00700 1 -- 172.21.15.67:0/2982991195 learned_addr learned my addr 172.21.15.67:0/2982991195 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc03f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abceca4f0 0x560abcedafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39922/0 (socket says 172.21.15.67:39922) 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.339+0000 7fc045200700 1 --2- 172.21.15.67:0/2982991195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 0x560abcecb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc03f400700 1 -- 172.21.15.67:0/2982991195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 msgr2=0x560abcecb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc03f400700 1 --2- 172.21.15.67:0/2982991195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 0x560abcecb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc03f400700 1 -- 172.21.15.67:0/2982991195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 msgr2=0x560abced2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc03f400700 1 --2- 172.21.15.67:0/2982991195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abced2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc03f400700 1 -- 172.21.15.67:0/2982991195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560abcddce50 con 0x560abceca4f0 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc03fe00700 1 --2- 172.21.15.67:0/2982991195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abced2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc03f400700 1 --2- 172.21.15.67:0/2982991195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abceca4f0 0x560abcedafc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0098e0 tx=0x7fc02c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54d50ba78ac932a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc036000700 1 -- 172.21.15.67:0/2982991195 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c039070 con 0x560abceca4f0 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc036000700 1 -- 172.21.15.67:0/2982991195 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc02c005940 con 0x560abceca4f0 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.340+0000 7fc036000700 1 -- 172.21.15.67:0/2982991195 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c01d040 con 0x560abceca4f0 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 -- 172.21.15.67:0/2982991195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abceca4f0 msgr2=0x560abcedafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/2982991195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abceca4f0 0x560abcedafc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0098e0 tx=0x7fc02c000bc0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 -- 172.21.15.67:0/2982991195 shutdown_connections 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/2982991195 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abcec9b90 0x560abced2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/2982991195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 0x560abcecb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/2982991195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abceca4f0 0x560abcedafc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 -- 172.21.15.67:0/2982991195 >> 172.21.15.67:0/2982991195 conn(0x560abce24650 msgr2=0x560abceb9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 -- 172.21.15.67:0/2982991195 shutdown_connections 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 -- 172.21.15.67:0/2982991195 wait complete. 2024-03-30T02:36:35.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.341+0000 7fc05ac98b80 1 -- start start 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc05ac98b80 1 Processor -- start 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcec9b90 0x560abcedad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc05ac98b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 0x560abced7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abced8290 con 0x560abceca4f0 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abced8400 con 0x560abcec9b90 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc05ac98b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560abcf334f0 con 0x560abcecb390 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc05ac98b80 10 client.0 _ll_register_callbacks cb 0x560abcdbc420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc03f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc03f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50624/0 (socket says 172.21.15.67:50624) 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc045200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 0x560abced7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc03f400700 1 -- 172.21.15.67:0/4253093308 learned_addr learned my addr 172.21.15.67:0/4253093308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.342+0000 7fc03fe00700 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcec9b90 0x560abcedad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc03fe00700 1 -- 172.21.15.67:0/4253093308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 msgr2=0x560abced7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc03fe00700 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 0x560abced7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc03fe00700 1 -- 172.21.15.67:0/4253093308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 msgr2=0x560abced7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc03fe00700 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc03fe00700 1 -- 172.21.15.67:0/4253093308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560abcc98360 con 0x560abcec9b90 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc03f400700 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc03fe00700 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcec9b90 0x560abcedad70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc030099ef0 tx=0x7fc030003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c031244eb49a6f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc02a000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc02a000700 1 -- 172.21.15.67:0/4253093308 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0300b1040 con 0x560abcec9b90 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc02a000700 1 -- 172.21.15.67:0/4253093308 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fc030098120 con 0x560abcec9b90 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc05ac98b80 1 -- 172.21.15.67:0/4253093308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc03009dd90 con 0x560abcec9b90 2024-03-30T02:36:35.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc02a000700 1 -- 172.21.15.67:0/4253093308 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc03009a1c0 con 0x560abcec9b90 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc05ac98b80 10 client.25427 fetch_fsmap learned FSMap version 28 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.343+0000 7fc05ac98b80 1 -- 172.21.15.67:0/4253093308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560abcc44b10 con 0x560abcec9b90 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.344+0000 7fc02a000700 1 -- 172.21.15.67:0/4253093308 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc03009dd90 con 0x560abcec9b90 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.344+0000 7fc02a000700 1 -- 172.21.15.67:0/4253093308 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0300994e0 con 0x560abcec9b90 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.344+0000 7fc05ac98b80 10 client.25427 fetch_fsmap finished waiting for FSMap version 28 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.344+0000 7fc05ac98b80 10 client.25427 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.344+0000 7fc05ac98b80 20 client.25427 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.344+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560abcf37f40 0x560abcf348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.344+0000 7fc05ac98b80 4 client.25427 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.344+0000 7fc05ac98b80 1 -- 172.21.15.67:0/4253093308 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560abcc65ca0 con 0x560abcf37f40 2024-03-30T02:36:35.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.344+0000 7fc03f400700 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560abcf37f40 0x560abcf348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc03f400700 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560abcf37f40 0x560abcf348b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5560a28ea11f46dd server_cookie=d4c9fa7edd94d078 in_seq=0 out_seq=0 2024-03-30T02:36:35.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc02a000700 10 client.25427 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:35.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc02a000700 1 -- 172.21.15.67:0/4253093308 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x560abcc65ca0 con 0x560abcf37f40 2024-03-30T02:36:35.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc02a000700 10 client.25427 handle_command_reply: tid=0 2024-03-30T02:36:35.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 client.25427 shutdown 2024-03-30T02:36:35.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 -- 172.21.15.67:0/4253093308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcec9b90 msgr2=0x560abcedad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcec9b90 0x560abcedad70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc030099ef0 tx=0x7fc030003e10 comp rx=0 tx=0).stop 2024-03-30T02:36:35.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 -- 172.21.15.67:0/4253093308 shutdown_connections 2024-03-30T02:36:35.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560abcecb390 0x560abced7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560abcec9b90 0x560abcedad70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560abcf37f40 0x560abcf348b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 --2- 172.21.15.67:0/4253093308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560abceca4f0 0x560abced7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc02a000700 0 client.25427 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:35.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 -- 172.21.15.67:0/4253093308 >> 172.21.15.67:0/4253093308 conn(0x560abce24650 msgr2=0x560abced2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 -- 172.21.15.67:0/4253093308 shutdown_connections 2024-03-30T02:36:35.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.345+0000 7fc05ac98b80 1 -- 172.21.15.67:0/4253093308 wait complete. 2024-03-30T02:36:35.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc05ac98b80 20 client.25427 trim_cache size 0 max 16384 2024-03-30T02:36:35.353 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:35.353 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "0ef0748e-ee3d-11ee-b647-cb9ed24678a4", 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24578, 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 108.629009074, 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 28, 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 87, 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 83, 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 113.983479249 2024-03-30T02:36:35.354 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:35.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 -- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc0240a11a0 msgr2=0x7fc0240a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc0240a11a0 0x7fc0240a3660 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc03800f590 tx=0x7fc03800b040 comp rx=0 tx=0).stop 2024-03-30T02:36:35.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 -- 172.21.15.67:0/1744776170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc040100d80 msgr2=0x7fc040196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:35.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc040100d80 0x7fc040196d10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc03000e970 tx=0x7fc03000ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:35.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 -- 172.21.15.67:0/1744776170 shutdown_connections 2024-03-30T02:36:35.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc0240a11a0 0x7fc0240a3660 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc040102910 0x7fc04019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc040100d80 0x7fc040196d10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 --2- 172.21.15.67:0/1744776170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc040101f70 0x7fc040197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:35.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 -- 172.21.15.67:0/1744776170 >> 172.21.15.67:0/1744776170 conn(0x7fc0400fc2f0 msgr2=0x7fc040105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:35.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 -- 172.21.15.67:0/1744776170 shutdown_connections 2024-03-30T02:36:35.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:35.346+0000 7fc047200700 1 -- 172.21.15.67:0/1744776170 wait complete. 2024-03-30T02:36:35.363 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:35 smithi067 ceph-mon[37808]: pgmap v291: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:35.366 INFO:teuthology.run_tasks:Running task kclient... 2024-03-30T02:36:35.379 INFO:tasks.kclient:Mounting kernel clients... 2024-03-30T02:36:35.379 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-30T02:36:35.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:35.380 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-03-30T02:36:35.421 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-30T02:36:35.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:35.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T02:36:35.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:35.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-03-30T02:36:35.642 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-30T02:36:35.642 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T02:36:35.642 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T02:36:35.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-03-30T02:36:35.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:35 smithi170 ceph-mon[78798]: pgmap v291: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:35.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:35 smithi082 ceph-mon[79176]: pgmap v291: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:36.014 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:36:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.242+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3996392855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c8106e40 msgr2=0x7fe5c8107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.242+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3996392855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c8106e40 0x7fe5c8107220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009990 tx=0x7fe5b8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.242+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3996392855 shutdown_connections 2024-03-30T02:36:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.242+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3996392855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c8106e40 0x7fe5c8107220 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.242+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3996392855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81017c0 0x7fe5c810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.242+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3996392855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5c8100e00 0x7fe5c8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.242+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3996392855 >> 172.21.15.67:0/3996392855 conn(0x7fe5c8077f10 msgr2=0x7fe5c8078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.242+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3996392855 shutdown_connections 2024-03-30T02:36:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.242+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3996392855 wait complete. 2024-03-30T02:36:37.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.243+0000 7fe5ccc00700 1 Processor -- start 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5ccc00700 1 -- start start 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5ccc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c8100e00 0x7fe5c81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5ccc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81017c0 0x7fe5c8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5ccc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5c8106e40 0x7fe5c819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5ccc00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c819e1e0 con 0x7fe5c8100e00 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5ccc00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c819e350 con 0x7fe5c8106e40 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5ccc00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c819e510 con 0x7fe5c81017c0 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5c5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81017c0 0x7fe5c8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5c5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81017c0 0x7fe5c8199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:46904/0 (socket says 172.21.15.67:46904) 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5c5400700 1 -- 172.21.15.67:0/3335874775 learned_addr learned my addr 172.21.15.67:0/3335874775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5c6800700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5c8106e40 0x7fe5c819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:37.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.244+0000 7fe5c5e00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c8100e00 0x7fe5c81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:37.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.245+0000 7fe5c5400700 1 -- 172.21.15.67:0/3335874775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5c8106e40 msgr2=0x7fe5c819dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:37.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.245+0000 7fe5c5400700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5c8106e40 0x7fe5c819dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.245+0000 7fe5c5400700 1 -- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c8100e00 msgr2=0x7fe5c81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:37.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.245+0000 7fe5c5400700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c8100e00 0x7fe5c81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.245+0000 7fe5c5400700 1 -- 172.21.15.67:0/3335874775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b80095f0 con 0x7fe5c81017c0 2024-03-30T02:36:37.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.245+0000 7fe5c5400700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81017c0 0x7fe5c8199820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e9a0 tx=0x7fe5b400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:37.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.245+0000 7fe5bea00700 1 -- 172.21.15.67:0/3335874775 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4009790 con 0x7fe5c81017c0 2024-03-30T02:36:37.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.246+0000 7fe5bea00700 1 -- 172.21.15.67:0/3335874775 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b4010430 con 0x7fe5c81017c0 2024-03-30T02:36:37.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.246+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c819e7f0 con 0x7fe5c81017c0 2024-03-30T02:36:37.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.246+0000 7fe5bea00700 1 -- 172.21.15.67:0/3335874775 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4018680 con 0x7fe5c81017c0 2024-03-30T02:36:37.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.246+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c81a6050 con 0x7fe5c81017c0 2024-03-30T02:36:37.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.247+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c8102480 con 0x7fe5c81017c0 2024-03-30T02:36:37.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.249+0000 7fe5bea00700 1 -- 172.21.15.67:0/3335874775 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fe5b40105a0 con 0x7fe5c81017c0 2024-03-30T02:36:37.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.250+0000 7fe5bea00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe5ac0a11a0 0x7fe5ac0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:37.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.250+0000 7fe5bea00700 1 -- 172.21.15.67:0/3335874775 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fe5b4014070 con 0x7fe5c81017c0 2024-03-30T02:36:37.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.250+0000 7fe5c5e00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe5ac0a11a0 0x7fe5ac0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:37.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.251+0000 7fe5c5e00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe5ac0a11a0 0x7fe5ac0a3660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80098e0 tx=0x7fe5b8005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:37.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.254+0000 7fe5bea00700 1 -- 172.21.15.67:0/3335874775 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe5b40529f0 con 0x7fe5c81017c0 2024-03-30T02:36:37.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:37 smithi067 ceph-mon[37808]: pgmap v292: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 730 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:37.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:37 smithi082 ceph-mon[79176]: pgmap v292: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 730 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:37.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.516+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe5c804ea80 con 0x7fe5c81017c0 2024-03-30T02:36:37.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.517+0000 7fe5bea00700 1 -- 172.21.15.67:0/3335874775 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe5b40961a0 con 0x7fe5c81017c0 2024-03-30T02:36:37.518 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T02:36:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe5ac0a11a0 msgr2=0x7fe5ac0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe5ac0a11a0 0x7fe5ac0a3660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80098e0 tx=0x7fe5b8005960 comp rx=0 tx=0).stop 2024-03-30T02:36:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81017c0 msgr2=0x7fe5c8199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81017c0 0x7fe5c8199820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5b400e9a0 tx=0x7fe5b400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 shutdown_connections 2024-03-30T02:36:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe5ac0a11a0 0x7fe5ac0a3660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5c8100e00 0x7fe5c81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5c8106e40 0x7fe5c819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 --2- 172.21.15.67:0/3335874775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5c81017c0 0x7fe5c8199820 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 >> 172.21.15.67:0/3335874775 conn(0x7fe5c8077f10 msgr2=0x7fe5c810c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 shutdown_connections 2024-03-30T02:36:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:37.519+0000 7fe5ccc00700 1 -- 172.21.15.67:0/3335874775 wait complete. 2024-03-30T02:36:37.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:37 smithi170 ceph-mon[78798]: pgmap v292: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 730 B/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:37.909 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T02:36:37.909 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T02:36:37.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-03-30T02:36:37.909 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T02:36:37.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T02:36:37.909 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T02:36:37.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T02:36:37.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T02:36:37.909 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T02:36:37.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:37.910 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-03-30T02:36:37.953 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T02:36:37.953 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T02:36:37.953 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-03-30T02:36:37.954 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-30T02:36:37.955 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41727sec preferred_lft 41727sec 2024-03-30T02:36:37.955 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-03-30T02:36:37.955 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-03-30T02:36:37.955 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T02:36:37.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:37.955 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:36:37.955 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:36:37.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-03-30T02:36:37.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-03-30T02:36:37.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-03-30T02:36:37.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T02:36:37.956 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:36:38.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:38.198 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T02:36:38.268 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-03-30T02:36:38.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:38.273 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-03-30T02:36:38.318 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-03-30T02:36:38.318 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T02:36:38.319 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T02:36:38.319 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T02:36:38.319 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T02:36:38.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:38.320 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:36:38.320 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:36:38.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T02:36:38.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T02:36:38.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T02:36:38.320 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:36:38.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:38 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3335874775' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-30T02:36:38.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:38.486 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-03-30T02:36:38.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:38 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3335874775' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-30T02:36:38.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:38.529 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-03-30T02:36:38.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:38.616 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:36:38.616 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:36:38.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T02:36:38.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-30T02:36:38.617 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:36:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:38 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3335874775' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-30T02:36:38.795 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-30T02:36:38.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:38.796 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:36:38.796 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:36:38.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-30T02:36:38.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T02:36:38.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-30T02:36:38.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-30T02:36:38.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-30T02:36:38.796 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:36:39.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:39.048 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec bash -c ' 2024-03-30T02:36:39.048 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-03-30T02:36:39.048 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-03-30T02:36:39.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T02:36:39.049 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-03-30T02:36:39.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T02:36:39.218 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T02:36:39.262 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T02:36:39.262 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T02:36:39.262 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T02:36:39.343 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-30T02:36:39.343 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-30T02:36:39.343 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/293e8158-1cf3-4427-b5aa-a50888e9e995 2024-03-30T02:36:39.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:39.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/293e8158-1cf3-4427-b5aa-a50888e9e995 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-03-30T02:36:39.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:39 smithi067 ceph-mon[37808]: pgmap v293: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 568 B/s wr, 12 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:39.485 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-03-30T02:36:39.485 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-03-30T02:36:39.485 INFO:teuthology.orchestra.run.smithi067.stdout:invalid new device string format 2024-03-30T02:36:39.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:39 smithi082 ceph-mon[79176]: pgmap v293: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 568 B/s wr, 12 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:39 smithi170 ceph-mon[78798]: pgmap v293: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 568 B/s wr, 12 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:40.118 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-03-30T02:36:40.118 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-03-30T02:36:40.119 INFO:teuthology.orchestra.run.smithi067.stdout:invalid new device string format 2024-03-30T02:36:40.119 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.82:3300,172.21.15.170:3300" 2024-03-30T02:36:40.119 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:3300,172.21.15.82:3300,172.21.15.170:3300:/volumes/qa/sv_1/293e8158-1cf3-4427-b5aa-a50888e9e995 2024-03-30T02:36:40.119 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=0ef0748e-ee3d-11ee-b647-cb9ed24678a4" will pass to kernel 2024-03-30T02:36:40.124 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-30T02:36:40.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:40.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T02:36:40.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:40.241 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T02:36:40.324 INFO:teuthology.orchestra.run.smithi067.stdout:client25429 2024-03-30T02:36:40.324 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:36:40.324 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T02:36:40.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:40.367 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T02:36:40.489 INFO:teuthology.orchestra.run.smithi067.stdout:client25429 2024-03-30T02:36:40.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:40.490 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-30T02:36:40.573 INFO:teuthology.orchestra.run.smithi067.stdout:client25429 2024-03-30T02:36:40.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:36:40.573 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4.client25429/status') 2024-03-30T02:36:40.676 INFO:teuthology.orchestra.run.smithi067.stdout:instance: client.25429 (3)172.21.15.67:0/3116968970 2024-03-30T02:36:40.676 INFO:teuthology.orchestra.run.smithi067.stdout:blocklisted: false 2024-03-30T02:36:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records in 2024-03-30T02:36:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records out 2024-03-30T02:36:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:71 bytes copied, 5.6671e-05 s, 1.3 MB/s 2024-03-30T02:36:40.679 INFO:teuthology.run_tasks:Running task exec... 2024-03-30T02:36:40.694 INFO:teuthology.task.exec:Executing custom commands... 2024-03-30T02:36:40.694 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:36:40.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-30T02:36:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.888+0000 7f973b600700 1 -- 172.21.15.67:0/1981883026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9734108bd0 msgr2=0x7f973410b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.888+0000 7f973b600700 1 --2- 172.21.15.67:0/1981883026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9734108bd0 0x7f973410b060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f972c00b0a0 tx=0x7f972c00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.888+0000 7f973b600700 1 -- 172.21.15.67:0/1981883026 shutdown_connections 2024-03-30T02:36:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.888+0000 7f973b600700 1 --2- 172.21.15.67:0/1981883026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9734108bd0 0x7f973410b060 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.888+0000 7f973b600700 1 --2- 172.21.15.67:0/1981883026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9734074df0 0x7f9734073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.888+0000 7f973b600700 1 --2- 172.21.15.67:0/1981883026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9734073810 0x7f9734073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.888+0000 7f973b600700 1 -- 172.21.15.67:0/1981883026 >> 172.21.15.67:0/1981883026 conn(0x7f97340fc280 msgr2=0x7f97340fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.888+0000 7f973b600700 1 -- 172.21.15.67:0/1981883026 shutdown_connections 2024-03-30T02:36:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.888+0000 7f973b600700 1 -- 172.21.15.67:0/1981883026 wait complete. 2024-03-30T02:36:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.890+0000 7f973b600700 1 Processor -- start 2024-03-30T02:36:40.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.890+0000 7f973b600700 1 -- start start 2024-03-30T02:36:40.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.890+0000 7f973b600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9734073810 0x7f9734198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.890+0000 7f973b600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9734074df0 0x7f97341994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.890+0000 7f973b600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9734108bd0 0x7f97341a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.890+0000 7f973b600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97341a0ac0 con 0x7f9734074df0 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.890+0000 7f973b600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97341a0c00 con 0x7f9734108bd0 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.890+0000 7f973b600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97341a0d90 con 0x7f9734073810 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9733e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9734074df0 0x7f97341994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9738c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9734073810 0x7f9734198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9738c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9734073810 0x7f9734198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:46916/0 (socket says 172.21.15.67:46916) 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9738c00700 1 -- 172.21.15.67:0/4085344482 learned_addr learned my addr 172.21.15.67:0/4085344482 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9739600700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9734108bd0 0x7f97341a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9733e00700 1 -- 172.21.15.67:0/4085344482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9734073810 msgr2=0x7f9734198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9733e00700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9734073810 0x7f9734198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9733e00700 1 -- 172.21.15.67:0/4085344482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9734108bd0 msgr2=0x7f97341a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9733e00700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9734108bd0 0x7f97341a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9733e00700 1 -- 172.21.15.67:0/4085344482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c009d00 con 0x7f9734074df0 2024-03-30T02:36:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9738c00700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9734073810 0x7f9734198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9739600700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9734108bd0 0x7f97341a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.891+0000 7f9733e00700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9734074df0 0x7f97341994d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f972000e9c0 tx=0x7f972000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.892+0000 7f9731600700 1 -- 172.21.15.67:0/4085344482 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9720004220 con 0x7f9734074df0 2024-03-30T02:36:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.892+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97341a1070 con 0x7f9734074df0 2024-03-30T02:36:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.892+0000 7f9731600700 1 -- 172.21.15.67:0/4085344482 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9720004380 con 0x7f9734074df0 2024-03-30T02:36:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.892+0000 7f9731600700 1 -- 172.21.15.67:0/4085344482 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9720003be0 con 0x7f9734074df0 2024-03-30T02:36:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.892+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97341a1580 con 0x7f9734074df0 2024-03-30T02:36:40.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.893+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9734107990 con 0x7f9734074df0 2024-03-30T02:36:40.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.895+0000 7f9731600700 1 -- 172.21.15.67:0/4085344482 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f9720003d40 con 0x7f9734074df0 2024-03-30T02:36:40.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.896+0000 7f9731600700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f971c0a1120 0x7f971c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:40.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.896+0000 7f9731600700 1 -- 172.21.15.67:0/4085344482 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f9720014070 con 0x7f9734074df0 2024-03-30T02:36:40.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.896+0000 7f9738c00700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f971c0a1120 0x7f971c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:40.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.897+0000 7f9738c00700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f971c0a1120 0x7f971c0a35e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9728011550 tx=0x7f9728009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:40.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:40.899+0000 7f9731600700 1 -- 172.21.15.67:0/4085344482 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9720096460 con 0x7f9734074df0 2024-03-30T02:36:41.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:41.163+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f9734067c30 con 0x7f9734074df0 2024-03-30T02:36:42.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.084+0000 7f9731600700 1 -- 172.21.15.67:0/4085344482 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9720099a80 con 0x7f9734074df0 2024-03-30T02:36:42.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.087+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f971c0a1120 msgr2=0x7f971c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.087+0000 7f973b600700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f971c0a1120 0x7f971c0a35e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9728011550 tx=0x7f9728009260 comp rx=0 tx=0).stop 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.087+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9734074df0 msgr2=0x7f97341994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.087+0000 7f973b600700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9734074df0 0x7f97341994d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f972000e9c0 tx=0x7f972000ecd0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.088+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 shutdown_connections 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.088+0000 7f973b600700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f971c0a1120 0x7f971c0a35e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.088+0000 7f973b600700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9734108bd0 0x7f97341a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.088+0000 7f973b600700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9734073810 0x7f9734198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.088+0000 7f973b600700 1 --2- 172.21.15.67:0/4085344482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9734074df0 0x7f97341994d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.088+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 >> 172.21.15.67:0/4085344482 conn(0x7f97340fc280 msgr2=0x7f9734104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.088+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 shutdown_connections 2024-03-30T02:36:42.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.088+0000 7f973b600700 1 -- 172.21.15.67:0/4085344482 wait complete. 2024-03-30T02:36:42.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-30T02:36:42.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.313+0000 7fcae1800700 1 -- 172.21.15.67:0/3470651174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc100d10 msgr2=0x7fcadc101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.313+0000 7fcae1800700 1 --2- 172.21.15.67:0/3470651174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc100d10 0x7fcadc101130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0099c0 tx=0x7fcacc009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.313+0000 7fcae1800700 1 -- 172.21.15.67:0/3470651174 shutdown_connections 2024-03-30T02:36:42.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.313+0000 7fcae1800700 1 --2- 172.21.15.67:0/3470651174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc100d10 0x7fcadc101130 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.313+0000 7fcae1800700 1 --2- 172.21.15.67:0/3470651174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcadc1028a0 0x7fcadc109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.313+0000 7fcae1800700 1 --2- 172.21.15.67:0/3470651174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcadc101f00 0x7fcadc102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.313+0000 7fcae1800700 1 -- 172.21.15.67:0/3470651174 >> 172.21.15.67:0/3470651174 conn(0x7fcadc0fc280 msgr2=0x7fcadc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.313+0000 7fcae1800700 1 -- 172.21.15.67:0/3470651174 shutdown_connections 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.313+0000 7fcae1800700 1 -- 172.21.15.67:0/3470651174 wait complete. 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.314+0000 7fcae1800700 1 Processor -- start 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.314+0000 7fcae1800700 1 -- start start 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.314+0000 7fcae1800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcadc100d10 0x7fcadc198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.314+0000 7fcae1800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc101f00 0x7fcadc199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.314+0000 7fcae1800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcadc1028a0 0x7fcadc1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.314+0000 7fcae1800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1a0a20 con 0x7fcadc101f00 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.314+0000 7fcae1800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1a0b60 con 0x7fcadc1028a0 2024-03-30T02:36:42.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.314+0000 7fcae1800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1a0cf0 con 0x7fcadc100d10 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcada000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc101f00 0x7fcadc199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcada000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc101f00 0x7fcadc199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57398/0 (socket says 172.21.15.67:57398) 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcada000700 1 -- 172.21.15.67:0/622822578 learned_addr learned my addr 172.21.15.67:0/622822578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcadb400700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcadc1028a0 0x7fcadc1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcadaa00700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcadc100d10 0x7fcadc198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcada000700 1 -- 172.21.15.67:0/622822578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcadc100d10 msgr2=0x7fcadc198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcada000700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcadc100d10 0x7fcadc198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcada000700 1 -- 172.21.15.67:0/622822578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcadc1028a0 msgr2=0x7fcadc1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcada000700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcadc1028a0 0x7fcadc1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcada000700 1 -- 172.21.15.67:0/622822578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc0095f0 con 0x7fcadc101f00 2024-03-30T02:36:42.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.315+0000 7fcada000700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc101f00 0x7fcadc199430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcac800d950 tx=0x7fcac800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:42.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.316+0000 7fcad3400700 1 -- 172.21.15.67:0/622822578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac800f890 con 0x7fcadc101f00 2024-03-30T02:36:42.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.316+0000 7fcad3400700 1 -- 172.21.15.67:0/622822578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcac8010460 con 0x7fcadc101f00 2024-03-30T02:36:42.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.316+0000 7fcad3400700 1 -- 172.21.15.67:0/622822578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac800e610 con 0x7fcadc101f00 2024-03-30T02:36:42.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.316+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc1a0fd0 con 0x7fcadc101f00 2024-03-30T02:36:42.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.316+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc1a1400 con 0x7fcadc101f00 2024-03-30T02:36:42.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.317+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc067c30 con 0x7fcadc101f00 2024-03-30T02:36:42.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.319+0000 7fcad3400700 1 -- 172.21.15.67:0/622822578 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fcac80108d0 con 0x7fcadc101f00 2024-03-30T02:36:42.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.320+0000 7fcad3400700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcac00a1120 0x7fcac00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:42.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.320+0000 7fcad3400700 1 -- 172.21.15.67:0/622822578 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fcac80cc9f0 con 0x7fcadc101f00 2024-03-30T02:36:42.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.320+0000 7fcadaa00700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcac00a1120 0x7fcac00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:42.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.321+0000 7fcadaa00700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcac00a1120 0x7fcac00a35e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcacc003730 tx=0x7fcacc000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:42.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.323+0000 7fcad3400700 1 -- 172.21.15.67:0/622822578 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcac8095de0 con 0x7fcadc101f00 2024-03-30T02:36:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:42 smithi067 ceph-mon[37808]: pgmap v294: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:42.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:42 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4085344482' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-30T02:36:42.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:42 smithi170 ceph-mon[78798]: pgmap v294: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:42.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:42 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4085344482' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-30T02:36:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:42 smithi082 ceph-mon[79176]: pgmap v294: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-03-30T02:36:42.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:42 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4085344482' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-30T02:36:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.578+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7fcadc108df0 con 0x7fcac00a1120 2024-03-30T02:36:42.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.597+0000 7fcad3400700 1 -- 172.21.15.67:0/622822578 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7fcadc108df0 con 0x7fcac00a1120 2024-03-30T02:36:42.598 INFO:teuthology.orchestra.run.smithi067.stdout:{} 2024-03-30T02:36:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcac00a1120 msgr2=0x7fcac00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcac00a1120 0x7fcac00a35e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcacc003730 tx=0x7fcacc000bc0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc101f00 msgr2=0x7fcadc199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc101f00 0x7fcadc199430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcac800d950 tx=0x7fcac800dc60 comp rx=0 tx=0).stop 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 shutdown_connections 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcac00a1120 0x7fcac00a35e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcadc1028a0 0x7fcadc1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcadc100d10 0x7fcadc198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 --2- 172.21.15.67:0/622822578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcadc101f00 0x7fcadc199430 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 >> 172.21.15.67:0/622822578 conn(0x7fcadc0fc280 msgr2=0x7fcadc1076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 shutdown_connections 2024-03-30T02:36:42.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.599+0000 7fcae1800700 1 -- 172.21.15.67:0/622822578 wait complete. 2024-03-30T02:36:42.619 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-30T02:36:42.630 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-30T02:36:42.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:42.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.885+0000 7fa2b2e00700 1 -- 172.21.15.67:0/1571955918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac102810 msgr2=0x7fa2ac1090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.885+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/1571955918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac102810 0x7fa2ac1090c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400b0d0 tx=0x7fa2a400b3e0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.885+0000 7fa2b2e00700 1 -- 172.21.15.67:0/1571955918 shutdown_connections 2024-03-30T02:36:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.885+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/1571955918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac102810 0x7fa2ac1090c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.885+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/1571955918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ac100c80 0x7fa2ac1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.885+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/1571955918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2ac101e70 0x7fa2ac1022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.885+0000 7fa2b2e00700 1 -- 172.21.15.67:0/1571955918 >> 172.21.15.67:0/1571955918 conn(0x7fa2ac0fc1f0 msgr2=0x7fa2ac0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.885+0000 7fa2b2e00700 1 -- 172.21.15.67:0/1571955918 shutdown_connections 2024-03-30T02:36:42.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.885+0000 7fa2b2e00700 1 -- 172.21.15.67:0/1571955918 wait complete. 2024-03-30T02:36:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.886+0000 7fa2b2e00700 1 Processor -- start 2024-03-30T02:36:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.886+0000 7fa2b2e00700 1 -- start start 2024-03-30T02:36:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b2e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2ac100c80 0x7fa2ac196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b2e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac101e70 0x7fa2ac197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b2e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ac102810 0x7fa2ac19e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:42.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b2e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac19e810 con 0x7fa2ac101e70 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b2e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac19e980 con 0x7fa2ac102810 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b2e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac19eb40 con 0x7fa2ac100c80 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2ab400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac101e70 0x7fa2ac197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b0e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ac102810 0x7fa2ac19e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2ab400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac101e70 0x7fa2ac197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57436/0 (socket says 172.21.15.67:57436) 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2ab400700 1 -- 172.21.15.67:0/3452170113 learned_addr learned my addr 172.21.15.67:0/3452170113 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2abe00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2ac100c80 0x7fa2ac196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b0e00700 1 -- 172.21.15.67:0/3452170113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2ac100c80 msgr2=0x7fa2ac196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b0e00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2ac100c80 0x7fa2ac196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b0e00700 1 -- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac101e70 msgr2=0x7fa2ac197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b0e00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac101e70 0x7fa2ac197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2b0e00700 1 -- 172.21.15.67:0/3452170113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c009520 con 0x7fa2ac102810 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.887+0000 7fa2ab400700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac101e70 0x7fa2ac197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:42.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.888+0000 7fa2b0e00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ac102810 0x7fa2ac19e240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4007b60 tx=0x7fa2a4008e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.888+0000 7fa2a8c00700 1 -- 172.21.15.67:0/3452170113 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400e040 con 0x7fa2ac102810 2024-03-30T02:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.888+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a4009d00 con 0x7fa2ac102810 2024-03-30T02:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.888+0000 7fa2a8c00700 1 -- 172.21.15.67:0/3452170113 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2a4005070 con 0x7fa2ac102810 2024-03-30T02:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.888+0000 7fa2a8c00700 1 -- 172.21.15.67:0/3452170113 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a401d6d0 con 0x7fa2ac102810 2024-03-30T02:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.888+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac19f060 con 0x7fa2ac102810 2024-03-30T02:36:42.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.889+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac1080b0 con 0x7fa2ac102810 2024-03-30T02:36:42.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.891+0000 7fa2a8c00700 1 -- 172.21.15.67:0/3452170113 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7fa2a40051e0 con 0x7fa2ac102810 2024-03-30T02:36:42.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.892+0000 7fa2a8c00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa2900a1120 0x7fa2900a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:42.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.892+0000 7fa2a8c00700 1 -- 172.21.15.67:0/3452170113 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fa2a40cdff0 con 0x7fa2ac102810 2024-03-30T02:36:42.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.893+0000 7fa2abe00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa2900a1120 0x7fa2900a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:42.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.893+0000 7fa2abe00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa2900a1120 0x7fa2900a35e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0fe590 tx=0x7fa29c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:42.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:42.895+0000 7fa2a8c00700 1 -- 172.21.15.67:0/3452170113 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa2a4097540 con 0x7fa2ac102810 2024-03-30T02:36:42.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:42.584+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:36:42.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:42.584+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:36:42.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:42.584+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:36:42.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:42.584+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:36:42.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:42.584+0000 7feb38e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:36:43.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.155+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2ac067a10 con 0x7fa2ac102810 2024-03-30T02:36:43.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.157+0000 7fa2a8c00700 1 -- 172.21.15.67:0/3452170113 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24479 (secure 0 0 0) 0x7fa2a409ab60 con 0x7fa2ac102810 2024-03-30T02:36:43.158 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:43.159 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:36:28.218474+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa2900a1120 msgr2=0x7fa2900a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa2900a1120 0x7fa2900a35e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0fe590 tx=0x7fa29c009260 comp rx=0 tx=0).stop 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ac102810 msgr2=0x7fa2ac19e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ac102810 0x7fa2ac19e240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4007b60 tx=0x7fa2a4008e40 comp rx=0 tx=0).stop 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 shutdown_connections 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa2900a1120 0x7fa2900a35e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ac102810 0x7fa2ac19e240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2ac100c80 0x7fa2ac196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 --2- 172.21.15.67:0/3452170113 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa2ac101e70 0x7fa2ac197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 >> 172.21.15.67:0/3452170113 conn(0x7fa2ac0fc1f0 msgr2=0x7fa2ac1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:43.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 shutdown_connections 2024-03-30T02:36:43.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.160+0000 7fa2b2e00700 1 -- 172.21.15.67:0/3452170113 wait complete. 2024-03-30T02:36:43.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:36:43.180 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:36:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.382+0000 7f5d04400700 1 -- 172.21.15.67:0/3552199238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc101f70 msgr2=0x7f5cfc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.382+0000 7f5d04400700 1 --2- 172.21.15.67:0/3552199238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc101f70 0x7f5cfc1023d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80099c0 tx=0x7f5ce8009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.382+0000 7f5d04400700 1 -- 172.21.15.67:0/3552199238 shutdown_connections 2024-03-30T02:36:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.382+0000 7f5d04400700 1 --2- 172.21.15.67:0/3552199238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cfc102910 0x7f5cfc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.382+0000 7f5d04400700 1 --2- 172.21.15.67:0/3552199238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cfc100d80 0x7f5cfc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.382+0000 7f5d04400700 1 --2- 172.21.15.67:0/3552199238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc101f70 0x7f5cfc1023d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.382+0000 7f5d04400700 1 -- 172.21.15.67:0/3552199238 >> 172.21.15.67:0/3552199238 conn(0x7f5cfc0fc2f0 msgr2=0x7f5cfc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:43.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.382+0000 7f5d04400700 1 -- 172.21.15.67:0/3552199238 shutdown_connections 2024-03-30T02:36:43.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.382+0000 7f5d04400700 1 -- 172.21.15.67:0/3552199238 wait complete. 2024-03-30T02:36:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.383+0000 7f5d04400700 1 Processor -- start 2024-03-30T02:36:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.383+0000 7f5d04400700 1 -- start start 2024-03-30T02:36:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.383+0000 7f5d04400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc100d80 0x7f5cfc198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.383+0000 7f5d04400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cfc101f70 0x7f5cfc199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.383+0000 7f5d04400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cfc102910 0x7f5cfc1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.383+0000 7f5d04400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc1a0a20 con 0x7f5cfc100d80 2024-03-30T02:36:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.383+0000 7f5d04400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc1a0b60 con 0x7f5cfc102910 2024-03-30T02:36:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.383+0000 7f5d04400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc1a0cf0 con 0x7f5cfc101f70 2024-03-30T02:36:43.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc100d80 0x7f5cfc198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc100d80 0x7f5cfc198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57464/0 (socket says 172.21.15.67:57464) 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01a00700 1 -- 172.21.15.67:0/362003232 learned_addr learned my addr 172.21.15.67:0/362003232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01000700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cfc101f70 0x7f5cfc199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d02400700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cfc102910 0x7f5cfc1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01a00700 1 -- 172.21.15.67:0/362003232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cfc101f70 msgr2=0x7f5cfc199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01a00700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cfc101f70 0x7f5cfc199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01a00700 1 -- 172.21.15.67:0/362003232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cfc102910 msgr2=0x7f5cfc1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01a00700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cfc102910 0x7f5cfc1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01a00700 1 -- 172.21.15.67:0/362003232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce80095f0 con 0x7f5cfc100d80 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.384+0000 7f5d01a00700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc100d80 0x7f5cfc198ef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e970 tx=0x7f5cf000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.385+0000 7f5cfa000700 1 -- 172.21.15.67:0/362003232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0009790 con 0x7f5cfc100d80 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.385+0000 7f5cfa000700 1 -- 172.21.15.67:0/362003232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cf000c7e0 con 0x7f5cfc100d80 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.385+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc1a0fd0 con 0x7f5cfc100d80 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.385+0000 7f5cfa000700 1 -- 172.21.15.67:0/362003232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0003c70 con 0x7f5cfc100d80 2024-03-30T02:36:43.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.385+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc1a1450 con 0x7f5cfc100d80 2024-03-30T02:36:43.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.386+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc067af0 con 0x7f5cfc100d80 2024-03-30T02:36:43.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.388+0000 7f5cfa000700 1 -- 172.21.15.67:0/362003232 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f5cf000cc50 con 0x7f5cfc100d80 2024-03-30T02:36:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.389+0000 7f5cfa000700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5ce40a11a0 0x7f5ce40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.389+0000 7f5cfa000700 1 -- 172.21.15.67:0/362003232 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f5cf0014070 con 0x7f5cfc100d80 2024-03-30T02:36:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.389+0000 7f5d01000700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5ce40a11a0 0x7f5ce40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:43.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.390+0000 7f5d01000700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5ce40a11a0 0x7f5ce40a3660 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8006010 tx=0x7f5ce800b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:43.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.393+0000 7f5cfa000700 1 -- 172.21.15.67:0/362003232 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5cf0096190 con 0x7f5cfc100d80 2024-03-30T02:36:43.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:43 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4085344482' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-30T02:36:43.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:43 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:36:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:43 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4085344482' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-30T02:36:43.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:43 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:36:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:43 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4085344482' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-30T02:36:43.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:43 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:36:43.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.608+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cfc1a17b0 con 0x7f5cfc100d80 2024-03-30T02:36:43.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.609+0000 7f5cfa000700 1 -- 172.21.15.67:0/362003232 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5cf0003dd0 con 0x7f5cfc100d80 2024-03-30T02:36:43.610 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:36:43.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5ce40a11a0 msgr2=0x7f5ce40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5ce40a11a0 0x7f5ce40a3660 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8006010 tx=0x7f5ce800b540 comp rx=0 tx=0).stop 2024-03-30T02:36:43.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc100d80 msgr2=0x7f5cfc198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc100d80 0x7f5cfc198ef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e970 tx=0x7f5cf000ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:43.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 shutdown_connections 2024-03-30T02:36:43.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f5ce40a11a0 0x7f5ce40a3660 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5cfc100d80 0x7f5cfc198ef0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cfc102910 0x7f5cfc1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 --2- 172.21.15.67:0/362003232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cfc101f70 0x7f5cfc199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 >> 172.21.15.67:0/362003232 conn(0x7f5cfc0fc2f0 msgr2=0x7f5cfc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:43.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 shutdown_connections 2024-03-30T02:36:43.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.611+0000 7f5d04400700 1 -- 172.21.15.67:0/362003232 wait complete. 2024-03-30T02:36:43.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:36:43.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.832+0000 7f51c8200700 1 -- 172.21.15.67:0/698774233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51c0102910 msgr2=0x7f51c01091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.832+0000 7f51c8200700 1 --2- 172.21.15.67:0/698774233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51c0102910 0x7f51c01091c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009fc0 tx=0x7f51b800b330 comp rx=0 tx=0).stop 2024-03-30T02:36:43.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.832+0000 7f51c8200700 1 -- 172.21.15.67:0/698774233 shutdown_connections 2024-03-30T02:36:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.832+0000 7f51c8200700 1 --2- 172.21.15.67:0/698774233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51c0102910 0x7f51c01091c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.832+0000 7f51c8200700 1 --2- 172.21.15.67:0/698774233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c0100d80 0x7f51c01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.832+0000 7f51c8200700 1 --2- 172.21.15.67:0/698774233 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c0101f70 0x7f51c01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.832+0000 7f51c8200700 1 -- 172.21.15.67:0/698774233 >> 172.21.15.67:0/698774233 conn(0x7f51c00fc2f0 msgr2=0x7f51c00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.832+0000 7f51c8200700 1 -- 172.21.15.67:0/698774233 shutdown_connections 2024-03-30T02:36:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.832+0000 7f51c8200700 1 -- 172.21.15.67:0/698774233 wait complete. 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.833+0000 7f51c8200700 1 Processor -- start 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.833+0000 7f51c8200700 1 -- start start 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c8200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c0100d80 0x7f51c0196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c8200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51c0101f70 0x7f51c0197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c8200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c0102910 0x7f51c019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c8200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c019e950 con 0x7f51c0100d80 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c8200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c019ea90 con 0x7f51c0101f70 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c8200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c019ec20 con 0x7f51c0102910 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c5800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c0100d80 0x7f51c0196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c5800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c0100d80 0x7f51c0196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57482/0 (socket says 172.21.15.67:57482) 2024-03-30T02:36:43.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c5800700 1 -- 172.21.15.67:0/4153062193 learned_addr learned my addr 172.21.15.67:0/4153062193 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c4e00700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51c0101f70 0x7f51c0197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c6200700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c0102910 0x7f51c019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c5800700 1 -- 172.21.15.67:0/4153062193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c0102910 msgr2=0x7f51c019e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c5800700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c0102910 0x7f51c019e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c5800700 1 -- 172.21.15.67:0/4153062193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51c0101f70 msgr2=0x7f51c0197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c5800700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51c0101f70 0x7f51c0197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c5800700 1 -- 172.21.15.67:0/4153062193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009520 con 0x7f51c0100d80 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.834+0000 7f51c5800700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c0100d80 0x7f51c0196e20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f51b400ea50 tx=0x7f51b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.835+0000 7f51be000700 1 -- 172.21.15.67:0/4153062193 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b400cbd0 con 0x7f51c0100d80 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.835+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac0095f0 con 0x7f51c0100d80 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.835+0000 7f51be000700 1 -- 172.21.15.67:0/4153062193 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b400cd30 con 0x7f51c0100d80 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.835+0000 7f51be000700 1 -- 172.21.15.67:0/4153062193 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4018680 con 0x7f51c0100d80 2024-03-30T02:36:43.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.835+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b8009c40 con 0x7f51c0100d80 2024-03-30T02:36:43.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.836+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c0108130 con 0x7f51c0100d80 2024-03-30T02:36:43.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.838+0000 7f51be000700 1 -- 172.21.15.67:0/4153062193 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) v1 ==== 135865+0+0 (secure 0 0 0) 0x7f51b4010430 con 0x7f51c0100d80 2024-03-30T02:36:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.839+0000 7f51be000700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f51a80a1120 0x7f51a80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.839+0000 7f51be000700 1 -- 172.21.15.67:0/4153062193 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f51b4014070 con 0x7f51c0100d80 2024-03-30T02:36:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.839+0000 7f51c4e00700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f51a80a1120 0x7f51a80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:43.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.840+0000 7f51c4e00700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f51a80a1120 0x7f51a80a35e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51ac005fd0 tx=0x7f51ac009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:43.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:43.843+0000 7f51be000700 1 -- 172.21.15.67:0/4153062193 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f51b4096500 con 0x7f51c0100d80 2024-03-30T02:36:44.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.061+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51c010c820 con 0x7f51c0100d80 2024-03-30T02:36:44.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.062+0000 7f51be000700 1 -- 172.21.15.67:0/4153062193 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51b4099b20 con 0x7f51c0100d80 2024-03-30T02:36:44.063 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f51a80a1120 msgr2=0x7f51a80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f51a80a1120 0x7f51a80a35e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51ac005fd0 tx=0x7f51ac009310 comp rx=0 tx=0).stop 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c0100d80 msgr2=0x7f51c0196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c0100d80 0x7f51c0196e20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f51b400ea50 tx=0x7f51b400ed60 comp rx=0 tx=0).stop 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 shutdown_connections 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f51a80a1120 0x7f51a80a35e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51c0100d80 0x7f51c0196e20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51c0102910 0x7f51c019e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 --2- 172.21.15.67:0/4153062193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51c0101f70 0x7f51c0197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.064+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 >> 172.21.15.67:0/4153062193 conn(0x7f51c00fc2f0 msgr2=0x7f51c0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.065+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 shutdown_connections 2024-03-30T02:36:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.065+0000 7f51c8200700 1 -- 172.21.15.67:0/4153062193 wait complete. 2024-03-30T02:36:44.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:36:44.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.292+0000 7fd327e00700 1 -- 172.21.15.67:0/3471088972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd320101f70 msgr2=0x7fd3201023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.292+0000 7fd327e00700 1 --2- 172.21.15.67:0/3471088972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd320101f70 0x7fd3201023d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009920 tx=0x7fd30c009c30 comp rx=0 tx=0).stop 2024-03-30T02:36:44.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.292+0000 7fd327e00700 1 -- 172.21.15.67:0/3471088972 shutdown_connections 2024-03-30T02:36:44.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.292+0000 7fd327e00700 1 --2- 172.21.15.67:0/3471088972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd320102910 0x7fd3201091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.292+0000 7fd327e00700 1 --2- 172.21.15.67:0/3471088972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd320100d80 0x7fd3201011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.292+0000 7fd327e00700 1 --2- 172.21.15.67:0/3471088972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd320101f70 0x7fd3201023d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.292+0000 7fd327e00700 1 -- 172.21.15.67:0/3471088972 >> 172.21.15.67:0/3471088972 conn(0x7fd3200fc2f0 msgr2=0x7fd3200fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:44.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.292+0000 7fd327e00700 1 -- 172.21.15.67:0/3471088972 shutdown_connections 2024-03-30T02:36:44.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.292+0000 7fd327e00700 1 -- 172.21.15.67:0/3471088972 wait complete. 2024-03-30T02:36:44.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.293+0000 7fd327e00700 1 Processor -- start 2024-03-30T02:36:44.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.293+0000 7fd327e00700 1 -- start start 2024-03-30T02:36:44.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd327e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd320100d80 0x7fd320196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:44.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd327e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd320101f70 0x7fd320197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:44.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd327e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd320102910 0x7fd32019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:44.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd327e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32019e8f0 con 0x7fd320101f70 2024-03-30T02:36:44.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd327e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32019ea30 con 0x7fd320102910 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd327e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32019ebc0 con 0x7fd320100d80 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd324a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd320101f70 0x7fd320197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd325400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd320100d80 0x7fd320196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd325400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd320100d80 0x7fd320196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47026/0 (socket says 172.21.15.67:47026) 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd324a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd320101f70 0x7fd320197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57496/0 (socket says 172.21.15.67:57496) 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd325400700 1 -- 172.21.15.67:0/2074472454 learned_addr learned my addr 172.21.15.67:0/2074472454 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.294+0000 7fd325e00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd320102910 0x7fd32019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd324a00700 1 -- 172.21.15.67:0/2074472454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd320100d80 msgr2=0x7fd320196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd324a00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd320100d80 0x7fd320196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd324a00700 1 -- 172.21.15.67:0/2074472454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd320102910 msgr2=0x7fd32019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd324a00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd320102910 0x7fd32019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd324a00700 1 -- 172.21.15.67:0/2074472454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c009520 con 0x7fd320101f70 2024-03-30T02:36:44.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd324a00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd320101f70 0x7fd320197300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009230 tx=0x7fd30c00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:44.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd31e000700 1 -- 172.21.15.67:0/2074472454 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c01c070 con 0x7fd320101f70 2024-03-30T02:36:44.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32019ee40 con 0x7fd320101f70 2024-03-30T02:36:44.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd31e000700 1 -- 172.21.15.67:0/2074472454 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd30c00bb20 con 0x7fd320101f70 2024-03-30T02:36:44.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd31e000700 1 -- 172.21.15.67:0/2074472454 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0049c0 con 0x7fd320101f70 2024-03-30T02:36:44.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.295+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32019f1b0 con 0x7fd320101f70 2024-03-30T02:36:44.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.296+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd320108240 con 0x7fd320101f70 2024-03-30T02:36:44.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.298+0000 7fd31e000700 1 -- 172.21.15.67:0/2074472454 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fd30c020070 con 0x7fd320101f70 2024-03-30T02:36:44.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.299+0000 7fd31e000700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd3080a11b0 0x7fd3080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:44.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.300+0000 7fd31e000700 1 -- 172.21.15.67:0/2074472454 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fd30c0cd9d0 con 0x7fd320101f70 2024-03-30T02:36:44.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.300+0000 7fd325400700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd3080a11b0 0x7fd3080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:44.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.300+0000 7fd325400700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd3080a11b0 0x7fd3080a3670 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd314005950 tx=0x7fd314009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:44.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.304+0000 7fd31e000700 1 -- 172.21.15.67:0/2074472454 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd30c096cf0 con 0x7fd320101f70 2024-03-30T02:36:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:44 smithi067 ceph-mon[37808]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:44 smithi067 ceph-mon[37808]: pgmap v295: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:36:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:44 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3452170113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:44 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/362003232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:44.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:44 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4153062193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:44 smithi170 ceph-mon[78798]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:44 smithi170 ceph-mon[78798]: pgmap v295: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:36:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:44 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3452170113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:44 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/362003232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:44.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:44 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4153062193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:44.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:44 smithi082 ceph-mon[79176]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:36:44.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:44 smithi082 ceph-mon[79176]: pgmap v295: 129 pgs: 129 active+clean; 646 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-30T02:36:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:44 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3452170113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:44 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/362003232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:44.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:44 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4153062193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:44.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.521+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd32010c5e0 con 0x7fd320101f70 2024-03-30T02:36:44.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.521+0000 7fd31e000700 1 -- 172.21.15.67:0/2074472454 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd30c09a310 con 0x7fd320101f70 2024-03-30T02:36:44.522 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd3080a11b0 msgr2=0x7fd3080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd3080a11b0 0x7fd3080a3670 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd314005950 tx=0x7fd314009260 comp rx=0 tx=0).stop 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd320101f70 msgr2=0x7fd320197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd320101f70 0x7fd320197300 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd30c009230 tx=0x7fd30c00bef0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 shutdown_connections 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd3080a11b0 0x7fd3080a3670 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd320102910 0x7fd32019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd320100d80 0x7fd320196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 --2- 172.21.15.67:0/2074472454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd320101f70 0x7fd320197300 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 >> 172.21.15.67:0/2074472454 conn(0x7fd3200fc2f0 msgr2=0x7fd320105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 shutdown_connections 2024-03-30T02:36:44.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.524+0000 7fd327e00700 1 -- 172.21.15.67:0/2074472454 wait complete. 2024-03-30T02:36:44.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:36:44.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.745+0000 7fd937c00700 1 -- 172.21.15.67:0/1047891742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 msgr2=0x7fd930073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.745+0000 7fd937c00700 1 --2- 172.21.15.67:0/1047891742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 0x7fd930073240 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd924009990 tx=0x7fd924009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.746+0000 7fd937c00700 1 -- 172.21.15.67:0/1047891742 shutdown_connections 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.746+0000 7fd937c00700 1 --2- 172.21.15.67:0/1047891742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 0x7fd930073240 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.746+0000 7fd937c00700 1 --2- 172.21.15.67:0/1047891742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd93010d350 0x7fd93010f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.746+0000 7fd937c00700 1 --2- 172.21.15.67:0/1047891742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd930073810 0x7fd930073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.746+0000 7fd937c00700 1 -- 172.21.15.67:0/1047891742 >> 172.21.15.67:0/1047891742 conn(0x7fd9300fc2f0 msgr2=0x7fd9300fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.746+0000 7fd937c00700 1 -- 172.21.15.67:0/1047891742 shutdown_connections 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.746+0000 7fd937c00700 1 -- 172.21.15.67:0/1047891742 wait complete. 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.747+0000 7fd937c00700 1 Processor -- start 2024-03-30T02:36:44.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.747+0000 7fd937c00700 1 -- start start 2024-03-30T02:36:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.747+0000 7fd937c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd930073810 0x7fd93019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:44.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.747+0000 7fd937c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 0x7fd93019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.747+0000 7fd937c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd93010d350 0x7fd9301a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.747+0000 7fd937c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9301a4e20 con 0x7fd930074df0 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.747+0000 7fd937c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9301a4f60 con 0x7fd93010d350 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.747+0000 7fd937c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9301a50f0 con 0x7fd930073810 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 0x7fd93019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 0x7fd93019d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57510/0 (socket says 172.21.15.67:57510) 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92fe00700 1 -- 172.21.15.67:0/3829760947 learned_addr learned my addr 172.21.15.67:0/3829760947 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd935200700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd930073810 0x7fd93019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd935c00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd93010d350 0x7fd9301a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92fe00700 1 -- 172.21.15.67:0/3829760947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd930073810 msgr2=0x7fd93019d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92fe00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd930073810 0x7fd93019d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92fe00700 1 -- 172.21.15.67:0/3829760947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd93010d350 msgr2=0x7fd9301a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92fe00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd93010d350 0x7fd9301a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92fe00700 1 -- 172.21.15.67:0/3829760947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9240095f0 con 0x7fd930074df0 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92fe00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 0x7fd93019d880 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00e970 tx=0x7fd91c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:44.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.748+0000 7fd92d600700 1 -- 172.21.15.67:0/3829760947 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c009790 con 0x7fd930074df0 2024-03-30T02:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.749+0000 7fd92d600700 1 -- 172.21.15.67:0/3829760947 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd91c00c7e0 con 0x7fd930074df0 2024-03-30T02:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.749+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9301a53d0 con 0x7fd930074df0 2024-03-30T02:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.749+0000 7fd92d600700 1 -- 172.21.15.67:0/3829760947 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c003c70 con 0x7fd930074df0 2024-03-30T02:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.749+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9301a58e0 con 0x7fd930074df0 2024-03-30T02:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.750+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9301035b0 con 0x7fd930074df0 2024-03-30T02:36:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.752+0000 7fd92d600700 1 -- 172.21.15.67:0/3829760947 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fd91c00cc50 con 0x7fd930074df0 2024-03-30T02:36:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.753+0000 7fd92d600700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd9180a11b0 0x7fd9180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.753+0000 7fd92d600700 1 -- 172.21.15.67:0/3829760947 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fd91c014070 con 0x7fd930074df0 2024-03-30T02:36:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.753+0000 7fd935200700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd9180a11b0 0x7fd9180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:44.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.754+0000 7fd935200700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd9180a11b0 0x7fd9180a3670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd92400b2e0 tx=0x7fd924005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:44.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.757+0000 7fd92d600700 1 -- 172.21.15.67:0/3829760947 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd91c096280 con 0x7fd930074df0 2024-03-30T02:36:44.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.975+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd930112cd0 con 0x7fd930074df0 2024-03-30T02:36:44.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.976+0000 7fd92d600700 1 -- 172.21.15.67:0/3829760947 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd91c003dd0 con 0x7fd930074df0 2024-03-30T02:36:44.977 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.978+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd9180a11b0 msgr2=0x7fd9180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.978+0000 7fd937c00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd9180a11b0 0x7fd9180a3670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd92400b2e0 tx=0x7fd924005fb0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.978+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 msgr2=0x7fd93019d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.978+0000 7fd937c00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 0x7fd93019d880 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00e970 tx=0x7fd91c00ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.979+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 shutdown_connections 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.979+0000 7fd937c00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fd9180a11b0 0x7fd9180a3670 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.979+0000 7fd937c00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd93010d350 0x7fd9301a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.979+0000 7fd937c00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd930073810 0x7fd93019d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.979+0000 7fd937c00700 1 --2- 172.21.15.67:0/3829760947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd930074df0 0x7fd93019d880 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.979+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 >> 172.21.15.67:0/3829760947 conn(0x7fd9300fc2f0 msgr2=0x7fd9301006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.979+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 shutdown_connections 2024-03-30T02:36:44.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:44.979+0000 7fd937c00700 1 -- 172.21.15.67:0/3829760947 wait complete. 2024-03-30T02:36:44.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:45.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.197+0000 7f85be600700 1 -- 172.21.15.67:0/4260777541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b81016e0 msgr2=0x7f85b8103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.197+0000 7f85be600700 1 --2- 172.21.15.67:0/4260777541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b81016e0 0x7f85b8103ad0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009990 tx=0x7f85a8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.198+0000 7f85be600700 1 -- 172.21.15.67:0/4260777541 shutdown_connections 2024-03-30T02:36:45.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.198+0000 7f85be600700 1 --2- 172.21.15.67:0/4260777541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b81016e0 0x7f85b8103ad0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.198+0000 7f85be600700 1 --2- 172.21.15.67:0/4260777541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b8106940 0x7f85b8108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.198+0000 7f85be600700 1 --2- 172.21.15.67:0/4260777541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b8104010 0x7f85b8106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.198+0000 7f85be600700 1 -- 172.21.15.67:0/4260777541 >> 172.21.15.67:0/4260777541 conn(0x7f85b80fb0d0 msgr2=0x7f85b80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:45.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.198+0000 7f85be600700 1 -- 172.21.15.67:0/4260777541 shutdown_connections 2024-03-30T02:36:45.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.198+0000 7f85be600700 1 -- 172.21.15.67:0/4260777541 wait complete. 2024-03-30T02:36:45.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.199+0000 7f85be600700 1 Processor -- start 2024-03-30T02:36:45.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.199+0000 7f85be600700 1 -- start start 2024-03-30T02:36:45.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.199+0000 7f85be600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b81016e0 0x7f85b8101000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:45.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.199+0000 7f85be600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b8104010 0x7f85b80ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:45.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.199+0000 7f85be600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8106940 0x7f85b80ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.199+0000 7f85be600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8100370 con 0x7f85b8106940 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.199+0000 7f85be600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81004e0 con 0x7f85b81016e0 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.199+0000 7f85be600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81a0bf0 con 0x7f85b8104010 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.199+0000 7f85b7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8106940 0x7f85b80ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b81016e0 0x7f85b8101000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8106940 0x7f85b80ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57540/0 (socket says 172.21.15.67:57540) 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7e00700 1 -- 172.21.15.67:0/718519183 learned_addr learned my addr 172.21.15.67:0/718519183 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b6a00700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b8104010 0x7f85b80ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7400700 1 -- 172.21.15.67:0/718519183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b8104010 msgr2=0x7f85b80ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7400700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b8104010 0x7f85b80ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7400700 1 -- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8106940 msgr2=0x7f85b80ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7400700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8106940 0x7f85b80ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7400700 1 -- 172.21.15.67:0/718519183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a80095f0 con 0x7f85b81016e0 2024-03-30T02:36:45.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7e00700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8106940 0x7f85b80ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:45.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.200+0000 7f85b7400700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b81016e0 0x7f85b8101000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:45.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.201+0000 7f85afe00700 1 -- 172.21.15.67:0/718519183 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a801d070 con 0x7f85b81016e0 2024-03-30T02:36:45.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.201+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b81a0e20 con 0x7f85b81016e0 2024-03-30T02:36:45.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.201+0000 7f85afe00700 1 -- 172.21.15.67:0/718519183 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a8021410 con 0x7f85b81016e0 2024-03-30T02:36:45.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.201+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b81a1190 con 0x7f85b81016e0 2024-03-30T02:36:45.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.201+0000 7f85afe00700 1 -- 172.21.15.67:0/718519183 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8021ea0 con 0x7f85b81016e0 2024-03-30T02:36:45.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.202+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b8067a80 con 0x7f85b81016e0 2024-03-30T02:36:45.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.203+0000 7f85afe00700 1 -- 172.21.15.67:0/718519183 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f85a8021580 con 0x7f85b81016e0 2024-03-30T02:36:45.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.205+0000 7f85afe00700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f859c0a11b0 0x7f859c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:45.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.205+0000 7f85afe00700 1 -- 172.21.15.67:0/718519183 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f85a80ce010 con 0x7f85b81016e0 2024-03-30T02:36:45.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.205+0000 7f85b6a00700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f859c0a11b0 0x7f859c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:45.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.205+0000 7f85b6a00700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f859c0a11b0 0x7f859c0a3670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009c60 tx=0x7f85a4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:45.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.209+0000 7f85afe00700 1 -- 172.21.15.67:0/718519183 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f85b8067a80 con 0x7f85b81016e0 2024-03-30T02:36:45.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:45 smithi067 ceph-mon[37808]: mgrmap e19: x(active, since 7m), standbys: z, y 2024-03-30T02:36:45.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:45 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2074472454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:45.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:45 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3829760947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:45.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:45 smithi170 ceph-mon[78798]: mgrmap e19: x(active, since 7m), standbys: z, y 2024-03-30T02:36:45.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:45 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2074472454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:45.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:45 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3829760947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:45.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.464+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85b810c440 con 0x7f85b81016e0 2024-03-30T02:36:45.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.466+0000 7f85afe00700 1 -- 172.21.15.67:0/718519183 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24479 (secure 0 0 0) 0x7f85a8097330 con 0x7f85b81016e0 2024-03-30T02:36:45.468 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:45.468 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:36:28.218474+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:45.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f859c0a11b0 msgr2=0x7f859c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f859c0a11b0 0x7f859c0a3670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009c60 tx=0x7f85a4009260 comp rx=0 tx=0).stop 2024-03-30T02:36:45.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b81016e0 msgr2=0x7f85b8101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b81016e0 0x7f85b8101000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a8004730 comp rx=0 tx=0).stop 2024-03-30T02:36:45.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 shutdown_connections 2024-03-30T02:36:45.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f859c0a11b0 0x7f859c0a3670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8106940 0x7f85b80ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85b81016e0 0x7f85b8101000 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 --2- 172.21.15.67:0/718519183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85b8104010 0x7f85b80ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 >> 172.21.15.67:0/718519183 conn(0x7f85b80fb0d0 msgr2=0x7f85b81035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:45.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 shutdown_connections 2024-03-30T02:36:45.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.469+0000 7f85be600700 1 -- 172.21.15.67:0/718519183 wait complete. 2024-03-30T02:36:45.490 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:36:45.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:36:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:45 smithi082 ceph-mon[79176]: mgrmap e19: x(active, since 7m), standbys: z, y 2024-03-30T02:36:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:45 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2074472454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:45.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:45 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3829760947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:45.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.692+0000 7f87c1400700 1 -- 172.21.15.67:0/4051303178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc100d80 msgr2=0x7f87bc1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.692+0000 7f87c1400700 1 --2- 172.21.15.67:0/4051303178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc100d80 0x7f87bc1011a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0099c0 tx=0x7f87ac009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.692+0000 7f87c1400700 1 -- 172.21.15.67:0/4051303178 shutdown_connections 2024-03-30T02:36:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.692+0000 7f87c1400700 1 --2- 172.21.15.67:0/4051303178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc100d80 0x7f87bc1011a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.692+0000 7f87c1400700 1 --2- 172.21.15.67:0/4051303178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc102910 0x7f87bc1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.692+0000 7f87c1400700 1 --2- 172.21.15.67:0/4051303178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87bc101f70 0x7f87bc1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.692+0000 7f87c1400700 1 -- 172.21.15.67:0/4051303178 >> 172.21.15.67:0/4051303178 conn(0x7f87bc0fc2f0 msgr2=0x7f87bc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.692+0000 7f87c1400700 1 -- 172.21.15.67:0/4051303178 shutdown_connections 2024-03-30T02:36:45.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.692+0000 7f87c1400700 1 -- 172.21.15.67:0/4051303178 wait complete. 2024-03-30T02:36:45.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.693+0000 7f87c1400700 1 Processor -- start 2024-03-30T02:36:45.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.693+0000 7f87c1400700 1 -- start start 2024-03-30T02:36:45.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87c1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87bc100d80 0x7f87bc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:45.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87c1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc101f70 0x7f87bc0731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87c1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc102910 0x7f87bc0736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87c1400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc073e80 con 0x7f87bc101f70 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87c1400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc073ff0 con 0x7f87bc102910 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87c1400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc1a0dc0 con 0x7f87bc100d80 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87b9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc101f70 0x7f87bc0731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87b9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc101f70 0x7f87bc0731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57546/0 (socket says 172.21.15.67:57546) 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87b9e00700 1 -- 172.21.15.67:0/1312630935 learned_addr learned my addr 172.21.15.67:0/1312630935 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87bb200700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc102910 0x7f87bc0736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87ba800700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87bc100d80 0x7f87bc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87bb200700 1 -- 172.21.15.67:0/1312630935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87bc100d80 msgr2=0x7f87bc074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87bb200700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87bc100d80 0x7f87bc074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87bb200700 1 -- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc101f70 msgr2=0x7f87bc0731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87bb200700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc101f70 0x7f87bc0731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87bb200700 1 -- 172.21.15.67:0/1312630935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac0095f0 con 0x7f87bc102910 2024-03-30T02:36:45.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.694+0000 7f87b9e00700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc101f70 0x7f87bc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:45.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.695+0000 7f87bb200700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc102910 0x7f87bc0736e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87b400efc0 tx=0x7f87b400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:45.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.695+0000 7f87b3400700 1 -- 172.21.15.67:0/1312630935 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4018070 con 0x7f87bc102910 2024-03-30T02:36:45.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.695+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc1a0ff0 con 0x7f87bc102910 2024-03-30T02:36:45.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.695+0000 7f87b3400700 1 -- 172.21.15.67:0/1312630935 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87b400f040 con 0x7f87bc102910 2024-03-30T02:36:45.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.695+0000 7f87b3400700 1 -- 172.21.15.67:0/1312630935 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400cce0 con 0x7f87bc102910 2024-03-30T02:36:45.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.695+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc1a1470 con 0x7f87bc102910 2024-03-30T02:36:45.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.696+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc108250 con 0x7f87bc102910 2024-03-30T02:36:45.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.698+0000 7f87b3400700 1 -- 172.21.15.67:0/1312630935 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f87b4003a00 con 0x7f87bc102910 2024-03-30T02:36:45.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.699+0000 7f87b3400700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87a00a11b0 0x7f87a00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:45.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.699+0000 7f87b3400700 1 -- 172.21.15.67:0/1312630935 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f87b40cd2a0 con 0x7f87bc102910 2024-03-30T02:36:45.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.700+0000 7f87ba800700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87a00a11b0 0x7f87a00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:45.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.700+0000 7f87ba800700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87a00a11b0 0x7f87a00a3670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f87ac003730 tx=0x7f87ac000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:45.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.701+0000 7f87b3400700 1 -- 172.21.15.67:0/1312630935 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f87b40965c0 con 0x7f87bc102910 2024-03-30T02:36:45.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.922+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87bc10c5e0 con 0x7f87bc102910 2024-03-30T02:36:45.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.923+0000 7f87b3400700 1 -- 172.21.15.67:0/1312630935 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87b4099be0 con 0x7f87bc102910 2024-03-30T02:36:45.924 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:36:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87a00a11b0 msgr2=0x7f87a00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87a00a11b0 0x7f87a00a3670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f87ac003730 tx=0x7f87ac000bc0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc102910 msgr2=0x7f87bc0736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc102910 0x7f87bc0736e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f87b400efc0 tx=0x7f87b400c3a0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 shutdown_connections 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f87a00a11b0 0x7f87a00a3670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc102910 0x7f87bc0736e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87bc100d80 0x7f87bc074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 --2- 172.21.15.67:0/1312630935 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f87bc101f70 0x7f87bc0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 >> 172.21.15.67:0/1312630935 conn(0x7f87bc0fc2f0 msgr2=0x7f87bc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 shutdown_connections 2024-03-30T02:36:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:45.925+0000 7f87c1400700 1 -- 172.21.15.67:0/1312630935 wait complete. 2024-03-30T02:36:45.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:36:46.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.147+0000 7fbfdd000700 1 -- 172.21.15.67:0/4274965261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd8100d80 msgr2=0x7fbfd81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.147+0000 7fbfdd000700 1 --2- 172.21.15.67:0/4274965261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd8100d80 0x7fbfd81011a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009990 tx=0x7fbfc8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.147+0000 7fbfdd000700 1 -- 172.21.15.67:0/4274965261 shutdown_connections 2024-03-30T02:36:46.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.147+0000 7fbfdd000700 1 --2- 172.21.15.67:0/4274965261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd8102910 0x7fbfd81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.147+0000 7fbfdd000700 1 --2- 172.21.15.67:0/4274965261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd8100d80 0x7fbfd81011a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.147+0000 7fbfdd000700 1 --2- 172.21.15.67:0/4274965261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd8101f70 0x7fbfd81023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.147+0000 7fbfdd000700 1 -- 172.21.15.67:0/4274965261 >> 172.21.15.67:0/4274965261 conn(0x7fbfd80fc2f0 msgr2=0x7fbfd80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:46.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.147+0000 7fbfdd000700 1 -- 172.21.15.67:0/4274965261 shutdown_connections 2024-03-30T02:36:46.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.147+0000 7fbfdd000700 1 -- 172.21.15.67:0/4274965261 wait complete. 2024-03-30T02:36:46.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.148+0000 7fbfdd000700 1 Processor -- start 2024-03-30T02:36:46.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfdd000700 1 -- start start 2024-03-30T02:36:46.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfdd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd8100d80 0x7fbfd8196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:46.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfdd000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd8101f70 0x7fbfd8197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:46.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfdd000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd8102910 0x7fbfd819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:46.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfdd000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd819e880 con 0x7fbfd8102910 2024-03-30T02:36:46.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfdd000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd819e9f0 con 0x7fbfd8100d80 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfdd000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd819ebb0 con 0x7fbfd8101f70 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfd5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd8100d80 0x7fbfd8196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfd6800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd8102910 0x7fbfd819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfd5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd8100d80 0x7fbfd8196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:44998/0 (socket says 172.21.15.67:44998) 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfd5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd8101f70 0x7fbfd8197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfd5e00700 1 -- 172.21.15.67:0/164066635 learned_addr learned my addr 172.21.15.67:0/164066635 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.149+0000 7fbfd6800700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd8102910 0x7fbfd819e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56584/0 (socket says 172.21.15.67:56584) 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfd5e00700 1 -- 172.21.15.67:0/164066635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd8101f70 msgr2=0x7fbfd8197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfd5e00700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd8101f70 0x7fbfd8197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfd5e00700 1 -- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd8102910 msgr2=0x7fbfd819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfd5e00700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd8102910 0x7fbfd819e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfd5e00700 1 -- 172.21.15.67:0/164066635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80095f0 con 0x7fbfd8100d80 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfd5e00700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd8100d80 0x7fbfd8196de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8000c00 tx=0x7fbfc8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfcea00700 1 -- 172.21.15.67:0/164066635 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc801d070 con 0x7fbfd8100d80 2024-03-30T02:36:46.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfcea00700 1 -- 172.21.15.67:0/164066635 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc8021410 con 0x7fbfd8100d80 2024-03-30T02:36:46.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd8072fe0 con 0x7fbfd8100d80 2024-03-30T02:36:46.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.150+0000 7fbfcea00700 1 -- 172.21.15.67:0/164066635 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8017650 con 0x7fbfd8100d80 2024-03-30T02:36:46.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.151+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd8073490 con 0x7fbfd8100d80 2024-03-30T02:36:46.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.152+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd8108270 con 0x7fbfd8100d80 2024-03-30T02:36:46.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.153+0000 7fbfcea00700 1 -- 172.21.15.67:0/164066635 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fbfc8021580 con 0x7fbfd8100d80 2024-03-30T02:36:46.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.155+0000 7fbfcea00700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfbc0a11b0 0x7fbfbc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:46.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.155+0000 7fbfcea00700 1 -- 172.21.15.67:0/164066635 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fbfc80ce0f0 con 0x7fbfd8100d80 2024-03-30T02:36:46.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.155+0000 7fbfd5400700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfbc0a11b0 0x7fbfbc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:46.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.155+0000 7fbfd5400700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfbc0a11b0 0x7fbfbc0a3670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009c60 tx=0x7fbfc4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:46.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.157+0000 7fbfcea00700 1 -- 172.21.15.67:0/164066635 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbfc8097410 con 0x7fbfd8100d80 2024-03-30T02:36:46.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.376+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbfd810c5e0 con 0x7fbfd8100d80 2024-03-30T02:36:46.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.377+0000 7fbfcea00700 1 -- 172.21.15.67:0/164066635 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbfc8005740 con 0x7fbfd8100d80 2024-03-30T02:36:46.378 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfbc0a11b0 msgr2=0x7fbfbc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfbc0a11b0 0x7fbfbc0a3670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009c60 tx=0x7fbfc4009260 comp rx=0 tx=0).stop 2024-03-30T02:36:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd8100d80 msgr2=0x7fbfd8196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd8100d80 0x7fbfd8196de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8000c00 tx=0x7fbfc8009580 comp rx=0 tx=0).stop 2024-03-30T02:36:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 shutdown_connections 2024-03-30T02:36:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fbfbc0a11b0 0x7fbfbc0a3670 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfd8102910 0x7fbfd819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfd8100d80 0x7fbfd8196de0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 --2- 172.21.15.67:0/164066635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd8101f70 0x7fbfd8197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 >> 172.21.15.67:0/164066635 conn(0x7fbfd80fc2f0 msgr2=0x7fbfd81053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 shutdown_connections 2024-03-30T02:36:46.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.379+0000 7fbfdd000700 1 -- 172.21.15.67:0/164066635 wait complete. 2024-03-30T02:36:46.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:46 smithi067 ceph-mon[37808]: pgmap v296: 129 pgs: 129 active+clean; 651 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-30T02:36:46.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:46 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/718519183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:46.396 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:46 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1312630935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:46.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:36:46.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:46 smithi170 ceph-mon[78798]: pgmap v296: 129 pgs: 129 active+clean; 651 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-30T02:36:46.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:46 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/718519183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:46.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:46 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1312630935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:46 smithi082 ceph-mon[79176]: pgmap v296: 129 pgs: 129 active+clean; 651 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2024-03-30T02:36:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:46 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/718519183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:46.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:46 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1312630935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.599+0000 7f11ac200700 1 -- 172.21.15.67:0/472546860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4100d80 msgr2=0x7f11a41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.599+0000 7f11ac200700 1 --2- 172.21.15.67:0/472546860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4100d80 0x7f11a41011a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f11980099c0 tx=0x7f1198009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.599+0000 7f11ac200700 1 -- 172.21.15.67:0/472546860 shutdown_connections 2024-03-30T02:36:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.599+0000 7f11ac200700 1 --2- 172.21.15.67:0/472546860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4100d80 0x7f11a41011a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.599+0000 7f11ac200700 1 --2- 172.21.15.67:0/472546860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a4102910 0x7f11a41091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.599+0000 7f11ac200700 1 --2- 172.21.15.67:0/472546860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a4101f70 0x7f11a41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.599+0000 7f11ac200700 1 -- 172.21.15.67:0/472546860 >> 172.21.15.67:0/472546860 conn(0x7f11a40fc2f0 msgr2=0x7f11a40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.599+0000 7f11ac200700 1 -- 172.21.15.67:0/472546860 shutdown_connections 2024-03-30T02:36:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.599+0000 7f11ac200700 1 -- 172.21.15.67:0/472546860 wait complete. 2024-03-30T02:36:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.600+0000 7f11ac200700 1 Processor -- start 2024-03-30T02:36:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.600+0000 7f11ac200700 1 -- start start 2024-03-30T02:36:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11ac200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a4100d80 0x7f11a4074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:46.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11ac200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a4101f70 0x7f11a40731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11ac200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4102910 0x7f11a40736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11ac200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4073de0 con 0x7f11a4102910 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11ac200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4073f50 con 0x7f11a4101f70 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11ac200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a41a0dc0 con 0x7f11a4100d80 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11aa200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4102910 0x7f11a40736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11aa200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4102910 0x7f11a40736e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56596/0 (socket says 172.21.15.67:56596) 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11aa200700 1 -- 172.21.15.67:0/1865990045 learned_addr learned my addr 172.21.15.67:0/1865990045 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11a9800700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a4100d80 0x7f11a4074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.601+0000 7f11a8e00700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a4101f70 0x7f11a40731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11a9800700 1 -- 172.21.15.67:0/1865990045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a4101f70 msgr2=0x7f11a40731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11a9800700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a4101f70 0x7f11a40731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11a9800700 1 -- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4102910 msgr2=0x7f11a40736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11a9800700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4102910 0x7f11a40736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11a9800700 1 -- 172.21.15.67:0/1865990045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11980095f0 con 0x7f11a4100d80 2024-03-30T02:36:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11aa200700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4102910 0x7f11a40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11a9800700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a4100d80 0x7f11a4074ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11980098e0 tx=0x7f119800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11a2000700 1 -- 172.21.15.67:0/1865990045 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119801d070 con 0x7f11a4100d80 2024-03-30T02:36:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a41a0fc0 con 0x7f11a4100d80 2024-03-30T02:36:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11a2000700 1 -- 172.21.15.67:0/1865990045 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f119800bda0 con 0x7f11a4100d80 2024-03-30T02:36:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11a2000700 1 -- 172.21.15.67:0/1865990045 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198004a90 con 0x7f11a4100d80 2024-03-30T02:36:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.602+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a41a1330 con 0x7f11a4100d80 2024-03-30T02:36:46.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.603+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a4108250 con 0x7f11a4100d80 2024-03-30T02:36:46.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.605+0000 7f11a2000700 1 -- 172.21.15.67:0/1865990045 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1198004c30 con 0x7f11a4100d80 2024-03-30T02:36:46.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.606+0000 7f11a2000700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f118c0a11b0 0x7f118c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:46.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.607+0000 7f11a2000700 1 -- 172.21.15.67:0/1865990045 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f11980cdb90 con 0x7f11a4100d80 2024-03-30T02:36:46.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.607+0000 7f11a8e00700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f118c0a11b0 0x7f118c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:46.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.607+0000 7f11a8e00700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f118c0a11b0 0x7f118c0a3670 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1190007c60 tx=0x7f11900073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:46.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.609+0000 7f11a2000700 1 -- 172.21.15.67:0/1865990045 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1198096fd0 con 0x7f11a4100d80 2024-03-30T02:36:46.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.829+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11a410c5e0 con 0x7f11a4100d80 2024-03-30T02:36:46.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.830+0000 7f11a2000700 1 -- 172.21.15.67:0/1865990045 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1198026020 con 0x7f11a4100d80 2024-03-30T02:36:46.831 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:36:46.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f118c0a11b0 msgr2=0x7f118c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f118c0a11b0 0x7f118c0a3670 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1190007c60 tx=0x7f11900073d0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a4100d80 msgr2=0x7f11a4074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a4100d80 0x7f11a4074ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11980098e0 tx=0x7f119800fa00 comp rx=0 tx=0).stop 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 shutdown_connections 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f118c0a11b0 0x7f118c0a3670 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f11a4102910 0x7f11a40736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a4100d80 0x7f11a4074ac0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 --2- 172.21.15.67:0/1865990045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a4101f70 0x7f11a40731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 >> 172.21.15.67:0/1865990045 conn(0x7f11a40fc2f0 msgr2=0x7f11a4105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 shutdown_connections 2024-03-30T02:36:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:46.832+0000 7f11ac200700 1 -- 172.21.15.67:0/1865990045 wait complete. 2024-03-30T02:36:46.852 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:36:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.054+0000 7fe11c600700 1 -- 172.21.15.67:0/3681264438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 msgr2=0x7fe1141023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.054+0000 7fe11c600700 1 --2- 172.21.15.67:0/3681264438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 0x7fe1141023d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe1000099c0 tx=0x7fe100009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.054+0000 7fe11c600700 1 -- 172.21.15.67:0/3681264438 shutdown_connections 2024-03-30T02:36:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.054+0000 7fe11c600700 1 --2- 172.21.15.67:0/3681264438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe114102910 0x7fe1141091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.054+0000 7fe11c600700 1 --2- 172.21.15.67:0/3681264438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe114100d80 0x7fe1141011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.054+0000 7fe11c600700 1 --2- 172.21.15.67:0/3681264438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 0x7fe1141023d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.054+0000 7fe11c600700 1 -- 172.21.15.67:0/3681264438 >> 172.21.15.67:0/3681264438 conn(0x7fe1140fc2f0 msgr2=0x7fe1140fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.054+0000 7fe11c600700 1 -- 172.21.15.67:0/3681264438 shutdown_connections 2024-03-30T02:36:47.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.054+0000 7fe11c600700 1 -- 172.21.15.67:0/3681264438 wait complete. 2024-03-30T02:36:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.055+0000 7fe11c600700 1 Processor -- start 2024-03-30T02:36:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe11c600700 1 -- start start 2024-03-30T02:36:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe11c600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe114100d80 0x7fe114198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe11c600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 0x7fe114199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe11c600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe114102910 0x7fe1141a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe11c600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1141a0a30 con 0x7fe114101f70 2024-03-30T02:36:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe11c600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1141a0b70 con 0x7fe114102910 2024-03-30T02:36:47.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe11c600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1141a0d00 con 0x7fe114100d80 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe119200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 0x7fe114199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe119200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 0x7fe114199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56608/0 (socket says 172.21.15.67:56608) 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe119200700 1 -- 172.21.15.67:0/2070338989 learned_addr learned my addr 172.21.15.67:0/2070338989 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe119c00700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe114100d80 0x7fe114198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.056+0000 7fe11a600700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe114102910 0x7fe1141a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe119200700 1 -- 172.21.15.67:0/2070338989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe114100d80 msgr2=0x7fe114198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe119200700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe114100d80 0x7fe114198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe119200700 1 -- 172.21.15.67:0/2070338989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe114102910 msgr2=0x7fe1141a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe119200700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe114102910 0x7fe1141a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe119200700 1 -- 172.21.15.67:0/2070338989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000095f0 con 0x7fe114101f70 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe119c00700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe114100d80 0x7fe114198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe119200700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 0x7fe114199440 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe100006010 tx=0x7fe10000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe112000700 1 -- 172.21.15.67:0/2070338989 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10001d070 con 0x7fe114101f70 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe112000700 1 -- 172.21.15.67:0/2070338989 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe100005030 con 0x7fe114101f70 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1141a0f80 con 0x7fe114101f70 2024-03-30T02:36:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe112000700 1 -- 172.21.15.67:0/2070338989 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100004b10 con 0x7fe114101f70 2024-03-30T02:36:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.057+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1141a1340 con 0x7fe114101f70 2024-03-30T02:36:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.059+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe114067af0 con 0x7fe114101f70 2024-03-30T02:36:47.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.060+0000 7fe112000700 1 -- 172.21.15.67:0/2070338989 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fe1000051a0 con 0x7fe114101f70 2024-03-30T02:36:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.061+0000 7fe112000700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe0fc0a11b0 0x7fe0fc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.062+0000 7fe112000700 1 -- 172.21.15.67:0/2070338989 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fe1000cda70 con 0x7fe114101f70 2024-03-30T02:36:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.062+0000 7fe119c00700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe0fc0a11b0 0x7fe0fc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.062+0000 7fe119c00700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe0fc0a11b0 0x7fe0fc0a3670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe114101dd0 tx=0x7fe108009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:47.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.066+0000 7fe112000700 1 -- 172.21.15.67:0/2070338989 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe100096d90 con 0x7fe114101f70 2024-03-30T02:36:47.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.285+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe1141a15d0 con 0x7fe114101f70 2024-03-30T02:36:47.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.286+0000 7fe112000700 1 -- 172.21.15.67:0/2070338989 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe10009a3b0 con 0x7fe114101f70 2024-03-30T02:36:47.287 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.288+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe0fc0a11b0 msgr2=0x7fe0fc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe0fc0a11b0 0x7fe0fc0a3670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe114101dd0 tx=0x7fe108009260 comp rx=0 tx=0).stop 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 msgr2=0x7fe114199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 0x7fe114199440 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe100006010 tx=0x7fe10000f690 comp rx=0 tx=0).stop 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 shutdown_connections 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fe0fc0a11b0 0x7fe0fc0a3670 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe114102910 0x7fe1141a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe114100d80 0x7fe114198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 --2- 172.21.15.67:0/2070338989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe114101f70 0x7fe114199440 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 >> 172.21.15.67:0/2070338989 conn(0x7fe1140fc2f0 msgr2=0x7fe114072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 shutdown_connections 2024-03-30T02:36:47.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.289+0000 7fe11c600700 1 -- 172.21.15.67:0/2070338989 wait complete. 2024-03-30T02:36:47.308 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-30T02:36:47.308 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:36:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:47 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/164066635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:47 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1865990045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:47.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:47 smithi082 ceph-mon[79176]: pgmap v297: 129 pgs: 129 active+clean; 651 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-03-30T02:36:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.507+0000 7f4109000700 1 -- 172.21.15.67:0/1048953297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 msgr2=0x7f41041010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.507+0000 7f4109000700 1 --2- 172.21.15.67:0/1048953297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 0x7f41041010a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f40f40098f0 tx=0x7f40f4009c00 comp rx=0 tx=0).stop 2024-03-30T02:36:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.507+0000 7f4109000700 1 -- 172.21.15.67:0/1048953297 shutdown_connections 2024-03-30T02:36:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.507+0000 7f4109000700 1 --2- 172.21.15.67:0/1048953297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4104102810 0x7f41041090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.507+0000 7f4109000700 1 --2- 172.21.15.67:0/1048953297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 0x7f41041010a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.507+0000 7f4109000700 1 --2- 172.21.15.67:0/1048953297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4104101e70 0x7f41041022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.507+0000 7f4109000700 1 -- 172.21.15.67:0/1048953297 >> 172.21.15.67:0/1048953297 conn(0x7f41040fc1f0 msgr2=0x7f41040fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.507+0000 7f4109000700 1 -- 172.21.15.67:0/1048953297 shutdown_connections 2024-03-30T02:36:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.507+0000 7f4109000700 1 -- 172.21.15.67:0/1048953297 wait complete. 2024-03-30T02:36:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.508+0000 7f4109000700 1 Processor -- start 2024-03-30T02:36:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4109000700 1 -- start start 2024-03-30T02:36:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4109000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 0x7f4104198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4109000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4104101e70 0x7f41041993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4109000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4104102810 0x7f41041a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4109000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41041a09a0 con 0x7f4104102810 2024-03-30T02:36:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4109000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41041a0ae0 con 0x7f4104101e70 2024-03-30T02:36:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4109000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41041a0c70 con 0x7f4104100c80 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4101e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 0x7f4104198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4101400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4104101e70 0x7f41041993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4101e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 0x7f4104198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40290/0 (socket says 172.21.15.67:40290) 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4101e00700 1 -- 172.21.15.67:0/888189465 learned_addr learned my addr 172.21.15.67:0/888189465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.509+0000 7f4102800700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4104102810 0x7f41041a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f4101400700 1 -- 172.21.15.67:0/888189465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 msgr2=0x7f4104198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f4101400700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 0x7f4104198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f4101400700 1 -- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4104102810 msgr2=0x7f41041a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f4101400700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4104102810 0x7f41041a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f4101400700 1 -- 172.21.15.67:0/888189465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f00095f0 con 0x7f4104101e70 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f4101e00700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 0x7f4104198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f4101400700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4104101e70 0x7f41041993b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40f000eb90 tx=0x7f40f000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f40faa00700 1 -- 172.21.15.67:0/888189465 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f0009970 con 0x7f4104101e70 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f4009520 con 0x7f4104101e70 2024-03-30T02:36:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f40faa00700 1 -- 172.21.15.67:0/888189465 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40f00044e0 con 0x7f4104101e70 2024-03-30T02:36:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.510+0000 7f40faa00700 1 -- 172.21.15.67:0/888189465 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f0003e50 con 0x7f4104101e70 2024-03-30T02:36:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.511+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41041a1250 con 0x7f4104101e70 2024-03-30T02:36:47.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.511+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4104067a10 con 0x7f4104101e70 2024-03-30T02:36:47.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.514+0000 7f40faa00700 1 -- 172.21.15.67:0/888189465 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f40f0004000 con 0x7f4104101e70 2024-03-30T02:36:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.515+0000 7f40faa00700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e80a11b0 0x7f40e80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.515+0000 7f40faa00700 1 -- 172.21.15.67:0/888189465 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f40f0014070 con 0x7f4104101e70 2024-03-30T02:36:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.515+0000 7f4101e00700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e80a11b0 0x7f40e80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.516+0000 7f4101e00700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e80a11b0 0x7f40e80a3670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009fd0 tx=0x7f40f4019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:47.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.517+0000 7f40faa00700 1 -- 172.21.15.67:0/888189465 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f40f00966c0 con 0x7f4104101e70 2024-03-30T02:36:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:47 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/164066635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:47 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1865990045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:47 smithi067 ceph-mon[37808]: pgmap v297: 129 pgs: 129 active+clean; 651 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-03-30T02:36:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:47 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/164066635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:47 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1865990045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:47 smithi170 ceph-mon[78798]: pgmap v297: 129 pgs: 129 active+clean; 651 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-03-30T02:36:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.786+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f410410c6e0 con 0x7f4104101e70 2024-03-30T02:36:47.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.787+0000 7f40faa00700 1 -- 172.21.15.67:0/888189465 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f40f0099ce0 con 0x7f4104101e70 2024-03-30T02:36:47.788 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:47.788 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:36:42.080059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:36:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e80a11b0 msgr2=0x7f40e80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e80a11b0 0x7f40e80a3670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009fd0 tx=0x7f40f4019040 comp rx=0 tx=0).stop 2024-03-30T02:36:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4104101e70 msgr2=0x7f41041993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4104101e70 0x7f41041993b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f40f000eb90 tx=0x7f40f000eea0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 shutdown_connections 2024-03-30T02:36:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f40e80a11b0 0x7f40e80a3670 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4104102810 0x7f41041a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4104100c80 0x7f4104198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 --2- 172.21.15.67:0/888189465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4104101e70 0x7f41041993b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:47.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 >> 172.21.15.67:0/888189465 conn(0x7f41040fc1f0 msgr2=0x7f4104107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:47.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 shutdown_connections 2024-03-30T02:36:47.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:47.790+0000 7f4109000700 1 -- 172.21.15.67:0/888189465 wait complete. 2024-03-30T02:36:47.792 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-03-30T02:36:47.809 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-30T02:36:47.810 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-30T02:36:47.810 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T02:36:48.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.008+0000 7fa671200700 1 -- 172.21.15.67:0/3081066199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 msgr2=0x7fa66c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.008+0000 7fa671200700 1 --2- 172.21.15.67:0/3081066199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 0x7fa66c073240 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009990 tx=0x7fa65c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.008+0000 7fa671200700 1 -- 172.21.15.67:0/3081066199 shutdown_connections 2024-03-30T02:36:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.008+0000 7fa671200700 1 --2- 172.21.15.67:0/3081066199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 0x7fa66c073240 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.008+0000 7fa671200700 1 --2- 172.21.15.67:0/3081066199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa66c108b40 0x7fa66c10afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.008+0000 7fa671200700 1 --2- 172.21.15.67:0/3081066199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c073810 0x7fa66c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.008+0000 7fa671200700 1 -- 172.21.15.67:0/3081066199 >> 172.21.15.67:0/3081066199 conn(0x7fa66c0fc1f0 msgr2=0x7fa66c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.008+0000 7fa671200700 1 -- 172.21.15.67:0/3081066199 shutdown_connections 2024-03-30T02:36:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.008+0000 7fa671200700 1 -- 172.21.15.67:0/3081066199 wait complete. 2024-03-30T02:36:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.010+0000 7fa671200700 1 Processor -- start 2024-03-30T02:36:48.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.010+0000 7fa671200700 1 -- start start 2024-03-30T02:36:48.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.010+0000 7fa671200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c073810 0x7fa66c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:48.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.010+0000 7fa671200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 0x7fa66c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:48.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.010+0000 7fa671200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa66c108b40 0x7fa66c1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:48.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.010+0000 7fa671200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c1a0960 con 0x7fa66c074df0 2024-03-30T02:36:48.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.010+0000 7fa671200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c1a0aa0 con 0x7fa66c108b40 2024-03-30T02:36:48.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.010+0000 7fa671200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c1a0c30 con 0x7fa66c073810 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 0x7fa66c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 0x7fa66c1993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56648/0 (socket says 172.21.15.67:56648) 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c073810 0x7fa66c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669400700 1 -- 172.21.15.67:0/3719518920 learned_addr learned my addr 172.21.15.67:0/3719518920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa66a800700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa66c108b40 0x7fa66c1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669400700 1 -- 172.21.15.67:0/3719518920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c073810 msgr2=0x7fa66c198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669400700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c073810 0x7fa66c198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669400700 1 -- 172.21.15.67:0/3719518920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa66c108b40 msgr2=0x7fa66c1a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669400700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa66c108b40 0x7fa66c1a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669400700 1 -- 172.21.15.67:0/3719518920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0095f0 con 0x7fa66c074df0 2024-03-30T02:36:48.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669e00700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c073810 0x7fa66c198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.011+0000 7fa669400700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 0x7fa66c1993c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa65800e970 tx=0x7fa65800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.012+0000 7fa662a00700 1 -- 172.21.15.67:0/3719518920 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658009790 con 0x7fa66c074df0 2024-03-30T02:36:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.012+0000 7fa662a00700 1 -- 172.21.15.67:0/3719518920 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa65800c7e0 con 0x7fa66c074df0 2024-03-30T02:36:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.012+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c1a0f10 con 0x7fa66c074df0 2024-03-30T02:36:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.012+0000 7fa662a00700 1 -- 172.21.15.67:0/3719518920 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658003c70 con 0x7fa66c074df0 2024-03-30T02:36:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.012+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c1a1390 con 0x7fa66c074df0 2024-03-30T02:36:48.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.013+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66c107900 con 0x7fa66c074df0 2024-03-30T02:36:48.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.015+0000 7fa662a00700 1 -- 172.21.15.67:0/3719518920 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fa65800cc50 con 0x7fa66c074df0 2024-03-30T02:36:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.016+0000 7fa662a00700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6500a11b0 0x7fa6500a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.016+0000 7fa662a00700 1 -- 172.21.15.67:0/3719518920 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fa658014070 con 0x7fa66c074df0 2024-03-30T02:36:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.016+0000 7fa669e00700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6500a11b0 0x7fa6500a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:48.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.017+0000 7fa669e00700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6500a11b0 0x7fa6500a3670 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b2e0 tx=0x7fa65c005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:48.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.020+0000 7fa662a00700 1 -- 172.21.15.67:0/3719518920 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa6580d0030 con 0x7fa66c074df0 2024-03-30T02:36:48.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.232+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa66c02cdc0 con 0x7fa6500a11b0 2024-03-30T02:36:48.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.247+0000 7fa662a00700 1 -- 172.21.15.67:0/3719518920 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428252 (secure 0 0 0) 0x7fa66c02cdc0 con 0x7fa6500a11b0 2024-03-30T02:36:48.255 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6500a11b0 msgr2=0x7fa6500a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6500a11b0 0x7fa6500a3670 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b2e0 tx=0x7fa65c005fd0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 msgr2=0x7fa66c1993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 0x7fa66c1993c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa65800e970 tx=0x7fa65800ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 shutdown_connections 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fa6500a11b0 0x7fa6500a3670 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa66c108b40 0x7fa66c1a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c073810 0x7fa66c198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 --2- 172.21.15.67:0/3719518920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa66c074df0 0x7fa66c1993c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 >> 172.21.15.67:0/3719518920 conn(0x7fa66c0fc1f0 msgr2=0x7fa66c104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 shutdown_connections 2024-03-30T02:36:48.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.253+0000 7fa671200700 1 -- 172.21.15.67:0/3719518920 wait complete. 2024-03-30T02:36:48.260 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-30T02:36:48.274 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-03-30T02:36:47.028962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666607,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1025,"num_read_kb":649,"num_write":442,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360624,"kb_used_data":11820,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581200,"statfs":{"total":1151940427776,"available":1151571148800,"internally_reserved":0,"allocated":12103680,"data_stored":7819336,"data_compressed":396372,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19130,"internal_metadata":357020998},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5319,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":63,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002916"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228788+0000","last_change":"2024-03-30T02:34:46.755257+0000","last_active":"2024-03-30T02:36:27.228788+0000","last_peered":"2024-03-30T02:36:27.228788+0000","last_clean":"2024-03-30T02:36:27.228788+0000","last_became_active":"2024-03-30T02:34:46.754922+0000","last_became_peered":"2024-03-30T02:34:46.754922+0000","last_unstale":"2024-03-30T02:36:27.228788+0000","last_undegraded":"2024-03-30T02:36:27.228788+0000","last_fullsized":"2024-03-30T02:36:27.228788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:09.980124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605237+0000","last_change":"2024-03-30T02:34:46.742846+0000","last_active":"2024-03-30T02:36:27.605237+0000","last_peered":"2024-03-30T02:36:27.605237+0000","last_clean":"2024-03-30T02:36:27.605237+0000","last_became_active":"2024-03-30T02:34:46.738096+0000","last_became_peered":"2024-03-30T02:34:46.738096+0000","last_unstale":"2024-03-30T02:36:27.605237+0000","last_undegraded":"2024-03-30T02:36:27.605237+0000","last_fullsized":"2024-03-30T02:36:27.605237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:43:16.982251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605205+0000","last_change":"2024-03-30T02:34:46.739607+0000","last_active":"2024-03-30T02:36:27.605205+0000","last_peered":"2024-03-30T02:36:27.605205+0000","last_clean":"2024-03-30T02:36:27.605205+0000","last_became_active":"2024-03-30T02:34:46.739230+0000","last_became_peered":"2024-03-30T02:34:46.739230+0000","last_unstale":"2024-03-30T02:36:27.605205+0000","last_undegraded":"2024-03-30T02:36:27.605205+0000","last_fullsized":"2024-03-30T02:36:27.605205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:17.818874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225333+0000","last_change":"2024-03-30T02:34:46.742118+0000","last_active":"2024-03-30T02:36:27.225333+0000","last_peered":"2024-03-30T02:36:27.225333+0000","last_clean":"2024-03-30T02:36:27.225333+0000","last_became_active":"2024-03-30T02:34:46.740845+0000","last_became_peered":"2024-03-30T02:34:46.740845+0000","last_unstale":"2024-03-30T02:36:27.225333+0000","last_undegraded":"2024-03-30T02:36:27.225333+0000","last_fullsized":"2024-03-30T02:36:27.225333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:33:28.595456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231314+0000","last_change":"2024-03-30T02:34:46.739557+0000","last_active":"2024-03-30T02:36:27.231314+0000","last_peered":"2024-03-30T02:36:27.231314+0000","last_clean":"2024-03-30T02:36:27.231314+0000","last_became_active":"2024-03-30T02:34:46.739147+0000","last_became_peered":"2024-03-30T02:34:46.739147+0000","last_unstale":"2024-03-30T02:36:27.231314+0000","last_undegraded":"2024-03-30T02:36:27.231314+0000","last_fullsized":"2024-03-30T02:36:27.231314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:12:52.307386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.227799+0000","last_change":"2024-03-30T02:35:28.211536+0000","last_active":"2024-03-30T02:36:27.227799+0000","last_peered":"2024-03-30T02:36:27.227799+0000","last_clean":"2024-03-30T02:36:27.227799+0000","last_became_active":"2024-03-30T02:35:28.211085+0000","last_became_peered":"2024-03-30T02:35:28.211085+0000","last_unstale":"2024-03-30T02:36:27.227799+0000","last_undegraded":"2024-03-30T02:36:27.227799+0000","last_fullsized":"2024-03-30T02:36:27.227799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:06:34.034277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223715+0000","last_change":"2024-03-30T02:34:46.735466+0000","last_active":"2024-03-30T02:36:27.223715+0000","last_peered":"2024-03-30T02:36:27.223715+0000","last_clean":"2024-03-30T02:36:27.223715+0000","last_became_active":"2024-03-30T02:34:46.734411+0000","last_became_peered":"2024-03-30T02:34:46.734411+0000","last_unstale":"2024-03-30T02:36:27.223715+0000","last_undegraded":"2024-03-30T02:36:27.223715+0000","last_fullsized":"2024-03-30T02:36:27.223715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:57.587768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224366+0000","last_change":"2024-03-30T02:34:46.747001+0000","last_active":"2024-03-30T02:36:27.224366+0000","last_peered":"2024-03-30T02:36:27.224366+0000","last_clean":"2024-03-30T02:36:27.224366+0000","last_became_active":"2024-03-30T02:34:46.740241+0000","last_became_peered":"2024-03-30T02:34:46.740241+0000","last_unstale":"2024-03-30T02:36:27.224366+0000","last_undegraded":"2024-03-30T02:36:27.224366+0000","last_fullsized":"2024-03-30T02:36:27.224366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:05.902190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231395+0000","last_change":"2024-03-30T02:34:46.745549+0000","last_active":"2024-03-30T02:36:27.231395+0000","last_peered":"2024-03-30T02:36:27.231395+0000","last_clean":"2024-03-30T02:36:27.231395+0000","last_became_active":"2024-03-30T02:34:46.745200+0000","last_became_peered":"2024-03-30T02:34:46.745200+0000","last_unstale":"2024-03-30T02:36:27.231395+0000","last_undegraded":"2024-03-30T02:36:27.231395+0000","last_fullsized":"2024-03-30T02:36:27.231395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:37.931440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228073+0000","last_change":"2024-03-30T02:34:46.735859+0000","last_active":"2024-03-30T02:36:27.228073+0000","last_peered":"2024-03-30T02:36:27.228073+0000","last_clean":"2024-03-30T02:36:27.228073+0000","last_became_active":"2024-03-30T02:34:46.735481+0000","last_became_peered":"2024-03-30T02:34:46.735481+0000","last_unstale":"2024-03-30T02:36:27.228073+0000","last_undegraded":"2024-03-30T02:36:27.228073+0000","last_fullsized":"2024-03-30T02:36:27.228073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:51.037887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224350+0000","last_change":"2024-03-30T02:34:46.741412+0000","last_active":"2024-03-30T02:36:27.224350+0000","last_peered":"2024-03-30T02:36:27.224350+0000","last_clean":"2024-03-30T02:36:27.224350+0000","last_became_active":"2024-03-30T02:34:46.741106+0000","last_became_peered":"2024-03-30T02:34:46.741106+0000","last_unstale":"2024-03-30T02:36:27.224350+0000","last_undegraded":"2024-03-30T02:36:27.224350+0000","last_fullsized":"2024-03-30T02:36:27.224350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:06:08.553028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225237+0000","last_change":"2024-03-30T02:34:46.746205+0000","last_active":"2024-03-30T02:36:27.225237+0000","last_peered":"2024-03-30T02:36:27.225237+0000","last_clean":"2024-03-30T02:36:27.225237+0000","last_became_active":"2024-03-30T02:34:46.745803+0000","last_became_peered":"2024-03-30T02:34:46.745803+0000","last_unstale":"2024-03-30T02:36:27.225237+0000","last_undegraded":"2024-03-30T02:36:27.225237+0000","last_fullsized":"2024-03-30T02:36:27.225237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:05.127095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221163+0000","last_change":"2024-03-30T02:34:46.722978+0000","last_active":"2024-03-30T02:36:27.221163+0000","last_peered":"2024-03-30T02:36:27.221163+0000","last_clean":"2024-03-30T02:36:27.221163+0000","last_became_active":"2024-03-30T02:34:46.722652+0000","last_became_peered":"2024-03-30T02:34:46.722652+0000","last_unstale":"2024-03-30T02:36:27.221163+0000","last_undegraded":"2024-03-30T02:36:27.221163+0000","last_fullsized":"2024-03-30T02:36:27.221163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:28:12.078216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.604850+0000","last_change":"2024-03-30T02:34:46.738895+0000","last_active":"2024-03-30T02:36:27.604850+0000","last_peered":"2024-03-30T02:36:27.604850+0000","last_clean":"2024-03-30T02:36:27.604850+0000","last_became_active":"2024-03-30T02:34:46.738510+0000","last_became_peered":"2024-03-30T02:34:46.738510+0000","last_unstale":"2024-03-30T02:36:27.604850+0000","last_undegraded":"2024-03-30T02:36:27.604850+0000","last_fullsized":"2024-03-30T02:36:27.604850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:09.859259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.604789+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:36:27.604789+0000","last_peered":"2024-03-30T02:36:27.604789+0000","last_clean":"2024-03-30T02:36:27.604789+0000","last_became_active":"2024-03-30T02:34:46.750301+0000","last_became_peered":"2024-03-30T02:34:46.750301+0000","last_unstale":"2024-03-30T02:36:27.604789+0000","last_undegraded":"2024-03-30T02:36:27.604789+0000","last_fullsized":"2024-03-30T02:36:27.604789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:06.475881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.230831+0000","last_change":"2024-03-30T02:34:46.748966+0000","last_active":"2024-03-30T02:36:27.230831+0000","last_peered":"2024-03-30T02:36:27.230831+0000","last_clean":"2024-03-30T02:36:27.230831+0000","last_became_active":"2024-03-30T02:34:46.748650+0000","last_became_peered":"2024-03-30T02:34:46.748650+0000","last_unstale":"2024-03-30T02:36:27.230831+0000","last_undegraded":"2024-03-30T02:36:27.230831+0000","last_fullsized":"2024-03-30T02:36:27.230831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:36.558864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228469+0000","last_change":"2024-03-30T02:34:46.746127+0000","last_active":"2024-03-30T02:36:27.228469+0000","last_peered":"2024-03-30T02:36:27.228469+0000","last_clean":"2024-03-30T02:36:27.228469+0000","last_became_active":"2024-03-30T02:34:46.745806+0000","last_became_peered":"2024-03-30T02:34:46.745806+0000","last_unstale":"2024-03-30T02:36:27.228469+0000","last_undegraded":"2024-03-30T02:36:27.228469+0000","last_fullsized":"2024-03-30T02:36:27.228469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:54.097996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.229068+0000","last_change":"2024-03-30T02:34:46.738391+0000","last_active":"2024-03-30T02:36:27.229068+0000","last_peered":"2024-03-30T02:36:27.229068+0000","last_clean":"2024-03-30T02:36:27.229068+0000","last_became_active":"2024-03-30T02:34:46.737110+0000","last_became_peered":"2024-03-30T02:34:46.737110+0000","last_unstale":"2024-03-30T02:36:27.229068+0000","last_undegraded":"2024-03-30T02:36:27.229068+0000","last_fullsized":"2024-03-30T02:36:27.229068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:20.326107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224284+0000","last_change":"2024-03-30T02:34:46.744522+0000","last_active":"2024-03-30T02:36:27.224284+0000","last_peered":"2024-03-30T02:36:27.224284+0000","last_clean":"2024-03-30T02:36:27.224284+0000","last_became_active":"2024-03-30T02:34:46.744088+0000","last_became_peered":"2024-03-30T02:34:46.744088+0000","last_unstale":"2024-03-30T02:36:27.224284+0000","last_undegraded":"2024-03-30T02:36:27.224284+0000","last_fullsized":"2024-03-30T02:36:27.224284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:35.698979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224043+0000","last_change":"2024-03-30T02:34:50.770855+0000","last_active":"2024-03-30T02:36:27.224043+0000","last_peered":"2024-03-30T02:36:27.224043+0000","last_clean":"2024-03-30T02:36:27.224043+0000","last_became_active":"2024-03-30T02:34:50.770420+0000","last_became_peered":"2024-03-30T02:34:50.770420+0000","last_unstale":"2024-03-30T02:36:27.224043+0000","last_undegraded":"2024-03-30T02:36:27.224043+0000","last_fullsized":"2024-03-30T02:36:27.224043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:00:08.847059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:44.255830+0000","last_change":"2024-03-30T02:34:46.207252+0000","last_active":"2024-03-30T02:36:44.255830+0000","last_peered":"2024-03-30T02:36:44.255830+0000","last_clean":"2024-03-30T02:36:44.255830+0000","last_became_active":"2024-03-30T02:34:46.206916+0000","last_became_peered":"2024-03-30T02:34:46.206916+0000","last_unstale":"2024-03-30T02:36:44.255830+0000","last_undegraded":"2024-03-30T02:36:44.255830+0000","last_fullsized":"2024-03-30T02:36:44.255830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:03:50.192245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225209+0000","last_change":"2024-03-30T02:34:46.745778+0000","last_active":"2024-03-30T02:36:27.225209+0000","last_peered":"2024-03-30T02:36:27.225209+0000","last_clean":"2024-03-30T02:36:27.225209+0000","last_became_active":"2024-03-30T02:34:46.745471+0000","last_became_peered":"2024-03-30T02:34:46.745471+0000","last_unstale":"2024-03-30T02:36:27.225209+0000","last_undegraded":"2024-03-30T02:36:27.225209+0000","last_fullsized":"2024-03-30T02:36:27.225209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:57.314163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225059+0000","last_change":"2024-03-30T02:34:50.769577+0000","last_active":"2024-03-30T02:36:27.225059+0000","last_peered":"2024-03-30T02:36:27.225059+0000","last_clean":"2024-03-30T02:36:27.225059+0000","last_became_active":"2024-03-30T02:34:50.769100+0000","last_became_peered":"2024-03-30T02:34:50.769100+0000","last_unstale":"2024-03-30T02:36:27.225059+0000","last_undegraded":"2024-03-30T02:36:27.225059+0000","last_fullsized":"2024-03-30T02:36:27.225059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:08:43.495671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225074+0000","last_change":"2024-03-30T02:34:46.218032+0000","last_active":"2024-03-30T02:36:27.225074+0000","last_peered":"2024-03-30T02:36:27.225074+0000","last_clean":"2024-03-30T02:36:27.225074+0000","last_became_active":"2024-03-30T02:34:46.217733+0000","last_became_peered":"2024-03-30T02:34:46.217733+0000","last_unstale":"2024-03-30T02:36:27.225074+0000","last_undegraded":"2024-03-30T02:36:27.225074+0000","last_fullsized":"2024-03-30T02:36:27.225074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:58.413590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605282+0000","last_change":"2024-03-30T02:34:46.750588+0000","last_active":"2024-03-30T02:36:27.605282+0000","last_peered":"2024-03-30T02:36:27.605282+0000","last_clean":"2024-03-30T02:36:27.605282+0000","last_became_active":"2024-03-30T02:34:46.750016+0000","last_became_peered":"2024-03-30T02:34:46.750016+0000","last_unstale":"2024-03-30T02:36:27.605282+0000","last_undegraded":"2024-03-30T02:36:27.605282+0000","last_fullsized":"2024-03-30T02:36:27.605282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:19.144544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225017+0000","last_change":"2024-03-30T02:35:28.213404+0000","last_active":"2024-03-30T02:36:27.225017+0000","last_peered":"2024-03-30T02:36:27.225017+0000","last_clean":"2024-03-30T02:36:27.225017+0000","last_became_active":"2024-03-30T02:35:28.212746+0000","last_became_peered":"2024-03-30T02:35:28.212746+0000","last_unstale":"2024-03-30T02:36:27.225017+0000","last_undegraded":"2024-03-30T02:36:27.225017+0000","last_fullsized":"2024-03-30T02:36:27.225017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:24.807469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.232111+0000","last_change":"2024-03-30T02:34:46.213741+0000","last_active":"2024-03-30T02:36:27.232111+0000","last_peered":"2024-03-30T02:36:27.232111+0000","last_clean":"2024-03-30T02:36:27.232111+0000","last_became_active":"2024-03-30T02:34:46.213395+0000","last_became_peered":"2024-03-30T02:34:46.213395+0000","last_unstale":"2024-03-30T02:36:27.232111+0000","last_undegraded":"2024-03-30T02:36:27.232111+0000","last_fullsized":"2024-03-30T02:36:27.232111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:13.192415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231037+0000","last_change":"2024-03-30T02:34:46.738401+0000","last_active":"2024-03-30T02:36:27.231037+0000","last_peered":"2024-03-30T02:36:27.231037+0000","last_clean":"2024-03-30T02:36:27.231037+0000","last_became_active":"2024-03-30T02:34:46.737804+0000","last_became_peered":"2024-03-30T02:34:46.737804+0000","last_unstale":"2024-03-30T02:36:27.231037+0000","last_undegraded":"2024-03-30T02:36:27.231037+0000","last_fullsized":"2024-03-30T02:36:27.231037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:56:08.194439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224866+0000","last_change":"2024-03-30T02:34:50.766222+0000","last_active":"2024-03-30T02:36:27.224866+0000","last_peered":"2024-03-30T02:36:27.224866+0000","last_clean":"2024-03-30T02:36:27.224866+0000","last_became_active":"2024-03-30T02:34:50.765836+0000","last_became_peered":"2024-03-30T02:34:50.765836+0000","last_unstale":"2024-03-30T02:36:27.224866+0000","last_undegraded":"2024-03-30T02:36:27.224866+0000","last_fullsized":"2024-03-30T02:36:27.224866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:32.529535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231217+0000","last_change":"2024-03-30T02:34:46.212586+0000","last_active":"2024-03-30T02:36:27.231217+0000","last_peered":"2024-03-30T02:36:27.231217+0000","last_clean":"2024-03-30T02:36:27.231217+0000","last_became_active":"2024-03-30T02:34:46.212110+0000","last_became_peered":"2024-03-30T02:34:46.212110+0000","last_unstale":"2024-03-30T02:36:27.231217+0000","last_undegraded":"2024-03-30T02:36:27.231217+0000","last_fullsized":"2024-03-30T02:36:27.231217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:38.630953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221516+0000","last_change":"2024-03-30T02:34:46.734923+0000","last_active":"2024-03-30T02:36:27.221516+0000","last_peered":"2024-03-30T02:36:27.221516+0000","last_clean":"2024-03-30T02:36:27.221516+0000","last_became_active":"2024-03-30T02:34:46.734572+0000","last_became_peered":"2024-03-30T02:34:46.734572+0000","last_unstale":"2024-03-30T02:36:27.221516+0000","last_undegraded":"2024-03-30T02:36:27.221516+0000","last_fullsized":"2024-03-30T02:36:27.221516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:28.647788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224173+0000","last_change":"2024-03-30T02:34:50.757803+0000","last_active":"2024-03-30T02:36:27.224173+0000","last_peered":"2024-03-30T02:36:27.224173+0000","last_clean":"2024-03-30T02:36:27.224173+0000","last_became_active":"2024-03-30T02:34:50.757198+0000","last_became_peered":"2024-03-30T02:34:50.757198+0000","last_unstale":"2024-03-30T02:36:27.224173+0000","last_undegraded":"2024-03-30T02:36:27.224173+0000","last_fullsized":"2024-03-30T02:36:27.224173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:54:49.399741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:41.254356+0000","last_change":"2024-03-30T02:34:46.214549+0000","last_active":"2024-03-30T02:36:41.254356+0000","last_peered":"2024-03-30T02:36:41.254356+0000","last_clean":"2024-03-30T02:36:41.254356+0000","last_became_active":"2024-03-30T02:34:46.214198+0000","last_became_peered":"2024-03-30T02:34:46.214198+0000","last_unstale":"2024-03-30T02:36:41.254356+0000","last_undegraded":"2024-03-30T02:36:41.254356+0000","last_fullsized":"2024-03-30T02:36:41.254356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:45:25.618982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1108,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231496+0000","last_change":"2024-03-30T02:34:46.737771+0000","last_active":"2024-03-30T02:36:27.231496+0000","last_peered":"2024-03-30T02:36:27.231496+0000","last_clean":"2024-03-30T02:36:27.231496+0000","last_became_active":"2024-03-30T02:34:46.737438+0000","last_became_peered":"2024-03-30T02:34:46.737438+0000","last_unstale":"2024-03-30T02:36:27.231496+0000","last_undegraded":"2024-03-30T02:36:27.231496+0000","last_fullsized":"2024-03-30T02:36:27.231496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:08.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224131+0000","last_change":"2024-03-30T02:34:50.767304+0000","last_active":"2024-03-30T02:36:27.224131+0000","last_peered":"2024-03-30T02:36:27.224131+0000","last_clean":"2024-03-30T02:36:27.224131+0000","last_became_active":"2024-03-30T02:34:50.766909+0000","last_became_peered":"2024-03-30T02:34:50.766909+0000","last_unstale":"2024-03-30T02:36:27.224131+0000","last_undegraded":"2024-03-30T02:36:27.224131+0000","last_fullsized":"2024-03-30T02:36:27.224131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:24.021991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.227724+0000","last_change":"2024-03-30T02:34:46.214937+0000","last_active":"2024-03-30T02:36:27.227724+0000","last_peered":"2024-03-30T02:36:27.227724+0000","last_clean":"2024-03-30T02:36:27.227724+0000","last_became_active":"2024-03-30T02:34:46.214503+0000","last_became_peered":"2024-03-30T02:34:46.214503+0000","last_unstale":"2024-03-30T02:36:27.227724+0000","last_undegraded":"2024-03-30T02:36:27.227724+0000","last_fullsized":"2024-03-30T02:36:27.227724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:20:11.624919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221928+0000","last_change":"2024-03-30T02:34:46.737374+0000","last_active":"2024-03-30T02:36:27.221928+0000","last_peered":"2024-03-30T02:36:27.221928+0000","last_clean":"2024-03-30T02:36:27.221928+0000","last_became_active":"2024-03-30T02:34:46.735307+0000","last_became_peered":"2024-03-30T02:34:46.735307+0000","last_unstale":"2024-03-30T02:36:27.221928+0000","last_undegraded":"2024-03-30T02:36:27.221928+0000","last_fullsized":"2024-03-30T02:36:27.221928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:30.482728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228360+0000","last_change":"2024-03-30T02:34:50.768016+0000","last_active":"2024-03-30T02:36:27.228360+0000","last_peered":"2024-03-30T02:36:27.228360+0000","last_clean":"2024-03-30T02:36:27.228360+0000","last_became_active":"2024-03-30T02:34:50.767564+0000","last_became_peered":"2024-03-30T02:34:50.767564+0000","last_unstale":"2024-03-30T02:36:27.228360+0000","last_undegraded":"2024-03-30T02:36:27.228360+0000","last_fullsized":"2024-03-30T02:36:27.228360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:16.888754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.220995+0000","last_change":"2024-03-30T02:34:46.212982+0000","last_active":"2024-03-30T02:36:27.220995+0000","last_peered":"2024-03-30T02:36:27.220995+0000","last_clean":"2024-03-30T02:36:27.220995+0000","last_became_active":"2024-03-30T02:34:46.212550+0000","last_became_peered":"2024-03-30T02:34:46.212550+0000","last_unstale":"2024-03-30T02:36:27.220995+0000","last_undegraded":"2024-03-30T02:36:27.220995+0000","last_fullsized":"2024-03-30T02:36:27.220995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:44:50.156615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228102+0000","last_change":"2024-03-30T02:35:28.213774+0000","last_active":"2024-03-30T02:36:27.228102+0000","last_peered":"2024-03-30T02:36:27.228102+0000","last_clean":"2024-03-30T02:36:27.228102+0000","last_became_active":"2024-03-30T02:35:28.213085+0000","last_became_peered":"2024-03-30T02:35:28.213085+0000","last_unstale":"2024-03-30T02:36:27.228102+0000","last_undegraded":"2024-03-30T02:36:27.228102+0000","last_fullsized":"2024-03-30T02:36:27.228102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:56:30.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605207+0000","last_change":"2024-03-30T02:34:50.773036+0000","last_active":"2024-03-30T02:36:27.605207+0000","last_peered":"2024-03-30T02:36:27.605207+0000","last_clean":"2024-03-30T02:36:27.605207+0000","last_became_active":"2024-03-30T02:34:50.772614+0000","last_became_peered":"2024-03-30T02:34:50.772614+0000","last_unstale":"2024-03-30T02:36:27.605207+0000","last_undegraded":"2024-03-30T02:36:27.605207+0000","last_fullsized":"2024-03-30T02:36:27.605207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:54:28.792793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221282+0000","last_change":"2024-03-30T02:34:46.209802+0000","last_active":"2024-03-30T02:36:27.221282+0000","last_peered":"2024-03-30T02:36:27.221282+0000","last_clean":"2024-03-30T02:36:27.221282+0000","last_became_active":"2024-03-30T02:34:46.209246+0000","last_became_peered":"2024-03-30T02:34:46.209246+0000","last_unstale":"2024-03-30T02:36:27.221282+0000","last_undegraded":"2024-03-30T02:36:27.221282+0000","last_fullsized":"2024-03-30T02:36:27.221282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:09:57.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228970+0000","last_change":"2024-03-30T02:34:46.743638+0000","last_active":"2024-03-30T02:36:27.228970+0000","last_peered":"2024-03-30T02:36:27.228970+0000","last_clean":"2024-03-30T02:36:27.228970+0000","last_became_active":"2024-03-30T02:34:46.743246+0000","last_became_peered":"2024-03-30T02:34:46.743246+0000","last_unstale":"2024-03-30T02:36:27.228970+0000","last_undegraded":"2024-03-30T02:36:27.228970+0000","last_fullsized":"2024-03-30T02:36:27.228970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:49:55.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225892+0000","last_change":"2024-03-30T02:34:50.768586+0000","last_active":"2024-03-30T02:36:27.225892+0000","last_peered":"2024-03-30T02:36:27.225892+0000","last_clean":"2024-03-30T02:36:27.225892+0000","last_became_active":"2024-03-30T02:34:50.766894+0000","last_became_peered":"2024-03-30T02:34:50.766894+0000","last_unstale":"2024-03-30T02:36:27.225892+0000","last_undegraded":"2024-03-30T02:36:27.225892+0000","last_fullsized":"2024-03-30T02:36:27.225892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:36:39.329608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'6","reported_seq":194,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:43.256303+0000","last_change":"2024-03-30T02:34:46.204994+0000","last_active":"2024-03-30T02:36:43.256303+0000","last_peered":"2024-03-30T02:36:43.256303+0000","last_clean":"2024-03-30T02:36:43.256303+0000","last_became_active":"2024-03-30T02:34:46.204710+0000","last_became_peered":"2024-03-30T02:34:46.204710+0000","last_unstale":"2024-03-30T02:36:43.256303+0000","last_undegraded":"2024-03-30T02:36:43.256303+0000","last_fullsized":"2024-03-30T02:36:43.256303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:39.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231396+0000","last_change":"2024-03-30T02:34:46.733573+0000","last_active":"2024-03-30T02:36:27.231396+0000","last_peered":"2024-03-30T02:36:27.231396+0000","last_clean":"2024-03-30T02:36:27.231396+0000","last_became_active":"2024-03-30T02:34:46.733241+0000","last_became_peered":"2024-03-30T02:34:46.733241+0000","last_unstale":"2024-03-30T02:36:27.231396+0000","last_undegraded":"2024-03-30T02:36:27.231396+0000","last_fullsized":"2024-03-30T02:36:27.231396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:07:10.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231018+0000","last_change":"2024-03-30T02:35:28.313493+0000","last_active":"2024-03-30T02:36:27.231018+0000","last_peered":"2024-03-30T02:36:27.231018+0000","last_clean":"2024-03-30T02:36:27.231018+0000","last_became_active":"2024-03-30T02:35:28.313065+0000","last_became_peered":"2024-03-30T02:35:28.313065+0000","last_unstale":"2024-03-30T02:36:27.231018+0000","last_undegraded":"2024-03-30T02:36:27.231018+0000","last_fullsized":"2024-03-30T02:36:27.231018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:44:52.675251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224410+0000","last_change":"2024-03-30T02:34:46.212732+0000","last_active":"2024-03-30T02:36:27.224410+0000","last_peered":"2024-03-30T02:36:27.224410+0000","last_clean":"2024-03-30T02:36:27.224410+0000","last_became_active":"2024-03-30T02:34:46.212306+0000","last_became_peered":"2024-03-30T02:34:46.212306+0000","last_unstale":"2024-03-30T02:36:27.224410+0000","last_undegraded":"2024-03-30T02:36:27.224410+0000","last_fullsized":"2024-03-30T02:36:27.224410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:27:15.766203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605314+0000","last_change":"2024-03-30T02:34:46.749420+0000","last_active":"2024-03-30T02:36:27.605314+0000","last_peered":"2024-03-30T02:36:27.605314+0000","last_clean":"2024-03-30T02:36:27.605314+0000","last_became_active":"2024-03-30T02:34:46.745286+0000","last_became_peered":"2024-03-30T02:34:46.745286+0000","last_unstale":"2024-03-30T02:36:27.605314+0000","last_undegraded":"2024-03-30T02:36:27.605314+0000","last_fullsized":"2024-03-30T02:36:27.605314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:01.489862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225302+0000","last_change":"2024-03-30T02:34:50.760542+0000","last_active":"2024-03-30T02:36:27.225302+0000","last_peered":"2024-03-30T02:36:27.225302+0000","last_clean":"2024-03-30T02:36:27.225302+0000","last_became_active":"2024-03-30T02:34:50.760087+0000","last_became_peered":"2024-03-30T02:34:50.760087+0000","last_unstale":"2024-03-30T02:36:27.225302+0000","last_undegraded":"2024-03-30T02:36:27.225302+0000","last_fullsized":"2024-03-30T02:36:27.225302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:46.352189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221437+0000","last_change":"2024-03-30T02:34:46.224058+0000","last_active":"2024-03-30T02:36:27.221437+0000","last_peered":"2024-03-30T02:36:27.221437+0000","last_clean":"2024-03-30T02:36:27.221437+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:36:27.221437+0000","last_undegraded":"2024-03-30T02:36:27.221437+0000","last_fullsized":"2024-03-30T02:36:27.221437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:31:10.124533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231126+0000","last_change":"2024-03-30T02:34:46.741413+0000","last_active":"2024-03-30T02:36:27.231126+0000","last_peered":"2024-03-30T02:36:27.231126+0000","last_clean":"2024-03-30T02:36:27.231126+0000","last_became_active":"2024-03-30T02:34:46.741112+0000","last_became_peered":"2024-03-30T02:34:46.741112+0000","last_unstale":"2024-03-30T02:36:27.231126+0000","last_undegraded":"2024-03-30T02:36:27.231126+0000","last_fullsized":"2024-03-30T02:36:27.231126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:34.820166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.229215+0000","last_change":"2024-03-30T02:34:50.752315+0000","last_active":"2024-03-30T02:36:27.229215+0000","last_peered":"2024-03-30T02:36:27.229215+0000","last_clean":"2024-03-30T02:36:27.229215+0000","last_became_active":"2024-03-30T02:34:50.750985+0000","last_became_peered":"2024-03-30T02:34:50.750985+0000","last_unstale":"2024-03-30T02:36:27.229215+0000","last_undegraded":"2024-03-30T02:36:27.229215+0000","last_fullsized":"2024-03-30T02:36:27.229215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:37.271063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'43","reported_seq":243,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:43.257780+0000","last_change":"2024-03-30T02:34:46.214759+0000","last_active":"2024-03-30T02:36:43.257780+0000","last_peered":"2024-03-30T02:36:43.257780+0000","last_clean":"2024-03-30T02:36:43.257780+0000","last_became_active":"2024-03-30T02:34:46.214388+0000","last_became_peered":"2024-03-30T02:34:46.214388+0000","last_unstale":"2024-03-30T02:36:43.257780+0000","last_undegraded":"2024-03-30T02:36:43.257780+0000","last_fullsized":"2024-03-30T02:36:43.257780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:41.100461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68401,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":180,"num_read_kb":158,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231853+0000","last_change":"2024-03-30T02:34:46.741112+0000","last_active":"2024-03-30T02:36:27.231853+0000","last_peered":"2024-03-30T02:36:27.231853+0000","last_clean":"2024-03-30T02:36:27.231853+0000","last_became_active":"2024-03-30T02:34:46.739664+0000","last_became_peered":"2024-03-30T02:34:46.739664+0000","last_unstale":"2024-03-30T02:36:27.231853+0000","last_undegraded":"2024-03-30T02:36:27.231853+0000","last_fullsized":"2024-03-30T02:36:27.231853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:07.763942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221733+0000","last_change":"2024-03-30T02:34:50.765645+0000","last_active":"2024-03-30T02:36:27.221733+0000","last_peered":"2024-03-30T02:36:27.221733+0000","last_clean":"2024-03-30T02:36:27.221733+0000","last_became_active":"2024-03-30T02:34:50.765345+0000","last_became_peered":"2024-03-30T02:34:50.765345+0000","last_unstale":"2024-03-30T02:36:27.221733+0000","last_undegraded":"2024-03-30T02:36:27.221733+0000","last_fullsized":"2024-03-30T02:36:27.221733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:25:38.591525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224473+0000","last_change":"2024-03-30T02:34:46.216643+0000","last_active":"2024-03-30T02:36:27.224473+0000","last_peered":"2024-03-30T02:36:27.224473+0000","last_clean":"2024-03-30T02:36:27.224473+0000","last_became_active":"2024-03-30T02:34:46.216188+0000","last_became_peered":"2024-03-30T02:34:46.216188+0000","last_unstale":"2024-03-30T02:36:27.224473+0000","last_undegraded":"2024-03-30T02:36:27.224473+0000","last_fullsized":"2024-03-30T02:36:27.224473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:49:10.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221353+0000","last_change":"2024-03-30T02:34:46.748313+0000","last_active":"2024-03-30T02:36:27.221353+0000","last_peered":"2024-03-30T02:36:27.221353+0000","last_clean":"2024-03-30T02:36:27.221353+0000","last_became_active":"2024-03-30T02:34:46.742897+0000","last_became_peered":"2024-03-30T02:34:46.742897+0000","last_unstale":"2024-03-30T02:36:27.221353+0000","last_undegraded":"2024-03-30T02:36:27.221353+0000","last_fullsized":"2024-03-30T02:36:27.221353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:00.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228185+0000","last_change":"2024-03-30T02:34:50.760993+0000","last_active":"2024-03-30T02:36:27.228185+0000","last_peered":"2024-03-30T02:36:27.228185+0000","last_clean":"2024-03-30T02:36:27.228185+0000","last_became_active":"2024-03-30T02:34:50.760535+0000","last_became_peered":"2024-03-30T02:34:50.760535+0000","last_unstale":"2024-03-30T02:36:27.228185+0000","last_undegraded":"2024-03-30T02:36:27.228185+0000","last_fullsized":"2024-03-30T02:36:27.228185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:15.569533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221659+0000","last_change":"2024-03-30T02:34:46.216603+0000","last_active":"2024-03-30T02:36:27.221659+0000","last_peered":"2024-03-30T02:36:27.221659+0000","last_clean":"2024-03-30T02:36:27.221659+0000","last_became_active":"2024-03-30T02:34:46.216231+0000","last_became_peered":"2024-03-30T02:34:46.216231+0000","last_unstale":"2024-03-30T02:36:27.221659+0000","last_undegraded":"2024-03-30T02:36:27.221659+0000","last_fullsized":"2024-03-30T02:36:27.221659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:51:00.659652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224126+0000","last_change":"2024-03-30T02:34:46.742815+0000","last_active":"2024-03-30T02:36:27.224126+0000","last_peered":"2024-03-30T02:36:27.224126+0000","last_clean":"2024-03-30T02:36:27.224126+0000","last_became_active":"2024-03-30T02:34:46.742487+0000","last_became_peered":"2024-03-30T02:34:46.742487+0000","last_unstale":"2024-03-30T02:36:27.224126+0000","last_undegraded":"2024-03-30T02:36:27.224126+0000","last_fullsized":"2024-03-30T02:36:27.224126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:07.182990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228857+0000","last_change":"2024-03-30T02:34:50.770398+0000","last_active":"2024-03-30T02:36:27.228857+0000","last_peered":"2024-03-30T02:36:27.228857+0000","last_clean":"2024-03-30T02:36:27.228857+0000","last_became_active":"2024-03-30T02:34:50.770000+0000","last_became_peered":"2024-03-30T02:34:50.770000+0000","last_unstale":"2024-03-30T02:36:27.228857+0000","last_undegraded":"2024-03-30T02:36:27.228857+0000","last_fullsized":"2024-03-30T02:36:27.228857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:14:49.057081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":116,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:44.256642+0000","last_change":"2024-03-30T02:34:46.209403+0000","last_active":"2024-03-30T02:36:44.256642+0000","last_peered":"2024-03-30T02:36:44.256642+0000","last_clean":"2024-03-30T02:36:44.256642+0000","last_became_active":"2024-03-30T02:34:46.209065+0000","last_became_peered":"2024-03-30T02:34:46.209065+0000","last_unstale":"2024-03-30T02:36:44.256642+0000","last_undegraded":"2024-03-30T02:36:44.256642+0000","last_fullsized":"2024-03-30T02:36:44.256642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:36:20.427720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223778+0000","last_change":"2024-03-30T02:34:46.734861+0000","last_active":"2024-03-30T02:36:27.223778+0000","last_peered":"2024-03-30T02:36:27.223778+0000","last_clean":"2024-03-30T02:36:27.223778+0000","last_became_active":"2024-03-30T02:34:46.734376+0000","last_became_peered":"2024-03-30T02:34:46.734376+0000","last_unstale":"2024-03-30T02:36:27.223778+0000","last_undegraded":"2024-03-30T02:36:27.223778+0000","last_fullsized":"2024-03-30T02:36:27.223778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:08:36.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221531+0000","last_change":"2024-03-30T02:34:50.767469+0000","last_active":"2024-03-30T02:36:27.221531+0000","last_peered":"2024-03-30T02:36:27.221531+0000","last_clean":"2024-03-30T02:36:27.221531+0000","last_became_active":"2024-03-30T02:34:50.765460+0000","last_became_peered":"2024-03-30T02:34:50.765460+0000","last_unstale":"2024-03-30T02:36:27.221531+0000","last_undegraded":"2024-03-30T02:36:27.221531+0000","last_fullsized":"2024-03-30T02:36:27.221531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:54:06.940171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231014+0000","last_change":"2024-03-30T02:34:46.224053+0000","last_active":"2024-03-30T02:36:27.231014+0000","last_peered":"2024-03-30T02:36:27.231014+0000","last_clean":"2024-03-30T02:36:27.231014+0000","last_became_active":"2024-03-30T02:34:46.223666+0000","last_became_peered":"2024-03-30T02:34:46.223666+0000","last_unstale":"2024-03-30T02:36:27.231014+0000","last_undegraded":"2024-03-30T02:36:27.231014+0000","last_fullsized":"2024-03-30T02:36:27.231014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:18:22.976298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221390+0000","last_change":"2024-03-30T02:34:46.734838+0000","last_active":"2024-03-30T02:36:27.221390+0000","last_peered":"2024-03-30T02:36:27.221390+0000","last_clean":"2024-03-30T02:36:27.221390+0000","last_became_active":"2024-03-30T02:34:46.734428+0000","last_became_peered":"2024-03-30T02:34:46.734428+0000","last_unstale":"2024-03-30T02:36:27.221390+0000","last_undegraded":"2024-03-30T02:36:27.221390+0000","last_fullsized":"2024-03-30T02:36:27.221390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:42:51.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221785+0000","last_change":"2024-03-30T02:34:50.771150+0000","last_active":"2024-03-30T02:36:27.221785+0000","last_peered":"2024-03-30T02:36:27.221785+0000","last_clean":"2024-03-30T02:36:27.221785+0000","last_became_active":"2024-03-30T02:34:50.770703+0000","last_became_peered":"2024-03-30T02:34:50.770703+0000","last_unstale":"2024-03-30T02:36:27.221785+0000","last_undegraded":"2024-03-30T02:36:27.221785+0000","last_fullsized":"2024-03-30T02:36:27.221785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:04.466664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231835+0000","last_change":"2024-03-30T02:34:46.203558+0000","last_active":"2024-03-30T02:36:27.231835+0000","last_peered":"2024-03-30T02:36:27.231835+0000","last_clean":"2024-03-30T02:36:27.231835+0000","last_became_active":"2024-03-30T02:34:46.202896+0000","last_became_peered":"2024-03-30T02:34:46.202896+0000","last_unstale":"2024-03-30T02:36:27.231835+0000","last_undegraded":"2024-03-30T02:36:27.231835+0000","last_fullsized":"2024-03-30T02:36:27.231835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:33.945665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228622+0000","last_change":"2024-03-30T02:35:28.212775+0000","last_active":"2024-03-30T02:36:27.228622+0000","last_peered":"2024-03-30T02:36:27.228622+0000","last_clean":"2024-03-30T02:36:27.228622+0000","last_became_active":"2024-03-30T02:35:28.212368+0000","last_became_peered":"2024-03-30T02:35:28.212368+0000","last_unstale":"2024-03-30T02:36:27.228622+0000","last_undegraded":"2024-03-30T02:36:27.228622+0000","last_fullsized":"2024-03-30T02:36:27.228622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:50.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231620+0000","last_change":"2024-03-30T02:34:50.797758+0000","last_active":"2024-03-30T02:36:27.231620+0000","last_peered":"2024-03-30T02:36:27.231620+0000","last_clean":"2024-03-30T02:36:27.231620+0000","last_became_active":"2024-03-30T02:34:50.797311+0000","last_became_peered":"2024-03-30T02:34:50.797311+0000","last_unstale":"2024-03-30T02:36:27.231620+0000","last_undegraded":"2024-03-30T02:36:27.231620+0000","last_fullsized":"2024-03-30T02:36:27.231620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:32.337422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228443+0000","last_change":"2024-03-30T02:34:46.224397+0000","last_active":"2024-03-30T02:36:27.228443+0000","last_peered":"2024-03-30T02:36:27.228443+0000","last_clean":"2024-03-30T02:36:27.228443+0000","last_became_active":"2024-03-30T02:34:46.223825+0000","last_became_peered":"2024-03-30T02:34:46.223825+0000","last_unstale":"2024-03-30T02:36:27.228443+0000","last_undegraded":"2024-03-30T02:36:27.228443+0000","last_fullsized":"2024-03-30T02:36:27.228443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:53.154665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.227952+0000","last_change":"2024-03-30T02:34:46.741358+0000","last_active":"2024-03-30T02:36:27.227952+0000","last_peered":"2024-03-30T02:36:27.227952+0000","last_clean":"2024-03-30T02:36:27.227952+0000","last_became_active":"2024-03-30T02:34:46.740734+0000","last_became_peered":"2024-03-30T02:34:46.740734+0000","last_unstale":"2024-03-30T02:36:27.227952+0000","last_undegraded":"2024-03-30T02:36:27.227952+0000","last_fullsized":"2024-03-30T02:36:27.227952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:59.678431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221512+0000","last_change":"2024-03-30T02:34:50.761998+0000","last_active":"2024-03-30T02:36:27.221512+0000","last_peered":"2024-03-30T02:36:27.221512+0000","last_clean":"2024-03-30T02:36:27.221512+0000","last_became_active":"2024-03-30T02:34:50.761560+0000","last_became_peered":"2024-03-30T02:34:50.761560+0000","last_unstale":"2024-03-30T02:36:27.221512+0000","last_undegraded":"2024-03-30T02:36:27.221512+0000","last_fullsized":"2024-03-30T02:36:27.221512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:57:17.213624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":112,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:44.256432+0000","last_change":"2024-03-30T02:34:46.213427+0000","last_active":"2024-03-30T02:36:44.256432+0000","last_peered":"2024-03-30T02:36:44.256432+0000","last_clean":"2024-03-30T02:36:44.256432+0000","last_became_active":"2024-03-30T02:34:46.213082+0000","last_became_peered":"2024-03-30T02:34:46.213082+0000","last_unstale":"2024-03-30T02:36:44.256432+0000","last_undegraded":"2024-03-30T02:36:44.256432+0000","last_fullsized":"2024-03-30T02:36:44.256432+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:31:35.513723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228590+0000","last_change":"2024-03-30T02:34:46.747197+0000","last_active":"2024-03-30T02:36:27.228590+0000","last_peered":"2024-03-30T02:36:27.228590+0000","last_clean":"2024-03-30T02:36:27.228590+0000","last_became_active":"2024-03-30T02:34:46.743841+0000","last_became_peered":"2024-03-30T02:34:46.743841+0000","last_unstale":"2024-03-30T02:36:27.228590+0000","last_undegraded":"2024-03-30T02:36:27.228590+0000","last_fullsized":"2024-03-30T02:36:27.228590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:09.234382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221642+0000","last_change":"2024-03-30T02:34:50.797423+0000","last_active":"2024-03-30T02:36:27.221642+0000","last_peered":"2024-03-30T02:36:27.221642+0000","last_clean":"2024-03-30T02:36:27.221642+0000","last_became_active":"2024-03-30T02:34:50.796746+0000","last_became_peered":"2024-03-30T02:34:50.796746+0000","last_unstale":"2024-03-30T02:36:27.221642+0000","last_undegraded":"2024-03-30T02:36:27.221642+0000","last_fullsized":"2024-03-30T02:36:27.221642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:24.014131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221459+0000","last_change":"2024-03-30T02:34:46.734972+0000","last_active":"2024-03-30T02:36:27.221459+0000","last_peered":"2024-03-30T02:36:27.221459+0000","last_clean":"2024-03-30T02:36:27.221459+0000","last_became_active":"2024-03-30T02:34:46.734520+0000","last_became_peered":"2024-03-30T02:34:46.734520+0000","last_unstale":"2024-03-30T02:36:27.221459+0000","last_undegraded":"2024-03-30T02:36:27.221459+0000","last_fullsized":"2024-03-30T02:36:27.221459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:15.635060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":112,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:43.255021+0000","last_change":"2024-03-30T02:34:46.224036+0000","last_active":"2024-03-30T02:36:43.255021+0000","last_peered":"2024-03-30T02:36:43.255021+0000","last_clean":"2024-03-30T02:36:43.255021+0000","last_became_active":"2024-03-30T02:34:46.223706+0000","last_became_peered":"2024-03-30T02:34:46.223706+0000","last_unstale":"2024-03-30T02:36:43.255021+0000","last_undegraded":"2024-03-30T02:36:43.255021+0000","last_fullsized":"2024-03-30T02:36:43.255021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:40.458345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224187+0000","last_change":"2024-03-30T02:34:50.757894+0000","last_active":"2024-03-30T02:36:27.224187+0000","last_peered":"2024-03-30T02:36:27.224187+0000","last_clean":"2024-03-30T02:36:27.224187+0000","last_became_active":"2024-03-30T02:34:50.757486+0000","last_became_peered":"2024-03-30T02:34:50.757486+0000","last_unstale":"2024-03-30T02:36:27.224187+0000","last_undegraded":"2024-03-30T02:36:27.224187+0000","last_fullsized":"2024-03-30T02:36:27.224187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:30.609851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225449+0000","last_change":"2024-03-30T02:34:46.745990+0000","last_active":"2024-03-30T02:36:27.225449+0000","last_peered":"2024-03-30T02:36:27.225449+0000","last_clean":"2024-03-30T02:36:27.225449+0000","last_became_active":"2024-03-30T02:34:46.741937+0000","last_became_peered":"2024-03-30T02:34:46.741937+0000","last_unstale":"2024-03-30T02:36:27.225449+0000","last_undegraded":"2024-03-30T02:36:27.225449+0000","last_fullsized":"2024-03-30T02:36:27.225449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:26:17.833164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225196+0000","last_change":"2024-03-30T02:34:46.208858+0000","last_active":"2024-03-30T02:36:27.225196+0000","last_peered":"2024-03-30T02:36:27.225196+0000","last_clean":"2024-03-30T02:36:27.225196+0000","last_became_active":"2024-03-30T02:34:46.208479+0000","last_became_peered":"2024-03-30T02:34:46.208479+0000","last_unstale":"2024-03-30T02:36:27.225196+0000","last_undegraded":"2024-03-30T02:36:27.225196+0000","last_fullsized":"2024-03-30T02:36:27.225196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:58:14.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223677+0000","last_change":"2024-03-30T02:33:56.208237+0000","last_active":"2024-03-30T02:36:27.223677+0000","last_peered":"2024-03-30T02:36:27.223677+0000","last_clean":"2024-03-30T02:36:27.223677+0000","last_became_active":"2024-03-30T02:33:56.207820+0000","last_became_peered":"2024-03-30T02:33:56.207820+0000","last_unstale":"2024-03-30T02:36:27.223677+0000","last_undegraded":"2024-03-30T02:36:27.223677+0000","last_fullsized":"2024-03-30T02:36:27.223677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_clean_scrub_stamp":"2024-03-30T02:31:23.922434+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:26.198503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221375+0000","last_change":"2024-03-30T02:35:28.213743+0000","last_active":"2024-03-30T02:36:27.221375+0000","last_peered":"2024-03-30T02:36:27.221375+0000","last_clean":"2024-03-30T02:36:27.221375+0000","last_became_active":"2024-03-30T02:35:28.213318+0000","last_became_peered":"2024-03-30T02:35:28.213318+0000","last_unstale":"2024-03-30T02:36:27.221375+0000","last_undegraded":"2024-03-30T02:36:27.221375+0000","last_fullsized":"2024-03-30T02:36:27.221375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:15:35.803168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.220877+0000","last_change":"2024-03-30T02:34:46.739757+0000","last_active":"2024-03-30T02:36:27.220877+0000","last_peered":"2024-03-30T02:36:27.220877+0000","last_clean":"2024-03-30T02:36:27.220877+0000","last_became_active":"2024-03-30T02:34:46.738269+0000","last_became_peered":"2024-03-30T02:34:46.738269+0000","last_unstale":"2024-03-30T02:36:27.220877+0000","last_undegraded":"2024-03-30T02:36:27.220877+0000","last_fullsized":"2024-03-30T02:36:27.220877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:03.411187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.235101+0000","last_change":"2024-03-30T02:34:46.212532+0000","last_active":"2024-03-30T02:36:28.235101+0000","last_peered":"2024-03-30T02:36:28.235101+0000","last_clean":"2024-03-30T02:36:28.235101+0000","last_became_active":"2024-03-30T02:34:46.212191+0000","last_became_peered":"2024-03-30T02:34:46.212191+0000","last_unstale":"2024-03-30T02:36:28.235101+0000","last_undegraded":"2024-03-30T02:36:28.235101+0000","last_fullsized":"2024-03-30T02:36:28.235101+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:19.120960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228188+0000","last_change":"2024-03-30T02:34:50.759911+0000","last_active":"2024-03-30T02:36:27.228188+0000","last_peered":"2024-03-30T02:36:27.228188+0000","last_clean":"2024-03-30T02:36:27.228188+0000","last_became_active":"2024-03-30T02:34:50.759487+0000","last_became_peered":"2024-03-30T02:34:50.759487+0000","last_unstale":"2024-03-30T02:36:27.228188+0000","last_undegraded":"2024-03-30T02:36:27.228188+0000","last_fullsized":"2024-03-30T02:36:27.228188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:46.403754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231948+0000","last_change":"2024-03-30T02:34:46.217092+0000","last_active":"2024-03-30T02:36:27.231948+0000","last_peered":"2024-03-30T02:36:27.231948+0000","last_clean":"2024-03-30T02:36:27.231948+0000","last_became_active":"2024-03-30T02:34:46.216779+0000","last_became_peered":"2024-03-30T02:34:46.216779+0000","last_unstale":"2024-03-30T02:36:27.231948+0000","last_undegraded":"2024-03-30T02:36:27.231948+0000","last_fullsized":"2024-03-30T02:36:27.231948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:10.894577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221586+0000","last_change":"2024-03-30T02:34:46.735212+0000","last_active":"2024-03-30T02:36:27.221586+0000","last_peered":"2024-03-30T02:36:27.221586+0000","last_clean":"2024-03-30T02:36:27.221586+0000","last_became_active":"2024-03-30T02:34:46.734282+0000","last_became_peered":"2024-03-30T02:34:46.734282+0000","last_unstale":"2024-03-30T02:36:27.221586+0000","last_undegraded":"2024-03-30T02:36:27.221586+0000","last_fullsized":"2024-03-30T02:36:27.221586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:28.596961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224232+0000","last_change":"2024-03-30T02:34:50.799290+0000","last_active":"2024-03-30T02:36:27.224232+0000","last_peered":"2024-03-30T02:36:27.224232+0000","last_clean":"2024-03-30T02:36:27.224232+0000","last_became_active":"2024-03-30T02:34:50.797524+0000","last_became_peered":"2024-03-30T02:34:50.797524+0000","last_unstale":"2024-03-30T02:36:27.224232+0000","last_undegraded":"2024-03-30T02:36:27.224232+0000","last_fullsized":"2024-03-30T02:36:27.224232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:53:27.931444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:31.369328+0000","last_change":"2024-03-30T02:36:28.236366+0000","last_active":"2024-03-30T02:36:31.369328+0000","last_peered":"2024-03-30T02:36:31.369328+0000","last_clean":"2024-03-30T02:36:31.369328+0000","last_became_active":"2024-03-30T02:36:28.228981+0000","last_became_peered":"2024-03-30T02:36:28.228981+0000","last_unstale":"2024-03-30T02:36:31.369328+0000","last_undegraded":"2024-03-30T02:36:31.369328+0000","last_fullsized":"2024-03-30T02:36:31.369328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:28:26.747807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224734+0000","last_change":"2024-03-30T02:34:46.740056+0000","last_active":"2024-03-30T02:36:27.224734+0000","last_peered":"2024-03-30T02:36:27.224734+0000","last_clean":"2024-03-30T02:36:27.224734+0000","last_became_active":"2024-03-30T02:34:46.739260+0000","last_became_peered":"2024-03-30T02:34:46.739260+0000","last_unstale":"2024-03-30T02:36:27.224734+0000","last_undegraded":"2024-03-30T02:36:27.224734+0000","last_fullsized":"2024-03-30T02:36:27.224734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:53.550978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223920+0000","last_change":"2024-03-30T02:34:50.756851+0000","last_active":"2024-03-30T02:36:27.223920+0000","last_peered":"2024-03-30T02:36:27.223920+0000","last_clean":"2024-03-30T02:36:27.223920+0000","last_became_active":"2024-03-30T02:34:50.756416+0000","last_became_peered":"2024-03-30T02:34:50.756416+0000","last_unstale":"2024-03-30T02:36:27.223920+0000","last_undegraded":"2024-03-30T02:36:27.223920+0000","last_fullsized":"2024-03-30T02:36:27.223920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:11.469599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225670+0000","last_change":"2024-03-30T02:34:46.215663+0000","last_active":"2024-03-30T02:36:27.225670+0000","last_peered":"2024-03-30T02:36:27.225670+0000","last_clean":"2024-03-30T02:36:27.225670+0000","last_became_active":"2024-03-30T02:34:46.214224+0000","last_became_peered":"2024-03-30T02:34:46.214224+0000","last_unstale":"2024-03-30T02:36:27.225670+0000","last_undegraded":"2024-03-30T02:36:27.225670+0000","last_fullsized":"2024-03-30T02:36:27.225670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:24:58.734434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605318+0000","last_change":"2024-03-30T02:34:46.754150+0000","last_active":"2024-03-30T02:36:27.605318+0000","last_peered":"2024-03-30T02:36:27.605318+0000","last_clean":"2024-03-30T02:36:27.605318+0000","last_became_active":"2024-03-30T02:34:46.753786+0000","last_became_peered":"2024-03-30T02:34:46.753786+0000","last_unstale":"2024-03-30T02:36:27.605318+0000","last_undegraded":"2024-03-30T02:36:27.605318+0000","last_fullsized":"2024-03-30T02:36:27.605318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:54:50.165020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225526+0000","last_change":"2024-03-30T02:34:50.797890+0000","last_active":"2024-03-30T02:36:27.225526+0000","last_peered":"2024-03-30T02:36:27.225526+0000","last_clean":"2024-03-30T02:36:27.225526+0000","last_became_active":"2024-03-30T02:34:50.797395+0000","last_became_peered":"2024-03-30T02:34:50.797395+0000","last_unstale":"2024-03-30T02:36:27.225526+0000","last_undegraded":"2024-03-30T02:36:27.225526+0000","last_fullsized":"2024-03-30T02:36:27.225526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:36:14.958804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.230907+0000","last_change":"2024-03-30T02:34:46.220873+0000","last_active":"2024-03-30T02:36:27.230907+0000","last_peered":"2024-03-30T02:36:27.230907+0000","last_clean":"2024-03-30T02:36:27.230907+0000","last_became_active":"2024-03-30T02:34:46.220544+0000","last_became_peered":"2024-03-30T02:34:46.220544+0000","last_unstale":"2024-03-30T02:36:27.230907+0000","last_undegraded":"2024-03-30T02:36:27.230907+0000","last_fullsized":"2024-03-30T02:36:27.230907+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:37.691602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231055+0000","last_change":"2024-03-30T02:34:46.739184+0000","last_active":"2024-03-30T02:36:27.231055+0000","last_peered":"2024-03-30T02:36:27.231055+0000","last_clean":"2024-03-30T02:36:27.231055+0000","last_became_active":"2024-03-30T02:34:46.738843+0000","last_became_peered":"2024-03-30T02:34:46.738843+0000","last_unstale":"2024-03-30T02:36:27.231055+0000","last_undegraded":"2024-03-30T02:36:27.231055+0000","last_fullsized":"2024-03-30T02:36:27.231055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:15:09.894754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.229023+0000","last_change":"2024-03-30T02:34:50.768763+0000","last_active":"2024-03-30T02:36:27.229023+0000","last_peered":"2024-03-30T02:36:27.229023+0000","last_clean":"2024-03-30T02:36:27.229023+0000","last_became_active":"2024-03-30T02:34:50.767363+0000","last_became_peered":"2024-03-30T02:34:50.767363+0000","last_unstale":"2024-03-30T02:36:27.229023+0000","last_undegraded":"2024-03-30T02:36:27.229023+0000","last_fullsized":"2024-03-30T02:36:27.229023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:06.932826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224532+0000","last_change":"2024-03-30T02:34:46.213006+0000","last_active":"2024-03-30T02:36:27.224532+0000","last_peered":"2024-03-30T02:36:27.224532+0000","last_clean":"2024-03-30T02:36:27.224532+0000","last_became_active":"2024-03-30T02:34:46.206271+0000","last_became_peered":"2024-03-30T02:34:46.206271+0000","last_unstale":"2024-03-30T02:36:27.224532+0000","last_undegraded":"2024-03-30T02:36:27.224532+0000","last_fullsized":"2024-03-30T02:36:27.224532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:00:36.447352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228446+0000","last_change":"2024-03-30T02:34:46.739566+0000","last_active":"2024-03-30T02:36:27.228446+0000","last_peered":"2024-03-30T02:36:27.228446+0000","last_clean":"2024-03-30T02:36:27.228446+0000","last_became_active":"2024-03-30T02:34:46.739205+0000","last_became_peered":"2024-03-30T02:34:46.739205+0000","last_unstale":"2024-03-30T02:36:27.228446+0000","last_undegraded":"2024-03-30T02:36:27.228446+0000","last_fullsized":"2024-03-30T02:36:27.228446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:01.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224920+0000","last_change":"2024-03-30T02:34:50.758016+0000","last_active":"2024-03-30T02:36:27.224920+0000","last_peered":"2024-03-30T02:36:27.224920+0000","last_clean":"2024-03-30T02:36:27.224920+0000","last_became_active":"2024-03-30T02:34:50.757616+0000","last_became_peered":"2024-03-30T02:34:50.757616+0000","last_unstale":"2024-03-30T02:36:27.224920+0000","last_undegraded":"2024-03-30T02:36:27.224920+0000","last_fullsized":"2024-03-30T02:36:27.224920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:35:07.902347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.240419+0000","last_change":"2024-03-30T02:36:28.240079+0000","last_active":"2024-03-30T02:36:28.240419+0000","last_peered":"2024-03-30T02:36:28.240419+0000","last_clean":"2024-03-30T02:36:28.240419+0000","last_became_active":"2024-03-30T02:36:28.228586+0000","last_became_peered":"2024-03-30T02:36:28.228586+0000","last_unstale":"2024-03-30T02:36:28.240419+0000","last_undegraded":"2024-03-30T02:36:28.240419+0000","last_fullsized":"2024-03-30T02:36:28.240419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:22.476600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224967+0000","last_change":"2024-03-30T02:34:46.754114+0000","last_active":"2024-03-30T02:36:27.224967+0000","last_peered":"2024-03-30T02:36:27.224967+0000","last_clean":"2024-03-30T02:36:27.224967+0000","last_became_active":"2024-03-30T02:34:46.752691+0000","last_became_peered":"2024-03-30T02:34:46.752691+0000","last_unstale":"2024-03-30T02:36:27.224967+0000","last_undegraded":"2024-03-30T02:36:27.224967+0000","last_fullsized":"2024-03-30T02:36:27.224967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:42.908877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225234+0000","last_change":"2024-03-30T02:34:50.798110+0000","last_active":"2024-03-30T02:36:27.225234+0000","last_peered":"2024-03-30T02:36:27.225234+0000","last_clean":"2024-03-30T02:36:27.225234+0000","last_became_active":"2024-03-30T02:34:50.797551+0000","last_became_peered":"2024-03-30T02:34:50.797551+0000","last_unstale":"2024-03-30T02:36:27.225234+0000","last_undegraded":"2024-03-30T02:36:27.225234+0000","last_fullsized":"2024-03-30T02:36:27.225234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:49.362018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231216+0000","last_change":"2024-03-30T02:34:46.211647+0000","last_active":"2024-03-30T02:36:27.231216+0000","last_peered":"2024-03-30T02:36:27.231216+0000","last_clean":"2024-03-30T02:36:27.231216+0000","last_became_active":"2024-03-30T02:34:46.211311+0000","last_became_peered":"2024-03-30T02:34:46.211311+0000","last_unstale":"2024-03-30T02:36:27.231216+0000","last_undegraded":"2024-03-30T02:36:27.231216+0000","last_fullsized":"2024-03-30T02:36:27.231216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:05:34.135289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224076+0000","last_change":"2024-03-30T02:34:46.740256+0000","last_active":"2024-03-30T02:36:27.224076+0000","last_peered":"2024-03-30T02:36:27.224076+0000","last_clean":"2024-03-30T02:36:27.224076+0000","last_became_active":"2024-03-30T02:34:46.739913+0000","last_became_peered":"2024-03-30T02:34:46.739913+0000","last_unstale":"2024-03-30T02:36:27.224076+0000","last_undegraded":"2024-03-30T02:36:27.224076+0000","last_fullsized":"2024-03-30T02:36:27.224076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:44.152475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.230907+0000","last_change":"2024-03-30T02:34:50.762149+0000","last_active":"2024-03-30T02:36:27.230907+0000","last_peered":"2024-03-30T02:36:27.230907+0000","last_clean":"2024-03-30T02:36:27.230907+0000","last_became_active":"2024-03-30T02:34:50.761717+0000","last_became_peered":"2024-03-30T02:34:50.761717+0000","last_unstale":"2024-03-30T02:36:27.230907+0000","last_undegraded":"2024-03-30T02:36:27.230907+0000","last_fullsized":"2024-03-30T02:36:27.230907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:22.373079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":111,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:40.253059+0000","last_change":"2024-03-30T02:34:46.224098+0000","last_active":"2024-03-30T02:36:40.253059+0000","last_peered":"2024-03-30T02:36:40.253059+0000","last_clean":"2024-03-30T02:36:40.253059+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:36:40.253059+0000","last_undegraded":"2024-03-30T02:36:40.253059+0000","last_fullsized":"2024-03-30T02:36:40.253059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:20:25.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224527+0000","last_change":"2024-03-30T02:34:46.737661+0000","last_active":"2024-03-30T02:36:27.224527+0000","last_peered":"2024-03-30T02:36:27.224527+0000","last_clean":"2024-03-30T02:36:27.224527+0000","last_became_active":"2024-03-30T02:34:46.737303+0000","last_became_peered":"2024-03-30T02:34:46.737303+0000","last_unstale":"2024-03-30T02:36:27.224527+0000","last_undegraded":"2024-03-30T02:36:27.224527+0000","last_fullsized":"2024-03-30T02:36:27.224527+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:44.929643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.604791+0000","last_change":"2024-03-30T02:34:50.767119+0000","last_active":"2024-03-30T02:36:27.604791+0000","last_peered":"2024-03-30T02:36:27.604791+0000","last_clean":"2024-03-30T02:36:27.604791+0000","last_became_active":"2024-03-30T02:34:50.766472+0000","last_became_peered":"2024-03-30T02:34:50.766472+0000","last_unstale":"2024-03-30T02:36:27.604791+0000","last_undegraded":"2024-03-30T02:36:27.604791+0000","last_fullsized":"2024-03-30T02:36:27.604791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:46.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221554+0000","last_change":"2024-03-30T02:34:46.216713+0000","last_active":"2024-03-30T02:36:27.221554+0000","last_peered":"2024-03-30T02:36:27.221554+0000","last_clean":"2024-03-30T02:36:27.221554+0000","last_became_active":"2024-03-30T02:34:46.216366+0000","last_became_peered":"2024-03-30T02:34:46.216366+0000","last_unstale":"2024-03-30T02:36:27.221554+0000","last_undegraded":"2024-03-30T02:36:27.221554+0000","last_fullsized":"2024-03-30T02:36:27.221554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:43.803955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224174+0000","last_change":"2024-03-30T02:34:46.736854+0000","last_active":"2024-03-30T02:36:27.224174+0000","last_peered":"2024-03-30T02:36:27.224174+0000","last_clean":"2024-03-30T02:36:27.224174+0000","last_became_active":"2024-03-30T02:34:46.736511+0000","last_became_peered":"2024-03-30T02:34:46.736511+0000","last_unstale":"2024-03-30T02:36:27.224174+0000","last_undegraded":"2024-03-30T02:36:27.224174+0000","last_fullsized":"2024-03-30T02:36:27.224174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:12.701467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221087+0000","last_change":"2024-03-30T02:34:50.757227+0000","last_active":"2024-03-30T02:36:27.221087+0000","last_peered":"2024-03-30T02:36:27.221087+0000","last_clean":"2024-03-30T02:36:27.221087+0000","last_became_active":"2024-03-30T02:34:50.756742+0000","last_became_peered":"2024-03-30T02:34:50.756742+0000","last_unstale":"2024-03-30T02:36:27.221087+0000","last_undegraded":"2024-03-30T02:36:27.221087+0000","last_fullsized":"2024-03-30T02:36:27.221087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:48.792202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.243069+0000","last_change":"2024-03-30T02:36:28.243069+0000","last_active":"2024-03-30T02:36:28.243069+0000","last_peered":"2024-03-30T02:36:28.243069+0000","last_clean":"2024-03-30T02:36:28.243069+0000","last_became_active":"2024-03-30T02:36:28.238658+0000","last_became_peered":"2024-03-30T02:36:28.238658+0000","last_unstale":"2024-03-30T02:36:28.243069+0000","last_undegraded":"2024-03-30T02:36:28.243069+0000","last_fullsized":"2024-03-30T02:36:28.243069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:47:59.741479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224872+0000","last_change":"2024-03-30T02:34:46.741735+0000","last_active":"2024-03-30T02:36:27.224872+0000","last_peered":"2024-03-30T02:36:27.224872+0000","last_clean":"2024-03-30T02:36:27.224872+0000","last_became_active":"2024-03-30T02:34:46.740663+0000","last_became_peered":"2024-03-30T02:34:46.740663+0000","last_unstale":"2024-03-30T02:36:27.224872+0000","last_undegraded":"2024-03-30T02:36:27.224872+0000","last_fullsized":"2024-03-30T02:36:27.224872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:09.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228383+0000","last_change":"2024-03-30T02:34:46.739082+0000","last_active":"2024-03-30T02:36:27.228383+0000","last_peered":"2024-03-30T02:36:27.228383+0000","last_clean":"2024-03-30T02:36:27.228383+0000","last_became_active":"2024-03-30T02:34:46.738732+0000","last_became_peered":"2024-03-30T02:34:46.738732+0000","last_unstale":"2024-03-30T02:36:27.228383+0000","last_undegraded":"2024-03-30T02:36:27.228383+0000","last_fullsized":"2024-03-30T02:36:27.228383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:53.961520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224158+0000","last_change":"2024-03-30T02:35:28.211005+0000","last_active":"2024-03-30T02:36:27.224158+0000","last_peered":"2024-03-30T02:36:27.224158+0000","last_clean":"2024-03-30T02:36:27.224158+0000","last_became_active":"2024-03-30T02:35:28.210610+0000","last_became_peered":"2024-03-30T02:35:28.210610+0000","last_unstale":"2024-03-30T02:36:27.224158+0000","last_undegraded":"2024-03-30T02:36:27.224158+0000","last_fullsized":"2024-03-30T02:36:27.224158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:53.054107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223977+0000","last_change":"2024-03-30T02:34:46.742590+0000","last_active":"2024-03-30T02:36:27.223977+0000","last_peered":"2024-03-30T02:36:27.223977+0000","last_clean":"2024-03-30T02:36:27.223977+0000","last_became_active":"2024-03-30T02:34:46.740904+0000","last_became_peered":"2024-03-30T02:34:46.740904+0000","last_unstale":"2024-03-30T02:36:27.223977+0000","last_undegraded":"2024-03-30T02:36:27.223977+0000","last_fullsized":"2024-03-30T02:36:27.223977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:22.653881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605025+0000","last_change":"2024-03-30T02:34:46.753119+0000","last_active":"2024-03-30T02:36:27.605025+0000","last_peered":"2024-03-30T02:36:27.605025+0000","last_clean":"2024-03-30T02:36:27.605025+0000","last_became_active":"2024-03-30T02:34:46.745680+0000","last_became_peered":"2024-03-30T02:34:46.745680+0000","last_unstale":"2024-03-30T02:36:27.605025+0000","last_undegraded":"2024-03-30T02:36:27.605025+0000","last_fullsized":"2024-03-30T02:36:27.605025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:02.352961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221594+0000","last_change":"2024-03-30T02:35:28.313317+0000","last_active":"2024-03-30T02:36:27.221594+0000","last_peered":"2024-03-30T02:36:27.221594+0000","last_clean":"2024-03-30T02:36:27.221594+0000","last_became_active":"2024-03-30T02:35:28.312929+0000","last_became_peered":"2024-03-30T02:35:28.312929+0000","last_unstale":"2024-03-30T02:36:27.221594+0000","last_undegraded":"2024-03-30T02:36:27.221594+0000","last_fullsized":"2024-03-30T02:36:27.221594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:06.390076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228760+0000","last_change":"2024-03-30T02:34:46.737882+0000","last_active":"2024-03-30T02:36:27.228760+0000","last_peered":"2024-03-30T02:36:27.228760+0000","last_clean":"2024-03-30T02:36:27.228760+0000","last_became_active":"2024-03-30T02:34:46.737546+0000","last_became_peered":"2024-03-30T02:34:46.737546+0000","last_unstale":"2024-03-30T02:36:27.228760+0000","last_undegraded":"2024-03-30T02:36:27.228760+0000","last_fullsized":"2024-03-30T02:36:27.228760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:48:41.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224639+0000","last_change":"2024-03-30T02:34:46.742703+0000","last_active":"2024-03-30T02:36:27.224639+0000","last_peered":"2024-03-30T02:36:27.224639+0000","last_clean":"2024-03-30T02:36:27.224639+0000","last_became_active":"2024-03-30T02:34:46.740689+0000","last_became_peered":"2024-03-30T02:34:46.740689+0000","last_unstale":"2024-03-30T02:36:27.224639+0000","last_undegraded":"2024-03-30T02:36:27.224639+0000","last_fullsized":"2024-03-30T02:36:27.224639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:45.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224909+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:36:27.224909+0000","last_peered":"2024-03-30T02:36:27.224909+0000","last_clean":"2024-03-30T02:36:27.224909+0000","last_became_active":"2024-03-30T02:34:46.750395+0000","last_became_peered":"2024-03-30T02:34:46.750395+0000","last_unstale":"2024-03-30T02:36:27.224909+0000","last_undegraded":"2024-03-30T02:36:27.224909+0000","last_fullsized":"2024-03-30T02:36:27.224909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:01.282677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221281+0000","last_change":"2024-03-30T02:34:46.734814+0000","last_active":"2024-03-30T02:36:27.221281+0000","last_peered":"2024-03-30T02:36:27.221281+0000","last_clean":"2024-03-30T02:36:27.221281+0000","last_became_active":"2024-03-30T02:34:46.733978+0000","last_became_peered":"2024-03-30T02:34:46.733978+0000","last_unstale":"2024-03-30T02:36:27.221281+0000","last_undegraded":"2024-03-30T02:36:27.221281+0000","last_fullsized":"2024-03-30T02:36:27.221281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:31:39.467478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224665+0000","last_change":"2024-03-30T02:34:46.748163+0000","last_active":"2024-03-30T02:36:27.224665+0000","last_peered":"2024-03-30T02:36:27.224665+0000","last_clean":"2024-03-30T02:36:27.224665+0000","last_became_active":"2024-03-30T02:34:46.738845+0000","last_became_peered":"2024-03-30T02:34:46.738845+0000","last_unstale":"2024-03-30T02:36:27.224665+0000","last_undegraded":"2024-03-30T02:36:27.224665+0000","last_fullsized":"2024-03-30T02:36:27.224665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:09.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231257+0000","last_change":"2024-03-30T02:34:46.743231+0000","last_active":"2024-03-30T02:36:27.231257+0000","last_peered":"2024-03-30T02:36:27.231257+0000","last_clean":"2024-03-30T02:36:27.231257+0000","last_became_active":"2024-03-30T02:34:46.742619+0000","last_became_peered":"2024-03-30T02:34:46.742619+0000","last_unstale":"2024-03-30T02:36:27.231257+0000","last_undegraded":"2024-03-30T02:36:27.231257+0000","last_fullsized":"2024-03-30T02:36:27.231257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:17:51.789139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221198+0000","last_change":"2024-03-30T02:34:46.745872+0000","last_active":"2024-03-30T02:36:27.221198+0000","last_peered":"2024-03-30T02:36:27.221198+0000","last_clean":"2024-03-30T02:36:27.221198+0000","last_became_active":"2024-03-30T02:34:46.745476+0000","last_became_peered":"2024-03-30T02:34:46.745476+0000","last_unstale":"2024-03-30T02:36:27.221198+0000","last_undegraded":"2024-03-30T02:36:27.221198+0000","last_fullsized":"2024-03-30T02:36:27.221198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:30.774316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228267+0000","last_change":"2024-03-30T02:34:46.746178+0000","last_active":"2024-03-30T02:36:27.228267+0000","last_peered":"2024-03-30T02:36:27.228267+0000","last_clean":"2024-03-30T02:36:27.228267+0000","last_became_active":"2024-03-30T02:34:46.745865+0000","last_became_peered":"2024-03-30T02:34:46.745865+0000","last_unstale":"2024-03-30T02:36:27.228267+0000","last_undegraded":"2024-03-30T02:36:27.228267+0000","last_fullsized":"2024-03-30T02:36:27.228267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:22.613637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248704,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76005,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":889,"num_read_kb":530,"num_write":259,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":228408,"data_compressed":5484,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579938,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":488244,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.746,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72299999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58299999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78700000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.34499999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.755,"15min":0.755},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72299999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70999999999999996}]},{"osd":3,"last update":"Sat Mar 30 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.749},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.72899999999999998}]},{"osd":4,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.82499999999999996,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.85099999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.40899999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.38800000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":1.089,"15min":1.089},"last":0.877}]},{"osd":6,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83999999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.323,"15min":1.323},"last":0.78900000000000003}]},{"osd":7,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88700000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.51400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91200000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.93799999999999994}]},{"osd":8,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.0229999999999999}]},{"osd":9,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.754,"15min":0.754},"min":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89700000000000002,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.60299999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96799999999999997,"5min":1.25,"15min":1.25},"last":0.96399999999999997}]},{"osd":10,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.629,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.89200000000000002,"5min":1.206,"15min":1.206},"last":0.91400000000000003}]}]},{"osd":10,"up_from":67,"seq":287762808870,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1114112,"data_stored":1143296,"data_compressed":61402,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.623}]},{"osd":1,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.312}]},{"osd":2,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.498,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66700000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.35599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.38}]},{"osd":5,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.65200000000000002}]},{"osd":6,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":8,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.59599999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69099999999999995}]},{"osd":9,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.60799999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75800000000000001,"15min":0.76100000000000001},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.753}]},{"osd":11,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.754,"15min":0.748},"min":{"1min":0.64300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.74199999999999999}]}]},{"osd":9,"up_from":62,"seq":266287972393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":491619,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.77000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":6.774,"5min":6.774,"15min":6.774},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.52100000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.56399999999999995}]},{"osd":3,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.278,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78200000000000003},"min":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.038,"5min":1.099,"15min":1.099},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":0.878},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3149999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.747},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.72199999999999998}]},{"osd":7,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.91200000000000003,"15min":0.93200000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3879999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.96199999999999997},"min":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.73299999999999998}]},{"osd":8,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88700000000000001,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.79300000000000004}]},{"osd":10,"last update":"Sat Mar 30 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.94499999999999995,"15min":0.95299999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84599999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83299999999999996}]},{"osd":11,"last update":"Sat Mar 30 02:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.81499999999999995}]}]},{"osd":8,"up_from":55,"seq":236223201325,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":487570,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.72399999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.45100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83099999999999996,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.59399999999999997,"15min":0.56599999999999995},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.53300000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.77100000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.38400000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.76200000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.45300000000000001,"15min":0.43099999999999999},"min":{"1min":0.318,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.62,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.63300000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.48299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.876,"15min":0.876},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.51100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.68300000000000005}]},{"osd":4,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.54800000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.55200000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89100000000000001,"5min":0.995,"15min":0.995},"last":0.83099999999999996}]},{"osd":5,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53000000000000003,"15min":0.499},"min":{"1min":0.436,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.71499999999999997}]},{"osd":6,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.73399999999999999,"15min":0.68200000000000005},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.499,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.65700000000000003,"15min":0.63500000000000001},"min":{"1min":0.56499999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.71499999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96399999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.90000000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.55500000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.751,"15min":0.73799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.91900000000000004}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.88700000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.69799999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92100000000000004,"5min":1.091,"15min":1.091},"last":0.94299999999999995}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.873}]}]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1084658,"data_compressed":59574,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42399999999999999,"15min":0.41899999999999998},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.442,"15min":0.436},"min":{"1min":0.35699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.56399999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.47499999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.504},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72999999999999998,"5min":1.034,"15min":1.034},"last":0.52500000000000002}]},{"osd":3,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.63400000000000001}]},{"osd":4,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.75,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59999999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41999999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85199999999999998,"5min":0.878,"15min":0.878},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83199999999999996,"5min":0.877,"15min":0.877},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83699999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.62}]},{"osd":8,"last update":"Sat Mar 30 02:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.55200000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70699999999999996}]},{"osd":9,"last update":"Sat Mar 30 02:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.59699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.71199999999999997},"min":{"1min":0.61799999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.753}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88500000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73099999999999998}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73799999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":962560,"data_stored":482979,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97699999999999998,"5min":0.998,"15min":0.998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39400000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.75800000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.069,"15min":1.069},"last":0.622}]},{"osd":3,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96199999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.74099999999999999},"min":{"1min":0.45700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1040000000000001,"5min":1.236,"15min":1.236},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":1.069,"15min":1.069},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0329999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0289999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.871},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96599999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Mar 30 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.83899999999999997,"15min":0.84699999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.111,"5min":1.393,"15min":1.393},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83899999999999997,"15min":0.84699999999999998},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":0.82899999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:36:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.89600000000000002,"15min":0.91600000000000004},"min":{"1min":0.62,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0700000000000001,"5min":1.355,"15min":1.355},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.875,"15min":0.88800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.72099999999999997}]},{"osd":9,"last update":"Sat Mar 30 02:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93000000000000005,"15min":0.96199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2030000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.88600000000000001,"15min":0.92000000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.70099999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.876,"15min":0.872},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93200000000000005,"15min":0.93100000000000005},"min":{"1min":0.745,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.24,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.81599999999999995}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":552410,"data_compressed":25402,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.75800000000000001},"min":{"1min":0.42599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59899999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57199999999999995,"15min":0.52400000000000002},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77500000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.52400000000000002},"min":{"1min":0.38500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92100000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.64400000000000002}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.438,"5min":0.378,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":1.028,"15min":1.028},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.56000000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.442,"15min":0.40799999999999997},"min":{"1min":0.33200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.433},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.313,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45000000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.64800000000000002,"15min":0.59399999999999997},"min":{"1min":0.54100000000000004,"5min":0.372,"15min":0.372},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.026},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59799999999999998,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.83999999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.62}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65700000000000003,"15min":0.63800000000000001},"min":{"1min":0.51600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77200000000000002}]},{"osd":7,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70699999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93600000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.73899999999999999,"15min":0.72899999999999998},"min":{"1min":0.58699999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.044,"15min":1.044},"last":0.81200000000000006}]},{"osd":9,"last update":"Sat Mar 30 02:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.57499999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.59799999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94899999999999995,"5min":1.056,"15min":1.056},"last":0.73899999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.58499999999999996,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.48799999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.874,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.75800000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":485134,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 02:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.54800000000000004},"min":{"1min":0.41999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.61599999999999999,"15min":0.55500000000000005},"min":{"1min":0.441,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.50900000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63200000000000001,"15min":0.59599999999999997},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.64300000000000002,"15min":0.60099999999999998},"min":{"1min":0.48599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.873}]},{"osd":3,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.42999999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.65600000000000003,"15min":0.63600000000000001},"min":{"1min":0.49099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63900000000000001,"15min":0.61799999999999999},"min":{"1min":0.51900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":1.006},"last":0.65600000000000003}]},{"osd":5,"last update":"Sat Mar 30 02:36:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.70999999999999996,"15min":0.66900000000000004},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.83699999999999997}]},{"osd":6,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84299999999999997,"5min":1.091,"15min":1.091},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.71199999999999997}]},{"osd":8,"last update":"Sat Mar 30 02:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.70999999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.84199999999999997,"15min":0.82599999999999996},"min":{"1min":0.60699999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.90600000000000003}]},{"osd":9,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0629999999999999,"5min":1.212,"15min":1.212},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.879,"15min":0.876},"min":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.79200000000000004}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.874,"15min":0.86499999999999999},"min":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.66800000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.621}]},{"osd":11,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.874,"15min":0.85199999999999998},"min":{"1min":0.76900000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.016,"5min":0.89300000000000002,"15min":0.873},"min":{"1min":0.76500000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.88500000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":553029,"data_compressed":25402,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.52200000000000002},"min":{"1min":0.41299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82899999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.51300000000000001},"min":{"1min":0.42499999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.41999999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85099999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59299999999999997,"15min":0.54300000000000004},"min":{"1min":0.46600000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.83699999999999997}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84599999999999997,"5min":0.878,"15min":0.878},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Mar 30 02:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.498,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.74099999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.58099999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85199999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59599999999999997}]},{"osd":7,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.876}]},{"osd":8,"last update":"Sat Mar 30 02:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.627,"15min":0.60099999999999998},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.747},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.63600000000000001}]},{"osd":9,"last update":"Sat Mar 30 02:36:39 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92000000000000004,"5min":1.292,"15min":1.292},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.86499999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.67800000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.755,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.90700000000000003}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.80200000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":974848,"data_stored":482735,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47099999999999997,"15min":0.40699999999999997},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.434},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.39700000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.60799999999999998,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.66100000000000003,"15min":0.58499999999999996},"min":{"1min":0.47299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.80100000000000005}]},{"osd":2,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.66700000000000004,"15min":0.623},"min":{"1min":0.56899999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.64800000000000002,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.73099999999999998,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.443,"15min":0.443},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.55900000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.628}]},{"osd":5,"last update":"Sat Mar 30 02:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.70999999999999996,"15min":0.65800000000000003},"min":{"1min":0.57899999999999996,"5min":0.432,"15min":0.432},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.79100000000000004,"15min":0.71799999999999997},"min":{"1min":0.64400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Sat Mar 30 02:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.80000000000000004,"15min":0.78000000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.47799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":0.53600000000000003}]},{"osd":7,"last update":"Sat Mar 30 02:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.73299999999999998,"15min":0.70599999999999996},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.90700000000000003,"15min":0.90300000000000002},"min":{"1min":0.58699999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":0.86599999999999999}]},{"osd":8,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.004}]},{"osd":9,"last update":"Sat Mar 30 02:36:39 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94199999999999995},"min":{"1min":0.66800000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2769999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.877,"15min":0.86699999999999999},"min":{"1min":0.57899999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.70599999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:35:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.871,"15min":0.83799999999999997},"min":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.97599999999999998,"15min":0.96199999999999997},"min":{"1min":0.71399999999999997,"5min":0.629,"15min":0.629},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.90400000000000003}]},{"osd":11,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.751},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.68700000000000006}]}]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":970752,"data_stored":487587,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.38,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.747,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.35299999999999998},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.63200000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.38,"15min":0.35599999999999998},"min":{"1min":0.32300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.748,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.52600000000000002},"min":{"1min":0.45500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.438}]},{"osd":5,"last update":"Sat Mar 30 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76000000000000001,"5min":0.878,"15min":0.878},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.56499999999999995},"min":{"1min":0.48499999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72199999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.625}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Mar 30 02:36:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82399999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.503},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.40899999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.496}]},{"osd":8,"last update":"Sat Mar 30 02:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92400000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69799999999999995,"15min":0.67500000000000004},"min":{"1min":0.53100000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.72699999999999998}]},{"osd":9,"last update":"Sat Mar 30 02:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.55300000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81100000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96099999999999997,"5min":1.079,"15min":1.079},"last":0.64900000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.50900000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92900000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.74199999999999999}]},{"osd":11,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78200000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":1044480,"data_stored":1080075,"data_compressed":59574,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.42799999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54100000000000004,"15min":0.51100000000000001},"min":{"1min":0.44700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.56299999999999994,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80600000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64000000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.37},"min":{"1min":0.35199999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.65100000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.36799999999999999},"min":{"1min":0.35299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.372}]},{"osd":3,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55200000000000005},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84899999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.58799999999999997},"min":{"1min":0.47499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.627,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84799999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.67600000000000005}]},{"osd":6,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54000000000000004}]},{"osd":7,"last update":"Sat Mar 30 02:36:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81000000000000005,"5min":1.006,"15min":1.006},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.625,"15min":0.60099999999999998},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.77400000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.73899999999999999}]},{"osd":8,"last update":"Sat Mar 30 02:36:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.437,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58199999999999996,"15min":0.56899999999999995},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52300000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.55400000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.622,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65700000000000003}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.86599999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.747,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.871,"5min":1.024,"15min":1.024},"last":0.80600000000000005}]},{"osd":11,"last update":"Sat Mar 30 02:36:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70105,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":70724,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":70623,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T02:36:48.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T02:36:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:48 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2070338989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:48.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:48 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/888189465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.513+0000 7f017be00700 1 -- 172.21.15.67:0/3567309035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174101e70 msgr2=0x7f01741022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.513+0000 7f017be00700 1 --2- 172.21.15.67:0/3567309035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174101e70 0x7f01741022d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f01600099c0 tx=0x7f0160009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.513+0000 7f017be00700 1 -- 172.21.15.67:0/3567309035 shutdown_connections 2024-03-30T02:36:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.513+0000 7f017be00700 1 --2- 172.21.15.67:0/3567309035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0174102810 0x7f01741090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.513+0000 7f017be00700 1 --2- 172.21.15.67:0/3567309035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0174100c80 0x7f01741010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.513+0000 7f017be00700 1 --2- 172.21.15.67:0/3567309035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174101e70 0x7f01741022d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.513+0000 7f017be00700 1 -- 172.21.15.67:0/3567309035 >> 172.21.15.67:0/3567309035 conn(0x7f01740fc1f0 msgr2=0x7f01740fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:48.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.513+0000 7f017be00700 1 -- 172.21.15.67:0/3567309035 shutdown_connections 2024-03-30T02:36:48.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.513+0000 7f017be00700 1 -- 172.21.15.67:0/3567309035 wait complete. 2024-03-30T02:36:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.514+0000 7f017be00700 1 Processor -- start 2024-03-30T02:36:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.514+0000 7f017be00700 1 -- start start 2024-03-30T02:36:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f017be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174100c80 0x7f0174074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f017be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0174101e70 0x7f01740731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f017be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0174102810 0x7f01740736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f017be00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0174073e80 con 0x7f0174100c80 2024-03-30T02:36:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f017be00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0174073ff0 con 0x7f0174101e70 2024-03-30T02:36:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f017be00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01741a0c20 con 0x7f0174102810 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0179400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174100c80 0x7f0174074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0179400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174100c80 0x7f0174074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56666/0 (socket says 172.21.15.67:56666) 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0179400700 1 -- 172.21.15.67:0/1011874569 learned_addr learned my addr 172.21.15.67:0/1011874569 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0178a00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0174101e70 0x7f01740731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0179e00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0174102810 0x7f01740736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0179400700 1 -- 172.21.15.67:0/1011874569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0174102810 msgr2=0x7f01740736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0179400700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0174102810 0x7f01740736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0179400700 1 -- 172.21.15.67:0/1011874569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0174101e70 msgr2=0x7f01740731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0179400700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0174101e70 0x7f01740731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.515+0000 7f0179400700 1 -- 172.21.15.67:0/1011874569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0168009520 con 0x7f0174100c80 2024-03-30T02:36:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.516+0000 7f0179400700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174100c80 0x7f0174074b10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f016800ea50 tx=0x7f016800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.516+0000 7f0172000700 1 -- 172.21.15.67:0/1011874569 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016800cb70 con 0x7f0174100c80 2024-03-30T02:36:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.516+0000 7f0172000700 1 -- 172.21.15.67:0/1011874569 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016800ccd0 con 0x7f0174100c80 2024-03-30T02:36:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.516+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01600095f0 con 0x7f0174100c80 2024-03-30T02:36:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.516+0000 7f0172000700 1 -- 172.21.15.67:0/1011874569 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168018680 con 0x7f0174100c80 2024-03-30T02:36:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.516+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01741a1180 con 0x7f0174100c80 2024-03-30T02:36:48.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.517+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0174067a80 con 0x7f0174100c80 2024-03-30T02:36:48.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.519+0000 7f0172000700 1 -- 172.21.15.67:0/1011874569 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f0168010430 con 0x7f0174100c80 2024-03-30T02:36:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.520+0000 7f0172000700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f015c0a11b0 0x7f015c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.520+0000 7f0172000700 1 -- 172.21.15.67:0/1011874569 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f0168014070 con 0x7f0174100c80 2024-03-30T02:36:48.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.520+0000 7f0178a00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f015c0a11b0 0x7f015c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:48.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.521+0000 7f0178a00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f015c0a11b0 0x7f015c0a3670 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0160003730 tx=0x7f016000b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:48.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.524+0000 7f0172000700 1 -- 172.21.15.67:0/1011874569 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0174067a80 con 0x7f0174100c80 2024-03-30T02:36:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:48 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2070338989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:48 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/888189465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:48 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2070338989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:48 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/888189465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:48.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.739+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0174067a10 con 0x7f015c0a11b0 2024-03-30T02:36:48.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.752+0000 7f0172000700 1 -- 172.21.15.67:0/1011874569 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428252 (secure 0 0 0) 0x7f0174067a10 con 0x7f015c0a11b0 2024-03-30T02:36:48.759 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:48.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f015c0a11b0 msgr2=0x7f015c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f015c0a11b0 0x7f015c0a3670 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0160003730 tx=0x7f016000b540 comp rx=0 tx=0).stop 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174100c80 msgr2=0x7f0174074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174100c80 0x7f0174074b10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f016800ea50 tx=0x7f016800ed60 comp rx=0 tx=0).stop 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 shutdown_connections 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f015c0a11b0 0x7f015c0a3670 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0174100c80 0x7f0174074b10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0174102810 0x7f01740736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 --2- 172.21.15.67:0/1011874569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0174101e70 0x7f01740731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 >> 172.21.15.67:0/1011874569 conn(0x7f01740fc1f0 msgr2=0x7f0174105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 shutdown_connections 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:48.757+0000 7f017be00700 1 -- 172.21.15.67:0/1011874569 wait complete. 2024-03-30T02:36:48.763 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-30T02:36:48.776 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-03-30T02:36:47.028962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666607,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1025,"num_read_kb":649,"num_write":442,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360624,"kb_used_data":11820,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581200,"statfs":{"total":1151940427776,"available":1151571148800,"internally_reserved":0,"allocated":12103680,"data_stored":7819336,"data_compressed":396372,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19130,"internal_metadata":357020998},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5319,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":63,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002916"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228788+0000","last_change":"2024-03-30T02:34:46.755257+0000","last_active":"2024-03-30T02:36:27.228788+0000","last_peered":"2024-03-30T02:36:27.228788+0000","last_clean":"2024-03-30T02:36:27.228788+0000","last_became_active":"2024-03-30T02:34:46.754922+0000","last_became_peered":"2024-03-30T02:34:46.754922+0000","last_unstale":"2024-03-30T02:36:27.228788+0000","last_undegraded":"2024-03-30T02:36:27.228788+0000","last_fullsized":"2024-03-30T02:36:27.228788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:09.980124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605237+0000","last_change":"2024-03-30T02:34:46.742846+0000","last_active":"2024-03-30T02:36:27.605237+0000","last_peered":"2024-03-30T02:36:27.605237+0000","last_clean":"2024-03-30T02:36:27.605237+0000","last_became_active":"2024-03-30T02:34:46.738096+0000","last_became_peered":"2024-03-30T02:34:46.738096+0000","last_unstale":"2024-03-30T02:36:27.605237+0000","last_undegraded":"2024-03-30T02:36:27.605237+0000","last_fullsized":"2024-03-30T02:36:27.605237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:43:16.982251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605205+0000","last_change":"2024-03-30T02:34:46.739607+0000","last_active":"2024-03-30T02:36:27.605205+0000","last_peered":"2024-03-30T02:36:27.605205+0000","last_clean":"2024-03-30T02:36:27.605205+0000","last_became_active":"2024-03-30T02:34:46.739230+0000","last_became_peered":"2024-03-30T02:34:46.739230+0000","last_unstale":"2024-03-30T02:36:27.605205+0000","last_undegraded":"2024-03-30T02:36:27.605205+0000","last_fullsized":"2024-03-30T02:36:27.605205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:17.818874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225333+0000","last_change":"2024-03-30T02:34:46.742118+0000","last_active":"2024-03-30T02:36:27.225333+0000","last_peered":"2024-03-30T02:36:27.225333+0000","last_clean":"2024-03-30T02:36:27.225333+0000","last_became_active":"2024-03-30T02:34:46.740845+0000","last_became_peered":"2024-03-30T02:34:46.740845+0000","last_unstale":"2024-03-30T02:36:27.225333+0000","last_undegraded":"2024-03-30T02:36:27.225333+0000","last_fullsized":"2024-03-30T02:36:27.225333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:33:28.595456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231314+0000","last_change":"2024-03-30T02:34:46.739557+0000","last_active":"2024-03-30T02:36:27.231314+0000","last_peered":"2024-03-30T02:36:27.231314+0000","last_clean":"2024-03-30T02:36:27.231314+0000","last_became_active":"2024-03-30T02:34:46.739147+0000","last_became_peered":"2024-03-30T02:34:46.739147+0000","last_unstale":"2024-03-30T02:36:27.231314+0000","last_undegraded":"2024-03-30T02:36:27.231314+0000","last_fullsized":"2024-03-30T02:36:27.231314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:12:52.307386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.227799+0000","last_change":"2024-03-30T02:35:28.211536+0000","last_active":"2024-03-30T02:36:27.227799+0000","last_peered":"2024-03-30T02:36:27.227799+0000","last_clean":"2024-03-30T02:36:27.227799+0000","last_became_active":"2024-03-30T02:35:28.211085+0000","last_became_peered":"2024-03-30T02:35:28.211085+0000","last_unstale":"2024-03-30T02:36:27.227799+0000","last_undegraded":"2024-03-30T02:36:27.227799+0000","last_fullsized":"2024-03-30T02:36:27.227799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:06:34.034277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223715+0000","last_change":"2024-03-30T02:34:46.735466+0000","last_active":"2024-03-30T02:36:27.223715+0000","last_peered":"2024-03-30T02:36:27.223715+0000","last_clean":"2024-03-30T02:36:27.223715+0000","last_became_active":"2024-03-30T02:34:46.734411+0000","last_became_peered":"2024-03-30T02:34:46.734411+0000","last_unstale":"2024-03-30T02:36:27.223715+0000","last_undegraded":"2024-03-30T02:36:27.223715+0000","last_fullsized":"2024-03-30T02:36:27.223715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:57.587768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224366+0000","last_change":"2024-03-30T02:34:46.747001+0000","last_active":"2024-03-30T02:36:27.224366+0000","last_peered":"2024-03-30T02:36:27.224366+0000","last_clean":"2024-03-30T02:36:27.224366+0000","last_became_active":"2024-03-30T02:34:46.740241+0000","last_became_peered":"2024-03-30T02:34:46.740241+0000","last_unstale":"2024-03-30T02:36:27.224366+0000","last_undegraded":"2024-03-30T02:36:27.224366+0000","last_fullsized":"2024-03-30T02:36:27.224366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:05.902190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231395+0000","last_change":"2024-03-30T02:34:46.745549+0000","last_active":"2024-03-30T02:36:27.231395+0000","last_peered":"2024-03-30T02:36:27.231395+0000","last_clean":"2024-03-30T02:36:27.231395+0000","last_became_active":"2024-03-30T02:34:46.745200+0000","last_became_peered":"2024-03-30T02:34:46.745200+0000","last_unstale":"2024-03-30T02:36:27.231395+0000","last_undegraded":"2024-03-30T02:36:27.231395+0000","last_fullsized":"2024-03-30T02:36:27.231395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:37.931440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228073+0000","last_change":"2024-03-30T02:34:46.735859+0000","last_active":"2024-03-30T02:36:27.228073+0000","last_peered":"2024-03-30T02:36:27.228073+0000","last_clean":"2024-03-30T02:36:27.228073+0000","last_became_active":"2024-03-30T02:34:46.735481+0000","last_became_peered":"2024-03-30T02:34:46.735481+0000","last_unstale":"2024-03-30T02:36:27.228073+0000","last_undegraded":"2024-03-30T02:36:27.228073+0000","last_fullsized":"2024-03-30T02:36:27.228073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:51.037887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224350+0000","last_change":"2024-03-30T02:34:46.741412+0000","last_active":"2024-03-30T02:36:27.224350+0000","last_peered":"2024-03-30T02:36:27.224350+0000","last_clean":"2024-03-30T02:36:27.224350+0000","last_became_active":"2024-03-30T02:34:46.741106+0000","last_became_peered":"2024-03-30T02:34:46.741106+0000","last_unstale":"2024-03-30T02:36:27.224350+0000","last_undegraded":"2024-03-30T02:36:27.224350+0000","last_fullsized":"2024-03-30T02:36:27.224350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:06:08.553028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225237+0000","last_change":"2024-03-30T02:34:46.746205+0000","last_active":"2024-03-30T02:36:27.225237+0000","last_peered":"2024-03-30T02:36:27.225237+0000","last_clean":"2024-03-30T02:36:27.225237+0000","last_became_active":"2024-03-30T02:34:46.745803+0000","last_became_peered":"2024-03-30T02:34:46.745803+0000","last_unstale":"2024-03-30T02:36:27.225237+0000","last_undegraded":"2024-03-30T02:36:27.225237+0000","last_fullsized":"2024-03-30T02:36:27.225237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:05.127095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221163+0000","last_change":"2024-03-30T02:34:46.722978+0000","last_active":"2024-03-30T02:36:27.221163+0000","last_peered":"2024-03-30T02:36:27.221163+0000","last_clean":"2024-03-30T02:36:27.221163+0000","last_became_active":"2024-03-30T02:34:46.722652+0000","last_became_peered":"2024-03-30T02:34:46.722652+0000","last_unstale":"2024-03-30T02:36:27.221163+0000","last_undegraded":"2024-03-30T02:36:27.221163+0000","last_fullsized":"2024-03-30T02:36:27.221163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:28:12.078216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.604850+0000","last_change":"2024-03-30T02:34:46.738895+0000","last_active":"2024-03-30T02:36:27.604850+0000","last_peered":"2024-03-30T02:36:27.604850+0000","last_clean":"2024-03-30T02:36:27.604850+0000","last_became_active":"2024-03-30T02:34:46.738510+0000","last_became_peered":"2024-03-30T02:34:46.738510+0000","last_unstale":"2024-03-30T02:36:27.604850+0000","last_undegraded":"2024-03-30T02:36:27.604850+0000","last_fullsized":"2024-03-30T02:36:27.604850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:09.859259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.604789+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:36:27.604789+0000","last_peered":"2024-03-30T02:36:27.604789+0000","last_clean":"2024-03-30T02:36:27.604789+0000","last_became_active":"2024-03-30T02:34:46.750301+0000","last_became_peered":"2024-03-30T02:34:46.750301+0000","last_unstale":"2024-03-30T02:36:27.604789+0000","last_undegraded":"2024-03-30T02:36:27.604789+0000","last_fullsized":"2024-03-30T02:36:27.604789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:06.475881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.230831+0000","last_change":"2024-03-30T02:34:46.748966+0000","last_active":"2024-03-30T02:36:27.230831+0000","last_peered":"2024-03-30T02:36:27.230831+0000","last_clean":"2024-03-30T02:36:27.230831+0000","last_became_active":"2024-03-30T02:34:46.748650+0000","last_became_peered":"2024-03-30T02:34:46.748650+0000","last_unstale":"2024-03-30T02:36:27.230831+0000","last_undegraded":"2024-03-30T02:36:27.230831+0000","last_fullsized":"2024-03-30T02:36:27.230831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:36.558864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228469+0000","last_change":"2024-03-30T02:34:46.746127+0000","last_active":"2024-03-30T02:36:27.228469+0000","last_peered":"2024-03-30T02:36:27.228469+0000","last_clean":"2024-03-30T02:36:27.228469+0000","last_became_active":"2024-03-30T02:34:46.745806+0000","last_became_peered":"2024-03-30T02:34:46.745806+0000","last_unstale":"2024-03-30T02:36:27.228469+0000","last_undegraded":"2024-03-30T02:36:27.228469+0000","last_fullsized":"2024-03-30T02:36:27.228469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:54.097996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.229068+0000","last_change":"2024-03-30T02:34:46.738391+0000","last_active":"2024-03-30T02:36:27.229068+0000","last_peered":"2024-03-30T02:36:27.229068+0000","last_clean":"2024-03-30T02:36:27.229068+0000","last_became_active":"2024-03-30T02:34:46.737110+0000","last_became_peered":"2024-03-30T02:34:46.737110+0000","last_unstale":"2024-03-30T02:36:27.229068+0000","last_undegraded":"2024-03-30T02:36:27.229068+0000","last_fullsized":"2024-03-30T02:36:27.229068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:20.326107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224284+0000","last_change":"2024-03-30T02:34:46.744522+0000","last_active":"2024-03-30T02:36:27.224284+0000","last_peered":"2024-03-30T02:36:27.224284+0000","last_clean":"2024-03-30T02:36:27.224284+0000","last_became_active":"2024-03-30T02:34:46.744088+0000","last_became_peered":"2024-03-30T02:34:46.744088+0000","last_unstale":"2024-03-30T02:36:27.224284+0000","last_undegraded":"2024-03-30T02:36:27.224284+0000","last_fullsized":"2024-03-30T02:36:27.224284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:35.698979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224043+0000","last_change":"2024-03-30T02:34:50.770855+0000","last_active":"2024-03-30T02:36:27.224043+0000","last_peered":"2024-03-30T02:36:27.224043+0000","last_clean":"2024-03-30T02:36:27.224043+0000","last_became_active":"2024-03-30T02:34:50.770420+0000","last_became_peered":"2024-03-30T02:34:50.770420+0000","last_unstale":"2024-03-30T02:36:27.224043+0000","last_undegraded":"2024-03-30T02:36:27.224043+0000","last_fullsized":"2024-03-30T02:36:27.224043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:00:08.847059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:44.255830+0000","last_change":"2024-03-30T02:34:46.207252+0000","last_active":"2024-03-30T02:36:44.255830+0000","last_peered":"2024-03-30T02:36:44.255830+0000","last_clean":"2024-03-30T02:36:44.255830+0000","last_became_active":"2024-03-30T02:34:46.206916+0000","last_became_peered":"2024-03-30T02:34:46.206916+0000","last_unstale":"2024-03-30T02:36:44.255830+0000","last_undegraded":"2024-03-30T02:36:44.255830+0000","last_fullsized":"2024-03-30T02:36:44.255830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:03:50.192245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225209+0000","last_change":"2024-03-30T02:34:46.745778+0000","last_active":"2024-03-30T02:36:27.225209+0000","last_peered":"2024-03-30T02:36:27.225209+0000","last_clean":"2024-03-30T02:36:27.225209+0000","last_became_active":"2024-03-30T02:34:46.745471+0000","last_became_peered":"2024-03-30T02:34:46.745471+0000","last_unstale":"2024-03-30T02:36:27.225209+0000","last_undegraded":"2024-03-30T02:36:27.225209+0000","last_fullsized":"2024-03-30T02:36:27.225209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:57.314163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225059+0000","last_change":"2024-03-30T02:34:50.769577+0000","last_active":"2024-03-30T02:36:27.225059+0000","last_peered":"2024-03-30T02:36:27.225059+0000","last_clean":"2024-03-30T02:36:27.225059+0000","last_became_active":"2024-03-30T02:34:50.769100+0000","last_became_peered":"2024-03-30T02:34:50.769100+0000","last_unstale":"2024-03-30T02:36:27.225059+0000","last_undegraded":"2024-03-30T02:36:27.225059+0000","last_fullsized":"2024-03-30T02:36:27.225059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:08:43.495671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225074+0000","last_change":"2024-03-30T02:34:46.218032+0000","last_active":"2024-03-30T02:36:27.225074+0000","last_peered":"2024-03-30T02:36:27.225074+0000","last_clean":"2024-03-30T02:36:27.225074+0000","last_became_active":"2024-03-30T02:34:46.217733+0000","last_became_peered":"2024-03-30T02:34:46.217733+0000","last_unstale":"2024-03-30T02:36:27.225074+0000","last_undegraded":"2024-03-30T02:36:27.225074+0000","last_fullsized":"2024-03-30T02:36:27.225074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:58.413590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605282+0000","last_change":"2024-03-30T02:34:46.750588+0000","last_active":"2024-03-30T02:36:27.605282+0000","last_peered":"2024-03-30T02:36:27.605282+0000","last_clean":"2024-03-30T02:36:27.605282+0000","last_became_active":"2024-03-30T02:34:46.750016+0000","last_became_peered":"2024-03-30T02:34:46.750016+0000","last_unstale":"2024-03-30T02:36:27.605282+0000","last_undegraded":"2024-03-30T02:36:27.605282+0000","last_fullsized":"2024-03-30T02:36:27.605282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:19.144544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225017+0000","last_change":"2024-03-30T02:35:28.213404+0000","last_active":"2024-03-30T02:36:27.225017+0000","last_peered":"2024-03-30T02:36:27.225017+0000","last_clean":"2024-03-30T02:36:27.225017+0000","last_became_active":"2024-03-30T02:35:28.212746+0000","last_became_peered":"2024-03-30T02:35:28.212746+0000","last_unstale":"2024-03-30T02:36:27.225017+0000","last_undegraded":"2024-03-30T02:36:27.225017+0000","last_fullsized":"2024-03-30T02:36:27.225017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:24.807469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.232111+0000","last_change":"2024-03-30T02:34:46.213741+0000","last_active":"2024-03-30T02:36:27.232111+0000","last_peered":"2024-03-30T02:36:27.232111+0000","last_clean":"2024-03-30T02:36:27.232111+0000","last_became_active":"2024-03-30T02:34:46.213395+0000","last_became_peered":"2024-03-30T02:34:46.213395+0000","last_unstale":"2024-03-30T02:36:27.232111+0000","last_undegraded":"2024-03-30T02:36:27.232111+0000","last_fullsized":"2024-03-30T02:36:27.232111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:13.192415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231037+0000","last_change":"2024-03-30T02:34:46.738401+0000","last_active":"2024-03-30T02:36:27.231037+0000","last_peered":"2024-03-30T02:36:27.231037+0000","last_clean":"2024-03-30T02:36:27.231037+0000","last_became_active":"2024-03-30T02:34:46.737804+0000","last_became_peered":"2024-03-30T02:34:46.737804+0000","last_unstale":"2024-03-30T02:36:27.231037+0000","last_undegraded":"2024-03-30T02:36:27.231037+0000","last_fullsized":"2024-03-30T02:36:27.231037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:56:08.194439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224866+0000","last_change":"2024-03-30T02:34:50.766222+0000","last_active":"2024-03-30T02:36:27.224866+0000","last_peered":"2024-03-30T02:36:27.224866+0000","last_clean":"2024-03-30T02:36:27.224866+0000","last_became_active":"2024-03-30T02:34:50.765836+0000","last_became_peered":"2024-03-30T02:34:50.765836+0000","last_unstale":"2024-03-30T02:36:27.224866+0000","last_undegraded":"2024-03-30T02:36:27.224866+0000","last_fullsized":"2024-03-30T02:36:27.224866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:32.529535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231217+0000","last_change":"2024-03-30T02:34:46.212586+0000","last_active":"2024-03-30T02:36:27.231217+0000","last_peered":"2024-03-30T02:36:27.231217+0000","last_clean":"2024-03-30T02:36:27.231217+0000","last_became_active":"2024-03-30T02:34:46.212110+0000","last_became_peered":"2024-03-30T02:34:46.212110+0000","last_unstale":"2024-03-30T02:36:27.231217+0000","last_undegraded":"2024-03-30T02:36:27.231217+0000","last_fullsized":"2024-03-30T02:36:27.231217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:38.630953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221516+0000","last_change":"2024-03-30T02:34:46.734923+0000","last_active":"2024-03-30T02:36:27.221516+0000","last_peered":"2024-03-30T02:36:27.221516+0000","last_clean":"2024-03-30T02:36:27.221516+0000","last_became_active":"2024-03-30T02:34:46.734572+0000","last_became_peered":"2024-03-30T02:34:46.734572+0000","last_unstale":"2024-03-30T02:36:27.221516+0000","last_undegraded":"2024-03-30T02:36:27.221516+0000","last_fullsized":"2024-03-30T02:36:27.221516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:28.647788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224173+0000","last_change":"2024-03-30T02:34:50.757803+0000","last_active":"2024-03-30T02:36:27.224173+0000","last_peered":"2024-03-30T02:36:27.224173+0000","last_clean":"2024-03-30T02:36:27.224173+0000","last_became_active":"2024-03-30T02:34:50.757198+0000","last_became_peered":"2024-03-30T02:34:50.757198+0000","last_unstale":"2024-03-30T02:36:27.224173+0000","last_undegraded":"2024-03-30T02:36:27.224173+0000","last_fullsized":"2024-03-30T02:36:27.224173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:54:49.399741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":113,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:41.254356+0000","last_change":"2024-03-30T02:34:46.214549+0000","last_active":"2024-03-30T02:36:41.254356+0000","last_peered":"2024-03-30T02:36:41.254356+0000","last_clean":"2024-03-30T02:36:41.254356+0000","last_became_active":"2024-03-30T02:34:46.214198+0000","last_became_peered":"2024-03-30T02:34:46.214198+0000","last_unstale":"2024-03-30T02:36:41.254356+0000","last_undegraded":"2024-03-30T02:36:41.254356+0000","last_fullsized":"2024-03-30T02:36:41.254356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:45:25.618982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1108,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231496+0000","last_change":"2024-03-30T02:34:46.737771+0000","last_active":"2024-03-30T02:36:27.231496+0000","last_peered":"2024-03-30T02:36:27.231496+0000","last_clean":"2024-03-30T02:36:27.231496+0000","last_became_active":"2024-03-30T02:34:46.737438+0000","last_became_peered":"2024-03-30T02:34:46.737438+0000","last_unstale":"2024-03-30T02:36:27.231496+0000","last_undegraded":"2024-03-30T02:36:27.231496+0000","last_fullsized":"2024-03-30T02:36:27.231496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:08.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224131+0000","last_change":"2024-03-30T02:34:50.767304+0000","last_active":"2024-03-30T02:36:27.224131+0000","last_peered":"2024-03-30T02:36:27.224131+0000","last_clean":"2024-03-30T02:36:27.224131+0000","last_became_active":"2024-03-30T02:34:50.766909+0000","last_became_peered":"2024-03-30T02:34:50.766909+0000","last_unstale":"2024-03-30T02:36:27.224131+0000","last_undegraded":"2024-03-30T02:36:27.224131+0000","last_fullsized":"2024-03-30T02:36:27.224131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:24.021991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.227724+0000","last_change":"2024-03-30T02:34:46.214937+0000","last_active":"2024-03-30T02:36:27.227724+0000","last_peered":"2024-03-30T02:36:27.227724+0000","last_clean":"2024-03-30T02:36:27.227724+0000","last_became_active":"2024-03-30T02:34:46.214503+0000","last_became_peered":"2024-03-30T02:34:46.214503+0000","last_unstale":"2024-03-30T02:36:27.227724+0000","last_undegraded":"2024-03-30T02:36:27.227724+0000","last_fullsized":"2024-03-30T02:36:27.227724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:20:11.624919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221928+0000","last_change":"2024-03-30T02:34:46.737374+0000","last_active":"2024-03-30T02:36:27.221928+0000","last_peered":"2024-03-30T02:36:27.221928+0000","last_clean":"2024-03-30T02:36:27.221928+0000","last_became_active":"2024-03-30T02:34:46.735307+0000","last_became_peered":"2024-03-30T02:34:46.735307+0000","last_unstale":"2024-03-30T02:36:27.221928+0000","last_undegraded":"2024-03-30T02:36:27.221928+0000","last_fullsized":"2024-03-30T02:36:27.221928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:30.482728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228360+0000","last_change":"2024-03-30T02:34:50.768016+0000","last_active":"2024-03-30T02:36:27.228360+0000","last_peered":"2024-03-30T02:36:27.228360+0000","last_clean":"2024-03-30T02:36:27.228360+0000","last_became_active":"2024-03-30T02:34:50.767564+0000","last_became_peered":"2024-03-30T02:34:50.767564+0000","last_unstale":"2024-03-30T02:36:27.228360+0000","last_undegraded":"2024-03-30T02:36:27.228360+0000","last_fullsized":"2024-03-30T02:36:27.228360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:16.888754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.220995+0000","last_change":"2024-03-30T02:34:46.212982+0000","last_active":"2024-03-30T02:36:27.220995+0000","last_peered":"2024-03-30T02:36:27.220995+0000","last_clean":"2024-03-30T02:36:27.220995+0000","last_became_active":"2024-03-30T02:34:46.212550+0000","last_became_peered":"2024-03-30T02:34:46.212550+0000","last_unstale":"2024-03-30T02:36:27.220995+0000","last_undegraded":"2024-03-30T02:36:27.220995+0000","last_fullsized":"2024-03-30T02:36:27.220995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:44:50.156615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228102+0000","last_change":"2024-03-30T02:35:28.213774+0000","last_active":"2024-03-30T02:36:27.228102+0000","last_peered":"2024-03-30T02:36:27.228102+0000","last_clean":"2024-03-30T02:36:27.228102+0000","last_became_active":"2024-03-30T02:35:28.213085+0000","last_became_peered":"2024-03-30T02:35:28.213085+0000","last_unstale":"2024-03-30T02:36:27.228102+0000","last_undegraded":"2024-03-30T02:36:27.228102+0000","last_fullsized":"2024-03-30T02:36:27.228102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:56:30.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605207+0000","last_change":"2024-03-30T02:34:50.773036+0000","last_active":"2024-03-30T02:36:27.605207+0000","last_peered":"2024-03-30T02:36:27.605207+0000","last_clean":"2024-03-30T02:36:27.605207+0000","last_became_active":"2024-03-30T02:34:50.772614+0000","last_became_peered":"2024-03-30T02:34:50.772614+0000","last_unstale":"2024-03-30T02:36:27.605207+0000","last_undegraded":"2024-03-30T02:36:27.605207+0000","last_fullsized":"2024-03-30T02:36:27.605207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:54:28.792793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221282+0000","last_change":"2024-03-30T02:34:46.209802+0000","last_active":"2024-03-30T02:36:27.221282+0000","last_peered":"2024-03-30T02:36:27.221282+0000","last_clean":"2024-03-30T02:36:27.221282+0000","last_became_active":"2024-03-30T02:34:46.209246+0000","last_became_peered":"2024-03-30T02:34:46.209246+0000","last_unstale":"2024-03-30T02:36:27.221282+0000","last_undegraded":"2024-03-30T02:36:27.221282+0000","last_fullsized":"2024-03-30T02:36:27.221282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:09:57.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228970+0000","last_change":"2024-03-30T02:34:46.743638+0000","last_active":"2024-03-30T02:36:27.228970+0000","last_peered":"2024-03-30T02:36:27.228970+0000","last_clean":"2024-03-30T02:36:27.228970+0000","last_became_active":"2024-03-30T02:34:46.743246+0000","last_became_peered":"2024-03-30T02:34:46.743246+0000","last_unstale":"2024-03-30T02:36:27.228970+0000","last_undegraded":"2024-03-30T02:36:27.228970+0000","last_fullsized":"2024-03-30T02:36:27.228970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:49:55.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225892+0000","last_change":"2024-03-30T02:34:50.768586+0000","last_active":"2024-03-30T02:36:27.225892+0000","last_peered":"2024-03-30T02:36:27.225892+0000","last_clean":"2024-03-30T02:36:27.225892+0000","last_became_active":"2024-03-30T02:34:50.766894+0000","last_became_peered":"2024-03-30T02:34:50.766894+0000","last_unstale":"2024-03-30T02:36:27.225892+0000","last_undegraded":"2024-03-30T02:36:27.225892+0000","last_fullsized":"2024-03-30T02:36:27.225892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:36:39.329608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'6","reported_seq":194,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:43.256303+0000","last_change":"2024-03-30T02:34:46.204994+0000","last_active":"2024-03-30T02:36:43.256303+0000","last_peered":"2024-03-30T02:36:43.256303+0000","last_clean":"2024-03-30T02:36:43.256303+0000","last_became_active":"2024-03-30T02:34:46.204710+0000","last_became_peered":"2024-03-30T02:34:46.204710+0000","last_unstale":"2024-03-30T02:36:43.256303+0000","last_undegraded":"2024-03-30T02:36:43.256303+0000","last_fullsized":"2024-03-30T02:36:43.256303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:39.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231396+0000","last_change":"2024-03-30T02:34:46.733573+0000","last_active":"2024-03-30T02:36:27.231396+0000","last_peered":"2024-03-30T02:36:27.231396+0000","last_clean":"2024-03-30T02:36:27.231396+0000","last_became_active":"2024-03-30T02:34:46.733241+0000","last_became_peered":"2024-03-30T02:34:46.733241+0000","last_unstale":"2024-03-30T02:36:27.231396+0000","last_undegraded":"2024-03-30T02:36:27.231396+0000","last_fullsized":"2024-03-30T02:36:27.231396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:07:10.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231018+0000","last_change":"2024-03-30T02:35:28.313493+0000","last_active":"2024-03-30T02:36:27.231018+0000","last_peered":"2024-03-30T02:36:27.231018+0000","last_clean":"2024-03-30T02:36:27.231018+0000","last_became_active":"2024-03-30T02:35:28.313065+0000","last_became_peered":"2024-03-30T02:35:28.313065+0000","last_unstale":"2024-03-30T02:36:27.231018+0000","last_undegraded":"2024-03-30T02:36:27.231018+0000","last_fullsized":"2024-03-30T02:36:27.231018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:44:52.675251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224410+0000","last_change":"2024-03-30T02:34:46.212732+0000","last_active":"2024-03-30T02:36:27.224410+0000","last_peered":"2024-03-30T02:36:27.224410+0000","last_clean":"2024-03-30T02:36:27.224410+0000","last_became_active":"2024-03-30T02:34:46.212306+0000","last_became_peered":"2024-03-30T02:34:46.212306+0000","last_unstale":"2024-03-30T02:36:27.224410+0000","last_undegraded":"2024-03-30T02:36:27.224410+0000","last_fullsized":"2024-03-30T02:36:27.224410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:27:15.766203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605314+0000","last_change":"2024-03-30T02:34:46.749420+0000","last_active":"2024-03-30T02:36:27.605314+0000","last_peered":"2024-03-30T02:36:27.605314+0000","last_clean":"2024-03-30T02:36:27.605314+0000","last_became_active":"2024-03-30T02:34:46.745286+0000","last_became_peered":"2024-03-30T02:34:46.745286+0000","last_unstale":"2024-03-30T02:36:27.605314+0000","last_undegraded":"2024-03-30T02:36:27.605314+0000","last_fullsized":"2024-03-30T02:36:27.605314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:01.489862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225302+0000","last_change":"2024-03-30T02:34:50.760542+0000","last_active":"2024-03-30T02:36:27.225302+0000","last_peered":"2024-03-30T02:36:27.225302+0000","last_clean":"2024-03-30T02:36:27.225302+0000","last_became_active":"2024-03-30T02:34:50.760087+0000","last_became_peered":"2024-03-30T02:34:50.760087+0000","last_unstale":"2024-03-30T02:36:27.225302+0000","last_undegraded":"2024-03-30T02:36:27.225302+0000","last_fullsized":"2024-03-30T02:36:27.225302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:46.352189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221437+0000","last_change":"2024-03-30T02:34:46.224058+0000","last_active":"2024-03-30T02:36:27.221437+0000","last_peered":"2024-03-30T02:36:27.221437+0000","last_clean":"2024-03-30T02:36:27.221437+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:36:27.221437+0000","last_undegraded":"2024-03-30T02:36:27.221437+0000","last_fullsized":"2024-03-30T02:36:27.221437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:31:10.124533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231126+0000","last_change":"2024-03-30T02:34:46.741413+0000","last_active":"2024-03-30T02:36:27.231126+0000","last_peered":"2024-03-30T02:36:27.231126+0000","last_clean":"2024-03-30T02:36:27.231126+0000","last_became_active":"2024-03-30T02:34:46.741112+0000","last_became_peered":"2024-03-30T02:34:46.741112+0000","last_unstale":"2024-03-30T02:36:27.231126+0000","last_undegraded":"2024-03-30T02:36:27.231126+0000","last_fullsized":"2024-03-30T02:36:27.231126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:34.820166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.229215+0000","last_change":"2024-03-30T02:34:50.752315+0000","last_active":"2024-03-30T02:36:27.229215+0000","last_peered":"2024-03-30T02:36:27.229215+0000","last_clean":"2024-03-30T02:36:27.229215+0000","last_became_active":"2024-03-30T02:34:50.750985+0000","last_became_peered":"2024-03-30T02:34:50.750985+0000","last_unstale":"2024-03-30T02:36:27.229215+0000","last_undegraded":"2024-03-30T02:36:27.229215+0000","last_fullsized":"2024-03-30T02:36:27.229215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:37.271063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'43","reported_seq":243,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:43.257780+0000","last_change":"2024-03-30T02:34:46.214759+0000","last_active":"2024-03-30T02:36:43.257780+0000","last_peered":"2024-03-30T02:36:43.257780+0000","last_clean":"2024-03-30T02:36:43.257780+0000","last_became_active":"2024-03-30T02:34:46.214388+0000","last_became_peered":"2024-03-30T02:34:46.214388+0000","last_unstale":"2024-03-30T02:36:43.257780+0000","last_undegraded":"2024-03-30T02:36:43.257780+0000","last_fullsized":"2024-03-30T02:36:43.257780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:41.100461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68401,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":180,"num_read_kb":158,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231853+0000","last_change":"2024-03-30T02:34:46.741112+0000","last_active":"2024-03-30T02:36:27.231853+0000","last_peered":"2024-03-30T02:36:27.231853+0000","last_clean":"2024-03-30T02:36:27.231853+0000","last_became_active":"2024-03-30T02:34:46.739664+0000","last_became_peered":"2024-03-30T02:34:46.739664+0000","last_unstale":"2024-03-30T02:36:27.231853+0000","last_undegraded":"2024-03-30T02:36:27.231853+0000","last_fullsized":"2024-03-30T02:36:27.231853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:07.763942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221733+0000","last_change":"2024-03-30T02:34:50.765645+0000","last_active":"2024-03-30T02:36:27.221733+0000","last_peered":"2024-03-30T02:36:27.221733+0000","last_clean":"2024-03-30T02:36:27.221733+0000","last_became_active":"2024-03-30T02:34:50.765345+0000","last_became_peered":"2024-03-30T02:34:50.765345+0000","last_unstale":"2024-03-30T02:36:27.221733+0000","last_undegraded":"2024-03-30T02:36:27.221733+0000","last_fullsized":"2024-03-30T02:36:27.221733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:25:38.591525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224473+0000","last_change":"2024-03-30T02:34:46.216643+0000","last_active":"2024-03-30T02:36:27.224473+0000","last_peered":"2024-03-30T02:36:27.224473+0000","last_clean":"2024-03-30T02:36:27.224473+0000","last_became_active":"2024-03-30T02:34:46.216188+0000","last_became_peered":"2024-03-30T02:34:46.216188+0000","last_unstale":"2024-03-30T02:36:27.224473+0000","last_undegraded":"2024-03-30T02:36:27.224473+0000","last_fullsized":"2024-03-30T02:36:27.224473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:49:10.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221353+0000","last_change":"2024-03-30T02:34:46.748313+0000","last_active":"2024-03-30T02:36:27.221353+0000","last_peered":"2024-03-30T02:36:27.221353+0000","last_clean":"2024-03-30T02:36:27.221353+0000","last_became_active":"2024-03-30T02:34:46.742897+0000","last_became_peered":"2024-03-30T02:34:46.742897+0000","last_unstale":"2024-03-30T02:36:27.221353+0000","last_undegraded":"2024-03-30T02:36:27.221353+0000","last_fullsized":"2024-03-30T02:36:27.221353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:00.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228185+0000","last_change":"2024-03-30T02:34:50.760993+0000","last_active":"2024-03-30T02:36:27.228185+0000","last_peered":"2024-03-30T02:36:27.228185+0000","last_clean":"2024-03-30T02:36:27.228185+0000","last_became_active":"2024-03-30T02:34:50.760535+0000","last_became_peered":"2024-03-30T02:34:50.760535+0000","last_unstale":"2024-03-30T02:36:27.228185+0000","last_undegraded":"2024-03-30T02:36:27.228185+0000","last_fullsized":"2024-03-30T02:36:27.228185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:15.569533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221659+0000","last_change":"2024-03-30T02:34:46.216603+0000","last_active":"2024-03-30T02:36:27.221659+0000","last_peered":"2024-03-30T02:36:27.221659+0000","last_clean":"2024-03-30T02:36:27.221659+0000","last_became_active":"2024-03-30T02:34:46.216231+0000","last_became_peered":"2024-03-30T02:34:46.216231+0000","last_unstale":"2024-03-30T02:36:27.221659+0000","last_undegraded":"2024-03-30T02:36:27.221659+0000","last_fullsized":"2024-03-30T02:36:27.221659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:51:00.659652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224126+0000","last_change":"2024-03-30T02:34:46.742815+0000","last_active":"2024-03-30T02:36:27.224126+0000","last_peered":"2024-03-30T02:36:27.224126+0000","last_clean":"2024-03-30T02:36:27.224126+0000","last_became_active":"2024-03-30T02:34:46.742487+0000","last_became_peered":"2024-03-30T02:34:46.742487+0000","last_unstale":"2024-03-30T02:36:27.224126+0000","last_undegraded":"2024-03-30T02:36:27.224126+0000","last_fullsized":"2024-03-30T02:36:27.224126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:07.182990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228857+0000","last_change":"2024-03-30T02:34:50.770398+0000","last_active":"2024-03-30T02:36:27.228857+0000","last_peered":"2024-03-30T02:36:27.228857+0000","last_clean":"2024-03-30T02:36:27.228857+0000","last_became_active":"2024-03-30T02:34:50.770000+0000","last_became_peered":"2024-03-30T02:34:50.770000+0000","last_unstale":"2024-03-30T02:36:27.228857+0000","last_undegraded":"2024-03-30T02:36:27.228857+0000","last_fullsized":"2024-03-30T02:36:27.228857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:14:49.057081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":116,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:44.256642+0000","last_change":"2024-03-30T02:34:46.209403+0000","last_active":"2024-03-30T02:36:44.256642+0000","last_peered":"2024-03-30T02:36:44.256642+0000","last_clean":"2024-03-30T02:36:44.256642+0000","last_became_active":"2024-03-30T02:34:46.209065+0000","last_became_peered":"2024-03-30T02:34:46.209065+0000","last_unstale":"2024-03-30T02:36:44.256642+0000","last_undegraded":"2024-03-30T02:36:44.256642+0000","last_fullsized":"2024-03-30T02:36:44.256642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:36:20.427720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223778+0000","last_change":"2024-03-30T02:34:46.734861+0000","last_active":"2024-03-30T02:36:27.223778+0000","last_peered":"2024-03-30T02:36:27.223778+0000","last_clean":"2024-03-30T02:36:27.223778+0000","last_became_active":"2024-03-30T02:34:46.734376+0000","last_became_peered":"2024-03-30T02:34:46.734376+0000","last_unstale":"2024-03-30T02:36:27.223778+0000","last_undegraded":"2024-03-30T02:36:27.223778+0000","last_fullsized":"2024-03-30T02:36:27.223778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:08:36.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221531+0000","last_change":"2024-03-30T02:34:50.767469+0000","last_active":"2024-03-30T02:36:27.221531+0000","last_peered":"2024-03-30T02:36:27.221531+0000","last_clean":"2024-03-30T02:36:27.221531+0000","last_became_active":"2024-03-30T02:34:50.765460+0000","last_became_peered":"2024-03-30T02:34:50.765460+0000","last_unstale":"2024-03-30T02:36:27.221531+0000","last_undegraded":"2024-03-30T02:36:27.221531+0000","last_fullsized":"2024-03-30T02:36:27.221531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:54:06.940171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231014+0000","last_change":"2024-03-30T02:34:46.224053+0000","last_active":"2024-03-30T02:36:27.231014+0000","last_peered":"2024-03-30T02:36:27.231014+0000","last_clean":"2024-03-30T02:36:27.231014+0000","last_became_active":"2024-03-30T02:34:46.223666+0000","last_became_peered":"2024-03-30T02:34:46.223666+0000","last_unstale":"2024-03-30T02:36:27.231014+0000","last_undegraded":"2024-03-30T02:36:27.231014+0000","last_fullsized":"2024-03-30T02:36:27.231014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:18:22.976298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221390+0000","last_change":"2024-03-30T02:34:46.734838+0000","last_active":"2024-03-30T02:36:27.221390+0000","last_peered":"2024-03-30T02:36:27.221390+0000","last_clean":"2024-03-30T02:36:27.221390+0000","last_became_active":"2024-03-30T02:34:46.734428+0000","last_became_peered":"2024-03-30T02:34:46.734428+0000","last_unstale":"2024-03-30T02:36:27.221390+0000","last_undegraded":"2024-03-30T02:36:27.221390+0000","last_fullsized":"2024-03-30T02:36:27.221390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:42:51.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221785+0000","last_change":"2024-03-30T02:34:50.771150+0000","last_active":"2024-03-30T02:36:27.221785+0000","last_peered":"2024-03-30T02:36:27.221785+0000","last_clean":"2024-03-30T02:36:27.221785+0000","last_became_active":"2024-03-30T02:34:50.770703+0000","last_became_peered":"2024-03-30T02:34:50.770703+0000","last_unstale":"2024-03-30T02:36:27.221785+0000","last_undegraded":"2024-03-30T02:36:27.221785+0000","last_fullsized":"2024-03-30T02:36:27.221785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:04.466664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231835+0000","last_change":"2024-03-30T02:34:46.203558+0000","last_active":"2024-03-30T02:36:27.231835+0000","last_peered":"2024-03-30T02:36:27.231835+0000","last_clean":"2024-03-30T02:36:27.231835+0000","last_became_active":"2024-03-30T02:34:46.202896+0000","last_became_peered":"2024-03-30T02:34:46.202896+0000","last_unstale":"2024-03-30T02:36:27.231835+0000","last_undegraded":"2024-03-30T02:36:27.231835+0000","last_fullsized":"2024-03-30T02:36:27.231835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:33.945665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228622+0000","last_change":"2024-03-30T02:35:28.212775+0000","last_active":"2024-03-30T02:36:27.228622+0000","last_peered":"2024-03-30T02:36:27.228622+0000","last_clean":"2024-03-30T02:36:27.228622+0000","last_became_active":"2024-03-30T02:35:28.212368+0000","last_became_peered":"2024-03-30T02:35:28.212368+0000","last_unstale":"2024-03-30T02:36:27.228622+0000","last_undegraded":"2024-03-30T02:36:27.228622+0000","last_fullsized":"2024-03-30T02:36:27.228622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:50.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231620+0000","last_change":"2024-03-30T02:34:50.797758+0000","last_active":"2024-03-30T02:36:27.231620+0000","last_peered":"2024-03-30T02:36:27.231620+0000","last_clean":"2024-03-30T02:36:27.231620+0000","last_became_active":"2024-03-30T02:34:50.797311+0000","last_became_peered":"2024-03-30T02:34:50.797311+0000","last_unstale":"2024-03-30T02:36:27.231620+0000","last_undegraded":"2024-03-30T02:36:27.231620+0000","last_fullsized":"2024-03-30T02:36:27.231620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:32.337422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228443+0000","last_change":"2024-03-30T02:34:46.224397+0000","last_active":"2024-03-30T02:36:27.228443+0000","last_peered":"2024-03-30T02:36:27.228443+0000","last_clean":"2024-03-30T02:36:27.228443+0000","last_became_active":"2024-03-30T02:34:46.223825+0000","last_became_peered":"2024-03-30T02:34:46.223825+0000","last_unstale":"2024-03-30T02:36:27.228443+0000","last_undegraded":"2024-03-30T02:36:27.228443+0000","last_fullsized":"2024-03-30T02:36:27.228443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:53.154665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.227952+0000","last_change":"2024-03-30T02:34:46.741358+0000","last_active":"2024-03-30T02:36:27.227952+0000","last_peered":"2024-03-30T02:36:27.227952+0000","last_clean":"2024-03-30T02:36:27.227952+0000","last_became_active":"2024-03-30T02:34:46.740734+0000","last_became_peered":"2024-03-30T02:34:46.740734+0000","last_unstale":"2024-03-30T02:36:27.227952+0000","last_undegraded":"2024-03-30T02:36:27.227952+0000","last_fullsized":"2024-03-30T02:36:27.227952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:59.678431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221512+0000","last_change":"2024-03-30T02:34:50.761998+0000","last_active":"2024-03-30T02:36:27.221512+0000","last_peered":"2024-03-30T02:36:27.221512+0000","last_clean":"2024-03-30T02:36:27.221512+0000","last_became_active":"2024-03-30T02:34:50.761560+0000","last_became_peered":"2024-03-30T02:34:50.761560+0000","last_unstale":"2024-03-30T02:36:27.221512+0000","last_undegraded":"2024-03-30T02:36:27.221512+0000","last_fullsized":"2024-03-30T02:36:27.221512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:57:17.213624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":112,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:44.256432+0000","last_change":"2024-03-30T02:34:46.213427+0000","last_active":"2024-03-30T02:36:44.256432+0000","last_peered":"2024-03-30T02:36:44.256432+0000","last_clean":"2024-03-30T02:36:44.256432+0000","last_became_active":"2024-03-30T02:34:46.213082+0000","last_became_peered":"2024-03-30T02:34:46.213082+0000","last_unstale":"2024-03-30T02:36:44.256432+0000","last_undegraded":"2024-03-30T02:36:44.256432+0000","last_fullsized":"2024-03-30T02:36:44.256432+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:31:35.513723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228590+0000","last_change":"2024-03-30T02:34:46.747197+0000","last_active":"2024-03-30T02:36:27.228590+0000","last_peered":"2024-03-30T02:36:27.228590+0000","last_clean":"2024-03-30T02:36:27.228590+0000","last_became_active":"2024-03-30T02:34:46.743841+0000","last_became_peered":"2024-03-30T02:34:46.743841+0000","last_unstale":"2024-03-30T02:36:27.228590+0000","last_undegraded":"2024-03-30T02:36:27.228590+0000","last_fullsized":"2024-03-30T02:36:27.228590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:09.234382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221642+0000","last_change":"2024-03-30T02:34:50.797423+0000","last_active":"2024-03-30T02:36:27.221642+0000","last_peered":"2024-03-30T02:36:27.221642+0000","last_clean":"2024-03-30T02:36:27.221642+0000","last_became_active":"2024-03-30T02:34:50.796746+0000","last_became_peered":"2024-03-30T02:34:50.796746+0000","last_unstale":"2024-03-30T02:36:27.221642+0000","last_undegraded":"2024-03-30T02:36:27.221642+0000","last_fullsized":"2024-03-30T02:36:27.221642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:24.014131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221459+0000","last_change":"2024-03-30T02:34:46.734972+0000","last_active":"2024-03-30T02:36:27.221459+0000","last_peered":"2024-03-30T02:36:27.221459+0000","last_clean":"2024-03-30T02:36:27.221459+0000","last_became_active":"2024-03-30T02:34:46.734520+0000","last_became_peered":"2024-03-30T02:34:46.734520+0000","last_unstale":"2024-03-30T02:36:27.221459+0000","last_undegraded":"2024-03-30T02:36:27.221459+0000","last_fullsized":"2024-03-30T02:36:27.221459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:15.635060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":112,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:43.255021+0000","last_change":"2024-03-30T02:34:46.224036+0000","last_active":"2024-03-30T02:36:43.255021+0000","last_peered":"2024-03-30T02:36:43.255021+0000","last_clean":"2024-03-30T02:36:43.255021+0000","last_became_active":"2024-03-30T02:34:46.223706+0000","last_became_peered":"2024-03-30T02:34:46.223706+0000","last_unstale":"2024-03-30T02:36:43.255021+0000","last_undegraded":"2024-03-30T02:36:43.255021+0000","last_fullsized":"2024-03-30T02:36:43.255021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:40.458345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224187+0000","last_change":"2024-03-30T02:34:50.757894+0000","last_active":"2024-03-30T02:36:27.224187+0000","last_peered":"2024-03-30T02:36:27.224187+0000","last_clean":"2024-03-30T02:36:27.224187+0000","last_became_active":"2024-03-30T02:34:50.757486+0000","last_became_peered":"2024-03-30T02:34:50.757486+0000","last_unstale":"2024-03-30T02:36:27.224187+0000","last_undegraded":"2024-03-30T02:36:27.224187+0000","last_fullsized":"2024-03-30T02:36:27.224187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:30.609851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225449+0000","last_change":"2024-03-30T02:34:46.745990+0000","last_active":"2024-03-30T02:36:27.225449+0000","last_peered":"2024-03-30T02:36:27.225449+0000","last_clean":"2024-03-30T02:36:27.225449+0000","last_became_active":"2024-03-30T02:34:46.741937+0000","last_became_peered":"2024-03-30T02:34:46.741937+0000","last_unstale":"2024-03-30T02:36:27.225449+0000","last_undegraded":"2024-03-30T02:36:27.225449+0000","last_fullsized":"2024-03-30T02:36:27.225449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:26:17.833164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225196+0000","last_change":"2024-03-30T02:34:46.208858+0000","last_active":"2024-03-30T02:36:27.225196+0000","last_peered":"2024-03-30T02:36:27.225196+0000","last_clean":"2024-03-30T02:36:27.225196+0000","last_became_active":"2024-03-30T02:34:46.208479+0000","last_became_peered":"2024-03-30T02:34:46.208479+0000","last_unstale":"2024-03-30T02:36:27.225196+0000","last_undegraded":"2024-03-30T02:36:27.225196+0000","last_fullsized":"2024-03-30T02:36:27.225196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:58:14.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223677+0000","last_change":"2024-03-30T02:33:56.208237+0000","last_active":"2024-03-30T02:36:27.223677+0000","last_peered":"2024-03-30T02:36:27.223677+0000","last_clean":"2024-03-30T02:36:27.223677+0000","last_became_active":"2024-03-30T02:33:56.207820+0000","last_became_peered":"2024-03-30T02:33:56.207820+0000","last_unstale":"2024-03-30T02:36:27.223677+0000","last_undegraded":"2024-03-30T02:36:27.223677+0000","last_fullsized":"2024-03-30T02:36:27.223677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_clean_scrub_stamp":"2024-03-30T02:31:23.922434+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:26.198503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221375+0000","last_change":"2024-03-30T02:35:28.213743+0000","last_active":"2024-03-30T02:36:27.221375+0000","last_peered":"2024-03-30T02:36:27.221375+0000","last_clean":"2024-03-30T02:36:27.221375+0000","last_became_active":"2024-03-30T02:35:28.213318+0000","last_became_peered":"2024-03-30T02:35:28.213318+0000","last_unstale":"2024-03-30T02:36:27.221375+0000","last_undegraded":"2024-03-30T02:36:27.221375+0000","last_fullsized":"2024-03-30T02:36:27.221375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:15:35.803168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.220877+0000","last_change":"2024-03-30T02:34:46.739757+0000","last_active":"2024-03-30T02:36:27.220877+0000","last_peered":"2024-03-30T02:36:27.220877+0000","last_clean":"2024-03-30T02:36:27.220877+0000","last_became_active":"2024-03-30T02:34:46.738269+0000","last_became_peered":"2024-03-30T02:34:46.738269+0000","last_unstale":"2024-03-30T02:36:27.220877+0000","last_undegraded":"2024-03-30T02:36:27.220877+0000","last_fullsized":"2024-03-30T02:36:27.220877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:03.411187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.235101+0000","last_change":"2024-03-30T02:34:46.212532+0000","last_active":"2024-03-30T02:36:28.235101+0000","last_peered":"2024-03-30T02:36:28.235101+0000","last_clean":"2024-03-30T02:36:28.235101+0000","last_became_active":"2024-03-30T02:34:46.212191+0000","last_became_peered":"2024-03-30T02:34:46.212191+0000","last_unstale":"2024-03-30T02:36:28.235101+0000","last_undegraded":"2024-03-30T02:36:28.235101+0000","last_fullsized":"2024-03-30T02:36:28.235101+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:19.120960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228188+0000","last_change":"2024-03-30T02:34:50.759911+0000","last_active":"2024-03-30T02:36:27.228188+0000","last_peered":"2024-03-30T02:36:27.228188+0000","last_clean":"2024-03-30T02:36:27.228188+0000","last_became_active":"2024-03-30T02:34:50.759487+0000","last_became_peered":"2024-03-30T02:34:50.759487+0000","last_unstale":"2024-03-30T02:36:27.228188+0000","last_undegraded":"2024-03-30T02:36:27.228188+0000","last_fullsized":"2024-03-30T02:36:27.228188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:46.403754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231948+0000","last_change":"2024-03-30T02:34:46.217092+0000","last_active":"2024-03-30T02:36:27.231948+0000","last_peered":"2024-03-30T02:36:27.231948+0000","last_clean":"2024-03-30T02:36:27.231948+0000","last_became_active":"2024-03-30T02:34:46.216779+0000","last_became_peered":"2024-03-30T02:34:46.216779+0000","last_unstale":"2024-03-30T02:36:27.231948+0000","last_undegraded":"2024-03-30T02:36:27.231948+0000","last_fullsized":"2024-03-30T02:36:27.231948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:10.894577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221586+0000","last_change":"2024-03-30T02:34:46.735212+0000","last_active":"2024-03-30T02:36:27.221586+0000","last_peered":"2024-03-30T02:36:27.221586+0000","last_clean":"2024-03-30T02:36:27.221586+0000","last_became_active":"2024-03-30T02:34:46.734282+0000","last_became_peered":"2024-03-30T02:34:46.734282+0000","last_unstale":"2024-03-30T02:36:27.221586+0000","last_undegraded":"2024-03-30T02:36:27.221586+0000","last_fullsized":"2024-03-30T02:36:27.221586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:28.596961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224232+0000","last_change":"2024-03-30T02:34:50.799290+0000","last_active":"2024-03-30T02:36:27.224232+0000","last_peered":"2024-03-30T02:36:27.224232+0000","last_clean":"2024-03-30T02:36:27.224232+0000","last_became_active":"2024-03-30T02:34:50.797524+0000","last_became_peered":"2024-03-30T02:34:50.797524+0000","last_unstale":"2024-03-30T02:36:27.224232+0000","last_undegraded":"2024-03-30T02:36:27.224232+0000","last_fullsized":"2024-03-30T02:36:27.224232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:53:27.931444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:31.369328+0000","last_change":"2024-03-30T02:36:28.236366+0000","last_active":"2024-03-30T02:36:31.369328+0000","last_peered":"2024-03-30T02:36:31.369328+0000","last_clean":"2024-03-30T02:36:31.369328+0000","last_became_active":"2024-03-30T02:36:28.228981+0000","last_became_peered":"2024-03-30T02:36:28.228981+0000","last_unstale":"2024-03-30T02:36:31.369328+0000","last_undegraded":"2024-03-30T02:36:31.369328+0000","last_fullsized":"2024-03-30T02:36:31.369328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:28:26.747807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224734+0000","last_change":"2024-03-30T02:34:46.740056+0000","last_active":"2024-03-30T02:36:27.224734+0000","last_peered":"2024-03-30T02:36:27.224734+0000","last_clean":"2024-03-30T02:36:27.224734+0000","last_became_active":"2024-03-30T02:34:46.739260+0000","last_became_peered":"2024-03-30T02:34:46.739260+0000","last_unstale":"2024-03-30T02:36:27.224734+0000","last_undegraded":"2024-03-30T02:36:27.224734+0000","last_fullsized":"2024-03-30T02:36:27.224734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:53.550978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223920+0000","last_change":"2024-03-30T02:34:50.756851+0000","last_active":"2024-03-30T02:36:27.223920+0000","last_peered":"2024-03-30T02:36:27.223920+0000","last_clean":"2024-03-30T02:36:27.223920+0000","last_became_active":"2024-03-30T02:34:50.756416+0000","last_became_peered":"2024-03-30T02:34:50.756416+0000","last_unstale":"2024-03-30T02:36:27.223920+0000","last_undegraded":"2024-03-30T02:36:27.223920+0000","last_fullsized":"2024-03-30T02:36:27.223920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:11.469599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225670+0000","last_change":"2024-03-30T02:34:46.215663+0000","last_active":"2024-03-30T02:36:27.225670+0000","last_peered":"2024-03-30T02:36:27.225670+0000","last_clean":"2024-03-30T02:36:27.225670+0000","last_became_active":"2024-03-30T02:34:46.214224+0000","last_became_peered":"2024-03-30T02:34:46.214224+0000","last_unstale":"2024-03-30T02:36:27.225670+0000","last_undegraded":"2024-03-30T02:36:27.225670+0000","last_fullsized":"2024-03-30T02:36:27.225670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:24:58.734434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605318+0000","last_change":"2024-03-30T02:34:46.754150+0000","last_active":"2024-03-30T02:36:27.605318+0000","last_peered":"2024-03-30T02:36:27.605318+0000","last_clean":"2024-03-30T02:36:27.605318+0000","last_became_active":"2024-03-30T02:34:46.753786+0000","last_became_peered":"2024-03-30T02:34:46.753786+0000","last_unstale":"2024-03-30T02:36:27.605318+0000","last_undegraded":"2024-03-30T02:36:27.605318+0000","last_fullsized":"2024-03-30T02:36:27.605318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:54:50.165020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225526+0000","last_change":"2024-03-30T02:34:50.797890+0000","last_active":"2024-03-30T02:36:27.225526+0000","last_peered":"2024-03-30T02:36:27.225526+0000","last_clean":"2024-03-30T02:36:27.225526+0000","last_became_active":"2024-03-30T02:34:50.797395+0000","last_became_peered":"2024-03-30T02:34:50.797395+0000","last_unstale":"2024-03-30T02:36:27.225526+0000","last_undegraded":"2024-03-30T02:36:27.225526+0000","last_fullsized":"2024-03-30T02:36:27.225526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:36:14.958804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.230907+0000","last_change":"2024-03-30T02:34:46.220873+0000","last_active":"2024-03-30T02:36:27.230907+0000","last_peered":"2024-03-30T02:36:27.230907+0000","last_clean":"2024-03-30T02:36:27.230907+0000","last_became_active":"2024-03-30T02:34:46.220544+0000","last_became_peered":"2024-03-30T02:34:46.220544+0000","last_unstale":"2024-03-30T02:36:27.230907+0000","last_undegraded":"2024-03-30T02:36:27.230907+0000","last_fullsized":"2024-03-30T02:36:27.230907+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:37.691602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231055+0000","last_change":"2024-03-30T02:34:46.739184+0000","last_active":"2024-03-30T02:36:27.231055+0000","last_peered":"2024-03-30T02:36:27.231055+0000","last_clean":"2024-03-30T02:36:27.231055+0000","last_became_active":"2024-03-30T02:34:46.738843+0000","last_became_peered":"2024-03-30T02:34:46.738843+0000","last_unstale":"2024-03-30T02:36:27.231055+0000","last_undegraded":"2024-03-30T02:36:27.231055+0000","last_fullsized":"2024-03-30T02:36:27.231055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:15:09.894754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.229023+0000","last_change":"2024-03-30T02:34:50.768763+0000","last_active":"2024-03-30T02:36:27.229023+0000","last_peered":"2024-03-30T02:36:27.229023+0000","last_clean":"2024-03-30T02:36:27.229023+0000","last_became_active":"2024-03-30T02:34:50.767363+0000","last_became_peered":"2024-03-30T02:34:50.767363+0000","last_unstale":"2024-03-30T02:36:27.229023+0000","last_undegraded":"2024-03-30T02:36:27.229023+0000","last_fullsized":"2024-03-30T02:36:27.229023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:06.932826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224532+0000","last_change":"2024-03-30T02:34:46.213006+0000","last_active":"2024-03-30T02:36:27.224532+0000","last_peered":"2024-03-30T02:36:27.224532+0000","last_clean":"2024-03-30T02:36:27.224532+0000","last_became_active":"2024-03-30T02:34:46.206271+0000","last_became_peered":"2024-03-30T02:34:46.206271+0000","last_unstale":"2024-03-30T02:36:27.224532+0000","last_undegraded":"2024-03-30T02:36:27.224532+0000","last_fullsized":"2024-03-30T02:36:27.224532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:00:36.447352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228446+0000","last_change":"2024-03-30T02:34:46.739566+0000","last_active":"2024-03-30T02:36:27.228446+0000","last_peered":"2024-03-30T02:36:27.228446+0000","last_clean":"2024-03-30T02:36:27.228446+0000","last_became_active":"2024-03-30T02:34:46.739205+0000","last_became_peered":"2024-03-30T02:34:46.739205+0000","last_unstale":"2024-03-30T02:36:27.228446+0000","last_undegraded":"2024-03-30T02:36:27.228446+0000","last_fullsized":"2024-03-30T02:36:27.228446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:01.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224920+0000","last_change":"2024-03-30T02:34:50.758016+0000","last_active":"2024-03-30T02:36:27.224920+0000","last_peered":"2024-03-30T02:36:27.224920+0000","last_clean":"2024-03-30T02:36:27.224920+0000","last_became_active":"2024-03-30T02:34:50.757616+0000","last_became_peered":"2024-03-30T02:34:50.757616+0000","last_unstale":"2024-03-30T02:36:27.224920+0000","last_undegraded":"2024-03-30T02:36:27.224920+0000","last_fullsized":"2024-03-30T02:36:27.224920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:35:07.902347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.240419+0000","last_change":"2024-03-30T02:36:28.240079+0000","last_active":"2024-03-30T02:36:28.240419+0000","last_peered":"2024-03-30T02:36:28.240419+0000","last_clean":"2024-03-30T02:36:28.240419+0000","last_became_active":"2024-03-30T02:36:28.228586+0000","last_became_peered":"2024-03-30T02:36:28.228586+0000","last_unstale":"2024-03-30T02:36:28.240419+0000","last_undegraded":"2024-03-30T02:36:28.240419+0000","last_fullsized":"2024-03-30T02:36:28.240419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:22.476600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224967+0000","last_change":"2024-03-30T02:34:46.754114+0000","last_active":"2024-03-30T02:36:27.224967+0000","last_peered":"2024-03-30T02:36:27.224967+0000","last_clean":"2024-03-30T02:36:27.224967+0000","last_became_active":"2024-03-30T02:34:46.752691+0000","last_became_peered":"2024-03-30T02:34:46.752691+0000","last_unstale":"2024-03-30T02:36:27.224967+0000","last_undegraded":"2024-03-30T02:36:27.224967+0000","last_fullsized":"2024-03-30T02:36:27.224967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:42.908877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225234+0000","last_change":"2024-03-30T02:34:50.798110+0000","last_active":"2024-03-30T02:36:27.225234+0000","last_peered":"2024-03-30T02:36:27.225234+0000","last_clean":"2024-03-30T02:36:27.225234+0000","last_became_active":"2024-03-30T02:34:50.797551+0000","last_became_peered":"2024-03-30T02:34:50.797551+0000","last_unstale":"2024-03-30T02:36:27.225234+0000","last_undegraded":"2024-03-30T02:36:27.225234+0000","last_fullsized":"2024-03-30T02:36:27.225234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:49.362018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231216+0000","last_change":"2024-03-30T02:34:46.211647+0000","last_active":"2024-03-30T02:36:27.231216+0000","last_peered":"2024-03-30T02:36:27.231216+0000","last_clean":"2024-03-30T02:36:27.231216+0000","last_became_active":"2024-03-30T02:34:46.211311+0000","last_became_peered":"2024-03-30T02:34:46.211311+0000","last_unstale":"2024-03-30T02:36:27.231216+0000","last_undegraded":"2024-03-30T02:36:27.231216+0000","last_fullsized":"2024-03-30T02:36:27.231216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:05:34.135289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224076+0000","last_change":"2024-03-30T02:34:46.740256+0000","last_active":"2024-03-30T02:36:27.224076+0000","last_peered":"2024-03-30T02:36:27.224076+0000","last_clean":"2024-03-30T02:36:27.224076+0000","last_became_active":"2024-03-30T02:34:46.739913+0000","last_became_peered":"2024-03-30T02:34:46.739913+0000","last_unstale":"2024-03-30T02:36:27.224076+0000","last_undegraded":"2024-03-30T02:36:27.224076+0000","last_fullsized":"2024-03-30T02:36:27.224076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:44.152475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.230907+0000","last_change":"2024-03-30T02:34:50.762149+0000","last_active":"2024-03-30T02:36:27.230907+0000","last_peered":"2024-03-30T02:36:27.230907+0000","last_clean":"2024-03-30T02:36:27.230907+0000","last_became_active":"2024-03-30T02:34:50.761717+0000","last_became_peered":"2024-03-30T02:34:50.761717+0000","last_unstale":"2024-03-30T02:36:27.230907+0000","last_undegraded":"2024-03-30T02:36:27.230907+0000","last_fullsized":"2024-03-30T02:36:27.230907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:22.373079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":111,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:40.253059+0000","last_change":"2024-03-30T02:34:46.224098+0000","last_active":"2024-03-30T02:36:40.253059+0000","last_peered":"2024-03-30T02:36:40.253059+0000","last_clean":"2024-03-30T02:36:40.253059+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:36:40.253059+0000","last_undegraded":"2024-03-30T02:36:40.253059+0000","last_fullsized":"2024-03-30T02:36:40.253059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:20:25.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":85,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224527+0000","last_change":"2024-03-30T02:34:46.737661+0000","last_active":"2024-03-30T02:36:27.224527+0000","last_peered":"2024-03-30T02:36:27.224527+0000","last_clean":"2024-03-30T02:36:27.224527+0000","last_became_active":"2024-03-30T02:34:46.737303+0000","last_became_peered":"2024-03-30T02:34:46.737303+0000","last_unstale":"2024-03-30T02:36:27.224527+0000","last_undegraded":"2024-03-30T02:36:27.224527+0000","last_fullsized":"2024-03-30T02:36:27.224527+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:44.929643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.604791+0000","last_change":"2024-03-30T02:34:50.767119+0000","last_active":"2024-03-30T02:36:27.604791+0000","last_peered":"2024-03-30T02:36:27.604791+0000","last_clean":"2024-03-30T02:36:27.604791+0000","last_became_active":"2024-03-30T02:34:50.766472+0000","last_became_peered":"2024-03-30T02:34:50.766472+0000","last_unstale":"2024-03-30T02:36:27.604791+0000","last_undegraded":"2024-03-30T02:36:27.604791+0000","last_fullsized":"2024-03-30T02:36:27.604791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:46.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221554+0000","last_change":"2024-03-30T02:34:46.216713+0000","last_active":"2024-03-30T02:36:27.221554+0000","last_peered":"2024-03-30T02:36:27.221554+0000","last_clean":"2024-03-30T02:36:27.221554+0000","last_became_active":"2024-03-30T02:34:46.216366+0000","last_became_peered":"2024-03-30T02:34:46.216366+0000","last_unstale":"2024-03-30T02:36:27.221554+0000","last_undegraded":"2024-03-30T02:36:27.221554+0000","last_fullsized":"2024-03-30T02:36:27.221554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:43.803955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224174+0000","last_change":"2024-03-30T02:34:46.736854+0000","last_active":"2024-03-30T02:36:27.224174+0000","last_peered":"2024-03-30T02:36:27.224174+0000","last_clean":"2024-03-30T02:36:27.224174+0000","last_became_active":"2024-03-30T02:34:46.736511+0000","last_became_peered":"2024-03-30T02:34:46.736511+0000","last_unstale":"2024-03-30T02:36:27.224174+0000","last_undegraded":"2024-03-30T02:36:27.224174+0000","last_fullsized":"2024-03-30T02:36:27.224174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:12.701467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221087+0000","last_change":"2024-03-30T02:34:50.757227+0000","last_active":"2024-03-30T02:36:27.221087+0000","last_peered":"2024-03-30T02:36:27.221087+0000","last_clean":"2024-03-30T02:36:27.221087+0000","last_became_active":"2024-03-30T02:34:50.756742+0000","last_became_peered":"2024-03-30T02:34:50.756742+0000","last_unstale":"2024-03-30T02:36:27.221087+0000","last_undegraded":"2024-03-30T02:36:27.221087+0000","last_fullsized":"2024-03-30T02:36:27.221087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:48.792202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.243069+0000","last_change":"2024-03-30T02:36:28.243069+0000","last_active":"2024-03-30T02:36:28.243069+0000","last_peered":"2024-03-30T02:36:28.243069+0000","last_clean":"2024-03-30T02:36:28.243069+0000","last_became_active":"2024-03-30T02:36:28.238658+0000","last_became_peered":"2024-03-30T02:36:28.238658+0000","last_unstale":"2024-03-30T02:36:28.243069+0000","last_undegraded":"2024-03-30T02:36:28.243069+0000","last_fullsized":"2024-03-30T02:36:28.243069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:47:59.741479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224872+0000","last_change":"2024-03-30T02:34:46.741735+0000","last_active":"2024-03-30T02:36:27.224872+0000","last_peered":"2024-03-30T02:36:27.224872+0000","last_clean":"2024-03-30T02:36:27.224872+0000","last_became_active":"2024-03-30T02:34:46.740663+0000","last_became_peered":"2024-03-30T02:34:46.740663+0000","last_unstale":"2024-03-30T02:36:27.224872+0000","last_undegraded":"2024-03-30T02:36:27.224872+0000","last_fullsized":"2024-03-30T02:36:27.224872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:09.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228383+0000","last_change":"2024-03-30T02:34:46.739082+0000","last_active":"2024-03-30T02:36:27.228383+0000","last_peered":"2024-03-30T02:36:27.228383+0000","last_clean":"2024-03-30T02:36:27.228383+0000","last_became_active":"2024-03-30T02:34:46.738732+0000","last_became_peered":"2024-03-30T02:34:46.738732+0000","last_unstale":"2024-03-30T02:36:27.228383+0000","last_undegraded":"2024-03-30T02:36:27.228383+0000","last_fullsized":"2024-03-30T02:36:27.228383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:53.961520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224158+0000","last_change":"2024-03-30T02:35:28.211005+0000","last_active":"2024-03-30T02:36:27.224158+0000","last_peered":"2024-03-30T02:36:27.224158+0000","last_clean":"2024-03-30T02:36:27.224158+0000","last_became_active":"2024-03-30T02:35:28.210610+0000","last_became_peered":"2024-03-30T02:35:28.210610+0000","last_unstale":"2024-03-30T02:36:27.224158+0000","last_undegraded":"2024-03-30T02:36:27.224158+0000","last_fullsized":"2024-03-30T02:36:27.224158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:53.054107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223977+0000","last_change":"2024-03-30T02:34:46.742590+0000","last_active":"2024-03-30T02:36:27.223977+0000","last_peered":"2024-03-30T02:36:27.223977+0000","last_clean":"2024-03-30T02:36:27.223977+0000","last_became_active":"2024-03-30T02:34:46.740904+0000","last_became_peered":"2024-03-30T02:34:46.740904+0000","last_unstale":"2024-03-30T02:36:27.223977+0000","last_undegraded":"2024-03-30T02:36:27.223977+0000","last_fullsized":"2024-03-30T02:36:27.223977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:22.653881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605025+0000","last_change":"2024-03-30T02:34:46.753119+0000","last_active":"2024-03-30T02:36:27.605025+0000","last_peered":"2024-03-30T02:36:27.605025+0000","last_clean":"2024-03-30T02:36:27.605025+0000","last_became_active":"2024-03-30T02:34:46.745680+0000","last_became_peered":"2024-03-30T02:34:46.745680+0000","last_unstale":"2024-03-30T02:36:27.605025+0000","last_undegraded":"2024-03-30T02:36:27.605025+0000","last_fullsized":"2024-03-30T02:36:27.605025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:02.352961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221594+0000","last_change":"2024-03-30T02:35:28.313317+0000","last_active":"2024-03-30T02:36:27.221594+0000","last_peered":"2024-03-30T02:36:27.221594+0000","last_clean":"2024-03-30T02:36:27.221594+0000","last_became_active":"2024-03-30T02:35:28.312929+0000","last_became_peered":"2024-03-30T02:35:28.312929+0000","last_unstale":"2024-03-30T02:36:27.221594+0000","last_undegraded":"2024-03-30T02:36:27.221594+0000","last_fullsized":"2024-03-30T02:36:27.221594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:06.390076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228760+0000","last_change":"2024-03-30T02:34:46.737882+0000","last_active":"2024-03-30T02:36:27.228760+0000","last_peered":"2024-03-30T02:36:27.228760+0000","last_clean":"2024-03-30T02:36:27.228760+0000","last_became_active":"2024-03-30T02:34:46.737546+0000","last_became_peered":"2024-03-30T02:34:46.737546+0000","last_unstale":"2024-03-30T02:36:27.228760+0000","last_undegraded":"2024-03-30T02:36:27.228760+0000","last_fullsized":"2024-03-30T02:36:27.228760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:48:41.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224639+0000","last_change":"2024-03-30T02:34:46.742703+0000","last_active":"2024-03-30T02:36:27.224639+0000","last_peered":"2024-03-30T02:36:27.224639+0000","last_clean":"2024-03-30T02:36:27.224639+0000","last_became_active":"2024-03-30T02:34:46.740689+0000","last_became_peered":"2024-03-30T02:34:46.740689+0000","last_unstale":"2024-03-30T02:36:27.224639+0000","last_undegraded":"2024-03-30T02:36:27.224639+0000","last_fullsized":"2024-03-30T02:36:27.224639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:45.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224909+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:36:27.224909+0000","last_peered":"2024-03-30T02:36:27.224909+0000","last_clean":"2024-03-30T02:36:27.224909+0000","last_became_active":"2024-03-30T02:34:46.750395+0000","last_became_peered":"2024-03-30T02:34:46.750395+0000","last_unstale":"2024-03-30T02:36:27.224909+0000","last_undegraded":"2024-03-30T02:36:27.224909+0000","last_fullsized":"2024-03-30T02:36:27.224909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:01.282677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221281+0000","last_change":"2024-03-30T02:34:46.734814+0000","last_active":"2024-03-30T02:36:27.221281+0000","last_peered":"2024-03-30T02:36:27.221281+0000","last_clean":"2024-03-30T02:36:27.221281+0000","last_became_active":"2024-03-30T02:34:46.733978+0000","last_became_peered":"2024-03-30T02:34:46.733978+0000","last_unstale":"2024-03-30T02:36:27.221281+0000","last_undegraded":"2024-03-30T02:36:27.221281+0000","last_fullsized":"2024-03-30T02:36:27.221281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:31:39.467478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224665+0000","last_change":"2024-03-30T02:34:46.748163+0000","last_active":"2024-03-30T02:36:27.224665+0000","last_peered":"2024-03-30T02:36:27.224665+0000","last_clean":"2024-03-30T02:36:27.224665+0000","last_became_active":"2024-03-30T02:34:46.738845+0000","last_became_peered":"2024-03-30T02:34:46.738845+0000","last_unstale":"2024-03-30T02:36:27.224665+0000","last_undegraded":"2024-03-30T02:36:27.224665+0000","last_fullsized":"2024-03-30T02:36:27.224665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:09.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231257+0000","last_change":"2024-03-30T02:34:46.743231+0000","last_active":"2024-03-30T02:36:27.231257+0000","last_peered":"2024-03-30T02:36:27.231257+0000","last_clean":"2024-03-30T02:36:27.231257+0000","last_became_active":"2024-03-30T02:34:46.742619+0000","last_became_peered":"2024-03-30T02:34:46.742619+0000","last_unstale":"2024-03-30T02:36:27.231257+0000","last_undegraded":"2024-03-30T02:36:27.231257+0000","last_fullsized":"2024-03-30T02:36:27.231257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:17:51.789139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221198+0000","last_change":"2024-03-30T02:34:46.745872+0000","last_active":"2024-03-30T02:36:27.221198+0000","last_peered":"2024-03-30T02:36:27.221198+0000","last_clean":"2024-03-30T02:36:27.221198+0000","last_became_active":"2024-03-30T02:34:46.745476+0000","last_became_peered":"2024-03-30T02:34:46.745476+0000","last_unstale":"2024-03-30T02:36:27.221198+0000","last_undegraded":"2024-03-30T02:36:27.221198+0000","last_fullsized":"2024-03-30T02:36:27.221198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:30.774316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228267+0000","last_change":"2024-03-30T02:34:46.746178+0000","last_active":"2024-03-30T02:36:27.228267+0000","last_peered":"2024-03-30T02:36:27.228267+0000","last_clean":"2024-03-30T02:36:27.228267+0000","last_became_active":"2024-03-30T02:34:46.745865+0000","last_became_peered":"2024-03-30T02:34:46.745865+0000","last_unstale":"2024-03-30T02:36:27.228267+0000","last_undegraded":"2024-03-30T02:36:27.228267+0000","last_fullsized":"2024-03-30T02:36:27.228267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:22.613637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248704,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76005,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":889,"num_read_kb":530,"num_write":259,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":228408,"data_compressed":5484,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579938,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":488244,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.746,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72299999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58299999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78700000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.34499999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.755,"15min":0.755},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72299999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70999999999999996}]},{"osd":3,"last update":"Sat Mar 30 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.749},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.72899999999999998}]},{"osd":4,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.82499999999999996,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.85099999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.40899999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.38800000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":1.089,"15min":1.089},"last":0.877}]},{"osd":6,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83999999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.323,"15min":1.323},"last":0.78900000000000003}]},{"osd":7,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88700000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.51400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91200000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.93799999999999994}]},{"osd":8,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.0229999999999999}]},{"osd":9,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.754,"15min":0.754},"min":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89700000000000002,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.60299999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96799999999999997,"5min":1.25,"15min":1.25},"last":0.96399999999999997}]},{"osd":10,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.629,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.89200000000000002,"5min":1.206,"15min":1.206},"last":0.91400000000000003}]}]},{"osd":10,"up_from":67,"seq":287762808870,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1114112,"data_stored":1143296,"data_compressed":61402,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.623}]},{"osd":1,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.312}]},{"osd":2,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.498,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.66700000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.35599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.38}]},{"osd":5,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.65200000000000002}]},{"osd":6,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":8,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.59599999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.69099999999999995}]},{"osd":9,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.60799999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75800000000000001,"15min":0.76100000000000001},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.753}]},{"osd":11,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.754,"15min":0.748},"min":{"1min":0.64300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.74199999999999999}]}]},{"osd":9,"up_from":62,"seq":266287972393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":491619,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.77000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":6.774,"5min":6.774,"15min":6.774},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.52100000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.56399999999999995}]},{"osd":3,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.278,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78200000000000003},"min":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.038,"5min":1.099,"15min":1.099},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":0.878},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3149999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.747},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.72199999999999998}]},{"osd":7,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.91200000000000003,"15min":0.93200000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3879999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.96199999999999997},"min":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.73299999999999998}]},{"osd":8,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88700000000000001,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.79300000000000004}]},{"osd":10,"last update":"Sat Mar 30 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.94499999999999995,"15min":0.95299999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84599999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83299999999999996}]},{"osd":11,"last update":"Sat Mar 30 02:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.81499999999999995}]}]},{"osd":8,"up_from":55,"seq":236223201325,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":487570,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.72399999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.45100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83099999999999996,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.59399999999999997,"15min":0.56599999999999995},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.53300000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.77100000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.38400000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.76200000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.45300000000000001,"15min":0.43099999999999999},"min":{"1min":0.318,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.62,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.63300000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.48299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.876,"15min":0.876},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.51100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.68300000000000005}]},{"osd":4,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.54800000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.55200000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89100000000000001,"5min":0.995,"15min":0.995},"last":0.83099999999999996}]},{"osd":5,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53000000000000003,"15min":0.499},"min":{"1min":0.436,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.71499999999999997}]},{"osd":6,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.73399999999999999,"15min":0.68200000000000005},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.499,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70299999999999996}]},{"osd":7,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.65700000000000003,"15min":0.63500000000000001},"min":{"1min":0.56499999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.71499999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96399999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.90000000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:35:38 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73499999999999999,"15min":0.72199999999999998},"min":{"1min":0.55500000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.751,"15min":0.73799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.91900000000000004}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.88700000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.69799999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92100000000000004,"5min":1.091,"15min":1.091},"last":0.94299999999999995}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.873}]}]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1084658,"data_compressed":59574,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42399999999999999,"15min":0.41899999999999998},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.442,"15min":0.436},"min":{"1min":0.35699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.56399999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.47499999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.504},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72999999999999998,"5min":1.034,"15min":1.034},"last":0.52500000000000002}]},{"osd":3,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.63400000000000001}]},{"osd":4,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.75,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59999999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41999999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85199999999999998,"5min":0.878,"15min":0.878},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83199999999999996,"5min":0.877,"15min":0.877},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83699999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.62}]},{"osd":8,"last update":"Sat Mar 30 02:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.55200000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70699999999999996}]},{"osd":9,"last update":"Sat Mar 30 02:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.59699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.71199999999999997},"min":{"1min":0.61799999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.753}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88500000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73099999999999998}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73799999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":962560,"data_stored":482979,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97699999999999998,"5min":0.998,"15min":0.998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39400000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.75800000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.069,"15min":1.069},"last":0.622}]},{"osd":3,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96199999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.74099999999999999},"min":{"1min":0.45700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1040000000000001,"5min":1.236,"15min":1.236},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":1.069,"15min":1.069},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0329999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0289999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.871},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96599999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Mar 30 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.83899999999999997,"15min":0.84699999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.111,"5min":1.393,"15min":1.393},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83899999999999997,"15min":0.84699999999999998},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":0.82899999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:36:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.89600000000000002,"15min":0.91600000000000004},"min":{"1min":0.62,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0700000000000001,"5min":1.355,"15min":1.355},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.875,"15min":0.88800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.72099999999999997}]},{"osd":9,"last update":"Sat Mar 30 02:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93000000000000005,"15min":0.96199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2030000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.88600000000000001,"15min":0.92000000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.70099999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.876,"15min":0.872},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93200000000000005,"15min":0.93100000000000005},"min":{"1min":0.745,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.24,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.81599999999999995}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":552410,"data_compressed":25402,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.75800000000000001},"min":{"1min":0.42599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59899999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57199999999999995,"15min":0.52400000000000002},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77500000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.52400000000000002},"min":{"1min":0.38500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92100000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.64400000000000002}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.438,"5min":0.378,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":1.028,"15min":1.028},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.56000000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.442,"15min":0.40799999999999997},"min":{"1min":0.33200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.433},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.313,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45000000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.64800000000000002,"15min":0.59399999999999997},"min":{"1min":0.54100000000000004,"5min":0.372,"15min":0.372},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.026},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59799999999999998,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.83999999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.62}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65700000000000003,"15min":0.63800000000000001},"min":{"1min":0.51600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77200000000000002}]},{"osd":7,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70699999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93600000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.73899999999999999,"15min":0.72899999999999998},"min":{"1min":0.58699999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.044,"15min":1.044},"last":0.81200000000000006}]},{"osd":9,"last update":"Sat Mar 30 02:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.57499999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.59799999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94899999999999995,"5min":1.056,"15min":1.056},"last":0.73899999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.58499999999999996,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.48799999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.874,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.75800000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":485134,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 02:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.54800000000000004},"min":{"1min":0.41999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.61599999999999999,"15min":0.55500000000000005},"min":{"1min":0.441,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.50900000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63200000000000001,"15min":0.59599999999999997},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.64300000000000002,"15min":0.60099999999999998},"min":{"1min":0.48599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.873}]},{"osd":3,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.42999999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.65600000000000003,"15min":0.63600000000000001},"min":{"1min":0.49099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63900000000000001,"15min":0.61799999999999999},"min":{"1min":0.51900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":1.006},"last":0.65600000000000003}]},{"osd":5,"last update":"Sat Mar 30 02:36:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.70999999999999996,"15min":0.66900000000000004},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.83699999999999997}]},{"osd":6,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84299999999999997,"5min":1.091,"15min":1.091},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.71199999999999997}]},{"osd":8,"last update":"Sat Mar 30 02:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.70999999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.84199999999999997,"15min":0.82599999999999996},"min":{"1min":0.60699999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.90600000000000003}]},{"osd":9,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0629999999999999,"5min":1.212,"15min":1.212},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.879,"15min":0.876},"min":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.79200000000000004}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.874,"15min":0.86499999999999999},"min":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.66800000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.621}]},{"osd":11,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.874,"15min":0.85199999999999998},"min":{"1min":0.76900000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.016,"5min":0.89300000000000002,"15min":0.873},"min":{"1min":0.76500000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.88500000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":553029,"data_compressed":25402,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.52200000000000002},"min":{"1min":0.41299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82899999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.51300000000000001},"min":{"1min":0.42499999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.41999999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85099999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59299999999999997,"15min":0.54300000000000004},"min":{"1min":0.46600000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.83699999999999997}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84599999999999997,"5min":0.878,"15min":0.878},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.68999999999999995}]},{"osd":5,"last update":"Sat Mar 30 02:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.498,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.74099999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.51100000000000001}]},{"osd":6,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.58099999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85199999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59599999999999997}]},{"osd":7,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.876}]},{"osd":8,"last update":"Sat Mar 30 02:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.627,"15min":0.60099999999999998},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.747},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.63600000000000001}]},{"osd":9,"last update":"Sat Mar 30 02:36:39 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92000000000000004,"5min":1.292,"15min":1.292},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.86499999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.67800000000000005}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.755,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.90700000000000003}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.80200000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":974848,"data_stored":482735,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47099999999999997,"15min":0.40699999999999997},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.434},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.39700000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.60799999999999998,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.66100000000000003,"15min":0.58499999999999996},"min":{"1min":0.47299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.80100000000000005}]},{"osd":2,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.66700000000000004,"15min":0.623},"min":{"1min":0.56899999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.64800000000000002,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.73099999999999998,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.443,"15min":0.443},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.55900000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.628}]},{"osd":5,"last update":"Sat Mar 30 02:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.70999999999999996,"15min":0.65800000000000003},"min":{"1min":0.57899999999999996,"5min":0.432,"15min":0.432},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.79100000000000004,"15min":0.71799999999999997},"min":{"1min":0.64400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Sat Mar 30 02:35:43 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.80000000000000004,"15min":0.78000000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.47799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":0.53600000000000003}]},{"osd":7,"last update":"Sat Mar 30 02:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.73299999999999998,"15min":0.70599999999999996},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.90700000000000003,"15min":0.90300000000000002},"min":{"1min":0.58699999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":0.86599999999999999}]},{"osd":8,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.004}]},{"osd":9,"last update":"Sat Mar 30 02:36:39 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94199999999999995},"min":{"1min":0.66800000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2769999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.877,"15min":0.86699999999999999},"min":{"1min":0.57899999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.70599999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:35:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.871,"15min":0.83799999999999997},"min":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.97599999999999998,"15min":0.96199999999999997},"min":{"1min":0.71399999999999997,"5min":0.629,"15min":0.629},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.90400000000000003}]},{"osd":11,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.751},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.68700000000000006}]}]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":970752,"data_stored":487587,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.38,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.747,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.35299999999999998},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.63200000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.38,"15min":0.35599999999999998},"min":{"1min":0.32300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.748,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.52600000000000002},"min":{"1min":0.45500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.438}]},{"osd":5,"last update":"Sat Mar 30 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76000000000000001,"5min":0.878,"15min":0.878},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.56499999999999995},"min":{"1min":0.48499999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72199999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.625}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Mar 30 02:36:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82399999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.503},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.40899999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.496}]},{"osd":8,"last update":"Sat Mar 30 02:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92400000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69799999999999995,"15min":0.67500000000000004},"min":{"1min":0.53100000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.72699999999999998}]},{"osd":9,"last update":"Sat Mar 30 02:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.55300000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81100000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96099999999999997,"5min":1.079,"15min":1.079},"last":0.64900000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.50900000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92900000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.74199999999999999}]},{"osd":11,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78200000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":1044480,"data_stored":1080075,"data_compressed":59574,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.42799999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54100000000000004,"15min":0.51100000000000001},"min":{"1min":0.44700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.56299999999999994,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80600000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64000000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.37},"min":{"1min":0.35199999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.65100000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.36799999999999999},"min":{"1min":0.35299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.372}]},{"osd":3,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55200000000000005},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84899999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.58799999999999997},"min":{"1min":0.47499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.627,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84799999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.67600000000000005}]},{"osd":6,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.54000000000000004}]},{"osd":7,"last update":"Sat Mar 30 02:36:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81000000000000005,"5min":1.006,"15min":1.006},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.625,"15min":0.60099999999999998},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.77400000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.73899999999999999}]},{"osd":8,"last update":"Sat Mar 30 02:36:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.437,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58199999999999996,"15min":0.56899999999999995},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52300000000000002}]},{"osd":9,"last update":"Sat Mar 30 02:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.55400000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.622,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65700000000000003}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.86599999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.747,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.871,"5min":1.024,"15min":1.024},"last":0.80600000000000005}]},{"osd":11,"last update":"Sat Mar 30 02:36:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70105,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":70724,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":70623,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T02:36:48.814 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-30T02:36:48.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T02:36:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.013+0000 7efc08c00700 1 -- 172.21.15.67:0/3166384990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04102810 msgr2=0x7efc041090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.013+0000 7efc08c00700 1 --2- 172.21.15.67:0/3166384990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04102810 0x7efc041090c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00b0d0 tx=0x7efbfc00b3e0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.013+0000 7efc08c00700 1 -- 172.21.15.67:0/3166384990 shutdown_connections 2024-03-30T02:36:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.013+0000 7efc08c00700 1 --2- 172.21.15.67:0/3166384990 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04102810 0x7efc041090c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.013+0000 7efc08c00700 1 --2- 172.21.15.67:0/3166384990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc04100c80 0x7efc041010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.013+0000 7efc08c00700 1 --2- 172.21.15.67:0/3166384990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc04101e70 0x7efc041022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.013+0000 7efc08c00700 1 -- 172.21.15.67:0/3166384990 >> 172.21.15.67:0/3166384990 conn(0x7efc040fc1f0 msgr2=0x7efc040fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:49.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.013+0000 7efc08c00700 1 -- 172.21.15.67:0/3166384990 shutdown_connections 2024-03-30T02:36:49.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.014+0000 7efc08c00700 1 -- 172.21.15.67:0/3166384990 wait complete. 2024-03-30T02:36:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.014+0000 7efc08c00700 1 Processor -- start 2024-03-30T02:36:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc08c00700 1 -- start start 2024-03-30T02:36:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc08c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04100c80 0x7efc04074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc08c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc04101e70 0x7efc040731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc08c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc04102810 0x7efc040736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc08c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc04073e80 con 0x7efc04100c80 2024-03-30T02:36:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc08c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc04073ff0 con 0x7efc04101e70 2024-03-30T02:36:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc08c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc041a0d50 con 0x7efc04102810 2024-03-30T02:36:49.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc01e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04100c80 0x7efc04074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc01e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04100c80 0x7efc04074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56676/0 (socket says 172.21.15.67:56676) 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc01e00700 1 -- 172.21.15.67:0/811720248 learned_addr learned my addr 172.21.15.67:0/811720248 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc01400700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc04101e70 0x7efc040731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.015+0000 7efc02800700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc04102810 0x7efc040736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efc01e00700 1 -- 172.21.15.67:0/811720248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc04102810 msgr2=0x7efc040736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efc01e00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc04102810 0x7efc040736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efc01e00700 1 -- 172.21.15.67:0/811720248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc04101e70 msgr2=0x7efc040731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efc01e00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc04101e70 0x7efc040731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efc01e00700 1 -- 172.21.15.67:0/811720248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc009d00 con 0x7efc04100c80 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efc01e00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04100c80 0x7efc04074b10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efbf400e970 tx=0x7efbf400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efbfaa00700 1 -- 172.21.15.67:0/811720248 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4009790 con 0x7efc04100c80 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efbfaa00700 1 -- 172.21.15.67:0/811720248 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbf400c780 con 0x7efc04100c80 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc041a0fe0 con 0x7efc04100c80 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efbfaa00700 1 -- 172.21.15.67:0/811720248 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4003be0 con 0x7efc04100c80 2024-03-30T02:36:49.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.016+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc041a1410 con 0x7efc04100c80 2024-03-30T02:36:49.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.018+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc041080b0 con 0x7efc04100c80 2024-03-30T02:36:49.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.019+0000 7efbfaa00700 1 -- 172.21.15.67:0/811720248 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7efbf400cbf0 con 0x7efc04100c80 2024-03-30T02:36:49.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.021+0000 7efbfaa00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efbe80a1230 0x7efbe80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:49.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.021+0000 7efbfaa00700 1 -- 172.21.15.67:0/811720248 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7efbf4014070 con 0x7efc04100c80 2024-03-30T02:36:49.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.021+0000 7efc01400700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efbe80a1230 0x7efbe80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:49.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.022+0000 7efc01400700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efbe80a1230 0x7efbe80a36f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009c60 tx=0x7efbf0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:49.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.025+0000 7efbfaa00700 1 -- 172.21.15.67:0/811720248 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7efbf4096190 con 0x7efc04100c80 2024-03-30T02:36:49.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.240+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc0402ce20 con 0x7efbe80a1230 2024-03-30T02:36:49.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.253+0000 7efbfaa00700 1 -- 172.21.15.67:0/811720248 <== mgr.14154 v2:172.21.15.67:6800/3801648476 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+428261 (secure 0 0 0) 0x7efc0402ce20 con 0x7efbe80a1230 2024-03-30T02:36:49.259 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efbe80a1230 msgr2=0x7efbe80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efbe80a1230 0x7efbe80a36f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009c60 tx=0x7efbf0009260 comp rx=0 tx=0).stop 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04100c80 msgr2=0x7efc04074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04100c80 0x7efc04074b10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efbf400e970 tx=0x7efbf400ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 shutdown_connections 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efbe80a1230 0x7efbe80a36f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc04100c80 0x7efc04074b10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efc04102810 0x7efc040736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 --2- 172.21.15.67:0/811720248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc04101e70 0x7efc040731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 >> 172.21.15.67:0/811720248 conn(0x7efc040fc1f0 msgr2=0x7efc041051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 shutdown_connections 2024-03-30T02:36:49.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.258+0000 7efc08c00700 1 -- 172.21.15.67:0/811720248 wait complete. 2024-03-30T02:36:49.264 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-03-30T02:36:49.278 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-03-30T02:36:49.029460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669520,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1057,"num_read_kb":668,"num_write":444,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360632,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":348653,"kb_avail":1124581192,"statfs":{"total":1151940427776,"available":1151571140608,"internally_reserved":0,"allocated":12111872,"data_stored":7825162,"data_compressed":396372,"data_compressed_allocated":540672,"data_compressed_original":2580480,"omap_allocated":19130,"internal_metadata":357020998},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":77,"num_write":12,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002944"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228788+0000","last_change":"2024-03-30T02:34:46.755257+0000","last_active":"2024-03-30T02:36:27.228788+0000","last_peered":"2024-03-30T02:36:27.228788+0000","last_clean":"2024-03-30T02:36:27.228788+0000","last_became_active":"2024-03-30T02:34:46.754922+0000","last_became_peered":"2024-03-30T02:34:46.754922+0000","last_unstale":"2024-03-30T02:36:27.228788+0000","last_undegraded":"2024-03-30T02:36:27.228788+0000","last_fullsized":"2024-03-30T02:36:27.228788+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:09.980124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605237+0000","last_change":"2024-03-30T02:34:46.742846+0000","last_active":"2024-03-30T02:36:27.605237+0000","last_peered":"2024-03-30T02:36:27.605237+0000","last_clean":"2024-03-30T02:36:27.605237+0000","last_became_active":"2024-03-30T02:34:46.738096+0000","last_became_peered":"2024-03-30T02:34:46.738096+0000","last_unstale":"2024-03-30T02:36:27.605237+0000","last_undegraded":"2024-03-30T02:36:27.605237+0000","last_fullsized":"2024-03-30T02:36:27.605237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:43:16.982251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605205+0000","last_change":"2024-03-30T02:34:46.739607+0000","last_active":"2024-03-30T02:36:27.605205+0000","last_peered":"2024-03-30T02:36:27.605205+0000","last_clean":"2024-03-30T02:36:27.605205+0000","last_became_active":"2024-03-30T02:34:46.739230+0000","last_became_peered":"2024-03-30T02:34:46.739230+0000","last_unstale":"2024-03-30T02:36:27.605205+0000","last_undegraded":"2024-03-30T02:36:27.605205+0000","last_fullsized":"2024-03-30T02:36:27.605205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:17.818874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225333+0000","last_change":"2024-03-30T02:34:46.742118+0000","last_active":"2024-03-30T02:36:27.225333+0000","last_peered":"2024-03-30T02:36:27.225333+0000","last_clean":"2024-03-30T02:36:27.225333+0000","last_became_active":"2024-03-30T02:34:46.740845+0000","last_became_peered":"2024-03-30T02:34:46.740845+0000","last_unstale":"2024-03-30T02:36:27.225333+0000","last_undegraded":"2024-03-30T02:36:27.225333+0000","last_fullsized":"2024-03-30T02:36:27.225333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:33:28.595456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231314+0000","last_change":"2024-03-30T02:34:46.739557+0000","last_active":"2024-03-30T02:36:27.231314+0000","last_peered":"2024-03-30T02:36:27.231314+0000","last_clean":"2024-03-30T02:36:27.231314+0000","last_became_active":"2024-03-30T02:34:46.739147+0000","last_became_peered":"2024-03-30T02:34:46.739147+0000","last_unstale":"2024-03-30T02:36:27.231314+0000","last_undegraded":"2024-03-30T02:36:27.231314+0000","last_fullsized":"2024-03-30T02:36:27.231314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:12:52.307386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.227799+0000","last_change":"2024-03-30T02:35:28.211536+0000","last_active":"2024-03-30T02:36:27.227799+0000","last_peered":"2024-03-30T02:36:27.227799+0000","last_clean":"2024-03-30T02:36:27.227799+0000","last_became_active":"2024-03-30T02:35:28.211085+0000","last_became_peered":"2024-03-30T02:35:28.211085+0000","last_unstale":"2024-03-30T02:36:27.227799+0000","last_undegraded":"2024-03-30T02:36:27.227799+0000","last_fullsized":"2024-03-30T02:36:27.227799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:06:34.034277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223715+0000","last_change":"2024-03-30T02:34:46.735466+0000","last_active":"2024-03-30T02:36:27.223715+0000","last_peered":"2024-03-30T02:36:27.223715+0000","last_clean":"2024-03-30T02:36:27.223715+0000","last_became_active":"2024-03-30T02:34:46.734411+0000","last_became_peered":"2024-03-30T02:34:46.734411+0000","last_unstale":"2024-03-30T02:36:27.223715+0000","last_undegraded":"2024-03-30T02:36:27.223715+0000","last_fullsized":"2024-03-30T02:36:27.223715+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:57.587768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224366+0000","last_change":"2024-03-30T02:34:46.747001+0000","last_active":"2024-03-30T02:36:27.224366+0000","last_peered":"2024-03-30T02:36:27.224366+0000","last_clean":"2024-03-30T02:36:27.224366+0000","last_became_active":"2024-03-30T02:34:46.740241+0000","last_became_peered":"2024-03-30T02:34:46.740241+0000","last_unstale":"2024-03-30T02:36:27.224366+0000","last_undegraded":"2024-03-30T02:36:27.224366+0000","last_fullsized":"2024-03-30T02:36:27.224366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:41:05.902190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231395+0000","last_change":"2024-03-30T02:34:46.745549+0000","last_active":"2024-03-30T02:36:27.231395+0000","last_peered":"2024-03-30T02:36:27.231395+0000","last_clean":"2024-03-30T02:36:27.231395+0000","last_became_active":"2024-03-30T02:34:46.745200+0000","last_became_peered":"2024-03-30T02:34:46.745200+0000","last_unstale":"2024-03-30T02:36:27.231395+0000","last_undegraded":"2024-03-30T02:36:27.231395+0000","last_fullsized":"2024-03-30T02:36:27.231395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:52:37.931440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228073+0000","last_change":"2024-03-30T02:34:46.735859+0000","last_active":"2024-03-30T02:36:27.228073+0000","last_peered":"2024-03-30T02:36:27.228073+0000","last_clean":"2024-03-30T02:36:27.228073+0000","last_became_active":"2024-03-30T02:34:46.735481+0000","last_became_peered":"2024-03-30T02:34:46.735481+0000","last_unstale":"2024-03-30T02:36:27.228073+0000","last_undegraded":"2024-03-30T02:36:27.228073+0000","last_fullsized":"2024-03-30T02:36:27.228073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:00:51.037887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224350+0000","last_change":"2024-03-30T02:34:46.741412+0000","last_active":"2024-03-30T02:36:27.224350+0000","last_peered":"2024-03-30T02:36:27.224350+0000","last_clean":"2024-03-30T02:36:27.224350+0000","last_became_active":"2024-03-30T02:34:46.741106+0000","last_became_peered":"2024-03-30T02:34:46.741106+0000","last_unstale":"2024-03-30T02:36:27.224350+0000","last_undegraded":"2024-03-30T02:36:27.224350+0000","last_fullsized":"2024-03-30T02:36:27.224350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:06:08.553028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225237+0000","last_change":"2024-03-30T02:34:46.746205+0000","last_active":"2024-03-30T02:36:27.225237+0000","last_peered":"2024-03-30T02:36:27.225237+0000","last_clean":"2024-03-30T02:36:27.225237+0000","last_became_active":"2024-03-30T02:34:46.745803+0000","last_became_peered":"2024-03-30T02:34:46.745803+0000","last_unstale":"2024-03-30T02:36:27.225237+0000","last_undegraded":"2024-03-30T02:36:27.225237+0000","last_fullsized":"2024-03-30T02:36:27.225237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:38:05.127095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221163+0000","last_change":"2024-03-30T02:34:46.722978+0000","last_active":"2024-03-30T02:36:27.221163+0000","last_peered":"2024-03-30T02:36:27.221163+0000","last_clean":"2024-03-30T02:36:27.221163+0000","last_became_active":"2024-03-30T02:34:46.722652+0000","last_became_peered":"2024-03-30T02:34:46.722652+0000","last_unstale":"2024-03-30T02:36:27.221163+0000","last_undegraded":"2024-03-30T02:36:27.221163+0000","last_fullsized":"2024-03-30T02:36:27.221163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:28:12.078216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.604850+0000","last_change":"2024-03-30T02:34:46.738895+0000","last_active":"2024-03-30T02:36:27.604850+0000","last_peered":"2024-03-30T02:36:27.604850+0000","last_clean":"2024-03-30T02:36:27.604850+0000","last_became_active":"2024-03-30T02:34:46.738510+0000","last_became_peered":"2024-03-30T02:34:46.738510+0000","last_unstale":"2024-03-30T02:36:27.604850+0000","last_undegraded":"2024-03-30T02:36:27.604850+0000","last_fullsized":"2024-03-30T02:36:27.604850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:02:09.859259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.604789+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:36:27.604789+0000","last_peered":"2024-03-30T02:36:27.604789+0000","last_clean":"2024-03-30T02:36:27.604789+0000","last_became_active":"2024-03-30T02:34:46.750301+0000","last_became_peered":"2024-03-30T02:34:46.750301+0000","last_unstale":"2024-03-30T02:36:27.604789+0000","last_undegraded":"2024-03-30T02:36:27.604789+0000","last_fullsized":"2024-03-30T02:36:27.604789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:16:06.475881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.230831+0000","last_change":"2024-03-30T02:34:46.748966+0000","last_active":"2024-03-30T02:36:27.230831+0000","last_peered":"2024-03-30T02:36:27.230831+0000","last_clean":"2024-03-30T02:36:27.230831+0000","last_became_active":"2024-03-30T02:34:46.748650+0000","last_became_peered":"2024-03-30T02:34:46.748650+0000","last_unstale":"2024-03-30T02:36:27.230831+0000","last_undegraded":"2024-03-30T02:36:27.230831+0000","last_fullsized":"2024-03-30T02:36:27.230831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:36.558864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228469+0000","last_change":"2024-03-30T02:34:46.746127+0000","last_active":"2024-03-30T02:36:27.228469+0000","last_peered":"2024-03-30T02:36:27.228469+0000","last_clean":"2024-03-30T02:36:27.228469+0000","last_became_active":"2024-03-30T02:34:46.745806+0000","last_became_peered":"2024-03-30T02:34:46.745806+0000","last_unstale":"2024-03-30T02:36:27.228469+0000","last_undegraded":"2024-03-30T02:36:27.228469+0000","last_fullsized":"2024-03-30T02:36:27.228469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:26:54.097996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.229068+0000","last_change":"2024-03-30T02:34:46.738391+0000","last_active":"2024-03-30T02:36:27.229068+0000","last_peered":"2024-03-30T02:36:27.229068+0000","last_clean":"2024-03-30T02:36:27.229068+0000","last_became_active":"2024-03-30T02:34:46.737110+0000","last_became_peered":"2024-03-30T02:34:46.737110+0000","last_unstale":"2024-03-30T02:36:27.229068+0000","last_undegraded":"2024-03-30T02:36:27.229068+0000","last_fullsized":"2024-03-30T02:36:27.229068+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:20.326107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224284+0000","last_change":"2024-03-30T02:34:46.744522+0000","last_active":"2024-03-30T02:36:27.224284+0000","last_peered":"2024-03-30T02:36:27.224284+0000","last_clean":"2024-03-30T02:36:27.224284+0000","last_became_active":"2024-03-30T02:34:46.744088+0000","last_became_peered":"2024-03-30T02:34:46.744088+0000","last_unstale":"2024-03-30T02:36:27.224284+0000","last_undegraded":"2024-03-30T02:36:27.224284+0000","last_fullsized":"2024-03-30T02:36:27.224284+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:35.698979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224043+0000","last_change":"2024-03-30T02:34:50.770855+0000","last_active":"2024-03-30T02:36:27.224043+0000","last_peered":"2024-03-30T02:36:27.224043+0000","last_clean":"2024-03-30T02:36:27.224043+0000","last_became_active":"2024-03-30T02:34:50.770420+0000","last_became_peered":"2024-03-30T02:34:50.770420+0000","last_unstale":"2024-03-30T02:36:27.224043+0000","last_undegraded":"2024-03-30T02:36:27.224043+0000","last_fullsized":"2024-03-30T02:36:27.224043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:00:08.847059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:44.255830+0000","last_change":"2024-03-30T02:34:46.207252+0000","last_active":"2024-03-30T02:36:44.255830+0000","last_peered":"2024-03-30T02:36:44.255830+0000","last_clean":"2024-03-30T02:36:44.255830+0000","last_became_active":"2024-03-30T02:34:46.206916+0000","last_became_peered":"2024-03-30T02:34:46.206916+0000","last_unstale":"2024-03-30T02:36:44.255830+0000","last_undegraded":"2024-03-30T02:36:44.255830+0000","last_fullsized":"2024-03-30T02:36:44.255830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:03:50.192245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.226846+0000","last_change":"2024-03-30T02:34:46.745778+0000","last_active":"2024-03-30T02:36:28.226846+0000","last_peered":"2024-03-30T02:36:28.226846+0000","last_clean":"2024-03-30T02:36:28.226846+0000","last_became_active":"2024-03-30T02:34:46.745471+0000","last_became_peered":"2024-03-30T02:34:46.745471+0000","last_unstale":"2024-03-30T02:36:28.226846+0000","last_undegraded":"2024-03-30T02:36:28.226846+0000","last_fullsized":"2024-03-30T02:36:28.226846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:58:57.314163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225059+0000","last_change":"2024-03-30T02:34:50.769577+0000","last_active":"2024-03-30T02:36:27.225059+0000","last_peered":"2024-03-30T02:36:27.225059+0000","last_clean":"2024-03-30T02:36:27.225059+0000","last_became_active":"2024-03-30T02:34:50.769100+0000","last_became_peered":"2024-03-30T02:34:50.769100+0000","last_unstale":"2024-03-30T02:36:27.225059+0000","last_undegraded":"2024-03-30T02:36:27.225059+0000","last_fullsized":"2024-03-30T02:36:27.225059+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:08:43.495671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.226939+0000","last_change":"2024-03-30T02:34:46.218032+0000","last_active":"2024-03-30T02:36:28.226939+0000","last_peered":"2024-03-30T02:36:28.226939+0000","last_clean":"2024-03-30T02:36:28.226939+0000","last_became_active":"2024-03-30T02:34:46.217733+0000","last_became_peered":"2024-03-30T02:34:46.217733+0000","last_unstale":"2024-03-30T02:36:28.226939+0000","last_undegraded":"2024-03-30T02:36:28.226939+0000","last_fullsized":"2024-03-30T02:36:28.226939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:14:58.413590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605282+0000","last_change":"2024-03-30T02:34:46.750588+0000","last_active":"2024-03-30T02:36:27.605282+0000","last_peered":"2024-03-30T02:36:27.605282+0000","last_clean":"2024-03-30T02:36:27.605282+0000","last_became_active":"2024-03-30T02:34:46.750016+0000","last_became_peered":"2024-03-30T02:34:46.750016+0000","last_unstale":"2024-03-30T02:36:27.605282+0000","last_undegraded":"2024-03-30T02:36:27.605282+0000","last_fullsized":"2024-03-30T02:36:27.605282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:28:19.144544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225017+0000","last_change":"2024-03-30T02:35:28.213404+0000","last_active":"2024-03-30T02:36:27.225017+0000","last_peered":"2024-03-30T02:36:27.225017+0000","last_clean":"2024-03-30T02:36:27.225017+0000","last_became_active":"2024-03-30T02:35:28.212746+0000","last_became_peered":"2024-03-30T02:35:28.212746+0000","last_unstale":"2024-03-30T02:36:27.225017+0000","last_undegraded":"2024-03-30T02:36:27.225017+0000","last_fullsized":"2024-03-30T02:36:27.225017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:24.807469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,10],"acting":[4,3,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.232111+0000","last_change":"2024-03-30T02:34:46.213741+0000","last_active":"2024-03-30T02:36:27.232111+0000","last_peered":"2024-03-30T02:36:27.232111+0000","last_clean":"2024-03-30T02:36:27.232111+0000","last_became_active":"2024-03-30T02:34:46.213395+0000","last_became_peered":"2024-03-30T02:34:46.213395+0000","last_unstale":"2024-03-30T02:36:27.232111+0000","last_undegraded":"2024-03-30T02:36:27.232111+0000","last_fullsized":"2024-03-30T02:36:27.232111+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:03:13.192415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231037+0000","last_change":"2024-03-30T02:34:46.738401+0000","last_active":"2024-03-30T02:36:27.231037+0000","last_peered":"2024-03-30T02:36:27.231037+0000","last_clean":"2024-03-30T02:36:27.231037+0000","last_became_active":"2024-03-30T02:34:46.737804+0000","last_became_peered":"2024-03-30T02:34:46.737804+0000","last_unstale":"2024-03-30T02:36:27.231037+0000","last_undegraded":"2024-03-30T02:36:27.231037+0000","last_fullsized":"2024-03-30T02:36:27.231037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:56:08.194439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224866+0000","last_change":"2024-03-30T02:34:50.766222+0000","last_active":"2024-03-30T02:36:27.224866+0000","last_peered":"2024-03-30T02:36:27.224866+0000","last_clean":"2024-03-30T02:36:27.224866+0000","last_became_active":"2024-03-30T02:34:50.765836+0000","last_became_peered":"2024-03-30T02:34:50.765836+0000","last_unstale":"2024-03-30T02:36:27.224866+0000","last_undegraded":"2024-03-30T02:36:27.224866+0000","last_fullsized":"2024-03-30T02:36:27.224866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:45:32.529535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231217+0000","last_change":"2024-03-30T02:34:46.212586+0000","last_active":"2024-03-30T02:36:27.231217+0000","last_peered":"2024-03-30T02:36:27.231217+0000","last_clean":"2024-03-30T02:36:27.231217+0000","last_became_active":"2024-03-30T02:34:46.212110+0000","last_became_peered":"2024-03-30T02:34:46.212110+0000","last_unstale":"2024-03-30T02:36:27.231217+0000","last_undegraded":"2024-03-30T02:36:27.231217+0000","last_fullsized":"2024-03-30T02:36:27.231217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:38.630953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221516+0000","last_change":"2024-03-30T02:34:46.734923+0000","last_active":"2024-03-30T02:36:27.221516+0000","last_peered":"2024-03-30T02:36:27.221516+0000","last_clean":"2024-03-30T02:36:27.221516+0000","last_became_active":"2024-03-30T02:34:46.734572+0000","last_became_peered":"2024-03-30T02:34:46.734572+0000","last_unstale":"2024-03-30T02:36:27.221516+0000","last_undegraded":"2024-03-30T02:36:27.221516+0000","last_fullsized":"2024-03-30T02:36:27.221516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:04:28.647788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224173+0000","last_change":"2024-03-30T02:34:50.757803+0000","last_active":"2024-03-30T02:36:27.224173+0000","last_peered":"2024-03-30T02:36:27.224173+0000","last_clean":"2024-03-30T02:36:27.224173+0000","last_became_active":"2024-03-30T02:34:50.757198+0000","last_became_peered":"2024-03-30T02:34:50.757198+0000","last_unstale":"2024-03-30T02:36:27.224173+0000","last_undegraded":"2024-03-30T02:36:27.224173+0000","last_fullsized":"2024-03-30T02:36:27.224173+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:54:49.399741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:46.261046+0000","last_change":"2024-03-30T02:34:46.214549+0000","last_active":"2024-03-30T02:36:46.261046+0000","last_peered":"2024-03-30T02:36:46.261046+0000","last_clean":"2024-03-30T02:36:46.261046+0000","last_became_active":"2024-03-30T02:34:46.214198+0000","last_became_peered":"2024-03-30T02:34:46.214198+0000","last_unstale":"2024-03-30T02:36:46.261046+0000","last_undegraded":"2024-03-30T02:36:46.261046+0000","last_fullsized":"2024-03-30T02:36:46.261046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:45:25.618982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1108,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231496+0000","last_change":"2024-03-30T02:34:46.737771+0000","last_active":"2024-03-30T02:36:27.231496+0000","last_peered":"2024-03-30T02:36:27.231496+0000","last_clean":"2024-03-30T02:36:27.231496+0000","last_became_active":"2024-03-30T02:34:46.737438+0000","last_became_peered":"2024-03-30T02:34:46.737438+0000","last_unstale":"2024-03-30T02:36:27.231496+0000","last_undegraded":"2024-03-30T02:36:27.231496+0000","last_fullsized":"2024-03-30T02:36:27.231496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:31:08.542435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224131+0000","last_change":"2024-03-30T02:34:50.767304+0000","last_active":"2024-03-30T02:36:27.224131+0000","last_peered":"2024-03-30T02:36:27.224131+0000","last_clean":"2024-03-30T02:36:27.224131+0000","last_became_active":"2024-03-30T02:34:50.766909+0000","last_became_peered":"2024-03-30T02:34:50.766909+0000","last_unstale":"2024-03-30T02:36:27.224131+0000","last_undegraded":"2024-03-30T02:36:27.224131+0000","last_fullsized":"2024-03-30T02:36:27.224131+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:40:24.021991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.227724+0000","last_change":"2024-03-30T02:34:46.214937+0000","last_active":"2024-03-30T02:36:27.227724+0000","last_peered":"2024-03-30T02:36:27.227724+0000","last_clean":"2024-03-30T02:36:27.227724+0000","last_became_active":"2024-03-30T02:34:46.214503+0000","last_became_peered":"2024-03-30T02:34:46.214503+0000","last_unstale":"2024-03-30T02:36:27.227724+0000","last_undegraded":"2024-03-30T02:36:27.227724+0000","last_fullsized":"2024-03-30T02:36:27.227724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:20:11.624919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221928+0000","last_change":"2024-03-30T02:34:46.737374+0000","last_active":"2024-03-30T02:36:27.221928+0000","last_peered":"2024-03-30T02:36:27.221928+0000","last_clean":"2024-03-30T02:36:27.221928+0000","last_became_active":"2024-03-30T02:34:46.735307+0000","last_became_peered":"2024-03-30T02:34:46.735307+0000","last_unstale":"2024-03-30T02:36:27.221928+0000","last_undegraded":"2024-03-30T02:36:27.221928+0000","last_fullsized":"2024-03-30T02:36:27.221928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:30.482728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228360+0000","last_change":"2024-03-30T02:34:50.768016+0000","last_active":"2024-03-30T02:36:27.228360+0000","last_peered":"2024-03-30T02:36:27.228360+0000","last_clean":"2024-03-30T02:36:27.228360+0000","last_became_active":"2024-03-30T02:34:50.767564+0000","last_became_peered":"2024-03-30T02:34:50.767564+0000","last_unstale":"2024-03-30T02:36:27.228360+0000","last_undegraded":"2024-03-30T02:36:27.228360+0000","last_fullsized":"2024-03-30T02:36:27.228360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:35:16.888754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.220995+0000","last_change":"2024-03-30T02:34:46.212982+0000","last_active":"2024-03-30T02:36:27.220995+0000","last_peered":"2024-03-30T02:36:27.220995+0000","last_clean":"2024-03-30T02:36:27.220995+0000","last_became_active":"2024-03-30T02:34:46.212550+0000","last_became_peered":"2024-03-30T02:34:46.212550+0000","last_unstale":"2024-03-30T02:36:27.220995+0000","last_undegraded":"2024-03-30T02:36:27.220995+0000","last_fullsized":"2024-03-30T02:36:27.220995+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:44:50.156615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228102+0000","last_change":"2024-03-30T02:35:28.213774+0000","last_active":"2024-03-30T02:36:27.228102+0000","last_peered":"2024-03-30T02:36:27.228102+0000","last_clean":"2024-03-30T02:36:27.228102+0000","last_became_active":"2024-03-30T02:35:28.213085+0000","last_became_peered":"2024-03-30T02:35:28.213085+0000","last_unstale":"2024-03-30T02:36:27.228102+0000","last_undegraded":"2024-03-30T02:36:27.228102+0000","last_fullsized":"2024-03-30T02:36:27.228102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:56:30.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605207+0000","last_change":"2024-03-30T02:34:50.773036+0000","last_active":"2024-03-30T02:36:27.605207+0000","last_peered":"2024-03-30T02:36:27.605207+0000","last_clean":"2024-03-30T02:36:27.605207+0000","last_became_active":"2024-03-30T02:34:50.772614+0000","last_became_peered":"2024-03-30T02:34:50.772614+0000","last_unstale":"2024-03-30T02:36:27.605207+0000","last_undegraded":"2024-03-30T02:36:27.605207+0000","last_fullsized":"2024-03-30T02:36:27.605207+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:54:28.792793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221282+0000","last_change":"2024-03-30T02:34:46.209802+0000","last_active":"2024-03-30T02:36:27.221282+0000","last_peered":"2024-03-30T02:36:27.221282+0000","last_clean":"2024-03-30T02:36:27.221282+0000","last_became_active":"2024-03-30T02:34:46.209246+0000","last_became_peered":"2024-03-30T02:34:46.209246+0000","last_unstale":"2024-03-30T02:36:27.221282+0000","last_undegraded":"2024-03-30T02:36:27.221282+0000","last_fullsized":"2024-03-30T02:36:27.221282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:09:57.073360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228970+0000","last_change":"2024-03-30T02:34:46.743638+0000","last_active":"2024-03-30T02:36:27.228970+0000","last_peered":"2024-03-30T02:36:27.228970+0000","last_clean":"2024-03-30T02:36:27.228970+0000","last_became_active":"2024-03-30T02:34:46.743246+0000","last_became_peered":"2024-03-30T02:34:46.743246+0000","last_unstale":"2024-03-30T02:36:27.228970+0000","last_undegraded":"2024-03-30T02:36:27.228970+0000","last_fullsized":"2024-03-30T02:36:27.228970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:49:55.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225892+0000","last_change":"2024-03-30T02:34:50.768586+0000","last_active":"2024-03-30T02:36:27.225892+0000","last_peered":"2024-03-30T02:36:27.225892+0000","last_clean":"2024-03-30T02:36:27.225892+0000","last_became_active":"2024-03-30T02:34:50.766894+0000","last_became_peered":"2024-03-30T02:34:50.766894+0000","last_unstale":"2024-03-30T02:36:27.225892+0000","last_undegraded":"2024-03-30T02:36:27.225892+0000","last_fullsized":"2024-03-30T02:36:27.225892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:36:39.329608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'6","reported_seq":204,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:48.263292+0000","last_change":"2024-03-30T02:34:46.204994+0000","last_active":"2024-03-30T02:36:48.263292+0000","last_peered":"2024-03-30T02:36:48.263292+0000","last_clean":"2024-03-30T02:36:48.263292+0000","last_became_active":"2024-03-30T02:34:46.204710+0000","last_became_peered":"2024-03-30T02:34:46.204710+0000","last_unstale":"2024-03-30T02:36:48.263292+0000","last_undegraded":"2024-03-30T02:36:48.263292+0000","last_fullsized":"2024-03-30T02:36:48.263292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:08:39.206516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231396+0000","last_change":"2024-03-30T02:34:46.733573+0000","last_active":"2024-03-30T02:36:27.231396+0000","last_peered":"2024-03-30T02:36:27.231396+0000","last_clean":"2024-03-30T02:36:27.231396+0000","last_became_active":"2024-03-30T02:34:46.733241+0000","last_became_peered":"2024-03-30T02:34:46.733241+0000","last_unstale":"2024-03-30T02:36:27.231396+0000","last_undegraded":"2024-03-30T02:36:27.231396+0000","last_fullsized":"2024-03-30T02:36:27.231396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:07:10.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231018+0000","last_change":"2024-03-30T02:35:28.313493+0000","last_active":"2024-03-30T02:36:27.231018+0000","last_peered":"2024-03-30T02:36:27.231018+0000","last_clean":"2024-03-30T02:36:27.231018+0000","last_became_active":"2024-03-30T02:35:28.313065+0000","last_became_peered":"2024-03-30T02:35:28.313065+0000","last_unstale":"2024-03-30T02:36:27.231018+0000","last_undegraded":"2024-03-30T02:36:27.231018+0000","last_fullsized":"2024-03-30T02:36:27.231018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:44:52.675251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224410+0000","last_change":"2024-03-30T02:34:46.212732+0000","last_active":"2024-03-30T02:36:27.224410+0000","last_peered":"2024-03-30T02:36:27.224410+0000","last_clean":"2024-03-30T02:36:27.224410+0000","last_became_active":"2024-03-30T02:34:46.212306+0000","last_became_peered":"2024-03-30T02:34:46.212306+0000","last_unstale":"2024-03-30T02:36:27.224410+0000","last_undegraded":"2024-03-30T02:36:27.224410+0000","last_fullsized":"2024-03-30T02:36:27.224410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:27:15.766203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605314+0000","last_change":"2024-03-30T02:34:46.749420+0000","last_active":"2024-03-30T02:36:27.605314+0000","last_peered":"2024-03-30T02:36:27.605314+0000","last_clean":"2024-03-30T02:36:27.605314+0000","last_became_active":"2024-03-30T02:34:46.745286+0000","last_became_peered":"2024-03-30T02:34:46.745286+0000","last_unstale":"2024-03-30T02:36:27.605314+0000","last_undegraded":"2024-03-30T02:36:27.605314+0000","last_fullsized":"2024-03-30T02:36:27.605314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:29:01.489862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225302+0000","last_change":"2024-03-30T02:34:50.760542+0000","last_active":"2024-03-30T02:36:27.225302+0000","last_peered":"2024-03-30T02:36:27.225302+0000","last_clean":"2024-03-30T02:36:27.225302+0000","last_became_active":"2024-03-30T02:34:50.760087+0000","last_became_peered":"2024-03-30T02:34:50.760087+0000","last_unstale":"2024-03-30T02:36:27.225302+0000","last_undegraded":"2024-03-30T02:36:27.225302+0000","last_fullsized":"2024-03-30T02:36:27.225302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:16:46.352189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221437+0000","last_change":"2024-03-30T02:34:46.224058+0000","last_active":"2024-03-30T02:36:27.221437+0000","last_peered":"2024-03-30T02:36:27.221437+0000","last_clean":"2024-03-30T02:36:27.221437+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:36:27.221437+0000","last_undegraded":"2024-03-30T02:36:27.221437+0000","last_fullsized":"2024-03-30T02:36:27.221437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:31:10.124533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231126+0000","last_change":"2024-03-30T02:34:46.741413+0000","last_active":"2024-03-30T02:36:27.231126+0000","last_peered":"2024-03-30T02:36:27.231126+0000","last_clean":"2024-03-30T02:36:27.231126+0000","last_became_active":"2024-03-30T02:34:46.741112+0000","last_became_peered":"2024-03-30T02:34:46.741112+0000","last_unstale":"2024-03-30T02:36:27.231126+0000","last_undegraded":"2024-03-30T02:36:27.231126+0000","last_fullsized":"2024-03-30T02:36:27.231126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:36:34.820166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.229215+0000","last_change":"2024-03-30T02:34:50.752315+0000","last_active":"2024-03-30T02:36:27.229215+0000","last_peered":"2024-03-30T02:36:27.229215+0000","last_clean":"2024-03-30T02:36:27.229215+0000","last_became_active":"2024-03-30T02:34:50.750985+0000","last_became_peered":"2024-03-30T02:34:50.750985+0000","last_unstale":"2024-03-30T02:36:27.229215+0000","last_undegraded":"2024-03-30T02:36:27.229215+0000","last_fullsized":"2024-03-30T02:36:27.229215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:37.271063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'44","reported_seq":255,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:48.264561+0000","last_change":"2024-03-30T02:34:46.214759+0000","last_active":"2024-03-30T02:36:48.264561+0000","last_peered":"2024-03-30T02:36:48.264561+0000","last_clean":"2024-03-30T02:36:48.264561+0000","last_became_active":"2024-03-30T02:34:46.214388+0000","last_became_peered":"2024-03-30T02:34:46.214388+0000","last_unstale":"2024-03-30T02:36:48.264561+0000","last_undegraded":"2024-03-30T02:36:48.264561+0000","last_fullsized":"2024-03-30T02:36:48.264561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:56:41.100461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70921,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":191,"num_read_kb":166,"num_write":46,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231853+0000","last_change":"2024-03-30T02:34:46.741112+0000","last_active":"2024-03-30T02:36:27.231853+0000","last_peered":"2024-03-30T02:36:27.231853+0000","last_clean":"2024-03-30T02:36:27.231853+0000","last_became_active":"2024-03-30T02:34:46.739664+0000","last_became_peered":"2024-03-30T02:34:46.739664+0000","last_unstale":"2024-03-30T02:36:27.231853+0000","last_undegraded":"2024-03-30T02:36:27.231853+0000","last_fullsized":"2024-03-30T02:36:27.231853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:13:07.763942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221733+0000","last_change":"2024-03-30T02:34:50.765645+0000","last_active":"2024-03-30T02:36:27.221733+0000","last_peered":"2024-03-30T02:36:27.221733+0000","last_clean":"2024-03-30T02:36:27.221733+0000","last_became_active":"2024-03-30T02:34:50.765345+0000","last_became_peered":"2024-03-30T02:34:50.765345+0000","last_unstale":"2024-03-30T02:36:27.221733+0000","last_undegraded":"2024-03-30T02:36:27.221733+0000","last_fullsized":"2024-03-30T02:36:27.221733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:25:38.591525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224473+0000","last_change":"2024-03-30T02:34:46.216643+0000","last_active":"2024-03-30T02:36:27.224473+0000","last_peered":"2024-03-30T02:36:27.224473+0000","last_clean":"2024-03-30T02:36:27.224473+0000","last_became_active":"2024-03-30T02:34:46.216188+0000","last_became_peered":"2024-03-30T02:34:46.216188+0000","last_unstale":"2024-03-30T02:36:27.224473+0000","last_undegraded":"2024-03-30T02:36:27.224473+0000","last_fullsized":"2024-03-30T02:36:27.224473+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:49:10.488446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221353+0000","last_change":"2024-03-30T02:34:46.748313+0000","last_active":"2024-03-30T02:36:27.221353+0000","last_peered":"2024-03-30T02:36:27.221353+0000","last_clean":"2024-03-30T02:36:27.221353+0000","last_became_active":"2024-03-30T02:34:46.742897+0000","last_became_peered":"2024-03-30T02:34:46.742897+0000","last_unstale":"2024-03-30T02:36:27.221353+0000","last_undegraded":"2024-03-30T02:36:27.221353+0000","last_fullsized":"2024-03-30T02:36:27.221353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:18:00.109790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228185+0000","last_change":"2024-03-30T02:34:50.760993+0000","last_active":"2024-03-30T02:36:27.228185+0000","last_peered":"2024-03-30T02:36:27.228185+0000","last_clean":"2024-03-30T02:36:27.228185+0000","last_became_active":"2024-03-30T02:34:50.760535+0000","last_became_peered":"2024-03-30T02:34:50.760535+0000","last_unstale":"2024-03-30T02:36:27.228185+0000","last_undegraded":"2024-03-30T02:36:27.228185+0000","last_fullsized":"2024-03-30T02:36:27.228185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:36:15.569533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221659+0000","last_change":"2024-03-30T02:34:46.216603+0000","last_active":"2024-03-30T02:36:27.221659+0000","last_peered":"2024-03-30T02:36:27.221659+0000","last_clean":"2024-03-30T02:36:27.221659+0000","last_became_active":"2024-03-30T02:34:46.216231+0000","last_became_peered":"2024-03-30T02:34:46.216231+0000","last_unstale":"2024-03-30T02:36:27.221659+0000","last_undegraded":"2024-03-30T02:36:27.221659+0000","last_fullsized":"2024-03-30T02:36:27.221659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:51:00.659652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224126+0000","last_change":"2024-03-30T02:34:46.742815+0000","last_active":"2024-03-30T02:36:27.224126+0000","last_peered":"2024-03-30T02:36:27.224126+0000","last_clean":"2024-03-30T02:36:27.224126+0000","last_became_active":"2024-03-30T02:34:46.742487+0000","last_became_peered":"2024-03-30T02:34:46.742487+0000","last_unstale":"2024-03-30T02:36:27.224126+0000","last_undegraded":"2024-03-30T02:36:27.224126+0000","last_fullsized":"2024-03-30T02:36:27.224126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:41:07.182990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228857+0000","last_change":"2024-03-30T02:34:50.770398+0000","last_active":"2024-03-30T02:36:27.228857+0000","last_peered":"2024-03-30T02:36:27.228857+0000","last_clean":"2024-03-30T02:36:27.228857+0000","last_became_active":"2024-03-30T02:34:50.770000+0000","last_became_peered":"2024-03-30T02:34:50.770000+0000","last_unstale":"2024-03-30T02:36:27.228857+0000","last_undegraded":"2024-03-30T02:36:27.228857+0000","last_fullsized":"2024-03-30T02:36:27.228857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:14:49.057081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":116,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:44.256642+0000","last_change":"2024-03-30T02:34:46.209403+0000","last_active":"2024-03-30T02:36:44.256642+0000","last_peered":"2024-03-30T02:36:44.256642+0000","last_clean":"2024-03-30T02:36:44.256642+0000","last_became_active":"2024-03-30T02:34:46.209065+0000","last_became_peered":"2024-03-30T02:34:46.209065+0000","last_unstale":"2024-03-30T02:36:44.256642+0000","last_undegraded":"2024-03-30T02:36:44.256642+0000","last_fullsized":"2024-03-30T02:36:44.256642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:36:20.427720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1074,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223778+0000","last_change":"2024-03-30T02:34:46.734861+0000","last_active":"2024-03-30T02:36:27.223778+0000","last_peered":"2024-03-30T02:36:27.223778+0000","last_clean":"2024-03-30T02:36:27.223778+0000","last_became_active":"2024-03-30T02:34:46.734376+0000","last_became_peered":"2024-03-30T02:34:46.734376+0000","last_unstale":"2024-03-30T02:36:27.223778+0000","last_undegraded":"2024-03-30T02:36:27.223778+0000","last_fullsized":"2024-03-30T02:36:27.223778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:08:36.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221531+0000","last_change":"2024-03-30T02:34:50.767469+0000","last_active":"2024-03-30T02:36:27.221531+0000","last_peered":"2024-03-30T02:36:27.221531+0000","last_clean":"2024-03-30T02:36:27.221531+0000","last_became_active":"2024-03-30T02:34:50.765460+0000","last_became_peered":"2024-03-30T02:34:50.765460+0000","last_unstale":"2024-03-30T02:36:27.221531+0000","last_undegraded":"2024-03-30T02:36:27.221531+0000","last_fullsized":"2024-03-30T02:36:27.221531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:54:06.940171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231014+0000","last_change":"2024-03-30T02:34:46.224053+0000","last_active":"2024-03-30T02:36:27.231014+0000","last_peered":"2024-03-30T02:36:27.231014+0000","last_clean":"2024-03-30T02:36:27.231014+0000","last_became_active":"2024-03-30T02:34:46.223666+0000","last_became_peered":"2024-03-30T02:34:46.223666+0000","last_unstale":"2024-03-30T02:36:27.231014+0000","last_undegraded":"2024-03-30T02:36:27.231014+0000","last_fullsized":"2024-03-30T02:36:27.231014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:18:22.976298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221390+0000","last_change":"2024-03-30T02:34:46.734838+0000","last_active":"2024-03-30T02:36:27.221390+0000","last_peered":"2024-03-30T02:36:27.221390+0000","last_clean":"2024-03-30T02:36:27.221390+0000","last_became_active":"2024-03-30T02:34:46.734428+0000","last_became_peered":"2024-03-30T02:34:46.734428+0000","last_unstale":"2024-03-30T02:36:27.221390+0000","last_undegraded":"2024-03-30T02:36:27.221390+0000","last_fullsized":"2024-03-30T02:36:27.221390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:42:51.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221785+0000","last_change":"2024-03-30T02:34:50.771150+0000","last_active":"2024-03-30T02:36:27.221785+0000","last_peered":"2024-03-30T02:36:27.221785+0000","last_clean":"2024-03-30T02:36:27.221785+0000","last_became_active":"2024-03-30T02:34:50.770703+0000","last_became_peered":"2024-03-30T02:34:50.770703+0000","last_unstale":"2024-03-30T02:36:27.221785+0000","last_undegraded":"2024-03-30T02:36:27.221785+0000","last_fullsized":"2024-03-30T02:36:27.221785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:21:04.466664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231835+0000","last_change":"2024-03-30T02:34:46.203558+0000","last_active":"2024-03-30T02:36:27.231835+0000","last_peered":"2024-03-30T02:36:27.231835+0000","last_clean":"2024-03-30T02:36:27.231835+0000","last_became_active":"2024-03-30T02:34:46.202896+0000","last_became_peered":"2024-03-30T02:34:46.202896+0000","last_unstale":"2024-03-30T02:36:27.231835+0000","last_undegraded":"2024-03-30T02:36:27.231835+0000","last_fullsized":"2024-03-30T02:36:27.231835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:30:33.945665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228622+0000","last_change":"2024-03-30T02:35:28.212775+0000","last_active":"2024-03-30T02:36:27.228622+0000","last_peered":"2024-03-30T02:36:27.228622+0000","last_clean":"2024-03-30T02:36:27.228622+0000","last_became_active":"2024-03-30T02:35:28.212368+0000","last_became_peered":"2024-03-30T02:35:28.212368+0000","last_unstale":"2024-03-30T02:36:27.228622+0000","last_undegraded":"2024-03-30T02:36:27.228622+0000","last_fullsized":"2024-03-30T02:36:27.228622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:11:50.828726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231620+0000","last_change":"2024-03-30T02:34:50.797758+0000","last_active":"2024-03-30T02:36:27.231620+0000","last_peered":"2024-03-30T02:36:27.231620+0000","last_clean":"2024-03-30T02:36:27.231620+0000","last_became_active":"2024-03-30T02:34:50.797311+0000","last_became_peered":"2024-03-30T02:34:50.797311+0000","last_unstale":"2024-03-30T02:36:27.231620+0000","last_undegraded":"2024-03-30T02:36:27.231620+0000","last_fullsized":"2024-03-30T02:36:27.231620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:23:32.337422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228443+0000","last_change":"2024-03-30T02:34:46.224397+0000","last_active":"2024-03-30T02:36:27.228443+0000","last_peered":"2024-03-30T02:36:27.228443+0000","last_clean":"2024-03-30T02:36:27.228443+0000","last_became_active":"2024-03-30T02:34:46.223825+0000","last_became_peered":"2024-03-30T02:34:46.223825+0000","last_unstale":"2024-03-30T02:36:27.228443+0000","last_undegraded":"2024-03-30T02:36:27.228443+0000","last_fullsized":"2024-03-30T02:36:27.228443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:53.154665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.227952+0000","last_change":"2024-03-30T02:34:46.741358+0000","last_active":"2024-03-30T02:36:27.227952+0000","last_peered":"2024-03-30T02:36:27.227952+0000","last_clean":"2024-03-30T02:36:27.227952+0000","last_became_active":"2024-03-30T02:34:46.740734+0000","last_became_peered":"2024-03-30T02:34:46.740734+0000","last_unstale":"2024-03-30T02:36:27.227952+0000","last_undegraded":"2024-03-30T02:36:27.227952+0000","last_fullsized":"2024-03-30T02:36:27.227952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:08:59.678431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221512+0000","last_change":"2024-03-30T02:34:50.761998+0000","last_active":"2024-03-30T02:36:27.221512+0000","last_peered":"2024-03-30T02:36:27.221512+0000","last_clean":"2024-03-30T02:36:27.221512+0000","last_became_active":"2024-03-30T02:34:50.761560+0000","last_became_peered":"2024-03-30T02:34:50.761560+0000","last_unstale":"2024-03-30T02:36:27.221512+0000","last_undegraded":"2024-03-30T02:36:27.221512+0000","last_fullsized":"2024-03-30T02:36:27.221512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:57:17.213624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":112,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:44.256432+0000","last_change":"2024-03-30T02:34:46.213427+0000","last_active":"2024-03-30T02:36:44.256432+0000","last_peered":"2024-03-30T02:36:44.256432+0000","last_clean":"2024-03-30T02:36:44.256432+0000","last_became_active":"2024-03-30T02:34:46.213082+0000","last_became_peered":"2024-03-30T02:34:46.213082+0000","last_unstale":"2024-03-30T02:36:44.256432+0000","last_undegraded":"2024-03-30T02:36:44.256432+0000","last_fullsized":"2024-03-30T02:36:44.256432+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:31:35.513723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228590+0000","last_change":"2024-03-30T02:34:46.747197+0000","last_active":"2024-03-30T02:36:27.228590+0000","last_peered":"2024-03-30T02:36:27.228590+0000","last_clean":"2024-03-30T02:36:27.228590+0000","last_became_active":"2024-03-30T02:34:46.743841+0000","last_became_peered":"2024-03-30T02:34:46.743841+0000","last_unstale":"2024-03-30T02:36:27.228590+0000","last_undegraded":"2024-03-30T02:36:27.228590+0000","last_fullsized":"2024-03-30T02:36:27.228590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:46:09.234382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221642+0000","last_change":"2024-03-30T02:34:50.797423+0000","last_active":"2024-03-30T02:36:27.221642+0000","last_peered":"2024-03-30T02:36:27.221642+0000","last_clean":"2024-03-30T02:36:27.221642+0000","last_became_active":"2024-03-30T02:34:50.796746+0000","last_became_peered":"2024-03-30T02:34:50.796746+0000","last_unstale":"2024-03-30T02:36:27.221642+0000","last_undegraded":"2024-03-30T02:36:27.221642+0000","last_fullsized":"2024-03-30T02:36:27.221642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:21:24.014131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.238257+0000","last_change":"2024-03-30T02:34:46.734972+0000","last_active":"2024-03-30T02:36:28.238257+0000","last_peered":"2024-03-30T02:36:28.238257+0000","last_clean":"2024-03-30T02:36:28.238257+0000","last_became_active":"2024-03-30T02:34:46.734520+0000","last_became_peered":"2024-03-30T02:34:46.734520+0000","last_unstale":"2024-03-30T02:36:28.238257+0000","last_undegraded":"2024-03-30T02:36:28.238257+0000","last_fullsized":"2024-03-30T02:36:28.238257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:10:15.635060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":112,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:43.255021+0000","last_change":"2024-03-30T02:34:46.224036+0000","last_active":"2024-03-30T02:36:43.255021+0000","last_peered":"2024-03-30T02:36:43.255021+0000","last_clean":"2024-03-30T02:36:43.255021+0000","last_became_active":"2024-03-30T02:34:46.223706+0000","last_became_peered":"2024-03-30T02:34:46.223706+0000","last_unstale":"2024-03-30T02:36:43.255021+0000","last_undegraded":"2024-03-30T02:36:43.255021+0000","last_fullsized":"2024-03-30T02:36:43.255021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:21:40.458345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"83'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224187+0000","last_change":"2024-03-30T02:34:50.757894+0000","last_active":"2024-03-30T02:36:27.224187+0000","last_peered":"2024-03-30T02:36:27.224187+0000","last_clean":"2024-03-30T02:36:27.224187+0000","last_became_active":"2024-03-30T02:34:50.757486+0000","last_became_peered":"2024-03-30T02:34:50.757486+0000","last_unstale":"2024-03-30T02:36:27.224187+0000","last_undegraded":"2024-03-30T02:36:27.224187+0000","last_fullsized":"2024-03-30T02:36:27.224187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:27:30.609851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225449+0000","last_change":"2024-03-30T02:34:46.745990+0000","last_active":"2024-03-30T02:36:27.225449+0000","last_peered":"2024-03-30T02:36:27.225449+0000","last_clean":"2024-03-30T02:36:27.225449+0000","last_became_active":"2024-03-30T02:34:46.741937+0000","last_became_peered":"2024-03-30T02:34:46.741937+0000","last_unstale":"2024-03-30T02:36:27.225449+0000","last_undegraded":"2024-03-30T02:36:27.225449+0000","last_fullsized":"2024-03-30T02:36:27.225449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:26:17.833164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225196+0000","last_change":"2024-03-30T02:34:46.208858+0000","last_active":"2024-03-30T02:36:27.225196+0000","last_peered":"2024-03-30T02:36:27.225196+0000","last_clean":"2024-03-30T02:36:27.225196+0000","last_became_active":"2024-03-30T02:34:46.208479+0000","last_became_peered":"2024-03-30T02:34:46.208479+0000","last_unstale":"2024-03-30T02:36:27.225196+0000","last_undegraded":"2024-03-30T02:36:27.225196+0000","last_fullsized":"2024-03-30T02:36:27.225196+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:58:14.512294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223677+0000","last_change":"2024-03-30T02:33:56.208237+0000","last_active":"2024-03-30T02:36:27.223677+0000","last_peered":"2024-03-30T02:36:27.223677+0000","last_clean":"2024-03-30T02:36:27.223677+0000","last_became_active":"2024-03-30T02:33:56.207820+0000","last_became_peered":"2024-03-30T02:33:56.207820+0000","last_unstale":"2024-03-30T02:36:27.223677+0000","last_undegraded":"2024-03-30T02:36:27.223677+0000","last_fullsized":"2024-03-30T02:36:27.223677+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:31:23.922434+0000","last_clean_scrub_stamp":"2024-03-30T02:31:23.922434+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:15:26.198503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221375+0000","last_change":"2024-03-30T02:35:28.213743+0000","last_active":"2024-03-30T02:36:27.221375+0000","last_peered":"2024-03-30T02:36:27.221375+0000","last_clean":"2024-03-30T02:36:27.221375+0000","last_became_active":"2024-03-30T02:35:28.213318+0000","last_became_peered":"2024-03-30T02:35:28.213318+0000","last_unstale":"2024-03-30T02:36:27.221375+0000","last_undegraded":"2024-03-30T02:36:27.221375+0000","last_fullsized":"2024-03-30T02:36:27.221375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:15:35.803168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.220877+0000","last_change":"2024-03-30T02:34:46.739757+0000","last_active":"2024-03-30T02:36:27.220877+0000","last_peered":"2024-03-30T02:36:27.220877+0000","last_clean":"2024-03-30T02:36:27.220877+0000","last_became_active":"2024-03-30T02:34:46.738269+0000","last_became_peered":"2024-03-30T02:34:46.738269+0000","last_unstale":"2024-03-30T02:36:27.220877+0000","last_undegraded":"2024-03-30T02:36:27.220877+0000","last_fullsized":"2024-03-30T02:36:27.220877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:12:03.411187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.235101+0000","last_change":"2024-03-30T02:34:46.212532+0000","last_active":"2024-03-30T02:36:28.235101+0000","last_peered":"2024-03-30T02:36:28.235101+0000","last_clean":"2024-03-30T02:36:28.235101+0000","last_became_active":"2024-03-30T02:34:46.212191+0000","last_became_peered":"2024-03-30T02:34:46.212191+0000","last_unstale":"2024-03-30T02:36:28.235101+0000","last_undegraded":"2024-03-30T02:36:28.235101+0000","last_fullsized":"2024-03-30T02:36:28.235101+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:19.120960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228188+0000","last_change":"2024-03-30T02:34:50.759911+0000","last_active":"2024-03-30T02:36:27.228188+0000","last_peered":"2024-03-30T02:36:27.228188+0000","last_clean":"2024-03-30T02:36:27.228188+0000","last_became_active":"2024-03-30T02:34:50.759487+0000","last_became_peered":"2024-03-30T02:34:50.759487+0000","last_unstale":"2024-03-30T02:36:27.228188+0000","last_undegraded":"2024-03-30T02:36:27.228188+0000","last_fullsized":"2024-03-30T02:36:27.228188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:16:46.403754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231948+0000","last_change":"2024-03-30T02:34:46.217092+0000","last_active":"2024-03-30T02:36:27.231948+0000","last_peered":"2024-03-30T02:36:27.231948+0000","last_clean":"2024-03-30T02:36:27.231948+0000","last_became_active":"2024-03-30T02:34:46.216779+0000","last_became_peered":"2024-03-30T02:34:46.216779+0000","last_unstale":"2024-03-30T02:36:27.231948+0000","last_undegraded":"2024-03-30T02:36:27.231948+0000","last_fullsized":"2024-03-30T02:36:27.231948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:53:10.894577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221586+0000","last_change":"2024-03-30T02:34:46.735212+0000","last_active":"2024-03-30T02:36:27.221586+0000","last_peered":"2024-03-30T02:36:27.221586+0000","last_clean":"2024-03-30T02:36:27.221586+0000","last_became_active":"2024-03-30T02:34:46.734282+0000","last_became_peered":"2024-03-30T02:34:46.734282+0000","last_unstale":"2024-03-30T02:36:27.221586+0000","last_undegraded":"2024-03-30T02:36:27.221586+0000","last_fullsized":"2024-03-30T02:36:27.221586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:59:28.596961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224232+0000","last_change":"2024-03-30T02:34:50.799290+0000","last_active":"2024-03-30T02:36:27.224232+0000","last_peered":"2024-03-30T02:36:27.224232+0000","last_clean":"2024-03-30T02:36:27.224232+0000","last_became_active":"2024-03-30T02:34:50.797524+0000","last_became_peered":"2024-03-30T02:34:50.797524+0000","last_unstale":"2024-03-30T02:36:27.224232+0000","last_undegraded":"2024-03-30T02:36:27.224232+0000","last_fullsized":"2024-03-30T02:36:27.224232+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:53:27.931444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:31.369328+0000","last_change":"2024-03-30T02:36:28.236366+0000","last_active":"2024-03-30T02:36:31.369328+0000","last_peered":"2024-03-30T02:36:31.369328+0000","last_clean":"2024-03-30T02:36:31.369328+0000","last_became_active":"2024-03-30T02:36:28.228981+0000","last_became_peered":"2024-03-30T02:36:28.228981+0000","last_unstale":"2024-03-30T02:36:31.369328+0000","last_undegraded":"2024-03-30T02:36:31.369328+0000","last_fullsized":"2024-03-30T02:36:31.369328+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:28:26.747807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224734+0000","last_change":"2024-03-30T02:34:46.740056+0000","last_active":"2024-03-30T02:36:27.224734+0000","last_peered":"2024-03-30T02:36:27.224734+0000","last_clean":"2024-03-30T02:36:27.224734+0000","last_became_active":"2024-03-30T02:34:46.739260+0000","last_became_peered":"2024-03-30T02:34:46.739260+0000","last_unstale":"2024-03-30T02:36:27.224734+0000","last_undegraded":"2024-03-30T02:36:27.224734+0000","last_fullsized":"2024-03-30T02:36:27.224734+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T08:08:53.550978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223920+0000","last_change":"2024-03-30T02:34:50.756851+0000","last_active":"2024-03-30T02:36:27.223920+0000","last_peered":"2024-03-30T02:36:27.223920+0000","last_clean":"2024-03-30T02:36:27.223920+0000","last_became_active":"2024-03-30T02:34:50.756416+0000","last_became_peered":"2024-03-30T02:34:50.756416+0000","last_unstale":"2024-03-30T02:36:27.223920+0000","last_undegraded":"2024-03-30T02:36:27.223920+0000","last_fullsized":"2024-03-30T02:36:27.223920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:16:11.469599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225670+0000","last_change":"2024-03-30T02:34:46.215663+0000","last_active":"2024-03-30T02:36:27.225670+0000","last_peered":"2024-03-30T02:36:27.225670+0000","last_clean":"2024-03-30T02:36:27.225670+0000","last_became_active":"2024-03-30T02:34:46.214224+0000","last_became_peered":"2024-03-30T02:34:46.214224+0000","last_unstale":"2024-03-30T02:36:27.225670+0000","last_undegraded":"2024-03-30T02:36:27.225670+0000","last_fullsized":"2024-03-30T02:36:27.225670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:24:58.734434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605318+0000","last_change":"2024-03-30T02:34:46.754150+0000","last_active":"2024-03-30T02:36:27.605318+0000","last_peered":"2024-03-30T02:36:27.605318+0000","last_clean":"2024-03-30T02:36:27.605318+0000","last_became_active":"2024-03-30T02:34:46.753786+0000","last_became_peered":"2024-03-30T02:34:46.753786+0000","last_unstale":"2024-03-30T02:36:27.605318+0000","last_undegraded":"2024-03-30T02:36:27.605318+0000","last_fullsized":"2024-03-30T02:36:27.605318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T11:54:50.165020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225526+0000","last_change":"2024-03-30T02:34:50.797890+0000","last_active":"2024-03-30T02:36:27.225526+0000","last_peered":"2024-03-30T02:36:27.225526+0000","last_clean":"2024-03-30T02:36:27.225526+0000","last_became_active":"2024-03-30T02:34:50.797395+0000","last_became_peered":"2024-03-30T02:34:50.797395+0000","last_unstale":"2024-03-30T02:36:27.225526+0000","last_undegraded":"2024-03-30T02:36:27.225526+0000","last_fullsized":"2024-03-30T02:36:27.225526+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:36:14.958804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.230907+0000","last_change":"2024-03-30T02:34:46.220873+0000","last_active":"2024-03-30T02:36:27.230907+0000","last_peered":"2024-03-30T02:36:27.230907+0000","last_clean":"2024-03-30T02:36:27.230907+0000","last_became_active":"2024-03-30T02:34:46.220544+0000","last_became_peered":"2024-03-30T02:34:46.220544+0000","last_unstale":"2024-03-30T02:36:27.230907+0000","last_undegraded":"2024-03-30T02:36:27.230907+0000","last_fullsized":"2024-03-30T02:36:27.230907+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:17:37.691602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231055+0000","last_change":"2024-03-30T02:34:46.739184+0000","last_active":"2024-03-30T02:36:27.231055+0000","last_peered":"2024-03-30T02:36:27.231055+0000","last_clean":"2024-03-30T02:36:27.231055+0000","last_became_active":"2024-03-30T02:34:46.738843+0000","last_became_peered":"2024-03-30T02:34:46.738843+0000","last_unstale":"2024-03-30T02:36:27.231055+0000","last_undegraded":"2024-03-30T02:36:27.231055+0000","last_fullsized":"2024-03-30T02:36:27.231055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:15:09.894754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.229023+0000","last_change":"2024-03-30T02:34:50.768763+0000","last_active":"2024-03-30T02:36:27.229023+0000","last_peered":"2024-03-30T02:36:27.229023+0000","last_clean":"2024-03-30T02:36:27.229023+0000","last_became_active":"2024-03-30T02:34:50.767363+0000","last_became_peered":"2024-03-30T02:34:50.767363+0000","last_unstale":"2024-03-30T02:36:27.229023+0000","last_undegraded":"2024-03-30T02:36:27.229023+0000","last_fullsized":"2024-03-30T02:36:27.229023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:06.932826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224532+0000","last_change":"2024-03-30T02:34:46.213006+0000","last_active":"2024-03-30T02:36:27.224532+0000","last_peered":"2024-03-30T02:36:27.224532+0000","last_clean":"2024-03-30T02:36:27.224532+0000","last_became_active":"2024-03-30T02:34:46.206271+0000","last_became_peered":"2024-03-30T02:34:46.206271+0000","last_unstale":"2024-03-30T02:36:27.224532+0000","last_undegraded":"2024-03-30T02:36:27.224532+0000","last_fullsized":"2024-03-30T02:36:27.224532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:00:36.447352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228446+0000","last_change":"2024-03-30T02:34:46.739566+0000","last_active":"2024-03-30T02:36:27.228446+0000","last_peered":"2024-03-30T02:36:27.228446+0000","last_clean":"2024-03-30T02:36:27.228446+0000","last_became_active":"2024-03-30T02:34:46.739205+0000","last_became_peered":"2024-03-30T02:34:46.739205+0000","last_unstale":"2024-03-30T02:36:27.228446+0000","last_undegraded":"2024-03-30T02:36:27.228446+0000","last_fullsized":"2024-03-30T02:36:27.228446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:22:01.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"83'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224920+0000","last_change":"2024-03-30T02:34:50.758016+0000","last_active":"2024-03-30T02:36:27.224920+0000","last_peered":"2024-03-30T02:36:27.224920+0000","last_clean":"2024-03-30T02:36:27.224920+0000","last_became_active":"2024-03-30T02:34:50.757616+0000","last_became_peered":"2024-03-30T02:34:50.757616+0000","last_unstale":"2024-03-30T02:36:27.224920+0000","last_undegraded":"2024-03-30T02:36:27.224920+0000","last_fullsized":"2024-03-30T02:36:27.224920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:35:07.902347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'4","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.240419+0000","last_change":"2024-03-30T02:36:28.240079+0000","last_active":"2024-03-30T02:36:28.240419+0000","last_peered":"2024-03-30T02:36:28.240419+0000","last_clean":"2024-03-30T02:36:28.240419+0000","last_became_active":"2024-03-30T02:36:28.228586+0000","last_became_peered":"2024-03-30T02:36:28.228586+0000","last_unstale":"2024-03-30T02:36:28.240419+0000","last_undegraded":"2024-03-30T02:36:28.240419+0000","last_fullsized":"2024-03-30T02:36:28.240419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:22.476600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224967+0000","last_change":"2024-03-30T02:34:46.754114+0000","last_active":"2024-03-30T02:36:27.224967+0000","last_peered":"2024-03-30T02:36:27.224967+0000","last_clean":"2024-03-30T02:36:27.224967+0000","last_became_active":"2024-03-30T02:34:46.752691+0000","last_became_peered":"2024-03-30T02:34:46.752691+0000","last_unstale":"2024-03-30T02:36:27.224967+0000","last_undegraded":"2024-03-30T02:36:27.224967+0000","last_fullsized":"2024-03-30T02:36:27.224967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T07:55:42.908877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.225234+0000","last_change":"2024-03-30T02:34:50.798110+0000","last_active":"2024-03-30T02:36:27.225234+0000","last_peered":"2024-03-30T02:36:27.225234+0000","last_clean":"2024-03-30T02:36:27.225234+0000","last_became_active":"2024-03-30T02:34:50.797551+0000","last_became_peered":"2024-03-30T02:34:50.797551+0000","last_unstale":"2024-03-30T02:36:27.225234+0000","last_undegraded":"2024-03-30T02:36:27.225234+0000","last_fullsized":"2024-03-30T02:36:27.225234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T06:33:49.362018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231216+0000","last_change":"2024-03-30T02:34:46.211647+0000","last_active":"2024-03-30T02:36:27.231216+0000","last_peered":"2024-03-30T02:36:27.231216+0000","last_clean":"2024-03-30T02:36:27.231216+0000","last_became_active":"2024-03-30T02:34:46.211311+0000","last_became_peered":"2024-03-30T02:34:46.211311+0000","last_unstale":"2024-03-30T02:36:27.231216+0000","last_undegraded":"2024-03-30T02:36:27.231216+0000","last_fullsized":"2024-03-30T02:36:27.231216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:05:34.135289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224076+0000","last_change":"2024-03-30T02:34:46.740256+0000","last_active":"2024-03-30T02:36:27.224076+0000","last_peered":"2024-03-30T02:36:27.224076+0000","last_clean":"2024-03-30T02:36:27.224076+0000","last_became_active":"2024-03-30T02:34:46.739913+0000","last_became_peered":"2024-03-30T02:34:46.739913+0000","last_unstale":"2024-03-30T02:36:27.224076+0000","last_undegraded":"2024-03-30T02:36:27.224076+0000","last_fullsized":"2024-03-30T02:36:27.224076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:13:44.152475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.230907+0000","last_change":"2024-03-30T02:34:50.762149+0000","last_active":"2024-03-30T02:36:27.230907+0000","last_peered":"2024-03-30T02:36:27.230907+0000","last_clean":"2024-03-30T02:36:27.230907+0000","last_became_active":"2024-03-30T02:34:50.761717+0000","last_became_peered":"2024-03-30T02:34:50.761717+0000","last_unstale":"2024-03-30T02:36:27.230907+0000","last_undegraded":"2024-03-30T02:36:27.230907+0000","last_fullsized":"2024-03-30T02:36:27.230907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:33:22.373079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:45.260777+0000","last_change":"2024-03-30T02:34:46.224098+0000","last_active":"2024-03-30T02:36:45.260777+0000","last_peered":"2024-03-30T02:36:45.260777+0000","last_clean":"2024-03-30T02:36:45.260777+0000","last_became_active":"2024-03-30T02:34:46.223729+0000","last_became_peered":"2024-03-30T02:34:46.223729+0000","last_unstale":"2024-03-30T02:36:45.260777+0000","last_undegraded":"2024-03-30T02:36:45.260777+0000","last_fullsized":"2024-03-30T02:36:45.260777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:20:25.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1821,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224527+0000","last_change":"2024-03-30T02:34:46.737661+0000","last_active":"2024-03-30T02:36:27.224527+0000","last_peered":"2024-03-30T02:36:27.224527+0000","last_clean":"2024-03-30T02:36:27.224527+0000","last_became_active":"2024-03-30T02:34:46.737303+0000","last_became_peered":"2024-03-30T02:34:46.737303+0000","last_unstale":"2024-03-30T02:36:27.224527+0000","last_undegraded":"2024-03-30T02:36:27.224527+0000","last_fullsized":"2024-03-30T02:36:27.224527+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:04:44.929643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.604791+0000","last_change":"2024-03-30T02:34:50.767119+0000","last_active":"2024-03-30T02:36:27.604791+0000","last_peered":"2024-03-30T02:36:27.604791+0000","last_clean":"2024-03-30T02:36:27.604791+0000","last_became_active":"2024-03-30T02:34:50.766472+0000","last_became_peered":"2024-03-30T02:34:50.766472+0000","last_unstale":"2024-03-30T02:36:27.604791+0000","last_undegraded":"2024-03-30T02:36:27.604791+0000","last_fullsized":"2024-03-30T02:36:27.604791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:27:46.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221554+0000","last_change":"2024-03-30T02:34:46.216713+0000","last_active":"2024-03-30T02:36:27.221554+0000","last_peered":"2024-03-30T02:36:27.221554+0000","last_clean":"2024-03-30T02:36:27.221554+0000","last_became_active":"2024-03-30T02:34:46.216366+0000","last_became_peered":"2024-03-30T02:34:46.216366+0000","last_unstale":"2024-03-30T02:36:27.221554+0000","last_undegraded":"2024-03-30T02:36:27.221554+0000","last_fullsized":"2024-03-30T02:36:27.221554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:59:43.803955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224174+0000","last_change":"2024-03-30T02:34:46.736854+0000","last_active":"2024-03-30T02:36:27.224174+0000","last_peered":"2024-03-30T02:36:27.224174+0000","last_clean":"2024-03-30T02:36:27.224174+0000","last_became_active":"2024-03-30T02:34:46.736511+0000","last_became_peered":"2024-03-30T02:34:46.736511+0000","last_unstale":"2024-03-30T02:36:27.224174+0000","last_undegraded":"2024-03-30T02:36:27.224174+0000","last_fullsized":"2024-03-30T02:36:27.224174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:12:12.701467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221087+0000","last_change":"2024-03-30T02:34:50.757227+0000","last_active":"2024-03-30T02:36:27.221087+0000","last_peered":"2024-03-30T02:36:27.221087+0000","last_clean":"2024-03-30T02:36:27.221087+0000","last_became_active":"2024-03-30T02:34:50.756742+0000","last_became_peered":"2024-03-30T02:34:50.756742+0000","last_unstale":"2024-03-30T02:36:27.221087+0000","last_undegraded":"2024-03-30T02:36:27.221087+0000","last_fullsized":"2024-03-30T02:36:27.221087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:49.730375+0000","last_clean_scrub_stamp":"2024-03-30T02:34:49.730375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:55:48.792202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-30T02:36:28.243069+0000","last_change":"2024-03-30T02:36:28.243069+0000","last_active":"2024-03-30T02:36:28.243069+0000","last_peered":"2024-03-30T02:36:28.243069+0000","last_clean":"2024-03-30T02:36:28.243069+0000","last_became_active":"2024-03-30T02:36:28.238658+0000","last_became_peered":"2024-03-30T02:36:28.238658+0000","last_unstale":"2024-03-30T02:36:28.243069+0000","last_undegraded":"2024-03-30T02:36:28.243069+0000","last_fullsized":"2024-03-30T02:36:28.243069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:45.110696+0000","last_clean_scrub_stamp":"2024-03-30T02:34:45.110696+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:47:59.741479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224872+0000","last_change":"2024-03-30T02:34:46.741735+0000","last_active":"2024-03-30T02:36:27.224872+0000","last_peered":"2024-03-30T02:36:27.224872+0000","last_clean":"2024-03-30T02:36:27.224872+0000","last_became_active":"2024-03-30T02:34:46.740663+0000","last_became_peered":"2024-03-30T02:34:46.740663+0000","last_unstale":"2024-03-30T02:36:27.224872+0000","last_undegraded":"2024-03-30T02:36:27.224872+0000","last_fullsized":"2024-03-30T02:36:27.224872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T10:14:09.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228383+0000","last_change":"2024-03-30T02:34:46.739082+0000","last_active":"2024-03-30T02:36:27.228383+0000","last_peered":"2024-03-30T02:36:27.228383+0000","last_clean":"2024-03-30T02:36:27.228383+0000","last_became_active":"2024-03-30T02:34:46.738732+0000","last_became_peered":"2024-03-30T02:34:46.738732+0000","last_unstale":"2024-03-30T02:36:27.228383+0000","last_undegraded":"2024-03-30T02:36:27.228383+0000","last_fullsized":"2024-03-30T02:36:27.228383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T02:44:53.961520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224158+0000","last_change":"2024-03-30T02:35:28.211005+0000","last_active":"2024-03-30T02:36:27.224158+0000","last_peered":"2024-03-30T02:36:27.224158+0000","last_clean":"2024-03-30T02:36:27.224158+0000","last_became_active":"2024-03-30T02:35:28.210610+0000","last_became_peered":"2024-03-30T02:35:28.210610+0000","last_unstale":"2024-03-30T02:36:27.224158+0000","last_undegraded":"2024-03-30T02:36:27.224158+0000","last_fullsized":"2024-03-30T02:36:27.224158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:23:53.054107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.223977+0000","last_change":"2024-03-30T02:34:46.742590+0000","last_active":"2024-03-30T02:36:27.223977+0000","last_peered":"2024-03-30T02:36:27.223977+0000","last_clean":"2024-03-30T02:36:27.223977+0000","last_became_active":"2024-03-30T02:34:46.740904+0000","last_became_peered":"2024-03-30T02:34:46.740904+0000","last_unstale":"2024-03-30T02:36:27.223977+0000","last_undegraded":"2024-03-30T02:36:27.223977+0000","last_fullsized":"2024-03-30T02:36:27.223977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:01:22.653881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.605025+0000","last_change":"2024-03-30T02:34:46.753119+0000","last_active":"2024-03-30T02:36:27.605025+0000","last_peered":"2024-03-30T02:36:27.605025+0000","last_clean":"2024-03-30T02:36:27.605025+0000","last_became_active":"2024-03-30T02:34:46.745680+0000","last_became_peered":"2024-03-30T02:34:46.745680+0000","last_unstale":"2024-03-30T02:36:27.605025+0000","last_undegraded":"2024-03-30T02:36:27.605025+0000","last_fullsized":"2024-03-30T02:36:27.605025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:30:02.352961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221594+0000","last_change":"2024-03-30T02:35:28.313317+0000","last_active":"2024-03-30T02:36:27.221594+0000","last_peered":"2024-03-30T02:36:27.221594+0000","last_clean":"2024-03-30T02:36:27.221594+0000","last_became_active":"2024-03-30T02:35:28.312929+0000","last_became_peered":"2024-03-30T02:35:28.312929+0000","last_unstale":"2024-03-30T02:36:27.221594+0000","last_undegraded":"2024-03-30T02:36:27.221594+0000","last_fullsized":"2024-03-30T02:36:27.221594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T05:47:06.390076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228760+0000","last_change":"2024-03-30T02:34:46.737882+0000","last_active":"2024-03-30T02:36:27.228760+0000","last_peered":"2024-03-30T02:36:27.228760+0000","last_clean":"2024-03-30T02:36:27.228760+0000","last_became_active":"2024-03-30T02:34:46.737546+0000","last_became_peered":"2024-03-30T02:34:46.737546+0000","last_unstale":"2024-03-30T02:36:27.228760+0000","last_undegraded":"2024-03-30T02:36:27.228760+0000","last_fullsized":"2024-03-30T02:36:27.228760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:48:41.709748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224639+0000","last_change":"2024-03-30T02:34:46.742703+0000","last_active":"2024-03-30T02:36:27.224639+0000","last_peered":"2024-03-30T02:36:27.224639+0000","last_clean":"2024-03-30T02:36:27.224639+0000","last_became_active":"2024-03-30T02:34:46.740689+0000","last_became_peered":"2024-03-30T02:34:46.740689+0000","last_unstale":"2024-03-30T02:36:27.224639+0000","last_undegraded":"2024-03-30T02:36:27.224639+0000","last_fullsized":"2024-03-30T02:36:27.224639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:05:45.300151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224909+0000","last_change":"2024-03-30T02:34:46.750811+0000","last_active":"2024-03-30T02:36:27.224909+0000","last_peered":"2024-03-30T02:36:27.224909+0000","last_clean":"2024-03-30T02:36:27.224909+0000","last_became_active":"2024-03-30T02:34:46.750395+0000","last_became_peered":"2024-03-30T02:34:46.750395+0000","last_unstale":"2024-03-30T02:36:27.224909+0000","last_undegraded":"2024-03-30T02:36:27.224909+0000","last_fullsized":"2024-03-30T02:36:27.224909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T04:23:01.282677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221281+0000","last_change":"2024-03-30T02:34:46.734814+0000","last_active":"2024-03-30T02:36:27.221281+0000","last_peered":"2024-03-30T02:36:27.221281+0000","last_clean":"2024-03-30T02:36:27.221281+0000","last_became_active":"2024-03-30T02:34:46.733978+0000","last_became_peered":"2024-03-30T02:34:46.733978+0000","last_unstale":"2024-03-30T02:36:27.221281+0000","last_undegraded":"2024-03-30T02:36:27.221281+0000","last_fullsized":"2024-03-30T02:36:27.221281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:31:39.467478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.224665+0000","last_change":"2024-03-30T02:34:46.748163+0000","last_active":"2024-03-30T02:36:27.224665+0000","last_peered":"2024-03-30T02:36:27.224665+0000","last_clean":"2024-03-30T02:36:27.224665+0000","last_became_active":"2024-03-30T02:34:46.738845+0000","last_became_peered":"2024-03-30T02:34:46.738845+0000","last_unstale":"2024-03-30T02:36:27.224665+0000","last_undegraded":"2024-03-30T02:36:27.224665+0000","last_fullsized":"2024-03-30T02:36:27.224665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T09:38:09.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.231257+0000","last_change":"2024-03-30T02:34:46.743231+0000","last_active":"2024-03-30T02:36:27.231257+0000","last_peered":"2024-03-30T02:36:27.231257+0000","last_clean":"2024-03-30T02:36:27.231257+0000","last_became_active":"2024-03-30T02:34:46.742619+0000","last_became_peered":"2024-03-30T02:34:46.742619+0000","last_unstale":"2024-03-30T02:36:27.231257+0000","last_undegraded":"2024-03-30T02:36:27.231257+0000","last_fullsized":"2024-03-30T02:36:27.231257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:17:51.789139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.221198+0000","last_change":"2024-03-30T02:34:46.745872+0000","last_active":"2024-03-30T02:36:27.221198+0000","last_peered":"2024-03-30T02:36:27.221198+0000","last_clean":"2024-03-30T02:36:27.221198+0000","last_became_active":"2024-03-30T02:34:46.745476+0000","last_became_peered":"2024-03-30T02:34:46.745476+0000","last_unstale":"2024-03-30T02:36:27.221198+0000","last_undegraded":"2024-03-30T02:36:27.221198+0000","last_fullsized":"2024-03-30T02:36:27.221198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T12:06:30.774316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-30T02:36:27.228267+0000","last_change":"2024-03-30T02:34:46.746178+0000","last_active":"2024-03-30T02:36:27.228267+0000","last_peered":"2024-03-30T02:36:27.228267+0000","last_clean":"2024-03-30T02:36:27.228267+0000","last_became_active":"2024-03-30T02:34:46.745865+0000","last_became_peered":"2024-03-30T02:34:46.745865+0000","last_unstale":"2024-03-30T02:36:27.228267+0000","last_undegraded":"2024-03-30T02:36:27.228267+0000","last_fullsized":"2024-03-30T02:36:27.228267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T02:34:46.181392+0000","last_clean_scrub_stamp":"2024-03-30T02:34:46.181392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T03:56:22.613637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":248704,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78918,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":921,"num_read_kb":549,"num_write":261,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":234234,"data_compressed":5484,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579938,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":488244,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.746,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72299999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58299999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78700000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.34499999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.751,"5min":0.755,"15min":0.755},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72299999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70999999999999996}]},{"osd":3,"last update":"Sat Mar 30 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.749},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.72899999999999998}]},{"osd":4,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.82499999999999996,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.85299999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.85099999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.40899999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.38800000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":1.089,"15min":1.089},"last":0.877}]},{"osd":6,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.83999999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":1.323,"15min":1.323},"last":0.78900000000000003}]},{"osd":7,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88700000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.51400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91200000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.93799999999999994}]},{"osd":8,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.0229999999999999}]},{"osd":9,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.754,"15min":0.754},"min":{"1min":0.61899999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.89700000000000002,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.83199999999999996,"15min":0.83599999999999997},"min":{"1min":0.60299999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.96799999999999997,"5min":1.25,"15min":1.25},"last":0.96399999999999997}]},{"osd":10,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.78700000000000003,"15min":0.78100000000000003},"min":{"1min":0.629,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.89200000000000002,"5min":1.206,"15min":1.206},"last":0.91400000000000003}]}]},{"osd":10,"up_from":67,"seq":287762808871,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1118208,"data_stored":1145816,"data_compressed":61402,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.313},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.371},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68600000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.434}]},{"osd":2,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.498,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.873},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.50700000000000001}]},{"osd":3,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60399999999999998,"15min":0.60899999999999999},"min":{"1min":0.44500000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.47899999999999998}]},{"osd":4,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.35599999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81000000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.76600000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63700000000000001,"15min":0.63},"min":{"1min":0.54200000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58599999999999997}]},{"osd":6,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70599999999999996,"15min":0.70199999999999996},"min":{"1min":0.54600000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":8,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.59599999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.94099999999999995}]},{"osd":9,"last update":"Sat Mar 30 02:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75700000000000001,"15min":0.755},"min":{"1min":0.60799999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75800000000000001,"15min":0.76100000000000001},"min":{"1min":0.55800000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.90900000000000003}]},{"osd":11,"last update":"Sat Mar 30 02:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.754,"15min":0.748},"min":{"1min":0.64300000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64700000000000002}]}]},{"osd":9,"up_from":62,"seq":266287972393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":491619,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81399999999999995,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.33300000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.79600000000000004,"15min":0.77000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":6.774,"5min":6.774,"15min":6.774},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.52100000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.56399999999999995}]},{"osd":3,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60399999999999998,"15min":0.61199999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.77400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.40500000000000003}]},{"osd":4,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.51000000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.278,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78200000000000003},"min":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.038,"5min":1.099,"15min":1.099},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.86199999999999999,"15min":0.878},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3149999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.747},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.72199999999999998}]},{"osd":7,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.91200000000000003,"15min":0.93200000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3879999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.96199999999999997},"min":{"1min":0.65900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.73299999999999998}]},{"osd":8,"last update":"Sat Mar 30 02:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88700000000000001,"15min":0.89700000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.79300000000000004}]},{"osd":10,"last update":"Sat Mar 30 02:35:59 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.94499999999999995,"15min":0.95299999999999996},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84599999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83299999999999996}]},{"osd":11,"last update":"Sat Mar 30 02:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.81499999999999995}]}]},{"osd":8,"up_from":55,"seq":236223201326,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":487570,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.52500000000000002},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.72399999999999998,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.45100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83099999999999996,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54100000000000004}]},{"osd":1,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.59399999999999997,"15min":0.56599999999999995},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.83499999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.53300000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.75,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51700000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.46999999999999997,"15min":0.44600000000000001},"min":{"1min":0.38400000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.76200000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.45300000000000001,"15min":0.43099999999999999},"min":{"1min":0.318,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.62,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.39300000000000002}]},{"osd":3,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.48299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.876,"15min":0.876},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.51100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56100000000000005}]},{"osd":4,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.621,"15min":0.59899999999999998},"min":{"1min":0.54800000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.55200000000000005,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89100000000000001,"5min":0.995,"15min":0.995},"last":0.70399999999999996}]},{"osd":5,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53000000000000003,"15min":0.499},"min":{"1min":0.436,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58099999999999996,"15min":0.55400000000000005},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.439}]},{"osd":6,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.73399999999999999,"15min":0.68200000000000005},"min":{"1min":0.53200000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65500000000000003,"15min":0.63300000000000001},"min":{"1min":0.499,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61599999999999999}]},{"osd":7,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.65700000000000003,"15min":0.63500000000000001},"min":{"1min":0.56499999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.71499999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96399999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.68999999999999995}]},{"osd":9,"last update":"Sat Mar 30 02:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.76300000000000001,"15min":0.73099999999999998},"min":{"1min":0.70099999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0249999999999999,"5min":1.137,"15min":1.137},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.77300000000000002,"15min":0.745},"min":{"1min":0.69199999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96899999999999997,"5min":1.034,"15min":1.034},"last":0.77200000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.752,"15min":0.75},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.88700000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.69799999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.92100000000000004,"5min":1.091,"15min":1.091},"last":0.79200000000000004}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73899999999999999}]}]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1084658,"data_compressed":59574,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:33:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.58499999999999996}]},{"osd":1,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42399999999999999,"15min":0.41899999999999998},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.55700000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.442,"15min":0.436},"min":{"1min":0.35699999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.56399999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.47499999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.433,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.504},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72999999999999998,"5min":1.034,"15min":1.034},"last":0.52500000000000002}]},{"osd":3,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59599999999999997},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78800000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60099999999999998,"15min":0.59599999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.63400000000000001}]},{"osd":4,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.5,"15min":0.49399999999999999},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.75,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.32200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59999999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41999999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61199999999999999,"15min":0.59999999999999998},"min":{"1min":0.47099999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85199999999999998,"5min":0.878,"15min":0.878},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60399999999999998}]},{"osd":6,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.83199999999999996,"5min":0.877,"15min":0.877},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83699999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.62}]},{"osd":8,"last update":"Sat Mar 30 02:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.68000000000000005,"15min":0.67000000000000004},"min":{"1min":0.55200000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90500000000000003,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70699999999999996}]},{"osd":9,"last update":"Sat Mar 30 02:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.59699999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71299999999999997,"15min":0.71199999999999997},"min":{"1min":0.61799999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.753}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91000000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88500000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73099999999999998}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.73799999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":962560,"data_stored":482979,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.97699999999999998,"5min":0.998,"15min":0.998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79500000000000004,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39400000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67800000000000005},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.75800000000000001}]},{"osd":2,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.069,"15min":1.069},"last":0.622}]},{"osd":3,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96199999999999997,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72499999999999998,"15min":0.74099999999999999},"min":{"1min":0.45700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.1040000000000001,"5min":1.236,"15min":1.236},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":1.069,"15min":1.069},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0329999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.52400000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.0289999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.85899999999999999,"15min":0.871},"min":{"1min":0.64700000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96599999999999997,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Sat Mar 30 02:36:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.83899999999999997,"15min":0.84699999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.111,"5min":1.393,"15min":1.393},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83899999999999997,"15min":0.84699999999999998},"min":{"1min":0.61099999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":0.82899999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:36:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.89600000000000002,"15min":0.91600000000000004},"min":{"1min":0.62,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0700000000000001,"5min":1.355,"15min":1.355},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.875,"15min":0.88800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.72099999999999997}]},{"osd":9,"last update":"Sat Mar 30 02:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.93000000000000005,"15min":0.96199999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2030000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.88600000000000001,"15min":0.92000000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.70099999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.876,"15min":0.872},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93200000000000005,"15min":0.93100000000000005},"min":{"1min":0.745,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.24,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.81599999999999995}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":552410,"data_compressed":25402,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.75800000000000001},"min":{"1min":0.42599999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71099999999999997,"5min":5.5439999999999996,"15min":5.5439999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59899999999999998}]},{"osd":2,"last update":"Sat Mar 30 02:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57199999999999995,"15min":0.52400000000000002},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77500000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.52400000000000002},"min":{"1min":0.38500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.92100000000000004,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.64400000000000002}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.438,"5min":0.378,"15min":0.378},"max":{"1min":0.70099999999999996,"5min":1.028,"15min":1.028},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59299999999999997,"15min":0.56000000000000005},"min":{"1min":0.436,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.442,"15min":0.40799999999999997},"min":{"1min":0.33200000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.433},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.439,"15min":0.41099999999999998},"min":{"1min":0.313,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45000000000000001}]},{"osd":5,"last update":"Sat Mar 30 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.64800000000000002,"15min":0.59399999999999997},"min":{"1min":0.54100000000000004,"5min":0.372,"15min":0.372},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.026},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59799999999999998,"15min":0.56000000000000005},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.83999999999999997,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.62}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65700000000000003,"15min":0.63800000000000001},"min":{"1min":0.51600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91100000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77200000000000002}]},{"osd":7,"last update":"Sat Mar 30 02:36:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.42699999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.36799999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.70699999999999996}]},{"osd":8,"last update":"Sat Mar 30 02:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93600000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.73899999999999999,"15min":0.72899999999999998},"min":{"1min":0.58699999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.0249999999999999,"5min":1.044,"15min":1.044},"last":0.81200000000000006}]},{"osd":9,"last update":"Sat Mar 30 02:36:41 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.57499999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.79300000000000004,"15min":0.80300000000000005},"min":{"1min":0.59799999999999998,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.94899999999999995,"5min":1.056,"15min":1.056},"last":0.73899999999999999}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.58499999999999996,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94899999999999995,"5min":1.04,"15min":1.04},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.48799999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.874,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.75800000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":485134,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Mar 30 02:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.54800000000000004},"min":{"1min":0.41999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.61599999999999999,"15min":0.55500000000000005},"min":{"1min":0.441,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.53800000000000003}]},{"osd":2,"last update":"Sat Mar 30 02:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63200000000000001,"15min":0.59599999999999997},"min":{"1min":0.46999999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.64300000000000002,"15min":0.60099999999999998},"min":{"1min":0.48599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.51300000000000001}]},{"osd":4,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.65600000000000003,"15min":0.63600000000000001},"min":{"1min":0.49099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63900000000000001,"15min":0.61799999999999999},"min":{"1min":0.51900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":1.006},"last":0.61199999999999999}]},{"osd":5,"last update":"Sat Mar 30 02:36:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.46999999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.70999999999999996,"15min":0.66900000000000004},"min":{"1min":0.504,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.81299999999999994}]},{"osd":6,"last update":"Sat Mar 30 02:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.69199999999999995,"15min":0.66200000000000003},"min":{"1min":0.46400000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.754,"15min":0.71199999999999997},"min":{"1min":0.53800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.68999999999999995}]},{"osd":7,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.99399999999999999}]},{"osd":8,"last update":"Sat Mar 30 02:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.81599999999999995,"15min":0.80000000000000004},"min":{"1min":0.70999999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.84199999999999997,"15min":0.82599999999999996},"min":{"1min":0.60699999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.88600000000000001}]},{"osd":9,"last update":"Sat Mar 30 02:36:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.88,"15min":0.878},"min":{"1min":0.66100000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.89100000000000001,"15min":0.88},"min":{"1min":0.76400000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.92300000000000004}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.874,"15min":0.86499999999999999},"min":{"1min":0.60899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.86599999999999999,"15min":0.85499999999999998},"min":{"1min":0.66800000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.83199999999999996}]},{"osd":11,"last update":"Sat Mar 30 02:36:16 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.874,"15min":0.85199999999999998},"min":{"1min":0.76900000000000002,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.016,"5min":0.89300000000000002,"15min":0.873},"min":{"1min":0.76500000000000001,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.014}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":556335,"data_compressed":25402,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55700000000000005,"15min":0.52200000000000002},"min":{"1min":0.41299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82899999999999996,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52900000000000003,"15min":0.51300000000000001},"min":{"1min":0.42499999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46500000000000002}]},{"osd":1,"last update":"Sat Mar 30 02:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85099999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59299999999999997,"15min":0.54300000000000004},"min":{"1min":0.46600000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.89200000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.73799999999999999}]},{"osd":3,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.41399999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84599999999999997,"5min":0.878,"15min":0.878},"last":0.622},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57599999999999996},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39600000000000002}]},{"osd":4,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.72199999999999998}]},{"osd":5,"last update":"Sat Mar 30 02:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.498,"15min":0.46600000000000003},"min":{"1min":0.30499999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.74099999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53600000000000003,"15min":0.52600000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.497}]},{"osd":6,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.58099999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85199999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.48799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.53500000000000003}]},{"osd":7,"last update":"Sat Mar 30 02:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.751},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78700000000000003}]},{"osd":8,"last update":"Sat Mar 30 02:36:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.627,"15min":0.60099999999999998},"min":{"1min":0.41299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.56799999999999995}]},{"osd":9,"last update":"Sat Mar 30 02:36:39 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.72699999999999998,"15min":0.71899999999999997},"min":{"1min":0.56100000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92000000000000004,"5min":1.292,"15min":1.292},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.86499999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.65100000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:35:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.755,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.76300000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.69799999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":974848,"data_stored":482735,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.47099999999999997,"15min":0.40699999999999997},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":2.1619999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.434},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.51400000000000001}]},{"osd":1,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.60799999999999998,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.66100000000000003,"15min":0.58499999999999996},"min":{"1min":0.47299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.83799999999999997}]},{"osd":2,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.66700000000000004,"15min":0.623},"min":{"1min":0.56899999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.64800000000000002,"15min":0.60099999999999998},"min":{"1min":0.42399999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":0.56299999999999994}]},{"osd":4,"last update":"Sat Mar 30 02:36:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.73099999999999998,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.443,"15min":0.443},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.68300000000000005,"15min":0.64500000000000002},"min":{"1min":0.55900000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.70999999999999996}]},{"osd":5,"last update":"Sat Mar 30 02:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.70999999999999996,"15min":0.65800000000000003},"min":{"1min":0.57899999999999996,"5min":0.432,"15min":0.432},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.79100000000000004,"15min":0.71799999999999997},"min":{"1min":0.64400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Sat Mar 30 02:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80200000000000005,"15min":0.78000000000000003},"min":{"1min":0.57599999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.234,"5min":2.387,"15min":2.387},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.752,"15min":0.71799999999999997},"min":{"1min":0.53600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.2149999999999999,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Sat Mar 30 02:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.73299999999999998,"15min":0.70599999999999996},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.1859999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.90700000000000003,"15min":0.90300000000000002},"min":{"1min":0.58699999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":0.93000000000000005}]},{"osd":8,"last update":"Sat Mar 30 02:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":1.004}]},{"osd":9,"last update":"Sat Mar 30 02:36:39 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94199999999999995},"min":{"1min":0.66800000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2769999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.877,"15min":0.86699999999999999},"min":{"1min":0.57899999999999996,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.258,"5min":2.258,"15min":2.258},"last":0.874}]},{"osd":10,"last update":"Sat Mar 30 02:35:58 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.871,"15min":0.83799999999999997},"min":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.97599999999999998,"15min":0.96199999999999997},"min":{"1min":0.71399999999999997,"5min":0.629,"15min":0.629},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.95299999999999996}]},{"osd":11,"last update":"Sat Mar 30 02:35:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.84999999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":970752,"data_stored":487587,"data_compressed":23574,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:34:03 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51000000000000001,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.5,"15min":0.48999999999999999},"min":{"1min":0.38,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.747,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35299999999999998}]},{"osd":1,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.35299999999999998},"min":{"1min":0.27700000000000002,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.63200000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.38,"15min":0.35599999999999998},"min":{"1min":0.32300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.39500000000000002}]},{"osd":2,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56000000000000005,"15min":0.52800000000000002},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.748,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.53900000000000003},"min":{"1min":0.45800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Sat Mar 30 02:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54600000000000004,"15min":0.52600000000000002},"min":{"1min":0.45500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.55300000000000005},"min":{"1min":0.35899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.438}]},{"osd":5,"last update":"Sat Mar 30 02:36:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59899999999999998,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76000000000000001,"5min":0.878,"15min":0.878},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.56499999999999995},"min":{"1min":0.48499999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.72199999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.625}]},{"osd":6,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.59999999999999998},"min":{"1min":0.45000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54200000000000004}]},{"osd":7,"last update":"Sat Mar 30 02:36:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53800000000000003,"15min":0.51900000000000002},"min":{"1min":0.39000000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82399999999999995,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.503},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.40899999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.496}]},{"osd":8,"last update":"Sat Mar 30 02:36:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.64700000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92400000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69799999999999995,"15min":0.67500000000000004},"min":{"1min":0.53100000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91500000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.72699999999999998}]},{"osd":9,"last update":"Sat Mar 30 02:36:42 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.55300000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.81100000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.96099999999999997,"5min":1.079,"15min":1.079},"last":0.64900000000000002}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.50900000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92900000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.74199999999999999}]},{"osd":11,"last update":"Sat Mar 30 02:35:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78200000000000003}]}]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":1044480,"data_stored":1080075,"data_compressed":59574,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.53500000000000003},"min":{"1min":0.42799999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53800000000000003,"15min":0.52800000000000002},"min":{"1min":0.41299999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54500000000000004}]},{"osd":1,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.54100000000000004,"15min":0.51100000000000001},"min":{"1min":0.44700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.56299999999999994,"15min":0.52800000000000002},"min":{"1min":0.40899999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80600000000000005,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.57199999999999995}]},{"osd":2,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.37},"min":{"1min":0.35199999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.65100000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.36799999999999999},"min":{"1min":0.35299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.74299999999999999}]},{"osd":3,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55200000000000005},"min":{"1min":0.39500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.48899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84899999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.71799999999999997}]},{"osd":4,"last update":"Sat Mar 30 02:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.58799999999999997},"min":{"1min":0.47499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.89600000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.627,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84799999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70199999999999996}]},{"osd":6,"last update":"Sat Mar 30 02:35:47 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61699999999999999,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.873}]},{"osd":7,"last update":"Sat Mar 30 02:36:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.53200000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81000000000000005,"5min":1.006,"15min":1.006},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.625,"15min":0.60099999999999998},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.77400000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.92500000000000004}]},{"osd":8,"last update":"Sat Mar 30 02:36:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.437,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58199999999999996,"15min":0.56899999999999995},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.83699999999999997}]},{"osd":9,"last update":"Sat Mar 30 02:36:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.55400000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.622,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.95599999999999996}]},{"osd":10,"last update":"Sat Mar 30 02:35:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.86599999999999999,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.747,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.871,"5min":1.024,"15min":1.024},"last":0.89000000000000001}]},{"osd":11,"last update":"Sat Mar 30 02:36:29 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":1.0229999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34352,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36000,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70105,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":74030,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":73143,"data_compressed":1828,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T02:36:49.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:49.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:49 smithi082 ceph-mon[79176]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:49.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:49 smithi082 ceph-mon[79176]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:49.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:49 smithi082 ceph-mon[79176]: pgmap v298: 129 pgs: 129 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-03-30T02:36:49.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.516+0000 7f04a6400700 1 -- 172.21.15.67:0/2288198609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0100c80 msgr2=0x7f04a01010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.516+0000 7f04a6400700 1 --2- 172.21.15.67:0/2288198609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0100c80 0x7f04a01010a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f04900099c0 tx=0x7f0490009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.516+0000 7f04a6400700 1 -- 172.21.15.67:0/2288198609 shutdown_connections 2024-03-30T02:36:49.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.516+0000 7f04a6400700 1 --2- 172.21.15.67:0/2288198609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0100c80 0x7f04a01010a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.516+0000 7f04a6400700 1 --2- 172.21.15.67:0/2288198609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a0102810 0x7f04a01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.516+0000 7f04a6400700 1 --2- 172.21.15.67:0/2288198609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a0101e70 0x7f04a01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.516+0000 7f04a6400700 1 -- 172.21.15.67:0/2288198609 >> 172.21.15.67:0/2288198609 conn(0x7f04a00fc1f0 msgr2=0x7f04a00fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.516+0000 7f04a6400700 1 -- 172.21.15.67:0/2288198609 shutdown_connections 2024-03-30T02:36:49.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.516+0000 7f04a6400700 1 -- 172.21.15.67:0/2288198609 wait complete. 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.517+0000 7f04a6400700 1 Processor -- start 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.517+0000 7f04a6400700 1 -- start start 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.517+0000 7f04a6400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a0100c80 0x7f04a0198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f04a6400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a0101e70 0x7f04a0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f04a6400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0102810 0x7f04a01a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f04a6400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a0980 con 0x7f04a0102810 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f04a6400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a0ac0 con 0x7f04a0100c80 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f04a6400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a0c50 con 0x7f04a0101e70 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0102810 0x7f04a01a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0102810 0x7f04a01a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56692/0 (socket says 172.21.15.67:56692) 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049fe00700 1 -- 172.21.15.67:0/1430309847 learned_addr learned my addr 172.21.15.67:0/1430309847 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049f400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a0100c80 0x7f04a0198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049ea00700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a0101e70 0x7f04a0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049fe00700 1 -- 172.21.15.67:0/1430309847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a0101e70 msgr2=0x7f04a0199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049fe00700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a0101e70 0x7f04a0199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049fe00700 1 -- 172.21.15.67:0/1430309847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a0100c80 msgr2=0x7f04a0198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049fe00700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a0100c80 0x7f04a0198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049fe00700 1 -- 172.21.15.67:0/1430309847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04900095f0 con 0x7f04a0102810 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.518+0000 7f049f400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a0100c80 0x7f04a0198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.519+0000 7f049fe00700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0102810 0x7f04a01a03b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0498007ee0 tx=0x7f049800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.519+0000 7f0497e00700 1 -- 172.21.15.67:0/1430309847 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498017070 con 0x7f04a0102810 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.519+0000 7f0497e00700 1 -- 172.21.15.67:0/1430309847 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f049800f040 con 0x7f04a0102810 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.519+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a01a0f30 con 0x7f04a0102810 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.519+0000 7f0497e00700 1 -- 172.21.15.67:0/1430309847 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498005580 con 0x7f04a0102810 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.519+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a01a13b0 con 0x7f04a0102810 2024-03-30T02:36:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.520+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a0067a10 con 0x7f04a0102810 2024-03-30T02:36:49.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.522+0000 7f0497e00700 1 -- 172.21.15.67:0/1430309847 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f049800db20 con 0x7f04a0102810 2024-03-30T02:36:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.523+0000 7f0497e00700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f04840a11b0 0x7f04840a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.523+0000 7f0497e00700 1 -- 172.21.15.67:0/1430309847 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f04980cc520 con 0x7f04a0102810 2024-03-30T02:36:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.523+0000 7f049f400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f04840a11b0 0x7f04840a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:49.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.524+0000 7f049f400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f04840a11b0 0x7f04840a3670 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0490003730 tx=0x7f049000b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:49.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.527+0000 7f0497e00700 1 -- 172.21.15.67:0/1430309847 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f04980957b0 con 0x7f04a0102810 2024-03-30T02:36:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:49 smithi067 ceph-mon[37808]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:49 smithi067 ceph-mon[37808]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:49 smithi067 ceph-mon[37808]: pgmap v298: 129 pgs: 129 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-03-30T02:36:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:49 smithi170 ceph-mon[78798]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:49 smithi170 ceph-mon[78798]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:49 smithi170 ceph-mon[78798]: pgmap v298: 129 pgs: 129 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-03-30T02:36:49.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.787+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04a010c4e0 con 0x7f04a0102810 2024-03-30T02:36:49.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.789+0000 7f0497e00700 1 -- 172.21.15.67:0/1430309847 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24479 (secure 0 0 0) 0x7f0498098dd0 con 0x7f04a0102810 2024-03-30T02:36:49.790 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:49.790 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:36:28.218474+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:49.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f04840a11b0 msgr2=0x7f04840a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f04840a11b0 0x7f04840a3670 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0490003730 tx=0x7f049000b540 comp rx=0 tx=0).stop 2024-03-30T02:36:49.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0102810 msgr2=0x7f04a01a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:49.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0102810 0x7f04a01a03b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0498007ee0 tx=0x7f049800d3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 shutdown_connections 2024-03-30T02:36:49.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f04840a11b0 0x7f04840a3670 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f04a0102810 0x7f04a01a03b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04a0100c80 0x7f04a0198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 --2- 172.21.15.67:0/1430309847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04a0101e70 0x7f04a0199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:49.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 >> 172.21.15.67:0/1430309847 conn(0x7f04a00fc1f0 msgr2=0x7f04a0107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:49.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 shutdown_connections 2024-03-30T02:36:49.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:49.791+0000 7f04a6400700 1 -- 172.21.15.67:0/1430309847 wait complete. 2024-03-30T02:36:49.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:36:49.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:36:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.012+0000 7f7563800700 1 -- 172.21.15.67:0/1852667978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c101f70 msgr2=0x7f755c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.012+0000 7f7563800700 1 --2- 172.21.15.67:0/1852667978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c101f70 0x7f755c1023d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75480099c0 tx=0x7f7548009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.012+0000 7f7563800700 1 -- 172.21.15.67:0/1852667978 shutdown_connections 2024-03-30T02:36:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.012+0000 7f7563800700 1 --2- 172.21.15.67:0/1852667978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f755c102910 0x7f755c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.012+0000 7f7563800700 1 --2- 172.21.15.67:0/1852667978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f755c100d80 0x7f755c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.012+0000 7f7563800700 1 --2- 172.21.15.67:0/1852667978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c101f70 0x7f755c1023d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.012+0000 7f7563800700 1 -- 172.21.15.67:0/1852667978 >> 172.21.15.67:0/1852667978 conn(0x7f755c0fc2f0 msgr2=0x7f755c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.013+0000 7f7563800700 1 -- 172.21.15.67:0/1852667978 shutdown_connections 2024-03-30T02:36:50.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.013+0000 7f7563800700 1 -- 172.21.15.67:0/1852667978 wait complete. 2024-03-30T02:36:50.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7563800700 1 Processor -- start 2024-03-30T02:36:50.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7563800700 1 -- start start 2024-03-30T02:36:50.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7563800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c100d80 0x7f755c198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7563800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f755c101f70 0x7f755c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7563800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f755c102910 0x7f755c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7563800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1a0a30 con 0x7f755c100d80 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7563800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1a0b70 con 0x7f755c102910 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7563800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1a0d00 con 0x7f755c101f70 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7560e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c100d80 0x7f755c198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f755be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f755c101f70 0x7f755c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7560e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c100d80 0x7f755c198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56708/0 (socket says 172.21.15.67:56708) 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.014+0000 7f7560e00700 1 -- 172.21.15.67:0/4135838953 learned_addr learned my addr 172.21.15.67:0/4135838953 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7561800700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f755c102910 0x7f755c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7560e00700 1 -- 172.21.15.67:0/4135838953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f755c101f70 msgr2=0x7f755c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7560e00700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f755c101f70 0x7f755c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7560e00700 1 -- 172.21.15.67:0/4135838953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f755c102910 msgr2=0x7f755c1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7560e00700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f755c102910 0x7f755c1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7560e00700 1 -- 172.21.15.67:0/4135838953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75480095f0 con 0x7f755c100d80 2024-03-30T02:36:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7560e00700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c100d80 0x7f755c198f00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f755000e970 tx=0x7f755000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7559600700 1 -- 172.21.15.67:0/4135838953 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550009790 con 0x7f755c100d80 2024-03-30T02:36:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7559600700 1 -- 172.21.15.67:0/4135838953 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f755000c7e0 con 0x7f755c100d80 2024-03-30T02:36:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c1a0fe0 con 0x7f755c100d80 2024-03-30T02:36:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.015+0000 7f7559600700 1 -- 172.21.15.67:0/4135838953 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550003c70 con 0x7f755c100d80 2024-03-30T02:36:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.016+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c1a1400 con 0x7f755c100d80 2024-03-30T02:36:50.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.017+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c067af0 con 0x7f755c100d80 2024-03-30T02:36:50.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.018+0000 7f7559600700 1 -- 172.21.15.67:0/4135838953 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f755000cc50 con 0x7f755c100d80 2024-03-30T02:36:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.020+0000 7f7559600700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f75440a11b0 0x7f75440a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.020+0000 7f7559600700 1 -- 172.21.15.67:0/4135838953 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f7550014070 con 0x7f755c100d80 2024-03-30T02:36:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.020+0000 7f755be00700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f75440a11b0 0x7f75440a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.020+0000 7f755be00700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f75440a11b0 0x7f75440a3670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7548003730 tx=0x7f754800b540 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:50.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.024+0000 7f7559600700 1 -- 172.21.15.67:0/4135838953 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f75500d0030 con 0x7f755c100d80 2024-03-30T02:36:50.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.244+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f755c10c860 con 0x7f755c100d80 2024-03-30T02:36:50.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.244+0000 7f7559600700 1 -- 172.21.15.67:0/4135838953 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7550096260 con 0x7f755c100d80 2024-03-30T02:36:50.245 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.246+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f75440a11b0 msgr2=0x7f75440a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f75440a11b0 0x7f75440a3670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7548003730 tx=0x7f754800b540 comp rx=0 tx=0).stop 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c100d80 msgr2=0x7f755c198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c100d80 0x7f755c198f00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f755000e970 tx=0x7f755000ec80 comp rx=0 tx=0).stop 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 shutdown_connections 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f75440a11b0 0x7f75440a3670 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f755c100d80 0x7f755c198f00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f755c102910 0x7f755c1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 --2- 172.21.15.67:0/4135838953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f755c101f70 0x7f755c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 >> 172.21.15.67:0/4135838953 conn(0x7f755c0fc2f0 msgr2=0x7f755c107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 shutdown_connections 2024-03-30T02:36:50.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.247+0000 7f7563800700 1 -- 172.21.15.67:0/4135838953 wait complete. 2024-03-30T02:36:50.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:36:50.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.468+0000 7f8d26400700 1 -- 172.21.15.67:0/1324646251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20100d80 msgr2=0x7f8d201011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.468+0000 7f8d26400700 1 --2- 172.21.15.67:0/1324646251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20100d80 0x7f8d201011a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8d100099c0 tx=0x7f8d10009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.468+0000 7f8d26400700 1 -- 172.21.15.67:0/1324646251 shutdown_connections 2024-03-30T02:36:50.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.468+0000 7f8d26400700 1 --2- 172.21.15.67:0/1324646251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20100d80 0x7f8d201011a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.468+0000 7f8d26400700 1 --2- 172.21.15.67:0/1324646251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20102910 0x7f8d201091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.468+0000 7f8d26400700 1 --2- 172.21.15.67:0/1324646251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d20101f70 0x7f8d201023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.468+0000 7f8d26400700 1 -- 172.21.15.67:0/1324646251 >> 172.21.15.67:0/1324646251 conn(0x7f8d200fc2f0 msgr2=0x7f8d200fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:50.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.468+0000 7f8d26400700 1 -- 172.21.15.67:0/1324646251 shutdown_connections 2024-03-30T02:36:50.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.468+0000 7f8d26400700 1 -- 172.21.15.67:0/1324646251 wait complete. 2024-03-30T02:36:50.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.469+0000 7f8d26400700 1 Processor -- start 2024-03-30T02:36:50.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.469+0000 7f8d26400700 1 -- start start 2024-03-30T02:36:50.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d26400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20100d80 0x7f8d20198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d26400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d20101f70 0x7f8d201994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d26400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102910 0x7f8d201a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d26400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d201a0a40 con 0x7f8d20102910 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d26400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d201a0b80 con 0x7f8d20101f70 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d26400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d201a0d10 con 0x7f8d20100d80 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d1ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d20101f70 0x7f8d201994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d1fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102910 0x7f8d201a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d1ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d20101f70 0x7f8d201994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:45152/0 (socket says 172.21.15.67:45152) 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d1ea00700 1 -- 172.21.15.67:0/3214897864 learned_addr learned my addr 172.21.15.67:0/3214897864 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.470+0000 7f8d1f400700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20100d80 0x7f8d20198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d1ea00700 1 -- 172.21.15.67:0/3214897864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20100d80 msgr2=0x7f8d20198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d1ea00700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20100d80 0x7f8d20198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d1ea00700 1 -- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102910 msgr2=0x7f8d201a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d1ea00700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102910 0x7f8d201a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d1ea00700 1 -- 172.21.15.67:0/3214897864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d100095f0 con 0x7f8d20101f70 2024-03-30T02:36:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d1fe00700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102910 0x7f8d201a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d1ea00700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d20101f70 0x7f8d201994a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00e9a0 tx=0x7f8d0c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d17e00700 1 -- 172.21.15.67:0/3214897864 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c009790 con 0x7f8d20101f70 2024-03-30T02:36:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d201a0ff0 con 0x7f8d20101f70 2024-03-30T02:36:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d17e00700 1 -- 172.21.15.67:0/3214897864 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d0c010430 con 0x7f8d20101f70 2024-03-30T02:36:50.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d17e00700 1 -- 172.21.15.67:0/3214897864 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c018680 con 0x7f8d20101f70 2024-03-30T02:36:50.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.471+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d201a14c0 con 0x7f8d20101f70 2024-03-30T02:36:50.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.472+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d20067af0 con 0x7f8d20101f70 2024-03-30T02:36:50.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.474+0000 7f8d17e00700 1 -- 172.21.15.67:0/3214897864 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f8d0c0105a0 con 0x7f8d20101f70 2024-03-30T02:36:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.476+0000 7f8d17e00700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d040a11b0 0x7f8d040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.476+0000 7f8d17e00700 1 -- 172.21.15.67:0/3214897864 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f8d0c014070 con 0x7f8d20101f70 2024-03-30T02:36:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.476+0000 7f8d1f400700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d040a11b0 0x7f8d040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.476+0000 7f8d1f400700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d040a11b0 0x7f8d040a3670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d10003730 tx=0x7f8d10000bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:50.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.478+0000 7f8d17e00700 1 -- 172.21.15.67:0/3214897864 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8d0c0961e0 con 0x7f8d20101f70 2024-03-30T02:36:50.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:50 smithi082 ceph-mon[79176]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:50.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:50 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1430309847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:50 smithi067 ceph-mon[37808]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:50 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1430309847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:50 smithi170 ceph-mon[78798]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-30T02:36:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:50 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1430309847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:50.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.707+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d2010c770 con 0x7f8d20101f70 2024-03-30T02:36:50.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.708+0000 7f8d17e00700 1 -- 172.21.15.67:0/3214897864 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d2010c770 con 0x7f8d20101f70 2024-03-30T02:36:50.709 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.710+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d040a11b0 msgr2=0x7f8d040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d040a11b0 0x7f8d040a3670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d10003730 tx=0x7f8d10000bc0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d20101f70 msgr2=0x7f8d201994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d20101f70 0x7f8d201994a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00e9a0 tx=0x7f8d0c00ecb0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 shutdown_connections 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f8d040a11b0 0x7f8d040a3670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d20102910 0x7f8d201a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20100d80 0x7f8d20198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 --2- 172.21.15.67:0/3214897864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d20101f70 0x7f8d201994a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 >> 172.21.15.67:0/3214897864 conn(0x7f8d200fc2f0 msgr2=0x7f8d20107790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 shutdown_connections 2024-03-30T02:36:50.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.711+0000 7f8d26400700 1 -- 172.21.15.67:0/3214897864 wait complete. 2024-03-30T02:36:50.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:36:50.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.934+0000 7f7d15600700 1 -- 172.21.15.67:0/2220131509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101f70 msgr2=0x7f7d101023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.934+0000 7f7d15600700 1 --2- 172.21.15.67:0/2220131509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101f70 0x7f7d101023d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0098f0 tx=0x7f7cfc009c00 comp rx=0 tx=0).stop 2024-03-30T02:36:50.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.934+0000 7f7d15600700 1 -- 172.21.15.67:0/2220131509 shutdown_connections 2024-03-30T02:36:50.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.934+0000 7f7d15600700 1 --2- 172.21.15.67:0/2220131509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d10100d80 0x7f7d101011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.934+0000 7f7d15600700 1 --2- 172.21.15.67:0/2220131509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10102910 0x7f7d101091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.934+0000 7f7d15600700 1 --2- 172.21.15.67:0/2220131509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101f70 0x7f7d101023d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.935+0000 7f7d15600700 1 -- 172.21.15.67:0/2220131509 >> 172.21.15.67:0/2220131509 conn(0x7f7d100fc2f0 msgr2=0x7f7d100fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:50.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.935+0000 7f7d15600700 1 -- 172.21.15.67:0/2220131509 shutdown_connections 2024-03-30T02:36:50.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.935+0000 7f7d15600700 1 -- 172.21.15.67:0/2220131509 wait complete. 2024-03-30T02:36:50.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d15600700 1 Processor -- start 2024-03-30T02:36:50.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d15600700 1 -- start start 2024-03-30T02:36:50.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d15600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10100d80 0x7f7d10196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d15600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101f70 0x7f7d101972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d15600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d10102910 0x7f7d1019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d15600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1019e8c0 con 0x7f7d10102910 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d15600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1019ea00 con 0x7f7d10100d80 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d15600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1019eb90 con 0x7f7d10101f70 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d0f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d10102910 0x7f7d1019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d0f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d10102910 0x7f7d1019e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56744/0 (socket says 172.21.15.67:56744) 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d0e000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101f70 0x7f7d101972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d0f400700 1 -- 172.21.15.67:0/1451926662 learned_addr learned my addr 172.21.15.67:0/1451926662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.936+0000 7f7d0ea00700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10100d80 0x7f7d10196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d0f400700 1 -- 172.21.15.67:0/1451926662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101f70 msgr2=0x7f7d101972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d0f400700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101f70 0x7f7d101972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d0f400700 1 -- 172.21.15.67:0/1451926662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10100d80 msgr2=0x7f7d10196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:50.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d0f400700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10100d80 0x7f7d10196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:50.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d0f400700 1 -- 172.21.15.67:0/1451926662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000095f0 con 0x7f7d10102910 2024-03-30T02:36:50.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d0f400700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d10102910 0x7f7d1019e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800ef30 tx=0x7f7d0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:50.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d07400700 1 -- 172.21.15.67:0/1451926662 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08018070 con 0x7f7d10102910 2024-03-30T02:36:50.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc009520 con 0x7f7d10102910 2024-03-30T02:36:50.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d07400700 1 -- 172.21.15.67:0/1451926662 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d0800f040 con 0x7f7d10102910 2024-03-30T02:36:50.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1019f190 con 0x7f7d10102910 2024-03-30T02:36:50.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.937+0000 7f7d07400700 1 -- 172.21.15.67:0/1451926662 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0800cca0 con 0x7f7d10102910 2024-03-30T02:36:50.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.938+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d101082a0 con 0x7f7d10102910 2024-03-30T02:36:50.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.940+0000 7f7d07400700 1 -- 172.21.15.67:0/1451926662 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f7d08003a00 con 0x7f7d10102910 2024-03-30T02:36:50.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.942+0000 7f7d07400700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7cf40a11b0 0x7f7cf40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:50.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.943+0000 7f7d07400700 1 -- 172.21.15.67:0/1451926662 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f7d080cc8a0 con 0x7f7d10102910 2024-03-30T02:36:50.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.943+0000 7f7d0ea00700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7cf40a11b0 0x7f7cf40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:50.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.943+0000 7f7d0ea00700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7cf40a11b0 0x7f7cf40a3670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d10101dd0 tx=0x7f7d00009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:50.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:50.947+0000 7f7d07400700 1 -- 172.21.15.67:0/1451926662 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7d08095bc0 con 0x7f7d10102910 2024-03-30T02:36:51.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.166+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d1010c6f0 con 0x7f7d10102910 2024-03-30T02:36:51.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.167+0000 7f7d07400700 1 -- 172.21.15.67:0/1451926662 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d080991e0 con 0x7f7d10102910 2024-03-30T02:36:51.168 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:36:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7cf40a11b0 msgr2=0x7f7cf40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7cf40a11b0 0x7f7cf40a3670 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d10101dd0 tx=0x7f7d00009310 comp rx=0 tx=0).stop 2024-03-30T02:36:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d10102910 msgr2=0x7f7d1019e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d10102910 0x7f7d1019e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800ef30 tx=0x7f7d0800c3a0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 shutdown_connections 2024-03-30T02:36:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f7cf40a11b0 0x7f7cf40a3670 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7d10102910 0x7f7d1019e2f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d10100d80 0x7f7d10196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 --2- 172.21.15.67:0/1451926662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101f70 0x7f7d101972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 >> 172.21.15.67:0/1451926662 conn(0x7f7d100fc2f0 msgr2=0x7f7d101053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.169+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 shutdown_connections 2024-03-30T02:36:51.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.170+0000 7f7d15600700 1 -- 172.21.15.67:0/1451926662 wait complete. 2024-03-30T02:36:51.191 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:36:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.396+0000 7eff14a00700 1 -- 172.21.15.67:0/776565538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff10100d80 msgr2=0x7eff101011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.396+0000 7eff14a00700 1 --2- 172.21.15.67:0/776565538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff10100d80 0x7eff101011a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7eff000099c0 tx=0x7eff00009cd0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.397+0000 7eff14a00700 1 -- 172.21.15.67:0/776565538 shutdown_connections 2024-03-30T02:36:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.397+0000 7eff14a00700 1 --2- 172.21.15.67:0/776565538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff10100d80 0x7eff101011a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.397+0000 7eff14a00700 1 --2- 172.21.15.67:0/776565538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff10102910 0x7eff101091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.397+0000 7eff14a00700 1 --2- 172.21.15.67:0/776565538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff10101f70 0x7eff101023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.397+0000 7eff14a00700 1 -- 172.21.15.67:0/776565538 >> 172.21.15.67:0/776565538 conn(0x7eff100fc2f0 msgr2=0x7eff100fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.397+0000 7eff14a00700 1 -- 172.21.15.67:0/776565538 shutdown_connections 2024-03-30T02:36:51.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.397+0000 7eff14a00700 1 -- 172.21.15.67:0/776565538 wait complete. 2024-03-30T02:36:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.398+0000 7eff14a00700 1 Processor -- start 2024-03-30T02:36:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.398+0000 7eff14a00700 1 -- start start 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.398+0000 7eff14a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff10100d80 0x7eff10074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.398+0000 7eff14a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff10101f70 0x7eff100731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.398+0000 7eff14a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff10102910 0x7eff100736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.398+0000 7eff14a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff10073de0 con 0x7eff10102910 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.398+0000 7eff14a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff10073f50 con 0x7eff10100d80 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.398+0000 7eff14a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff101a0e10 con 0x7eff10101f70 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff10100d80 0x7eff10074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0e800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff10102910 0x7eff100736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff10100d80 0x7eff10074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:45206/0 (socket says 172.21.15.67:45206) 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0de00700 1 -- 172.21.15.67:0/74815485 learned_addr learned my addr 172.21.15.67:0/74815485 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0d400700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff10101f70 0x7eff100731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0de00700 1 -- 172.21.15.67:0/74815485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff10101f70 msgr2=0x7eff100731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0de00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff10101f70 0x7eff100731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0de00700 1 -- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff10102910 msgr2=0x7eff100736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0de00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff10102910 0x7eff100736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0de00700 1 -- 172.21.15.67:0/74815485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff000095f0 con 0x7eff10100d80 2024-03-30T02:36:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0e800700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff10102910 0x7eff100736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.399+0000 7eff0de00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff10100d80 0x7eff10074ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7eff00006010 tx=0x7eff00004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.400+0000 7eff06a00700 1 -- 172.21.15.67:0/74815485 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0001d070 con 0x7eff10100d80 2024-03-30T02:36:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.400+0000 7eff06a00700 1 -- 172.21.15.67:0/74815485 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff00004050 con 0x7eff10100d80 2024-03-30T02:36:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.400+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff101a1010 con 0x7eff10100d80 2024-03-30T02:36:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.400+0000 7eff06a00700 1 -- 172.21.15.67:0/74815485 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00005280 con 0x7eff10100d80 2024-03-30T02:36:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.400+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff101a13d0 con 0x7eff10100d80 2024-03-30T02:36:51.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.401+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff101082a0 con 0x7eff10100d80 2024-03-30T02:36:51.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.403+0000 7eff06a00700 1 -- 172.21.15.67:0/74815485 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7eff000048e0 con 0x7eff10100d80 2024-03-30T02:36:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.404+0000 7eff06a00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efef40a11b0 0x7efef40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.404+0000 7eff06a00700 1 -- 172.21.15.67:0/74815485 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7eff000cdc00 con 0x7eff10100d80 2024-03-30T02:36:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.404+0000 7eff0d400700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efef40a11b0 0x7efef40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.405+0000 7eff0d400700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efef40a11b0 0x7efef40a3670 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efefc005950 tx=0x7efefc00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:51.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.406+0000 7eff06a00700 1 -- 172.21.15.67:0/74815485 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7eff00096f20 con 0x7eff10100d80 2024-03-30T02:36:51.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:51 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4135838953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:51.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:51 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3214897864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:51.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:51 smithi082 ceph-mon[79176]: pgmap v299: 129 pgs: 129 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-30T02:36:51.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:51 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1451926662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:51.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.626+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7eff1010c5e0 con 0x7eff10100d80 2024-03-30T02:36:51.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.627+0000 7eff06a00700 1 -- 172.21.15.67:0/74815485 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7eff0009a540 con 0x7eff10100d80 2024-03-30T02:36:51.628 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:36:51.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efef40a11b0 msgr2=0x7efef40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efef40a11b0 0x7efef40a3670 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efefc005950 tx=0x7efefc00b410 comp rx=0 tx=0).stop 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff10100d80 msgr2=0x7eff10074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff10100d80 0x7eff10074ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7eff00006010 tx=0x7eff00004730 comp rx=0 tx=0).stop 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 shutdown_connections 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7efef40a11b0 0x7efef40a3670 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7eff10102910 0x7eff100736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff10100d80 0x7eff10074ac0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 --2- 172.21.15.67:0/74815485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff10101f70 0x7eff100731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.629+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 >> 172.21.15.67:0/74815485 conn(0x7eff100fc2f0 msgr2=0x7eff101053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.630+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 shutdown_connections 2024-03-30T02:36:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.630+0000 7eff14a00700 1 -- 172.21.15.67:0/74815485 wait complete. 2024-03-30T02:36:51.647 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:51 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4135838953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:51.647 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:51 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3214897864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:51.647 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:51 smithi067 ceph-mon[37808]: pgmap v299: 129 pgs: 129 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-30T02:36:51.647 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:51 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1451926662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:51.649 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:36:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:51 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4135838953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:36:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:51 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3214897864' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:36:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:51 smithi170 ceph-mon[78798]: pgmap v299: 129 pgs: 129 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-30T02:36:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:51 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1451926662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:36:51.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.846+0000 7f457a800700 1 -- 172.21.15.67:0/3292673240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 msgr2=0x7f4574108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.846+0000 7f457a800700 1 --2- 172.21.15.67:0/3292673240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 0x7f4574108de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f456c00b0a0 tx=0x7f456c00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.846+0000 7f457a800700 1 -- 172.21.15.67:0/3292673240 shutdown_connections 2024-03-30T02:36:51.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.846+0000 7f457a800700 1 --2- 172.21.15.67:0/3292673240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 0x7f4574108de0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.846+0000 7f457a800700 1 --2- 172.21.15.67:0/3292673240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4574101710 0x7f4574103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.846+0000 7f457a800700 1 --2- 172.21.15.67:0/3292673240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4574104040 0x7f4574106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.846+0000 7f457a800700 1 -- 172.21.15.67:0/3292673240 >> 172.21.15.67:0/3292673240 conn(0x7f45740fb100 msgr2=0x7f45740fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:51.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.846+0000 7f457a800700 1 -- 172.21.15.67:0/3292673240 shutdown_connections 2024-03-30T02:36:51.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.846+0000 7f457a800700 1 -- 172.21.15.67:0/3292673240 wait complete. 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.847+0000 7f457a800700 1 Processor -- start 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.847+0000 7f457a800700 1 -- start start 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.847+0000 7f457a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4574101710 0x7f4574198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.847+0000 7f457a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4574104040 0x7f4574199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.847+0000 7f457a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 0x7f45741a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.847+0000 7f457a800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45741a0a00 con 0x7f4574106970 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.847+0000 7f457a800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45741a0b40 con 0x7f4574104040 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.847+0000 7f457a800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45741a0cd0 con 0x7f4574101710 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 0x7f45741a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:51.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 0x7f45741a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56790/0 (socket says 172.21.15.67:56790) 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573e00700 1 -- 172.21.15.67:0/699865152 learned_addr learned my addr 172.21.15.67:0/699865152 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4572a00700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4574104040 0x7f4574199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573400700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4574101710 0x7f4574198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573e00700 1 -- 172.21.15.67:0/699865152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4574101710 msgr2=0x7f4574198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573e00700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4574101710 0x7f4574198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573e00700 1 -- 172.21.15.67:0/699865152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4574104040 msgr2=0x7f4574199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573e00700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4574104040 0x7f4574199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573e00700 1 -- 172.21.15.67:0/699865152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4560009520 con 0x7f4574106970 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.848+0000 7f4573e00700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 0x7f45741a0430 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f456c0078c0 tx=0x7f456c0078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.849+0000 7f456be00700 1 -- 172.21.15.67:0/699865152 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c00e040 con 0x7f4574106970 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.849+0000 7f456be00700 1 -- 172.21.15.67:0/699865152 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f456c00be40 con 0x7f4574106970 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.849+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456c009d00 con 0x7f4574106970 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.849+0000 7f456be00700 1 -- 172.21.15.67:0/699865152 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c0052a0 con 0x7f4574106970 2024-03-30T02:36:51.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.849+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45741a11e0 con 0x7f4574106970 2024-03-30T02:36:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.850+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45741084f0 con 0x7f4574106970 2024-03-30T02:36:51.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.852+0000 7f456be00700 1 -- 172.21.15.67:0/699865152 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f456c01d470 con 0x7f4574106970 2024-03-30T02:36:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.853+0000 7f456be00700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f45580a1230 0x7f45580a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.853+0000 7f456be00700 1 -- 172.21.15.67:0/699865152 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f456c0ce360 con 0x7f4574106970 2024-03-30T02:36:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.853+0000 7f4573400700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f45580a1230 0x7f45580a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:51.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.854+0000 7f4573400700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f45580a1230 0x7f45580a36f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4564009c60 tx=0x7f4564009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:51.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:51.857+0000 7f456be00700 1 -- 172.21.15.67:0/699865152 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f456c097680 con 0x7f4574106970 2024-03-30T02:36:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.131+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4574067180 con 0x7f4574106970 2024-03-30T02:36:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.132+0000 7f456be00700 1 -- 172.21.15.67:0/699865152 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f456c09aca0 con 0x7f4574106970 2024-03-30T02:36:52.133 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:52.133 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:36:42.080059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.134+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f45580a1230 msgr2=0x7f45580a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.134+0000 7f457a800700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f45580a1230 0x7f45580a36f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4564009c60 tx=0x7f4564009260 comp rx=0 tx=0).stop 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.134+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 msgr2=0x7f45741a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.134+0000 7f457a800700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 0x7f45741a0430 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f456c0078c0 tx=0x7f456c0078f0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.134+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 shutdown_connections 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.134+0000 7f457a800700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f45580a1230 0x7f45580a36f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.134+0000 7f457a800700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4574106970 0x7f45741a0430 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.134+0000 7f457a800700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4574101710 0x7f4574198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.134+0000 7f457a800700 1 --2- 172.21.15.67:0/699865152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4574104040 0x7f4574199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.135+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 >> 172.21.15.67:0/699865152 conn(0x7f45740fb100 msgr2=0x7f4574100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.135+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 shutdown_connections 2024-03-30T02:36:52.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.135+0000 7f457a800700 1 -- 172.21.15.67:0/699865152 wait complete. 2024-03-30T02:36:52.137 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-03-30T02:36:52.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:36:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.353+0000 7f1e26e00700 1 -- 172.21.15.67:0/2077223873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20102810 msgr2=0x7f1e201090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.353+0000 7f1e26e00700 1 --2- 172.21.15.67:0/2077223873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20102810 0x7f1e201090c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800b0d0 tx=0x7f1e1800b3e0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.353+0000 7f1e26e00700 1 -- 172.21.15.67:0/2077223873 shutdown_connections 2024-03-30T02:36:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.353+0000 7f1e26e00700 1 --2- 172.21.15.67:0/2077223873 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20102810 0x7f1e201090c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.353+0000 7f1e26e00700 1 --2- 172.21.15.67:0/2077223873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e20100c80 0x7f1e201010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.353+0000 7f1e26e00700 1 --2- 172.21.15.67:0/2077223873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e20101e70 0x7f1e201022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.353+0000 7f1e26e00700 1 -- 172.21.15.67:0/2077223873 >> 172.21.15.67:0/2077223873 conn(0x7f1e200fc1f0 msgr2=0x7f1e200fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.353+0000 7f1e26e00700 1 -- 172.21.15.67:0/2077223873 shutdown_connections 2024-03-30T02:36:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.354+0000 7f1e26e00700 1 -- 172.21.15.67:0/2077223873 wait complete. 2024-03-30T02:36:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.355+0000 7f1e26e00700 1 Processor -- start 2024-03-30T02:36:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.355+0000 7f1e26e00700 1 -- start start 2024-03-30T02:36:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e26e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20100c80 0x7f1e20074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e26e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e20101e70 0x7f1e200731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e26e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e20102810 0x7f1e200736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e26e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20073e80 con 0x7f1e20100c80 2024-03-30T02:36:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e26e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20073ff0 con 0x7f1e20102810 2024-03-30T02:36:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e26e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201a0d50 con 0x7f1e20101e70 2024-03-30T02:36:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20100c80 0x7f1e20074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20100c80 0x7f1e20074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56816/0 (socket says 172.21.15.67:56816) 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e1fe00700 1 -- 172.21.15.67:0/538814977 learned_addr learned my addr 172.21.15.67:0/538814977 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e24e00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e20102810 0x7f1e200736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e1f400700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e20101e70 0x7f1e200731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e1fe00700 1 -- 172.21.15.67:0/538814977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e20101e70 msgr2=0x7f1e200731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e1fe00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e20101e70 0x7f1e200731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e1fe00700 1 -- 172.21.15.67:0/538814977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e20102810 msgr2=0x7f1e200736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e1fe00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e20102810 0x7f1e200736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.356+0000 7f1e1fe00700 1 -- 172.21.15.67:0/538814977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e18009d00 con 0x7f1e20100c80 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.357+0000 7f1e1fe00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20100c80 0x7f1e20074b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000d980 tx=0x7f1e1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.357+0000 7f1e1cc00700 1 -- 172.21.15.67:0/538814977 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10004020 con 0x7f1e20100c80 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.357+0000 7f1e1cc00700 1 -- 172.21.15.67:0/538814977 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e100096d0 con 0x7f1e20100c80 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.357+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e201a0fe0 con 0x7f1e20100c80 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.357+0000 7f1e1cc00700 1 -- 172.21.15.67:0/538814977 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10003a80 con 0x7f1e20100c80 2024-03-30T02:36:52.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.357+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e201a1400 con 0x7f1e20100c80 2024-03-30T02:36:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.358+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e201080b0 con 0x7f1e20100c80 2024-03-30T02:36:52.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.360+0000 7f1e1cc00700 1 -- 172.21.15.67:0/538814977 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f1e10009840 con 0x7f1e20100c80 2024-03-30T02:36:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.361+0000 7f1e1cc00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1e040a11b0 0x7f1e040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.361+0000 7f1e1cc00700 1 -- 172.21.15.67:0/538814977 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f1e100cca40 con 0x7f1e20100c80 2024-03-30T02:36:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.361+0000 7f1e1f400700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1e040a11b0 0x7f1e040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.362+0000 7f1e1f400700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1e040a11b0 0x7f1e040a3670 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c005fd0 tx=0x7f1e0c005f60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.365+0000 7f1e1cc00700 1 -- 172.21.15.67:0/538814977 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1e10095d30 con 0x7f1e20100c80 2024-03-30T02:36:52.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:52 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/74815485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:52.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:52 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/699865152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:52.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:52 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/74815485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:52.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:52 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/699865152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:52.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.616+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e2010c570 con 0x7f1e20100c80 2024-03-30T02:36:52.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.618+0000 7f1e1cc00700 1 -- 172.21.15.67:0/538814977 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24479 (secure 0 0 0) 0x7f1e10099350 con 0x7f1e20100c80 2024-03-30T02:36:52.619 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:36:52.619 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:36:28.218474+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:36:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1e040a11b0 msgr2=0x7f1e040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1e040a11b0 0x7f1e040a3670 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c005fd0 tx=0x7f1e0c005f60 comp rx=0 tx=0).stop 2024-03-30T02:36:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20100c80 msgr2=0x7f1e20074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20100c80 0x7f1e20074b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000d980 tx=0x7f1e1000dc90 comp rx=0 tx=0).stop 2024-03-30T02:36:52.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 shutdown_connections 2024-03-30T02:36:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1e040a11b0 0x7f1e040a3670 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e20100c80 0x7f1e20074b10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e20102810 0x7f1e200736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 --2- 172.21.15.67:0/538814977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e20101e70 0x7f1e200731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 >> 172.21.15.67:0/538814977 conn(0x7f1e200fc1f0 msgr2=0x7f1e201051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 shutdown_connections 2024-03-30T02:36:52.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.620+0000 7f1e26e00700 1 -- 172.21.15.67:0/538814977 wait complete. 2024-03-30T02:36:52.640 DEBUG:tasks.fwd_scrub:Yielding 2024-03-30T02:36:52.640 INFO:teuthology.run_tasks:Running task exec... 2024-03-30T02:36:52.651 INFO:teuthology.task.exec:Executing custom commands... 2024-03-30T02:36:52.651 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:36:52.651 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-30T02:36:52.652 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-30T02:36:52.652 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:36:52.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:36:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:52 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/74815485' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:36:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:52 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/699865152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:36:52.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.848+0000 7fcda3800700 1 -- 172.21.15.67:0/3688221205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd9c074df0 msgr2=0x7fcd9c073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.848+0000 7fcda3800700 1 --2- 172.21.15.67:0/3688221205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd9c074df0 0x7fcd9c073240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd900098f0 tx=0x7fcd90009c00 comp rx=0 tx=0).stop 2024-03-30T02:36:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.848+0000 7fcda3800700 1 -- 172.21.15.67:0/3688221205 shutdown_connections 2024-03-30T02:36:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.848+0000 7fcda3800700 1 --2- 172.21.15.67:0/3688221205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c108af0 0x7fcd9c10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.848+0000 7fcda3800700 1 --2- 172.21.15.67:0/3688221205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd9c074df0 0x7fcd9c073240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.848+0000 7fcda3800700 1 --2- 172.21.15.67:0/3688221205 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd9c073810 0x7fcd9c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.848+0000 7fcda3800700 1 -- 172.21.15.67:0/3688221205 >> 172.21.15.67:0/3688221205 conn(0x7fcd9c0fc1c0 msgr2=0x7fcd9c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.848+0000 7fcda3800700 1 -- 172.21.15.67:0/3688221205 shutdown_connections 2024-03-30T02:36:52.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.848+0000 7fcda3800700 1 -- 172.21.15.67:0/3688221205 wait complete. 2024-03-30T02:36:52.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.849+0000 7fcda3800700 1 Processor -- start 2024-03-30T02:36:52.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcda3800700 1 -- start start 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcda3800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c073810 0x7fcd9c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcda3800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd9c074df0 0x7fcd9c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcda3800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd9c108af0 0x7fcd9c1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcda3800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1a0960 con 0x7fcd9c074df0 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcda3800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1a0aa0 con 0x7fcd9c108af0 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcda3800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1a0c30 con 0x7fcd9c073810 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcd9be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd9c074df0 0x7fcd9c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcd9be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd9c074df0 0x7fcd9c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56820/0 (socket says 172.21.15.67:56820) 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcd9be00700 1 -- 172.21.15.67:0/54029565 learned_addr learned my addr 172.21.15.67:0/54029565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:52.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcda1800700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd9c108af0 0x7fcd9c1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.850+0000 7fcda0e00700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c073810 0x7fcd9c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcd9be00700 1 -- 172.21.15.67:0/54029565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c073810 msgr2=0x7fcd9c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcd9be00700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c073810 0x7fcd9c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcd9be00700 1 -- 172.21.15.67:0/54029565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd9c108af0 msgr2=0x7fcd9c1a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcd9be00700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd9c108af0 0x7fcd9c1a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcd9be00700 1 -- 172.21.15.67:0/54029565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd90009520 con 0x7fcd9c074df0 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcd9be00700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd9c074df0 0x7fcd9c199370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800e9f0 tx=0x7fcd8800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcd99600700 1 -- 172.21.15.67:0/54029565 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88004220 con 0x7fcd9c074df0 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcd99600700 1 -- 172.21.15.67:0/54029565 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd88004380 con 0x7fcd9c074df0 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcd99600700 1 -- 172.21.15.67:0/54029565 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88003be0 con 0x7fcd9c074df0 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c1a0f10 con 0x7fcd9c074df0 2024-03-30T02:36:52.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.851+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c1a1340 con 0x7fcd9c074df0 2024-03-30T02:36:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.852+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c1078b0 con 0x7fcd9c074df0 2024-03-30T02:36:52.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.854+0000 7fcd99600700 1 -- 172.21.15.67:0/54029565 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fcd88003d40 con 0x7fcd9c074df0 2024-03-30T02:36:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.856+0000 7fcd99600700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcd840a1220 0x7fcd840a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.856+0000 7fcd99600700 1 -- 172.21.15.67:0/54029565 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fcd88014070 con 0x7fcd9c074df0 2024-03-30T02:36:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.856+0000 7fcda0e00700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcd840a1220 0x7fcd840a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.857+0000 7fcda0e00700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcd840a1220 0x7fcd840a36e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000fa40 tx=0x7fcd90019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.860+0000 7fcd99600700 1 -- 172.21.15.67:0/54029565 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fcd88096180 con 0x7fcd9c074df0 2024-03-30T02:36:52.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.862+0000 7f9504600700 1 -- 172.21.15.67:0/2837350441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94fc073810 msgr2=0x7f94fc073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.862+0000 7f9504600700 1 --2- 172.21.15.67:0/2837350441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94fc073810 0x7f94fc073c90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f94e80098f0 tx=0x7f94e8009c00 comp rx=0 tx=0).stop 2024-03-30T02:36:52.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.862+0000 7f9504600700 1 -- 172.21.15.67:0/2837350441 shutdown_connections 2024-03-30T02:36:52.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.862+0000 7f9504600700 1 --2- 172.21.15.67:0/2837350441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94fc108c40 0x7f94fc10b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.862+0000 7f9504600700 1 --2- 172.21.15.67:0/2837350441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc074df0 0x7f94fc073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.862+0000 7f9504600700 1 --2- 172.21.15.67:0/2837350441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94fc073810 0x7f94fc073c90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.862+0000 7f9504600700 1 -- 172.21.15.67:0/2837350441 >> 172.21.15.67:0/2837350441 conn(0x7f94fc0fc2f0 msgr2=0x7f94fc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.863+0000 7f9504600700 1 -- 172.21.15.67:0/2837350441 shutdown_connections 2024-03-30T02:36:52.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.863+0000 7f9504600700 1 -- 172.21.15.67:0/2837350441 wait complete. 2024-03-30T02:36:52.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.864+0000 7f9504600700 1 Processor -- start 2024-03-30T02:36:52.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.864+0000 7f9504600700 1 -- start start 2024-03-30T02:36:52.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9504600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94fc073810 0x7f94fc198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9504600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc074df0 0x7f94fc1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9504600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94fc108c40 0x7f94fc1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9504600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc1a0aa0 con 0x7f94fc073810 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9504600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc1a0be0 con 0x7f94fc074df0 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9504600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc1a0d70 con 0x7f94fc108c40 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9501c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94fc073810 0x7f94fc198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9501c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94fc073810 0x7f94fc198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56836/0 (socket says 172.21.15.67:56836) 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9501c00700 1 -- 172.21.15.67:0/1772031813 learned_addr learned my addr 172.21.15.67:0/1772031813 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9501200700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc074df0 0x7f94fc1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.865+0000 7f9502600700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94fc108c40 0x7f94fc1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f9501c00700 1 -- 172.21.15.67:0/1772031813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94fc108c40 msgr2=0x7f94fc1a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f9501c00700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94fc108c40 0x7f94fc1a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f9501c00700 1 -- 172.21.15.67:0/1772031813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc074df0 msgr2=0x7f94fc1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f9501c00700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc074df0 0x7f94fc1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f9501c00700 1 -- 172.21.15.67:0/1772031813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00095f0 con 0x7f94fc073810 2024-03-30T02:36:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f9502600700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94fc108c40 0x7f94fc1a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f9501c00700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94fc073810 0x7f94fc198f70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f94f000ec10 tx=0x7f94f000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f94fa000700 1 -- 172.21.15.67:0/1772031813 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f000cd40 con 0x7f94fc073810 2024-03-30T02:36:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f94fa000700 1 -- 172.21.15.67:0/1772031813 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f00044e0 con 0x7f94fc073810 2024-03-30T02:36:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f9504600700 1 -- 172.21.15.67:0/1772031813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e8009520 con 0x7f94fc073810 2024-03-30T02:36:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f94fa000700 1 -- 172.21.15.67:0/1772031813 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0010690 con 0x7f94fc073810 2024-03-30T02:36:52.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.866+0000 7f9504600700 1 -- 172.21.15.67:0/1772031813 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc1a12e0 con 0x7f94fc073810 2024-03-30T02:36:52.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.870+0000 7f94fa000700 1 -- 172.21.15.67:0/1772031813 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7f94f0003710 con 0x7f94fc073810 2024-03-30T02:36:52.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.871+0000 7f9518217b80 1 Processor -- start 2024-03-30T02:36:52.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.871+0000 7f9518217b80 1 -- start start 2024-03-30T02:36:52.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.871+0000 7f9518217b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a55f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.871+0000 7f9518217b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f71b0 0x55e4a55f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.871+0000 7f9518217b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f8050 0x55e4a55f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.871+0000 7f9518217b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a554ea10 con 0x55e4a55f6850 2024-03-30T02:36:52.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.871+0000 7f9518217b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a554c3d0 con 0x55e4a55f8050 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.871+0000 7f9518217b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a5547cc0 con 0x55e4a55f71b0 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9502600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a55f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9502600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a55f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56840/0 (socket says 172.21.15.67:56840) 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9502600700 1 -- 172.21.15.67:0/2482760307 learned_addr learned my addr 172.21.15.67:0/2482760307 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9501c00700 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f71b0 0x55e4a55f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9501200700 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f8050 0x55e4a55f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9502600700 1 -- 172.21.15.67:0/2482760307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f71b0 msgr2=0x55e4a55f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9502600700 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f71b0 0x55e4a55f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9502600700 1 -- 172.21.15.67:0/2482760307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f8050 msgr2=0x55e4a55f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:52.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9502600700 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f8050 0x55e4a55f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9502600700 1 -- 172.21.15.67:0/2482760307 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4a53c9430 con 0x55e4a55f6850 2024-03-30T02:36:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9501200700 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f8050 0x55e4a55f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f94fa000700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f94e40a11b0 0x7f94e40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.872+0000 7f9502600700 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a55f6c70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009440 tx=0x7f94f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220e731e3d1646e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.873+0000 7f94eea00700 1 -- 172.21.15.67:0/2482760307 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f400e070 con 0x55e4a55f6850 2024-03-30T02:36:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.873+0000 7f94fa000700 1 -- 172.21.15.67:0/1772031813 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f94f0014070 con 0x7f94fc073810 2024-03-30T02:36:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.873+0000 7f9501c00700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f94e40a11b0 0x7f94e40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.873+0000 7f94eea00700 1 -- 172.21.15.67:0/2482760307 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f400f040 con 0x55e4a55f6850 2024-03-30T02:36:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.873+0000 7f94eea00700 1 -- 172.21.15.67:0/2482760307 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4015c90 con 0x55e4a55f6850 2024-03-30T02:36:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 -- 172.21.15.67:0/2482760307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 msgr2=0x55e4a55f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9501c00700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f94e40a11b0 0x7f94e40a3670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f94f0096d70 tx=0x7f94f0096040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a55f6c70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009440 tx=0x7f94f4009920 comp rx=0 tx=0).stop 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 -- 172.21.15.67:0/2482760307 shutdown_connections 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a55f6c70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f8050 0x55e4a55f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 --2- 172.21.15.67:0/2482760307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f71b0 0x55e4a55f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 -- 172.21.15.67:0/2482760307 >> 172.21.15.67:0/2482760307 conn(0x55e4a5555720 msgr2=0x55e4a55f1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 -- 172.21.15.67:0/2482760307 shutdown_connections 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 -- 172.21.15.67:0/2482760307 wait complete. 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.874+0000 7f9518217b80 1 -- start start 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.875+0000 7f9518217b80 1 Processor -- start 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.875+0000 7f9518217b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 0x55e4a56006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9518217b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5600bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9518217b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a5607c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9518217b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a54e3e00 con 0x55e4a55f71b0 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9518217b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a54e3aa0 con 0x55e4a55f6850 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9518217b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a53cb720 con 0x55e4a55f8050 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9518217b80 10 client.0 _ll_register_callbacks cb 0x55e4a54ed4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9501200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5600bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9502600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 0x55e4a56006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9501200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5600bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56850/0 (socket says 172.21.15.67:56850) 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9501200700 1 -- 172.21.15.67:0/1318747062 learned_addr learned my addr 172.21.15.67:0/1318747062 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9502600700 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a5607c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9501200700 1 -- 172.21.15.67:0/1318747062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 msgr2=0x55e4a5607c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:52.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9501200700 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a5607c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9501200700 1 -- 172.21.15.67:0/1318747062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 msgr2=0x55e4a56006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9501200700 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 0x55e4a56006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9501200700 1 -- 172.21.15.67:0/1318747062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc10bcf0 con 0x55e4a55f71b0 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.876+0000 7f9502600700 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a5607c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.877+0000 7f9501200700 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5600bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94e8005e90 tx=0x7f94e800f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225e687fbec419e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.877+0000 7f94e3400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.877+0000 7f94e3400700 1 -- 172.21.15.67:0/1318747062 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8027070 con 0x55e4a55f71b0 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.877+0000 7f94e3400700 1 -- 172.21.15.67:0/1318747062 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f94e800bb50 con 0x55e4a55f71b0 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.877+0000 7f9518217b80 1 -- 172.21.15.67:0/1318747062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4a5508400 con 0x55e4a55f71b0 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.877+0000 7f94e3400700 1 -- 172.21.15.67:0/1318747062 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94e8005040 con 0x55e4a55f71b0 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.877+0000 7f9518217b80 10 client.15549 fetch_fsmap learned FSMap version 29 2024-03-30T02:36:52.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.877+0000 7f9518217b80 1 -- 172.21.15.67:0/1318747062 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f94f40090f0 con 0x55e4a55f71b0 2024-03-30T02:36:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.877+0000 7f94e3400700 1 -- 172.21.15.67:0/1318747062 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4a5508400 con 0x55e4a55f71b0 2024-03-30T02:36:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.878+0000 7f94e3400700 1 -- 172.21.15.67:0/1318747062 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f94e802a020 con 0x55e4a55f71b0 2024-03-30T02:36:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.878+0000 7f9518217b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:36:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.878+0000 7f9518217b80 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:36:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.878+0000 7f9518217b80 20 client.15549 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.878+0000 7f9518217b80 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e4a560c490 0x55e4a560e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.878+0000 7f9518217b80 4 client.15549 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.878+0000 7f9518217b80 1 -- 172.21.15.67:0/1318747062 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4a5540780 con 0x55e4a560c490 2024-03-30T02:36:52.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.878+0000 7f9502600700 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e4a560c490 0x55e4a560e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.879+0000 7f94e3400700 10 client.15549 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.879+0000 7f9502600700 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e4a560c490 0x55e4a560e8b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d62f0057f7df1a server_cookie=fbd8086b58c5460a in_seq=0 out_seq=0 2024-03-30T02:36:52.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.879+0000 7f94e3400700 1 -- 172.21.15.67:0/1318747062 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e4a5540780 con 0x55e4a560c490 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.879+0000 7f94e3400700 10 client.15549 handle_command_reply: tid=0 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.879+0000 7f9518217b80 1 client.15549 shutdown 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.879+0000 7f9518217b80 1 -- 172.21.15.67:0/1318747062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 msgr2=0x55e4a5600bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.879+0000 7f9518217b80 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5600bf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f94e8005e90 tx=0x7f94e800f960 comp rx=0 tx=0).stop 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f9518217b80 1 -- 172.21.15.67:0/1318747062 shutdown_connections 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f9518217b80 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a5607c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f9518217b80 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 0x55e4a56006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f9518217b80 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e4a560c490 0x55e4a560e8b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f9518217b80 1 --2- 172.21.15.67:0/1318747062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5600bf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f9518217b80 1 -- 172.21.15.67:0/1318747062 >> 172.21.15.67:0/1318747062 conn(0x55e4a5555720 msgr2=0x55e4a55f51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f94e3400700 0 client.15549 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f9518217b80 1 -- 172.21.15.67:0/1318747062 shutdown_connections 2024-03-30T02:36:52.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f9518217b80 1 -- 172.21.15.67:0/1318747062 wait complete. 2024-03-30T02:36:52.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.880+0000 7f9518217b80 20 client.15549 trim_cache size 0 max 16384 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9518217b80 1 Processor -- start 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9518217b80 1 -- start start 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9518217b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 0x55e4a5600460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9518217b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5607bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9518217b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a56009a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9518217b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a55f2530 con 0x55e4a55f71b0 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9518217b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a55f7ed0 con 0x55e4a55f6850 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9518217b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a5608db0 con 0x55e4a55f8050 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9501200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5607bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9501200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5607bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56854/0 (socket says 172.21.15.67:56854) 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9501200700 1 -- 172.21.15.67:0/2501931871 learned_addr learned my addr 172.21.15.67:0/2501931871 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:52.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.898+0000 7f9502600700 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 0x55e4a5600460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f9502600700 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a56009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f9501200700 1 -- 172.21.15.67:0/2501931871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 msgr2=0x55e4a56009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:52.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f9501200700 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a56009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f9501200700 1 -- 172.21.15.67:0/2501931871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 msgr2=0x55e4a5600460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:52.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f9501200700 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 0x55e4a5600460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f9501200700 1 -- 172.21.15.67:0/2501931871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4a56134e0 con 0x55e4a55f71b0 2024-03-30T02:36:52.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f9502600700 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a56009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:36:52.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f9501200700 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5607bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000c00 tx=0x7f94e80213b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87488f673c5d7f15 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f94ef400700 1 -- 172.21.15.67:0/2501931871 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8005940 con 0x55e4a55f71b0 2024-03-30T02:36:52.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f94ef400700 1 -- 172.21.15.67:0/2501931871 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94e8028020 con 0x55e4a55f71b0 2024-03-30T02:36:52.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.899+0000 7f94ef400700 1 -- 172.21.15.67:0/2501931871 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8005690 con 0x55e4a55f71b0 2024-03-30T02:36:52.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.900+0000 7f9518217b80 1 -- 172.21.15.67:0/2501931871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 msgr2=0x55e4a5607bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.900+0000 7f9518217b80 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5607bf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f94e8000c00 tx=0x7f94e80213b0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.901+0000 7f9518217b80 1 -- 172.21.15.67:0/2501931871 shutdown_connections 2024-03-30T02:36:52.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.901+0000 7f9518217b80 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a56009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.901+0000 7f9518217b80 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f6850 0x55e4a5600460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.901+0000 7f9518217b80 1 --2- 172.21.15.67:0/2501931871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f71b0 0x55e4a5607bf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.901+0000 7f9518217b80 1 -- 172.21.15.67:0/2501931871 >> 172.21.15.67:0/2501931871 conn(0x55e4a5555720 msgr2=0x55e4a55fb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.901+0000 7f9518217b80 1 -- 172.21.15.67:0/2501931871 shutdown_connections 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.901+0000 7f9518217b80 1 -- 172.21.15.67:0/2501931871 wait complete. 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.901+0000 7f9518217b80 1 -- start start 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.901+0000 7f9518217b80 1 Processor -- start 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9518217b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a5606c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9518217b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f71b0 0x55e4a56071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9518217b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a5660340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9518217b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a5660880 con 0x55e4a55f6850 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9518217b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a56609c0 con 0x55e4a55f71b0 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9518217b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a5660b50 con 0x55e4a55f8050 2024-03-30T02:36:52.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9518217b80 10 client.0 _ll_register_callbacks cb 0x55e4a54ed4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9502600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a5606c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9501200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f71b0 0x55e4a56071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9502600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a5606c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56864/0 (socket says 172.21.15.67:56864) 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9501200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f71b0 0x55e4a56071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:45312/0 (socket says 172.21.15.67:45312) 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9502600700 1 -- 172.21.15.67:0/2641751368 learned_addr learned my addr 172.21.15.67:0/2641751368 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9501200700 1 -- 172.21.15.67:0/2641751368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 msgr2=0x55e4a5660340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.902+0000 7f9501200700 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a5660340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f9501200700 1 -- 172.21.15.67:0/2641751368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 msgr2=0x55e4a5606c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f9501200700 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a5606c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f9501200700 1 -- 172.21.15.67:0/2641751368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f40090f0 con 0x55e4a55f71b0 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f9502600700 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a5606c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:52.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f9501200700 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f71b0 0x55e4a56071b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55e4a55f7a40 tx=0x7f94e800f8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aeeda3282b44df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f94e3400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f94e3400700 1 -- 172.21.15.67:0/2641751368 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8027070 con 0x55e4a55f71b0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f9518217b80 1 -- 172.21.15.67:0/2641751368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94f400f040 con 0x55e4a55f71b0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f94e3400700 1 -- 172.21.15.67:0/2641751368 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f94e8000e40 con 0x55e4a55f71b0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f94e3400700 1 -- 172.21.15.67:0/2641751368 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94e8023470 con 0x55e4a55f71b0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f9518217b80 10 client.25601 fetch_fsmap learned FSMap version 29 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.903+0000 7f9518217b80 1 -- 172.21.15.67:0/2641751368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e4a5375be0 con 0x55e4a55f71b0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.904+0000 7f94e3400700 1 -- 172.21.15.67:0/2641751368 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94f400f040 con 0x55e4a55f71b0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.904+0000 7f94e3400700 1 -- 172.21.15.67:0/2641751368 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f94e802a070 con 0x55e4a55f71b0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.904+0000 7f9518217b80 10 client.25601 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.904+0000 7f9518217b80 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.904+0000 7f9518217b80 20 client.25601 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.904+0000 7f9518217b80 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e4a56620f0 0x55e4a5664510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.904+0000 7f9518217b80 4 client.25601 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.904+0000 7f9518217b80 1 -- 172.21.15.67:0/2641751368 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e4a5396d70 con 0x55e4a56620f0 2024-03-30T02:36:52.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.904+0000 7f9502600700 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e4a56620f0 0x55e4a5664510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:52.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.905+0000 7f9502600700 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e4a56620f0 0x55e4a5664510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0c1a93b91b92c server_cookie=6dd7d1053059f51 in_seq=0 out_seq=0 2024-03-30T02:36:52.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.905+0000 7f94e3400700 10 client.25601 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.906+0000 7f94e3400700 1 -- 172.21.15.67:0/2641751368 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e4a5396d70 con 0x55e4a56620f0 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.906+0000 7f94e3400700 10 client.25601 handle_command_reply: tid=0 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.906+0000 7f9518217b80 1 client.25601 shutdown 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 -- 172.21.15.67:0/2641751368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f71b0 msgr2=0x55e4a56071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f71b0 0x55e4a56071b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55e4a55f7a40 tx=0x7f94e800f8d0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 -- 172.21.15.67:0/2641751368 shutdown_connections 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e4a56620f0 0x55e4a5664510 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4a55f6850 0x55e4a5606c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f94e3400700 0 client.25601 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4a55f8050 0x55e4a5660340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 --2- 172.21.15.67:0/2641751368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4a55f71b0 0x55e4a56071b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 -- 172.21.15.67:0/2641751368 >> 172.21.15.67:0/2641751368 conn(0x55e4a5555720 msgr2=0x55e4a5600460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 -- 172.21.15.67:0/2641751368 shutdown_connections 2024-03-30T02:36:52.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 1 -- 172.21.15.67:0/2641751368 wait complete. 2024-03-30T02:36:52.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.907+0000 7f9518217b80 20 client.25601 trim_cache size 0 max 16384 2024-03-30T02:36:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:52.912 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:36:52.912 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "32e3e246-de95-4cff-ad83-da993b27beca", 2024-03-30T02:36:52.912 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:36:52.912 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 -- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f94e40a11b0 msgr2=0x7f94e40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f94e40a11b0 0x7f94e40a3670 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f94f0096d70 tx=0x7f94f0096040 comp rx=0 tx=0).stop 2024-03-30T02:36:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 -- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94fc073810 msgr2=0x7f94fc198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94fc073810 0x7f94fc198f70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f94f000ec10 tx=0x7f94f000ef20 comp rx=0 tx=0).stop 2024-03-30T02:36:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 -- 172.21.15.67:0/1772031813 shutdown_connections 2024-03-30T02:36:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f94e40a11b0 0x7f94e40a3670 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94fc073810 0x7f94fc198f70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94fc108c40 0x7f94fc1a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 --2- 172.21.15.67:0/1772031813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc074df0 0x7f94fc1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 -- 172.21.15.67:0/1772031813 >> 172.21.15.67:0/1772031813 conn(0x7f94fc0fc2f0 msgr2=0x7f94fc104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 -- 172.21.15.67:0/1772031813 shutdown_connections 2024-03-30T02:36:52.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:52.908+0000 7f9504600700 1 -- 172.21.15.67:0/1772031813 wait complete. 2024-03-30T02:36:52.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:36:53.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.127+0000 7fc45f000700 1 -- 172.21.15.67:0/1690681424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc458074df0 msgr2=0x7fc458073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.127+0000 7fc45f000700 1 --2- 172.21.15.67:0/1690681424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc458074df0 0x7fc458073240 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc448009990 tx=0x7fc448009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.127+0000 7fc45f000700 1 -- 172.21.15.67:0/1690681424 shutdown_connections 2024-03-30T02:36:53.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.127+0000 7fc45f000700 1 --2- 172.21.15.67:0/1690681424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc458074df0 0x7fc458073240 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.127+0000 7fc45f000700 1 --2- 172.21.15.67:0/1690681424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45810d350 0x7fc45810f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.127+0000 7fc45f000700 1 --2- 172.21.15.67:0/1690681424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc458073810 0x7fc458073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.127+0000 7fc45f000700 1 -- 172.21.15.67:0/1690681424 >> 172.21.15.67:0/1690681424 conn(0x7fc4580fc2f0 msgr2=0x7fc4580fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.127+0000 7fc45f000700 1 -- 172.21.15.67:0/1690681424 shutdown_connections 2024-03-30T02:36:53.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.127+0000 7fc45f000700 1 -- 172.21.15.67:0/1690681424 wait complete. 2024-03-30T02:36:53.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.128+0000 7fc45f000700 1 Processor -- start 2024-03-30T02:36:53.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc45f000700 1 -- start start 2024-03-30T02:36:53.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc45f000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc458073810 0x7fc45819d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc45f000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc458074df0 0x7fc45819d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc45f000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc45810d350 0x7fc4581a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc45f000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4581a4e20 con 0x7fc458073810 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc45f000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4581a4f60 con 0x7fc458074df0 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc45f000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4581a50f0 con 0x7fc45810d350 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc457e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc458073810 0x7fc45819d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc457e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc458073810 0x7fc45819d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56888/0 (socket says 172.21.15.67:56888) 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc45d000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc45810d350 0x7fc4581a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc457e00700 1 -- 172.21.15.67:0/2765547477 learned_addr learned my addr 172.21.15.67:0/2765547477 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.129+0000 7fc457400700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc458074df0 0x7fc45819d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc45d000700 1 -- 172.21.15.67:0/2765547477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc458074df0 msgr2=0x7fc45819d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc45d000700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc458074df0 0x7fc45819d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc45d000700 1 -- 172.21.15.67:0/2765547477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc458073810 msgr2=0x7fc45819d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc45d000700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc458073810 0x7fc45819d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc45d000700 1 -- 172.21.15.67:0/2765547477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480095f0 con 0x7fc45810d350 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc45d000700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc45810d350 0x7fc4581a4850 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc450007f10 tx=0x7fc45000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc454c00700 1 -- 172.21.15.67:0/2765547477 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450004db0 con 0x7fc45810d350 2024-03-30T02:36:53.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc45f000700 1 -- 172.21.15.67:0/2765547477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4581a53d0 con 0x7fc45810d350 2024-03-30T02:36:53.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc454c00700 1 -- 172.21.15.67:0/2765547477 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc45000f040 con 0x7fc45810d350 2024-03-30T02:36:53.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc454c00700 1 -- 172.21.15.67:0/2765547477 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450005990 con 0x7fc45810d350 2024-03-30T02:36:53.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.130+0000 7fc45f000700 1 -- 172.21.15.67:0/2765547477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4581a5930 con 0x7fc45810d350 2024-03-30T02:36:53.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.134+0000 7fc454c00700 1 -- 172.21.15.67:0/2765547477 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) v1 ==== 135916+0+0 (secure 0 0 0) 0x7fc45000db20 con 0x7fc45810d350 2024-03-30T02:36:53.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc472bb9b80 1 Processor -- start 2024-03-30T02:36:53.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc454c00700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc43c0a11b0 0x7fc43c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc472bb9b80 1 -- start start 2024-03-30T02:36:53.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc454c00700 1 -- 172.21.15.67:0/2765547477 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fc4500cc5f0 con 0x7fc45810d350 2024-03-30T02:36:53.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc457400700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc43c0a11b0 0x7fc43c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8e130 0x55e19cb8e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb8f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19cae5990 con 0x55e19cb8efd0 2024-03-30T02:36:53.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19cae3350 con 0x55e19cb8e130 2024-03-30T02:36:53.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.135+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19cadec40 con 0x55e19cb8d7d0 2024-03-30T02:36:53.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc457e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc457400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8e130 0x55e19cb8e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc45d000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb8f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc457e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40578/0 (socket says 172.21.15.67:40578) 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc457e00700 1 -- 172.21.15.67:0/2293373362 learned_addr learned my addr 172.21.15.67:0/2293373362 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc457400700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc43c0a11b0 0x7fc43c0a3670 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc458074b10 tx=0x7fc444011440 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc45d000700 1 -- 172.21.15.67:0/2293373362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 msgr2=0x55e19cb8dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc45d000700 1 --2- 172.21.15.67:0/2293373362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc45d000700 1 -- 172.21.15.67:0/2293373362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8e130 msgr2=0x55e19cb8e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc45d000700 1 --2- 172.21.15.67:0/2293373362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8e130 0x55e19cb8e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc45d000700 1 -- 172.21.15.67:0/2293373362 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e19c9603b0 con 0x55e19cb8efd0 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc457400700 1 --2- 172.21.15.67:0/2293373362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8e130 0x55e19cb8e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.136+0000 7fc457e00700 1 --2- 172.21.15.67:0/2293373362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.137+0000 7fc45d000700 1 --2- 172.21.15.67:0/2293373362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb8f450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c9c60 tx=0x7fc450004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319bc72a8df307f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.137+0000 7fc44d600700 1 -- 172.21.15.67:0/2293373362 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450098f00 con 0x55e19cb8efd0 2024-03-30T02:36:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.137+0000 7fc44d600700 1 -- 172.21.15.67:0/2293373362 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc450099ca0 con 0x55e19cb8efd0 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.137+0000 7fc44d600700 1 -- 172.21.15.67:0/2293373362 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450097940 con 0x55e19cb8efd0 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.137+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2293373362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 msgr2=0x55e19cb8f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.138+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/2293373362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb8f450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c9c60 tx=0x7fc450004f10 comp rx=0 tx=0).stop 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.138+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2293373362 shutdown_connections 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.138+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/2293373362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb8f450 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.138+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/2293373362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.138+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/2293373362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8e130 0x55e19cb8e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.138+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2293373362 >> 172.21.15.67:0/2293373362 conn(0x55e19caec6a0 msgr2=0x55e19cb889a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.138+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2293373362 shutdown_connections 2024-03-30T02:36:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.138+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2293373362 wait complete. 2024-03-30T02:36:53.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.138+0000 7fc472bb9b80 1 -- start start 2024-03-30T02:36:53.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.139+0000 7fc472bb9b80 1 Processor -- start 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.139+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb97710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.139+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8e130 0x55e19cb97c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.139+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.139+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19ca7ad80 con 0x55e19cb8e130 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.139+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19ca7aa20 con 0x55e19cb8efd0 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.139+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19c9626a0 con 0x55e19cb8d7d0 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.139+0000 7fc472bb9b80 10 client.0 _ll_register_callbacks cb 0x55e19ca84470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc457400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8e130 0x55e19cb97c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc457400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8e130 0x55e19cb97c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56910/0 (socket says 172.21.15.67:56910) 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc457400700 1 -- 172.21.15.67:0/1136109178 learned_addr learned my addr 172.21.15.67:0/1136109178 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:53.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc457e00700 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb97710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc45d000700 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc457400700 1 -- 172.21.15.67:0/1136109178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 msgr2=0x55e19cb97710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc457400700 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb97710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc457400700 1 -- 172.21.15.67:0/1136109178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 msgr2=0x55e19cb9ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc457400700 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc457400700 1 -- 172.21.15.67:0/1136109178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc444011b60 con 0x55e19cb8e130 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.140+0000 7fc45d000700 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.141+0000 7fc457400700 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8e130 0x55e19cb97c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc44400fb40 tx=0x7fc4440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1164c7d29962cdd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.141+0000 7fc442000700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.141+0000 7fc442000700 1 -- 172.21.15.67:0/1136109178 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44401ec70 con 0x55e19cb8e130 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.141+0000 7fc442000700 1 -- 172.21.15.67:0/1136109178 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fc44401edd0 con 0x55e19cb8e130 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.141+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1136109178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e19ca9f380 con 0x55e19cb8e130 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.141+0000 7fc442000700 1 -- 172.21.15.67:0/1136109178 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4440263f0 con 0x55e19cb8e130 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.141+0000 7fc472bb9b80 10 client.15567 fetch_fsmap learned FSMap version 29 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.141+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1136109178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fc4581a5930 con 0x55e19cb8e130 2024-03-30T02:36:53.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.141+0000 7fc442000700 1 -- 172.21.15.67:0/1136109178 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e19ca9f380 con 0x55e19cb8e130 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.142+0000 7fc442000700 1 -- 172.21.15.67:0/1136109178 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc44400e650 con 0x55e19cb8e130 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.142+0000 7fc472bb9b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.142+0000 7fc472bb9b80 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.142+0000 7fc472bb9b80 20 client.15567 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.142+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e19cba3580 0x55e19cba59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.142+0000 7fc472bb9b80 4 client.15567 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.142+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1136109178 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e19cad7700 con 0x55e19cba3580 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.142+0000 7fc457e00700 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e19cba3580 0x55e19cba59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.143+0000 7fc457e00700 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e19cba3580 0x55e19cba59a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7bf843e9e2ee61 server_cookie=e6b122ec54169afc in_seq=0 out_seq=0 2024-03-30T02:36:53.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.143+0000 7fc442000700 10 client.15567 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:53.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.143+0000 7fc442000700 1 -- 172.21.15.67:0/1136109178 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e19cad7700 con 0x55e19cba3580 2024-03-30T02:36:53.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.143+0000 7fc442000700 10 client.15567 handle_command_reply: tid=0 2024-03-30T02:36:53.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.143+0000 7fc472bb9b80 1 client.15567 shutdown 2024-03-30T02:36:53.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1136109178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8e130 msgr2=0x55e19cb97c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8e130 0x55e19cb97c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc44400fb40 tx=0x7fc4440047c0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1136109178 shutdown_connections 2024-03-30T02:36:53.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb8d7d0 0x55e19cb97710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e19cba3580 0x55e19cba59a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1136109178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8e130 0x55e19cb97c50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1136109178 >> 172.21.15.67:0/1136109178 conn(0x55e19caec6a0 msgr2=0x55e19cb8c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc442000700 0 client.15567 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1136109178 shutdown_connections 2024-03-30T02:36:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1136109178 wait complete. 2024-03-30T02:36:53.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.144+0000 7fc472bb9b80 20 client.15567 trim_cache size 0 max 16384 2024-03-30T02:36:53.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.155+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fcd9c10e730 con 0x7fcd9c074df0 2024-03-30T02:36:53.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.162+0000 7fc472bb9b80 1 Processor -- start 2024-03-30T02:36:53.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.162+0000 7fc472bb9b80 1 -- start start 2024-03-30T02:36:53.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.162+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.162+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb96c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.162+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 0x55e19cb9f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.162+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19cb8ee50 con 0x55e19cb8efd0 2024-03-30T02:36:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.162+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19cb97280 con 0x55e19cb8d7d0 2024-03-30T02:36:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.162+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19cb98390 con 0x55e19cb9ec00 2024-03-30T02:36:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.162+0000 7fc457400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb96c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc45d000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 0x55e19cb9f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc457400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb96c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56924/0 (socket says 172.21.15.67:56924) 2024-03-30T02:36:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc457400700 1 -- 172.21.15.67:0/2629826496 learned_addr learned my addr 172.21.15.67:0/2629826496 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:53.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc457e00700 1 --2- 172.21.15.67:0/2629826496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc45d000700 1 -- 172.21.15.67:0/2629826496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8d7d0 msgr2=0x55e19cb8f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc45d000700 1 --2- 172.21.15.67:0/2629826496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc45d000700 1 -- 172.21.15.67:0/2629826496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 msgr2=0x55e19cb96c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc45d000700 1 --2- 172.21.15.67:0/2629826496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb96c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc45d000700 1 -- 172.21.15.67:0/2629826496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e19c90cb60 con 0x55e19cb9ec00 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc45d000700 1 --2- 172.21.15.67:0/2629826496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 0x55e19cb9f2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55e19cb95ce0 tx=0x7fc4500b9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c388e18190ffb2f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc44e000700 1 -- 172.21.15.67:0/2629826496 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450098f00 con 0x55e19cb9ec00 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc44e000700 1 -- 172.21.15.67:0/2629826496 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc45009d3d0 con 0x55e19cb9ec00 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.163+0000 7fc44e000700 1 -- 172.21.15.67:0/2629826496 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45009d1b0 con 0x55e19cb9ec00 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.164+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2629826496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 msgr2=0x55e19cb9f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.164+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/2629826496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 0x55e19cb9f2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55e19cb95ce0 tx=0x7fc4500b9040 comp rx=0 tx=0).stop 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.164+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2629826496 shutdown_connections 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.164+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/2629826496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 0x55e19cb9f2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.164+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/2629826496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8d7d0 0x55e19cb8f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.164+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/2629826496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8efd0 0x55e19cb96c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.164+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2629826496 >> 172.21.15.67:0/2629826496 conn(0x55e19caec6a0 msgr2=0x55e19cb95560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.164+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2629826496 shutdown_connections 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.164+0000 7fc472bb9b80 1 -- 172.21.15.67:0/2629826496 wait complete. 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.165+0000 7fc472bb9b80 1 -- start start 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.165+0000 7fc472bb9b80 1 Processor -- start 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.165+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8d7d0 0x55e19cb9de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.165+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.165+0000 7fc472bb9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 0x55e19cb9c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.165+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19cb9c9e0 con 0x55e19cb8d7d0 2024-03-30T02:36:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.165+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19cb9cb50 con 0x55e19cb8efd0 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.165+0000 7fc472bb9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19cb9cd10 con 0x55e19cb9ec00 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.165+0000 7fc472bb9b80 10 client.0 _ll_register_callbacks cb 0x55e19ca84470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8d7d0 0x55e19cb9de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:45384/0 (socket says 172.21.15.67:45384) 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457400700 1 -- 172.21.15.67:0/1155084225 learned_addr learned my addr 172.21.15.67:0/1155084225 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc45d000700 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 0x55e19cb9c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457400700 1 -- 172.21.15.67:0/1155084225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 msgr2=0x55e19cb9c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457400700 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 0x55e19cb9c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457400700 1 -- 172.21.15.67:0/1155084225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8d7d0 msgr2=0x55e19cb9de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457400700 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8d7d0 0x55e19cb9de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457400700 1 -- 172.21.15.67:0/1155084225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e19c9603b0 con 0x55e19cb8efd0 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457e00700 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8d7d0 0x55e19cb9de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:53.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc457400700 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9e3d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc44400fb10 tx=0x7fc44401ed20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d45389861fad272 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc442000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc442000700 1 -- 172.21.15.67:0/1155084225 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44400e7c0 con 0x55e19cb8efd0 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1155084225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc450099ca0 con 0x55e19cb8efd0 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.166+0000 7fc442000700 1 -- 172.21.15.67:0/1155084225 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7fc444015030 con 0x55e19cb8efd0 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc442000700 1 -- 172.21.15.67:0/1155084225 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc444005b40 con 0x55e19cb8efd0 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc472bb9b80 10 client.25619 fetch_fsmap learned FSMap version 29 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1155084225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e19caa4ea0 con 0x55e19cb8efd0 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc442000700 1 -- 172.21.15.67:0/1155084225 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc450099ca0 con 0x55e19cb8efd0 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc442000700 1 -- 172.21.15.67:0/1155084225 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc44400fb70 con 0x55e19cb8efd0 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc472bb9b80 10 client.25619 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc472bb9b80 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc472bb9b80 20 client.25619 populate_metadata read hostname 'smithi067' 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e19cbf8570 0x55e19cbfa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc472bb9b80 4 client.25619 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.167+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1155084225 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e19c92dcf0 con 0x55e19cbf8570 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.168+0000 7fc457e00700 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e19cbf8570 0x55e19cbfa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.168+0000 7fc457e00700 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e19cbf8570 0x55e19cbfa990 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b1a7d5b6402397 server_cookie=96942c41597e3f6 in_seq=0 out_seq=0 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.168+0000 7fc442000700 10 client.25619 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.168+0000 7fc442000700 1 -- 172.21.15.67:0/1155084225 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e19c92dcf0 con 0x55e19cbf8570 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.168+0000 7fc442000700 10 client.25619 handle_command_reply: tid=0 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.168+0000 7fc472bb9b80 1 client.25619 shutdown 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.168+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1155084225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 msgr2=0x55e19cb9e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.168+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9e3d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc44400fb10 tx=0x7fc44401ed20 comp rx=0 tx=0).stop 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1155084225 shutdown_connections 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55e19cbf8570 0x55e19cbfa990 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e19cb8d7d0 0x55e19cb9de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e19cb9ec00 0x55e19cb9c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc442000700 0 client.25619 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc472bb9b80 1 --2- 172.21.15.67:0/1155084225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e19cb8efd0 0x55e19cb9e3d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1155084225 >> 172.21.15.67:0/1155084225 conn(0x55e19caec6a0 msgr2=0x55e19cb954f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1155084225 shutdown_connections 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc472bb9b80 1 -- 172.21.15.67:0/1155084225 wait complete. 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc472bb9b80 20 client.25619 trim_cache size 0 max 16384 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stdout: "32e3e246-de95-4cff-ad83-da993b27beca": { 2024-03-30T02:36:53.175 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:36:53.176 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "32e3e246-de95-4cff-ad83-da993b27beca", 2024-03-30T02:36:53.176 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:36:53.176 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:36:53.176 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:36:53.176 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:36:53.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc45f000700 1 -- 172.21.15.67:0/2765547477 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc43c0a11b0 msgr2=0x7fc43c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc45f000700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc43c0a11b0 0x7fc43c0a3670 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc458074b10 tx=0x7fc444011440 comp rx=0 tx=0).stop 2024-03-30T02:36:53.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc45f000700 1 -- 172.21.15.67:0/2765547477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc45810d350 msgr2=0x7fc4581a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.169+0000 7fc45f000700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc45810d350 0x7fc4581a4850 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc450007f10 tx=0x7fc45000d3b0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.170+0000 7fc45f000700 1 -- 172.21.15.67:0/2765547477 shutdown_connections 2024-03-30T02:36:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.170+0000 7fc45f000700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fc43c0a11b0 0x7fc43c0a3670 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.170+0000 7fc45f000700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc458073810 0x7fc45819d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.170+0000 7fc45f000700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc45810d350 0x7fc4581a4850 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.170+0000 7fc45f000700 1 --2- 172.21.15.67:0/2765547477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc458074df0 0x7fc45819d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.170+0000 7fc45f000700 1 -- 172.21.15.67:0/2765547477 >> 172.21.15.67:0/2765547477 conn(0x7fc4580fc2f0 msgr2=0x7fc4581006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.170+0000 7fc45f000700 1 -- 172.21.15.67:0/2765547477 shutdown_connections 2024-03-30T02:36:53.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.170+0000 7fc45f000700 1 -- 172.21.15.67:0/2765547477 wait complete. 2024-03-30T02:36:53.188 INFO:tasks.cephfs.filesystem:scrub status for tag:32e3e246-de95-4cff-ad83-da993b27beca - {'path': '/', 'tag': '32e3e246-de95-4cff-ad83-da993b27beca', 'options': 'recursive,force'} 2024-03-30T02:36:53.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.259+0000 7fcd99600700 1 -- 172.21.15.67:0/54029565 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 20) v1 ==== 135933+0+0 (secure 0 0 0) 0x7fcd88096550 con 0x7fcd9c074df0 2024-03-30T02:36:53.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.260+0000 7fcd99600700 1 -- 172.21.15.67:0/54029565 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7fcd88099c10 con 0x7fcd9c074df0 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.263+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcd840a1220 msgr2=0x7fcd840a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.263+0000 7fcda3800700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcd840a1220 0x7fcd840a36e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000fa40 tx=0x7fcd90019040 comp rx=0 tx=0).stop 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.263+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd9c074df0 msgr2=0x7fcd9c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.263+0000 7fcda3800700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd9c074df0 0x7fcd9c199370 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800e9f0 tx=0x7fcd8800ed00 comp rx=0 tx=0).stop 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.264+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 shutdown_connections 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.264+0000 7fcda3800700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7fcd840a1220 0x7fcd840a36e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.264+0000 7fcda3800700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd9c108af0 0x7fcd9c1a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.264+0000 7fcda3800700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd9c073810 0x7fcd9c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.264+0000 7fcda3800700 1 --2- 172.21.15.67:0/54029565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd9c074df0 0x7fcd9c199370 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.264+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 >> 172.21.15.67:0/54029565 conn(0x7fcd9c0fc1c0 msgr2=0x7fcd9c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.264+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 shutdown_connections 2024-03-30T02:36:53.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.264+0000 7fcda3800700 1 -- 172.21.15.67:0/54029565 wait complete. 2024-03-30T02:36:53.284 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-30T02:36:53.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.490+0000 7f1ab0200700 1 -- 172.21.15.67:0/1456201637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8074d70 msgr2=0x7f1aa80731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.490+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1456201637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8074d70 0x7f1aa80731c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0098f0 tx=0x7f1a9c009c00 comp rx=0 tx=0).stop 2024-03-30T02:36:53.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.490+0000 7f1ab0200700 1 -- 172.21.15.67:0/1456201637 shutdown_connections 2024-03-30T02:36:53.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.490+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1456201637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa8108c00 0x7f1aa810b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.490+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1456201637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8074d70 0x7f1aa80731c0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0098f0 tx=0x7f1a9c009c00 comp rx=0 tx=0).stop 2024-03-30T02:36:53.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.490+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1456201637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8073790 0x7f1aa8073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.490+0000 7f1ab0200700 1 -- 172.21.15.67:0/1456201637 >> 172.21.15.67:0/1456201637 conn(0x7f1aa80fc2a0 msgr2=0x7f1aa80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.490+0000 7f1ab0200700 1 -- 172.21.15.67:0/1456201637 shutdown_connections 2024-03-30T02:36:53.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.490+0000 7f1ab0200700 1 -- 172.21.15.67:0/1456201637 wait complete. 2024-03-30T02:36:53.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.491+0000 7f1ab0200700 1 Processor -- start 2024-03-30T02:36:53.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.491+0000 7f1ab0200700 1 -- start start 2024-03-30T02:36:53.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1ab0200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8073790 0x7f1aa81990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1ab0200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8108c00 0x7f1aa8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1ab0200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa819e630 0x7f1aa81a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1ab0200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa81a0ff0 con 0x7f1aa819e630 2024-03-30T02:36:53.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1ab0200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa81a1130 con 0x7f1aa8108c00 2024-03-30T02:36:53.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1ab0200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa81a12c0 con 0x7f1aa8073790 2024-03-30T02:36:53.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aace00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8108c00 0x7f1aa8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aace00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8108c00 0x7f1aa8199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:45412/0 (socket says 172.21.15.67:45412) 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aace00700 1 -- 172.21.15.67:0/1420059539 learned_addr learned my addr 172.21.15.67:0/1420059539 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aad800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8073790 0x7f1aa81990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aae200700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa819e630 0x7f1aa81a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aace00700 1 -- 172.21.15.67:0/1420059539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8073790 msgr2=0x7f1aa81990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aace00700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8073790 0x7f1aa81990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aace00700 1 -- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa819e630 msgr2=0x7f1aa81a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aace00700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa819e630 0x7f1aa81a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aace00700 1 -- 172.21.15.67:0/1420059539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa0009d00 con 0x7f1aa8108c00 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.492+0000 7f1aad800700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8073790 0x7f1aa81990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.493+0000 7f1aace00700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8108c00 0x7f1aa8199620 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e9a0 tx=0x7f1a9400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.493+0000 7f1aa6000700 1 -- 172.21.15.67:0/1420059539 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94009790 con 0x7f1aa8108c00 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.493+0000 7f1aa6000700 1 -- 172.21.15.67:0/1420059539 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a94010430 con 0x7f1aa8108c00 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.493+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c009520 con 0x7f1aa8108c00 2024-03-30T02:36:53.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.493+0000 7f1aa6000700 1 -- 172.21.15.67:0/1420059539 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a94018680 con 0x7f1aa8108c00 2024-03-30T02:36:53.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.493+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa81a1970 con 0x7f1aa8108c00 2024-03-30T02:36:53.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.494+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa8107a10 con 0x7f1aa8108c00 2024-03-30T02:36:53.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.496+0000 7f1aa6000700 1 -- 172.21.15.67:0/1420059539 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 20) v1 ==== 135933+0+0 (secure 0 0 0) 0x7f1a940105a0 con 0x7f1aa8108c00 2024-03-30T02:36:53.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.497+0000 7f1aa6000700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1a900a1280 0x7f1a900a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.497+0000 7f1aad800700 1 -- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1a900a1280 msgr2=0x7f1a900a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:36:53.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.497+0000 7f1aa6000700 1 -- 172.21.15.67:0/1420059539 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f1a94014070 con 0x7f1aa8108c00 2024-03-30T02:36:53.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.498+0000 7f1aad800700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1a900a1280 0x7f1a900a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:36:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.501+0000 7f1aa6000700 1 -- 172.21.15.67:0/1420059539 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1a940529f0 con 0x7f1aa8108c00 2024-03-30T02:36:53.665 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: ignoring --setuser ceph since I am not root 2024-03-30T02:36:53.665 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: ignoring --setgroup ceph since I am not root 2024-03-30T02:36:53.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:53 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/538814977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:53.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:53 smithi067 ceph-mon[37808]: pgmap v300: 129 pgs: 129 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-03-30T02:36:53.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:53 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/54029565' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-30T02:36:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/538814977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-mon[78798]: pgmap v300: 129 pgs: 129 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-03-30T02:36:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/54029565' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-30T02:36:53.689 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: ignoring --setuser ceph since I am not root 2024-03-30T02:36:53.689 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: ignoring --setgroup ceph since I am not root 2024-03-30T02:36:53.689 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-mgr[80689]: -- 172.21.15.170:0/649019728 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d892641a0 con 0x555d8923a800 2024-03-30T02:36:53.690 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/538814977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:36:53.690 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-mon[79176]: pgmap v300: 129 pgs: 129 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-03-30T02:36:53.690 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/54029565' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-30T02:36:53.690 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: ignoring --setuser ceph since I am not root 2024-03-30T02:36:53.690 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: ignoring --setgroup ceph since I am not root 2024-03-30T02:36:53.691 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-mgr[80680]: -- 172.21.15.82:0/3625871506 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56071156c1a0 con 0x560711542800 2024-03-30T02:36:53.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.699+0000 7f1aad800700 1 -- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1a900a1280 msgr2=0x7f1a900a3740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:36:53.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.699+0000 7f1aad800700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1a900a1280 0x7f1a900a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T02:36:53.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.759+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1aa80673a0 con 0x7f1aa8108c00 2024-03-30T02:36:53.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.766+0000 7f1aa6000700 1 -- 172.21.15.67:0/1420059539 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1a94099cc0 con 0x7f1aa8108c00 2024-03-30T02:36:53.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1a900a1280 msgr2=0x7f1a900a3740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1a900a1280 0x7f1a900a3740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8108c00 msgr2=0x7f1aa8199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8108c00 0x7f1aa8199620 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400e9a0 tx=0x7f1a9400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 shutdown_connections 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f1a900a1280 0x7f1a900a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1aa819e630 0x7f1aa81a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1aa8073790 0x7f1aa81990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 --2- 172.21.15.67:0/1420059539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1aa8108c00 0x7f1aa8199620 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.768+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 >> 172.21.15.67:0/1420059539 conn(0x7f1aa80fc2a0 msgr2=0x7f1aa8104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.769+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 shutdown_connections 2024-03-30T02:36:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.769+0000 7f1ab0200700 1 -- 172.21.15.67:0/1420059539 wait complete. 2024-03-30T02:36:53.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-30T02:36:53.953 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:53.689+0000 7f609fc08200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T02:36:53.953 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:53.782+0000 7f609fc08200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T02:36:53.953 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:53.868+0000 7f609fc08200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T02:36:53.957 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:53.693+0000 7f507c7c8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T02:36:53.957 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:53.787+0000 7f507c7c8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T02:36:53.957 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:53.873+0000 7f507c7c8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T02:36:53.957 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:53.956+0000 7f507c7c8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T02:36:53.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.986+0000 7f6ad4600700 1 -- 172.21.15.67:0/1684678885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6acc106a10 msgr2=0x7f6acc108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.986+0000 7f6ad4600700 1 --2- 172.21.15.67:0/1684678885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6acc106a10 0x7f6acc108e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009f90 tx=0x7f6ac400b330 comp rx=0 tx=0).stop 2024-03-30T02:36:53.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.986+0000 7f6ad4600700 1 -- 172.21.15.67:0/1684678885 shutdown_connections 2024-03-30T02:36:53.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.986+0000 7f6ad4600700 1 --2- 172.21.15.67:0/1684678885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6acc106a10 0x7f6acc108e80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.986+0000 7f6ad4600700 1 --2- 172.21.15.67:0/1684678885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6acc1017b0 0x7f6acc103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.986+0000 7f6ad4600700 1 --2- 172.21.15.67:0/1684678885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6acc1040e0 0x7f6acc1064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.986+0000 7f6ad4600700 1 -- 172.21.15.67:0/1684678885 >> 172.21.15.67:0/1684678885 conn(0x7f6acc0fb1c0 msgr2=0x7f6acc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:53.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.986+0000 7f6ad4600700 1 -- 172.21.15.67:0/1684678885 shutdown_connections 2024-03-30T02:36:53.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.986+0000 7f6ad4600700 1 -- 172.21.15.67:0/1684678885 wait complete. 2024-03-30T02:36:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.987+0000 7f6ad4600700 1 Processor -- start 2024-03-30T02:36:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.987+0000 7f6ad4600700 1 -- start start 2024-03-30T02:36:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.987+0000 7f6ad4600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6acc1017b0 0x7f6acc1010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad4600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6acc1040e0 0x7f6acc0ff760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad4600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6acc106a10 0x7f6acc0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad4600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc100440 con 0x7f6acc106a10 2024-03-30T02:36:53.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad4600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc1005b0 con 0x7f6acc1017b0 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad4600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc1a0da0 con 0x7f6acc1040e0 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6acc1040e0 0x7f6acc0ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6acc1017b0 0x7f6acc1010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6acc1040e0 0x7f6acc0ff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40654/0 (socket says 172.21.15.67:40654) 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6acc1017b0 0x7f6acc1010d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:45422/0 (socket says 172.21.15.67:45422) 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1200700 1 -- 172.21.15.67:0/2398996550 learned_addr learned my addr 172.21.15.67:0/2398996550 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad2600700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6acc106a10 0x7f6acc0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1200700 1 -- 172.21.15.67:0/2398996550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6acc1017b0 msgr2=0x7f6acc1010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1200700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6acc1017b0 0x7f6acc1010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1200700 1 -- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6acc106a10 msgr2=0x7f6acc0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1200700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6acc106a10 0x7f6acc0ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.988+0000 7f6ad1200700 1 -- 172.21.15.67:0/2398996550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac0009520 con 0x7f6acc1040e0 2024-03-30T02:36:53.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.989+0000 7f6ad1200700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6acc1040e0 0x7f6acc0ff760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800e9c0 tx=0x7f6ab800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:53.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.989+0000 7f6aca000700 1 -- 172.21.15.67:0/2398996550 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8004a30 con 0x7f6acc1040e0 2024-03-30T02:36:53.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.989+0000 7f6aca000700 1 -- 172.21.15.67:0/2398996550 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ab800cc80 con 0x7f6acc1040e0 2024-03-30T02:36:53.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.989+0000 7f6aca000700 1 -- 172.21.15.67:0/2398996550 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80054e0 con 0x7f6acc1040e0 2024-03-30T02:36:53.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.989+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac4009c40 con 0x7f6acc1040e0 2024-03-30T02:36:53.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.989+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc1a1270 con 0x7f6acc1040e0 2024-03-30T02:36:53.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.990+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc0fcd40 con 0x7f6acc1040e0 2024-03-30T02:36:53.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.992+0000 7f6aca000700 1 -- 172.21.15.67:0/2398996550 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 20) v1 ==== 135933+0+0 (secure 0 0 0) 0x7f6ab8004b90 con 0x7f6acc1040e0 2024-03-30T02:36:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.993+0000 7f6aca000700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6ab40a1200 0x7f6ab40a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.993+0000 7f6ad1c00700 1 -- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6ab40a1200 msgr2=0x7f6ab40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:36:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.993+0000 7f6aca000700 1 -- 172.21.15.67:0/2398996550 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f6ab8014070 con 0x7f6acc1040e0 2024-03-30T02:36:53.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.993+0000 7f6ad1c00700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6ab40a1200 0x7f6ab40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:36:53.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:53.997+0000 7f6aca000700 1 -- 172.21.15.67:0/2398996550 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f6ab8096290 con 0x7f6acc1040e0 2024-03-30T02:36:54.148 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:53.897+0000 7f6c2e0a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T02:36:54.148 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:54.028+0000 7f6c2e0a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T02:36:54.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.194+0000 7f6ad1c00700 1 -- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6ab40a1200 msgr2=0x7f6ab40a36c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:36:54.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.194+0000 7f6ad1c00700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6ab40a1200 0x7f6ab40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T02:36:54.249 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:53.952+0000 7f609fc08200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T02:36:54.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.254+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f6acc0673a0 con 0x7f6acc1040e0 2024-03-30T02:36:54.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.260+0000 7f6aca000700 1 -- 172.21.15.67:0/2398996550 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6ab80998b0 con 0x7f6acc1040e0 2024-03-30T02:36:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6ab40a1200 msgr2=0x7f6ab40a36c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:36:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6ab40a1200 0x7f6ab40a36c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6acc1040e0 msgr2=0x7f6acc0ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6acc1040e0 0x7f6acc0ff760 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800e9c0 tx=0x7f6ab800ecd0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 shutdown_connections 2024-03-30T02:36:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f6ab40a1200 0x7f6ab40a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6acc106a10 0x7f6acc0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6acc1017b0 0x7f6acc1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 --2- 172.21.15.67:0/2398996550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6acc1040e0 0x7f6acc0ff760 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 >> 172.21.15.67:0/2398996550 conn(0x7f6acc0fb1c0 msgr2=0x7f6acc1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 shutdown_connections 2024-03-30T02:36:54.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.263+0000 7f6ad4600700 1 -- 172.21.15.67:0/2398996550 wait complete. 2024-03-30T02:36:54.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-30T02:36:54.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:54.146+0000 7f6c2e0a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T02:36:54.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:54.270+0000 7f6c2e0a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T02:36:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:36:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:36:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:36:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:36:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/54029565' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-30T02:36:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-mon[37808]: mgrmap e20: x(active, since 7m), standbys: z, y 2024-03-30T02:36:54.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:54 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' 2024-03-30T02:36:54.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.481+0000 7f09efa00700 1 -- 172.21.15.67:0/3499137003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09e8101df0 msgr2=0x7f09e8104280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:54.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.481+0000 7f09efa00700 1 --2- 172.21.15.67:0/3499137003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09e8101df0 0x7f09e8104280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009990 tx=0x7f09d4009ca0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.481+0000 7f09efa00700 1 -- 172.21.15.67:0/3499137003 shutdown_connections 2024-03-30T02:36:54.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.481+0000 7f09efa00700 1 --2- 172.21.15.67:0/3499137003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09e81047c0 0x7f09e8104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.481+0000 7f09efa00700 1 --2- 172.21.15.67:0/3499137003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e80ff480 0x7f09e81018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.481+0000 7f09efa00700 1 --2- 172.21.15.67:0/3499137003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09e8101df0 0x7f09e8104280 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.481+0000 7f09efa00700 1 -- 172.21.15.67:0/3499137003 >> 172.21.15.67:0/3499137003 conn(0x7f09e80fb0a0 msgr2=0x7f09e80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:36:54.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.481+0000 7f09efa00700 1 -- 172.21.15.67:0/3499137003 shutdown_connections 2024-03-30T02:36:54.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.481+0000 7f09efa00700 1 -- 172.21.15.67:0/3499137003 wait complete. 2024-03-30T02:36:54.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.482+0000 7f09efa00700 1 Processor -- start 2024-03-30T02:36:54.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09efa00700 1 -- start start 2024-03-30T02:36:54.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09efa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09e80ff480 0x7f09e819d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:54.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09efa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e8101df0 0x7f09e819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09efa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09e81047c0 0x7f09e81a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09efa00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e81a4ce0 con 0x7f09e80ff480 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09efa00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e81a4e20 con 0x7f09e81047c0 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09efa00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e81a4fb0 con 0x7f09e8101df0 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09ed000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09e80ff480 0x7f09e819d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09e7e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e8101df0 0x7f09e819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09eda00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09e81047c0 0x7f09e81a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09ed000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09e80ff480 0x7f09e819d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56992/0 (socket says 172.21.15.67:56992) 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.483+0000 7f09ed000700 1 -- 172.21.15.67:0/3197848146 learned_addr learned my addr 172.21.15.67:0/3197848146 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09ed000700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e8101df0 msgr2=0x7f09e819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09ed000700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e8101df0 0x7f09e819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09ed000700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09e81047c0 msgr2=0x7f09e81a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09ed000700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09e81047c0 0x7f09e81a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09ed000700 1 -- 172.21.15.67:0/3197848146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc009520 con 0x7f09e80ff480 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09eda00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09e81047c0 0x7f09e81a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09e7e00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e8101df0 0x7f09e819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09ed000700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09e80ff480 0x7f09e819d200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00eb30 tx=0x7f09dc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc009880 con 0x7f09e80ff480 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09dc0044e0 con 0x7f09e80ff480 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d40095f0 con 0x7f09e80ff480 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.484+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc003d80 con 0x7f09e80ff480 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.485+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e81a5610 con 0x7f09e80ff480 2024-03-30T02:36:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.486+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e80fcc20 con 0x7f09e80ff480 2024-03-30T02:36:54.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.487+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 135933+0+0 (secure 0 0 0) 0x7f09dc003f20 con 0x7f09e80ff480 2024-03-30T02:36:54.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.489+0000 7f09e5600700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 0x7f09d00a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:36:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.489+0000 7f09e7e00700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 msgr2=0x7f09d00a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:36:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.489+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9065+0+0 (secure 0 0 0) 0x7f09dc014070 con 0x7f09e80ff480 2024-03-30T02:36:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.489+0000 7f09e7e00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 0x7f09d00a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T02:36:54.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.493+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f09dc096670 con 0x7f09e80ff480 2024-03-30T02:36:54.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:36:54.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:36:54.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:36:54.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:36:54.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/54029565' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-30T02:36:54.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-mon[79176]: mgrmap e20: x(active, since 7m), standbys: z, y 2024-03-30T02:36:54.579 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' 2024-03-30T02:36:54.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:36:54.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:36:54.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:36:54.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:36:54.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/54029565' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-30T02:36:54.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-mon[78798]: mgrmap e20: x(active, since 7m), standbys: z, y 2024-03-30T02:36:54.587 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' 2024-03-30T02:36:54.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.690+0000 7f09e7e00700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 msgr2=0x7f09d00a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:36:54.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.690+0000 7f09e7e00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 0x7f09d00a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T02:36:54.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:54.761+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 --> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f09e80611f0 con 0x7f09d00a11f0 2024-03-30T02:36:54.831 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:54.578+0000 7f609fc08200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:36:54.842 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:54.585+0000 7f507c7c8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:36:55.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:55.091+0000 7f09e7e00700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 msgr2=0x7f09d00a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:36:55.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:55.091+0000 7f09e7e00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 0x7f09d00a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-30T02:36:55.180 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:54.830+0000 7f609fc08200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T02:36:55.180 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:54.915+0000 7f609fc08200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T02:36:55.180 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:55.022+0000 7f609fc08200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T02:36:55.189 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:54.841+0000 7f507c7c8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T02:36:55.189 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:54.926+0000 7f507c7c8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T02:36:55.189 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:55.033+0000 7f507c7c8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T02:36:55.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:55 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' 2024-03-30T02:36:55.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:55.161+0000 7f6c2e0a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:36:55.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:55 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' 2024-03-30T02:36:55.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:55.178+0000 7f609fc08200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T02:36:55.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:55.306+0000 7f609fc08200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T02:36:55.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:55.391+0000 7f609fc08200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T02:36:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:55 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' 2024-03-30T02:36:55.689 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:55.193+0000 7f507c7c8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T02:36:55.690 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:55.320+0000 7f507c7c8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T02:36:55.690 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:55.405+0000 7f507c7c8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T02:36:55.751 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:55.488+0000 7f6c2e0a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T02:36:55.751 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:55.604+0000 7f6c2e0a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T02:36:55.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:55.892+0000 7f09e7e00700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 msgr2=0x7f09d00a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:36:55.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:55.892+0000 7f09e7e00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 0x7f09d00a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-30T02:36:56.129 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:55.750+0000 7f6c2e0a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T02:36:56.129 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:55.953+0000 7f6c2e0a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T02:36:56.141 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:55.755+0000 7f609fc08200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T02:36:56.163 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:55.774+0000 7f507c7c8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T02:36:56.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:56 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:56.128+0000 7f6c2e0a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T02:36:56.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:56 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:56.247+0000 7f6c2e0a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T02:36:56.429 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:56 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:56.140+0000 7f609fc08200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T02:36:56.429 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:56 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:56.334+0000 7f609fc08200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T02:36:56.439 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:56.161+0000 7f507c7c8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T02:36:56.439 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:56.360+0000 7f507c7c8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T02:36:56.749 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:56 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:56.428+0000 7f609fc08200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T02:36:56.939 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:56.454+0000 7f507c7c8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T02:36:57.163 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:56 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:56.733+0000 7f6c2e0a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T02:36:57.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:57.494+0000 7f09e7e00700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 msgr2=0x7f09d00a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:36:57.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:57.494+0000 7f09e7e00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 0x7f09d00a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-30T02:36:57.499 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:57 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:57.182+0000 7f609fc08200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T02:36:57.499 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:57 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:57.278+0000 7f609fc08200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T02:36:57.500 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:57 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:57.424+0000 7f609fc08200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T02:36:57.511 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:57.248+0000 7f6c2e0a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T02:36:57.689 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:57.216+0000 7f507c7c8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T02:36:57.689 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:57.313+0000 7f507c7c8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T02:36:57.689 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:57.460+0000 7f507c7c8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T02:36:57.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:57.510+0000 7f6c2e0a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-30T02:36:57.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:57.639+0000 7f6c2e0a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T02:36:58.417 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:58 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:58.043+0000 7f609fc08200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T02:36:58.417 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:58 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:58.223+0000 7f609fc08200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T02:36:58.439 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:58.084+0000 7f507c7c8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T02:36:58.439 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:58.264+0000 7f507c7c8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T02:36:58.749 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:58 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:58.416+0000 7f609fc08200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T02:36:58.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:58.640+0000 7f6c2e0a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T02:36:58.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:58.774+0000 7f6c2e0a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T02:36:58.939 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:58.457+0000 7f507c7c8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T02:36:59.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:58.978+0000 7f6c2e0a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T02:36:59.627 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:59.342+0000 7f609fc08200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T02:36:59.677 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:59.391+0000 7f507c7c8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T02:36:59.906 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:59.626+0000 7f609fc08200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:36:59.906 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:59.714+0000 7f609fc08200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T02:36:59.939 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:59.677+0000 7f507c7c8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:36:59.939 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:59.766+0000 7f507c7c8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T02:36:59.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:36:59.979+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 21) v1 ==== 135933+0+0 (secure 0 0 0) 0x7f09dc096a40 con 0x7f09e80ff480 2024-03-30T02:37:00.089 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:36:59 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:36:59.829+0000 7f6c2e0a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-30T02:37:00.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:59 smithi067 ceph-mon[37808]: Standby manager daemon y restarted 2024-03-30T02:37:00.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:59 smithi067 ceph-mon[37808]: Standby manager daemon y started 2024-03-30T02:37:00.090 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:59 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-30T02:37:00.090 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:59 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:37:00.090 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:59 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-30T02:37:00.090 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:36:59 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:37:00.250 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y[80658]: 2024-03-30T02:36:59.905+0000 7f609fc08200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T02:37:00.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-mon[79176]: Standby manager daemon y restarted 2024-03-30T02:37:00.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-mon[79176]: Standby manager daemon y started 2024-03-30T02:37:00.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-30T02:37:00.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:37:00.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-30T02:37:00.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:36:59 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:37:00.360 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:00.088+0000 7f6c2e0a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T02:37:00.439 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z[80666]: 2024-03-30T02:36:59.956+0000 7f507c7c8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T02:37:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-mon[78798]: Standby manager daemon y restarted 2024-03-30T02:37:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-mon[78798]: Standby manager daemon y started 2024-03-30T02:37:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-30T02:37:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:37:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-30T02:37:00.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:36:59 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.82:0/2662199195' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:37:00.663 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:00.360+0000 7f6c2e0a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T02:37:00.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:00.698+0000 7f09e7e00700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 msgr2=0x7f09d00a36b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/3801648476 2024-03-30T02:37:00.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:00.698+0000 7f09e7e00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 0x7f09d00a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-30T02:37:00.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:00.983+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mgrmap(e 22) v1 ==== 135933+0+0 (secure 0 0 0) 0x7f09dc099820 con 0x7f09e80ff480 2024-03-30T02:37:01.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:00 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-30T02:37:01.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:00 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:37:01.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:00 smithi082 ceph-mon[79176]: Standby manager daemon z restarted 2024-03-30T02:37:01.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:00 smithi082 ceph-mon[79176]: Standby manager daemon z started 2024-03-30T02:37:01.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:00 smithi082 ceph-mon[79176]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-03-30T02:37:01.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:00 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-30T02:37:01.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:00 smithi082 ceph-mon[79176]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:37:01.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:00 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-30T02:37:01.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:00 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:37:01.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:00 smithi067 ceph-mon[37808]: Standby manager daemon z restarted 2024-03-30T02:37:01.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:00 smithi067 ceph-mon[37808]: Standby manager daemon z started 2024-03-30T02:37:01.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:00 smithi067 ceph-mon[37808]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-03-30T02:37:01.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:00 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-30T02:37:01.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:00 smithi067 ceph-mon[37808]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:37:01.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:00 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-30T02:37:01.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:00 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-30T02:37:01.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:00 smithi170 ceph-mon[78798]: Standby manager daemon z restarted 2024-03-30T02:37:01.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:00 smithi170 ceph-mon[78798]: Standby manager daemon z started 2024-03-30T02:37:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:00 smithi170 ceph-mon[78798]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-03-30T02:37:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:00 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-30T02:37:01.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:00 smithi170 ceph-mon[78798]: from='mgr.? 172.21.15.170:0/3000325798' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-30T02:37:01.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:01.566+0000 7f6c2e0a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T02:37:02.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:01 smithi082 ceph-mon[79176]: mgrmap e22: x(active, since 7m), standbys: z, y 2024-03-30T02:37:02.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:01 smithi067 ceph-mon[37808]: mgrmap e22: x(active, since 7m), standbys: z, y 2024-03-30T02:37:02.360 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:01.963+0000 7f6c2e0a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T02:37:02.360 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.089+0000 7f6c2e0a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T02:37:02.360 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.359+0000 7f6c2e0a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T02:37:02.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:02.373+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 9 ==== mgrmap(e 23) v1 ==== 135550+0+0 (secure 0 0 0) 0x7f09dc0425c0 con 0x7f09e80ff480 2024-03-30T02:37:02.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:02.374+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 msgr2=0x7f09d00a36b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:37:02.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:02.374+0000 7f09e5600700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/3801648476,v1:172.21.15.67:6801/3801648476] conn(0x7f09d00a11f0 0x7f09d00a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:02.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:01 smithi170 ceph-mon[78798]: mgrmap e22: x(active, since 7m), standbys: z, y 2024-03-30T02:37:02.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.649+0000 7f6b7ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:02.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.649+0000 7f6b7ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:02.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.649+0000 7f6b7ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:02.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.649+0000 7f6b7ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:02.913 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.649+0000 7f6b7ee00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:02.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.651+0000 7f6b84800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:02.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.651+0000 7f6b84800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:02.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.651+0000 7f6b84800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:02.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.651+0000 7f6b84800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:02.914 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:37:02.651+0000 7f6b84800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:37:03.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: Active manager daemon x restarted 2024-03-30T02:37:03.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: Activating manager daemon x 2024-03-30T02:37:03.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: osdmap e88: 12 total, 12 up, 12 in 2024-03-30T02:37:03.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: mgrmap e23: x(active, starting, since 0.0124707s), standbys: z, y 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T02:37:03.251 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:37:03.252 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:37:03.253 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:37:03.253 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-30T02:37:03.253 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-30T02:37:03.253 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-30T02:37:03.253 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: Manager daemon x is now available 2024-03-30T02:37:03.253 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:37:03.253 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-30T02:37:03.253 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:02 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-30T02:37:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.380+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mon.0 v2:172.21.15.67:3300/0 10 ==== mgrmap(e 24) v1 ==== 135677+0+0 (secure 0 0 0) 0x7f09dc097310 con 0x7f09e80ff480 2024-03-30T02:37:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.381+0000 7f09e5600700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f09d004c720 0x7f09d004cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:03.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.381+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 --> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f09e80611f0 con 0x7f09d004c720 2024-03-30T02:37:03.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.388+0000 7f09e7e00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f09d004c720 0x7f09d004cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:03.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.394+0000 7f09e7e00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f09d004c720 0x7f09d004cb00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f09d400b2e0 tx=0x7f09d4005fb0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:03.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: Active manager daemon x restarted 2024-03-30T02:37:03.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: Activating manager daemon x 2024-03-30T02:37:03.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: osdmap e88: 12 total, 12 up, 12 in 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: mgrmap e23: x(active, starting, since 0.0124707s), standbys: z, y 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T02:37:03.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:37:03.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: Manager daemon x is now available 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-30T02:37:03.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:02 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: Active manager daemon x restarted 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: Activating manager daemon x 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: osdmap e88: 12 total, 12 up, 12 in 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: mgrmap e23: x(active, starting, since 0.0124707s), standbys: z, y 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-30T02:37:03.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-30T02:37:03.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: Manager daemon x is now available 2024-03-30T02:37:03.442 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:37:03.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-30T02:37:03.443 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:02 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-30T02:37:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.466+0000 7f09e5600700 1 -- 172.21.15.67:0/3197848146 <== mgr.25634 v2:172.21.15.67:6800/1710497985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f09e80611f0 con 0x7f09d004c720 2024-03-30T02:37:03.466 INFO:teuthology.orchestra.run.smithi067.stdout:Schedule set for path / 2024-03-30T02:37:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.468+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f09d004c720 msgr2=0x7f09d004cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.468+0000 7f09efa00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f09d004c720 0x7f09d004cb00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f09d400b2e0 tx=0x7f09d4005fb0 comp rx=0 tx=0).stop 2024-03-30T02:37:03.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.468+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09e80ff480 msgr2=0x7f09e819d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.468+0000 7f09efa00700 1 --2- 172.21.15.67:0/3197848146 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09e80ff480 0x7f09e819d200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00eb30 tx=0x7f09dc00ee40 comp rx=0 tx=0).stop 2024-03-30T02:37:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.468+0000 7f09eda00700 1 -- 172.21.15.67:0/3197848146 reap_dead start 2024-03-30T02:37:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.468+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 shutdown_connections 2024-03-30T02:37:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.468+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 >> 172.21.15.67:0/3197848146 conn(0x7f09e80fb0a0 msgr2=0x7f09e8100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.469+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 shutdown_connections 2024-03-30T02:37:03.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.469+0000 7f09efa00700 1 -- 172.21.15.67:0/3197848146 wait complete. 2024-03-30T02:37:03.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-30T02:37:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.708+0000 7f0326200700 1 -- 172.21.15.67:0/984246817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320101790 msgr2=0x7f0320108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.708+0000 7f0326200700 1 --2- 172.21.15.67:0/984246817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320101790 0x7f0320108040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f031800b0d0 tx=0x7f031800b3e0 comp rx=0 tx=0).stop 2024-03-30T02:37:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.708+0000 7f0326200700 1 -- 172.21.15.67:0/984246817 shutdown_connections 2024-03-30T02:37:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.708+0000 7f0326200700 1 --2- 172.21.15.67:0/984246817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320101790 0x7f0320108040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.708+0000 7f0326200700 1 --2- 172.21.15.67:0/984246817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03200ffc00 0x7f0320100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.708+0000 7f0326200700 1 --2- 172.21.15.67:0/984246817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0320100df0 0x7f0320101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.708+0000 7f0326200700 1 -- 172.21.15.67:0/984246817 >> 172.21.15.67:0/984246817 conn(0x7f03200fb170 msgr2=0x7f03200fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.708+0000 7f0326200700 1 -- 172.21.15.67:0/984246817 shutdown_connections 2024-03-30T02:37:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.709+0000 7f0326200700 1 -- 172.21.15.67:0/984246817 wait complete. 2024-03-30T02:37:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f0326200700 1 Processor -- start 2024-03-30T02:37:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f0326200700 1 -- start start 2024-03-30T02:37:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f0326200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03200ffc00 0x7f0320199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f0326200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100df0 0x7f0320199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f0326200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0320101790 0x7f03201a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f0326200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03201a0b60 con 0x7f0320101790 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f0326200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03201a0ca0 con 0x7f0320100df0 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f0326200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03201a0e30 con 0x7f03200ffc00 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f031f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03200ffc00 0x7f0320199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f031f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03200ffc00 0x7f0320199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60760/0 (socket says 172.21.15.67:60760) 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f031ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100df0 0x7f0320199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f031f400700 1 -- 172.21.15.67:0/3929692650 learned_addr learned my addr 172.21.15.67:0/3929692650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.710+0000 7f031fe00700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0320101790 0x7f03201a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f031f400700 1 -- 172.21.15.67:0/3929692650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100df0 msgr2=0x7f0320199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f031f400700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100df0 0x7f0320199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f031f400700 1 -- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0320101790 msgr2=0x7f03201a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f031f400700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0320101790 0x7f03201a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f031f400700 1 -- 172.21.15.67:0/3929692650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c009520 con 0x7f03200ffc00 2024-03-30T02:37:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f031f400700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03200ffc00 0x7f0320199030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f031000e970 tx=0x7f031000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f0317e00700 1 -- 172.21.15.67:0/3929692650 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310009790 con 0x7f03200ffc00 2024-03-30T02:37:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f0317e00700 1 -- 172.21.15.67:0/3929692650 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f031000c7e0 con 0x7f03200ffc00 2024-03-30T02:37:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0318009d00 con 0x7f03200ffc00 2024-03-30T02:37:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.711+0000 7f0317e00700 1 -- 172.21.15.67:0/3929692650 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310003c70 con 0x7f03200ffc00 2024-03-30T02:37:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.712+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201a14e0 con 0x7f03200ffc00 2024-03-30T02:37:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.713+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0320067c30 con 0x7f03200ffc00 2024-03-30T02:37:03.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.714+0000 7f0317e00700 1 -- 172.21.15.67:0/3929692650 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) v1 ==== 135677+0+0 (secure 0 0 0) 0x7f031000cc50 con 0x7f03200ffc00 2024-03-30T02:37:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.716+0000 7f0317e00700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a0fa0 0x7f03040a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.716+0000 7f0317e00700 1 -- 172.21.15.67:0/3929692650 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f0310014070 con 0x7f03200ffc00 2024-03-30T02:37:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.716+0000 7f031ea00700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a0fa0 0x7f03040a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.717+0000 7f031ea00700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a0fa0 0x7f03040a3460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f030c009df0 tx=0x7f030c009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:03.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:03.720+0000 7f0317e00700 1 -- 172.21.15.67:0/3929692650 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f03100960b0 con 0x7f03200ffc00 2024-03-30T02:37:04.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.026+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 --> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f03200fd390 con 0x7f03040a0fa0 2024-03-30T02:37:04.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.097+0000 7f0317e00700 1 -- 172.21.15.67:0/3929692650 <== mgr.25634 v2:172.21.15.67:6800/1710497985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f03200fd390 con 0x7f03040a0fa0 2024-03-30T02:37:04.097 INFO:teuthology.orchestra.run.smithi067.stdout:Retention added to path / 2024-03-30T02:37:04.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a0fa0 msgr2=0x7f03040a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a0fa0 0x7f03040a3460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f030c009df0 tx=0x7f030c009260 comp rx=0 tx=0).stop 2024-03-30T02:37:04.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03200ffc00 msgr2=0x7f0320199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03200ffc00 0x7f0320199030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f031000e970 tx=0x7f031000ec80 comp rx=0 tx=0).stop 2024-03-30T02:37:04.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 shutdown_connections 2024-03-30T02:37:04.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a0fa0 0x7f03040a3460 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0320101790 0x7f03201a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03200ffc00 0x7f0320199030 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 --2- 172.21.15.67:0/3929692650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100df0 0x7f0320199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 >> 172.21.15.67:0/3929692650 conn(0x7f03200fb170 msgr2=0x7f03200fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:04.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 shutdown_connections 2024-03-30T02:37:04.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.100+0000 7f0326200700 1 -- 172.21.15.67:0/3929692650 wait complete. 2024-03-30T02:37:04.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-30T02:37:04.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.336+0000 7f69b7200700 1 -- 172.21.15.67:0/1002220964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 msgr2=0x7f69b0103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.336+0000 7f69b7200700 1 --2- 172.21.15.67:0/1002220964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 0x7f69b0103c10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009990 tx=0x7f69a0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.336+0000 7f69b7200700 1 -- 172.21.15.67:0/1002220964 shutdown_connections 2024-03-30T02:37:04.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.336+0000 7f69b7200700 1 --2- 172.21.15.67:0/1002220964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69b0106a80 0x7f69b0108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.336+0000 7f69b7200700 1 --2- 172.21.15.67:0/1002220964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 0x7f69b0103c10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.336+0000 7f69b7200700 1 --2- 172.21.15.67:0/1002220964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0104150 0x7f69b0106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.336+0000 7f69b7200700 1 -- 172.21.15.67:0/1002220964 >> 172.21.15.67:0/1002220964 conn(0x7f69b00fb170 msgr2=0x7f69b00fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:04.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.336+0000 7f69b7200700 1 -- 172.21.15.67:0/1002220964 shutdown_connections 2024-03-30T02:37:04.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.336+0000 7f69b7200700 1 -- 172.21.15.67:0/1002220964 wait complete. 2024-03-30T02:37:04.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.337+0000 7f69b7200700 1 Processor -- start 2024-03-30T02:37:04.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.337+0000 7f69b7200700 1 -- start start 2024-03-30T02:37:04.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.337+0000 7f69b7200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 0x7f69b0198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69b7200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69b0104150 0x7f69b01993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69b7200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0106a80 0x7f69b01a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69b7200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b01a09e0 con 0x7f69b0104150 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69b7200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b01a0b20 con 0x7f69b0101820 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69b7200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b01a0cb0 con 0x7f69b0106a80 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69afe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 0x7f69b0198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69afe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 0x7f69b0198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:58320/0 (socket says 172.21.15.67:58320) 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69afe00700 1 -- 172.21.15.67:0/150834268 learned_addr learned my addr 172.21.15.67:0/150834268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69b5200700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0106a80 0x7f69b01a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69af400700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69b0104150 0x7f69b01993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69afe00700 1 -- 172.21.15.67:0/150834268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0106a80 msgr2=0x7f69b01a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69afe00700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0106a80 0x7f69b01a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69afe00700 1 -- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69b0104150 msgr2=0x7f69b01993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69afe00700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69b0104150 0x7f69b01993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.338+0000 7f69afe00700 1 -- 172.21.15.67:0/150834268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00095f0 con 0x7f69b0101820 2024-03-30T02:37:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.339+0000 7f69afe00700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 0x7f69b0198eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f69a0000c00 tx=0x7f69a0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.339+0000 7f69acc00700 1 -- 172.21.15.67:0/150834268 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a001d070 con 0x7f69b0101820 2024-03-30T02:37:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.339+0000 7f69acc00700 1 -- 172.21.15.67:0/150834268 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69a0021410 con 0x7f69b0101820 2024-03-30T02:37:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.339+0000 7f69acc00700 1 -- 172.21.15.67:0/150834268 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0017650 con 0x7f69b0101820 2024-03-30T02:37:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.339+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b01a0f30 con 0x7f69b0101820 2024-03-30T02:37:04.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.339+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b01a12a0 con 0x7f69b0101820 2024-03-30T02:37:04.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.340+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b0108600 con 0x7f69b0101820 2024-03-30T02:37:04.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.342+0000 7f69acc00700 1 -- 172.21.15.67:0/150834268 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135677+0+0 (secure 0 0 0) 0x7f69a00217f0 con 0x7f69b0101820 2024-03-30T02:37:04.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.343+0000 7f69acc00700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f69940a0fa0 0x7f69940a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:04.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.343+0000 7f69acc00700 1 -- 172.21.15.67:0/150834268 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f69a00cdd80 con 0x7f69b0101820 2024-03-30T02:37:04.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.344+0000 7f69af400700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f69940a0fa0 0x7f69940a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:04.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.344+0000 7f69af400700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f69940a0fa0 0x7f69940a3460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f699c009c60 tx=0x7f699c009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:04.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.351+0000 7f69acc00700 1 -- 172.21.15.67:0/150834268 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f69a00973e0 con 0x7f69b0101820 2024-03-30T02:37:04.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.624+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 --> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f69b002cdf0 con 0x7f69940a0fa0 2024-03-30T02:37:04.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.632+0000 7f69acc00700 1 -- 172.21.15.67:0/150834268 <== mgr.25634 v2:172.21.15.67:6800/1710497985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f69b002cdf0 con 0x7f69940a0fa0 2024-03-30T02:37:04.632 INFO:teuthology.orchestra.run.smithi067.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-30T00:00:00", "created": "2024-03-30T02:37:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-30T02:37:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f69940a0fa0 msgr2=0x7f69940a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f69940a0fa0 0x7f69940a3460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f699c009c60 tx=0x7f699c009260 comp rx=0 tx=0).stop 2024-03-30T02:37:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 msgr2=0x7f69b0198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 0x7f69b0198eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f69a0000c00 tx=0x7f69a0005740 comp rx=0 tx=0).stop 2024-03-30T02:37:04.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 shutdown_connections 2024-03-30T02:37:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f69940a0fa0 0x7f69940a3460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0106a80 0x7f69b01a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69b0101820 0x7f69b0198eb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 --2- 172.21.15.67:0/150834268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69b0104150 0x7f69b01993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 >> 172.21.15.67:0/150834268 conn(0x7f69b00fb170 msgr2=0x7f69b01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 shutdown_connections 2024-03-30T02:37:04.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.635+0000 7f69b7200700 1 -- 172.21.15.67:0/150834268 wait complete. 2024-03-30T02:37:04.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:04 smithi067 ceph-mon[37808]: [30/Mar/2024:02:37:03] ENGINE Bus STARTING 2024-03-30T02:37:04.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:04 smithi067 ceph-mon[37808]: [30/Mar/2024:02:37:03] ENGINE Serving on http://172.21.15.67:8765 2024-03-30T02:37:04.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:04 smithi067 ceph-mon[37808]: [30/Mar/2024:02:37:03] ENGINE Serving on https://172.21.15.67:7150 2024-03-30T02:37:04.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:04 smithi067 ceph-mon[37808]: [30/Mar/2024:02:37:03] ENGINE Bus STARTED 2024-03-30T02:37:04.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:04 smithi067 ceph-mon[37808]: mgrmap e24: x(active, since 1.01875s), standbys: z, y 2024-03-30T02:37:04.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:04 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:04 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:04 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:04 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-30T02:37:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:04 smithi170 ceph-mon[78798]: [30/Mar/2024:02:37:03] ENGINE Bus STARTING 2024-03-30T02:37:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:04 smithi170 ceph-mon[78798]: [30/Mar/2024:02:37:03] ENGINE Serving on http://172.21.15.67:8765 2024-03-30T02:37:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:04 smithi170 ceph-mon[78798]: [30/Mar/2024:02:37:03] ENGINE Serving on https://172.21.15.67:7150 2024-03-30T02:37:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:04 smithi170 ceph-mon[78798]: [30/Mar/2024:02:37:03] ENGINE Bus STARTED 2024-03-30T02:37:04.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:04 smithi170 ceph-mon[78798]: mgrmap e24: x(active, since 1.01875s), standbys: z, y 2024-03-30T02:37:04.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:04 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:04 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:04 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:04 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:04 smithi082 ceph-mon[79176]: [30/Mar/2024:02:37:03] ENGINE Bus STARTING 2024-03-30T02:37:04.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:04 smithi082 ceph-mon[79176]: [30/Mar/2024:02:37:03] ENGINE Serving on http://172.21.15.67:8765 2024-03-30T02:37:04.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:04 smithi082 ceph-mon[79176]: [30/Mar/2024:02:37:03] ENGINE Serving on https://172.21.15.67:7150 2024-03-30T02:37:04.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:04 smithi082 ceph-mon[79176]: [30/Mar/2024:02:37:03] ENGINE Bus STARTED 2024-03-30T02:37:04.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:04 smithi082 ceph-mon[79176]: mgrmap e24: x(active, since 1.01875s), standbys: z, y 2024-03-30T02:37:04.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:04 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:04 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:04 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:04 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.872+0000 7fd34ec00700 1 -- 172.21.15.67:0/4164767045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd348100e00 msgr2=0x7fd348101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.872+0000 7fd34ec00700 1 --2- 172.21.15.67:0/4164767045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd348100e00 0x7fd348101260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3340099c0 tx=0x7fd334009cd0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.872+0000 7fd34ec00700 1 -- 172.21.15.67:0/4164767045 shutdown_connections 2024-03-30T02:37:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.872+0000 7fd34ec00700 1 --2- 172.21.15.67:0/4164767045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3481017a0 0x7fd348108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.872+0000 7fd34ec00700 1 --2- 172.21.15.67:0/4164767045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3480ffc10 0x7fd348100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.872+0000 7fd34ec00700 1 --2- 172.21.15.67:0/4164767045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd348100e00 0x7fd348101260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.872+0000 7fd34ec00700 1 -- 172.21.15.67:0/4164767045 >> 172.21.15.67:0/4164767045 conn(0x7fd3480fb1a0 msgr2=0x7fd3480fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.872+0000 7fd34ec00700 1 -- 172.21.15.67:0/4164767045 shutdown_connections 2024-03-30T02:37:04.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.873+0000 7fd34ec00700 1 -- 172.21.15.67:0/4164767045 wait complete. 2024-03-30T02:37:04.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.873+0000 7fd34ec00700 1 Processor -- start 2024-03-30T02:37:04.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd34ec00700 1 -- start start 2024-03-30T02:37:04.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd34ec00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3480ffc10 0x7fd348198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:04.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd34ec00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348100e00 0x7fd348199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd34ec00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3481017a0 0x7fd3481a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd34ec00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481a0a30 con 0x7fd3481017a0 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd34ec00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481a0b70 con 0x7fd3480ffc10 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd34ec00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481a0d00 con 0x7fd348100e00 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd347e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3480ffc10 0x7fd348198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd347e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3480ffc10 0x7fd348198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:58360/0 (socket says 172.21.15.67:58360) 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd347e00700 1 -- 172.21.15.67:0/82430938 learned_addr learned my addr 172.21.15.67:0/82430938 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd347400700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348100e00 0x7fd348199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.874+0000 7fd34cc00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3481017a0 0x7fd3481a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd347e00700 1 -- 172.21.15.67:0/82430938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348100e00 msgr2=0x7fd348199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd347e00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348100e00 0x7fd348199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd347e00700 1 -- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3481017a0 msgr2=0x7fd3481a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd347e00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3481017a0 0x7fd3481a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:04.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd347e00700 1 -- 172.21.15.67:0/82430938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3340095f0 con 0x7fd3480ffc10 2024-03-30T02:37:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd347e00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3480ffc10 0x7fd348198f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd33800e970 tx=0x7fd33800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd344c00700 1 -- 172.21.15.67:0/82430938 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338009790 con 0x7fd3480ffc10 2024-03-30T02:37:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd344c00700 1 -- 172.21.15.67:0/82430938 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd33800c7e0 con 0x7fd3480ffc10 2024-03-30T02:37:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd344c00700 1 -- 172.21.15.67:0/82430938 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338003c70 con 0x7fd3480ffc10 2024-03-30T02:37:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3481a0fe0 con 0x7fd3480ffc10 2024-03-30T02:37:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.875+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3481a13b0 con 0x7fd3480ffc10 2024-03-30T02:37:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.877+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd348068080 con 0x7fd3480ffc10 2024-03-30T02:37:04.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.878+0000 7fd344c00700 1 -- 172.21.15.67:0/82430938 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 24) v1 ==== 135677+0+0 (secure 0 0 0) 0x7fd33800cc50 con 0x7fd3480ffc10 2024-03-30T02:37:04.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.879+0000 7fd344c00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd32c0a0fa0 0x7fd32c0a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:04.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.879+0000 7fd344c00700 1 -- 172.21.15.67:0/82430938 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd338014070 con 0x7fd3480ffc10 2024-03-30T02:37:04.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.880+0000 7fd347400700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd32c0a0fa0 0x7fd32c0a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:04.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.880+0000 7fd347400700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd32c0a0fa0 0x7fd32c0a3460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd334000c00 tx=0x7fd334019040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:04.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:04.883+0000 7fd344c00700 1 -- 172.21.15.67:0/82430938 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd3380960b0 con 0x7fd3480ffc10 2024-03-30T02:37:05.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.067+0000 7fd344c00700 1 -- 172.21.15.67:0/82430938 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 25) v1 ==== 135985+0+0 (secure 0 0 0) 0x7fd33800c950 con 0x7fd3480ffc10 2024-03-30T02:37:05.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.144+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 --> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd348069b90 con 0x7fd32c0a0fa0 2024-03-30T02:37:05.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.151+0000 7fd344c00700 1 -- 172.21.15.67:0/82430938 <== mgr.25634 v2:172.21.15.67:6800/1710497985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fd348069b90 con 0x7fd32c0a0fa0 2024-03-30T02:37:05.151 INFO:teuthology.orchestra.run.smithi067.stdout:/ 1m 6m3h 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd32c0a0fa0 msgr2=0x7fd32c0a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd32c0a0fa0 0x7fd32c0a3460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd334000c00 tx=0x7fd334019040 comp rx=0 tx=0).stop 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3480ffc10 msgr2=0x7fd348198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3480ffc10 0x7fd348198f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd33800e970 tx=0x7fd33800ec80 comp rx=0 tx=0).stop 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 shutdown_connections 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd32c0a0fa0 0x7fd32c0a3460 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3481017a0 0x7fd3481a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3480ffc10 0x7fd348198f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 --2- 172.21.15.67:0/82430938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd348100e00 0x7fd348199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 >> 172.21.15.67:0/82430938 conn(0x7fd3480fb1a0 msgr2=0x7fd348069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 shutdown_connections 2024-03-30T02:37:05.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:05.154+0000 7fd34ec00700 1 -- 172.21.15.67:0/82430938 wait complete. 2024-03-30T02:37:05.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-30T02:37:05.262 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-30T02:37:05.276 INFO:teuthology.run_tasks:Running task workunit... 2024-03-30T02:37:05.308 INFO:tasks.workunit:Pulling workunits from ref 470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:37:05.309 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-30T02:37:05.309 INFO:tasks.workunit:timeout=3h 2024-03-30T02:37:05.309 INFO:tasks.workunit:cleanup=True 2024-03-30T02:37:05.309 DEBUG:teuthology.orchestra.run.smithi067:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-30T02:37:05.352 INFO:teuthology.orchestra.run.smithi067.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-30T02:37:05.352 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-30T02:37:05.352 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 7ah/122d Inode: 1099511628283 Links: 2 2024-03-30T02:37:05.352 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T02:37:05.352 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:cephfs_t:s0 2024-03-30T02:37:05.352 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-03-30 02:35:20.659857678 +0000 2024-03-30T02:37:05.352 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-03-30 02:35:20.659857678 +0000 2024-03-30T02:37:05.352 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-03-30 02:36:40.230836919 +0000 2024-03-30T02:37:05.352 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-03-30 02:35:20.659857678 +0000 2024-03-30T02:37:05.353 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-30T02:37:05.353 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-30T02:37:05.454 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='client.25681 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:05 smithi067 ceph-mon[37808]: mgrmap e25: x(active, since 2s), standbys: z, y 2024-03-30T02:37:05.476 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 470100a4ccbcf7769052bf09a5413881ccf1b97f 2024-03-30T02:37:05.524 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-30T02:37:05.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:05.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:37:05.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='client.25681 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:05.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:05 smithi170 ceph-mon[78798]: mgrmap e25: x(active, since 2s), standbys: z, y 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='client.25681 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:05 smithi082 ceph-mon[79176]: mgrmap e25: x(active, since 2s), standbys: z, y 2024-03-30T02:37:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:37:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='client.25697 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi170 to 182.5M 2024-03-30T02:37:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: Unable to set osd_memory_target on smithi170 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-03-30T02:37:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi082 to 182.4M 2024-03-30T02:37:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: Unable to set osd_memory_target on smithi082 to 191351193: error parsing value: Value '191351193' is below minimum 939524096 2024-03-30T02:37:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='client.25709 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:06.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:06.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:06.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:37:06.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:37:06.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:06 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:37:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:37:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='client.25697 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi170 to 182.5M 2024-03-30T02:37:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: Unable to set osd_memory_target on smithi170 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-03-30T02:37:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi082 to 182.4M 2024-03-30T02:37:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: Unable to set osd_memory_target on smithi082 to 191351193: error parsing value: Value '191351193' is below minimum 939524096 2024-03-30T02:37:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='client.25709 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:37:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:37:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:06 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:37:06.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='client.25697 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi170 to 182.5M 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: Unable to set osd_memory_target on smithi170 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi082 to 182.4M 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: Unable to set osd_memory_target on smithi082 to 191351193: error parsing value: Value '191351193' is below minimum 939524096 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='client.25709 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:37:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:06 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:37:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi067 to 221.5M 2024-03-30T02:37:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: Unable to set osd_memory_target on smithi067 to 232361472: error parsing value: Value '232361472' is below minimum 939524096 2024-03-30T02:37:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: Updating smithi067:/etc/ceph/ceph.conf 2024-03-30T02:37:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: Updating smithi082:/etc/ceph/ceph.conf 2024-03-30T02:37:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-30T02:37:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: Updating smithi082:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:37:07.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:37:07.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:07 smithi067 ceph-mon[37808]: mgrmap e26: x(active, since 4s), standbys: z, y 2024-03-30T02:37:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi067 to 221.5M 2024-03-30T02:37:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: Unable to set osd_memory_target on smithi067 to 232361472: error parsing value: Value '232361472' is below minimum 939524096 2024-03-30T02:37:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: Updating smithi067:/etc/ceph/ceph.conf 2024-03-30T02:37:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: Updating smithi082:/etc/ceph/ceph.conf 2024-03-30T02:37:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-30T02:37:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: Updating smithi082:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:37:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:37:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:07 smithi170 ceph-mon[78798]: mgrmap e26: x(active, since 4s), standbys: z, y 2024-03-30T02:37:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi067 to 221.5M 2024-03-30T02:37:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: Unable to set osd_memory_target on smithi067 to 232361472: error parsing value: Value '232361472' is below minimum 939524096 2024-03-30T02:37:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: Updating smithi067:/etc/ceph/ceph.conf 2024-03-30T02:37:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: Updating smithi082:/etc/ceph/ceph.conf 2024-03-30T02:37:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: Updating smithi170:/etc/ceph/ceph.conf 2024-03-30T02:37:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: Updating smithi082:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:37:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:37:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:07 smithi082 ceph-mon[79176]: mgrmap e26: x(active, since 4s), standbys: z, y 2024-03-30T02:37:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:37:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: Updating smithi067:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: Updating smithi082:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: Updating smithi067:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:37:08.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:08 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:37:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: Updating smithi067:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:37:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:37:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:37:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: Updating smithi082:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:37:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:37:08.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:37:08.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: Updating smithi067:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:37:08.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:37:08.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:37:08.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:37:08.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:08 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: Updating smithi067:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: Updating smithi082:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: Updating smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: Updating smithi067:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:37:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:37:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:37:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:08 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:37:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:10 smithi067 ceph-mon[37808]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 91 KiB/s wr, 97 op/s 2024-03-30T02:37:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:10 smithi170 ceph-mon[78798]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 91 KiB/s wr, 97 op/s 2024-03-30T02:37:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:10 smithi082 ceph-mon[79176]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 91 KiB/s wr, 97 op/s 2024-03-30T02:37:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:12 smithi067 ceph-mon[37808]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-30T02:37:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:12 smithi170 ceph-mon[78798]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-30T02:37:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:12 smithi082 ceph-mon[79176]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-03-30T02:37:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:14 smithi067 ceph-mon[37808]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-30T02:37:14.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:14 smithi170 ceph-mon[78798]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-30T02:37:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:14 smithi082 ceph-mon[79176]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-30T02:37:16.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:16 smithi067 ceph-mon[37808]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-30T02:37:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:16 smithi170 ceph-mon[78798]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-30T02:37:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:16 smithi082 ceph-mon[79176]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-03-30T02:37:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:18 smithi067 ceph-mon[37808]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-30T02:37:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:18 smithi170 ceph-mon[78798]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-30T02:37:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:18 smithi082 ceph-mon[79176]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-30T02:37:20.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:20 smithi067 ceph-mon[37808]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-03-30T02:37:20.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:20 smithi170 ceph-mon[78798]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-03-30T02:37:20.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:20 smithi082 ceph-mon[79176]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-03-30T02:37:22.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:22 smithi067 ceph-mon[37808]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-03-30T02:37:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:22 smithi170 ceph-mon[78798]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-03-30T02:37:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:22 smithi082 ceph-mon[79176]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-03-30T02:37:23.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:37:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.405+0000 7fb6aec00700 1 -- 172.21.15.67:0/2259762620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 msgr2=0x7fb6a8100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.405+0000 7fb6aec00700 1 --2- 172.21.15.67:0/2259762620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 0x7fb6a8100fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb6940099c0 tx=0x7fb694009cd0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.405+0000 7fb6aec00700 1 -- 172.21.15.67:0/2259762620 shutdown_connections 2024-03-30T02:37:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.405+0000 7fb6aec00700 1 --2- 172.21.15.67:0/2259762620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a8101510 0x7fb6a810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.405+0000 7fb6aec00700 1 --2- 172.21.15.67:0/2259762620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6a8106c10 0x7fb6a8106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.405+0000 7fb6aec00700 1 --2- 172.21.15.67:0/2259762620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 0x7fb6a8100fd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.405+0000 7fb6aec00700 1 -- 172.21.15.67:0/2259762620 >> 172.21.15.67:0/2259762620 conn(0x7fb6a80fc620 msgr2=0x7fb6a80fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.405+0000 7fb6aec00700 1 -- 172.21.15.67:0/2259762620 shutdown_connections 2024-03-30T02:37:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.405+0000 7fb6aec00700 1 -- 172.21.15.67:0/2259762620 wait complete. 2024-03-30T02:37:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.406+0000 7fb6aec00700 1 Processor -- start 2024-03-30T02:37:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.406+0000 7fb6aec00700 1 -- start start 2024-03-30T02:37:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6aec00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 0x7fb6a8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6aec00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6a8101510 0x7fb6a8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6aec00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a8106c10 0x7fb6a819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6aec00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a819e0e0 con 0x7fb6a8100b50 2024-03-30T02:37:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6aec00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a819e250 con 0x7fb6a8106c10 2024-03-30T02:37:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6aec00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a819e410 con 0x7fb6a8101510 2024-03-30T02:37:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ad800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 0x7fb6a8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ad800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 0x7fb6a8199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35362/0 (socket says 172.21.15.67:35362) 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ad800700 1 -- 172.21.15.67:0/3015595170 learned_addr learned my addr 172.21.15.67:0/3015595170 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ae200700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a8106c10 0x7fb6a819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ace00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6a8101510 0x7fb6a8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ad800700 1 -- 172.21.15.67:0/3015595170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6a8101510 msgr2=0x7fb6a8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ad800700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6a8101510 0x7fb6a8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ad800700 1 -- 172.21.15.67:0/3015595170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a8106c10 msgr2=0x7fb6a819daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ad800700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a8106c10 0x7fb6a819daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ad800700 1 -- 172.21.15.67:0/3015595170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6940095f0 con 0x7fb6a8100b50 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.407+0000 7fb6ace00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6a8101510 0x7fb6a8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.408+0000 7fb6ad800700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 0x7fb6a8199210 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00d950 tx=0x7fb69c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.408+0000 7fb6a6000700 1 -- 172.21.15.67:0/3015595170 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c009940 con 0x7fb6a8100b50 2024-03-30T02:37:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.408+0000 7fb6a6000700 1 -- 172.21.15.67:0/3015595170 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb69c010460 con 0x7fb6a8100b50 2024-03-30T02:37:23.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.408+0000 7fb6a6000700 1 -- 172.21.15.67:0/3015595170 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00f620 con 0x7fb6a8100b50 2024-03-30T02:37:23.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.408+0000 7fb6aec00700 1 -- 172.21.15.67:0/3015595170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a819e6f0 con 0x7fb6a8100b50 2024-03-30T02:37:23.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.408+0000 7fb6aec00700 1 -- 172.21.15.67:0/3015595170 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a81a5f30 con 0x7fb6a8100b50 2024-03-30T02:37:23.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.411+0000 7fb6a6000700 1 -- 172.21.15.67:0/3015595170 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fb69c0108d0 con 0x7fb6a8100b50 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.412+0000 7fb6a6000700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb6900a1320 0x7fb6900a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.412+0000 7fb6c3cf1b80 1 Processor -- start 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.412+0000 7fb6a6000700 1 -- 172.21.15.67:0/3015595170 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fb69c0ccff0 con 0x7fb6a8100b50 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.412+0000 7fb6c3cf1b80 1 -- start start 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6ace00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb6900a1320 0x7fb6900a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64e97a0 0x55f9c64e9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64f4fc0 0x55f9c64f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c6442990 con 0x55f9c64f4fc0 2024-03-30T02:37:23.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c6440350 con 0x55f9c64ea0c0 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c643bc40 con 0x55f9c64e97a0 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6ad800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64e97a0 0x55f9c64e9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6ad800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64e97a0 0x55f9c64e9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:51952/0 (socket says 172.21.15.67:51952) 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6ad800700 1 -- 172.21.15.67:0/2651318278 learned_addr learned my addr 172.21.15.67:0/2651318278 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6ae200700 1 --2- 172.21.15.67:0/2651318278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64f4fc0 0x55f9c64f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.413+0000 7fb6ace00700 1 --2- 172.21.15.67:0/2651318278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb6ace00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb6900a1320 0x7fb6900a37e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6a819a710 tx=0x7fb694005fb0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb6ad800700 1 -- 172.21.15.67:0/2651318278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 msgr2=0x55f9c64f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb6ad800700 1 --2- 172.21.15.67:0/2651318278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb6ad800700 1 -- 172.21.15.67:0/2651318278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64f4fc0 msgr2=0x55f9c64f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb6ad800700 1 --2- 172.21.15.67:0/2651318278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64f4fc0 0x55f9c64f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb6ad800700 1 -- 172.21.15.67:0/2651318278 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9c62bd3b0 con 0x55f9c64e97a0 2024-03-30T02:37:23.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb6ad800700 1 --2- 172.21.15.67:0/2651318278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64e97a0 0x55f9c64e9b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb69c099900 tx=0x7fb69c0995e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9c11530bfd3a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:23.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb69aa00700 1 -- 172.21.15.67:0/2651318278 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c09da30 con 0x55f9c64e97a0 2024-03-30T02:37:23.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb69aa00700 1 -- 172.21.15.67:0/2651318278 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb69c0a2080 con 0x55f9c64e97a0 2024-03-30T02:37:23.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.414+0000 7fb69aa00700 1 -- 172.21.15.67:0/2651318278 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0990f0 con 0x55f9c64e97a0 2024-03-30T02:37:23.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.415+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2651318278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64e97a0 msgr2=0x55f9c64e9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.415+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2651318278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64e97a0 0x55f9c64e9b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb69c099900 tx=0x7fb69c0995e0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.415+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2651318278 shutdown_connections 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.415+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2651318278 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64f4fc0 0x55f9c64f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.415+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2651318278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64e97a0 0x55f9c64e9b80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.415+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2651318278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.415+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2651318278 >> 172.21.15.67:0/2651318278 conn(0x55f9c64496a0 msgr2=0x55f9c64e5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.415+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2651318278 shutdown_connections 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.415+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2651318278 wait complete. 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.416+0000 7fb6c3cf1b80 1 -- start start 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.416+0000 7fb6c3cf1b80 1 Processor -- start 2024-03-30T02:37:23.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.416+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.416+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.416+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64f4fc0 0x55f9c64f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.416+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c63d7d80 con 0x55f9c64e97a0 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.416+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c63d7a20 con 0x55f9c64ea0c0 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.416+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c62bf6a0 con 0x55f9c64f4fc0 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.416+0000 7fb6c3cf1b80 10 client.0 _ll_register_callbacks cb 0x55f9c63e1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ad800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ad800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35370/0 (socket says 172.21.15.67:35370) 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ae200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64f4fc0 0x55f9c64f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ad800700 1 -- 172.21.15.67:0/860138977 learned_addr learned my addr 172.21.15.67:0/860138977 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ace00700 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ad800700 1 -- 172.21.15.67:0/860138977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64f4fc0 msgr2=0x55f9c64f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ad800700 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64f4fc0 0x55f9c64f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ad800700 1 -- 172.21.15.67:0/860138977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 msgr2=0x55f9c64f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ad800700 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ad800700 1 -- 172.21.15.67:0/860138977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a81a5f30 con 0x55f9c64e97a0 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ae200700 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64f4fc0 0x55f9c64f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb6ad800700 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f49f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0a1b50 tx=0x7fb69c0ca7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abf11d0815934db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.417+0000 7fb68f400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb68f400700 1 -- 172.21.15.67:0/860138977 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0b6070 con 0x55f9c64e97a0 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb68f400700 1 -- 172.21.15.67:0/860138977 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fb69c09da30 con 0x55f9c64e97a0 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/860138977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9c63fc380 con 0x55f9c64e97a0 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb68f400700 1 -- 172.21.15.67:0/860138977 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb69c09a070 con 0x55f9c64e97a0 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb6c3cf1b80 10 client.15672 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:23.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/860138977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f9c6269b60 con 0x55f9c64e97a0 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb68f400700 1 -- 172.21.15.67:0/860138977 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9c63fc380 con 0x55f9c64e97a0 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb68f400700 1 -- 172.21.15.67:0/860138977 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb69c0959f0 con 0x55f9c64e97a0 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb6c3cf1b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb6c3cf1b80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb6c3cf1b80 20 client.15672 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f9c6504650 0x55f9c6506a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb6c3cf1b80 4 client.15672 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.418+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/860138977 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9c6434700 con 0x55f9c6504650 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.419+0000 7fb6ace00700 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f9c6504650 0x55f9c6506a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.419+0000 7fb68f400700 10 client.15672 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.419+0000 7fb6ace00700 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f9c6504650 0x55f9c6506a40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7a196cea4fda13 server_cookie=64a991d3dea51122 in_seq=0 out_seq=0 2024-03-30T02:37:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb68f400700 1 -- 172.21.15.67:0/860138977 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f9c6434700 con 0x55f9c6504650 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb68f400700 10 client.15672 handle_command_reply: tid=0 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 client.15672 shutdown 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/860138977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 msgr2=0x55f9c64f49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f49f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0a1b50 tx=0x7fb69c0ca7d0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/860138977 shutdown_connections 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f9c6504650 0x55f9c6506a40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f49f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64f4fc0 0x55f9c64f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/860138977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb68f400700 0 client.15672 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/860138977 >> 172.21.15.67:0/860138977 conn(0x55f9c64496a0 msgr2=0x55f9c64eb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/860138977 shutdown_connections 2024-03-30T02:37:23.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.420+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/860138977 wait complete. 2024-03-30T02:37:23.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.421+0000 7fb6c3cf1b80 20 client.15672 trim_cache size 0 max 16384 2024-03-30T02:37:23.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.438+0000 7fb6c3cf1b80 1 Processor -- start 2024-03-30T02:37:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.438+0000 7fb6c3cf1b80 1 -- start start 2024-03-30T02:37:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.438+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.438+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 0x55f9c64fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.438+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 0x55f9c64df010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.438+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c64fc120 con 0x55f9c64e97a0 2024-03-30T02:37:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.438+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c64df7c0 con 0x55f9c64f4fc0 2024-03-30T02:37:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.438+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c64ea4a0 con 0x55f9c64ea0c0 2024-03-30T02:37:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ad800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ad800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35376/0 (socket says 172.21.15.67:35376) 2024-03-30T02:37:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ad800700 1 -- 172.21.15.67:0/2823853689 learned_addr learned my addr 172.21.15.67:0/2823853689 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ae200700 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 0x55f9c64df010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ace00700 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 0x55f9c64fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ad800700 1 -- 172.21.15.67:0/2823853689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 msgr2=0x55f9c64fbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ad800700 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 0x55f9c64fbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ad800700 1 -- 172.21.15.67:0/2823853689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 msgr2=0x55f9c64df010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ad800700 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 0x55f9c64df010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ad800700 1 -- 172.21.15.67:0/2823853689 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9c6401ea0 con 0x55f9c64e97a0 2024-03-30T02:37:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ace00700 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 0x55f9c64fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb6ad800700 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f23e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb69c010fd0 tx=0x7fb69c010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7331fd47e13a99f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.439+0000 7fb69b400700 1 -- 172.21.15.67:0/2823853689 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0ca230 con 0x55f9c64e97a0 2024-03-30T02:37:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb69b400700 1 -- 172.21.15.67:0/2823853689 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb69c0ca840 con 0x55f9c64e97a0 2024-03-30T02:37:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb69b400700 1 -- 172.21.15.67:0/2823853689 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0990f0 con 0x55f9c64e97a0 2024-03-30T02:37:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2823853689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 msgr2=0x55f9c64f23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f23e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb69c010fd0 tx=0x7fb69c010bd0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2823853689 shutdown_connections 2024-03-30T02:37:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f23e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 0x55f9c64df010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2823853689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 0x55f9c64fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2823853689 >> 172.21.15.67:0/2823853689 conn(0x55f9c64496a0 msgr2=0x55f9c64de7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2823853689 shutdown_connections 2024-03-30T02:37:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.440+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2823853689 wait complete. 2024-03-30T02:37:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6c3cf1b80 1 -- start start 2024-03-30T02:37:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6c3cf1b80 1 Processor -- start 2024-03-30T02:37:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6c3cf1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 0x55f9c65573e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c64fa680 con 0x55f9c64e97a0 2024-03-30T02:37:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c6557920 con 0x55f9c64f4fc0 2024-03-30T02:37:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6c3cf1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9c6557ae0 con 0x55f9c64ea0c0 2024-03-30T02:37:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6c3cf1b80 10 client.0 _ll_register_callbacks cb 0x55f9c63e1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.441+0000 7fb6ad800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ae200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 0x55f9c65573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ae200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 0x55f9c65573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43726/0 (socket says 172.21.15.67:43726) 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ae200700 1 -- 172.21.15.67:0/2618299899 learned_addr learned my addr 172.21.15.67:0/2618299899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ace00700 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ad800700 1 -- 172.21.15.67:0/2618299899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 msgr2=0x55f9c64f9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ad800700 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ad800700 1 -- 172.21.15.67:0/2618299899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 msgr2=0x55f9c65573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ad800700 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 0x55f9c65573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ad800700 1 -- 172.21.15.67:0/2618299899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9c6269b60 con 0x55f9c64e97a0 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6ad800700 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f99d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb69c095d80 tx=0x7fb69c010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da043d32b08e39dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb68f400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb68f400700 1 -- 172.21.15.67:0/2618299899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c0992f0 con 0x55f9c64e97a0 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb68f400700 1 -- 172.21.15.67:0/2618299899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fb69c09da30 con 0x55f9c64e97a0 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.442+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2618299899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb69c0a2080 con 0x55f9c64e97a0 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb68f400700 1 -- 172.21.15.67:0/2618299899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb69c0ca230 con 0x55f9c64e97a0 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb6c3cf1b80 10 client.15678 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2618299899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f9c62c1cd0 con 0x55f9c64e97a0 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb68f400700 1 -- 172.21.15.67:0/2618299899 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb69c0a2080 con 0x55f9c64e97a0 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb68f400700 1 -- 172.21.15.67:0/2618299899 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb69c05fab0 con 0x55f9c64e97a0 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb6c3cf1b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb6c3cf1b80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb6c3cf1b80 20 client.15678 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f9c6558e30 0x55f9c655b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb6c3cf1b80 4 client.15678 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2618299899 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f9c628acf0 con 0x55f9c6558e30 2024-03-30T02:37:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.443+0000 7fb6ace00700 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f9c6558e30 0x55f9c655b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6ace00700 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f9c6558e30 0x55f9c655b220 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c76046cfe79b99 server_cookie=dc20623c33a4086 in_seq=0 out_seq=0 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb68f400700 10 client.15678 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb68f400700 1 -- 172.21.15.67:0/2618299899 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f9c628acf0 con 0x55f9c6558e30 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb68f400700 10 client.15678 handle_command_reply: tid=0 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 client.15678 shutdown 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2618299899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 msgr2=0x55f9c64f99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f99d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb69c095d80 tx=0x7fb69c010cd0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2618299899 shutdown_connections 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f9c6558e30 0x55f9c655b220 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f9c64e97a0 0x55f9c64f99d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f9c64f4fc0 0x55f9c65573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb68f400700 0 client.15678 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 --2- 172.21.15.67:0/2618299899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c64ea0c0 0x55f9c64f9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2618299899 >> 172.21.15.67:0/2618299899 conn(0x55f9c64496a0 msgr2=0x55f9c64f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.444+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2618299899 shutdown_connections 2024-03-30T02:37:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6c3cf1b80 1 -- 172.21.15.67:0/2618299899 wait complete. 2024-03-30T02:37:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6c3cf1b80 20 client.15678 trim_cache size 0 max 16384 2024-03-30T02:37:23.453 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:37:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:37:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:37:23.453 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:37:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 -- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb6900a1320 msgr2=0x7fb6900a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb6900a1320 0x7fb6900a37e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb6a819a710 tx=0x7fb694005fb0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 -- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 msgr2=0x7fb6a8199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 0x7fb6a8199210 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00d950 tx=0x7fb69c00dc60 comp rx=0 tx=0).stop 2024-03-30T02:37:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 -- 172.21.15.67:0/3015595170 shutdown_connections 2024-03-30T02:37:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb6900a1320 0x7fb6900a37e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6a8100b50 0x7fb6a8199210 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a8106c10 0x7fb6a819daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 --2- 172.21.15.67:0/3015595170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6a8101510 0x7fb6a8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 -- 172.21.15.67:0/3015595170 >> 172.21.15.67:0/3015595170 conn(0x7fb6a80fc620 msgr2=0x7fb6a80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 -- 172.21.15.67:0/3015595170 shutdown_connections 2024-03-30T02:37:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.445+0000 7fb6aec00700 1 -- 172.21.15.67:0/3015595170 wait complete. 2024-03-30T02:37:23.465 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:37:23.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:37:23.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.663+0000 7fa205000700 1 -- 172.21.15.67:0/2264351232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa200100e60 msgr2=0x7fa200101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.663+0000 7fa205000700 1 --2- 172.21.15.67:0/2264351232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa200100e60 0x7fa200101240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009990 tx=0x7fa1f0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.663+0000 7fa205000700 1 -- 172.21.15.67:0/2264351232 shutdown_connections 2024-03-30T02:37:23.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.663+0000 7fa205000700 1 --2- 172.21.15.67:0/2264351232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa200105f80 0x7fa20010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.663+0000 7fa205000700 1 --2- 172.21.15.67:0/2264351232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa200100e60 0x7fa200101240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.663+0000 7fa205000700 1 --2- 172.21.15.67:0/2264351232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa200101810 0x7fa200105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.663+0000 7fa205000700 1 -- 172.21.15.67:0/2264351232 >> 172.21.15.67:0/2264351232 conn(0x7fa2000fc760 msgr2=0x7fa2000feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:23.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.663+0000 7fa205000700 1 -- 172.21.15.67:0/2264351232 shutdown_connections 2024-03-30T02:37:23.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.663+0000 7fa205000700 1 -- 172.21.15.67:0/2264351232 wait complete. 2024-03-30T02:37:23.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.664+0000 7fa205000700 1 Processor -- start 2024-03-30T02:37:23.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.664+0000 7fa205000700 1 -- start start 2024-03-30T02:37:23.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa205000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa200100e60 0x7fa2000730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa205000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa200105f80 0x7fa2000735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa205000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa200073b30 0x7fa2001a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa205000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001a5900 con 0x7fa200100e60 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa205000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001a5a40 con 0x7fa200073b30 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa205000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001a5c00 con 0x7fa200105f80 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fde00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa200100e60 0x7fa2000730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fde00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa200100e60 0x7fa2000730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35392/0 (socket says 172.21.15.67:35392) 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fde00700 1 -- 172.21.15.67:0/4094009837 learned_addr learned my addr 172.21.15.67:0/4094009837 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fd400700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa200105f80 0x7fa2000735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fe800700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa200073b30 0x7fa2001a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fde00700 1 -- 172.21.15.67:0/4094009837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa200105f80 msgr2=0x7fa2000735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fde00700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa200105f80 0x7fa2000735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fde00700 1 -- 172.21.15.67:0/4094009837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa200073b30 msgr2=0x7fa2001a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fde00700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa200073b30 0x7fa2001a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.665+0000 7fa1fde00700 1 -- 172.21.15.67:0/4094009837 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00095f0 con 0x7fa200100e60 2024-03-30T02:37:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.666+0000 7fa1fde00700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa200100e60 0x7fa2000730b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0005910 tx=0x7fa1f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.666+0000 7fa1f6a00700 1 -- 172.21.15.67:0/4094009837 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f001d070 con 0x7fa200100e60 2024-03-30T02:37:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.666+0000 7fa1f6a00700 1 -- 172.21.15.67:0/4094009837 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f0022470 con 0x7fa200100e60 2024-03-30T02:37:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.666+0000 7fa1f6a00700 1 -- 172.21.15.67:0/4094009837 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0017650 con 0x7fa200100e60 2024-03-30T02:37:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.666+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2001a5e80 con 0x7fa200100e60 2024-03-30T02:37:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.666+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2001a61f0 con 0x7fa200100e60 2024-03-30T02:37:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.667+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2001091a0 con 0x7fa200100e60 2024-03-30T02:37:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.669+0000 7fa1f6a00700 1 -- 172.21.15.67:0/4094009837 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fa1f00225e0 con 0x7fa200100e60 2024-03-30T02:37:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.670+0000 7fa1f6a00700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa1e40a1320 0x7fa1e40a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.670+0000 7fa1f6a00700 1 -- 172.21.15.67:0/4094009837 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa1f00ce4c0 con 0x7fa200100e60 2024-03-30T02:37:23.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.670+0000 7fa1fd400700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa1e40a1320 0x7fa1e40a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:23.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.671+0000 7fa1fd400700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa1e40a1320 0x7fa1e40a37e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa2000fed80 tx=0x7fa1ec009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:23.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.673+0000 7fa1f6a00700 1 -- 172.21.15.67:0/4094009837 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fa1f00974e0 con 0x7fa200100e60 2024-03-30T02:37:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.940+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa200102330 con 0x7fa200100e60 2024-03-30T02:37:23.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.941+0000 7fa1f6a00700 1 -- 172.21.15.67:0/4094009837 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10909 (secure 0 0 0) 0x7fa1f009ab00 con 0x7fa200100e60 2024-03-30T02:37:23.942 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:37:23.942 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:36:42.080059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa1e40a1320 msgr2=0x7fa1e40a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa1e40a1320 0x7fa1e40a37e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa2000fed80 tx=0x7fa1ec009260 comp rx=0 tx=0).stop 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa200100e60 msgr2=0x7fa2000730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa200100e60 0x7fa2000730b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0005910 tx=0x7fa1f0004730 comp rx=0 tx=0).stop 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 shutdown_connections 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa1e40a1320 0x7fa1e40a37e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa200100e60 0x7fa2000730b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa200073b30 0x7fa2001a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 --2- 172.21.15.67:0/4094009837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa200105f80 0x7fa2000735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 >> 172.21.15.67:0/4094009837 conn(0x7fa2000fc760 msgr2=0x7fa2000fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:23.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 shutdown_connections 2024-03-30T02:37:23.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:23.944+0000 7fa205000700 1 -- 172.21.15.67:0/4094009837 wait complete. 2024-03-30T02:37:23.946 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-03-30T02:37:23.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.166+0000 7f74ccc00700 1 -- 172.21.15.67:0/3840636069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 msgr2=0x7f74c8101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.166+0000 7f74ccc00700 1 --2- 172.21.15.67:0/3840636069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 0x7f74c8101340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009990 tx=0x7f74b8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.166+0000 7f74ccc00700 1 -- 172.21.15.67:0/3840636069 shutdown_connections 2024-03-30T02:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.166+0000 7f74ccc00700 1 --2- 172.21.15.67:0/3840636069 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 0x7f74c8101340 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.166+0000 7f74ccc00700 1 --2- 172.21.15.67:0/3840636069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c8106080 0x7f74c810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.166+0000 7f74ccc00700 1 --2- 172.21.15.67:0/3840636069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8101910 0x7f74c8105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.166+0000 7f74ccc00700 1 -- 172.21.15.67:0/3840636069 >> 172.21.15.67:0/3840636069 conn(0x7f74c80fc7c0 msgr2=0x7f74c80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.166+0000 7f74ccc00700 1 -- 172.21.15.67:0/3840636069 shutdown_connections 2024-03-30T02:37:24.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.166+0000 7f74ccc00700 1 -- 172.21.15.67:0/3840636069 wait complete. 2024-03-30T02:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.167+0000 7f74ccc00700 1 Processor -- start 2024-03-30T02:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.167+0000 7f74ccc00700 1 -- start start 2024-03-30T02:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.167+0000 7f74ccc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 0x7f74c80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.167+0000 7f74ccc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8106080 0x7f74c80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.167+0000 7f74ccc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c8073b30 0x7f74c81a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.167+0000 7f74ccc00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81a59f0 con 0x7f74c8100f60 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.167+0000 7f74ccc00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81a5b60 con 0x7f74c8073b30 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.167+0000 7f74ccc00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81a5d20 con 0x7f74c8106080 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 0x7f74c80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 0x7f74c80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35400/0 (socket says 172.21.15.67:35400) 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5e00700 1 -- 172.21.15.67:0/131798825 learned_addr learned my addr 172.21.15.67:0/131798825 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5400700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8106080 0x7f74c80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c6800700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c8073b30 0x7f74c81a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5e00700 1 -- 172.21.15.67:0/131798825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8106080 msgr2=0x7f74c80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5e00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8106080 0x7f74c80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5e00700 1 -- 172.21.15.67:0/131798825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c8073b30 msgr2=0x7f74c81a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5e00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c8073b30 0x7f74c81a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5e00700 1 -- 172.21.15.67:0/131798825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b80095f0 con 0x7f74c8100f60 2024-03-30T02:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5400700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8106080 0x7f74c80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.168+0000 7f74c5e00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 0x7f74c80730b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f74b8005910 tx=0x7f74b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.169+0000 7f74bea00700 1 -- 172.21.15.67:0/131798825 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b801d070 con 0x7f74c8100f60 2024-03-30T02:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.169+0000 7f74bea00700 1 -- 172.21.15.67:0/131798825 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74b8022470 con 0x7f74c8100f60 2024-03-30T02:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.169+0000 7f74ccc00700 1 -- 172.21.15.67:0/131798825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c81a5fa0 con 0x7f74c8100f60 2024-03-30T02:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.169+0000 7f74bea00700 1 -- 172.21.15.67:0/131798825 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8017650 con 0x7f74c8100f60 2024-03-30T02:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.169+0000 7f74ccc00700 1 -- 172.21.15.67:0/131798825 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c81a63c0 con 0x7f74c8100f60 2024-03-30T02:37:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.172+0000 7f74bea00700 1 -- 172.21.15.67:0/131798825 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f74b80225e0 con 0x7f74c8100f60 2024-03-30T02:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.173+0000 7f74bea00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74ac0a13a0 0x7f74ac0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.173+0000 7f74bea00700 1 -- 172.21.15.67:0/131798825 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f74b80ce5b0 con 0x7f74c8100f60 2024-03-30T02:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.173+0000 7f74e07cab80 1 Processor -- start 2024-03-30T02:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.173+0000 7f74c5400700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74ac0a13a0 0x7f74ac0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74e07cab80 1 -- start start 2024-03-30T02:37:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d05d0 0x5649889d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74c5400700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74ac0a13a0 0x7f74ac0a3860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f74c80747e0 tx=0x7f74b4009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889db8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889dbdf0 0x5649889de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74e07cab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564988929990 con 0x5649889d05d0 2024-03-30T02:37:24.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74e07cab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564988927350 con 0x5649889d0ef0 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74e07cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564988922c40 con 0x5649889dbdf0 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d05d0 0x5649889d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.174+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d05d0 0x5649889d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35410/0 (socket says 172.21.15.67:35410) 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74c5e00700 1 -- 172.21.15.67:0/1931231915 learned_addr learned my addr 172.21.15.67:0/1931231915 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74c6800700 1 --2- 172.21.15.67:0/1931231915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889dbdf0 0x5649889de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74c5400700 1 --2- 172.21.15.67:0/1931231915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889db8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74c5e00700 1 -- 172.21.15.67:0/1931231915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889dbdf0 msgr2=0x5649889de1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74c5e00700 1 --2- 172.21.15.67:0/1931231915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889dbdf0 0x5649889de1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74c5e00700 1 -- 172.21.15.67:0/1931231915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 msgr2=0x5649889db8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74c5e00700 1 --2- 172.21.15.67:0/1931231915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889db8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74c5e00700 1 -- 172.21.15.67:0/1931231915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649887a43b0 con 0x5649889d05d0 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74c5e00700 1 --2- 172.21.15.67:0/1931231915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d05d0 0x5649889d09b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f74b8097850 tx=0x7f74b8022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ee442ec1c534d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74b3000700 1 -- 172.21.15.67:0/1931231915 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b809acb0 con 0x5649889d05d0 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74b3000700 1 -- 172.21.15.67:0/1931231915 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74b809ba50 con 0x5649889d05d0 2024-03-30T02:37:24.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.175+0000 7f74b3000700 1 -- 172.21.15.67:0/1931231915 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b809b5b0 con 0x5649889d05d0 2024-03-30T02:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.176+0000 7f74e07cab80 1 -- 172.21.15.67:0/1931231915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d05d0 msgr2=0x5649889d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.176+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1931231915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d05d0 0x5649889d09b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f74b8097850 tx=0x7f74b8022f90 comp rx=0 tx=0).stop 2024-03-30T02:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.176+0000 7f74e07cab80 1 -- 172.21.15.67:0/1931231915 shutdown_connections 2024-03-30T02:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.176+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1931231915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d05d0 0x5649889d09b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.176+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1931231915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889dbdf0 0x5649889de1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.176+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1931231915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889db8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.176+0000 7f74e07cab80 1 -- 172.21.15.67:0/1931231915 >> 172.21.15.67:0/1931231915 conn(0x5649889306a0 msgr2=0x5649889cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.176+0000 7f74e07cab80 1 -- 172.21.15.67:0/1931231915 shutdown_connections 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.176+0000 7f74e07cab80 1 -- 172.21.15.67:0/1931231915 wait complete. 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.177+0000 7f74e07cab80 1 -- start start 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.177+0000 7f74e07cab80 1 Processor -- start 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.177+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 0x5649889dc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.177+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d0ef0 0x5649889dc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889dbdf0 0x5649889e7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74e07cab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649888bed80 con 0x5649889d0ef0 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74e07cab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649888bea20 con 0x5649889dbdf0 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74e07cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649887a66a0 con 0x5649889d05d0 2024-03-30T02:37:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74e07cab80 10 client.0 _ll_register_callbacks cb 0x5649888c8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d0ef0 0x5649889dc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d0ef0 0x5649889dc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35416/0 (socket says 172.21.15.67:35416) 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5400700 1 -- 172.21.15.67:0/3888776788 learned_addr learned my addr 172.21.15.67:0/3888776788 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5e00700 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 0x5649889dc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c6800700 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889dbdf0 0x5649889e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5400700 1 -- 172.21.15.67:0/3888776788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 msgr2=0x5649889dc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5400700 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 0x5649889dc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5400700 1 -- 172.21.15.67:0/3888776788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889dbdf0 msgr2=0x5649889e7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5400700 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889dbdf0 0x5649889e7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5400700 1 -- 172.21.15.67:0/3888776788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c81a63c0 con 0x5649889d0ef0 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.178+0000 7f74c5400700 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d0ef0 0x5649889dc710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f74b4003930 tx=0x7f74b40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86074609f6f3b1ea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74a7400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74a7400700 1 -- 172.21.15.67:0/3888776788 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4011ee0 con 0x5649889d0ef0 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74a7400700 1 -- 172.21.15.67:0/3888776788 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f74b4018030 con 0x5649889d0ef0 2024-03-30T02:37:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74e07cab80 1 -- 172.21.15.67:0/3888776788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649888e3380 con 0x5649889d0ef0 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74a7400700 1 -- 172.21.15.67:0/3888776788 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b4012450 con 0x5649889d0ef0 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74e07cab80 10 client.15696 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74e07cab80 1 -- 172.21.15.67:0/3888776788 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564988750b60 con 0x5649889d0ef0 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74a7400700 1 -- 172.21.15.67:0/3888776788 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649888e3380 con 0x5649889d0ef0 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74a7400700 1 -- 172.21.15.67:0/3888776788 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74b4026020 con 0x5649889d0ef0 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74e07cab80 10 client.15696 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.179+0000 7f74e07cab80 10 client.15696 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.180+0000 7f74e07cab80 20 client.15696 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.180+0000 7f74e07cab80 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5649889eb670 0x5649889eda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.180+0000 7f74e07cab80 4 client.15696 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.180+0000 7f74e07cab80 1 -- 172.21.15.67:0/3888776788 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56498891b700 con 0x5649889eb670 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.180+0000 7f74c5e00700 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5649889eb670 0x5649889eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.180+0000 7f74c5e00700 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5649889eb670 0x5649889eda90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac56bdb2b427eb10 server_cookie=1e02c17295ca24ae in_seq=0 out_seq=0 2024-03-30T02:37:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.180+0000 7f74a7400700 10 client.15696 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74a7400700 1 -- 172.21.15.67:0/3888776788 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56498891b700 con 0x5649889eb670 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74a7400700 10 client.15696 handle_command_reply: tid=0 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 client.15696 shutdown 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 -- 172.21.15.67:0/3888776788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d0ef0 msgr2=0x5649889dc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d0ef0 0x5649889dc710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f74b4003930 tx=0x7f74b40047c0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 -- 172.21.15.67:0/3888776788 shutdown_connections 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5649889eb670 0x5649889eda90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889dbdf0 0x5649889e7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 0x5649889dc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74a7400700 0 client.15696 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 --2- 172.21.15.67:0/3888776788 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889d0ef0 0x5649889dc710 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 -- 172.21.15.67:0/3888776788 >> 172.21.15.67:0/3888776788 conn(0x5649889306a0 msgr2=0x5649889d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 -- 172.21.15.67:0/3888776788 shutdown_connections 2024-03-30T02:37:24.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.181+0000 7f74e07cab80 1 -- 172.21.15.67:0/3888776788 wait complete. 2024-03-30T02:37:24.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.182+0000 7f74e07cab80 20 client.15696 trim_cache size 0 max 16384 2024-03-30T02:37:24.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.195+0000 7f74e07cab80 1 Processor -- start 2024-03-30T02:37:24.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.195+0000 7f74e07cab80 1 -- start start 2024-03-30T02:37:24.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.195+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 0x5649889eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.195+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889e1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.195+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 0x5649889c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.195+0000 7f74e07cab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649889d1640 con 0x5649889dbdf0 2024-03-30T02:37:24.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.195+0000 7f74e07cab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649889e79f0 con 0x5649889d0ef0 2024-03-30T02:37:24.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.195+0000 7f74e07cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649889c8830 con 0x5649889d05d0 2024-03-30T02:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c6800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 0x5649889c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889e1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 0x5649889eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c6800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 0x5649889c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35418/0 (socket says 172.21.15.67:35418) 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c6800700 1 -- 172.21.15.67:0/2995501849 learned_addr learned my addr 172.21.15.67:0/2995501849 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c5400700 1 --2- 172.21.15.67:0/2995501849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889e1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43798/0 (socket says 172.21.15.67:43798) 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c5400700 1 -- 172.21.15.67:0/2995501849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 msgr2=0x5649889eda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c5400700 1 --2- 172.21.15.67:0/2995501849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 0x5649889eda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c5400700 1 -- 172.21.15.67:0/2995501849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 msgr2=0x5649889c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c5400700 1 --2- 172.21.15.67:0/2995501849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 0x5649889c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c5400700 1 -- 172.21.15.67:0/2995501849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649889d0440 con 0x5649889d0ef0 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.196+0000 7f74c5400700 1 --2- 172.21.15.67:0/2995501849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889e1040 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74b40096d0 tx=0x7f74b4011ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22fffe4b1d2089e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74b3a00700 1 -- 172.21.15.67:0/2995501849 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4004630 con 0x5649889d0ef0 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74b3a00700 1 -- 172.21.15.67:0/2995501849 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74b4003c40 con 0x5649889d0ef0 2024-03-30T02:37:24.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74b3a00700 1 -- 172.21.15.67:0/2995501849 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4012b70 con 0x5649889d0ef0 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74e07cab80 1 -- 172.21.15.67:0/2995501849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 msgr2=0x5649889e1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74e07cab80 1 --2- 172.21.15.67:0/2995501849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889e1040 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f74b40096d0 tx=0x7f74b4011ad0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74e07cab80 1 -- 172.21.15.67:0/2995501849 shutdown_connections 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74e07cab80 1 --2- 172.21.15.67:0/2995501849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 0x5649889c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74e07cab80 1 --2- 172.21.15.67:0/2995501849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d05d0 0x5649889eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74e07cab80 1 --2- 172.21.15.67:0/2995501849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d0ef0 0x5649889e1040 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74e07cab80 1 -- 172.21.15.67:0/2995501849 >> 172.21.15.67:0/2995501849 conn(0x5649889306a0 msgr2=0x5649889d5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74e07cab80 1 -- 172.21.15.67:0/2995501849 shutdown_connections 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.197+0000 7f74e07cab80 1 -- 172.21.15.67:0/2995501849 wait complete. 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.198+0000 7f74e07cab80 1 -- start start 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.198+0000 7f74e07cab80 1 Processor -- start 2024-03-30T02:37:24.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.198+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d05d0 0x564988a33620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.198+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d0ef0 0x564988a33b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.198+0000 7f74e07cab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 0x564988a38560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.198+0000 7f74e07cab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564988a38aa0 con 0x5649889dbdf0 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.198+0000 7f74e07cab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564988a38c10 con 0x5649889d05d0 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.198+0000 7f74e07cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564988a38dd0 con 0x5649889d0ef0 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.198+0000 7f74e07cab80 10 client.0 _ll_register_callbacks cb 0x5649888c8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d0ef0 0x564988a33b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c6800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 0x564988a38560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d0ef0 0x564988a33b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52068/0 (socket says 172.21.15.67:52068) 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5400700 1 -- 172.21.15.67:0/1465010998 learned_addr learned my addr 172.21.15.67:0/1465010998 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5e00700 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d05d0 0x564988a33620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5400700 1 -- 172.21.15.67:0/1465010998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d05d0 msgr2=0x564988a33620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5400700 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d05d0 0x564988a33620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5400700 1 -- 172.21.15.67:0/1465010998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 msgr2=0x564988a38560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5400700 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 0x564988a38560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5400700 1 -- 172.21.15.67:0/1465010998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b402d040 con 0x5649889d0ef0 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74c5400700 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d0ef0 0x564988a33b60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74b4003900 tx=0x7f74b4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ad9afb067d9364 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.199+0000 7f74a7400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.200+0000 7f74a7400700 1 -- 172.21.15.67:0/1465010998 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4028a20 con 0x5649889d0ef0 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.200+0000 7f74e07cab80 1 -- 172.21.15.67:0/1465010998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74b809ba50 con 0x5649889d0ef0 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.200+0000 7f74a7400700 1 -- 172.21.15.67:0/1465010998 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f74b4018070 con 0x5649889d0ef0 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.200+0000 7f74a7400700 1 -- 172.21.15.67:0/1465010998 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b4012450 con 0x5649889d0ef0 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.200+0000 7f74e07cab80 10 client.25741 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:24.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.200+0000 7f74e07cab80 1 -- 172.21.15.67:0/1465010998 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5649888e8ea0 con 0x5649889d0ef0 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.200+0000 7f74a7400700 1 -- 172.21.15.67:0/1465010998 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74b809ba50 con 0x5649889d0ef0 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74a7400700 1 -- 172.21.15.67:0/1465010998 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74b401e040 con 0x5649889d0ef0 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74e07cab80 10 client.25741 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74e07cab80 10 client.25741 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74e07cab80 20 client.25741 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x564988a401b0 0x564988a425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74e07cab80 4 client.25741 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74e07cab80 1 -- 172.21.15.67:0/1465010998 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564988771cf0 con 0x564988a401b0 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74c5e00700 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x564988a401b0 0x564988a425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74a7400700 10 client.25741 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.201+0000 7f74c5e00700 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x564988a401b0 0x564988a425a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d26ba151e6ad0008 server_cookie=7a8d3671afafd8e8 in_seq=0 out_seq=0 2024-03-30T02:37:24.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74a7400700 1 -- 172.21.15.67:0/1465010998 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564988771cf0 con 0x564988a401b0 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74a7400700 10 client.25741 handle_command_reply: tid=0 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 client.25741 shutdown 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 -- 172.21.15.67:0/1465010998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d0ef0 msgr2=0x564988a33b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d0ef0 0x564988a33b60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74b4003900 tx=0x7f74b4014000 comp rx=0 tx=0).stop 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 -- 172.21.15.67:0/1465010998 shutdown_connections 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649889dbdf0 0x564988a38560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x564988a401b0 0x564988a425a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649889d05d0 0x564988a33620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 --2- 172.21.15.67:0/1465010998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649889d0ef0 0x564988a33b60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 -- 172.21.15.67:0/1465010998 >> 172.21.15.67:0/1465010998 conn(0x5649889306a0 msgr2=0x5649889c55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74a7400700 0 client.25741 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 -- 172.21.15.67:0/1465010998 shutdown_connections 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.202+0000 7f74e07cab80 1 -- 172.21.15.67:0/1465010998 wait complete. 2024-03-30T02:37:24.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74e07cab80 20 client.25741 trim_cache size 0 max 16384 2024-03-30T02:37:24.207 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 -- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74ac0a13a0 msgr2=0x7f74ac0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74ac0a13a0 0x7f74ac0a3860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f74c80747e0 tx=0x7f74b4009260 comp rx=0 tx=0).stop 2024-03-30T02:37:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 -- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 msgr2=0x7f74c80730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 0x7f74c80730b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f74b8005910 tx=0x7f74b8004730 comp rx=0 tx=0).stop 2024-03-30T02:37:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 -- 172.21.15.67:0/131798825 shutdown_connections 2024-03-30T02:37:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74ac0a13a0 0x7f74ac0a3860 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c8100f60 0x7f74c80730b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c8073b30 0x7f74c81a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 --2- 172.21.15.67:0/131798825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8106080 0x7f74c80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 -- 172.21.15.67:0/131798825 >> 172.21.15.67:0/131798825 conn(0x7f74c80fc7c0 msgr2=0x7f74c80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 -- 172.21.15.67:0/131798825 shutdown_connections 2024-03-30T02:37:24.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.203+0000 7f74ccc00700 1 -- 172.21.15.67:0/131798825 wait complete. 2024-03-30T02:37:24.223 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:37:24.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.425+0000 7f0138600700 1 -- 172.21.15.67:0/2505313644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130101920 msgr2=0x7f0130105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.425+0000 7f0138600700 1 --2- 172.21.15.67:0/2505313644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130101920 0x7f0130105960 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f011c009990 tx=0x7f011c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.425+0000 7f0138600700 1 -- 172.21.15.67:0/2505313644 shutdown_connections 2024-03-30T02:37:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.425+0000 7f0138600700 1 --2- 172.21.15.67:0/2505313644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0130106090 0x7f013010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.425+0000 7f0138600700 1 --2- 172.21.15.67:0/2505313644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0130100f70 0x7f0130101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.425+0000 7f0138600700 1 --2- 172.21.15.67:0/2505313644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130101920 0x7f0130105960 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.425+0000 7f0138600700 1 -- 172.21.15.67:0/2505313644 >> 172.21.15.67:0/2505313644 conn(0x7f01300fc7f0 msgr2=0x7f01300fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.425+0000 7f0138600700 1 -- 172.21.15.67:0/2505313644 shutdown_connections 2024-03-30T02:37:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.425+0000 7f0138600700 1 -- 172.21.15.67:0/2505313644 wait complete. 2024-03-30T02:37:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.426+0000 7f0138600700 1 Processor -- start 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0138600700 1 -- start start 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0138600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130100f70 0x7f0130199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0138600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0130106090 0x7f0130199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0138600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f013019dce0 0x7f013019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0138600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013019e760 con 0x7f0130100f70 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0138600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013019e8d0 con 0x7f0130106090 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0138600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013019ea90 con 0x7f013019dce0 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0135c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130100f70 0x7f0130199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0135c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130100f70 0x7f0130199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35438/0 (socket says 172.21.15.67:35438) 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0135c00700 1 -- 172.21.15.67:0/1279381136 learned_addr learned my addr 172.21.15.67:0/1279381136 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0135200700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0130106090 0x7f0130199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.427+0000 7f0136600700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f013019dce0 0x7f013019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f0135c00700 1 -- 172.21.15.67:0/1279381136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f013019dce0 msgr2=0x7f013019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f0135c00700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f013019dce0 0x7f013019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f0135c00700 1 -- 172.21.15.67:0/1279381136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0130106090 msgr2=0x7f0130199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f0135c00700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0130106090 0x7f0130199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f0135c00700 1 -- 172.21.15.67:0/1279381136 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c0095f0 con 0x7f0130100f70 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f0135200700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0130106090 0x7f0130199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f0135c00700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130100f70 0x7f0130199400 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f012400e9a0 tx=0x7f012400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f012e000700 1 -- 172.21.15.67:0/1279381136 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124009790 con 0x7f0130100f70 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f012e000700 1 -- 172.21.15.67:0/1279381136 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0124010430 con 0x7f0130100f70 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f0138600700 1 -- 172.21.15.67:0/1279381136 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01301a5ff0 con 0x7f0130100f70 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f012e000700 1 -- 172.21.15.67:0/1279381136 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124018680 con 0x7f0130100f70 2024-03-30T02:37:24.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.428+0000 7f0138600700 1 -- 172.21.15.67:0/1279381136 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01301a64c0 con 0x7f0130100f70 2024-03-30T02:37:24.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.431+0000 7f012e000700 1 -- 172.21.15.67:0/1279381136 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f01240105a0 con 0x7f0130100f70 2024-03-30T02:37:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.433+0000 7f014c1d5b80 1 Processor -- start 2024-03-30T02:37:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f014c1d5b80 1 -- start start 2024-03-30T02:37:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f012e000700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f01180a13a0 0x7f01180a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f012e000700 1 -- 172.21.15.67:0/1279381136 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f0124014070 con 0x7f0130100f70 2024-03-30T02:37:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574351785d0 0x5574351789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f0136600700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f01180a13a0 0x7f01180a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435178ef0 0x5574351838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 0x5574351861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574350d1990 con 0x5574351785d0 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574350cf350 con 0x557435183df0 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574350cac40 con 0x557435178ef0 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.434+0000 7f0135c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574351785d0 0x5574351789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0135200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435178ef0 0x5574351838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0135c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574351785d0 0x5574351789b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35446/0 (socket says 172.21.15.67:35446) 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0135c00700 1 -- 172.21.15.67:0/2069568985 learned_addr learned my addr 172.21.15.67:0/2069568985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0136600700 1 --2- 172.21.15.67:0/2069568985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 0x5574351861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0135c00700 1 -- 172.21.15.67:0/2069568985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435178ef0 msgr2=0x5574351838b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0135c00700 1 --2- 172.21.15.67:0/2069568985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435178ef0 0x5574351838b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0135c00700 1 -- 172.21.15.67:0/2069568985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 msgr2=0x5574351861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0135c00700 1 --2- 172.21.15.67:0/2069568985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 0x5574351861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0135c00700 1 -- 172.21.15.67:0/2069568985 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557434f4c3b0 con 0x5574351785d0 2024-03-30T02:37:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0136600700 1 --2- 172.21.15.67:0/2069568985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 0x5574351861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0136600700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f01180a13a0 0x7f01180a3860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f012800f540 tx=0x7f0128009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0135c00700 1 --2- 172.21.15.67:0/2069568985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574351785d0 0x5574351789b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0124099c60 tx=0x7f01240999b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed4fb09e607102b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0122a00700 1 -- 172.21.15.67:0/2069568985 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01240cd170 con 0x5574351785d0 2024-03-30T02:37:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0122a00700 1 -- 172.21.15.67:0/2069568985 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f012409dd90 con 0x5574351785d0 2024-03-30T02:37:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.435+0000 7f0122a00700 1 -- 172.21.15.67:0/2069568985 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124099450 con 0x5574351785d0 2024-03-30T02:37:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.436+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2069568985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574351785d0 msgr2=0x5574351789b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.436+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/2069568985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574351785d0 0x5574351789b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0124099c60 tx=0x7f01240999b0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.436+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2069568985 shutdown_connections 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.436+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/2069568985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5574351785d0 0x5574351789b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.436+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/2069568985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 0x5574351861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.436+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/2069568985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435178ef0 0x5574351838b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.436+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2069568985 >> 172.21.15.67:0/2069568985 conn(0x5574350d86a0 msgr2=0x557435174a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.436+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2069568985 shutdown_connections 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.436+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2069568985 wait complete. 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.437+0000 7f014c1d5b80 1 -- start start 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.437+0000 7f014c1d5b80 1 Processor -- start 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.437+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574351785d0 0x5574351837d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.437+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x557435180b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.437+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 0x557435181190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.437+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557435066d80 con 0x557435178ef0 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.437+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557435066a20 con 0x557435183df0 2024-03-30T02:37:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.437+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557434f4e6a0 con 0x5574351785d0 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f014c1d5b80 10 client.0 _ll_register_callbacks cb 0x557435070470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x557435180b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x557435180b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35450/0 (socket says 172.21.15.67:35450) 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135200700 1 -- 172.21.15.67:0/1047101744 learned_addr learned my addr 172.21.15.67:0/1047101744 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135c00700 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574351785d0 0x5574351837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0136600700 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 0x557435181190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135200700 1 -- 172.21.15.67:0/1047101744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574351785d0 msgr2=0x5574351837d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135200700 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574351785d0 0x5574351837d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135200700 1 -- 172.21.15.67:0/1047101744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 msgr2=0x557435181190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135200700 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 0x557435181190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135200700 1 -- 172.21.15.67:0/1047101744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01301a64c0 con 0x557435178ef0 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0135200700 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x557435180b30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f011c009990 tx=0x7f011c025b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7552043961e065b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.438+0000 7f0117400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:37:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f0117400700 1 -- 172.21.15.67:0/1047101744 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c02a070 con 0x557435178ef0 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f0117400700 1 -- 172.21.15.67:0/1047101744 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f011c005250 con 0x557435178ef0 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1047101744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55743508b380 con 0x557435178ef0 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f0117400700 1 -- 172.21.15.67:0/1047101744 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f011c005410 con 0x557435178ef0 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f014c1d5b80 10 client.15714 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1047101744 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557434ef8b60 con 0x557435178ef0 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f0117400700 1 -- 172.21.15.67:0/1047101744 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55743508b380 con 0x557435178ef0 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f0117400700 1 -- 172.21.15.67:0/1047101744 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f011c004cf0 con 0x557435178ef0 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f014c1d5b80 10 client.15714 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f014c1d5b80 10 client.15714 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.439+0000 7f014c1d5b80 20 client.15714 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.440+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557435193ab0 0x557435195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.440+0000 7f014c1d5b80 4 client.15714 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.440+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1047101744 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574350c3700 con 0x557435193ab0 2024-03-30T02:37:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.440+0000 7f0135c00700 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557435193ab0 0x557435195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.440+0000 7f0135c00700 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557435193ab0 0x557435195ed0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=432a4eca908db187 server_cookie=17c532ce4800b53a in_seq=0 out_seq=0 2024-03-30T02:37:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.440+0000 7f0117400700 10 client.15714 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:37:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f0117400700 1 -- 172.21.15.67:0/1047101744 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5574350c3700 con 0x557435193ab0 2024-03-30T02:37:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f0117400700 10 client.15714 handle_command_reply: tid=0 2024-03-30T02:37:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 client.15714 shutdown 2024-03-30T02:37:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1047101744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 msgr2=0x557435180b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x557435180b30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f011c009990 tx=0x7f011c025b80 comp rx=0 tx=0).stop 2024-03-30T02:37:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1047101744 shutdown_connections 2024-03-30T02:37:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557435193ab0 0x557435195ed0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557435183df0 0x557435181190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574351785d0 0x5574351837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1047101744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x557435180b30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f0117400700 0 client.15714 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:37:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1047101744 >> 172.21.15.67:0/1047101744 conn(0x5574350d86a0 msgr2=0x55743517a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1047101744 shutdown_connections 2024-03-30T02:37:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.441+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1047101744 wait complete. 2024-03-30T02:37:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.442+0000 7f014c1d5b80 20 client.15714 trim_cache size 0 max 16384 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.455+0000 7f014c1d5b80 1 Processor -- start 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.455+0000 7f014c1d5b80 1 -- start start 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.455+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x557435181130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.455+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.455+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 0x557435183470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.455+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557435181c40 con 0x557435178ef0 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.455+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743517b9d0 con 0x5574351785d0 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.455+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743518afb0 con 0x557435183df0 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35466/0 (socket says 172.21.15.67:35466) 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135200700 1 -- 172.21.15.67:0/2437158917 learned_addr learned my addr 172.21.15.67:0/2437158917 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135c00700 1 --2- 172.21.15.67:0/2437158917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x557435181130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0136600700 1 --2- 172.21.15.67:0/2437158917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 0x557435183470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135c00700 1 -- 172.21.15.67:0/2437158917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 msgr2=0x557435183470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135c00700 1 --2- 172.21.15.67:0/2437158917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 0x557435183470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135c00700 1 -- 172.21.15.67:0/2437158917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 msgr2=0x55743518aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135c00700 1 --2- 172.21.15.67:0/2437158917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135c00700 1 -- 172.21.15.67:0/2437158917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557435090ea0 con 0x5574351785d0 2024-03-30T02:37:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0135c00700 1 --2- 172.21.15.67:0/2437158917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x557435181130 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55743517a640 tx=0x7f0124010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a78ec560806895 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0123400700 1 -- 172.21.15.67:0/2437158917 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01240cd170 con 0x5574351785d0 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.456+0000 7f0123400700 1 -- 172.21.15.67:0/2437158917 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0124010880 con 0x5574351785d0 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f0123400700 1 -- 172.21.15.67:0/2437158917 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012409a1c0 con 0x5574351785d0 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2437158917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 msgr2=0x557435181130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/2437158917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x557435181130 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55743517a640 tx=0x7f0124010c80 comp rx=0 tx=0).stop 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2437158917 shutdown_connections 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/2437158917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 0x557435183470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/2437158917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x557435181130 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/2437158917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2437158917 >> 172.21.15.67:0/2437158917 conn(0x5574350d86a0 msgr2=0x55743516d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2437158917 shutdown_connections 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.457+0000 7f014c1d5b80 1 -- 172.21.15.67:0/2437158917 wait complete. 2024-03-30T02:37:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.458+0000 7f014c1d5b80 1 -- start start 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.458+0000 7f014c1d5b80 1 Processor -- start 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x55743518a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f014c1d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 0x557435188a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557435189130 con 0x557435178ef0 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574351892a0 con 0x5574351785d0 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f014c1d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557435189460 con 0x557435183df0 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f014c1d5b80 10 client.0 _ll_register_callbacks cb 0x557435070470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x55743518a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35470/0 (socket says 172.21.15.67:35470) 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135200700 1 -- 172.21.15.67:0/1234084415 learned_addr learned my addr 172.21.15.67:0/1234084415 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0136600700 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 0x557435188a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135c00700 1 -- 172.21.15.67:0/1234084415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 msgr2=0x557435188a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135c00700 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 0x557435188a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135c00700 1 -- 172.21.15.67:0/1234084415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 msgr2=0x55743518a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135c00700 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135c00700 1 -- 172.21.15.67:0/1234084415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557434f4c3b0 con 0x5574351785d0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.459+0000 7f0135c00700 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x55743518a260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f012400c2d0 tx=0x7f012409a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5a3857dce9729fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f0117400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f0117400700 1 -- 172.21.15.67:0/1234084415 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124099d30 con 0x5574351785d0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f0117400700 1 -- 172.21.15.67:0/1234084415 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f012409a1c0 con 0x5574351785d0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1234084415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f012409dd90 con 0x5574351785d0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f0117400700 1 -- 172.21.15.67:0/1234084415 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f012409a3a0 con 0x5574351785d0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f014c1d5b80 10 client.25769 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1234084415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557434f50cd0 con 0x5574351785d0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f0117400700 1 -- 172.21.15.67:0/1234084415 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f012409dd90 con 0x5574351785d0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f0117400700 1 -- 172.21.15.67:0/1234084415 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01240a1f50 con 0x5574351785d0 2024-03-30T02:37:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f014c1d5b80 10 client.25769 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f014c1d5b80 10 client.25769 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f014c1d5b80 20 client.25769 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5574351e8540 0x5574351ea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f014c1d5b80 4 client.25769 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.460+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1234084415 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557434f19cf0 con 0x5574351e8540 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f0135200700 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5574351e8540 0x5574351ea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f0117400700 10 client.25769 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f0135200700 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5574351e8540 0x5574351ea930 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3fe41f90b41917c0 server_cookie=bbec4ad03c9aeb5 in_seq=0 out_seq=0 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f0117400700 1 -- 172.21.15.67:0/1234084415 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557434f19cf0 con 0x5574351e8540 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f0117400700 10 client.25769 handle_command_reply: tid=0 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f014c1d5b80 1 client.25769 shutdown 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1234084415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 msgr2=0x55743518a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x55743518a260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f012400c2d0 tx=0x7f012409a0b0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1234084415 shutdown_connections 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5574351e8540 0x5574351ea930 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557435183df0 0x557435188a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5574351785d0 0x55743518a260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f014c1d5b80 1 --2- 172.21.15.67:0/1234084415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557435178ef0 0x55743518a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f0117400700 0 client.25769 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.461+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1234084415 >> 172.21.15.67:0/1234084415 conn(0x5574350d86a0 msgr2=0x55743517f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1234084415 shutdown_connections 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f014c1d5b80 1 -- 172.21.15.67:0/1234084415 wait complete. 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f014c1d5b80 20 client.25769 trim_cache size 0 max 16384 2024-03-30T02:37:24.467 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 -- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f01180a13a0 msgr2=0x7f01180a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f01180a13a0 0x7f01180a3860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f012800f540 tx=0x7f0128009920 comp rx=0 tx=0).stop 2024-03-30T02:37:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 -- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130100f70 msgr2=0x7f0130199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130100f70 0x7f0130199400 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f012400e9a0 tx=0x7f012400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 -- 172.21.15.67:0/1279381136 shutdown_connections 2024-03-30T02:37:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f01180a13a0 0x7f01180a3860 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0130100f70 0x7f0130199400 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f013019dce0 0x7f013019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 --2- 172.21.15.67:0/1279381136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0130106090 0x7f0130199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 -- 172.21.15.67:0/1279381136 >> 172.21.15.67:0/1279381136 conn(0x7f01300fc7f0 msgr2=0x7f01300feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.462+0000 7f0138600700 1 -- 172.21.15.67:0/1279381136 shutdown_connections 2024-03-30T02:37:24.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.463+0000 7f0138600700 1 -- 172.21.15.67:0/1279381136 wait complete. 2024-03-30T02:37:24.479 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:24 smithi067 ceph-mon[37808]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:24.479 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:24 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4094009837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:37:24.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:37:24.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.680+0000 7f0304200700 1 -- 172.21.15.67:0/2983793678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02fc101900 msgr2=0x7f02fc105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.680+0000 7f0304200700 1 --2- 172.21.15.67:0/2983793678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02fc101900 0x7f02fc105940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f02e80098f0 tx=0x7f02e8009c00 comp rx=0 tx=0).stop 2024-03-30T02:37:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.681+0000 7f0304200700 1 -- 172.21.15.67:0/2983793678 shutdown_connections 2024-03-30T02:37:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.681+0000 7f0304200700 1 --2- 172.21.15.67:0/2983793678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02fc100f50 0x7f02fc101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.681+0000 7f0304200700 1 --2- 172.21.15.67:0/2983793678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc106070 0x7f02fc10dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.681+0000 7f0304200700 1 --2- 172.21.15.67:0/2983793678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02fc101900 0x7f02fc105940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.681+0000 7f0304200700 1 -- 172.21.15.67:0/2983793678 >> 172.21.15.67:0/2983793678 conn(0x7f02fc0fc7f0 msgr2=0x7f02fc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.681+0000 7f0304200700 1 -- 172.21.15.67:0/2983793678 shutdown_connections 2024-03-30T02:37:24.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.681+0000 7f0304200700 1 -- 172.21.15.67:0/2983793678 wait complete. 2024-03-30T02:37:24.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:24 smithi170 ceph-mon[78798]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:24.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:24 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4094009837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.682+0000 7f0304200700 1 Processor -- start 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.682+0000 7f0304200700 1 -- start start 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.682+0000 7f0304200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc100f50 0x7f02fc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.682+0000 7f0304200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02fc106070 0x7f02fc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.682+0000 7f0304200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02fc073b30 0x7f02fc1a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.682+0000 7f0304200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc1a5a50 con 0x7f02fc106070 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.682+0000 7f0304200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc1a5bc0 con 0x7f02fc073b30 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.682+0000 7f0304200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc1a5d80 con 0x7f02fc100f50 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0301800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc100f50 0x7f02fc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0300e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02fc106070 0x7f02fc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0301800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc100f50 0x7f02fc0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52160/0 (socket says 172.21.15.67:52160) 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0301800700 1 -- 172.21.15.67:0/3019945991 learned_addr learned my addr 172.21.15.67:0/3019945991 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0302200700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02fc073b30 0x7f02fc1a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0301800700 1 -- 172.21.15.67:0/3019945991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02fc073b30 msgr2=0x7f02fc1a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0301800700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02fc073b30 0x7f02fc1a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0301800700 1 -- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02fc106070 msgr2=0x7f02fc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0301800700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02fc106070 0x7f02fc0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0301800700 1 -- 172.21.15.67:0/3019945991 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f00095f0 con 0x7f02fc100f50 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0300e00700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02fc106070 0x7f02fc0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.683+0000 7f0301800700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc100f50 0x7f02fc0730b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f02f000ec10 tx=0x7f02f000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.684+0000 7f02fa000700 1 -- 172.21.15.67:0/3019945991 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f000cc90 con 0x7f02fc100f50 2024-03-30T02:37:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.684+0000 7f02fa000700 1 -- 172.21.15.67:0/3019945991 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02f00044e0 con 0x7f02fc100f50 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.684+0000 7f0304200700 1 -- 172.21.15.67:0/3019945991 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e8009520 con 0x7f02fc100f50 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.684+0000 7f02fa000700 1 -- 172.21.15.67:0/3019945991 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0010430 con 0x7f02fc100f50 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.684+0000 7f0304200700 1 -- 172.21.15.67:0/3019945991 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc1a6400 con 0x7f02fc100f50 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.688+0000 7f02fa000700 1 -- 172.21.15.67:0/3019945991 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f02f0003fb0 con 0x7f02fc100f50 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.689+0000 7f0317dc4b80 1 Processor -- start 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.689+0000 7f0317dc4b80 1 -- start start 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f02fa000700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f02e40a13a0 0x7f02e40a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f02fa000700 1 -- 172.21.15.67:0/3019945991 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f02f0014070 con 0x7f02fc100f50 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09565b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 0x55f0956668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0302200700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f02e40a13a0 0x7f02e40a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 0x55f0956691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0955b4990 con 0x55f095666df0 2024-03-30T02:37:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0955b2350 con 0x55f09565b5d0 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0955adc40 con 0x55f09565bef0 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0301800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09565b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0300e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 0x55f0956668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0302200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 0x55f0956691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0301800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09565b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43884/0 (socket says 172.21.15.67:43884) 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.690+0000 7f0301800700 1 -- 172.21.15.67:0/1855977364 learned_addr learned my addr 172.21.15.67:0/1855977364 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f0301800700 1 -- 172.21.15.67:0/1855977364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 msgr2=0x55f0956668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f0301800700 1 --2- 172.21.15.67:0/1855977364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 0x55f0956668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f0301800700 1 -- 172.21.15.67:0/1855977364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 msgr2=0x55f0956691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f0301800700 1 --2- 172.21.15.67:0/1855977364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 0x55f0956691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f0301800700 1 -- 172.21.15.67:0/1855977364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f09542f3b0 con 0x55f09565b5d0 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f0301800700 1 --2- 172.21.15.67:0/1855977364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09565b9b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f02f0096d50 tx=0x7f02f0096020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4fa4af8e4bae466 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f0302200700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f02e40a13a0 0x7f02e40a3860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009ec0 tx=0x7f02f4016450 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f02eea00700 1 -- 172.21.15.67:0/1855977364 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0016330 con 0x55f09565b5d0 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f02eea00700 1 -- 172.21.15.67:0/1855977364 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02f00b2e90 con 0x55f09565b5d0 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.691+0000 7f02eea00700 1 -- 172.21.15.67:0/1855977364 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0098680 con 0x55f09565b5d0 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 -- 172.21.15.67:0/1855977364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 msgr2=0x55f09565b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/1855977364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09565b9b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f02f0096d50 tx=0x7f02f0096020 comp rx=0 tx=0).stop 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 -- 172.21.15.67:0/1855977364 shutdown_connections 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/1855977364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 0x55f0956691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/1855977364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09565b9b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/1855977364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 0x55f0956668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 -- 172.21.15.67:0/1855977364 >> 172.21.15.67:0/1855977364 conn(0x55f0955bb6a0 msgr2=0x55f095657a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 -- 172.21.15.67:0/1855977364 shutdown_connections 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 -- 172.21.15.67:0/1855977364 wait complete. 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.692+0000 7f0317dc4b80 1 -- start start 2024-03-30T02:37:24.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0317dc4b80 1 Processor -- start 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f095666810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 0x55f095665320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f095666df0 0x55f0956659a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f095549d80 con 0x55f09565bef0 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f095549a20 con 0x55f09565b5d0 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0954316a0 con 0x55f095666df0 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0317dc4b80 10 client.0 _ll_register_callbacks cb 0x55f095553470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0300e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 0x55f095665320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0302200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f095666df0 0x55f0956659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0301800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f095666810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0301800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f095666810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43896/0 (socket says 172.21.15.67:43896) 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.693+0000 7f0301800700 1 -- 172.21.15.67:0/2737317612 learned_addr learned my addr 172.21.15.67:0/2737317612 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f0302200700 1 -- 172.21.15.67:0/2737317612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 msgr2=0x55f095666810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f0302200700 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f095666810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f0302200700 1 -- 172.21.15.67:0/2737317612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 msgr2=0x55f095665320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f0302200700 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 0x55f095665320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f0302200700 1 -- 172.21.15.67:0/2737317612 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc1a6400 con 0x55f095666df0 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f0301800700 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f095666810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f0302200700 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f095666df0 0x55f0956659a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f02f401aa50 tx=0x7f02f4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=713d22c40478150e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f02e3400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f02e3400700 1 -- 172.21.15.67:0/2737317612 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f400e040 con 0x55f095666df0 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f0317dc4b80 1 -- 172.21.15.67:0/2737317612 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f09556e380 con 0x55f095666df0 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.694+0000 7f02e3400700 1 -- 172.21.15.67:0/2737317612 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f02f4021030 con 0x55f095666df0 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f02e3400700 1 -- 172.21.15.67:0/2737317612 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02f401aee0 con 0x55f095666df0 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f0317dc4b80 10 client.25771 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f0317dc4b80 1 -- 172.21.15.67:0/2737317612 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f0953dbb60 con 0x55f095666df0 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f02e3400700 1 -- 172.21.15.67:0/2737317612 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f09556e380 con 0x55f095666df0 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f02e3400700 1 -- 172.21.15.67:0/2737317612 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02f40042f0 con 0x55f095666df0 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f0317dc4b80 10 client.25771 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f0317dc4b80 10 client.25771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f0317dc4b80 20 client.25771 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f095676930 0x55f095678d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f0317dc4b80 4 client.25771 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.695+0000 7f0317dc4b80 1 -- 172.21.15.67:0/2737317612 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0955a6700 con 0x55f095676930 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.696+0000 7f0301800700 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f095676930 0x55f095678d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.696+0000 7f0301800700 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f095676930 0x55f095678d50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15022eafb085e614 server_cookie=f9f7436648fac1ab in_seq=0 out_seq=0 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.696+0000 7f02e3400700 10 client.25771 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f02e3400700 1 -- 172.21.15.67:0/2737317612 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f0955a6700 con 0x55f095676930 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f02e3400700 10 client.25771 handle_command_reply: tid=0 2024-03-30T02:37:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 client.25771 shutdown 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 -- 172.21.15.67:0/2737317612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f095666df0 msgr2=0x55f0956659a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f095666df0 0x55f0956659a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f02f401aa50 tx=0x7f02f4004b40 comp rx=0 tx=0).stop 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 -- 172.21.15.67:0/2737317612 shutdown_connections 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f095676930 0x55f095678d50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f095666df0 0x55f0956659a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f095666810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/2737317612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 0x55f095665320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f02e3400700 0 client.25771 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 -- 172.21.15.67:0/2737317612 >> 172.21.15.67:0/2737317612 conn(0x55f0955bb6a0 msgr2=0x55f09565d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 -- 172.21.15.67:0/2737317612 shutdown_connections 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 1 -- 172.21.15.67:0/2737317612 wait complete. 2024-03-30T02:37:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.697+0000 7f0317dc4b80 20 client.25771 trim_cache size 0 max 16384 2024-03-30T02:37:24.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0317dc4b80 1 Processor -- start 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0317dc4b80 1 -- start start 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565b5d0 0x55f095678d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 0x55f09566da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f095666df0 0x55f095665320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09565c850 con 0x55f09565bef0 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f095664ce0 con 0x55f095666df0 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09566dfb0 con 0x55f09565b5d0 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0301800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565b5d0 0x55f095678d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0300e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 0x55f09566da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0301800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565b5d0 0x55f095678d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52180/0 (socket says 172.21.15.67:52180) 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0301800700 1 -- 172.21.15.67:0/4029026577 learned_addr learned my addr 172.21.15.67:0/4029026577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.711+0000 7f0302200700 1 --2- 172.21.15.67:0/4029026577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f095666df0 0x55f095665320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f0301800700 1 -- 172.21.15.67:0/4029026577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f095666df0 msgr2=0x55f095665320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f0301800700 1 --2- 172.21.15.67:0/4029026577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f095666df0 0x55f095665320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f0301800700 1 -- 172.21.15.67:0/4029026577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 msgr2=0x55f09566da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f0301800700 1 --2- 172.21.15.67:0/4029026577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 0x55f09566da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f0301800700 1 -- 172.21.15.67:0/4029026577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f095573ea0 con 0x55f09565b5d0 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f0302200700 1 --2- 172.21.15.67:0/4029026577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f095666df0 0x55f095665320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f0301800700 1 --2- 172.21.15.67:0/4029026577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565b5d0 0x55f095678d20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f02f0099fe0 tx=0x7f02f0085040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7033611a398c1ad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f02ef400700 1 -- 172.21.15.67:0/4029026577 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00b7040 con 0x55f09565b5d0 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f02ef400700 1 -- 172.21.15.67:0/4029026577 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02f00ca910 con 0x55f09565b5d0 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.712+0000 7f02ef400700 1 -- 172.21.15.67:0/4029026577 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00036f0 con 0x55f09565b5d0 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 -- 172.21.15.67:0/4029026577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565b5d0 msgr2=0x55f095678d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/4029026577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565b5d0 0x55f095678d20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f02f0099fe0 tx=0x7f02f0085040 comp rx=0 tx=0).stop 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 -- 172.21.15.67:0/4029026577 shutdown_connections 2024-03-30T02:37:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/4029026577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f095666df0 0x55f095665320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/4029026577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565b5d0 0x55f095678d20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/4029026577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f09565bef0 0x55f09566da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 -- 172.21.15.67:0/4029026577 >> 172.21.15.67:0/4029026577 conn(0x55f0955bb6a0 msgr2=0x55f09565ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 -- 172.21.15.67:0/4029026577 shutdown_connections 2024-03-30T02:37:24.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 -- 172.21.15.67:0/4029026577 wait complete. 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.713+0000 7f0317dc4b80 1 -- start start 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.714+0000 7f0317dc4b80 1 Processor -- start 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.714+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09566bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.714+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 0x55f09566c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.714+0000 7f0317dc4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 0x55f0956c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.714+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f09566c8f0 con 0x55f095666df0 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.714+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0956c9ca0 con 0x55f09565b5d0 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.714+0000 7f0317dc4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0956c9e60 con 0x55f09565bef0 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.714+0000 7f0317dc4b80 10 client.0 _ll_register_callbacks cb 0x55f095553470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0302200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 0x55f0956c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0300e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 0x55f09566c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0302200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 0x55f0956c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35530/0 (socket says 172.21.15.67:35530) 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0302200700 1 -- 172.21.15.67:0/3247761973 learned_addr learned my addr 172.21.15.67:0/3247761973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0301800700 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09566bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0301800700 1 -- 172.21.15.67:0/3247761973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 msgr2=0x55f09566c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0301800700 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 0x55f09566c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0301800700 1 -- 172.21.15.67:0/3247761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 msgr2=0x55f0956c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0301800700 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 0x55f0956c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0301800700 1 -- 172.21.15.67:0/3247761973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f09542f3b0 con 0x55f09565b5d0 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0300e00700 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 0x55f09566c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0301800700 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09566bc40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02f00967e0 tx=0x7f02f000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661262ae69d182e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f02e3400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f02e3400700 1 -- 172.21.15.67:0/3247761973 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00b7070 con 0x55f09565b5d0 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f02e3400700 1 -- 172.21.15.67:0/3247761973 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f02f009a9e0 con 0x55f09565b5d0 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.715+0000 7f0317dc4b80 1 -- 172.21.15.67:0/3247761973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0953fccf0 con 0x55f09565b5d0 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f02e3400700 1 -- 172.21.15.67:0/3247761973 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02f00ca910 con 0x55f09565b5d0 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f0317dc4b80 10 client.25787 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f0317dc4b80 1 -- 172.21.15.67:0/3247761973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f095433cd0 con 0x55f09565b5d0 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f02e3400700 1 -- 172.21.15.67:0/3247761973 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02f00ad020 con 0x55f09565b5d0 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f02e3400700 1 -- 172.21.15.67:0/3247761973 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02f00b1070 con 0x55f09565b5d0 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f0317dc4b80 10 client.25787 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f0317dc4b80 10 client.25787 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f0317dc4b80 20 client.25787 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f0956cb150 0x55f0956cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f0317dc4b80 4 client.25787 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f0317dc4b80 1 -- 172.21.15.67:0/3247761973 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f095431ce0 con 0x55f0956cb150 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.716+0000 7f0300e00700 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f0956cb150 0x55f0956cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0300e00700 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f0956cb150 0x55f0956cd540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=649d53d197622bc server_cookie=72f8da2993940aac in_seq=0 out_seq=0 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f02e3400700 10 client.25787 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f02e3400700 1 -- 172.21.15.67:0/3247761973 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f095431ce0 con 0x55f0956cb150 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f02e3400700 10 client.25787 handle_command_reply: tid=0 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 client.25787 shutdown 2024-03-30T02:37:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 -- 172.21.15.67:0/3247761973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 msgr2=0x55f09566bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09566bc40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02f00967e0 tx=0x7f02f000ef90 comp rx=0 tx=0).stop 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 -- 172.21.15.67:0/3247761973 shutdown_connections 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f0956cb150 0x55f0956cd540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f095666df0 0x55f0956c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f09565b5d0 0x55f09566bc40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 --2- 172.21.15.67:0/3247761973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f09565bef0 0x55f09566c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f02e3400700 0 client.25787 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 -- 172.21.15.67:0/3247761973 >> 172.21.15.67:0/3247761973 conn(0x55f0955bb6a0 msgr2=0x55f095665320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 -- 172.21.15.67:0/3247761973 shutdown_connections 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.717+0000 7f0317dc4b80 1 -- 172.21.15.67:0/3247761973 wait complete. 2024-03-30T02:37:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0317dc4b80 20 client.25787 trim_cache size 0 max 16384 2024-03-30T02:37:24.729 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 -- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f02e40a13a0 msgr2=0x7f02e40a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f02e40a13a0 0x7f02e40a3860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009ec0 tx=0x7f02f4016450 comp rx=0 tx=0).stop 2024-03-30T02:37:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 -- 172.21.15.67:0/3019945991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc100f50 msgr2=0x7f02fc0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc100f50 0x7f02fc0730b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f02f000ec10 tx=0x7f02f000ef20 comp rx=0 tx=0).stop 2024-03-30T02:37:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 -- 172.21.15.67:0/3019945991 shutdown_connections 2024-03-30T02:37:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f02e40a13a0 0x7f02e40a3860 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02fc073b30 0x7f02fc1a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc100f50 0x7f02fc0730b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 --2- 172.21.15.67:0/3019945991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f02fc106070 0x7f02fc0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 -- 172.21.15.67:0/3019945991 >> 172.21.15.67:0/3019945991 conn(0x7f02fc0fc7f0 msgr2=0x7f02fc0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.718+0000 7f0304200700 1 -- 172.21.15.67:0/3019945991 shutdown_connections 2024-03-30T02:37:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.719+0000 7f0304200700 1 -- 172.21.15.67:0/3019945991 wait complete. 2024-03-30T02:37:24.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:37:24.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:24 smithi082 ceph-mon[79176]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:24.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:24 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4094009837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:37:24.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.938+0000 7f8985a00700 1 -- 172.21.15.67:0/2788921961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 msgr2=0x7f898010dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.938+0000 7f8985a00700 1 --2- 172.21.15.67:0/2788921961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 0x7f898010dc10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f897800b0a0 tx=0x7f897800b3b0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.938+0000 7f8985a00700 1 -- 172.21.15.67:0/2788921961 shutdown_connections 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.938+0000 7f8985a00700 1 --2- 172.21.15.67:0/2788921961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 0x7f898010dc10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.938+0000 7f8985a00700 1 --2- 172.21.15.67:0/2788921961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980100f70 0x7f8980101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.938+0000 7f8985a00700 1 --2- 172.21.15.67:0/2788921961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8980101920 0x7f8980105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.938+0000 7f8985a00700 1 -- 172.21.15.67:0/2788921961 >> 172.21.15.67:0/2788921961 conn(0x7f89800fc7f0 msgr2=0x7f89800fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.938+0000 7f8985a00700 1 -- 172.21.15.67:0/2788921961 shutdown_connections 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.938+0000 7f8985a00700 1 -- 172.21.15.67:0/2788921961 wait complete. 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.939+0000 7f8985a00700 1 Processor -- start 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.939+0000 7f8985a00700 1 -- start start 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f8985a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980100f70 0x7f89801994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f8985a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 0x7f8980199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f8985a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f898019ddd0 0x7f898019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f8985a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019e850 con 0x7f8980106090 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f8985a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019e9c0 con 0x7f8980100f70 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f8985a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019eb80 con 0x7f898019ddd0 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 0x7f8980199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980100f70 0x7f89801994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 0x7f8980199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35552/0 (socket says 172.21.15.67:35552) 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897e000700 1 -- 172.21.15.67:0/777723393 learned_addr learned my addr 172.21.15.67:0/777723393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980100f70 0x7f89801994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43932/0 (socket says 172.21.15.67:43932) 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897f400700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f898019ddd0 0x7f898019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897e000700 1 -- 172.21.15.67:0/777723393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f898019ddd0 msgr2=0x7f898019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897e000700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f898019ddd0 0x7f898019e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897e000700 1 -- 172.21.15.67:0/777723393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980100f70 msgr2=0x7f89801994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897e000700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980100f70 0x7f89801994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897e000700 1 -- 172.21.15.67:0/777723393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8978009d00 con 0x7f8980106090 2024-03-30T02:37:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.940+0000 7f897ea00700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980100f70 0x7f89801994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.941+0000 7f897e000700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 0x7f8980199a30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f896c00e970 tx=0x7f896c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.941+0000 7f8977400700 1 -- 172.21.15.67:0/777723393 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c009790 con 0x7f8980106090 2024-03-30T02:37:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.941+0000 7f8977400700 1 -- 172.21.15.67:0/777723393 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c00c7e0 con 0x7f8980106090 2024-03-30T02:37:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.941+0000 7f8985a00700 1 -- 172.21.15.67:0/777723393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89801a60e0 con 0x7f8980106090 2024-03-30T02:37:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.941+0000 7f8977400700 1 -- 172.21.15.67:0/777723393 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c0054e0 con 0x7f8980106090 2024-03-30T02:37:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.941+0000 7f8985a00700 1 -- 172.21.15.67:0/777723393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89801a6610 con 0x7f8980106090 2024-03-30T02:37:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.944+0000 7f8977400700 1 -- 172.21.15.67:0/777723393 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f896c00c950 con 0x7f8980106090 2024-03-30T02:37:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.945+0000 7f8977400700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89640a13a0 0x7f89640a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.945+0000 7f8977400700 1 -- 172.21.15.67:0/777723393 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f896c014070 con 0x7f8980106090 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f897ea00700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89640a13a0 0x7f89640a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f89995aab80 1 Processor -- start 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f89995aab80 1 -- start start 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f89995aab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 0x55f5561dc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f89995aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f89995aab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561e7df0 0x55f5561ea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f89995aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f556135990 con 0x55f5561dcef0 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f89995aab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f556133350 con 0x55f5561e7df0 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f897ea00700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89640a13a0 0x7f89640a3860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8970011490 tx=0x7f8970009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f89995aab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55612ec40 con 0x55f5561dc5d0 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.946+0000 7f897e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35568/0 (socket says 172.21.15.67:35568) 2024-03-30T02:37:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897e000700 1 -- 172.21.15.67:0/1663623973 learned_addr learned my addr 172.21.15.67:0/1663623973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897ea00700 1 --2- 172.21.15.67:0/1663623973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 0x55f5561dc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897f400700 1 --2- 172.21.15.67:0/1663623973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561e7df0 0x55f5561ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897e000700 1 -- 172.21.15.67:0/1663623973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 msgr2=0x55f5561dc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897e000700 1 --2- 172.21.15.67:0/1663623973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 0x55f5561dc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897e000700 1 -- 172.21.15.67:0/1663623973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561e7df0 msgr2=0x55f5561ea1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897e000700 1 --2- 172.21.15.67:0/1663623973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561e7df0 0x55f5561ea1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897e000700 1 -- 172.21.15.67:0/1663623973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f555fb03b0 con 0x55f5561dcef0 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f897e000700 1 --2- 172.21.15.67:0/1663623973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e78b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f896c099c90 tx=0x7f896c099a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c1713a44dc15de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f896be00700 1 -- 172.21.15.67:0/1663623973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c0cd170 con 0x55f5561dcef0 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f896be00700 1 -- 172.21.15.67:0/1663623973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c0cae90 con 0x55f5561dcef0 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.947+0000 7f896be00700 1 -- 172.21.15.67:0/1663623973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c0ca930 con 0x55f5561dcef0 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.948+0000 7f89995aab80 1 -- 172.21.15.67:0/1663623973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 msgr2=0x55f5561e78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.948+0000 7f89995aab80 1 --2- 172.21.15.67:0/1663623973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e78b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f896c099c90 tx=0x7f896c099a20 comp rx=0 tx=0).stop 2024-03-30T02:37:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.948+0000 7f89995aab80 1 -- 172.21.15.67:0/1663623973 shutdown_connections 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.948+0000 7f89995aab80 1 --2- 172.21.15.67:0/1663623973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561e7df0 0x55f5561ea1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.948+0000 7f89995aab80 1 --2- 172.21.15.67:0/1663623973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 0x55f5561dc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.948+0000 7f89995aab80 1 --2- 172.21.15.67:0/1663623973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e78b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.948+0000 7f89995aab80 1 -- 172.21.15.67:0/1663623973 >> 172.21.15.67:0/1663623973 conn(0x55f55613c6a0 msgr2=0x55f5561d8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.948+0000 7f89995aab80 1 -- 172.21.15.67:0/1663623973 shutdown_connections 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.948+0000 7f89995aab80 1 -- 172.21.15.67:0/1663623973 wait complete. 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.949+0000 7f89995aab80 1 -- start start 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.949+0000 7f89995aab80 1 Processor -- start 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.949+0000 7f89995aab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 0x55f5561e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.949+0000 7f89995aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.949+0000 7f89995aab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561e7df0 0x55f5561e51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.949+0000 7f89995aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5560cad80 con 0x55f5561dcef0 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.949+0000 7f89995aab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5560caa20 con 0x55f5561dc5d0 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.949+0000 7f89995aab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f555fb26a0 con 0x55f5561e7df0 2024-03-30T02:37:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f89995aab80 10 client.0 _ll_register_callbacks cb 0x55f5560d4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35574/0 (socket says 172.21.15.67:35574) 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897e000700 1 -- 172.21.15.67:0/1709324895 learned_addr learned my addr 172.21.15.67:0/1709324895 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897ea00700 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 0x55f5561e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897f400700 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561e7df0 0x55f5561e51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897e000700 1 -- 172.21.15.67:0/1709324895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561e7df0 msgr2=0x55f5561e51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897e000700 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561e7df0 0x55f5561e51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897e000700 1 -- 172.21.15.67:0/1709324895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 msgr2=0x55f5561e7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897e000700 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 0x55f5561e7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897e000700 1 -- 172.21.15.67:0/1709324895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89801a6610 con 0x55f5561dcef0 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f897e000700 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e4ab0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f896c0a1ee0 tx=0x7f896c005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5db338fbcc365f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.950+0000 7f8968c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f8968c00700 1 -- 172.21.15.67:0/1709324895 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c099f90 con 0x55f5561dcef0 2024-03-30T02:37:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f8968c00700 1 -- 172.21.15.67:0/1709324895 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f896c09ddc0 con 0x55f5561dcef0 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f89995aab80 1 -- 172.21.15.67:0/1709324895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5560ef380 con 0x55f5561dcef0 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f8968c00700 1 -- 172.21.15.67:0/1709324895 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f896c0cd170 con 0x55f5561dcef0 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f89995aab80 10 client.15750 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f89995aab80 1 -- 172.21.15.67:0/1709324895 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f555f5cb60 con 0x55f5561dcef0 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f8968c00700 1 -- 172.21.15.67:0/1709324895 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5560ef380 con 0x55f5561dcef0 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f8968c00700 1 -- 172.21.15.67:0/1709324895 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f896c099510 con 0x55f5561dcef0 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f89995aab80 10 client.15750 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f89995aab80 10 client.15750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f89995aab80 20 client.15750 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f89995aab80 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55f5561f74f0 0x55f5561f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f89995aab80 4 client.15750 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.951+0000 7f89995aab80 1 -- 172.21.15.67:0/1709324895 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f556127700 con 0x55f5561f74f0 2024-03-30T02:37:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.952+0000 7f897ea00700 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55f5561f74f0 0x55f5561f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.952+0000 7f897ea00700 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55f5561f74f0 0x55f5561f98e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=703a84b862d68232 server_cookie=61fe7d868dcdc924 in_seq=0 out_seq=0 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.952+0000 7f8968c00700 10 client.15750 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f8968c00700 1 -- 172.21.15.67:0/1709324895 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f556127700 con 0x55f5561f74f0 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f8968c00700 10 client.15750 handle_command_reply: tid=0 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 client.15750 shutdown 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 -- 172.21.15.67:0/1709324895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 msgr2=0x55f5561e4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e4ab0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f896c0a1ee0 tx=0x7f896c005870 comp rx=0 tx=0).stop 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 -- 172.21.15.67:0/1709324895 shutdown_connections 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561e7df0 0x55f5561e51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 0x55f5561e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55f5561f74f0 0x55f5561f98e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 --2- 172.21.15.67:0/1709324895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561dcef0 0x55f5561e4ab0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f8968c00700 0 client.15750 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:37:24.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 -- 172.21.15.67:0/1709324895 >> 172.21.15.67:0/1709324895 conn(0x55f55613c6a0 msgr2=0x55f5561de6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 -- 172.21.15.67:0/1709324895 shutdown_connections 2024-03-30T02:37:24.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.953+0000 7f89995aab80 1 -- 172.21.15.67:0/1709324895 wait complete. 2024-03-30T02:37:24.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.954+0000 7f89995aab80 20 client.15750 trim_cache size 0 max 16384 2024-03-30T02:37:24.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.967+0000 7f89995aab80 1 Processor -- start 2024-03-30T02:37:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.967+0000 7f89995aab80 1 -- start start 2024-03-30T02:37:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.967+0000 7f89995aab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 0x55f5561eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.967+0000 7f89995aab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dcef0 0x55f5561e7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.967+0000 7f89995aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.967+0000 7f89995aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5561df570 con 0x55f5561e7df0 2024-03-30T02:37:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.967+0000 7f89995aab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5561eefb0 con 0x55f5561dcef0 2024-03-30T02:37:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.967+0000 7f89995aab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5561dc3c0 con 0x55f5561dc5d0 2024-03-30T02:37:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561e78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35586/0 (socket says 172.21.15.67:35586) 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897f400700 1 -- 172.21.15.67:0/2576115282 learned_addr learned my addr 172.21.15.67:0/2576115282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897ea00700 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 0x55f5561eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897e000700 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dcef0 0x55f5561e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897f400700 1 -- 172.21.15.67:0/2576115282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 msgr2=0x55f5561eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897f400700 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 0x55f5561eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897f400700 1 -- 172.21.15.67:0/2576115282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dcef0 msgr2=0x55f5561e7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897f400700 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dcef0 0x55f5561e7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897f400700 1 -- 172.21.15.67:0/2576115282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5560f4ea0 con 0x55f5561e7df0 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897ea00700 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 0x55f5561eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897e000700 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dcef0 0x55f5561e7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.968+0000 7f897f400700 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561e78a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f897800b0a0 tx=0x7f8978007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd09c69ae5782d06 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f8974c00700 1 -- 172.21.15.67:0/2576115282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897800e040 con 0x55f5561e7df0 2024-03-30T02:37:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f8974c00700 1 -- 172.21.15.67:0/2576115282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8978022070 con 0x55f5561e7df0 2024-03-30T02:37:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f8974c00700 1 -- 172.21.15.67:0/2576115282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978003e70 con 0x55f5561e7df0 2024-03-30T02:37:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f89995aab80 1 -- 172.21.15.67:0/2576115282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 msgr2=0x55f5561e78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f89995aab80 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561e78a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f897800b0a0 tx=0x7f8978007b70 comp rx=0 tx=0).stop 2024-03-30T02:37:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f89995aab80 1 -- 172.21.15.67:0/2576115282 shutdown_connections 2024-03-30T02:37:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f89995aab80 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561e78a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f89995aab80 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dc5d0 0x55f5561eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f89995aab80 1 --2- 172.21.15.67:0/2576115282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dcef0 0x55f5561e7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f89995aab80 1 -- 172.21.15.67:0/2576115282 >> 172.21.15.67:0/2576115282 conn(0x55f55613c6a0 msgr2=0x55f5561dfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f89995aab80 1 -- 172.21.15.67:0/2576115282 shutdown_connections 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.969+0000 7f89995aab80 1 -- 172.21.15.67:0/2576115282 wait complete. 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.970+0000 7f89995aab80 1 -- start start 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.970+0000 7f89995aab80 1 Processor -- start 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.970+0000 7f89995aab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 0x55f5561ee990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.970+0000 7f89995aab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dcef0 0x55f5561ece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.970+0000 7f89995aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.970+0000 7f89995aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5561edb10 con 0x55f5561e7df0 2024-03-30T02:37:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.970+0000 7f89995aab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5561edc80 con 0x55f5561dc5d0 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.970+0000 7f89995aab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f55624b2b0 con 0x55f5561dcef0 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f89995aab80 10 client.0 _ll_register_callbacks cb 0x55f5560d4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561ed440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35602/0 (socket says 172.21.15.67:35602) 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897f400700 1 -- 172.21.15.67:0/715281816 learned_addr learned my addr 172.21.15.67:0/715281816 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897e000700 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dcef0 0x55f5561ece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897ea00700 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 0x55f5561ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897e000700 1 -- 172.21.15.67:0/715281816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 msgr2=0x55f5561ee990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897e000700 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 0x55f5561ee990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897e000700 1 -- 172.21.15.67:0/715281816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 msgr2=0x55f5561ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897e000700 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897e000700 1 -- 172.21.15.67:0/715281816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898010e480 con 0x55f5561dcef0 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897f400700 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f897e000700 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dcef0 0x55f5561ece30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f896c0a2550 tx=0x7f896c0999a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68f023d737c31508 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.971+0000 7f8968c00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f8968c00700 1 -- 172.21.15.67:0/715281816 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c09a3f0 con 0x55f5561dcef0 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f8968c00700 1 -- 172.21.15.67:0/715281816 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f896c098d10 con 0x55f5561dcef0 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f89995aab80 1 -- 172.21.15.67:0/715281816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f896c0cae90 con 0x55f5561dcef0 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f8968c00700 1 -- 172.21.15.67:0/715281816 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f896c0ca5c0 con 0x55f5561dcef0 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f89995aab80 10 client.25795 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f89995aab80 1 -- 172.21.15.67:0/715281816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f555fb4cd0 con 0x55f5561dcef0 2024-03-30T02:37:24.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f8968c00700 1 -- 172.21.15.67:0/715281816 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f896c0cae90 con 0x55f5561dcef0 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f8968c00700 1 -- 172.21.15.67:0/715281816 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f896c094ba0 con 0x55f5561dcef0 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f89995aab80 10 client.25795 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f89995aab80 10 client.25795 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f89995aab80 20 client.25795 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f89995aab80 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55f55624c5a0 0x55f55624e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f89995aab80 4 client.25795 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.972+0000 7f89995aab80 1 -- 172.21.15.67:0/715281816 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f555f7dcf0 con 0x55f55624c5a0 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.973+0000 7f897ea00700 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55f55624c5a0 0x55f55624e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.973+0000 7f897ea00700 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55f55624c5a0 0x55f55624e990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abb3bfad1d0dac52 server_cookie=6a518c667670559 in_seq=0 out_seq=0 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.973+0000 7f8968c00700 10 client.25795 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:37:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.973+0000 7f8968c00700 1 -- 172.21.15.67:0/715281816 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f555f7dcf0 con 0x55f55624c5a0 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.973+0000 7f8968c00700 10 client.25795 handle_command_reply: tid=0 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.973+0000 7f89995aab80 1 client.25795 shutdown 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.973+0000 7f89995aab80 1 -- 172.21.15.67:0/715281816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dcef0 msgr2=0x55f5561ece30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.973+0000 7f89995aab80 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dcef0 0x55f5561ece30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f896c0a2550 tx=0x7f896c0999a0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f89995aab80 1 -- 172.21.15.67:0/715281816 shutdown_connections 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f89995aab80 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55f55624c5a0 0x55f55624e990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f89995aab80 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5561e7df0 0x55f5561ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f89995aab80 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5561dc5d0 0x55f5561ee990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f89995aab80 1 --2- 172.21.15.67:0/715281816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5561dcef0 0x55f5561ece30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f8968c00700 0 client.25795 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:37:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f89995aab80 1 -- 172.21.15.67:0/715281816 >> 172.21.15.67:0/715281816 conn(0x55f55613c6a0 msgr2=0x55f5561de5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f89995aab80 1 -- 172.21.15.67:0/715281816 shutdown_connections 2024-03-30T02:37:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f89995aab80 1 -- 172.21.15.67:0/715281816 wait complete. 2024-03-30T02:37:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f89995aab80 20 client.25795 trim_cache size 0 max 16384 2024-03-30T02:37:24.983 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:24.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f8985a00700 1 -- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89640a13a0 msgr2=0x7f89640a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f8985a00700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89640a13a0 0x7f89640a3860 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8970011490 tx=0x7f8970009260 comp rx=0 tx=0).stop 2024-03-30T02:37:24.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f8985a00700 1 -- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 msgr2=0x7f8980199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:24.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.974+0000 7f8985a00700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 0x7f8980199a30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f896c00e970 tx=0x7f896c00ec80 comp rx=0 tx=0).stop 2024-03-30T02:37:24.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.975+0000 7f8985a00700 1 -- 172.21.15.67:0/777723393 shutdown_connections 2024-03-30T02:37:24.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.975+0000 7f8985a00700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89640a13a0 0x7f89640a3860 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.975+0000 7f8985a00700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f898019ddd0 0x7f898019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.975+0000 7f8985a00700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980100f70 0x7f89801994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.975+0000 7f8985a00700 1 --2- 172.21.15.67:0/777723393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8980106090 0x7f8980199a30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.975+0000 7f8985a00700 1 -- 172.21.15.67:0/777723393 >> 172.21.15.67:0/777723393 conn(0x7f89800fc7f0 msgr2=0x7f898010c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.975+0000 7f8985a00700 1 -- 172.21.15.67:0/777723393 shutdown_connections 2024-03-30T02:37:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:24.975+0000 7f8985a00700 1 -- 172.21.15.67:0/777723393 wait complete. 2024-03-30T02:37:24.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:37:25.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.196+0000 7f2690a00700 1 -- 172.21.15.67:0/103376863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c106080 msgr2=0x7f268c10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.196+0000 7f2690a00700 1 --2- 172.21.15.67:0/103376863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c106080 0x7f268c10dc00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f268400b0a0 tx=0x7f268400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.196+0000 7f2690a00700 1 -- 172.21.15.67:0/103376863 shutdown_connections 2024-03-30T02:37:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.196+0000 7f2690a00700 1 --2- 172.21.15.67:0/103376863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c106080 0x7f268c10dc00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.196+0000 7f2690a00700 1 --2- 172.21.15.67:0/103376863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268c100f60 0x7f268c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.196+0000 7f2690a00700 1 --2- 172.21.15.67:0/103376863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f268c101910 0x7f268c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.196+0000 7f2690a00700 1 -- 172.21.15.67:0/103376863 >> 172.21.15.67:0/103376863 conn(0x7f268c0fc7c0 msgr2=0x7f268c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.196+0000 7f2690a00700 1 -- 172.21.15.67:0/103376863 shutdown_connections 2024-03-30T02:37:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.196+0000 7f2690a00700 1 -- 172.21.15.67:0/103376863 wait complete. 2024-03-30T02:37:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.197+0000 7f2690a00700 1 Processor -- start 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.197+0000 7f2690a00700 1 -- start start 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.197+0000 7f2690a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268c100f60 0x7f268c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.197+0000 7f2690a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f268c106080 0x7f268c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.197+0000 7f2690a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c19de10 0x7f268c19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.197+0000 7f2690a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c19e890 con 0x7f268c19de10 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.197+0000 7f2690a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c19ea00 con 0x7f268c100f60 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.197+0000 7f2690a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c19ebc0 con 0x7f268c106080 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f268a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c19de10 0x7f268c19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f268a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c19de10 0x7f268c19e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35610/0 (socket says 172.21.15.67:35610) 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f268a800700 1 -- 172.21.15.67:0/684131579 learned_addr learned my addr 172.21.15.67:0/684131579 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f2683e00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f268c106080 0x7f268c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f2689e00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268c100f60 0x7f268c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f268a800700 1 -- 172.21.15.67:0/684131579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f268c106080 msgr2=0x7f268c199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f268a800700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f268c106080 0x7f268c199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f268a800700 1 -- 172.21.15.67:0/684131579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268c100f60 msgr2=0x7f268c199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f268a800700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268c100f60 0x7f268c199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f268a800700 1 -- 172.21.15.67:0/684131579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c009520 con 0x7f268c19de10 2024-03-30T02:37:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.198+0000 7f268a800700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c19de10 0x7f268c19e290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f268400b070 tx=0x7f268400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.199+0000 7f2682a00700 1 -- 172.21.15.67:0/684131579 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400e040 con 0x7f268c19de10 2024-03-30T02:37:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.199+0000 7f2682a00700 1 -- 172.21.15.67:0/684131579 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f268401d480 con 0x7f268c19de10 2024-03-30T02:37:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.199+0000 7f2682a00700 1 -- 172.21.15.67:0/684131579 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840052a0 con 0x7f268c19de10 2024-03-30T02:37:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.199+0000 7f2690a00700 1 -- 172.21.15.67:0/684131579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2674009520 con 0x7f268c19de10 2024-03-30T02:37:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.199+0000 7f2690a00700 1 -- 172.21.15.67:0/684131579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2684009d00 con 0x7f268c19de10 2024-03-30T02:37:25.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.202+0000 7f2682a00700 1 -- 172.21.15.67:0/684131579 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f268401d5f0 con 0x7f268c19de10 2024-03-30T02:37:25.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.203+0000 7f2682a00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f26700a13a0 0x7f26700a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.203+0000 7f26a449db80 1 Processor -- start 2024-03-30T02:37:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.203+0000 7f2682a00700 1 -- 172.21.15.67:0/684131579 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f26840ce1d0 con 0x7f268c19de10 2024-03-30T02:37:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.203+0000 7f2689e00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f26700a13a0 0x7f26700a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.203+0000 7f26a449db80 1 -- start start 2024-03-30T02:37:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f26a449db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 0x564d0b29e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f26a449db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f26a449db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 0x564d0b2ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f26a449db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b1f7990 con 0x564d0b29eef0 2024-03-30T02:37:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f26a449db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b1f5350 con 0x564d0b2a9df0 2024-03-30T02:37:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f26a449db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b1f0c40 con 0x564d0b29e5d0 2024-03-30T02:37:25.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f2683e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f268a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 0x564d0b2ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f2683e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35622/0 (socket says 172.21.15.67:35622) 2024-03-30T02:37:25.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f2683e00700 1 -- 172.21.15.67:0/3885976925 learned_addr learned my addr 172.21.15.67:0/3885976925 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:25.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f2689e00700 1 --2- 172.21.15.67:0/3885976925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 0x564d0b29e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.204+0000 7f2689e00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f26700a13a0 0x7f26700a3860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f267c009e40 tx=0x7f267c009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:25.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f268a800700 1 -- 172.21.15.67:0/3885976925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 msgr2=0x564d0b29e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:25.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f268a800700 1 --2- 172.21.15.67:0/3885976925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 0x564d0b29e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f268a800700 1 -- 172.21.15.67:0/3885976925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 msgr2=0x564d0b2a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f268a800700 1 --2- 172.21.15.67:0/3885976925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f268a800700 1 -- 172.21.15.67:0/3885976925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d0b0723b0 con 0x564d0b2a9df0 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f2689e00700 1 --2- 172.21.15.67:0/3885976925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 0x564d0b29e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f268a800700 1 --2- 172.21.15.67:0/3885976925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 0x564d0b2ac1e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26840cb4b0 tx=0x7f268400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35d6a471e632997a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f267b000700 1 -- 172.21.15.67:0/3885976925 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268409ec80 con 0x564d0b2a9df0 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f267b000700 1 -- 172.21.15.67:0/3885976925 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26840a3240 con 0x564d0b2a9df0 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.205+0000 7f267b000700 1 -- 172.21.15.67:0/3885976925 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268409b310 con 0x564d0b2a9df0 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 -- 172.21.15.67:0/3885976925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 msgr2=0x564d0b2ac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 --2- 172.21.15.67:0/3885976925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 0x564d0b2ac1e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26840cb4b0 tx=0x7f268400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 -- 172.21.15.67:0/3885976925 shutdown_connections 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 --2- 172.21.15.67:0/3885976925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 0x564d0b2ac1e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 --2- 172.21.15.67:0/3885976925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 0x564d0b29e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 --2- 172.21.15.67:0/3885976925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 -- 172.21.15.67:0/3885976925 >> 172.21.15.67:0/3885976925 conn(0x564d0b1fe6a0 msgr2=0x564d0b29aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 -- 172.21.15.67:0/3885976925 shutdown_connections 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 -- 172.21.15.67:0/3885976925 wait complete. 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.206+0000 7f26a449db80 1 -- start start 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.207+0000 7f26a449db80 1 Processor -- start 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.207+0000 7f26a449db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29e5d0 0x564d0b2a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.207+0000 7f26a449db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.207+0000 7f26a449db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 0x564d0b2a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.207+0000 7f26a449db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b18cd80 con 0x564d0b29eef0 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.207+0000 7f26a449db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b18ca20 con 0x564d0b29e5d0 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.207+0000 7f26a449db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b0746a0 con 0x564d0b2a9df0 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.207+0000 7f26a449db80 10 client.0 _ll_register_callbacks cb 0x564d0b196470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:25.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.207+0000 7f2683e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f2683e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35636/0 (socket says 172.21.15.67:35636) 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f2683e00700 1 -- 172.21.15.67:0/3032777114 learned_addr learned my addr 172.21.15.67:0/3032777114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f2689e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29e5d0 0x564d0b2a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f268a800700 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 0x564d0b2a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f2683e00700 1 -- 172.21.15.67:0/3032777114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 msgr2=0x564d0b2a70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f2683e00700 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 0x564d0b2a70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f2683e00700 1 -- 172.21.15.67:0/3032777114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29e5d0 msgr2=0x564d0b2a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f2683e00700 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29e5d0 0x564d0b2a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f2683e00700 1 -- 172.21.15.67:0/3032777114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2684009d00 con 0x564d0b29eef0 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f2683e00700 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a6a80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2674009fd0 tx=0x7f2674012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a445c62330a8f93 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f266f400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f266f400700 1 -- 172.21.15.67:0/3032777114 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400f8c0 con 0x564d0b29eef0 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f266f400700 1 -- 172.21.15.67:0/3032777114 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f267400fa20 con 0x564d0b29eef0 2024-03-30T02:37:25.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.208+0000 7f26a449db80 1 -- 172.21.15.67:0/3032777114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d0b1b1380 con 0x564d0b29eef0 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f266f400700 1 -- 172.21.15.67:0/3032777114 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f267400fbe0 con 0x564d0b29eef0 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f26a449db80 10 client.15768 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f26a449db80 1 -- 172.21.15.67:0/3032777114 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564d0b01eb60 con 0x564d0b29eef0 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f266f400700 1 -- 172.21.15.67:0/3032777114 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d0b1b1380 con 0x564d0b29eef0 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f266f400700 1 -- 172.21.15.67:0/3032777114 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2674010430 con 0x564d0b29eef0 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f26a449db80 10 client.15768 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f26a449db80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f26a449db80 20 client.15768 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f26a449db80 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564d0b2b97d0 0x564d0b2bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f26a449db80 4 client.15768 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.209+0000 7f26a449db80 1 -- 172.21.15.67:0/3032777114 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d0b1e9700 con 0x564d0b2b97d0 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.210+0000 7f2689e00700 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564d0b2b97d0 0x564d0b2bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.210+0000 7f266f400700 10 client.15768 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.210+0000 7f2689e00700 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564d0b2b97d0 0x564d0b2bbbf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcffd8a30dd318d2 server_cookie=73b8a71470e1885f in_seq=0 out_seq=0 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f266f400700 1 -- 172.21.15.67:0/3032777114 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564d0b1e9700 con 0x564d0b2b97d0 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f266f400700 10 client.15768 handle_command_reply: tid=0 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 client.15768 shutdown 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 -- 172.21.15.67:0/3032777114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 msgr2=0x564d0b2a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a6a80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2674009fd0 tx=0x7f2674012c50 comp rx=0 tx=0).stop 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 -- 172.21.15.67:0/3032777114 shutdown_connections 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 0x564d0b2a70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29e5d0 0x564d0b2a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564d0b2b97d0 0x564d0b2bbbf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 --2- 172.21.15.67:0/3032777114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2a6a80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 -- 172.21.15.67:0/3032777114 >> 172.21.15.67:0/3032777114 conn(0x564d0b1fe6a0 msgr2=0x564d0b2a06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f266f400700 0 client.15768 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:25.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 -- 172.21.15.67:0/3032777114 shutdown_connections 2024-03-30T02:37:25.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 1 -- 172.21.15.67:0/3032777114 wait complete. 2024-03-30T02:37:25.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.211+0000 7f26a449db80 20 client.15768 trim_cache size 0 max 16384 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.225+0000 7f26a449db80 1 Processor -- start 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.225+0000 7f26a449db80 1 -- start start 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.225+0000 7f26a449db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 0x564d0b2a7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.225+0000 7f26a449db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.225+0000 7f26a449db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 0x564d0b2935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.225+0000 7f26a449db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b2a7990 con 0x564d0b29eef0 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.225+0000 7f26a449db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b2b0dc0 con 0x564d0b2a9df0 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.225+0000 7f26a449db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b2a2660 con 0x564d0b29e5d0 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2683e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2683e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35638/0 (socket says 172.21.15.67:35638) 2024-03-30T02:37:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2683e00700 1 -- 172.21.15.67:0/1795427514 learned_addr learned my addr 172.21.15.67:0/1795427514 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2689e00700 1 --2- 172.21.15.67:0/1795427514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 0x564d0b2a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f268a800700 1 --2- 172.21.15.67:0/1795427514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 0x564d0b2935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2683e00700 1 -- 172.21.15.67:0/1795427514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 msgr2=0x564d0b2a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2683e00700 1 --2- 172.21.15.67:0/1795427514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 0x564d0b2a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2683e00700 1 -- 172.21.15.67:0/1795427514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 msgr2=0x564d0b2935a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2683e00700 1 --2- 172.21.15.67:0/1795427514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 0x564d0b2935a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2683e00700 1 -- 172.21.15.67:0/1795427514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d0b1b6ea0 con 0x564d0b29eef0 2024-03-30T02:37:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f2683e00700 1 --2- 172.21.15.67:0/1795427514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2b0880 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2674001eb0 tx=0x7f2674009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb5688916f26893 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:25.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f267ba00700 1 -- 172.21.15.67:0/1795427514 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2674010430 con 0x564d0b29eef0 2024-03-30T02:37:25.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.226+0000 7f267ba00700 1 -- 172.21.15.67:0/1795427514 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2674010590 con 0x564d0b29eef0 2024-03-30T02:37:25.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f267ba00700 1 -- 172.21.15.67:0/1795427514 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400f9d0 con 0x564d0b29eef0 2024-03-30T02:37:25.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f26a449db80 1 -- 172.21.15.67:0/1795427514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 msgr2=0x564d0b2b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:25.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f26a449db80 1 --2- 172.21.15.67:0/1795427514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2b0880 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2674001eb0 tx=0x7f2674009400 comp rx=0 tx=0).stop 2024-03-30T02:37:25.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f26a449db80 1 -- 172.21.15.67:0/1795427514 shutdown_connections 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f26a449db80 1 --2- 172.21.15.67:0/1795427514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b2a9df0 0x564d0b2935a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f26a449db80 1 --2- 172.21.15.67:0/1795427514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b29e5d0 0x564d0b2a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f26a449db80 1 --2- 172.21.15.67:0/1795427514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29eef0 0x564d0b2b0880 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f26a449db80 1 -- 172.21.15.67:0/1795427514 >> 172.21.15.67:0/1795427514 conn(0x564d0b1fe6a0 msgr2=0x564d0b293f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f26a449db80 1 -- 172.21.15.67:0/1795427514 shutdown_connections 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.227+0000 7f26a449db80 1 -- 172.21.15.67:0/1795427514 wait complete. 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.228+0000 7f26a449db80 1 -- start start 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.228+0000 7f26a449db80 1 Processor -- start 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f26a449db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29e5d0 0x564d0b2ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f26a449db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29eef0 0x564d0b2aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f26a449db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 0x564d0b2afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f26a449db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b2b0390 con 0x564d0b29e5d0 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f26a449db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b2b0500 con 0x564d0b29eef0 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f26a449db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0b2b06c0 con 0x564d0b2a9df0 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f26a449db80 10 client.0 _ll_register_callbacks cb 0x564d0b196470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f2689e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29e5d0 0x564d0b2ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f2689e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29e5d0 0x564d0b2ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35652/0 (socket says 172.21.15.67:35652) 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f2689e00700 1 -- 172.21.15.67:0/1760009180 learned_addr learned my addr 172.21.15.67:0/1760009180 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f2683e00700 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29eef0 0x564d0b2aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f268a800700 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 0x564d0b2afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f268a800700 1 -- 172.21.15.67:0/1760009180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29eef0 msgr2=0x564d0b2aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f268a800700 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29eef0 0x564d0b2aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f268a800700 1 -- 172.21.15.67:0/1760009180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29e5d0 msgr2=0x564d0b2ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f268a800700 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29e5d0 0x564d0b2ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f268a800700 1 -- 172.21.15.67:0/1760009180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d0b01eb60 con 0x564d0b2a9df0 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f2683e00700 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29eef0 0x564d0b2aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.229+0000 7f2689e00700 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29e5d0 0x564d0b2ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f268a800700 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 0x564d0b2afdb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26840cb4b0 tx=0x7f26840cb4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=629d984c267e975e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f266f400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f266f400700 1 -- 172.21.15.67:0/1760009180 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268409ec80 con 0x564d0b2a9df0 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f26a449db80 1 -- 172.21.15.67:0/1760009180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d0b03fcf0 con 0x564d0b2a9df0 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f266f400700 1 -- 172.21.15.67:0/1760009180 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f268409b0b0 con 0x564d0b2a9df0 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f266f400700 1 -- 172.21.15.67:0/1760009180 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f268409b270 con 0x564d0b2a9df0 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f26a449db80 10 client.25813 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f26a449db80 1 -- 172.21.15.67:0/1760009180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564d0b076cd0 con 0x564d0b2a9df0 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f266f400700 1 -- 172.21.15.67:0/1760009180 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d0b03fcf0 con 0x564d0b2a9df0 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.230+0000 7f266f400700 1 -- 172.21.15.67:0/1760009180 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f268409fba0 con 0x564d0b2a9df0 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.231+0000 7f26a449db80 10 client.25813 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.231+0000 7f26a449db80 10 client.25813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.231+0000 7f26a449db80 20 client.25813 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.231+0000 7f26a449db80 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564d0b30e490 0x564d0b310880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.231+0000 7f26a449db80 4 client.25813 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.231+0000 7f26a449db80 1 -- 172.21.15.67:0/1760009180 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d0b074ce0 con 0x564d0b30e490 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.231+0000 7f2689e00700 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564d0b30e490 0x564d0b310880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.231+0000 7f2689e00700 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564d0b30e490 0x564d0b310880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ec65a7985d85dd server_cookie=d0aaa20da24326ee in_seq=0 out_seq=0 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.231+0000 7f266f400700 10 client.25813 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f266f400700 1 -- 172.21.15.67:0/1760009180 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d0b074ce0 con 0x564d0b30e490 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f266f400700 10 client.25813 handle_command_reply: tid=0 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 client.25813 shutdown 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 -- 172.21.15.67:0/1760009180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 msgr2=0x564d0b2afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 0x564d0b2afdb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26840cb4b0 tx=0x7f26840cb4e0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 -- 172.21.15.67:0/1760009180 shutdown_connections 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564d0b30e490 0x564d0b310880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564d0b29e5d0 0x564d0b2ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f266f400700 0 client.25813 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d0b2a9df0 0x564d0b2afdb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 --2- 172.21.15.67:0/1760009180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d0b29eef0 0x564d0b2aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 -- 172.21.15.67:0/1760009180 >> 172.21.15.67:0/1760009180 conn(0x564d0b1fe6a0 msgr2=0x564d0b2935a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:25.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 -- 172.21.15.67:0/1760009180 shutdown_connections 2024-03-30T02:37:25.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 1 -- 172.21.15.67:0/1760009180 wait complete. 2024-03-30T02:37:25.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.232+0000 7f26a449db80 20 client.25813 trim_cache size 0 max 16384 2024-03-30T02:37:25.238 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 -- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f26700a13a0 msgr2=0x7f26700a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f26700a13a0 0x7f26700a3860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f267c009e40 tx=0x7f267c009260 comp rx=0 tx=0).stop 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 -- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c19de10 msgr2=0x7f268c19e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c19de10 0x7f268c19e290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f268400b070 tx=0x7f268400bda0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 -- 172.21.15.67:0/684131579 shutdown_connections 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f26700a13a0 0x7f26700a3860 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f268c19de10 0x7f268c19e290 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f268c100f60 0x7f268c199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 --2- 172.21.15.67:0/684131579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f268c106080 0x7f268c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 -- 172.21.15.67:0/684131579 >> 172.21.15.67:0/684131579 conn(0x7f268c0fc7c0 msgr2=0x7f268c0fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:25.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 -- 172.21.15.67:0/684131579 shutdown_connections 2024-03-30T02:37:25.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:25.233+0000 7f2690a00700 1 -- 172.21.15.67:0/684131579 wait complete. 2024-03-30T02:37:26.252 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:37:26.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:37:26.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.468+0000 7f111ce00700 1 -- 172.21.15.67:0/1430250547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118101910 msgr2=0x7f1118105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.468+0000 7f111ce00700 1 --2- 172.21.15.67:0/1430250547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118101910 0x7f1118105950 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1104009990 tx=0x7f1104009ca0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.468+0000 7f111ce00700 1 -- 172.21.15.67:0/1430250547 shutdown_connections 2024-03-30T02:37:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.469+0000 7f111ce00700 1 --2- 172.21.15.67:0/1430250547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1118106080 0x7f111810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.469+0000 7f111ce00700 1 --2- 172.21.15.67:0/1430250547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1118100f60 0x7f1118101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.469+0000 7f111ce00700 1 --2- 172.21.15.67:0/1430250547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118101910 0x7f1118105950 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.469+0000 7f111ce00700 1 -- 172.21.15.67:0/1430250547 >> 172.21.15.67:0/1430250547 conn(0x7f11180fc7c0 msgr2=0x7f11180febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.469+0000 7f111ce00700 1 -- 172.21.15.67:0/1430250547 shutdown_connections 2024-03-30T02:37:26.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.469+0000 7f111ce00700 1 -- 172.21.15.67:0/1430250547 wait complete. 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.470+0000 7f111ce00700 1 Processor -- start 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.470+0000 7f111ce00700 1 -- start start 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.470+0000 7f111ce00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118100f60 0x7f1118197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.470+0000 7f111ce00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1118106080 0x7f11181977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.470+0000 7f111ce00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111819bb40 0x7f111819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.470+0000 7f111ce00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111819c5c0 con 0x7f1118100f60 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.470+0000 7f111ce00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111819c730 con 0x7f111819bb40 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.470+0000 7f111ce00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111819c8f0 con 0x7f1118106080 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.470+0000 7f1115e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118100f60 0x7f1118197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1115e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118100f60 0x7f1118197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34872/0 (socket says 172.21.15.67:34872) 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1115e00700 1 -- 172.21.15.67:0/1591125046 learned_addr learned my addr 172.21.15.67:0/1591125046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1116800700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111819bb40 0x7f111819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1115400700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1118106080 0x7f11181977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1115e00700 1 -- 172.21.15.67:0/1591125046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1118106080 msgr2=0x7f11181977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1115e00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1118106080 0x7f11181977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1115e00700 1 -- 172.21.15.67:0/1591125046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111819bb40 msgr2=0x7f111819bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1115e00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111819bb40 0x7f111819bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1115e00700 1 -- 172.21.15.67:0/1591125046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11040095f0 con 0x7f1118100f60 2024-03-30T02:37:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1116800700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111819bb40 0x7f111819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:26.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f1115e00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118100f60 0x7f1118197260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f110800e9a0 tx=0x7f110800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.471+0000 7f110ea00700 1 -- 172.21.15.67:0/1591125046 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108009790 con 0x7f1118100f60 2024-03-30T02:37:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.472+0000 7f110ea00700 1 -- 172.21.15.67:0/1591125046 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1108010430 con 0x7f1118100f60 2024-03-30T02:37:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.472+0000 7f111ce00700 1 -- 172.21.15.67:0/1591125046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11181a3e50 con 0x7f1118100f60 2024-03-30T02:37:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.472+0000 7f110ea00700 1 -- 172.21.15.67:0/1591125046 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108018680 con 0x7f1118100f60 2024-03-30T02:37:26.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.472+0000 7f111ce00700 1 -- 172.21.15.67:0/1591125046 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11181a4320 con 0x7f1118100f60 2024-03-30T02:37:26.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.476+0000 7f110ea00700 1 -- 172.21.15.67:0/1591125046 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f11080105a0 con 0x7f1118100f60 2024-03-30T02:37:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.477+0000 7f113086bb80 1 Processor -- start 2024-03-30T02:37:26.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.477+0000 7f113086bb80 1 -- start start 2024-03-30T02:37:26.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.477+0000 7f110ea00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f10fc0a13a0 0x7f10fc0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.477+0000 7f113086bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecc2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f113086bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f110ea00700 1 -- 172.21.15.67:0/1591125046 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1108014070 con 0x7f1118100f60 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f113086bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 0x55d02ecd0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f113086bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ec17a10 con 0x55d02ecc2ac0 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f113086bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ec153d0 con 0x55d02ecce290 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f113086bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ec10cc0 con 0x55d02ecc33e0 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1116800700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f10fc0a13a0 0x7f10fc0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecc2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecc2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34886/0 (socket says 172.21.15.67:34886) 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115e00700 1 -- 172.21.15.67:0/3561150644 learned_addr learned my addr 172.21.15.67:0/3561150644 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115400700 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1116800700 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 0x55d02ecd0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115e00700 1 -- 172.21.15.67:0/3561150644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 msgr2=0x55d02eccdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115e00700 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115e00700 1 -- 172.21.15.67:0/3561150644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 msgr2=0x55d02ecd0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115e00700 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 0x55d02ecd0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115e00700 1 -- 172.21.15.67:0/3561150644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d02ea92430 con 0x55d02ecc2ac0 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1115400700 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1116800700 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 0x55d02ecd0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.478+0000 7f1116800700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f10fc0a13a0 0x7f10fc0a3860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1110003e40 tx=0x7f1110009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.479+0000 7f1115e00700 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecc2ea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f11080ca6e0 tx=0x7f1108099940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219e3d4ab11b7cd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.479+0000 7f1103400700 1 -- 172.21.15.67:0/3561150644 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11080cd170 con 0x55d02ecc2ac0 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.479+0000 7f1103400700 1 -- 172.21.15.67:0/3561150644 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110809dd90 con 0x55d02ecc2ac0 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.479+0000 7f1103400700 1 -- 172.21.15.67:0/3561150644 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108099450 con 0x55d02ecc2ac0 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.479+0000 7f113086bb80 1 -- 172.21.15.67:0/3561150644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 msgr2=0x55d02ecc2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.479+0000 7f113086bb80 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecc2ea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f11080ca6e0 tx=0x7f1108099940 comp rx=0 tx=0).stop 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.480+0000 7f113086bb80 1 -- 172.21.15.67:0/3561150644 shutdown_connections 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.480+0000 7f113086bb80 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecc2ea0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.480+0000 7f113086bb80 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 0x55d02ecd0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.480+0000 7f113086bb80 1 --2- 172.21.15.67:0/3561150644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.480+0000 7f113086bb80 1 -- 172.21.15.67:0/3561150644 >> 172.21.15.67:0/3561150644 conn(0x55d02ec1e720 msgr2=0x55d02ecbaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.480+0000 7f113086bb80 1 -- 172.21.15.67:0/3561150644 shutdown_connections 2024-03-30T02:37:26.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.480+0000 7f113086bb80 1 -- 172.21.15.67:0/3561150644 wait complete. 2024-03-30T02:37:26.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.480+0000 7f113086bb80 1 -- start start 2024-03-30T02:37:26.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.480+0000 7f113086bb80 1 Processor -- start 2024-03-30T02:37:26.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f113086bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02eccdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f113086bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f113086bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 0x55d02eccb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f113086bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ebace00 con 0x55d02ecc2ac0 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f113086bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ebacaa0 con 0x55d02ecce290 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f113086bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ea94720 con 0x55d02ecc33e0 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f113086bb80 10 client.0 _ll_register_callbacks cb 0x55d02ebb64f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f1115e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02eccdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f1115e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02eccdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34900/0 (socket says 172.21.15.67:34900) 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f1115e00700 1 -- 172.21.15.67:0/929885174 learned_addr learned my addr 172.21.15.67:0/929885174 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f1115400700 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f1116800700 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 0x55d02eccb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f1115e00700 1 -- 172.21.15.67:0/929885174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 msgr2=0x55d02eccaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f1115e00700 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f1115e00700 1 -- 172.21.15.67:0/929885174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 msgr2=0x55d02eccb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.481+0000 7f1115e00700 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 0x55d02eccb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f1115e00700 1 -- 172.21.15.67:0/929885174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11181a4320 con 0x55d02ecc2ac0 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f1115400700 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f1115e00700 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02eccdc60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f110800c510 tx=0x7f11080ca5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b384714c2a45694c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f10f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f10f7e00700 1 -- 172.21.15.67:0/929885174 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11080b8320 con 0x55d02ecc2ac0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f10f7e00700 1 -- 172.21.15.67:0/929885174 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1108099450 con 0x55d02ecc2ac0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f113086bb80 1 -- 172.21.15.67:0/929885174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d02ebd1400 con 0x55d02ecc2ac0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f10f7e00700 1 -- 172.21.15.67:0/929885174 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1108099610 con 0x55d02ecc2ac0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f113086bb80 10 client.15786 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f113086bb80 1 -- 172.21.15.67:0/929885174 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d02ea3ebe0 con 0x55d02ecc2ac0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.482+0000 7f10f7e00700 1 -- 172.21.15.67:0/929885174 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d02ebd1400 con 0x55d02ecc2ac0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.483+0000 7f10f7e00700 1 -- 172.21.15.67:0/929885174 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1108095e70 con 0x55d02ecc2ac0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.483+0000 7f113086bb80 10 client.15786 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.483+0000 7f113086bb80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.483+0000 7f113086bb80 20 client.15786 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.483+0000 7f113086bb80 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d02ecdd9b0 0x55d02ecdfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.483+0000 7f113086bb80 4 client.15786 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.483+0000 7f113086bb80 1 -- 172.21.15.67:0/929885174 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d02ec09780 con 0x55d02ecdd9b0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.483+0000 7f1115400700 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d02ecdd9b0 0x55d02ecdfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.484+0000 7f1115400700 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d02ecdd9b0 0x55d02ecdfda0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5439727dbe32ef6a server_cookie=6e6cb6f50d0dfb55 in_seq=0 out_seq=0 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.484+0000 7f10f7e00700 10 client.15786 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.484+0000 7f10f7e00700 1 -- 172.21.15.67:0/929885174 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d02ec09780 con 0x55d02ecdd9b0 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.484+0000 7f10f7e00700 10 client.15786 handle_command_reply: tid=0 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.484+0000 7f113086bb80 1 client.15786 shutdown 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 -- 172.21.15.67:0/929885174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 msgr2=0x55d02eccdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02eccdc60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f110800c510 tx=0x7f11080ca5c0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 -- 172.21.15.67:0/929885174 shutdown_connections 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d02ecdd9b0 0x55d02ecdfda0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02eccdc60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecce290 0x55d02eccb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 --2- 172.21.15.67:0/929885174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecc33e0 0x55d02eccaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f10f7e00700 0 client.15786 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 -- 172.21.15.67:0/929885174 >> 172.21.15.67:0/929885174 conn(0x55d02ec1e720 msgr2=0x55d02ecc4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 -- 172.21.15.67:0/929885174 shutdown_connections 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 1 -- 172.21.15.67:0/929885174 wait complete. 2024-03-30T02:37:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.485+0000 7f113086bb80 20 client.15786 trim_cache size 0 max 16384 2024-03-30T02:37:26.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.503+0000 7f113086bb80 1 Processor -- start 2024-03-30T02:37:26.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.503+0000 7f113086bb80 1 -- start start 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.503+0000 7f113086bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.503+0000 7f113086bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc33e0 0x55d02ecca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.503+0000 7f113086bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 0x55d02eccad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.503+0000 7f113086bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ecc3d70 con 0x55d02ecc2ac0 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.503+0000 7f113086bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ecc5910 con 0x55d02ecc33e0 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.503+0000 7f113086bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ecbb000 con 0x55d02ecce290 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.503+0000 7f1115e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1115e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34904/0 (socket says 172.21.15.67:34904) 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1115e00700 1 -- 172.21.15.67:0/2133404519 learned_addr learned my addr 172.21.15.67:0/2133404519 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1116800700 1 --2- 172.21.15.67:0/2133404519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 0x55d02eccad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1115400700 1 --2- 172.21.15.67:0/2133404519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc33e0 0x55d02ecca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1115e00700 1 -- 172.21.15.67:0/2133404519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 msgr2=0x55d02eccad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1115e00700 1 --2- 172.21.15.67:0/2133404519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 0x55d02eccad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1115e00700 1 -- 172.21.15.67:0/2133404519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc33e0 msgr2=0x55d02ecca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1115e00700 1 --2- 172.21.15.67:0/2133404519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc33e0 0x55d02ecca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1115e00700 1 -- 172.21.15.67:0/2133404519 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d02ebd6f20 con 0x55d02ecc2ac0 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1115e00700 1 --2- 172.21.15.67:0/2133404519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd4f60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1108099ef0 tx=0x7f1108098510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97253c503ea3075a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1103e00700 1 -- 172.21.15.67:0/2133404519 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110807aea0 con 0x55d02ecc2ac0 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1103e00700 1 -- 172.21.15.67:0/2133404519 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110809a660 con 0x55d02ecc2ac0 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.504+0000 7f1103e00700 1 -- 172.21.15.67:0/2133404519 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108099450 con 0x55d02ecc2ac0 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.505+0000 7f113086bb80 1 -- 172.21.15.67:0/2133404519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 msgr2=0x55d02ecd4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.505+0000 7f113086bb80 1 --2- 172.21.15.67:0/2133404519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd4f60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1108099ef0 tx=0x7f1108098510 comp rx=0 tx=0).stop 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.505+0000 7f113086bb80 1 -- 172.21.15.67:0/2133404519 shutdown_connections 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.505+0000 7f113086bb80 1 --2- 172.21.15.67:0/2133404519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd4f60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.505+0000 7f113086bb80 1 --2- 172.21.15.67:0/2133404519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 0x55d02eccad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.505+0000 7f113086bb80 1 --2- 172.21.15.67:0/2133404519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc33e0 0x55d02ecca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.505+0000 7f113086bb80 1 -- 172.21.15.67:0/2133404519 >> 172.21.15.67:0/2133404519 conn(0x55d02ec1e720 msgr2=0x55d02ecb3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.505+0000 7f113086bb80 1 -- 172.21.15.67:0/2133404519 shutdown_connections 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.505+0000 7f113086bb80 1 -- 172.21.15.67:0/2133404519 wait complete. 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.506+0000 7f113086bb80 1 -- start start 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.506+0000 7f113086bb80 1 Processor -- start 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.506+0000 7f113086bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.506+0000 7f113086bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc33e0 0x55d02ecd4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.506+0000 7f113086bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 0x55d02ed31760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.506+0000 7f113086bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ecd4a50 con 0x55d02ecc33e0 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.506+0000 7f113086bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ed31ca0 con 0x55d02ecc2ac0 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.506+0000 7f113086bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02ed31e10 con 0x55d02ecce290 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.506+0000 7f113086bb80 10 client.0 _ll_register_callbacks cb 0x55d02ebb64f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc33e0 0x55d02ecd4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1116800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 0x55d02ed31760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:36998/0 (socket says 172.21.15.67:36998) 2024-03-30T02:37:26.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115e00700 1 -- 172.21.15.67:0/3521178139 learned_addr learned my addr 172.21.15.67:0/3521178139 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115e00700 1 -- 172.21.15.67:0/3521178139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 msgr2=0x55d02ed31760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115e00700 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 0x55d02ed31760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115e00700 1 -- 172.21.15.67:0/3521178139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc33e0 msgr2=0x55d02ecd4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115e00700 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc33e0 0x55d02ecd4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115e00700 1 -- 172.21.15.67:0/3521178139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d02ea3ebe0 con 0x55d02ecc2ac0 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f1115e00700 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd3e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1108000c00 tx=0x7f11080ca5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ac024054e428d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f10f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f10f7e00700 1 -- 172.21.15.67:0/3521178139 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108099450 con 0x55d02ecc2ac0 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f10f7e00700 1 -- 172.21.15.67:0/3521178139 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f11080995b0 con 0x55d02ecc2ac0 2024-03-30T02:37:26.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.507+0000 7f113086bb80 1 -- 172.21.15.67:0/3521178139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f110809dd90 con 0x55d02ecc2ac0 2024-03-30T02:37:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f10f7e00700 1 -- 172.21.15.67:0/3521178139 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1108095d50 con 0x55d02ecc2ac0 2024-03-30T02:37:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f113086bb80 10 client.25838 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f113086bb80 1 -- 172.21.15.67:0/3521178139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d02ea96d50 con 0x55d02ecc2ac0 2024-03-30T02:37:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f10f7e00700 1 -- 172.21.15.67:0/3521178139 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110809dd90 con 0x55d02ecc2ac0 2024-03-30T02:37:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f10f7e00700 1 -- 172.21.15.67:0/3521178139 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1108099030 con 0x55d02ecc2ac0 2024-03-30T02:37:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f113086bb80 10 client.25838 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f113086bb80 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:26.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f113086bb80 20 client.25838 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f113086bb80 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d02ed33140 0x55d02ed35530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f113086bb80 4 client.25838 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f113086bb80 1 -- 172.21.15.67:0/3521178139 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d02ea5fd70 con 0x55d02ed33140 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.508+0000 7f1115400700 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d02ed33140 0x55d02ed35530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.509+0000 7f10f7e00700 10 client.25838 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.509+0000 7f1115400700 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d02ed33140 0x55d02ed35530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9564d81849bd655e server_cookie=5371df8de451a53d in_seq=0 out_seq=0 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.510+0000 7f10f7e00700 1 -- 172.21.15.67:0/3521178139 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d02ea5fd70 con 0x55d02ed33140 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f10f7e00700 10 client.25838 handle_command_reply: tid=0 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 client.25838 shutdown 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 -- 172.21.15.67:0/3521178139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc2ac0 msgr2=0x55d02ecd3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd3e00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1108000c00 tx=0x7f11080ca5c0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 -- 172.21.15.67:0/3521178139 shutdown_connections 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d02ecce290 0x55d02ed31760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d02ecc2ac0 0x55d02ecd3e00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d02ed33140 0x55d02ed35530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 --2- 172.21.15.67:0/3521178139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d02ecc33e0 0x55d02ecd4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 -- 172.21.15.67:0/3521178139 >> 172.21.15.67:0/3521178139 conn(0x55d02ec1e720 msgr2=0x55d02ecc4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f10f7e00700 0 client.25838 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 -- 172.21.15.67:0/3521178139 shutdown_connections 2024-03-30T02:37:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 1 -- 172.21.15.67:0/3521178139 wait complete. 2024-03-30T02:37:26.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.511+0000 7f113086bb80 20 client.25838 trim_cache size 0 max 16384 2024-03-30T02:37:26.514 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:37:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:37:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "69107c60-0646-421c-b569-9c942bf30b96", 2024-03-30T02:37:26.514 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:37:26.514 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:37:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 -- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f10fc0a13a0 msgr2=0x7f10fc0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f10fc0a13a0 0x7f10fc0a3860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1110003e40 tx=0x7f1110009920 comp rx=0 tx=0).stop 2024-03-30T02:37:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 -- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118100f60 msgr2=0x7f1118197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118100f60 0x7f1118197260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f110800e9a0 tx=0x7f110800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 -- 172.21.15.67:0/1591125046 shutdown_connections 2024-03-30T02:37:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f10fc0a13a0 0x7f10fc0a3860 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1118100f60 0x7f1118197260 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f111819bb40 0x7f111819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 --2- 172.21.15.67:0/1591125046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1118106080 0x7f11181977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 -- 172.21.15.67:0/1591125046 >> 172.21.15.67:0/1591125046 conn(0x7f11180fc7c0 msgr2=0x7f11180fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 -- 172.21.15.67:0/1591125046 shutdown_connections 2024-03-30T02:37:26.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.512+0000 7f111ce00700 1 -- 172.21.15.67:0/1591125046 wait complete. 2024-03-30T02:37:26.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:37:26.589 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:26 smithi067 ceph-mon[37808]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-30T02:37:26.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:26 smithi170 ceph-mon[78798]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-30T02:37:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.733+0000 7f0826600700 1 -- 172.21.15.67:0/1422508410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f082010d520 msgr2=0x7f082010f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.733+0000 7f0826600700 1 --2- 172.21.15.67:0/1422508410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f082010d520 0x7f082010f990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f081800b0a0 tx=0x7f081800b3b0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.733+0000 7f0826600700 1 -- 172.21.15.67:0/1422508410 shutdown_connections 2024-03-30T02:37:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.733+0000 7f0826600700 1 --2- 172.21.15.67:0/1422508410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f082010d520 0x7f082010f990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.733+0000 7f0826600700 1 --2- 172.21.15.67:0/1422508410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08200731e0 0x7f08200735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.733+0000 7f0826600700 1 --2- 172.21.15.67:0/1422508410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0820073b00 0x7f082010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.733+0000 7f0826600700 1 -- 172.21.15.67:0/1422508410 >> 172.21.15.67:0/1422508410 conn(0x7f08200fc7f0 msgr2=0x7f08200fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.733+0000 7f0826600700 1 -- 172.21.15.67:0/1422508410 shutdown_connections 2024-03-30T02:37:26.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.733+0000 7f0826600700 1 -- 172.21.15.67:0/1422508410 wait complete. 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.734+0000 7f0826600700 1 Processor -- start 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.734+0000 7f0826600700 1 -- start start 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.734+0000 7f0826600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08200731e0 0x7f0820199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.734+0000 7f0826600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0820073b00 0x7f0820199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.734+0000 7f0826600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082010d520 0x7f082019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.734+0000 7f0826600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082019e420 con 0x7f0820073b00 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.734+0000 7f0826600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082019e590 con 0x7f08200731e0 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.734+0000 7f0826600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082019e750 con 0x7f082010d520 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0820073b00 0x7f0820199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0820073b00 0x7f0820199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34948/0 (socket says 172.21.15.67:34948) 2024-03-30T02:37:26.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081ea00700 1 -- 172.21.15.67:0/4095554552 learned_addr learned my addr 172.21.15.67:0/4095554552 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081f400700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08200731e0 0x7f0820199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081fe00700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082010d520 0x7f082019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081ea00700 1 -- 172.21.15.67:0/4095554552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082010d520 msgr2=0x7f082019dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081ea00700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082010d520 0x7f082019dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081ea00700 1 -- 172.21.15.67:0/4095554552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08200731e0 msgr2=0x7f0820199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081ea00700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08200731e0 0x7f0820199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081ea00700 1 -- 172.21.15.67:0/4095554552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818009d00 con 0x7f0820073b00 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.735+0000 7f081ea00700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0820073b00 0x7f0820199a40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f080c00e9a0 tx=0x7f080c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.736+0000 7f0817e00700 1 -- 172.21.15.67:0/4095554552 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c009790 con 0x7f0820073b00 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.736+0000 7f0817e00700 1 -- 172.21.15.67:0/4095554552 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080c010430 con 0x7f0820073b00 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.736+0000 7f0826600700 1 -- 172.21.15.67:0/4095554552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082019ea30 con 0x7f0820073b00 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.736+0000 7f0817e00700 1 -- 172.21.15.67:0/4095554552 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c018680 con 0x7f0820073b00 2024-03-30T02:37:26.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.736+0000 7f0826600700 1 -- 172.21.15.67:0/4095554552 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08201a6270 con 0x7f0820073b00 2024-03-30T02:37:26.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.739+0000 7f0817e00700 1 -- 172.21.15.67:0/4095554552 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f080c0105a0 con 0x7f0820073b00 2024-03-30T02:37:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.741+0000 7f0817e00700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08040a1320 0x7f08040a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.741+0000 7f083a096b80 1 Processor -- start 2024-03-30T02:37:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.741+0000 7f0817e00700 1 -- 172.21.15.67:0/4095554552 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f080c014070 con 0x7f0820073b00 2024-03-30T02:37:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.741+0000 7f083a096b80 1 -- start start 2024-03-30T02:37:26.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.741+0000 7f081f400700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08040a1320 0x7f08040a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f083a096b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f0439b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f083a096b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f043ef0 0x56168f0570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f083a096b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f0599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f083a096b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168ef9c990 con 0x56168f057600 2024-03-30T02:37:26.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f083a096b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168ef9a350 con 0x56168f0435d0 2024-03-30T02:37:26.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f083a096b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168ef95c40 con 0x56168f043ef0 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f081ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f043ef0 0x56168f0570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f081f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f0439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f081ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f043ef0 0x56168f0570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44272/0 (socket says 172.21.15.67:44272) 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.742+0000 7f081ea00700 1 -- 172.21.15.67:0/62612075 learned_addr learned my addr 172.21.15.67:0/62612075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.743+0000 7f081fe00700 1 --2- 172.21.15.67:0/62612075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f0599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.743+0000 7f081f400700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08040a1320 0x7f08040a37e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0810011550 tx=0x7f0810011910 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.743+0000 7f081f400700 1 -- 172.21.15.67:0/62612075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f043ef0 msgr2=0x56168f0570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.743+0000 7f081f400700 1 --2- 172.21.15.67:0/62612075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f043ef0 0x56168f0570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.743+0000 7f081f400700 1 -- 172.21.15.67:0/62612075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 msgr2=0x56168f0599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.743+0000 7f081f400700 1 --2- 172.21.15.67:0/62612075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f0599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.743+0000 7f081f400700 1 -- 172.21.15.67:0/62612075 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56168ee173b0 con 0x56168f0435d0 2024-03-30T02:37:26.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.743+0000 7f081fe00700 1 --2- 172.21.15.67:0/62612075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f0599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:37:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.743+0000 7f081f400700 1 --2- 172.21.15.67:0/62612075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f0439b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08100043e0 tx=0x7f081001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=730418f457f61266 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.744+0000 7f0814c00700 1 -- 172.21.15.67:0/62612075 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810003c50 con 0x56168f0435d0 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.744+0000 7f0814c00700 1 -- 172.21.15.67:0/62612075 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0810024070 con 0x56168f0435d0 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.744+0000 7f0814c00700 1 -- 172.21.15.67:0/62612075 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810020450 con 0x56168f0435d0 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 -- 172.21.15.67:0/62612075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 msgr2=0x56168f0439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 --2- 172.21.15.67:0/62612075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f0439b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08100043e0 tx=0x7f081001d000 comp rx=0 tx=0).stop 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 -- 172.21.15.67:0/62612075 shutdown_connections 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 --2- 172.21.15.67:0/62612075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f0599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 --2- 172.21.15.67:0/62612075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f0439b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 --2- 172.21.15.67:0/62612075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f043ef0 0x56168f0570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 -- 172.21.15.67:0/62612075 >> 172.21.15.67:0/62612075 conn(0x56168efa36a0 msgr2=0x56168f03fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 -- 172.21.15.67:0/62612075 shutdown_connections 2024-03-30T02:37:26.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 -- 172.21.15.67:0/62612075 wait complete. 2024-03-30T02:37:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.745+0000 7f083a096b80 1 -- start start 2024-03-30T02:37:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.746+0000 7f083a096b80 1 Processor -- start 2024-03-30T02:37:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.746+0000 7f083a096b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f0435d0 0x56168f057020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.746+0000 7f083a096b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 0x56168f0542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.746+0000 7f083a096b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f054920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.746+0000 7f083a096b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168ef31d80 con 0x56168f057600 2024-03-30T02:37:26.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.746+0000 7f083a096b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168ef31a20 con 0x56168f043ef0 2024-03-30T02:37:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.746+0000 7f083a096b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168ee196a0 con 0x56168f0435d0 2024-03-30T02:37:26.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.746+0000 7f083a096b80 10 client.0 _ll_register_callbacks cb 0x56168ef3b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:26.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f054920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f054920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34966/0 (socket says 172.21.15.67:34966) 2024-03-30T02:37:26.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081fe00700 1 -- 172.21.15.67:0/3419790282 learned_addr learned my addr 172.21.15.67:0/3419790282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081f400700 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f0435d0 0x56168f057020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081ea00700 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 0x56168f0542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081f400700 1 -- 172.21.15.67:0/3419790282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 msgr2=0x56168f0542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081f400700 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 0x56168f0542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081f400700 1 -- 172.21.15.67:0/3419790282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 msgr2=0x56168f054920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081f400700 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f054920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.747+0000 7f081f400700 1 -- 172.21.15.67:0/3419790282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0810004150 con 0x56168f0435d0 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.748+0000 7f081f400700 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f0435d0 0x56168f057020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0810003960 tx=0x7f0810005c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ff90322aa9840c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.748+0000 7f0809600700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.748+0000 7f0809600700 1 -- 172.21.15.67:0/3419790282 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810003a50 con 0x56168f0435d0 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.748+0000 7f083a096b80 1 -- 172.21.15.67:0/3419790282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56168ef56380 con 0x56168f0435d0 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.748+0000 7f0809600700 1 -- 172.21.15.67:0/3419790282 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f0810003bb0 con 0x56168f0435d0 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.748+0000 7f0809600700 1 -- 172.21.15.67:0/3419790282 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0810003d70 con 0x56168f0435d0 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.748+0000 7f083a096b80 10 client.25840 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.748+0000 7f083a096b80 1 -- 172.21.15.67:0/3419790282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56168edc3b60 con 0x56168f0435d0 2024-03-30T02:37:26.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.749+0000 7f0809600700 1 -- 172.21.15.67:0/3419790282 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56168ef56380 con 0x56168f0435d0 2024-03-30T02:37:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:26 smithi082 ceph-mon[79176]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-30T02:37:26.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.749+0000 7f0809600700 1 -- 172.21.15.67:0/3419790282 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0810031020 con 0x56168f0435d0 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.749+0000 7f083a096b80 10 client.25840 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.749+0000 7f083a096b80 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.749+0000 7f083a096b80 20 client.25840 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.749+0000 7f083a096b80 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56168f066f30 0x56168f069350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.749+0000 7f083a096b80 4 client.25840 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.749+0000 7f083a096b80 1 -- 172.21.15.67:0/3419790282 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56168ef8e700 con 0x56168f066f30 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.749+0000 7f081ea00700 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56168f066f30 0x56168f069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.750+0000 7f0809600700 10 client.25840 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.750+0000 7f081ea00700 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56168f066f30 0x56168f069350 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd539bca42e71b8d server_cookie=cb86638099c82e3 in_seq=0 out_seq=0 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.750+0000 7f0809600700 1 -- 172.21.15.67:0/3419790282 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56168ef8e700 con 0x56168f066f30 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.750+0000 7f0809600700 10 client.25840 handle_command_reply: tid=0 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.750+0000 7f083a096b80 1 client.25840 shutdown 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 -- 172.21.15.67:0/3419790282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f0435d0 msgr2=0x56168f057020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f0435d0 0x56168f057020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0810003960 tx=0x7f0810005c20 comp rx=0 tx=0).stop 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 -- 172.21.15.67:0/3419790282 shutdown_connections 2024-03-30T02:37:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56168f066f30 0x56168f069350 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f057600 0x56168f054920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f0809600700 0 client.25840 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f0435d0 0x56168f057020 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 --2- 172.21.15.67:0/3419790282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 0x56168f0542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 -- 172.21.15.67:0/3419790282 >> 172.21.15.67:0/3419790282 conn(0x56168efa36a0 msgr2=0x56168f0456c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 -- 172.21.15.67:0/3419790282 shutdown_connections 2024-03-30T02:37:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 1 -- 172.21.15.67:0/3419790282 wait complete. 2024-03-30T02:37:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.751+0000 7f083a096b80 20 client.25840 trim_cache size 0 max 16384 2024-03-30T02:37:26.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.769+0000 7f083a096b80 1 Processor -- start 2024-03-30T02:37:26.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.769+0000 7f083a096b80 1 -- start start 2024-03-30T02:37:26.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.769+0000 7f083a096b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f0435d0 0x56168f05e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.769+0000 7f083a096b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 0x56168f062970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.769+0000 7f083a096b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 0x56168f062eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.769+0000 7f083a096b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168f05e760 con 0x56168f0435d0 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.769+0000 7f083a096b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168f0433c0 con 0x56168f043ef0 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.769+0000 7f083a096b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168f03ff20 con 0x56168f057600 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f0435d0 0x56168f05e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f0435d0 0x56168f05e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34980/0 (socket says 172.21.15.67:34980) 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081f400700 1 -- 172.21.15.67:0/1199144134 learned_addr learned my addr 172.21.15.67:0/1199144134 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081fe00700 1 --2- 172.21.15.67:0/1199144134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 0x56168f062eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081ea00700 1 --2- 172.21.15.67:0/1199144134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 0x56168f062970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081f400700 1 -- 172.21.15.67:0/1199144134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 msgr2=0x56168f062eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081f400700 1 --2- 172.21.15.67:0/1199144134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 0x56168f062eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081f400700 1 -- 172.21.15.67:0/1199144134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 msgr2=0x56168f062970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081f400700 1 --2- 172.21.15.67:0/1199144134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 0x56168f062970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081f400700 1 -- 172.21.15.67:0/1199144134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0820104df0 con 0x56168f0435d0 2024-03-30T02:37:26.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.770+0000 7f081f400700 1 --2- 172.21.15.67:0/1199144134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f0435d0 0x56168f05e220 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0810020c90 tx=0x7f0810020cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32748d763a52058 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f0815600700 1 -- 172.21.15.67:0/1199144134 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810009b60 con 0x56168f0435d0 2024-03-30T02:37:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f0815600700 1 -- 172.21.15.67:0/1199144134 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0810018070 con 0x56168f0435d0 2024-03-30T02:37:26.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f0815600700 1 -- 172.21.15.67:0/1199144134 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081000f6d0 con 0x56168f0435d0 2024-03-30T02:37:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f083a096b80 1 -- 172.21.15.67:0/1199144134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f0435d0 msgr2=0x56168f05e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f083a096b80 1 --2- 172.21.15.67:0/1199144134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f0435d0 0x56168f05e220 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0810020c90 tx=0x7f0810020cc0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f083a096b80 1 -- 172.21.15.67:0/1199144134 shutdown_connections 2024-03-30T02:37:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f083a096b80 1 --2- 172.21.15.67:0/1199144134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f0435d0 0x56168f05e220 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f083a096b80 1 --2- 172.21.15.67:0/1199144134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 0x56168f062eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f083a096b80 1 --2- 172.21.15.67:0/1199144134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f043ef0 0x56168f062970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.771+0000 7f083a096b80 1 -- 172.21.15.67:0/1199144134 >> 172.21.15.67:0/1199144134 conn(0x56168efa36a0 msgr2=0x56168f038f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.772+0000 7f083a096b80 1 -- 172.21.15.67:0/1199144134 shutdown_connections 2024-03-30T02:37:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.772+0000 7f083a096b80 1 -- 172.21.15.67:0/1199144134 wait complete. 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.772+0000 7f083a096b80 1 -- start start 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.772+0000 7f083a096b80 1 Processor -- start 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f083a096b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f05dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f083a096b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f043ef0 0x56168f05e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f083a096b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 0x56168f05c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f083a096b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168f05cb30 con 0x56168f043ef0 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f083a096b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168f05cca0 con 0x56168f0435d0 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f083a096b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56168f05ce60 con 0x56168f057600 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f083a096b80 10 client.0 _ll_register_callbacks cb 0x56168ef3b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f05dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f043ef0 0x56168f05e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f05dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37054/0 (socket says 172.21.15.67:37054) 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081f400700 1 -- 172.21.15.67:0/153430701 learned_addr learned my addr 172.21.15.67:0/153430701 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081fe00700 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 0x56168f05c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081f400700 1 -- 172.21.15.67:0/153430701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 msgr2=0x56168f05c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081f400700 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 0x56168f05c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081f400700 1 -- 172.21.15.67:0/153430701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f043ef0 msgr2=0x56168f05e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081f400700 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f043ef0 0x56168f05e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081f400700 1 -- 172.21.15.67:0/153430701 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56168edc3b60 con 0x56168f0435d0 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081ea00700 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f043ef0 0x56168f05e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.773+0000 7f081fe00700 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 0x56168f05c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f081f400700 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f05dde0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0810005850 tx=0x7f0810009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=734c8c7c0005ce5b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f0809600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f0809600700 1 -- 172.21.15.67:0/153430701 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08100206a0 con 0x56168f0435d0 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f083a096b80 1 -- 172.21.15.67:0/153430701 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0810024070 con 0x56168f0435d0 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f0809600700 1 -- 172.21.15.67:0/153430701 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f0810003a50 con 0x56168f0435d0 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f0809600700 1 -- 172.21.15.67:0/153430701 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0810003c30 con 0x56168f0435d0 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f083a096b80 10 client.25856 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f083a096b80 1 -- 172.21.15.67:0/153430701 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56168ef5bea0 con 0x56168f0435d0 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f0809600700 1 -- 172.21.15.67:0/153430701 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0810024070 con 0x56168f0435d0 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.774+0000 7f0809600700 1 -- 172.21.15.67:0/153430701 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0810031070 con 0x56168f0435d0 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.775+0000 7f083a096b80 10 client.25856 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.775+0000 7f083a096b80 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.775+0000 7f083a096b80 20 client.25856 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.775+0000 7f083a096b80 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56168f0bbca0 0x56168f0be090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.775+0000 7f083a096b80 4 client.25856 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.775+0000 7f083a096b80 1 -- 172.21.15.67:0/153430701 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56168ede4cf0 con 0x56168f0bbca0 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.775+0000 7f081ea00700 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56168f0bbca0 0x56168f0be090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.775+0000 7f081ea00700 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56168f0bbca0 0x56168f0be090 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2976595b8c235ce server_cookie=604db34f99392e82 in_seq=0 out_seq=0 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.775+0000 7f0809600700 10 client.25856 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f0809600700 1 -- 172.21.15.67:0/153430701 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56168ede4cf0 con 0x56168f0bbca0 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f0809600700 10 client.25856 handle_command_reply: tid=0 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 client.25856 shutdown 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 -- 172.21.15.67:0/153430701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 msgr2=0x56168f05dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f05dde0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0810005850 tx=0x7f0810009eb0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 -- 172.21.15.67:0/153430701 shutdown_connections 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56168f057600 0x56168f05c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56168f0435d0 0x56168f05dde0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56168f0bbca0 0x56168f0be090 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 --2- 172.21.15.67:0/153430701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56168f043ef0 0x56168f05e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f0809600700 0 client.25856 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 -- 172.21.15.67:0/153430701 >> 172.21.15.67:0/153430701 conn(0x56168efa36a0 msgr2=0x56168f062970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 -- 172.21.15.67:0/153430701 shutdown_connections 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 1 -- 172.21.15.67:0/153430701 wait complete. 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.776+0000 7f083a096b80 20 client.25856 trim_cache size 0 max 16384 2024-03-30T02:37:26.781 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:37:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T02:37:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:37:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "69107c60-0646-421c-b569-9c942bf30b96": { 2024-03-30T02:37:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:37:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "69107c60-0646-421c-b569-9c942bf30b96", 2024-03-30T02:37:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:37:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:37:26.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:37:26.782 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 -- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08040a1320 msgr2=0x7f08040a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08040a1320 0x7f08040a37e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0810011550 tx=0x7f0810011910 comp rx=0 tx=0).stop 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 -- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0820073b00 msgr2=0x7f0820199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0820073b00 0x7f0820199a40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f080c00e9a0 tx=0x7f080c00ecb0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 -- 172.21.15.67:0/4095554552 shutdown_connections 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08040a1320 0x7f08040a37e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f082010d520 0x7f082019dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08200731e0 0x7f0820199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 --2- 172.21.15.67:0/4095554552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0820073b00 0x7f0820199a40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 -- 172.21.15.67:0/4095554552 >> 172.21.15.67:0/4095554552 conn(0x7f08200fc7f0 msgr2=0x7f08200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 -- 172.21.15.67:0/4095554552 shutdown_connections 2024-03-30T02:37:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:26.777+0000 7f0826600700 1 -- 172.21.15.67:0/4095554552 wait complete. 2024-03-30T02:37:26.796 INFO:tasks.cephfs.filesystem:scrub status for tag:69107c60-0646-421c-b569-9c942bf30b96 - {'path': '/', 'tag': '69107c60-0646-421c-b569-9c942bf30b96', 'options': 'recursive,force'} 2024-03-30T02:37:28.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:28 smithi067 ceph-mon[37808]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:28.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:28 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:37:28.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:37:28.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:37:28.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:37:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:28 smithi170 ceph-mon[78798]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:28 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:37:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:37:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:37:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:37:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:28 smithi082 ceph-mon[79176]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:28 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:37:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:37:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:37:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:37:30.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:30 smithi067 ceph-mon[37808]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-03-30T02:37:30.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:30 smithi170 ceph-mon[78798]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-03-30T02:37:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:30 smithi082 ceph-mon[79176]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-03-30T02:37:32.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:32 smithi082 ceph-mon[79176]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-03-30T02:37:32.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:32 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:37:32.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:32 smithi067 ceph-mon[37808]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-03-30T02:37:32.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:32 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:37:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:32 smithi170 ceph-mon[78798]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-03-30T02:37:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:32 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:37:34.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:34 smithi082 ceph-mon[79176]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-30T02:37:34.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:34 smithi067 ceph-mon[37808]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-30T02:37:34.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:34 smithi170 ceph-mon[78798]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-30T02:37:36.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:36 smithi082 ceph-mon[79176]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-30T02:37:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:36 smithi067 ceph-mon[37808]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-30T02:37:36.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:36 smithi170 ceph-mon[78798]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-03-30T02:37:38.500 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:38 smithi082 ceph-mon[79176]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-30T02:37:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:38 smithi067 ceph-mon[37808]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-30T02:37:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:38 smithi170 ceph-mon[78798]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-03-30T02:37:40.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:40 smithi082 ceph-mon[79176]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-03-30T02:37:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:40 smithi067 ceph-mon[37808]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-03-30T02:37:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:40 smithi170 ceph-mon[78798]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-03-30T02:37:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:42 smithi067 ceph-mon[37808]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:37:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:42 smithi170 ceph-mon[78798]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:37:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:42 smithi082 ceph-mon[79176]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-30T02:37:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:44 smithi067 ceph-mon[37808]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:44 smithi170 ceph-mon[78798]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:44 smithi082 ceph-mon[79176]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:46 smithi067 ceph-mon[37808]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-30T02:37:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:46 smithi170 ceph-mon[78798]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-30T02:37:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:46 smithi082 ceph-mon[79176]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-30T02:37:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:48 smithi067 ceph-mon[37808]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-30T02:37:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:48 smithi170 ceph-mon[78798]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-30T02:37:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:48 smithi082 ceph-mon[79176]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-30T02:37:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:50 smithi067 ceph-mon[37808]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-30T02:37:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:50 smithi170 ceph-mon[78798]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-30T02:37:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:50 smithi082 ceph-mon[79176]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-30T02:37:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:52 smithi067 ceph-mon[37808]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-30T02:37:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:52 smithi170 ceph-mon[78798]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-30T02:37:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:52 smithi082 ceph-mon[79176]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-03-30T02:37:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:54 smithi067 ceph-mon[37808]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-30T02:37:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:54 smithi170 ceph-mon[78798]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-30T02:37:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:54 smithi082 ceph-mon[79176]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-30T02:37:56.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:56 smithi067 ceph-mon[37808]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-30T02:37:56.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:56 smithi170 ceph-mon[78798]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-30T02:37:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:56 smithi082 ceph-mon[79176]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-03-30T02:37:56.797 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:37:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.073+0000 7f1509000700 1 -- 172.21.15.67:0/2821279008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 msgr2=0x7f15041020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.073+0000 7f1509000700 1 --2- 172.21.15.67:0/2821279008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 0x7f15041020d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009990 tx=0x7f14f4009ca0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.074+0000 7f1509000700 1 -- 172.21.15.67:0/2821279008 shutdown_connections 2024-03-30T02:37:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.074+0000 7f1509000700 1 --2- 172.21.15.67:0/2821279008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 0x7f15041020d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.074+0000 7f1509000700 1 --2- 172.21.15.67:0/2821279008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f150410b110 0x7f150410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.074+0000 7f1509000700 1 --2- 172.21.15.67:0/2821279008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1504102610 0x7f150410aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.074+0000 7f1509000700 1 -- 172.21.15.67:0/2821279008 >> 172.21.15.67:0/2821279008 conn(0x7f15040fb510 msgr2=0x7f15040fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.074+0000 7f1509000700 1 -- 172.21.15.67:0/2821279008 shutdown_connections 2024-03-30T02:37:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.074+0000 7f1509000700 1 -- 172.21.15.67:0/2821279008 wait complete. 2024-03-30T02:37:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.075+0000 7f1509000700 1 Processor -- start 2024-03-30T02:37:57.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.075+0000 7f1509000700 1 -- start start 2024-03-30T02:37:57.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.075+0000 7f1509000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 0x7f15041992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.075+0000 7f1509000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504102610 0x7f15041997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.075+0000 7f1509000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f150410b110 0x7f150419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.075+0000 7f1509000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419e180 con 0x7f1504101cf0 2024-03-30T02:37:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.075+0000 7f1509000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419e2f0 con 0x7f150410b110 2024-03-30T02:37:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.075+0000 7f1509000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419e4b0 con 0x7f1504102610 2024-03-30T02:37:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1503400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 0x7f15041992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1503400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 0x7f15041992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40402/0 (socket says 172.21.15.67:40402) 2024-03-30T02:37:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1503400700 1 -- 172.21.15.67:0/1293566716 learned_addr learned my addr 172.21.15.67:0/1293566716 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1503e00700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f150410b110 0x7f150419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1503400700 1 -- 172.21.15.67:0/1293566716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504102610 msgr2=0x7f15041997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1502a00700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504102610 0x7f15041997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1503400700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504102610 0x7f15041997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1503400700 1 -- 172.21.15.67:0/1293566716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f150410b110 msgr2=0x7f150419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1503400700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f150410b110 0x7f150419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1503400700 1 -- 172.21.15.67:0/1293566716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f40095f0 con 0x7f1504101cf0 2024-03-30T02:37:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.076+0000 7f1502a00700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504102610 0x7f15041997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:37:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.077+0000 7f1503400700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 0x7f15041992b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005910 tx=0x7f14f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.077+0000 7f14fbe00700 1 -- 172.21.15.67:0/1293566716 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f401d070 con 0x7f1504101cf0 2024-03-30T02:37:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.077+0000 7f1509000700 1 -- 172.21.15.67:0/1293566716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150419e730 con 0x7f1504101cf0 2024-03-30T02:37:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.077+0000 7f14fbe00700 1 -- 172.21.15.67:0/1293566716 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f4022470 con 0x7f1504101cf0 2024-03-30T02:37:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.077+0000 7f14fbe00700 1 -- 172.21.15.67:0/1293566716 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4017650 con 0x7f1504101cf0 2024-03-30T02:37:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.077+0000 7f1509000700 1 -- 172.21.15.67:0/1293566716 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15041a5fd0 con 0x7f1504101cf0 2024-03-30T02:37:57.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.082+0000 7f14fbe00700 1 -- 172.21.15.67:0/1293566716 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f14f40225e0 con 0x7f1504101cf0 2024-03-30T02:37:57.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.083+0000 7f151e06fb80 1 Processor -- start 2024-03-30T02:37:57.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.084+0000 7f151e06fb80 1 -- start start 2024-03-30T02:37:57.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.084+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed63b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.084+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.084+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.084+0000 7f151e06fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fecbc990 con 0x55c1fed640c0 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.084+0000 7f151e06fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fecba350 con 0x55c1fed637a0 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.084+0000 7f151e06fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fecb5c40 con 0x55c1fed6efc0 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.084+0000 7f14fbe00700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f14e80a1320 0x7f14e80a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed63b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37026/0 (socket says 172.21.15.67:37026) 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503e00700 1 -- 172.21.15.67:0/517528402 learned_addr learned my addr 172.21.15.67:0/517528402 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1502a00700 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f14fbe00700 1 -- 172.21.15.67:0/1293566716 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f14f40ce290 con 0x7f1504101cf0 2024-03-30T02:37:57.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503e00700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f14e80a1320 0x7f14e80a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503400700 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503e00700 1 -- 172.21.15.67:0/517528402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 msgr2=0x55c1fed713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503e00700 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503e00700 1 -- 172.21.15.67:0/517528402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 msgr2=0x55c1fed6ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503e00700 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503e00700 1 -- 172.21.15.67:0/517528402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1feb373b0 con 0x55c1fed637a0 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.085+0000 7f1503400700 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.086+0000 7f1503e00700 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed63b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00fc70 tx=0x7f14fc00ff80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e0673fddb70359 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.086+0000 7f1503e00700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f14e80a1320 0x7f14e80a37e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009000 tx=0x7f14fc01e000 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.086+0000 7f14f8c00700 1 -- 172.21.15.67:0/517528402 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00e040 con 0x55c1fed637a0 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.086+0000 7f14f8c00700 1 -- 172.21.15.67:0/517528402 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14fc00f040 con 0x55c1fed637a0 2024-03-30T02:37:57.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.086+0000 7f14f8c00700 1 -- 172.21.15.67:0/517528402 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc01a940 con 0x55c1fed637a0 2024-03-30T02:37:57.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.087+0000 7f151e06fb80 1 -- 172.21.15.67:0/517528402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 msgr2=0x55c1fed63b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.087+0000 7f151e06fb80 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed63b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00fc70 tx=0x7f14fc00ff80 comp rx=0 tx=0).stop 2024-03-30T02:37:57.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.087+0000 7f151e06fb80 1 -- 172.21.15.67:0/517528402 shutdown_connections 2024-03-30T02:37:57.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.087+0000 7f151e06fb80 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.087+0000 7f151e06fb80 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed63b80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.087+0000 7f151e06fb80 1 --2- 172.21.15.67:0/517528402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.087+0000 7f151e06fb80 1 -- 172.21.15.67:0/517528402 >> 172.21.15.67:0/517528402 conn(0x55c1fecc36a0 msgr2=0x55c1fed5fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:57.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.088+0000 7f151e06fb80 1 -- 172.21.15.67:0/517528402 shutdown_connections 2024-03-30T02:37:57.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.088+0000 7f151e06fb80 1 -- 172.21.15.67:0/517528402 wait complete. 2024-03-30T02:37:57.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.088+0000 7f151e06fb80 1 -- start start 2024-03-30T02:37:57.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.089+0000 7f151e06fb80 1 Processor -- start 2024-03-30T02:37:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.090+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed637a0 0x55c1fed6ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.090+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.090+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed6efc0 0x55c1fed6c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.090+0000 7f151e06fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fec51d80 con 0x55c1fed640c0 2024-03-30T02:37:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.090+0000 7f151e06fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fec51a20 con 0x55c1fed6efc0 2024-03-30T02:37:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.090+0000 7f151e06fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1feb396a0 con 0x55c1fed637a0 2024-03-30T02:37:57.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.090+0000 7f151e06fb80 10 client.0 _ll_register_callbacks cb 0x55c1fec5b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:57.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.090+0000 7f1502a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1502a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40422/0 (socket says 172.21.15.67:40422) 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1502a00700 1 -- 172.21.15.67:0/392313817 learned_addr learned my addr 172.21.15.67:0/392313817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1503400700 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed637a0 0x55c1fed6ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1503e00700 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed6efc0 0x55c1fed6c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1503400700 1 -- 172.21.15.67:0/392313817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed6efc0 msgr2=0x55c1fed6c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1503400700 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed6efc0 0x55c1fed6c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1503400700 1 -- 172.21.15.67:0/392313817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 msgr2=0x55c1fed6bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1503400700 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1503400700 1 -- 172.21.15.67:0/392313817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc00f820 con 0x55c1fed637a0 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f1503400700 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed637a0 0x55c1fed6ea00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f14f40cb5d0 tx=0x7f14f4099520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd6b9b1e917f2793 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.091+0000 7f14ed600700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.092+0000 7f14ed600700 1 -- 172.21.15.67:0/392313817 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f409a8d0 con 0x55c1fed637a0 2024-03-30T02:37:57.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.092+0000 7f14ed600700 1 -- 172.21.15.67:0/392313817 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f14f409b670 con 0x55c1fed637a0 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.092+0000 7f151e06fb80 1 -- 172.21.15.67:0/392313817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1fec76380 con 0x55c1fed637a0 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.092+0000 7f14ed600700 1 -- 172.21.15.67:0/392313817 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14f409af70 con 0x55c1fed637a0 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.092+0000 7f151e06fb80 10 client.25852 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.092+0000 7f151e06fb80 1 -- 172.21.15.67:0/392313817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c1feae3b60 con 0x55c1fed637a0 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.092+0000 7f14ed600700 1 -- 172.21.15.67:0/392313817 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1fec76380 con 0x55c1fed637a0 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.092+0000 7f14ed600700 1 -- 172.21.15.67:0/392313817 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f14f40a2ed0 con 0x55c1fed637a0 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.093+0000 7f151e06fb80 10 client.25852 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.093+0000 7f151e06fb80 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.093+0000 7f151e06fb80 20 client.25852 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.093+0000 7f151e06fb80 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1fed86f60 0x55c1fed89380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.093+0000 7f151e06fb80 4 client.25852 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.093+0000 7f151e06fb80 1 -- 172.21.15.67:0/392313817 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1fecae700 con 0x55c1fed86f60 2024-03-30T02:37:57.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.093+0000 7f1502a00700 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1fed86f60 0x55c1fed89380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.094+0000 7f1502a00700 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1fed86f60 0x55c1fed89380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc1c4ef0dcc9b94 server_cookie=4eba901fb9af1ef5 in_seq=0 out_seq=0 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.094+0000 7f14ed600700 10 client.25852 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.094+0000 7f14ed600700 1 -- 172.21.15.67:0/392313817 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c1fecae700 con 0x55c1fed86f60 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.094+0000 7f14ed600700 10 client.25852 handle_command_reply: tid=0 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.094+0000 7f151e06fb80 1 client.25852 shutdown 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 -- 172.21.15.67:0/392313817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed637a0 msgr2=0x55c1fed6ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed637a0 0x55c1fed6ea00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f14f40cb5d0 tx=0x7f14f4099520 comp rx=0 tx=0).stop 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 -- 172.21.15.67:0/392313817 shutdown_connections 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed6efc0 0x55c1fed6c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed637a0 0x55c1fed6ea00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1fed86f60 0x55c1fed89380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 --2- 172.21.15.67:0/392313817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f14ed600700 0 client.25852 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 -- 172.21.15.67:0/392313817 >> 172.21.15.67:0/392313817 conn(0x55c1fecc36a0 msgr2=0x55c1fed65890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 -- 172.21.15.67:0/392313817 shutdown_connections 2024-03-30T02:37:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 1 -- 172.21.15.67:0/392313817 wait complete. 2024-03-30T02:37:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.095+0000 7f151e06fb80 20 client.25852 trim_cache size 0 max 16384 2024-03-30T02:37:57.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.125+0000 7f151e06fb80 1 Processor -- start 2024-03-30T02:37:57.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.125+0000 7f151e06fb80 1 -- start start 2024-03-30T02:37:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed6bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed75c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f151e06fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fed6d2f0 con 0x55c1fed640c0 2024-03-30T02:37:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f151e06fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fed5f520 con 0x55c1fed637a0 2024-03-30T02:37:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f151e06fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fed76180 con 0x55c1fed6efc0 2024-03-30T02:37:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f1503400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed6bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f1502a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f1503400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed6bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37048/0 (socket says 172.21.15.67:37048) 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f1503400700 1 -- 172.21.15.67:0/3402983471 learned_addr learned my addr 172.21.15.67:0/3402983471 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.126+0000 7f1503e00700 1 --2- 172.21.15.67:0/3402983471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed75c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f1503400700 1 -- 172.21.15.67:0/3402983471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 msgr2=0x55c1fed75c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f1503400700 1 --2- 172.21.15.67:0/3402983471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed75c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f1503400700 1 -- 172.21.15.67:0/3402983471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 msgr2=0x55c1fed6c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f1503400700 1 --2- 172.21.15.67:0/3402983471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f1503400700 1 -- 172.21.15.67:0/3402983471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1fec7bea0 con 0x55c1fed637a0 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f1502a00700 1 --2- 172.21.15.67:0/3402983471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f1503400700 1 --2- 172.21.15.67:0/3402983471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed6bd50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f14f4000c00 tx=0x7f14f4022ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6181418dd3ea07a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f14f9600700 1 -- 172.21.15.67:0/3402983471 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f409a720 con 0x55c1fed637a0 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f14f9600700 1 -- 172.21.15.67:0/3402983471 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f40cb510 con 0x55c1fed637a0 2024-03-30T02:37:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.127+0000 7f14f9600700 1 -- 172.21.15.67:0/3402983471 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f409af70 con 0x55c1fed637a0 2024-03-30T02:37:57.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.128+0000 7f151e06fb80 1 -- 172.21.15.67:0/3402983471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 msgr2=0x55c1fed6bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.128+0000 7f151e06fb80 1 --2- 172.21.15.67:0/3402983471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed6bd50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f14f4000c00 tx=0x7f14f4022ad0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.128+0000 7f151e06fb80 1 -- 172.21.15.67:0/3402983471 shutdown_connections 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.128+0000 7f151e06fb80 1 --2- 172.21.15.67:0/3402983471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed75c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.128+0000 7f151e06fb80 1 --2- 172.21.15.67:0/3402983471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed6bd50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.128+0000 7f151e06fb80 1 --2- 172.21.15.67:0/3402983471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed6c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.128+0000 7f151e06fb80 1 -- 172.21.15.67:0/3402983471 >> 172.21.15.67:0/3402983471 conn(0x55c1fecc36a0 msgr2=0x55c1fed6b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.128+0000 7f151e06fb80 1 -- 172.21.15.67:0/3402983471 shutdown_connections 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.128+0000 7f151e06fb80 1 -- 172.21.15.67:0/3402983471 wait complete. 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.129+0000 7f151e06fb80 1 -- start start 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.129+0000 7f151e06fb80 1 Processor -- start 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed73d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f151e06fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed74250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f151e06fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fed747d0 con 0x55c1fed640c0 2024-03-30T02:37:57.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f151e06fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fed74940 con 0x55c1fed637a0 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f151e06fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1fed74b00 con 0x55c1fed6efc0 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f151e06fb80 10 client.0 _ll_register_callbacks cb 0x55c1fec5b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1503400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1503400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37058/0 (socket says 172.21.15.67:37058) 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1503400700 1 -- 172.21.15.67:0/1541182379 learned_addr learned my addr 172.21.15.67:0/1541182379 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1503e00700 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed74250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1502a00700 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed73d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1503400700 1 -- 172.21.15.67:0/1541182379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 msgr2=0x55c1fed74250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1503400700 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed74250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1503400700 1 -- 172.21.15.67:0/1541182379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 msgr2=0x55c1fed73d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1503400700 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed73d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.130+0000 7f1503400700 1 -- 172.21.15.67:0/1541182379 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1feae3b60 con 0x55c1fed637a0 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.131+0000 7f1503e00700 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed74250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.131+0000 7f1503400700 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed758a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14f409ef60 tx=0x7f14f4005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd89578851819ab0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:57.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.131+0000 7f14ed600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.131+0000 7f14ed600700 1 -- 172.21.15.67:0/1541182379 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f409a720 con 0x55c1fed637a0 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.131+0000 7f14ed600700 1 -- 172.21.15.67:0/1541182379 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f14f409b670 con 0x55c1fed637a0 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.131+0000 7f151e06fb80 1 -- 172.21.15.67:0/1541182379 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1feb04cf0 con 0x55c1fed637a0 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.131+0000 7f14ed600700 1 -- 172.21.15.67:0/1541182379 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14f40cb510 con 0x55c1fed637a0 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.131+0000 7f151e06fb80 10 client.25874 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.131+0000 7f151e06fb80 1 -- 172.21.15.67:0/1541182379 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c1feb3bcd0 con 0x55c1fed637a0 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.132+0000 7f14ed600700 1 -- 172.21.15.67:0/1541182379 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f40a7080 con 0x55c1fed637a0 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.132+0000 7f14ed600700 1 -- 172.21.15.67:0/1541182379 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f14f4099ef0 con 0x55c1fed637a0 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.132+0000 7f151e06fb80 10 client.25874 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.132+0000 7f151e06fb80 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.132+0000 7f151e06fb80 20 client.25874 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.132+0000 7f151e06fb80 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1feddbe40 0x55c1fedde230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.132+0000 7f151e06fb80 4 client.25874 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:37:57.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.132+0000 7f151e06fb80 1 -- 172.21.15.67:0/1541182379 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1feb39ce0 con 0x55c1feddbe40 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.132+0000 7f1502a00700 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1feddbe40 0x55c1fedde230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f1502a00700 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1feddbe40 0x55c1fedde230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb7284cea913d2d6 server_cookie=286a1387f6a8ea11 in_seq=0 out_seq=0 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f14ed600700 10 client.25874 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f14ed600700 1 -- 172.21.15.67:0/1541182379 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c1feb39ce0 con 0x55c1feddbe40 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f14ed600700 10 client.25874 handle_command_reply: tid=0 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 client.25874 shutdown 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 -- 172.21.15.67:0/1541182379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 msgr2=0x55c1fed758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed758a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14f409ef60 tx=0x7f14f4005fb0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 -- 172.21.15.67:0/1541182379 shutdown_connections 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1fed6efc0 0x55c1fed74250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1fed637a0 0x55c1fed758a0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14f409ef60 tx=0x7f14f4005fb0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1feddbe40 0x55c1fedde230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 --2- 172.21.15.67:0/1541182379 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1fed640c0 0x55c1fed73d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 -- 172.21.15.67:0/1541182379 >> 172.21.15.67:0/1541182379 conn(0x55c1fecc36a0 msgr2=0x55c1fed59010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f14ed600700 0 client.25874 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.133+0000 7f151e06fb80 1 -- 172.21.15.67:0/1541182379 shutdown_connections 2024-03-30T02:37:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.134+0000 7f151e06fb80 1 -- 172.21.15.67:0/1541182379 wait complete. 2024-03-30T02:37:57.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.134+0000 7f151e06fb80 20 client.25874 trim_cache size 0 max 16384 2024-03-30T02:37:57.137 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:37:57.137 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:37:57.137 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:37:57.137 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:37:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.134+0000 7f1509000700 1 -- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f14e80a1320 msgr2=0x7f14e80a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.134+0000 7f1509000700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f14e80a1320 0x7f14e80a37e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009000 tx=0x7f14fc01e000 comp rx=0 tx=0).stop 2024-03-30T02:37:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 -- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 msgr2=0x7f15041992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 0x7f15041992b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005910 tx=0x7f14f4004730 comp rx=0 tx=0).stop 2024-03-30T02:37:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 -- 172.21.15.67:0/1293566716 shutdown_connections 2024-03-30T02:37:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f14e80a1320 0x7f14e80a37e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1504101cf0 0x7f15041992b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f150410b110 0x7f150419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 --2- 172.21.15.67:0/1293566716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1504102610 0x7f15041997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 -- 172.21.15.67:0/1293566716 >> 172.21.15.67:0/1293566716 conn(0x7f15040fb510 msgr2=0x7f1504100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:57.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 -- 172.21.15.67:0/1293566716 shutdown_connections 2024-03-30T02:37:57.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.135+0000 7f1509000700 1 -- 172.21.15.67:0/1293566716 wait complete. 2024-03-30T02:37:57.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:37:57.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:37:57.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.410+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1538726445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28102710 msgr2=0x7fdc2810abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.410+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1538726445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28102710 0x7fdc2810abf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdc140098f0 tx=0x7fdc14009c00 comp rx=0 tx=0).stop 2024-03-30T02:37:57.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.410+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1538726445 shutdown_connections 2024-03-30T02:37:57.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.410+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1538726445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc28101df0 0x7fdc281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.410+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1538726445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc2810b1c0 0x7fdc2810d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.410+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1538726445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28102710 0x7fdc2810abf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.410+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1538726445 >> 172.21.15.67:0/1538726445 conn(0x7fdc280fb590 msgr2=0x7fdc280fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:57.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.410+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1538726445 shutdown_connections 2024-03-30T02:37:57.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.410+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1538726445 wait complete. 2024-03-30T02:37:57.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.412+0000 7fdc2ca00700 1 Processor -- start 2024-03-30T02:37:57.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.412+0000 7fdc2ca00700 1 -- start start 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc2ca00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28101df0 0x7fdc280ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc2ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28102710 0x7fdc280fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc2ca00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2810b1c0 0x7fdc281004c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc2ca00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc281a54d0 con 0x7fdc2810b1c0 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc2ca00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc281a5640 con 0x7fdc28102710 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc2ca00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc281a5800 con 0x7fdc28101df0 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc27400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28101df0 0x7fdc280ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc27400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28101df0 0x7fdc280ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55138/0 (socket says 172.21.15.67:55138) 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc27e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2810b1c0 0x7fdc281004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc27400700 1 -- 172.21.15.67:0/1574175162 learned_addr learned my addr 172.21.15.67:0/1574175162 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:57.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.413+0000 7fdc26a00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28102710 0x7fdc280fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.414+0000 7fdc27400700 1 -- 172.21.15.67:0/1574175162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28102710 msgr2=0x7fdc280fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.414+0000 7fdc27400700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28102710 0x7fdc280fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.414+0000 7fdc27400700 1 -- 172.21.15.67:0/1574175162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2810b1c0 msgr2=0x7fdc281004c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.414+0000 7fdc27400700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2810b1c0 0x7fdc281004c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.414+0000 7fdc27400700 1 -- 172.21.15.67:0/1574175162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14009520 con 0x7fdc28101df0 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.414+0000 7fdc27400700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28101df0 0x7fdc280ffa40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800e9c0 tx=0x7fdc1800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.414+0000 7fdc1fe00700 1 -- 172.21.15.67:0/1574175162 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18004a30 con 0x7fdc28101df0 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.414+0000 7fdc1fe00700 1 -- 172.21.15.67:0/1574175162 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1800cc80 con 0x7fdc28101df0 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.415+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc281a5ae0 con 0x7fdc28101df0 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.415+0000 7fdc1fe00700 1 -- 172.21.15.67:0/1574175162 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180054e0 con 0x7fdc28101df0 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.415+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc281a6010 con 0x7fdc28101df0 2024-03-30T02:37:57.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.416+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc281062e0 con 0x7fdc28101df0 2024-03-30T02:37:57.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.419+0000 7fdc1fe00700 1 -- 172.21.15.67:0/1574175162 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7fdc18004b90 con 0x7fdc28101df0 2024-03-30T02:37:57.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.421+0000 7fdc1fe00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdc0c0a13a0 0x7fdc0c0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:57.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.421+0000 7fdc1fe00700 1 -- 172.21.15.67:0/1574175162 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fdc18014070 con 0x7fdc28101df0 2024-03-30T02:37:57.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.421+0000 7fdc26a00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdc0c0a13a0 0x7fdc0c0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:57.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.422+0000 7fdc26a00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdc0c0a13a0 0x7fdc0c0a3860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc140098c0 tx=0x7fdc14019040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:57.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.425+0000 7fdc1fe00700 1 -- 172.21.15.67:0/1574175162 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdc180961e0 con 0x7fdc28101df0 2024-03-30T02:37:57.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.768+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc28068650 con 0x7fdc28101df0 2024-03-30T02:37:57.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.769+0000 7fdc1fe00700 1 -- 172.21.15.67:0/1574175162 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10909 (secure 0 0 0) 0x7fdc18099800 con 0x7fdc28101df0 2024-03-30T02:37:57.769 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:37:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:36:42.080059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdc0c0a13a0 msgr2=0x7fdc0c0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdc0c0a13a0 0x7fdc0c0a3860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc140098c0 tx=0x7fdc14019040 comp rx=0 tx=0).stop 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28101df0 msgr2=0x7fdc280ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28101df0 0x7fdc280ffa40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800e9c0 tx=0x7fdc1800ecd0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 shutdown_connections 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdc0c0a13a0 0x7fdc0c0a3860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdc2810b1c0 0x7fdc281004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc28101df0 0x7fdc280ffa40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 --2- 172.21.15.67:0/1574175162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc28102710 0x7fdc280fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:57.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 >> 172.21.15.67:0/1574175162 conn(0x7fdc280fb590 msgr2=0x7fdc280fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 shutdown_connections 2024-03-30T02:37:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:57.773+0000 7fdc2ca00700 1 -- 172.21.15.67:0/1574175162 wait complete. 2024-03-30T02:37:57.774 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-03-30T02:37:57.801 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:37:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.039+0000 7ff39f200700 1 -- 172.21.15.67:0/3523185362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff398067e30 msgr2=0x7ff3980682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.039+0000 7ff39f200700 1 --2- 172.21.15.67:0/3523185362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff398067e30 0x7ff3980682b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff3840099c0 tx=0x7ff384009cd0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.040+0000 7ff39f200700 1 -- 172.21.15.67:0/3523185362 shutdown_connections 2024-03-30T02:37:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.040+0000 7ff39f200700 1 --2- 172.21.15.67:0/3523185362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff398105ce0 0x7ff3981060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.040+0000 7ff39f200700 1 --2- 172.21.15.67:0/3523185362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3980687f0 0x7ff39810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.040+0000 7ff39f200700 1 --2- 172.21.15.67:0/3523185362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff398067e30 0x7ff3980682b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.040+0000 7ff39f200700 1 -- 172.21.15.67:0/3523185362 >> 172.21.15.67:0/3523185362 conn(0x7ff398076d40 msgr2=0x7ff398077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.040+0000 7ff39f200700 1 -- 172.21.15.67:0/3523185362 shutdown_connections 2024-03-30T02:37:58.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.040+0000 7ff39f200700 1 -- 172.21.15.67:0/3523185362 wait complete. 2024-03-30T02:37:58.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.041+0000 7ff39f200700 1 Processor -- start 2024-03-30T02:37:58.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.042+0000 7ff39f200700 1 -- start start 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.042+0000 7ff39f200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff398067e30 0x7ff398197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.042+0000 7ff39f200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3980687f0 0x7ff3981977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.042+0000 7ff39f200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff398105ce0 0x7ff39819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.042+0000 7ff39f200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39819c180 con 0x7ff3980687f0 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.042+0000 7ff39f200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39819c2f0 con 0x7ff398105ce0 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.042+0000 7ff39f200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39819c4b0 con 0x7ff398067e30 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.042+0000 7ff397400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3980687f0 0x7ff3981977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.042+0000 7ff397e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff398067e30 0x7ff398197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff397e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff398067e30 0x7ff398197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55150/0 (socket says 172.21.15.67:55150) 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff397e00700 1 -- 172.21.15.67:0/730184833 learned_addr learned my addr 172.21.15.67:0/730184833 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff39d200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff398105ce0 0x7ff39819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff397400700 1 -- 172.21.15.67:0/730184833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff398067e30 msgr2=0x7ff398197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff397400700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff398067e30 0x7ff398197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff397400700 1 -- 172.21.15.67:0/730184833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff398105ce0 msgr2=0x7ff39819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff397400700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff398105ce0 0x7ff39819bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff397400700 1 -- 172.21.15.67:0/730184833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3840095f0 con 0x7ff3980687f0 2024-03-30T02:37:58.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff397e00700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff398067e30 0x7ff398197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:58.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff39d200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff398105ce0 0x7ff39819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:58.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.043+0000 7ff397400700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3980687f0 0x7ff3981977a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff3840098e0 tx=0x7ff38400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.044+0000 7ff394c00700 1 -- 172.21.15.67:0/730184833 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38401d070 con 0x7ff3980687f0 2024-03-30T02:37:58.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.044+0000 7ff394c00700 1 -- 172.21.15.67:0/730184833 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff38400be00 con 0x7ff3980687f0 2024-03-30T02:37:58.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.044+0000 7ff39f200700 1 -- 172.21.15.67:0/730184833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39819c730 con 0x7ff3980687f0 2024-03-30T02:37:58.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.044+0000 7ff394c00700 1 -- 172.21.15.67:0/730184833 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38400f9a0 con 0x7ff3980687f0 2024-03-30T02:37:58.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.044+0000 7ff39f200700 1 -- 172.21.15.67:0/730184833 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3981a3fd0 con 0x7ff3980687f0 2024-03-30T02:37:58.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.049+0000 7ff394c00700 1 -- 172.21.15.67:0/730184833 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7ff384004c60 con 0x7ff3980687f0 2024-03-30T02:37:58.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.050+0000 7ff3b2cb0b80 1 Processor -- start 2024-03-30T02:37:58.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.050+0000 7ff3b2cb0b80 1 -- start start 2024-03-30T02:37:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2d5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df2eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df22e990 con 0x5599df2d71c0 2024-03-30T02:37:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df22c350 con 0x5599df2d5b10 2024-03-30T02:37:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df227c40 con 0x5599df2e9490 2024-03-30T02:37:58.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff394c00700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff37c0a1320 0x7ff37c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff394c00700 1 -- 172.21.15.67:0/730184833 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff3840cdd80 con 0x7ff3980687f0 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff397e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2d5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff397e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2d5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37112/0 (socket says 172.21.15.67:37112) 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.051+0000 7ff397e00700 1 -- 172.21.15.67:0/1727341331 learned_addr learned my addr 172.21.15.67:0/1727341331 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.052+0000 7ff39d200700 1 --2- 172.21.15.67:0/1727341331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.052+0000 7ff397400700 1 --2- 172.21.15.67:0/1727341331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df2eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.052+0000 7ff39d200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff37c0a1320 0x7ff37c0a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.052+0000 7ff397e00700 1 -- 172.21.15.67:0/1727341331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 msgr2=0x5599df2eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.052+0000 7ff397e00700 1 --2- 172.21.15.67:0/1727341331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df2eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.052+0000 7ff397e00700 1 -- 172.21.15.67:0/1727341331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 msgr2=0x5599df2d55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.052+0000 7ff397e00700 1 --2- 172.21.15.67:0/1727341331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.052+0000 7ff397e00700 1 -- 172.21.15.67:0/1727341331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599df0a93b0 con 0x5599df2d5b10 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.052+0000 7ff397e00700 1 --2- 172.21.15.67:0/1727341331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2d5f90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3880117e0 tx=0x7ff388011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a9ca1d6d87d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.053+0000 7ff39d200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff37c0a1320 0x7ff37c0a37e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff390009fd0 tx=0x7ff39000b040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.053+0000 7ff38d600700 1 -- 172.21.15.67:0/1727341331 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800f480 con 0x5599df2d5b10 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.053+0000 7ff38d600700 1 -- 172.21.15.67:0/1727341331 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff38800faa0 con 0x5599df2d5b10 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.053+0000 7ff38d600700 1 -- 172.21.15.67:0/1727341331 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388019b60 con 0x5599df2d5b10 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.054+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/1727341331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 msgr2=0x5599df2d5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.054+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/1727341331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2d5f90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3880117e0 tx=0x7ff388011af0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.054+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/1727341331 shutdown_connections 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.054+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/1727341331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.054+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/1727341331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df2eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.054+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/1727341331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2d5f90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.054+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/1727341331 >> 172.21.15.67:0/1727341331 conn(0x5599df2356a0 msgr2=0x5599df2d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.054+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/1727341331 shutdown_connections 2024-03-30T02:37:58.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.054+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/1727341331 wait complete. 2024-03-30T02:37:58.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.055+0000 7ff3b2cb0b80 1 -- start start 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.055+0000 7ff3b2cb0b80 1 Processor -- start 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.055+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.055+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.055+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df2d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.055+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df1c3d80 con 0x5599df2d71c0 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.055+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df1c3a20 con 0x5599df2d5b10 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.055+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df0ab6a0 con 0x5599df2e9490 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff3b2cb0b80 10 client.0 _ll_register_callbacks cb 0x5599df1cd470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40474/0 (socket says 172.21.15.67:40474) 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397400700 1 -- 172.21.15.67:0/800105224 learned_addr learned my addr 172.21.15.67:0/800105224 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff39d200700 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df2d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397e00700 1 -- 172.21.15.67:0/800105224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 msgr2=0x5599df2d9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397e00700 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df2d9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397e00700 1 -- 172.21.15.67:0/800105224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 msgr2=0x5599df2d95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397e00700 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397e00700 1 -- 172.21.15.67:0/800105224 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff388011440 con 0x5599df2d5b10 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.056+0000 7ff397400700 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.057+0000 7ff397e00700 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2dc270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff388000c00 tx=0x7ff38800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=872e5a68cbabf1e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.057+0000 7ff382000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.057+0000 7ff382000700 1 -- 172.21.15.67:0/800105224 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800f920 con 0x5599df2d5b10 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.057+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/800105224 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599df1e8380 con 0x5599df2d5b10 2024-03-30T02:37:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.057+0000 7ff382000700 1 -- 172.21.15.67:0/800105224 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff388005ce0 con 0x5599df2d5b10 2024-03-30T02:37:58.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.057+0000 7ff382000700 1 -- 172.21.15.67:0/800105224 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff388004250 con 0x5599df2d5b10 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.057+0000 7ff3b2cb0b80 10 client.25892 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.057+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/800105224 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5599df055b60 con 0x5599df2d5b10 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.058+0000 7ff382000700 1 -- 172.21.15.67:0/800105224 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599df1e8380 con 0x5599df2d5b10 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.058+0000 7ff382000700 1 -- 172.21.15.67:0/800105224 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff388003710 con 0x5599df2d5b10 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.058+0000 7ff3b2cb0b80 10 client.25892 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.058+0000 7ff3b2cb0b80 10 client.25892 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.058+0000 7ff3b2cb0b80 20 client.25892 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.058+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5599df2f8fc0 0x5599df2fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.058+0000 7ff3b2cb0b80 4 client.25892 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.058+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/800105224 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599df220700 con 0x5599df2f8fc0 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.059+0000 7ff397400700 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5599df2f8fc0 0x5599df2fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.059+0000 7ff397400700 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5599df2f8fc0 0x5599df2fb3b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3fec066c41ac689 server_cookie=4d3a0d56079c32ac in_seq=0 out_seq=0 2024-03-30T02:37:58.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.059+0000 7ff382000700 10 client.25892 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff382000700 1 -- 172.21.15.67:0/800105224 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5599df220700 con 0x5599df2f8fc0 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff382000700 10 client.25892 handle_command_reply: tid=0 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff3b2cb0b80 1 client.25892 shutdown 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/800105224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 msgr2=0x5599df2dc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2dc270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff388000c00 tx=0x7ff38800f730 comp rx=0 tx=0).stop 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/800105224 shutdown_connections 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5599df2f8fc0 0x5599df2fb3b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df2d9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d5b10 0x5599df2dc270 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/800105224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d71c0 0x5599df2d95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/800105224 >> 172.21.15.67:0/800105224 conn(0x5599df2356a0 msgr2=0x5599df2d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.060+0000 7ff382000700 0 client.25892 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.061+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/800105224 shutdown_connections 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.061+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/800105224 wait complete. 2024-03-30T02:37:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.061+0000 7ff3b2cb0b80 20 client.25892 trim_cache size 0 max 16384 2024-03-30T02:37:58.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.074+0000 7ff3b2cb0b80 1 Processor -- start 2024-03-30T02:37:58.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.074+0000 7ff3b2cb0b80 1 -- start start 2024-03-30T02:37:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2d5b10 0x5599df2eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2e9490 0x5599df2e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df2d0f10 con 0x5599df2e9490 2024-03-30T02:37:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df2e8740 con 0x5599df2d71c0 2024-03-30T02:37:58.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df2d3160 con 0x5599df2d5b10 2024-03-30T02:37:58.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff39d200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2e9490 0x5599df2e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff39d200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2e9490 0x5599df2e8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40484/0 (socket says 172.21.15.67:40484) 2024-03-30T02:37:58.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff39d200700 1 -- 172.21.15.67:0/603898135 learned_addr learned my addr 172.21.15.67:0/603898135 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff397e00700 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2d5b10 0x5599df2eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.075+0000 7ff397400700 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.076+0000 7ff39d200700 1 -- 172.21.15.67:0/603898135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2d5b10 msgr2=0x5599df2eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.076+0000 7ff39d200700 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2d5b10 0x5599df2eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.076+0000 7ff39d200700 1 -- 172.21.15.67:0/603898135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 msgr2=0x5599df2dbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.076+0000 7ff39d200700 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2dbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.076+0000 7ff39d200700 1 -- 172.21.15.67:0/603898135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599df1edea0 con 0x5599df2e9490 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.076+0000 7ff397e00700 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2d5b10 0x5599df2eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.076+0000 7ff39d200700 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2e9490 0x5599df2e8ac0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e9a0 tx=0x7ff390009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be92f6d04ad2cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.076+0000 7ff38e000700 1 -- 172.21.15.67:0/603898135 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390007ea0 con 0x5599df2e9490 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.077+0000 7ff38e000700 1 -- 172.21.15.67:0/603898135 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff39000f040 con 0x5599df2e9490 2024-03-30T02:37:58.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.077+0000 7ff38e000700 1 -- 172.21.15.67:0/603898135 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390018570 con 0x5599df2e9490 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.077+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/603898135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2e9490 msgr2=0x5599df2e8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.077+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2e9490 0x5599df2e8ac0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e9a0 tx=0x7ff390009920 comp rx=0 tx=0).stop 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.078+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/603898135 shutdown_connections 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.078+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2e9490 0x5599df2e8ac0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.078+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2d5b10 0x5599df2eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.078+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/603898135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2dbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.078+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/603898135 >> 172.21.15.67:0/603898135 conn(0x5599df2356a0 msgr2=0x5599df2e5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.078+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/603898135 shutdown_connections 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.078+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/603898135 wait complete. 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.078+0000 7ff3b2cb0b80 1 -- start start 2024-03-30T02:37:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.079+0000 7ff3b2cb0b80 1 Processor -- start 2024-03-30T02:37:58.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.079+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d5b10 0x5599df2edc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.079+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2ee150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.079+0000 7ff3b2cb0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df34bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.079+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df2eea10 con 0x5599df2d5b10 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.079+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df34c320 con 0x5599df2d71c0 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.079+0000 7ff3b2cb0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599df34c460 con 0x5599df2e9490 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.079+0000 7ff3b2cb0b80 10 client.0 _ll_register_callbacks cb 0x5599df1cd470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d5b10 0x5599df2edc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2ee150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37152/0 (socket says 172.21.15.67:37152) 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397400700 1 -- 172.21.15.67:0/625304641 learned_addr learned my addr 172.21.15.67:0/625304641 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff39d200700 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df34bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397400700 1 -- 172.21.15.67:0/625304641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 msgr2=0x5599df34bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397400700 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df34bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397400700 1 -- 172.21.15.67:0/625304641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d5b10 msgr2=0x5599df2edc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397400700 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d5b10 0x5599df2edc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397400700 1 -- 172.21.15.67:0/625304641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599df055b60 con 0x5599df2d71c0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.080+0000 7ff397e00700 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d5b10 0x5599df2edc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.081+0000 7ff397400700 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2ee150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff384097470 tx=0x7ff3840cb5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e24d629507653 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.081+0000 7ff382000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.081+0000 7ff382000700 1 -- 172.21.15.67:0/625304641 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3840b3320 con 0x5599df2d71c0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.081+0000 7ff382000700 1 -- 172.21.15.67:0/625304641 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff38409aa00 con 0x5599df2d71c0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.081+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/625304641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff38800faa0 con 0x5599df2d71c0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.081+0000 7ff382000700 1 -- 172.21.15.67:0/625304641 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff38409e830 con 0x5599df2d71c0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.081+0000 7ff3b2cb0b80 10 client.25898 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.081+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/625304641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7ff39000e6e0 con 0x5599df2d71c0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.081+0000 7ff382000700 1 -- 172.21.15.67:0/625304641 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff38800faa0 con 0x5599df2d71c0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.082+0000 7ff382000700 1 -- 172.21.15.67:0/625304641 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff3840a2960 con 0x5599df2d71c0 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.082+0000 7ff3b2cb0b80 10 client.25898 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:58.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.082+0000 7ff3b2cb0b80 10 client.25898 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:37:58.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.082+0000 7ff3b2cb0b80 20 client.25898 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:58.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.082+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5599df34d880 0x5599df34fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.082+0000 7ff3b2cb0b80 4 client.25898 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:58.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.082+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/625304641 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5599df076cf0 con 0x5599df34d880 2024-03-30T02:37:58.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.082+0000 7ff397e00700 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5599df34d880 0x5599df34fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.083+0000 7ff397e00700 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5599df34d880 0x5599df34fca0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6157fd5f2a18868e server_cookie=3b6ecae0dfd8884d in_seq=0 out_seq=0 2024-03-30T02:37:58.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.083+0000 7ff382000700 10 client.25898 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:37:58.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.083+0000 7ff382000700 1 -- 172.21.15.67:0/625304641 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5599df076cf0 con 0x5599df34d880 2024-03-30T02:37:58.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.083+0000 7ff382000700 10 client.25898 handle_command_reply: tid=0 2024-03-30T02:37:58.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.083+0000 7ff3b2cb0b80 1 client.25898 shutdown 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.083+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/625304641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 msgr2=0x5599df2ee150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.083+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2ee150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff384097470 tx=0x7ff3840cb5f0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/625304641 shutdown_connections 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5599df2d5b10 0x5599df2edc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5599df34d880 0x5599df34fca0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599df2e9490 0x5599df34bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff382000700 0 client.25898 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff3b2cb0b80 1 --2- 172.21.15.67:0/625304641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599df2d71c0 0x5599df2ee150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/625304641 >> 172.21.15.67:0/625304641 conn(0x5599df2356a0 msgr2=0x5599df2e8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/625304641 shutdown_connections 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff3b2cb0b80 1 -- 172.21.15.67:0/625304641 wait complete. 2024-03-30T02:37:58.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.084+0000 7ff3b2cb0b80 20 client.25898 trim_cache size 0 max 16384 2024-03-30T02:37:58.087 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:58.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 -- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff37c0a1320 msgr2=0x7ff37c0a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff37c0a1320 0x7ff37c0a37e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff390009fd0 tx=0x7ff39000b040 comp rx=0 tx=0).stop 2024-03-30T02:37:58.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 -- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3980687f0 msgr2=0x7ff3981977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3980687f0 0x7ff3981977a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff3840098e0 tx=0x7ff38400b9c0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 -- 172.21.15.67:0/730184833 shutdown_connections 2024-03-30T02:37:58.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff37c0a1320 0x7ff37c0a37e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff398105ce0 0x7ff39819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff398067e30 0x7ff398197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 --2- 172.21.15.67:0/730184833 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3980687f0 0x7ff3981977a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 -- 172.21.15.67:0/730184833 >> 172.21.15.67:0/730184833 conn(0x7ff398076d40 msgr2=0x7ff3980fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 -- 172.21.15.67:0/730184833 shutdown_connections 2024-03-30T02:37:58.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.085+0000 7ff39f200700 1 -- 172.21.15.67:0/730184833 wait complete. 2024-03-30T02:37:58.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:37:58.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.377+0000 7f6d9be00700 1 -- 172.21.15.67:0/2330725484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d94105ce0 msgr2=0x7f6d941060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.377+0000 7f6d9be00700 1 --2- 172.21.15.67:0/2330725484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d94105ce0 0x7f6d941060c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d880099c0 tx=0x7f6d88009cd0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.377+0000 7f6d9be00700 1 -- 172.21.15.67:0/2330725484 shutdown_connections 2024-03-30T02:37:58.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.377+0000 7f6d9be00700 1 --2- 172.21.15.67:0/2330725484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d940687f0 0x7f6d9410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.377+0000 7f6d9be00700 1 --2- 172.21.15.67:0/2330725484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d94105ce0 0x7f6d941060c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.377+0000 7f6d9be00700 1 --2- 172.21.15.67:0/2330725484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d94067e30 0x7f6d940682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.377+0000 7f6d9be00700 1 -- 172.21.15.67:0/2330725484 >> 172.21.15.67:0/2330725484 conn(0x7f6d94076d40 msgr2=0x7f6d94077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.377+0000 7f6d9be00700 1 -- 172.21.15.67:0/2330725484 shutdown_connections 2024-03-30T02:37:58.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.377+0000 7f6d9be00700 1 -- 172.21.15.67:0/2330725484 wait complete. 2024-03-30T02:37:58.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.378+0000 7f6d9be00700 1 Processor -- start 2024-03-30T02:37:58.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.378+0000 7f6d9be00700 1 -- start start 2024-03-30T02:37:58.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d9be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d94067e30 0x7f6d94194ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d9be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d940687f0 0x7f6d94195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d9be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d94105ce0 0x7f6d941998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d9be00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d94199f10 con 0x7f6d94067e30 2024-03-30T02:37:58.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d9be00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9419a080 con 0x7f6d940687f0 2024-03-30T02:37:58.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d9be00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9419a240 con 0x7f6d94105ce0 2024-03-30T02:37:58.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d99400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d94067e30 0x7f6d94194ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d99e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d94105ce0 0x7f6d941998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d99e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d94105ce0 0x7f6d941998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55212/0 (socket says 172.21.15.67:55212) 2024-03-30T02:37:58.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.379+0000 7f6d99e00700 1 -- 172.21.15.67:0/4239854743 learned_addr learned my addr 172.21.15.67:0/4239854743 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.380+0000 7f6d98a00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d940687f0 0x7f6d94195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.380+0000 7f6d99e00700 1 -- 172.21.15.67:0/4239854743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d940687f0 msgr2=0x7f6d94195530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.380+0000 7f6d99e00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d940687f0 0x7f6d94195530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.380+0000 7f6d99e00700 1 -- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d94067e30 msgr2=0x7f6d94194ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.380+0000 7f6d99e00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d94067e30 0x7f6d94194ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.380+0000 7f6d99e00700 1 -- 172.21.15.67:0/4239854743 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d880095f0 con 0x7f6d94105ce0 2024-03-30T02:37:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.380+0000 7f6d98a00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d940687f0 0x7f6d94195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.380+0000 7f6d99400700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d94067e30 0x7f6d94194ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.380+0000 7f6d99e00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d94105ce0 0x7f6d941998d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c007ee0 tx=0x7f6d8c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.381+0000 7f6d92000700 1 -- 172.21.15.67:0/4239854743 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c00ddd0 con 0x7f6d94105ce0 2024-03-30T02:37:58.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.381+0000 7f6d92000700 1 -- 172.21.15.67:0/4239854743 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8c00f040 con 0x7f6d94105ce0 2024-03-30T02:37:58.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.381+0000 7f6d92000700 1 -- 172.21.15.67:0/4239854743 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c014600 con 0x7f6d94105ce0 2024-03-30T02:37:58.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.381+0000 7f6d9be00700 1 -- 172.21.15.67:0/4239854743 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9419a520 con 0x7f6d94105ce0 2024-03-30T02:37:58.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.381+0000 7f6d9be00700 1 -- 172.21.15.67:0/4239854743 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d941a1d60 con 0x7f6d94105ce0 2024-03-30T02:37:58.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.385+0000 7f6d92000700 1 -- 172.21.15.67:0/4239854743 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f6d8c014760 con 0x7f6d94105ce0 2024-03-30T02:37:58.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.386+0000 7f6daf9ddb80 1 Processor -- start 2024-03-30T02:37:58.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.386+0000 7f6daf9ddb80 1 -- start start 2024-03-30T02:37:58.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9769b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c98a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c98c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c8cf990 con 0x55f37c98a600 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c8cd350 con 0x55f37c9765d0 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c8c8c40 con 0x55f37c976ef0 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6d92000700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6d7c0a13a0 0x7f6d7c0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6d98a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6d98a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9769b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37184/0 (socket says 172.21.15.67:37184) 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6d98a00700 1 -- 172.21.15.67:0/2433046842 learned_addr learned my addr 172.21.15.67:0/2433046842 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6d92000700 1 -- 172.21.15.67:0/4239854743 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f6d8c0ccca0 con 0x7f6d94105ce0 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2433046842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c98c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6d99e00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6d7c0a13a0 0x7f6d7c0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.387+0000 7f6d99400700 1 --2- 172.21.15.67:0/2433046842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c98a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.388+0000 7f6d98a00700 1 -- 172.21.15.67:0/2433046842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 msgr2=0x55f37c98a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.388+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2433046842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c98a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.388+0000 7f6d98a00700 1 -- 172.21.15.67:0/2433046842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 msgr2=0x55f37c9769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.388+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2433046842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.388+0000 7f6d98a00700 1 -- 172.21.15.67:0/2433046842 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f37c74a3b0 con 0x55f37c98a600 2024-03-30T02:37:58.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.388+0000 7f6d99e00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6d7c0a13a0 0x7f6d7c0a3860 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0c9fe0 tx=0x7f6d8c00df30 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.388+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2433046842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c98c9f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6d80005950 tx=0x7f6d80011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3651c36c0aa37c3c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.389+0000 7f6d86600700 1 -- 172.21.15.67:0/2433046842 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80014070 con 0x55f37c98a600 2024-03-30T02:37:58.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.389+0000 7f6d86600700 1 -- 172.21.15.67:0/2433046842 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d80009500 con 0x55f37c98a600 2024-03-30T02:37:58.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.389+0000 7f6d86600700 1 -- 172.21.15.67:0/2433046842 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8001c440 con 0x55f37c98a600 2024-03-30T02:37:58.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.390+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2433046842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 msgr2=0x55f37c98c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.390+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2433046842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c98c9f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6d80005950 tx=0x7f6d80011040 comp rx=0 tx=0).stop 2024-03-30T02:37:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.390+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2433046842 shutdown_connections 2024-03-30T02:37:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.390+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2433046842 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c98c9f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.390+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2433046842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.390+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2433046842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c98a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.390+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2433046842 >> 172.21.15.67:0/2433046842 conn(0x55f37c8d66a0 msgr2=0x55f37c972a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.390+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2433046842 shutdown_connections 2024-03-30T02:37:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.390+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2433046842 wait complete. 2024-03-30T02:37:58.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.391+0000 7f6daf9ddb80 1 -- start start 2024-03-30T02:37:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.391+0000 7f6daf9ddb80 1 Processor -- start 2024-03-30T02:37:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.392+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c9765d0 0x55f37c989fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.392+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c9872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.392+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c98a600 0x55f37c9878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.392+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c864d80 con 0x55f37c9765d0 2024-03-30T02:37:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.392+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c864a20 con 0x55f37c98a600 2024-03-30T02:37:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.392+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c74c6a0 con 0x55f37c976ef0 2024-03-30T02:37:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.392+0000 7f6daf9ddb80 10 client.0 _ll_register_callbacks cb 0x55f37c86e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:58.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d99400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c9872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d99400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c9872e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55228/0 (socket says 172.21.15.67:55228) 2024-03-30T02:37:58.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d99400700 1 -- 172.21.15.67:0/2813502408 learned_addr learned my addr 172.21.15.67:0/2813502408 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c9765d0 0x55f37c989fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d99400700 1 -- 172.21.15.67:0/2813502408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c98a600 msgr2=0x55f37c9878a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:37:58.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d99400700 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c98a600 0x55f37c9878a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d99400700 1 -- 172.21.15.67:0/2813502408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c9765d0 msgr2=0x55f37c989fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d99400700 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c9765d0 0x55f37c989fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d99400700 1 -- 172.21.15.67:0/2813502408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d80009670 con 0x55f37c976ef0 2024-03-30T02:37:58.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d99400700 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c9872e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d880099c0 tx=0x7f6d8800ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64454704ef5fc58b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.393+0000 7f6d7aa00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:58.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.394+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2813502408 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8801d030 con 0x55f37c976ef0 2024-03-30T02:37:58.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.394+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2813502408 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f6d88003880 con 0x55f37c976ef0 2024-03-30T02:37:58.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.394+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2813502408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f37c889380 con 0x55f37c976ef0 2024-03-30T02:37:58.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.395+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2813502408 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d88003a40 con 0x55f37c976ef0 2024-03-30T02:37:58.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.395+0000 7f6daf9ddb80 10 client.25894 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:58.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.395+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2813502408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f37c6f6b60 con 0x55f37c976ef0 2024-03-30T02:37:58.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.395+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2813502408 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f37c889380 con 0x55f37c976ef0 2024-03-30T02:37:58.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.396+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2813502408 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d8800f460 con 0x55f37c976ef0 2024-03-30T02:37:58.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.396+0000 7f6daf9ddb80 10 client.25894 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:58.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.396+0000 7f6daf9ddb80 10 client.25894 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:37:58.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.396+0000 7f6daf9ddb80 20 client.25894 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:58.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.396+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55f37c99a130 0x55f37c99c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.396+0000 7f6daf9ddb80 4 client.25894 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:58.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.396+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2813502408 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f37c8c1700 con 0x55f37c99a130 2024-03-30T02:37:58.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.396+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55f37c99a130 0x55f37c99c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.397+0000 7f6d7aa00700 10 client.25894 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:37:58.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.397+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55f37c99a130 0x55f37c99c550 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1cbe4a942f2beb90 server_cookie=11bfb72c3c270c39 in_seq=0 out_seq=0 2024-03-30T02:37:58.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.398+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2813502408 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f37c8c1700 con 0x55f37c99a130 2024-03-30T02:37:58.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.398+0000 7f6d7aa00700 10 client.25894 handle_command_reply: tid=0 2024-03-30T02:37:58.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.398+0000 7f6daf9ddb80 1 client.25894 shutdown 2024-03-30T02:37:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.398+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2813502408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 msgr2=0x55f37c9872e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.398+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c9872e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6d880099c0 tx=0x7f6d8800ba30 comp rx=0 tx=0).stop 2024-03-30T02:37:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2813502408 shutdown_connections 2024-03-30T02:37:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55f37c99a130 0x55f37c99c550 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c9765d0 0x55f37c989fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c98a600 0x55f37c9878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2813502408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c9872e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2813502408 >> 172.21.15.67:0/2813502408 conn(0x55f37c8d66a0 msgr2=0x55f37c978690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6d7aa00700 0 client.25894 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:37:58.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2813502408 shutdown_connections 2024-03-30T02:37:58.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2813502408 wait complete. 2024-03-30T02:37:58.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.399+0000 7f6daf9ddb80 20 client.25894 trim_cache size 0 max 16384 2024-03-30T02:37:58.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:58 smithi067 ceph-mon[37808]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:58.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:37:58 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1574175162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:37:58.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.415+0000 7f6daf9ddb80 1 Processor -- start 2024-03-30T02:37:58.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.415+0000 7f6daf9ddb80 1 -- start start 2024-03-30T02:37:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.415+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.415+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c98c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.415+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c99c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.415+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c977850 con 0x55f37c98a600 2024-03-30T02:37:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.415+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c9763f0 con 0x55f37c9765d0 2024-03-30T02:37:58.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.415+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c972f20 con 0x55f37c976ef0 2024-03-30T02:37:58.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d98a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d98a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9878a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37198/0 (socket says 172.21.15.67:37198) 2024-03-30T02:37:58.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d98a00700 1 -- 172.21.15.67:0/2363322714 learned_addr learned my addr 172.21.15.67:0/2363322714 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d99400700 1 --2- 172.21.15.67:0/2363322714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c98c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2363322714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c99c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d99400700 1 -- 172.21.15.67:0/2363322714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 msgr2=0x55f37c9878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d99400700 1 --2- 172.21.15.67:0/2363322714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9878a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d99400700 1 -- 172.21.15.67:0/2363322714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 msgr2=0x55f37c99c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d99400700 1 --2- 172.21.15.67:0/2363322714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c99c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d99400700 1 -- 172.21.15.67:0/2363322714 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f37c88eea0 con 0x55f37c976ef0 2024-03-30T02:37:58.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.416+0000 7f6d99400700 1 --2- 172.21.15.67:0/2363322714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c98c9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6d880058e0 tx=0x7f6d8800f990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf033469dac235c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.417+0000 7f6d87000700 1 -- 172.21.15.67:0/2363322714 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8803b070 con 0x55f37c976ef0 2024-03-30T02:37:58.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.417+0000 7f6d87000700 1 -- 172.21.15.67:0/2363322714 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8801e040 con 0x55f37c976ef0 2024-03-30T02:37:58.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.417+0000 7f6d87000700 1 -- 172.21.15.67:0/2363322714 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8801d040 con 0x55f37c976ef0 2024-03-30T02:37:58.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.417+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2363322714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 msgr2=0x55f37c98c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.417+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2363322714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c98c9f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6d880058e0 tx=0x7f6d8800f990 comp rx=0 tx=0).stop 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.418+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2363322714 shutdown_connections 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.418+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2363322714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c99c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.418+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2363322714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c9765d0 0x55f37c9878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.418+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2363322714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c976ef0 0x55f37c98c9f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.418+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2363322714 >> 172.21.15.67:0/2363322714 conn(0x55f37c8d66a0 msgr2=0x55f37c96bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.418+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2363322714 shutdown_connections 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.418+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2363322714 wait complete. 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.418+0000 7f6daf9ddb80 1 -- start start 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.418+0000 7f6daf9ddb80 1 Processor -- start 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c9765d0 0x55f37c98f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c976ef0 0x55f37c98f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6daf9ddb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c98fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c9ed040 con 0x55f37c98a600 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c9ed180 con 0x55f37c976ef0 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6daf9ddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37c9ed340 con 0x55f37c9765d0 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6daf9ddb80 10 client.0 _ll_register_callbacks cb 0x55f37c86e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6d98a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c9765d0 0x55f37c98f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6d98a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c9765d0 0x55f37c98f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55240/0 (socket says 172.21.15.67:55240) 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6d98a00700 1 -- 172.21.15.67:0/2543341888 learned_addr learned my addr 172.21.15.67:0/2543341888 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6d99400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c976ef0 0x55f37c98f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.419+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c98fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6d98a00700 1 -- 172.21.15.67:0/2543341888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c976ef0 msgr2=0x55f37c98f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c976ef0 0x55f37c98f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6d98a00700 1 -- 172.21.15.67:0/2543341888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 msgr2=0x55f37c98fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c98fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6d98a00700 1 -- 172.21.15.67:0/2543341888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f37c6f6b60 con 0x55f37c9765d0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6d99400700 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c976ef0 0x55f37c98f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6d98a00700 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c9765d0 0x55f37c98f290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000f010 tx=0x7f6d80009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9706639e6cd54dc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6d7aa00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2543341888 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80014040 con 0x55f37c9765d0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.421+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2543341888 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f6d8002b0d0 con 0x55f37c9765d0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.420+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2543341888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d80009500 con 0x55f37c9765d0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.421+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2543341888 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d8002b290 con 0x55f37c9765d0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.421+0000 7f6daf9ddb80 10 client.25900 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.421+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2543341888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f37c74ecd0 con 0x55f37c9765d0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.421+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2543341888 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d80009500 con 0x55f37c9765d0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.422+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2543341888 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d8000e460 con 0x55f37c9765d0 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.422+0000 7f6daf9ddb80 10 client.25900 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.422+0000 7f6daf9ddb80 10 client.25900 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.422+0000 7f6daf9ddb80 20 client.25900 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:58.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.422+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55f37c9ee980 0x55f37c9f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.422+0000 7f6daf9ddb80 4 client.25900 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:58.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.422+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2543341888 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f37c717cf0 con 0x55f37c9ee980 2024-03-30T02:37:58.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.422+0000 7f6d99400700 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55f37c9ee980 0x55f37c9f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.423+0000 7f6d99400700 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55f37c9ee980 0x55f37c9f0d70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e18d9003a3a12502 server_cookie=7928c292e15b3c04 in_seq=0 out_seq=0 2024-03-30T02:37:58.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.423+0000 7f6d7aa00700 10 client.25900 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:37:58.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.423+0000 7f6d7aa00700 1 -- 172.21.15.67:0/2543341888 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f37c717cf0 con 0x55f37c9ee980 2024-03-30T02:37:58.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.423+0000 7f6d7aa00700 10 client.25900 handle_command_reply: tid=0 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.423+0000 7f6daf9ddb80 1 client.25900 shutdown 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.423+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2543341888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c9765d0 msgr2=0x55f37c98f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.423+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c9765d0 0x55f37c98f290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000f010 tx=0x7f6d80009f80 comp rx=0 tx=0).stop 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2543341888 shutdown_connections 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55f37c9ee980 0x55f37c9f0d70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f37c98a600 0x55f37c98fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f37c9765d0 0x55f37c98f290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6d7aa00700 0 client.25900 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6daf9ddb80 1 --2- 172.21.15.67:0/2543341888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37c976ef0 0x55f37c98f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2543341888 >> 172.21.15.67:0/2543341888 conn(0x55f37c8d66a0 msgr2=0x55f37c985a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2543341888 shutdown_connections 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6daf9ddb80 1 -- 172.21.15.67:0/2543341888 wait complete. 2024-03-30T02:37:58.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.424+0000 7f6daf9ddb80 20 client.25900 trim_cache size 0 max 16384 2024-03-30T02:37:58.428 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 -- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6d7c0a13a0 msgr2=0x7f6d7c0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6d7c0a13a0 0x7f6d7c0a3860 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0c9fe0 tx=0x7f6d8c00df30 comp rx=0 tx=0).stop 2024-03-30T02:37:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 -- 172.21.15.67:0/4239854743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d94105ce0 msgr2=0x7f6d941998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d94105ce0 0x7f6d941998d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c007ee0 tx=0x7f6d8c00d3b0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 -- 172.21.15.67:0/4239854743 shutdown_connections 2024-03-30T02:37:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6d7c0a13a0 0x7f6d7c0a3860 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d94067e30 0x7f6d94194ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d94105ce0 0x7f6d941998d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 --2- 172.21.15.67:0/4239854743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d940687f0 0x7f6d94195530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 -- 172.21.15.67:0/4239854743 >> 172.21.15.67:0/4239854743 conn(0x7f6d94076d40 msgr2=0x7f6d940fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 -- 172.21.15.67:0/4239854743 shutdown_connections 2024-03-30T02:37:58.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.425+0000 7f6d9be00700 1 -- 172.21.15.67:0/4239854743 wait complete. 2024-03-30T02:37:58.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:37:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.663+0000 7ff841a00700 1 -- 172.21.15.67:0/3710798814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff83c101cf0 msgr2=0x7ff83c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.663+0000 7ff841a00700 1 --2- 172.21.15.67:0/3710798814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff83c101cf0 0x7ff83c1020d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0098f0 tx=0x7ff82c009c00 comp rx=0 tx=0).stop 2024-03-30T02:37:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.663+0000 7ff841a00700 1 -- 172.21.15.67:0/3710798814 shutdown_connections 2024-03-30T02:37:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.664+0000 7ff841a00700 1 --2- 172.21.15.67:0/3710798814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c10b110 0x7ff83c10d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.664+0000 7ff841a00700 1 --2- 172.21.15.67:0/3710798814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff83c101cf0 0x7ff83c1020d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.664+0000 7ff841a00700 1 --2- 172.21.15.67:0/3710798814 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83c102610 0x7ff83c10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.664+0000 7ff841a00700 1 -- 172.21.15.67:0/3710798814 >> 172.21.15.67:0/3710798814 conn(0x7ff83c0fb510 msgr2=0x7ff83c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.664+0000 7ff841a00700 1 -- 172.21.15.67:0/3710798814 shutdown_connections 2024-03-30T02:37:58.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.664+0000 7ff841a00700 1 -- 172.21.15.67:0/3710798814 wait complete. 2024-03-30T02:37:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.665+0000 7ff841a00700 1 Processor -- start 2024-03-30T02:37:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.665+0000 7ff841a00700 1 -- start start 2024-03-30T02:37:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff841a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c101cf0 0x7ff83c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff841a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83c102610 0x7ff83c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff841a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff83c10b110 0x7ff83c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff841a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c19e190 con 0x7ff83c102610 2024-03-30T02:37:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff841a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c19e300 con 0x7ff83c101cf0 2024-03-30T02:37:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff841a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c19e4c0 con 0x7ff83c10b110 2024-03-30T02:37:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff83be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c101cf0 0x7ff83c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff83be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c101cf0 0x7ff83c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37230/0 (socket says 172.21.15.67:37230) 2024-03-30T02:37:58.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff83be00700 1 -- 172.21.15.67:0/2500057357 learned_addr learned my addr 172.21.15.67:0/2500057357 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff83b400700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83c102610 0x7ff83c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff841000700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff83c10b110 0x7ff83c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff83be00700 1 -- 172.21.15.67:0/2500057357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff83c10b110 msgr2=0x7ff83c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff83be00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff83c10b110 0x7ff83c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff83be00700 1 -- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83c102610 msgr2=0x7ff83c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff83be00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83c102610 0x7ff83c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.666+0000 7ff83be00700 1 -- 172.21.15.67:0/2500057357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c009520 con 0x7ff83c101cf0 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.667+0000 7ff83be00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c101cf0 0x7ff83c1992c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00fa00 tx=0x7ff82c00fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.667+0000 7ff838c00700 1 -- 172.21.15.67:0/2500057357 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c01c070 con 0x7ff83c101cf0 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.667+0000 7ff841a00700 1 -- 172.21.15.67:0/2500057357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83c19e740 con 0x7ff83c101cf0 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.667+0000 7ff838c00700 1 -- 172.21.15.67:0/2500057357 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff82c00bb20 con 0x7ff83c101cf0 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.667+0000 7ff838c00700 1 -- 172.21.15.67:0/2500057357 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c004930 con 0x7ff83c101cf0 2024-03-30T02:37:58.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.667+0000 7ff841a00700 1 -- 172.21.15.67:0/2500057357 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83c1a5fe0 con 0x7ff83c101cf0 2024-03-30T02:37:58.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.672+0000 7ff838c00700 1 -- 172.21.15.67:0/2500057357 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7ff82c020070 con 0x7ff83c101cf0 2024-03-30T02:37:58.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.673+0000 7ff856c1bb80 1 Processor -- start 2024-03-30T02:37:58.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.674+0000 7ff856c1bb80 1 -- start start 2024-03-30T02:37:58.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.674+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.674+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 0x55bd40082290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.674+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 0x55bd40084bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.674+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd3ffc7990 con 0x55bd400827d0 2024-03-30T02:37:58.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.674+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd3ffc5350 con 0x55bd4006f0c0 2024-03-30T02:37:58.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.674+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd3ffc0c40 con 0x55bd4006e7a0 2024-03-30T02:37:58.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff841000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff841000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4006eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55260/0 (socket says 172.21.15.67:55260) 2024-03-30T02:37:58.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff841000700 1 -- 172.21.15.67:0/4236692415 learned_addr learned my addr 172.21.15.67:0/4236692415 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff83be00700 1 --2- 172.21.15.67:0/4236692415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 0x55bd40082290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff838c00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff8200a1320 0x7ff8200a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff841000700 1 -- 172.21.15.67:0/4236692415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 msgr2=0x55bd40082290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff841000700 1 --2- 172.21.15.67:0/4236692415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 0x55bd40082290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff841000700 1 -- 172.21.15.67:0/4236692415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 msgr2=0x55bd40084bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff841000700 1 --2- 172.21.15.67:0/4236692415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 0x55bd40084bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff841000700 1 -- 172.21.15.67:0/4236692415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd3fe423b0 con 0x55bd4006e7a0 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.675+0000 7ff838c00700 1 -- 172.21.15.67:0/2500057357 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff82c0cdea0 con 0x7ff83c101cf0 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.676+0000 7ff841000700 1 --2- 172.21.15.67:0/4236692415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4006eb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff834009440 tx=0x7ff834009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a14ffe2c904839e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.676+0000 7ff841000700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff8200a1320 0x7ff8200a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.676+0000 7ff831600700 1 -- 172.21.15.67:0/4236692415 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83401a070 con 0x55bd4006e7a0 2024-03-30T02:37:58.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.676+0000 7ff831600700 1 -- 172.21.15.67:0/4236692415 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff83400f040 con 0x55bd4006e7a0 2024-03-30T02:37:58.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.676+0000 7ff831600700 1 -- 172.21.15.67:0/4236692415 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83400e070 con 0x55bd4006e7a0 2024-03-30T02:37:58.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff841000700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff8200a1320 0x7ff8200a37e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff834009550 tx=0x7ff834014700 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff856c1bb80 1 -- 172.21.15.67:0/4236692415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 msgr2=0x55bd4006eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/4236692415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4006eb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff834009440 tx=0x7ff834009920 comp rx=0 tx=0).stop 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff856c1bb80 1 -- 172.21.15.67:0/4236692415 shutdown_connections 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/4236692415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 0x55bd40084bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/4236692415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4006eb80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/4236692415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 0x55bd40082290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff856c1bb80 1 -- 172.21.15.67:0/4236692415 >> 172.21.15.67:0/4236692415 conn(0x55bd3ffce6a0 msgr2=0x55bd4006ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff856c1bb80 1 -- 172.21.15.67:0/4236692415 shutdown_connections 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.677+0000 7ff856c1bb80 1 -- 172.21.15.67:0/4236692415 wait complete. 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.678+0000 7ff856c1bb80 1 -- start start 2024-03-30T02:37:58.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.678+0000 7ff856c1bb80 1 Processor -- start 2024-03-30T02:37:58.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.678+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006e7a0 0x55bd4007bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.678+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd4007c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd400827d0 0x55bd40081a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd3ff5cd80 con 0x55bd4006f0c0 2024-03-30T02:37:58.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd3ff5ca20 con 0x55bd4006e7a0 2024-03-30T02:37:58.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd3fe446a0 con 0x55bd400827d0 2024-03-30T02:37:58.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff856c1bb80 10 client.0 _ll_register_callbacks cb 0x55bd3ff66470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:58.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff83be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006e7a0 0x55bd4007bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff83b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd4007c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff841000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd400827d0 0x55bd40081a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff841000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd400827d0 0x55bd40081a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55262/0 (socket says 172.21.15.67:55262) 2024-03-30T02:37:58.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff83b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd4007c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40608/0 (socket says 172.21.15.67:40608) 2024-03-30T02:37:58.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.679+0000 7ff841000700 1 -- 172.21.15.67:0/2694892558 learned_addr learned my addr 172.21.15.67:0/2694892558 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff83be00700 1 -- 172.21.15.67:0/2694892558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd400827d0 msgr2=0x55bd40081a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff83be00700 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd400827d0 0x55bd40081a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff83be00700 1 -- 172.21.15.67:0/2694892558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 msgr2=0x55bd4007c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff83be00700 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd4007c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff83be00700 1 -- 172.21.15.67:0/2694892558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8340090f0 con 0x55bd4006e7a0 2024-03-30T02:37:58.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff841000700 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd400827d0 0x55bd40081a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:58.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff83b400700 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd4007c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:58.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff83be00700 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006e7a0 0x55bd4007bcc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0cb180 tx=0x7ff82c00bc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3578694a12eb02b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff826000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.680+0000 7ff826000700 1 -- 172.21.15.67:0/2694892558 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c09a8f0 con 0x55bd4006e7a0 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.681+0000 7ff826000700 1 -- 172.21.15.67:0/2694892558 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff82c09b220 con 0x55bd4006e7a0 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.681+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2694892558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd3ff81380 con 0x55bd4006e7a0 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.681+0000 7ff826000700 1 -- 172.21.15.67:0/2694892558 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff82c09a660 con 0x55bd4006e7a0 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.681+0000 7ff856c1bb80 10 client.25925 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.681+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2694892558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bd3fdeeb60 con 0x55bd4006e7a0 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.682+0000 7ff826000700 1 -- 172.21.15.67:0/2694892558 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bd3ff81380 con 0x55bd4006e7a0 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.682+0000 7ff826000700 1 -- 172.21.15.67:0/2694892558 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff82c09a0a0 con 0x55bd4006e7a0 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.682+0000 7ff856c1bb80 10 client.25925 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.682+0000 7ff856c1bb80 10 client.25925 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.682+0000 7ff856c1bb80 20 client.25925 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.682+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55bd4009a910 0x55bd4009cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.682+0000 7ff856c1bb80 4 client.25925 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:58.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.682+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2694892558 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd3ffb9700 con 0x55bd4009a910 2024-03-30T02:37:58.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.682+0000 7ff83b400700 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55bd4009a910 0x55bd4009cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.683+0000 7ff83b400700 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55bd4009a910 0x55bd4009cd00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47cd140232861ed3 server_cookie=73da1df39a6201f7 in_seq=0 out_seq=0 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.683+0000 7ff826000700 10 client.25925 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff826000700 1 -- 172.21.15.67:0/2694892558 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bd3ffb9700 con 0x55bd4009a910 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff826000700 10 client.25925 handle_command_reply: tid=0 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff856c1bb80 1 client.25925 shutdown 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2694892558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006e7a0 msgr2=0x55bd4007bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006e7a0 0x55bd4007bcc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0cb180 tx=0x7ff82c00bc90 comp rx=0 tx=0).stop 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2694892558 shutdown_connections 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55bd4009a910 0x55bd4009cd00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd400827d0 0x55bd40081a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006e7a0 0x55bd4007bcc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2694892558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd4007c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2694892558 >> 172.21.15.67:0/2694892558 conn(0x55bd3ffce6a0 msgr2=0x55bd40070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.684+0000 7ff826000700 0 client.25925 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:37:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.685+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2694892558 shutdown_connections 2024-03-30T02:37:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.685+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2694892558 wait complete. 2024-03-30T02:37:58.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.685+0000 7ff856c1bb80 20 client.25925 trim_cache size 0 max 16384 2024-03-30T02:37:58.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:58 smithi170 ceph-mon[78798]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:58.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:37:58 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1574175162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:37:58.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.711+0000 7ff856c1bb80 1 Processor -- start 2024-03-30T02:37:58.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.711+0000 7ff856c1bb80 1 -- start start 2024-03-30T02:37:58.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.711+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd400666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.711+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd40081980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.711+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd400827d0 0x55bd400637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.711+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd40071840 con 0x55bd4006f0c0 2024-03-30T02:37:58.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.711+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd40077fd0 con 0x55bd400827d0 2024-03-30T02:37:58.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.711+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd40070300 con 0x55bd4006e7a0 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd40081980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd400666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd400666d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55270/0 (socket says 172.21.15.67:55270) 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd40081980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40624/0 (socket says 172.21.15.67:40624) 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83be00700 1 -- 172.21.15.67:0/2222549661 learned_addr learned my addr 172.21.15.67:0/2222549661 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff841000700 1 --2- 172.21.15.67:0/2222549661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd400827d0 0x55bd400637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83b400700 1 -- 172.21.15.67:0/2222549661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 msgr2=0x55bd400666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83b400700 1 --2- 172.21.15.67:0/2222549661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd400666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83b400700 1 -- 172.21.15.67:0/2222549661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd400827d0 msgr2=0x55bd400637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83b400700 1 --2- 172.21.15.67:0/2222549661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd400827d0 0x55bd400637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.712+0000 7ff83b400700 1 -- 172.21.15.67:0/2222549661 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd3ff86ea0 con 0x55bd4006f0c0 2024-03-30T02:37:58.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.713+0000 7ff83b400700 1 --2- 172.21.15.67:0/2222549661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd40081980 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55bd4009d4b0 tx=0x7ff82801bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b313bfd1dee0ad95 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.713+0000 7ff832000700 1 -- 172.21.15.67:0/2222549661 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828017030 con 0x55bd4006f0c0 2024-03-30T02:37:58.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.713+0000 7ff832000700 1 -- 172.21.15.67:0/2222549661 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff828018040 con 0x55bd4006f0c0 2024-03-30T02:37:58.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.713+0000 7ff832000700 1 -- 172.21.15.67:0/2222549661 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82801b730 con 0x55bd4006f0c0 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.714+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2222549661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 msgr2=0x55bd40081980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.714+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2222549661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd40081980 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55bd4009d4b0 tx=0x7ff82801bb70 comp rx=0 tx=0).stop 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.714+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2222549661 shutdown_connections 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.714+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2222549661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd400827d0 0x55bd400637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.714+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2222549661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd400666d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.714+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/2222549661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd4006f0c0 0x55bd40081980 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.714+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2222549661 >> 172.21.15.67:0/2222549661 conn(0x55bd3ffce6a0 msgr2=0x55bd4009cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.714+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2222549661 shutdown_connections 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.714+0000 7ff856c1bb80 1 -- 172.21.15.67:0/2222549661 wait complete. 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.715+0000 7ff856c1bb80 1 -- start start 2024-03-30T02:37:58.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.715+0000 7ff856c1bb80 1 Processor -- start 2024-03-30T02:37:58.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.716+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4007f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.716+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 0x55bd4007fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.716+0000 7ff856c1bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 0x55bd400ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.716+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd400edaf0 con 0x55bd400827d0 2024-03-30T02:37:58.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.716+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd400edc30 con 0x55bd4006f0c0 2024-03-30T02:37:58.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.716+0000 7ff856c1bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd400edd70 con 0x55bd4006e7a0 2024-03-30T02:37:58.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff856c1bb80 10 client.0 _ll_register_callbacks cb 0x55bd3ff66470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:58.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff83be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4007f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff841000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 0x55bd400ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff83be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4007f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55272/0 (socket says 172.21.15.67:55272) 2024-03-30T02:37:58.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff83be00700 1 -- 172.21.15.67:0/3065642239 learned_addr learned my addr 172.21.15.67:0/3065642239 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:58.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff841000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 0x55bd400ed5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40632/0 (socket says 172.21.15.67:40632) 2024-03-30T02:37:58.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff83b400700 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 0x55bd4007fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff83be00700 1 -- 172.21.15.67:0/3065642239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 msgr2=0x55bd4007fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff83be00700 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 0x55bd4007fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff83be00700 1 -- 172.21.15.67:0/3065642239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 msgr2=0x55bd400ed5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff83be00700 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 0x55bd400ed5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.717+0000 7ff83be00700 1 -- 172.21.15.67:0/3065642239 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff828012050 con 0x55bd4006e7a0 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.718+0000 7ff83be00700 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4007f700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff82c098e70 tx=0x7ff82c0d3fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6719a0be6aaba491 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.718+0000 7ff826000700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.718+0000 7ff826000700 1 -- 172.21.15.67:0/3065642239 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c09a220 con 0x55bd4006e7a0 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.718+0000 7ff856c1bb80 1 -- 172.21.15.67:0/3065642239 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff83400f040 con 0x55bd4006e7a0 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.718+0000 7ff826000700 1 -- 172.21.15.67:0/3065642239 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff82c09b220 con 0x55bd4006e7a0 2024-03-30T02:37:58.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.718+0000 7ff826000700 1 -- 172.21.15.67:0/3065642239 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff82c09a480 con 0x55bd4006e7a0 2024-03-30T02:37:58.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.718+0000 7ff856c1bb80 10 client.25918 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:58.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.718+0000 7ff856c1bb80 1 -- 172.21.15.67:0/3065642239 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bd3fe46cd0 con 0x55bd4006e7a0 2024-03-30T02:37:58.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.718+0000 7ff826000700 1 -- 172.21.15.67:0/3065642239 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff83400f040 con 0x55bd4006e7a0 2024-03-30T02:37:58.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.719+0000 7ff826000700 1 -- 172.21.15.67:0/3065642239 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff82c0d0070 con 0x55bd4006e7a0 2024-03-30T02:37:58.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.719+0000 7ff856c1bb80 10 client.25918 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:58.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.719+0000 7ff856c1bb80 10 client.25918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:37:58.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.719+0000 7ff856c1bb80 20 client.25918 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:58.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.719+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55bd400ef010 0x55bd400f1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:58.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.719+0000 7ff856c1bb80 4 client.25918 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.719+0000 7ff856c1bb80 1 -- 172.21.15.67:0/3065642239 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd3fe0fcf0 con 0x55bd400ef010 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.719+0000 7ff83b400700 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55bd400ef010 0x55bd400f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.720+0000 7ff826000700 10 client.25918 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.720+0000 7ff83b400700 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55bd400ef010 0x55bd400f1400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=575690860afbb8eb server_cookie=91fcf069a225dc49 in_seq=0 out_seq=0 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.720+0000 7ff826000700 1 -- 172.21.15.67:0/3065642239 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bd3fe0fcf0 con 0x55bd400ef010 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.720+0000 7ff826000700 10 client.25918 handle_command_reply: tid=0 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.720+0000 7ff856c1bb80 1 client.25918 shutdown 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.720+0000 7ff856c1bb80 1 -- 172.21.15.67:0/3065642239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 msgr2=0x55bd4007f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.720+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4007f700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff82c098e70 tx=0x7ff82c0d3fb0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff856c1bb80 1 -- 172.21.15.67:0/3065642239 shutdown_connections 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55bd400ef010 0x55bd400f1400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bd400827d0 0x55bd400ed5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd4006e7a0 0x55bd4007f700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff826000700 0 client.25918 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff856c1bb80 1 --2- 172.21.15.67:0/3065642239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd4006f0c0 0x55bd4007fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff856c1bb80 1 -- 172.21.15.67:0/3065642239 >> 172.21.15.67:0/3065642239 conn(0x55bd3ffce6a0 msgr2=0x55bd40075800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff856c1bb80 1 -- 172.21.15.67:0/3065642239 shutdown_connections 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff856c1bb80 1 -- 172.21.15.67:0/3065642239 wait complete. 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.721+0000 7ff856c1bb80 20 client.25918 trim_cache size 0 max 16384 2024-03-30T02:37:58.724 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 -- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff8200a1320 msgr2=0x7ff8200a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff8200a1320 0x7ff8200a37e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff834009550 tx=0x7ff834014700 comp rx=0 tx=0).stop 2024-03-30T02:37:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 -- 172.21.15.67:0/2500057357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c101cf0 msgr2=0x7ff83c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c101cf0 0x7ff83c1992c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff82c00fa00 tx=0x7ff82c00fa30 comp rx=0 tx=0).stop 2024-03-30T02:37:58.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 -- 172.21.15.67:0/2500057357 shutdown_connections 2024-03-30T02:37:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff8200a1320 0x7ff8200a37e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff83c10b110 0x7ff83c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff83c101cf0 0x7ff83c1992c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 --2- 172.21.15.67:0/2500057357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff83c102610 0x7ff83c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 -- 172.21.15.67:0/2500057357 >> 172.21.15.67:0/2500057357 conn(0x7ff83c0fb510 msgr2=0x7ff83c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 -- 172.21.15.67:0/2500057357 shutdown_connections 2024-03-30T02:37:58.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:58.722+0000 7ff841a00700 1 -- 172.21.15.67:0/2500057357 wait complete. 2024-03-30T02:37:58.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:37:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:58 smithi082 ceph-mon[79176]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:37:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:37:58 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1574175162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:37:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.058+0000 7f44b8e00700 1 -- 172.21.15.67:0/872976914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b4102690 msgr2=0x7f44b410ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.058+0000 7f44b8e00700 1 --2- 172.21.15.67:0/872976914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b4102690 0x7f44b410ab70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009990 tx=0x7f44a0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.058+0000 7f44b8e00700 1 -- 172.21.15.67:0/872976914 shutdown_connections 2024-03-30T02:37:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.058+0000 7f44b8e00700 1 --2- 172.21.15.67:0/872976914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b410b140 0x7f44b410d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.058+0000 7f44b8e00700 1 --2- 172.21.15.67:0/872976914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b4101d70 0x7f44b4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.058+0000 7f44b8e00700 1 --2- 172.21.15.67:0/872976914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b4102690 0x7f44b410ab70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.058+0000 7f44b8e00700 1 -- 172.21.15.67:0/872976914 >> 172.21.15.67:0/872976914 conn(0x7f44b40fb510 msgr2=0x7f44b40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.058+0000 7f44b8e00700 1 -- 172.21.15.67:0/872976914 shutdown_connections 2024-03-30T02:37:59.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.058+0000 7f44b8e00700 1 -- 172.21.15.67:0/872976914 wait complete. 2024-03-30T02:37:59.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.060+0000 7f44b8e00700 1 Processor -- start 2024-03-30T02:37:59.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.060+0000 7f44b8e00700 1 -- start start 2024-03-30T02:37:59.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.060+0000 7f44b8e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b4101d70 0x7f44b4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.060+0000 7f44b8e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b4102690 0x7f44b4199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.060+0000 7f44b8e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b410b140 0x7f44b419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.061+0000 7f44b8e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b419e150 con 0x7f44b4102690 2024-03-30T02:37:59.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.061+0000 7f44b8e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b419e2c0 con 0x7f44b410b140 2024-03-30T02:37:59.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.061+0000 7f44b8e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b419e480 con 0x7f44b4101d70 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.061+0000 7f44b3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b410b140 0x7f44b419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.061+0000 7f44b3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b410b140 0x7f44b419db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37288/0 (socket says 172.21.15.67:37288) 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.061+0000 7f44b2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b4102690 0x7f44b4199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.061+0000 7f44b3e00700 1 -- 172.21.15.67:0/1178810952 learned_addr learned my addr 172.21.15.67:0/1178810952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.061+0000 7f44b3400700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b4101d70 0x7f44b4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.062+0000 7f44b2a00700 1 -- 172.21.15.67:0/1178810952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b4101d70 msgr2=0x7f44b4199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.062+0000 7f44b2a00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b4101d70 0x7f44b4199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.062+0000 7f44b2a00700 1 -- 172.21.15.67:0/1178810952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b410b140 msgr2=0x7f44b419db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.062+0000 7f44b2a00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b410b140 0x7f44b419db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.062+0000 7f44b2a00700 1 -- 172.21.15.67:0/1178810952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4009520 con 0x7f44b4102690 2024-03-30T02:37:59.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.062+0000 7f44b3400700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b4101d70 0x7f44b4199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:59.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.062+0000 7f44b2a00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b4102690 0x7f44b4199770 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f44a00098e0 tx=0x7f44a0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.062+0000 7f44abe00700 1 -- 172.21.15.67:0/1178810952 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a001d070 con 0x7f44b4102690 2024-03-30T02:37:59.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.063+0000 7f44abe00700 1 -- 172.21.15.67:0/1178810952 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a0022470 con 0x7f44b4102690 2024-03-30T02:37:59.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.063+0000 7f44b8e00700 1 -- 172.21.15.67:0/1178810952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a00095f0 con 0x7f44b4102690 2024-03-30T02:37:59.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.063+0000 7f44abe00700 1 -- 172.21.15.67:0/1178810952 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0017650 con 0x7f44b4102690 2024-03-30T02:37:59.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.063+0000 7f44b8e00700 1 -- 172.21.15.67:0/1178810952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b41a5dc0 con 0x7f44b4102690 2024-03-30T02:37:59.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.069+0000 7f44cde66b80 1 Processor -- start 2024-03-30T02:37:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.069+0000 7f44cde66b80 1 -- start start 2024-03-30T02:37:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.069+0000 7f44abe00700 1 -- 172.21.15.67:0/1178810952 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f44a00228a0 con 0x7f44b4102690 2024-03-30T02:37:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.069+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2617a0 0x557c2b261b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.069+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2620c0 0x557c2b26ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.069+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b26f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.069+0000 7f44cde66b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b1ba990 con 0x557c2b26cfc0 2024-03-30T02:37:59.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.069+0000 7f44cde66b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b1b8350 con 0x557c2b2620c0 2024-03-30T02:37:59.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.069+0000 7f44cde66b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b1b3c40 con 0x557c2b2617a0 2024-03-30T02:37:59.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.070+0000 7f44b3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2620c0 0x557c2b26ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.070+0000 7f44b2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b26f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.070+0000 7f44b3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2620c0 0x557c2b26ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37304/0 (socket says 172.21.15.67:37304) 2024-03-30T02:37:59.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.070+0000 7f44b3400700 1 -- 172.21.15.67:0/1514536661 learned_addr learned my addr 172.21.15.67:0/1514536661 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.070+0000 7f44b3e00700 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2617a0 0x557c2b261b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.071+0000 7f44b3400700 1 -- 172.21.15.67:0/1514536661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2617a0 msgr2=0x557c2b261b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.071+0000 7f44b3400700 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2617a0 0x557c2b261b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.071+0000 7f44b3400700 1 -- 172.21.15.67:0/1514536661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 msgr2=0x557c2b26f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.071+0000 7f44b3400700 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b26f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.071+0000 7f44b3400700 1 -- 172.21.15.67:0/1514536661 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c2b0353b0 con 0x557c2b2620c0 2024-03-30T02:37:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.071+0000 7f44b3e00700 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2617a0 0x557c2b261b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.071+0000 7f44b2a00700 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b26f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.071+0000 7f44b3400700 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2620c0 0x557c2b26ca80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009f70 tx=0x7f44a40116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e5429703589a151 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.071+0000 7f44abe00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f44980a14f0 0x7f44980a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.072+0000 7f44abe00700 1 -- 172.21.15.67:0/1178810952 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f44a00ce450 con 0x7f44b4102690 2024-03-30T02:37:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.072+0000 7f44a8c00700 1 -- 172.21.15.67:0/1514536661 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4011ee0 con 0x557c2b2620c0 2024-03-30T02:37:59.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.072+0000 7f44b3e00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f44980a14f0 0x7f44980a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.072+0000 7f44a8c00700 1 -- 172.21.15.67:0/1514536661 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a4012910 con 0x557c2b2620c0 2024-03-30T02:37:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.072+0000 7f44a8c00700 1 -- 172.21.15.67:0/1514536661 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400f990 con 0x557c2b2620c0 2024-03-30T02:37:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44cde66b80 1 -- 172.21.15.67:0/1514536661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2620c0 msgr2=0x557c2b26ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44b3e00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f44980a14f0 0x7f44980a39b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009fd0 tx=0x7f44ac00b040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44cde66b80 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2620c0 0x557c2b26ca80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009f70 tx=0x7f44a40116c0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44cde66b80 1 -- 172.21.15.67:0/1514536661 shutdown_connections 2024-03-30T02:37:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44cde66b80 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b26f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44cde66b80 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2617a0 0x557c2b261b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44cde66b80 1 --2- 172.21.15.67:0/1514536661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2620c0 0x557c2b26ca80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44cde66b80 1 -- 172.21.15.67:0/1514536661 >> 172.21.15.67:0/1514536661 conn(0x557c2b1c16a0 msgr2=0x557c2b25dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44cde66b80 1 -- 172.21.15.67:0/1514536661 shutdown_connections 2024-03-30T02:37:59.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.073+0000 7f44cde66b80 1 -- 172.21.15.67:0/1514536661 wait complete. 2024-03-30T02:37:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.074+0000 7f44cde66b80 1 -- start start 2024-03-30T02:37:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.074+0000 7f44cde66b80 1 Processor -- start 2024-03-30T02:37:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.075+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b26c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.075+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b2620c0 0x557c2b269d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.075+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b26cfc0 0x557c2b26a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.075+0000 7f44cde66b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b14fd80 con 0x557c2b2620c0 2024-03-30T02:37:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.075+0000 7f44cde66b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b14fa20 con 0x557c2b2617a0 2024-03-30T02:37:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.075+0000 7f44cde66b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b0376a0 con 0x557c2b26cfc0 2024-03-30T02:37:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.075+0000 7f44cde66b80 10 client.0 _ll_register_callbacks cb 0x557c2b159470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b2620c0 0x557c2b269d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b2620c0 0x557c2b269d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40670/0 (socket says 172.21.15.67:40670) 2024-03-30T02:37:59.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b2a00700 1 -- 172.21.15.67:0/2385962559 learned_addr learned my addr 172.21.15.67:0/2385962559 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b3e00700 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b26cfc0 0x557c2b26a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b3400700 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b26c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b3e00700 1 -- 172.21.15.67:0/2385962559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 msgr2=0x557c2b26c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b3e00700 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b26c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b3e00700 1 -- 172.21.15.67:0/2385962559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b2620c0 msgr2=0x557c2b269d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b3e00700 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b2620c0 0x557c2b269d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b3e00700 1 -- 172.21.15.67:0/2385962559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b41a5dc0 con 0x557c2b26cfc0 2024-03-30T02:37:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.076+0000 7f44b3400700 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b26c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:37:59.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.077+0000 7f44b2a00700 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b2620c0 0x557c2b269d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.077+0000 7f44b3e00700 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b26cfc0 0x557c2b26a320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f44ac016a60 tx=0x7f44ac004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d72a6012e2c280 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.077+0000 7f449d600700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.077+0000 7f449d600700 1 -- 172.21.15.67:0/2385962559 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00e040 con 0x557c2b26cfc0 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.077+0000 7f44cde66b80 1 -- 172.21.15.67:0/2385962559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c2b174380 con 0x557c2b26cfc0 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.077+0000 7f449d600700 1 -- 172.21.15.67:0/2385962559 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f44ac016e20 con 0x557c2b26cfc0 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f449d600700 1 -- 172.21.15.67:0/2385962559 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44ac015a20 con 0x557c2b26cfc0 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f44cde66b80 10 client.25930 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f44cde66b80 1 -- 172.21.15.67:0/2385962559 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f44a4009ba0 con 0x557c2b26cfc0 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f449d600700 1 -- 172.21.15.67:0/2385962559 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c2b174380 con 0x557c2b26cfc0 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f449d600700 1 -- 172.21.15.67:0/2385962559 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f44ac023d10 con 0x557c2b26cfc0 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f44cde66b80 10 client.25930 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f44cde66b80 10 client.25930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f44cde66b80 20 client.25930 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:59.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f44cde66b80 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557c2b285400 0x557c2b287820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f44cde66b80 4 client.25930 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:59.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.078+0000 7f44cde66b80 1 -- 172.21.15.67:0/2385962559 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c2b1ac700 con 0x557c2b285400 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.079+0000 7f44b3400700 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557c2b285400 0x557c2b287820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.079+0000 7f44b3400700 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557c2b285400 0x557c2b287820 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cdd3ecc6f1928a6 server_cookie=69de1043bc84f2d9 in_seq=0 out_seq=0 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.079+0000 7f449d600700 10 client.25930 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.080+0000 7f449d600700 1 -- 172.21.15.67:0/2385962559 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c2b1ac700 con 0x557c2b285400 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.080+0000 7f449d600700 10 client.25930 handle_command_reply: tid=0 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.080+0000 7f44cde66b80 1 client.25930 shutdown 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.080+0000 7f44cde66b80 1 -- 172.21.15.67:0/2385962559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b26cfc0 msgr2=0x557c2b26a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.080+0000 7f44cde66b80 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b26cfc0 0x557c2b26a320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f44ac016a60 tx=0x7f44ac004b40 comp rx=0 tx=0).stop 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f44cde66b80 1 -- 172.21.15.67:0/2385962559 shutdown_connections 2024-03-30T02:37:59.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f44cde66b80 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b26cfc0 0x557c2b26a320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f44cde66b80 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b26c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f44cde66b80 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557c2b285400 0x557c2b287820 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f44cde66b80 1 --2- 172.21.15.67:0/2385962559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b2620c0 0x557c2b269d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f449d600700 0 client.25930 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:37:59.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f44cde66b80 1 -- 172.21.15.67:0/2385962559 >> 172.21.15.67:0/2385962559 conn(0x557c2b1c16a0 msgr2=0x557c2b263890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f44cde66b80 1 -- 172.21.15.67:0/2385962559 shutdown_connections 2024-03-30T02:37:59.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f44cde66b80 1 -- 172.21.15.67:0/2385962559 wait complete. 2024-03-30T02:37:59.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.081+0000 7f44cde66b80 20 client.25930 trim_cache size 0 max 16384 2024-03-30T02:37:59.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.096+0000 7f44cde66b80 1 Processor -- start 2024-03-30T02:37:59.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.096+0000 7f44cde66b80 1 -- start start 2024-03-30T02:37:59.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.096+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b25d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.096+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 0x557c2b27c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.096+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b2567f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.096+0000 7f44cde66b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b268240 con 0x557c2b26cfc0 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.096+0000 7f44cde66b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b256d30 con 0x557c2b2617a0 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.096+0000 7f44cde66b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b26b1d0 con 0x557c2b2620c0 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b25d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b2a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 0x557c2b27c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b25d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37328/0 (socket says 172.21.15.67:37328) 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b3400700 1 -- 172.21.15.67:0/1727438310 learned_addr learned my addr 172.21.15.67:0/1727438310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b3e00700 1 --2- 172.21.15.67:0/1727438310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b2567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b2a00700 1 -- 172.21.15.67:0/1727438310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 msgr2=0x557c2b25d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b2a00700 1 --2- 172.21.15.67:0/1727438310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b25d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b2a00700 1 -- 172.21.15.67:0/1727438310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 msgr2=0x557c2b2567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b2a00700 1 --2- 172.21.15.67:0/1727438310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b2567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b2a00700 1 -- 172.21.15.67:0/1727438310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c2afe1b60 con 0x557c2b2620c0 2024-03-30T02:37:59.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.097+0000 7f44b3e00700 1 --2- 172.21.15.67:0/1727438310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b2567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:59.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.098+0000 7f44b2a00700 1 --2- 172.21.15.67:0/1727438310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 0x557c2b27c450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44a0000c00 tx=0x7f44a00d1d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d24aee3d21957cd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.098+0000 7f44a9600700 1 -- 172.21.15.67:0/1727438310 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0097530 con 0x557c2b2620c0 2024-03-30T02:37:59.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.098+0000 7f44a9600700 1 -- 172.21.15.67:0/1727438310 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a009afc0 con 0x557c2b2620c0 2024-03-30T02:37:59.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.098+0000 7f44a9600700 1 -- 172.21.15.67:0/1727438310 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a009ab50 con 0x557c2b2620c0 2024-03-30T02:37:59.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.099+0000 7f44cde66b80 1 -- 172.21.15.67:0/1727438310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 msgr2=0x557c2b27c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.099+0000 7f44cde66b80 1 --2- 172.21.15.67:0/1727438310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 0x557c2b27c450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44a0000c00 tx=0x7f44a00d1d90 comp rx=0 tx=0).stop 2024-03-30T02:37:59.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.099+0000 7f44cde66b80 1 -- 172.21.15.67:0/1727438310 shutdown_connections 2024-03-30T02:37:59.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.099+0000 7f44cde66b80 1 --2- 172.21.15.67:0/1727438310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b2567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.099+0000 7f44cde66b80 1 --2- 172.21.15.67:0/1727438310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b25d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.099+0000 7f44cde66b80 1 --2- 172.21.15.67:0/1727438310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 0x557c2b27c450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.099+0000 7f44cde66b80 1 -- 172.21.15.67:0/1727438310 >> 172.21.15.67:0/1727438310 conn(0x557c2b1c16a0 msgr2=0x557c2b26c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.099+0000 7f44cde66b80 1 -- 172.21.15.67:0/1727438310 shutdown_connections 2024-03-30T02:37:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.099+0000 7f44cde66b80 1 -- 172.21.15.67:0/1727438310 wait complete. 2024-03-30T02:37:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.100+0000 7f44cde66b80 1 -- start start 2024-03-30T02:37:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.100+0000 7f44cde66b80 1 Processor -- start 2024-03-30T02:37:59.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b27a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 0x557c2b27ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44cde66b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b2d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44cde66b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b2d8770 con 0x557c2b26cfc0 2024-03-30T02:37:59.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44cde66b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b2d88b0 con 0x557c2b2617a0 2024-03-30T02:37:59.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44cde66b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2b2d8a70 con 0x557c2b2620c0 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44cde66b80 10 client.0 _ll_register_callbacks cb 0x557c2b159470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44b3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b27a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44b3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b27a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37330/0 (socket says 172.21.15.67:37330) 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44b3400700 1 -- 172.21.15.67:0/522999170 learned_addr learned my addr 172.21.15.67:0/522999170 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44b2a00700 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 0x557c2b27ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44b3400700 1 -- 172.21.15.67:0/522999170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 msgr2=0x557c2b27ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44b3400700 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 0x557c2b27ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44b3400700 1 -- 172.21.15.67:0/522999170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 msgr2=0x557c2b2d8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44b3400700 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b2d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.101+0000 7f44b3400700 1 -- 172.21.15.67:0/522999170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a00095f0 con 0x557c2b2617a0 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.102+0000 7f44b3400700 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b27a620 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f44a401c040 tx=0x7f44a4009850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8e5b7fb9f9625b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.102+0000 7f449d600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:59.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.102+0000 7f449d600700 1 -- 172.21.15.67:0/522999170 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4029040 con 0x557c2b2617a0 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.102+0000 7f44cde66b80 1 -- 172.21.15.67:0/522999170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44a4012910 con 0x557c2b2617a0 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.102+0000 7f449d600700 1 -- 172.21.15.67:0/522999170 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f44a402f070 con 0x557c2b2617a0 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.102+0000 7f449d600700 1 -- 172.21.15.67:0/522999170 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44a4012650 con 0x557c2b2617a0 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f44cde66b80 10 client.25946 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f44cde66b80 1 -- 172.21.15.67:0/522999170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557c2b179ea0 con 0x557c2b2617a0 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f449d600700 1 -- 172.21.15.67:0/522999170 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a4012910 con 0x557c2b2617a0 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f449d600700 1 -- 172.21.15.67:0/522999170 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f44a40113e0 con 0x557c2b2617a0 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f44cde66b80 10 client.25946 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f44cde66b80 10 client.25946 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f44cde66b80 20 client.25946 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f44cde66b80 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557c2b2da180 0x557c2b2dc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f44cde66b80 4 client.25946 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:59.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.103+0000 7f44cde66b80 1 -- 172.21.15.67:0/522999170 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c2b002cf0 con 0x557c2b2da180 2024-03-30T02:37:59.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.104+0000 7f44b2a00700 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557c2b2da180 0x557c2b2dc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.104+0000 7f44b2a00700 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557c2b2da180 0x557c2b2dc5a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49ac43bbf2761a83 server_cookie=223349fb21af79ef in_seq=0 out_seq=0 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.104+0000 7f449d600700 10 client.25946 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f449d600700 1 -- 172.21.15.67:0/522999170 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c2b002cf0 con 0x557c2b2da180 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f449d600700 10 client.25946 handle_command_reply: tid=0 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f44cde66b80 1 client.25946 shutdown 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f44cde66b80 1 -- 172.21.15.67:0/522999170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 msgr2=0x557c2b27a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f44cde66b80 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b27a620 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f44a401c040 tx=0x7f44a4009850 comp rx=0 tx=0).stop 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f44cde66b80 1 -- 172.21.15.67:0/522999170 shutdown_connections 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f44cde66b80 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557c2b2da180 0x557c2b2dc5a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f449d600700 0 client.25946 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f44cde66b80 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c2b26cfc0 0x557c2b2d8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f44cde66b80 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c2b2617a0 0x557c2b27a620 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f44cde66b80 1 --2- 172.21.15.67:0/522999170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c2b2620c0 0x557c2b27ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.105+0000 7f44cde66b80 1 -- 172.21.15.67:0/522999170 >> 172.21.15.67:0/522999170 conn(0x557c2b1c16a0 msgr2=0x557c2b25d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.106+0000 7f44cde66b80 1 -- 172.21.15.67:0/522999170 shutdown_connections 2024-03-30T02:37:59.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.106+0000 7f44cde66b80 1 -- 172.21.15.67:0/522999170 wait complete. 2024-03-30T02:37:59.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.106+0000 7f44cde66b80 20 client.25946 trim_cache size 0 max 16384 2024-03-30T02:37:59.108 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:59.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.106+0000 7f44b8e00700 1 -- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f44980a14f0 msgr2=0x7f44980a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.106+0000 7f44b8e00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f44980a14f0 0x7f44980a39b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009fd0 tx=0x7f44ac00b040 comp rx=0 tx=0).stop 2024-03-30T02:37:59.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.106+0000 7f44b8e00700 1 -- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b4102690 msgr2=0x7f44b4199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.106+0000 7f44b8e00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b4102690 0x7f44b4199770 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f44a00098e0 tx=0x7f44a0005740 comp rx=0 tx=0).stop 2024-03-30T02:37:59.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.107+0000 7f44b8e00700 1 -- 172.21.15.67:0/1178810952 shutdown_connections 2024-03-30T02:37:59.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.107+0000 7f44b8e00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f44980a14f0 0x7f44980a39b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.107+0000 7f44b8e00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b410b140 0x7f44b419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.107+0000 7f44b8e00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44b4101d70 0x7f44b4199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.107+0000 7f44b8e00700 1 --2- 172.21.15.67:0/1178810952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f44b4102690 0x7f44b4199770 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.107+0000 7f44b8e00700 1 -- 172.21.15.67:0/1178810952 >> 172.21.15.67:0/1178810952 conn(0x7f44b40fb510 msgr2=0x7f44b4100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.107+0000 7f44b8e00700 1 -- 172.21.15.67:0/1178810952 shutdown_connections 2024-03-30T02:37:59.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.107+0000 7f44b8e00700 1 -- 172.21.15.67:0/1178810952 wait complete. 2024-03-30T02:37:59.128 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:37:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.380+0000 7f92e3e00700 1 -- 172.21.15.67:0/3902188555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e410d4a0 msgr2=0x7f92e410f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.380+0000 7f92e3e00700 1 --2- 172.21.15.67:0/3902188555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e410d4a0 0x7f92e410f890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009f90 tx=0x7f92dc00b330 comp rx=0 tx=0).stop 2024-03-30T02:37:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.380+0000 7f92e3e00700 1 -- 172.21.15.67:0/3902188555 shutdown_connections 2024-03-30T02:37:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.380+0000 7f92e3e00700 1 --2- 172.21.15.67:0/3902188555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ff8f0 0x7f92e40ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.380+0000 7f92e3e00700 1 --2- 172.21.15.67:0/3902188555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e410d4a0 0x7f92e410f890 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.380+0000 7f92e3e00700 1 --2- 172.21.15.67:0/3902188555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92e41002a0 0x7f92e410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.380+0000 7f92e3e00700 1 -- 172.21.15.67:0/3902188555 >> 172.21.15.67:0/3902188555 conn(0x7f92e40fb510 msgr2=0x7f92e40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.380+0000 7f92e3e00700 1 -- 172.21.15.67:0/3902188555 shutdown_connections 2024-03-30T02:37:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.381+0000 7f92e3e00700 1 -- 172.21.15.67:0/3902188555 wait complete. 2024-03-30T02:37:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.382+0000 7f92e3e00700 1 Processor -- start 2024-03-30T02:37:59.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.382+0000 7f92e3e00700 1 -- start start 2024-03-30T02:37:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ff8f0 0x7f92e4199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e41002a0 0x7f92e41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e3e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92e410d4a0 0x7f92e419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e3e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e419e2b0 con 0x7f92e40ff8f0 2024-03-30T02:37:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e3e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e419e420 con 0x7f92e41002a0 2024-03-30T02:37:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e3e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e419e5e0 con 0x7f92e410d4a0 2024-03-30T02:37:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ff8f0 0x7f92e4199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ff8f0 0x7f92e4199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40708/0 (socket says 172.21.15.67:40708) 2024-03-30T02:37:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e2a00700 1 -- 172.21.15.67:0/69720871 learned_addr learned my addr 172.21.15.67:0/69720871 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e2000700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e41002a0 0x7f92e41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.383+0000 7f92e3400700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92e410d4a0 0x7f92e419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.384+0000 7f92e2a00700 1 -- 172.21.15.67:0/69720871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92e410d4a0 msgr2=0x7f92e419dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.384+0000 7f92e2a00700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92e410d4a0 0x7f92e419dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.384+0000 7f92e2a00700 1 -- 172.21.15.67:0/69720871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e41002a0 msgr2=0x7f92e41998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.384+0000 7f92e2a00700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e41002a0 0x7f92e41998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.384+0000 7f92e2a00700 1 -- 172.21.15.67:0/69720871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc009c40 con 0x7f92e40ff8f0 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.384+0000 7f92e2000700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e41002a0 0x7f92e41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.384+0000 7f92e3400700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92e410d4a0 0x7f92e419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.384+0000 7f92e2a00700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ff8f0 0x7f92e4199390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9c0 tx=0x7f92d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.384+0000 7f92db400700 1 -- 172.21.15.67:0/69720871 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d400cbd0 con 0x7f92e40ff8f0 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.385+0000 7f92e3e00700 1 -- 172.21.15.67:0/69720871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e419e8c0 con 0x7f92e40ff8f0 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.385+0000 7f92db400700 1 -- 172.21.15.67:0/69720871 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92d400eeb0 con 0x7f92e40ff8f0 2024-03-30T02:37:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.385+0000 7f92e3e00700 1 -- 172.21.15.67:0/69720871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e41a6100 con 0x7f92e40ff8f0 2024-03-30T02:37:59.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.385+0000 7f92db400700 1 -- 172.21.15.67:0/69720871 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40054b0 con 0x7f92e40ff8f0 2024-03-30T02:37:59.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.391+0000 7f92db400700 1 -- 172.21.15.67:0/69720871 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f92d4005610 con 0x7f92e40ff8f0 2024-03-30T02:37:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.391+0000 7f92fd5d7b80 1 Processor -- start 2024-03-30T02:37:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.391+0000 7f92fd5d7b80 1 -- start start 2024-03-30T02:37:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e49b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e54a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 0x55fde2e573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2da2990 con 0x55fde2e4a0c0 2024-03-30T02:37:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2da0350 con 0x55fde2e54fc0 2024-03-30T02:37:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2d9bc40 con 0x55fde2e497a0 2024-03-30T02:37:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e54a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e54a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40720/0 (socket says 172.21.15.67:40720) 2024-03-30T02:37:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92e2a00700 1 -- 172.21.15.67:0/2779183921 learned_addr learned my addr 172.21.15.67:0/2779183921 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92e3400700 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e49b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.392+0000 7f92e2000700 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 0x55fde2e573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92e2a00700 1 -- 172.21.15.67:0/2779183921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 msgr2=0x55fde2e49b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92e2a00700 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e49b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92e2a00700 1 -- 172.21.15.67:0/2779183921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 msgr2=0x55fde2e573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92e2a00700 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 0x55fde2e573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92e2a00700 1 -- 172.21.15.67:0/2779183921 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde2c1d3b0 con 0x55fde2e4a0c0 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92e2000700 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 0x55fde2e573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92db400700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f92c80a1320 0x7f92c80a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92e2a00700 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e54a80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f92d40ca5f0 tx=0x7f92d40d5160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0cfe062bc13e6e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92db400700 1 -- 172.21.15.67:0/69720871 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f92d4014070 con 0x7f92e40ff8f0 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92e3400700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f92c80a1320 0x7f92c80a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.393+0000 7f92cfe00700 1 -- 172.21.15.67:0/2779183921 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40962d0 con 0x55fde2e4a0c0 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.394+0000 7f92cfe00700 1 -- 172.21.15.67:0/2779183921 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92d40d0cb0 con 0x55fde2e4a0c0 2024-03-30T02:37:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.394+0000 7f92cfe00700 1 -- 172.21.15.67:0/2779183921 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d409a690 con 0x55fde2e4a0c0 2024-03-30T02:37:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.394+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2779183921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 msgr2=0x55fde2e54a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.394+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e54a80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f92d40ca5f0 tx=0x7f92d40d5160 comp rx=0 tx=0).stop 2024-03-30T02:37:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.394+0000 7f92e3400700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f92c80a1320 0x7f92c80a37e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009790 tx=0x7f92dc0096e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.395+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2779183921 shutdown_connections 2024-03-30T02:37:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.395+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 0x55fde2e573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.395+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e49b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.395+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2779183921 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e54a80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.395+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2779183921 >> 172.21.15.67:0/2779183921 conn(0x55fde2da96a0 msgr2=0x55fde2e45c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.395+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2779183921 shutdown_connections 2024-03-30T02:37:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.395+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2779183921 wait complete. 2024-03-30T02:37:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.395+0000 7f92fd5d7b80 1 -- start start 2024-03-30T02:37:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.396+0000 7f92fd5d7b80 1 Processor -- start 2024-03-30T02:37:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.396+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e497a0 0x55fde2e548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.396+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.396+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 0x55fde2e521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.396+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2d37d80 con 0x55fde2e4a0c0 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.396+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2d37a20 con 0x55fde2e497a0 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.396+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2c1f6a0 con 0x55fde2e54fc0 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92fd5d7b80 10 client.0 _ll_register_callbacks cb 0x55fde2d41470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e497a0 0x55fde2e548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e497a0 0x55fde2e548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37370/0 (socket says 172.21.15.67:37370) 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2a00700 1 -- 172.21.15.67:0/2409733689 learned_addr learned my addr 172.21.15.67:0/2409733689 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e3400700 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 0x55fde2e521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2a00700 1 -- 172.21.15.67:0/2409733689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 msgr2=0x55fde2e521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2a00700 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 0x55fde2e521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2a00700 1 -- 172.21.15.67:0/2409733689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 msgr2=0x55fde2e51c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2a00700 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e51c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2a00700 1 -- 172.21.15.67:0/2409733689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e41a6100 con 0x55fde2e497a0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.397+0000 7f92e2000700 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e51c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.398+0000 7f92e2a00700 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e497a0 0x55fde2e548b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f92d40ca6e0 tx=0x7f92d4096960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749b17c391ffa8dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.398+0000 7f92ccc00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.398+0000 7f92ccc00700 1 -- 172.21.15.67:0/2409733689 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40962d0 con 0x55fde2e497a0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.398+0000 7f92ccc00700 1 -- 172.21.15.67:0/2409733689 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f92d40cd170 con 0x55fde2e497a0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.398+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2409733689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fde2d94700 con 0x55fde2e497a0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.398+0000 7f92ccc00700 1 -- 172.21.15.67:0/2409733689 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92d4099d60 con 0x55fde2e497a0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.398+0000 7f92fd5d7b80 10 client.25958 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.398+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2409733689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fde2bc9b60 con 0x55fde2e497a0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.399+0000 7f92ccc00700 1 -- 172.21.15.67:0/2409733689 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fde2d94700 con 0x55fde2e497a0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.399+0000 7f92ccc00700 1 -- 172.21.15.67:0/2409733689 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f92d40b8170 con 0x55fde2e497a0 2024-03-30T02:37:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.399+0000 7f92fd5d7b80 10 client.25958 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:59.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.399+0000 7f92fd5d7b80 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:59.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.399+0000 7f92fd5d7b80 20 client.25958 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:59.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.399+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fde2e64850 0x55fde2e66c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.399+0000 7f92fd5d7b80 4 client.25958 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:37:59.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.399+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2409733689 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fde2beacf0 con 0x55fde2e64850 2024-03-30T02:37:59.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.399+0000 7f92e2000700 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fde2e64850 0x55fde2e66c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.400+0000 7f92e2000700 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fde2e64850 0x55fde2e66c70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d072a74eecc53024 server_cookie=1095e77e80caf8bb in_seq=0 out_seq=0 2024-03-30T02:37:59.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.400+0000 7f92ccc00700 10 client.25958 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:59.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92ccc00700 1 -- 172.21.15.67:0/2409733689 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fde2beacf0 con 0x55fde2e64850 2024-03-30T02:37:59.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92ccc00700 10 client.25958 handle_command_reply: tid=0 2024-03-30T02:37:59.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92fd5d7b80 1 client.25958 shutdown 2024-03-30T02:37:59.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2409733689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e497a0 msgr2=0x55fde2e548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e497a0 0x55fde2e548b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f92d40ca6e0 tx=0x7f92d4096960 comp rx=0 tx=0).stop 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2409733689 shutdown_connections 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 0x55fde2e521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e497a0 0x55fde2e548b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fde2e64850 0x55fde2e66c70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/2409733689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e51c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2409733689 >> 172.21.15.67:0/2409733689 conn(0x55fde2da96a0 msgr2=0x55fde2e4b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.401+0000 7f92ccc00700 0 client.25958 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.402+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2409733689 shutdown_connections 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.402+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/2409733689 wait complete. 2024-03-30T02:37:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.402+0000 7f92fd5d7b80 20 client.25958 trim_cache size 0 max 16384 2024-03-30T02:37:59.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92fd5d7b80 1 Processor -- start 2024-03-30T02:37:59.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92fd5d7b80 1 -- start start 2024-03-30T02:37:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e497a0 0x55fde2e51520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e4a0c0 0x55fde2e5bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 0x55fde2e45700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2e41ad0 con 0x55fde2e497a0 2024-03-30T02:37:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2e5c240 con 0x55fde2e4a0c0 2024-03-30T02:37:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2e45160 con 0x55fde2e54fc0 2024-03-30T02:37:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e497a0 0x55fde2e51520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.418+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e497a0 0x55fde2e51520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40730/0 (socket says 172.21.15.67:40730) 2024-03-30T02:37:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.419+0000 7f92e2a00700 1 -- 172.21.15.67:0/4085794846 learned_addr learned my addr 172.21.15.67:0/4085794846 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.419+0000 7f92e3400700 1 --2- 172.21.15.67:0/4085794846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 0x55fde2e45700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.419+0000 7f92e2a00700 1 -- 172.21.15.67:0/4085794846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 msgr2=0x55fde2e45700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.419+0000 7f92e2a00700 1 --2- 172.21.15.67:0/4085794846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 0x55fde2e45700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.419+0000 7f92e2a00700 1 -- 172.21.15.67:0/4085794846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e4a0c0 msgr2=0x55fde2e5bcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:37:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.419+0000 7f92e2a00700 1 --2- 172.21.15.67:0/4085794846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e4a0c0 0x55fde2e5bcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.419+0000 7f92e2a00700 1 -- 172.21.15.67:0/4085794846 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fde2d61ea0 con 0x55fde2e497a0 2024-03-30T02:37:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.419+0000 7f92e2a00700 1 --2- 172.21.15.67:0/4085794846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e497a0 0x55fde2e51520 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f92d4004fd0 tx=0x7f92d40d5140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfe66fed03f0493 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.420+0000 7f92d8c00700 1 -- 172.21.15.67:0/4085794846 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40a2490 con 0x55fde2e497a0 2024-03-30T02:37:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.420+0000 7f92d8c00700 1 -- 172.21.15.67:0/4085794846 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92d40cae90 con 0x55fde2e497a0 2024-03-30T02:37:59.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.420+0000 7f92d8c00700 1 -- 172.21.15.67:0/4085794846 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4099f90 con 0x55fde2e497a0 2024-03-30T02:37:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.421+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/4085794846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e497a0 msgr2=0x55fde2e51520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.421+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/4085794846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e497a0 0x55fde2e51520 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f92d4004fd0 tx=0x7f92d40d5140 comp rx=0 tx=0).stop 2024-03-30T02:37:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.421+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/4085794846 shutdown_connections 2024-03-30T02:37:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.421+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/4085794846 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e497a0 0x55fde2e51520 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.421+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/4085794846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e54fc0 0x55fde2e45700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.421+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/4085794846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e4a0c0 0x55fde2e5bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.421+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/4085794846 >> 172.21.15.67:0/4085794846 conn(0x55fde2da96a0 msgr2=0x55fde2e51c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.421+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/4085794846 shutdown_connections 2024-03-30T02:37:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.421+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/4085794846 wait complete. 2024-03-30T02:37:59.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.422+0000 7f92fd5d7b80 1 -- start start 2024-03-30T02:37:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.422+0000 7f92fd5d7b80 1 Processor -- start 2024-03-30T02:37:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.422+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e5b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.422+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e5b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.422+0000 7f92fd5d7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 0x55fde2e59cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.422+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2e5a3c0 con 0x55fde2e4a0c0 2024-03-30T02:37:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.422+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2e5a530 con 0x55fde2e54fc0 2024-03-30T02:37:59.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.422+0000 7f92fd5d7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fde2e5a6f0 con 0x55fde2e497a0 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.422+0000 7f92fd5d7b80 10 client.0 _ll_register_callbacks cb 0x55fde2d41470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e5b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e2a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e5b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55404/0 (socket says 172.21.15.67:55404) 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e2a00700 1 -- 172.21.15.67:0/3748215629 learned_addr learned my addr 172.21.15.67:0/3748215629 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e2000700 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e5b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e3400700 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 0x55fde2e59cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e2a00700 1 -- 172.21.15.67:0/3748215629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 msgr2=0x55fde2e59cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e2a00700 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 0x55fde2e59cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e2a00700 1 -- 172.21.15.67:0/3748215629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 msgr2=0x55fde2e5b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e2a00700 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e5b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.423+0000 7f92e2a00700 1 -- 172.21.15.67:0/3748215629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fde2bc9b60 con 0x55fde2e497a0 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.424+0000 7f92e2a00700 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e5b380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f92d4096a00 tx=0x7f92d40a2490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1d162af2c096ae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:37:59.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.424+0000 7f92ccc00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:37:59.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.424+0000 7f92ccc00700 1 -- 172.21.15.67:0/3748215629 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40b8370 con 0x55fde2e497a0 2024-03-30T02:37:59.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.424+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/3748215629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fde2d5c380 con 0x55fde2e497a0 2024-03-30T02:37:59.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.424+0000 7f92ccc00700 1 -- 172.21.15.67:0/3748215629 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f92d405fe40 con 0x55fde2e497a0 2024-03-30T02:37:59.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.424+0000 7f92ccc00700 1 -- 172.21.15.67:0/3748215629 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92d4099ad0 con 0x55fde2e497a0 2024-03-30T02:37:59.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.424+0000 7f92fd5d7b80 10 client.25951 fetch_fsmap learned FSMap version 29 2024-03-30T02:37:59.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.424+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/3748215629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fde2c21cd0 con 0x55fde2e497a0 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.425+0000 7f92ccc00700 1 -- 172.21.15.67:0/3748215629 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fde2d5c380 con 0x55fde2e497a0 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.425+0000 7f92ccc00700 1 -- 172.21.15.67:0/3748215629 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f92d40a1f80 con 0x55fde2e497a0 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.425+0000 7f92fd5d7b80 10 client.25951 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.425+0000 7f92fd5d7b80 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.425+0000 7f92fd5d7b80 20 client.25951 populate_metadata read hostname 'smithi067' 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.425+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fde2ec1ac0 0x55fde2ec3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.425+0000 7f92fd5d7b80 4 client.25951 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.425+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/3748215629 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fde2c1fce0 con 0x55fde2ec1ac0 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.425+0000 7f92e2000700 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fde2ec1ac0 0x55fde2ec3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92e2000700 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fde2ec1ac0 0x55fde2ec3eb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ac7fe0e94e681b server_cookie=2eebc8fbb8017ea0 in_seq=0 out_seq=0 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92ccc00700 10 client.25951 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92ccc00700 1 -- 172.21.15.67:0/3748215629 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fde2c1fce0 con 0x55fde2ec1ac0 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92ccc00700 10 client.25951 handle_command_reply: tid=0 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 client.25951 shutdown 2024-03-30T02:37:59.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/3748215629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 msgr2=0x55fde2e5b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e5b380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f92d4096a00 tx=0x7f92d40a2490 comp rx=0 tx=0).stop 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/3748215629 shutdown_connections 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fde2e54fc0 0x55fde2e59cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fde2e497a0 0x55fde2e5b380 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fde2ec1ac0 0x55fde2ec3eb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92ccc00700 0 client.25951 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 --2- 172.21.15.67:0/3748215629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fde2e4a0c0 0x55fde2e5b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/3748215629 >> 172.21.15.67:0/3748215629 conn(0x55fde2da96a0 msgr2=0x55fde2e45660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.426+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/3748215629 shutdown_connections 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.427+0000 7f92fd5d7b80 1 -- 172.21.15.67:0/3748215629 wait complete. 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.427+0000 7f92fd5d7b80 20 client.25951 trim_cache size 0 max 16384 2024-03-30T02:37:59.429 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:37:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.427+0000 7f92e3e00700 1 -- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f92c80a1320 msgr2=0x7f92c80a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.427+0000 7f92e3e00700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f92c80a1320 0x7f92c80a37e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009790 tx=0x7f92dc0096e0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.427+0000 7f92e3e00700 1 -- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ff8f0 msgr2=0x7f92e4199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:37:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.427+0000 7f92e3e00700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ff8f0 0x7f92e4199390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9c0 tx=0x7f92d400ecd0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.428+0000 7f92e3e00700 1 -- 172.21.15.67:0/69720871 shutdown_connections 2024-03-30T02:37:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.428+0000 7f92e3e00700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f92c80a1320 0x7f92c80a37e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.428+0000 7f92e3e00700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f92e40ff8f0 0x7f92e4199390 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.428+0000 7f92e3e00700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92e410d4a0 0x7f92e419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.428+0000 7f92e3e00700 1 --2- 172.21.15.67:0/69720871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e41002a0 0x7f92e41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:37:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.428+0000 7f92e3e00700 1 -- 172.21.15.67:0/69720871 >> 172.21.15.67:0/69720871 conn(0x7f92e40fb510 msgr2=0x7f92e40fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:37:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.428+0000 7f92e3e00700 1 -- 172.21.15.67:0/69720871 shutdown_connections 2024-03-30T02:37:59.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:37:59.428+0000 7f92e3e00700 1 -- 172.21.15.67:0/69720871 wait complete. 2024-03-30T02:38:00.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:38:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:38:00.110+0000 7f6b5a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:38:00.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:38:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:38:00.110+0000 7f6b5a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:38:00.413 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:38:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:38:00.110+0000 7f6b5a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:38:00.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:38:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:38:00.110+0000 7f6b5a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:38:00.414 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:38:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:38:00.110+0000 7f6b5a000700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-30T02:38:00.414 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:00 smithi067 ceph-mon[37808]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-30T02:38:00.447 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:38:00.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:38:00.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:00 smithi170 ceph-mon[78798]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-30T02:38:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.693+0000 7f41cf200700 1 -- 172.21.15.67:0/209245966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c81002a0 msgr2=0x7f41c810cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.693+0000 7f41cf200700 1 --2- 172.21.15.67:0/209245966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c81002a0 0x7f41c810cf60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f41b80098f0 tx=0x7f41b8009c00 comp rx=0 tx=0).stop 2024-03-30T02:38:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.693+0000 7f41cf200700 1 -- 172.21.15.67:0/209245966 shutdown_connections 2024-03-30T02:38:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.693+0000 7f41cf200700 1 --2- 172.21.15.67:0/209245966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c810d4a0 0x7f41c810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.693+0000 7f41cf200700 1 --2- 172.21.15.67:0/209245966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c80ff8f0 0x7f41c80ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.693+0000 7f41cf200700 1 --2- 172.21.15.67:0/209245966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c81002a0 0x7f41c810cf60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.693+0000 7f41cf200700 1 -- 172.21.15.67:0/209245966 >> 172.21.15.67:0/209245966 conn(0x7f41c80fb510 msgr2=0x7f41c80fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.693+0000 7f41cf200700 1 -- 172.21.15.67:0/209245966 shutdown_connections 2024-03-30T02:38:00.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.693+0000 7f41cf200700 1 -- 172.21.15.67:0/209245966 wait complete. 2024-03-30T02:38:00.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.694+0000 7f41cf200700 1 Processor -- start 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.694+0000 7f41cf200700 1 -- start start 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cf200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c80ff8f0 0x7f41c81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cf200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c81002a0 0x7f41c8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cf200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c810d4a0 0x7f41c819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cf200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c819e200 con 0x7f41c81002a0 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cf200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c819e370 con 0x7f41c80ff8f0 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cf200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c819e530 con 0x7f41c810d4a0 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cde00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c80ff8f0 0x7f41c81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cde00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c80ff8f0 0x7f41c81992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37420/0 (socket says 172.21.15.67:37420) 2024-03-30T02:38:00.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cde00700 1 -- 172.21.15.67:0/99278878 learned_addr learned my addr 172.21.15.67:0/99278878 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:00.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cd400700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c81002a0 0x7f41c8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41ce800700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c810d4a0 0x7f41c819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cde00700 1 -- 172.21.15.67:0/99278878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c810d4a0 msgr2=0x7f41c819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.695+0000 7f41cde00700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c810d4a0 0x7f41c819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.696+0000 7f41cde00700 1 -- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c81002a0 msgr2=0x7f41c8199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.696+0000 7f41cde00700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c81002a0 0x7f41c8199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.696+0000 7f41cde00700 1 -- 172.21.15.67:0/99278878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b8009520 con 0x7f41c80ff8f0 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.696+0000 7f41cde00700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c80ff8f0 0x7f41c81992e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e9c0 tx=0x7f41bc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.696+0000 7f41c2a00700 1 -- 172.21.15.67:0/99278878 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc004a30 con 0x7f41c80ff8f0 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.696+0000 7f41c2a00700 1 -- 172.21.15.67:0/99278878 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41bc00cc80 con 0x7f41c80ff8f0 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.696+0000 7f41c2a00700 1 -- 172.21.15.67:0/99278878 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc0054e0 con 0x7f41c80ff8f0 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.696+0000 7f41cf200700 1 -- 172.21.15.67:0/99278878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c819e810 con 0x7f41c80ff8f0 2024-03-30T02:38:00.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.697+0000 7f41cf200700 1 -- 172.21.15.67:0/99278878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c81a6050 con 0x7f41c80ff8f0 2024-03-30T02:38:00.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.702+0000 7f41c2a00700 1 -- 172.21.15.67:0/99278878 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f41bc004b90 con 0x7f41c80ff8f0 2024-03-30T02:38:00.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.703+0000 7f41e430ab80 1 Processor -- start 2024-03-30T02:38:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.703+0000 7f41c2a00700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f41b00a13a0 0x7f41b00a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.703+0000 7f41c2a00700 1 -- 172.21.15.67:0/99278878 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f41bc014070 con 0x7f41c80ff8f0 2024-03-30T02:38:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41e430ab80 1 -- start start 2024-03-30T02:38:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe30410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 0x56189fe3b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe3b830 0x56189fe3dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41e430ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fd80a10 con 0x56189fe3b830 2024-03-30T02:38:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41e430ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fd7e3d0 con 0x56189fe30950 2024-03-30T02:38:00.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41e430ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fd79cc0 con 0x56189fe30030 2024-03-30T02:38:00.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41cde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe30410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41ce800700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f41b00a13a0 0x7f41b00a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41cde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe30410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55430/0 (socket says 172.21.15.67:55430) 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41cde00700 1 -- 172.21.15.67:0/1933721722 learned_addr learned my addr 172.21.15.67:0/1933721722 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.704+0000 7f41cd400700 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 0x56189fe3b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.705+0000 7f41cde00700 1 -- 172.21.15.67:0/1933721722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 msgr2=0x56189fe3b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.705+0000 7f41cde00700 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 0x56189fe3b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.705+0000 7f41cde00700 1 -- 172.21.15.67:0/1933721722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe3b830 msgr2=0x56189fe3dc20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.705+0000 7f41ce800700 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe3b830 0x56189fe3dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.705+0000 7f41cde00700 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe3b830 0x56189fe3dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.705+0000 7f41cde00700 1 -- 172.21.15.67:0/1933721722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56189fbfb430 con 0x56189fe30030 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.705+0000 7f41ce800700 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe3b830 0x56189fe3dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.705+0000 7f41cde00700 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe30410 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00cfd0 tx=0x7f41bc099880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0ea5e267202d89 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.706+0000 7f41b7000700 1 -- 172.21.15.67:0/1933721722 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc099ea0 con 0x56189fe30030 2024-03-30T02:38:00.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.706+0000 7f41b7000700 1 -- 172.21.15.67:0/1933721722 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41bc09dcd0 con 0x56189fe30030 2024-03-30T02:38:00.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.706+0000 7f41b7000700 1 -- 172.21.15.67:0/1933721722 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc09a100 con 0x56189fe30030 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.706+0000 7f41e430ab80 1 -- 172.21.15.67:0/1933721722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 msgr2=0x56189fe30410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.706+0000 7f41ce800700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f41b00a13a0 0x7f41b00a3860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41c8068ed0 tx=0x7f41c4009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.706+0000 7f41e430ab80 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe30410 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00cfd0 tx=0x7f41bc099880 comp rx=0 tx=0).stop 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.707+0000 7f41e430ab80 1 -- 172.21.15.67:0/1933721722 shutdown_connections 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.707+0000 7f41e430ab80 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe3b830 0x56189fe3dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.707+0000 7f41e430ab80 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe30410 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.707+0000 7f41e430ab80 1 --2- 172.21.15.67:0/1933721722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 0x56189fe3b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.707+0000 7f41e430ab80 1 -- 172.21.15.67:0/1933721722 >> 172.21.15.67:0/1933721722 conn(0x56189fd87720 msgr2=0x56189fe23c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.707+0000 7f41e430ab80 1 -- 172.21.15.67:0/1933721722 shutdown_connections 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.707+0000 7f41e430ab80 1 -- 172.21.15.67:0/1933721722 wait complete. 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.707+0000 7f41e430ab80 1 -- start start 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.708+0000 7f41e430ab80 1 Processor -- start 2024-03-30T02:38:00.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.708+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe3b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.708+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe38690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.708+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe3b830 0x56189fe38d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.708+0000 7f41e430ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fd15e00 con 0x56189fe30950 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.708+0000 7f41e430ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fd15aa0 con 0x56189fe3b830 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.708+0000 7f41e430ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fbfd720 con 0x56189fe30030 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.708+0000 7f41e430ab80 10 client.0 _ll_register_callbacks cb 0x56189fd1f4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cd400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe38690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cd400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe38690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40796/0 (socket says 172.21.15.67:40796) 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cd400700 1 -- 172.21.15.67:0/3169011400 learned_addr learned my addr 172.21.15.67:0/3169011400 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41ce800700 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe3b830 0x56189fe38d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cde00700 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe3b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cd400700 1 -- 172.21.15.67:0/3169011400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 msgr2=0x56189fe3b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:00.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cd400700 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe3b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cd400700 1 -- 172.21.15.67:0/3169011400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe3b830 msgr2=0x56189fe38d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:00.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cd400700 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe3b830 0x56189fe38d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cd400700 1 -- 172.21.15.67:0/3169011400 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c81a6050 con 0x56189fe30950 2024-03-30T02:38:00.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.709+0000 7f41cde00700 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe3b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:00.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.710+0000 7f41cd400700 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe38690 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009230 tx=0x7f41b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6350ab097cd8fb5d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:00.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.710+0000 7f41ab400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:00.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.710+0000 7f41ab400700 1 -- 172.21.15.67:0/3169011400 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b800b970 con 0x56189fe30950 2024-03-30T02:38:00.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.710+0000 7f41ab400700 1 -- 172.21.15.67:0/3169011400 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f41b800bad0 con 0x56189fe30950 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.711+0000 7f41e430ab80 1 -- 172.21.15.67:0/3169011400 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56189fd3a400 con 0x56189fe30950 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.711+0000 7f41ab400700 1 -- 172.21.15.67:0/3169011400 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41b800f970 con 0x56189fe30950 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.711+0000 7f41e430ab80 10 client.15924 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.711+0000 7f41e430ab80 1 -- 172.21.15.67:0/3169011400 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56189fba7be0 con 0x56189fe30950 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.711+0000 7f41ab400700 1 -- 172.21.15.67:0/3169011400 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56189fd3a400 con 0x56189fe30950 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.712+0000 7f41ab400700 1 -- 172.21.15.67:0/3169011400 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41b8015020 con 0x56189fe30950 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.712+0000 7f41e430ab80 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.712+0000 7f41e430ab80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.712+0000 7f41e430ab80 20 client.15924 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.712+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56189fe4b3c0 0x56189fe4d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.712+0000 7f41e430ab80 4 client.15924 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.712+0000 7f41e430ab80 1 -- 172.21.15.67:0/3169011400 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56189fd72780 con 0x56189fe4b3c0 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.712+0000 7f41cde00700 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56189fe4b3c0 0x56189fe4d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.713+0000 7f41cde00700 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56189fe4b3c0 0x56189fe4d7e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c2ce60d9eb4089a server_cookie=219e00a992039347 in_seq=0 out_seq=0 2024-03-30T02:38:00.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.713+0000 7f41ab400700 10 client.15924 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.713+0000 7f41ab400700 1 -- 172.21.15.67:0/3169011400 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56189fd72780 con 0x56189fe4b3c0 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.713+0000 7f41ab400700 10 client.15924 handle_command_reply: tid=0 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.713+0000 7f41e430ab80 1 client.15924 shutdown 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 -- 172.21.15.67:0/3169011400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 msgr2=0x56189fe38690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe38690 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009230 tx=0x7f41b800f690 comp rx=0 tx=0).stop 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 -- 172.21.15.67:0/3169011400 shutdown_connections 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe3b830 0x56189fe38d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe3b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56189fe4b3c0 0x56189fe4d7e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3169011400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe38690 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 -- 172.21.15.67:0/3169011400 >> 172.21.15.67:0/3169011400 conn(0x56189fd87720 msgr2=0x56189fe32140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41ab400700 0 client.15924 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:00.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 -- 172.21.15.67:0/3169011400 shutdown_connections 2024-03-30T02:38:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 1 -- 172.21.15.67:0/3169011400 wait complete. 2024-03-30T02:38:00.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.714+0000 7f41e430ab80 20 client.15924 trim_cache size 0 max 16384 2024-03-30T02:38:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.746+0000 7f41e430ab80 1 Processor -- start 2024-03-30T02:38:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.746+0000 7f41e430ab80 1 -- start start 2024-03-30T02:38:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.746+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30030 0x56189fe424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 0x56189fe36f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe3b830 0x56189fe374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41e430ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fe399e0 con 0x56189fe30030 2024-03-30T02:38:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41e430ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fe2fea0 con 0x56189fe30950 2024-03-30T02:38:00.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41e430ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fe395e0 con 0x56189fe3b830 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cde00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30030 0x56189fe424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cde00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30030 0x56189fe424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40810/0 (socket says 172.21.15.67:40810) 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cde00700 1 -- 172.21.15.67:0/3061286637 learned_addr learned my addr 172.21.15.67:0/3061286637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cd400700 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 0x56189fe36f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41ce800700 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe3b830 0x56189fe374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cd400700 1 -- 172.21.15.67:0/3061286637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe3b830 msgr2=0x56189fe374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cd400700 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe3b830 0x56189fe374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cd400700 1 -- 172.21.15.67:0/3061286637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30030 msgr2=0x56189fe424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cd400700 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30030 0x56189fe424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cd400700 1 -- 172.21.15.67:0/3061286637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56189fd3ff20 con 0x56189fe30950 2024-03-30T02:38:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41ce800700 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe3b830 0x56189fe374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:38:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.747+0000 7f41cde00700 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30030 0x56189fe424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.748+0000 7f41cd400700 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 0x56189fe36f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009810 tx=0x7f41b8009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed350ae035e98a33 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.748+0000 7f41b7a00700 1 -- 172.21.15.67:0/3061286637 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8003880 con 0x56189fe30950 2024-03-30T02:38:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.748+0000 7f41b7a00700 1 -- 172.21.15.67:0/3061286637 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41b801d070 con 0x56189fe30950 2024-03-30T02:38:00.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.748+0000 7f41b7a00700 1 -- 172.21.15.67:0/3061286637 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b80053d0 con 0x56189fe30950 2024-03-30T02:38:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:00 smithi082 ceph-mon[79176]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-03-30T02:38:00.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.749+0000 7f41e430ab80 1 -- 172.21.15.67:0/3061286637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 msgr2=0x56189fe36f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:00.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.749+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 0x56189fe36f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009810 tx=0x7f41b8009fd0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.749+0000 7f41e430ab80 1 -- 172.21.15.67:0/3061286637 shutdown_connections 2024-03-30T02:38:00.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.749+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30030 0x56189fe424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.749+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe3b830 0x56189fe374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.749+0000 7f41e430ab80 1 --2- 172.21.15.67:0/3061286637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe30950 0x56189fe36f80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.749+0000 7f41e430ab80 1 -- 172.21.15.67:0/3061286637 >> 172.21.15.67:0/3061286637 conn(0x56189fd87720 msgr2=0x56189fe339d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:00.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.749+0000 7f41e430ab80 1 -- 172.21.15.67:0/3061286637 shutdown_connections 2024-03-30T02:38:00.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.749+0000 7f41e430ab80 1 -- 172.21.15.67:0/3061286637 wait complete. 2024-03-30T02:38:00.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.750+0000 7f41e430ab80 1 -- start start 2024-03-30T02:38:00.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.750+0000 7f41e430ab80 1 Processor -- start 2024-03-30T02:38:00.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe40ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41e430ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe3b830 0x56189fe40ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41e430ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fe41700 con 0x56189fe30950 2024-03-30T02:38:00.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41e430ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fe41870 con 0x56189fe3b830 2024-03-30T02:38:00.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41e430ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56189fe9f220 con 0x56189fe30030 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41e430ab80 10 client.0 _ll_register_callbacks cb 0x56189fd1f4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41cde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41cde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe424a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55450/0 (socket says 172.21.15.67:55450) 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.751+0000 7f41cde00700 1 -- 172.21.15.67:0/366197690 learned_addr learned my addr 172.21.15.67:0/366197690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.752+0000 7f41cd400700 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe40ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.752+0000 7f41cd400700 1 -- 172.21.15.67:0/366197690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 msgr2=0x56189fe424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.752+0000 7f41cd400700 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe424a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.752+0000 7f41cd400700 1 -- 172.21.15.67:0/366197690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe3b830 msgr2=0x56189fe40ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.752+0000 7f41cd400700 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe3b830 0x56189fe40ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.752+0000 7f41cd400700 1 -- 172.21.15.67:0/366197690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56189fbfb430 con 0x56189fe30950 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.752+0000 7f41cde00700 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.753+0000 7f41cd400700 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe40ab0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x56189fe24bd0 tx=0x7f41b800bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc094095b6bc8d7e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:00.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.753+0000 7f41ab400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.753+0000 7f41ab400700 1 -- 172.21.15.67:0/366197690 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8003ea0 con 0x56189fe30950 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.753+0000 7f41e430ab80 1 -- 172.21.15.67:0/366197690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41bc09dcd0 con 0x56189fe30950 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.753+0000 7f41ab400700 1 -- 172.21.15.67:0/366197690 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f41b8000e40 con 0x56189fe30950 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.753+0000 7f41ab400700 1 -- 172.21.15.67:0/366197690 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41b800b770 con 0x56189fe30950 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.753+0000 7f41e430ab80 10 client.15930 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.753+0000 7f41e430ab80 1 -- 172.21.15.67:0/366197690 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56189fba7be0 con 0x56189fe30950 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.754+0000 7f41ab400700 1 -- 172.21.15.67:0/366197690 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41bc09dcd0 con 0x56189fe30950 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.754+0000 7f41ab400700 1 -- 172.21.15.67:0/366197690 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41b8015040 con 0x56189fe30950 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.754+0000 7f41e430ab80 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:00.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.754+0000 7f41e430ab80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.754+0000 7f41e430ab80 20 client.15930 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.754+0000 7f41e430ab80 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56189fea0360 0x56189fea2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.754+0000 7f41e430ab80 4 client.15930 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:38:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.754+0000 7f41e430ab80 1 -- 172.21.15.67:0/366197690 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56189fbc8d70 con 0x56189fea0360 2024-03-30T02:38:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.755+0000 7f41cde00700 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56189fea0360 0x56189fea2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.755+0000 7f41ab400700 10 client.15930 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:00.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.755+0000 7f41cde00700 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56189fea0360 0x56189fea2750 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe411800ce2436c server_cookie=eaa95c497f320c96 in_seq=0 out_seq=0 2024-03-30T02:38:00.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.756+0000 7f41ab400700 1 -- 172.21.15.67:0/366197690 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56189fbc8d70 con 0x56189fea0360 2024-03-30T02:38:00.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.756+0000 7f41ab400700 10 client.15930 handle_command_reply: tid=0 2024-03-30T02:38:00.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.756+0000 7f41e430ab80 1 client.15930 shutdown 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 -- 172.21.15.67:0/366197690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 msgr2=0x56189fe40ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe40ab0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x56189fe24bd0 tx=0x7f41b800bf70 comp rx=0 tx=0).stop 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 -- 172.21.15.67:0/366197690 shutdown_connections 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56189fe3b830 0x56189fe40ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56189fe30030 0x56189fe424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56189fea0360 0x56189fea2750 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 --2- 172.21.15.67:0/366197690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56189fe30950 0x56189fe40ab0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 -- 172.21.15.67:0/366197690 >> 172.21.15.67:0/366197690 conn(0x56189fd87720 msgr2=0x56189fe36f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41ab400700 0 client.15930 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 -- 172.21.15.67:0/366197690 shutdown_connections 2024-03-30T02:38:00.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 1 -- 172.21.15.67:0/366197690 wait complete. 2024-03-30T02:38:00.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.757+0000 7f41e430ab80 20 client.15930 trim_cache size 0 max 16384 2024-03-30T02:38:00.759 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:38:00.759 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:38:00.759 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "cfcfeede-0437-4cb0-bc99-55ac6a5d63e9", 2024-03-30T02:38:00.759 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:38:00.759 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:38:00.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 -- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f41b00a13a0 msgr2=0x7f41b00a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:00.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f41b00a13a0 0x7f41b00a3860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41c8068ed0 tx=0x7f41c4009920 comp rx=0 tx=0).stop 2024-03-30T02:38:00.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 -- 172.21.15.67:0/99278878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c80ff8f0 msgr2=0x7f41c81992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:00.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c80ff8f0 0x7f41c81992e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e9c0 tx=0x7f41bc00ecd0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 -- 172.21.15.67:0/99278878 shutdown_connections 2024-03-30T02:38:00.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f41b00a13a0 0x7f41b00a3860 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41c810d4a0 0x7f41c819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41c80ff8f0 0x7f41c81992e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 --2- 172.21.15.67:0/99278878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c81002a0 0x7f41c8199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:00.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.758+0000 7f41cf200700 1 -- 172.21.15.67:0/99278878 >> 172.21.15.67:0/99278878 conn(0x7f41c80fb510 msgr2=0x7f41c80fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:00.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.759+0000 7f41cf200700 1 -- 172.21.15.67:0/99278878 shutdown_connections 2024-03-30T02:38:00.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:00.759+0000 7f41cf200700 1 -- 172.21.15.67:0/99278878 wait complete. 2024-03-30T02:38:00.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:38:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.018+0000 7f6ca9e00700 1 -- 172.21.15.67:0/2601141251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca4101d90 msgr2=0x7f6ca4102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.018+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/2601141251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca4101d90 0x7f6ca4102170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009990 tx=0x7f6c98009ca0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.019+0000 7f6ca9e00700 1 -- 172.21.15.67:0/2601141251 shutdown_connections 2024-03-30T02:38:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.019+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/2601141251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca4101d90 0x7f6ca4102170 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.019+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/2601141251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca410b160 0x7f6ca410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.019+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/2601141251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ca41026b0 0x7f6ca410ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.019+0000 7f6ca9e00700 1 -- 172.21.15.67:0/2601141251 >> 172.21.15.67:0/2601141251 conn(0x7f6ca40fb510 msgr2=0x7f6ca40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:01.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.019+0000 7f6ca9e00700 1 -- 172.21.15.67:0/2601141251 shutdown_connections 2024-03-30T02:38:01.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.019+0000 7f6ca9e00700 1 -- 172.21.15.67:0/2601141251 wait complete. 2024-03-30T02:38:01.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.020+0000 7f6ca9e00700 1 Processor -- start 2024-03-30T02:38:01.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.021+0000 7f6ca9e00700 1 -- start start 2024-03-30T02:38:01.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.021+0000 7f6ca9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101d90 0x7f6ca4109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.021+0000 7f6ca9e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ca41026b0 0x7f6ca410a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.021+0000 7f6ca9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca410b160 0x7f6ca41a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.021+0000 7f6ca9e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41a2580 con 0x7f6ca410b160 2024-03-30T02:38:01.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.021+0000 7f6ca9e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41a26f0 con 0x7f6ca4101d90 2024-03-30T02:38:01.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.021+0000 7f6ca9e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41a28b0 con 0x7f6ca41026b0 2024-03-30T02:38:01.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.021+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ca41026b0 0x7f6ca410a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.021+0000 7f6ca8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101d90 0x7f6ca4109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ca41026b0 0x7f6ca410a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55466/0 (socket says 172.21.15.67:55466) 2024-03-30T02:38:01.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca3e00700 1 -- 172.21.15.67:0/4229217644 learned_addr learned my addr 172.21.15.67:0/4229217644 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:01.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca9400700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca410b160 0x7f6ca41a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101d90 0x7f6ca4109ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37458/0 (socket says 172.21.15.67:37458) 2024-03-30T02:38:01.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca3e00700 1 -- 172.21.15.67:0/4229217644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101d90 msgr2=0x7f6ca4109ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:01.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101d90 0x7f6ca4109ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca3e00700 1 -- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca410b160 msgr2=0x7f6ca41a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:01.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca410b160 0x7f6ca41a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca3e00700 1 -- 172.21.15.67:0/4229217644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c980095f0 con 0x7f6ca41026b0 2024-03-30T02:38:01.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.022+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ca41026b0 0x7f6ca410a420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000e9a0 tx=0x7f6c9000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:01.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.023+0000 7f6ca1600700 1 -- 172.21.15.67:0/4229217644 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90009790 con 0x7f6ca41026b0 2024-03-30T02:38:01.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.023+0000 7f6ca9e00700 1 -- 172.21.15.67:0/4229217644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca41a2b90 con 0x7f6ca41026b0 2024-03-30T02:38:01.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.023+0000 7f6ca9e00700 1 -- 172.21.15.67:0/4229217644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca41aa400 con 0x7f6ca41026b0 2024-03-30T02:38:01.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.023+0000 7f6ca1600700 1 -- 172.21.15.67:0/4229217644 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9000cbd0 con 0x7f6ca41026b0 2024-03-30T02:38:01.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.023+0000 7f6ca1600700 1 -- 172.21.15.67:0/4229217644 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90018680 con 0x7f6ca41026b0 2024-03-30T02:38:01.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.028+0000 7f6cbefe8b80 1 Processor -- start 2024-03-30T02:38:01.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.028+0000 7f6ca1600700 1 -- 172.21.15.67:0/4229217644 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 26) v1 ==== 136041+0+0 (secure 0 0 0) 0x7f6c900187e0 con 0x7f6ca41026b0 2024-03-30T02:38:01.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.028+0000 7f6cbefe8b80 1 -- start start 2024-03-30T02:38:01.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489b30c0 0x555c489bda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489bdfc0 0x555c489c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c4890b990 con 0x555c489b27a0 2024-03-30T02:38:01.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c48909350 con 0x555c489bdfc0 2024-03-30T02:38:01.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c48904c40 con 0x555c489b30c0 2024-03-30T02:38:01.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489bdfc0 0x555c489c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489bdfc0 0x555c489c03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37470/0 (socket says 172.21.15.67:37470) 2024-03-30T02:38:01.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6ca3e00700 1 -- 172.21.15.67:0/1353688628 learned_addr learned my addr 172.21.15.67:0/1353688628 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:01.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.029+0000 7f6ca9400700 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489b30c0 0x555c489bda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca3e00700 1 -- 172.21.15.67:0/1353688628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489b30c0 msgr2=0x555c489bda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:01.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489b30c0 0x555c489bda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca3e00700 1 -- 172.21.15.67:0/1353688628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 msgr2=0x555c489b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:01.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca3e00700 1 -- 172.21.15.67:0/1353688628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c487863b0 con 0x555c489bdfc0 2024-03-30T02:38:01.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489b30c0 0x555c489bda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca1600700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c8c0a13a0 0x7f6c8c0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca1600700 1 -- 172.21.15.67:0/4229217644 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f6c90014070 con 0x7f6ca41026b0 2024-03-30T02:38:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489bdfc0 0x555c489c03b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6c900ca620 tx=0x7f6c900d5440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b91cc957bd5d0682 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.030+0000 7f6ca9400700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c8c0a13a0 0x7f6c8c0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.031+0000 7f6c95c00700 1 -- 172.21.15.67:0/1353688628 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90096300 con 0x555c489bdfc0 2024-03-30T02:38:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.031+0000 7f6c95c00700 1 -- 172.21.15.67:0/1353688628 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c900d0ce0 con 0x555c489bdfc0 2024-03-30T02:38:01.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.031+0000 7f6c95c00700 1 -- 172.21.15.67:0/1353688628 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90099fc0 con 0x555c489bdfc0 2024-03-30T02:38:01.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.031+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/1353688628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489bdfc0 msgr2=0x555c489c03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:01.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.031+0000 7f6ca9400700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c8c0a13a0 0x7f6c8c0a3860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0127f0 tx=0x7f6c9c00b040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.031+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489bdfc0 0x555c489c03b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6c900ca620 tx=0x7f6c900d5440 comp rx=0 tx=0).stop 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.032+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/1353688628 shutdown_connections 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.032+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.032+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489bdfc0 0x555c489c03b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.032+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/1353688628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489b30c0 0x555c489bda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.032+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/1353688628 >> 172.21.15.67:0/1353688628 conn(0x555c489126a0 msgr2=0x555c489aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.032+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/1353688628 shutdown_connections 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.032+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/1353688628 wait complete. 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.032+0000 7f6cbefe8b80 1 -- start start 2024-03-30T02:38:01.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.033+0000 7f6cbefe8b80 1 Processor -- start 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489bd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489babd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489bb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c488a0d80 con 0x555c489b27a0 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c488a0a20 con 0x555c489b30c0 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c487886a0 con 0x555c489bdfc0 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6cbefe8b80 10 client.0 _ll_register_callbacks cb 0x555c488aa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6ca3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489babd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37480/0 (socket says 172.21.15.67:37480) 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6ca3e00700 1 -- 172.21.15.67:0/704712332 learned_addr learned my addr 172.21.15.67:0/704712332 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:01.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6ca9400700 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489bb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489bd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.034+0000 7f6ca3e00700 1 -- 172.21.15.67:0/704712332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 msgr2=0x555c489bb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489bb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6ca3e00700 1 -- 172.21.15.67:0/704712332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 msgr2=0x555c489bd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489bd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6ca3e00700 1 -- 172.21.15.67:0/704712332 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca41aa400 con 0x555c489b30c0 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489bd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489babd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c900a1f10 tx=0x7f6c900cd450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329a9884bcbcdca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6c8a000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6c8a000700 1 -- 172.21.15.67:0/704712332 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90096300 con 0x555c489b30c0 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6c8a000700 1 -- 172.21.15.67:0/704712332 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f6c9009a6c0 con 0x555c489b30c0 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/704712332 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c488fd700 con 0x555c489b30c0 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6c8a000700 1 -- 172.21.15.67:0/704712332 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c90099d90 con 0x555c489b30c0 2024-03-30T02:38:01.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.035+0000 7f6cbefe8b80 10 client.25994 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.036+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/704712332 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555c48732b60 con 0x555c489b30c0 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.036+0000 7f6c8a000700 1 -- 172.21.15.67:0/704712332 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c488fd700 con 0x555c489b30c0 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.036+0000 7f6c8a000700 1 -- 172.21.15.67:0/704712332 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c900cd030 con 0x555c489b30c0 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.036+0000 7f6cbefe8b80 10 client.25994 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.036+0000 7f6cbefe8b80 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.036+0000 7f6cbefe8b80 20 client.25994 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.036+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555c489cd840 0x555c489cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.036+0000 7f6cbefe8b80 4 client.25994 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.036+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/704712332 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c48753cf0 con 0x555c489cd840 2024-03-30T02:38:01.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.037+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555c489cd840 0x555c489cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.037+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555c489cd840 0x555c489cfc60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad189a9120379fa8 server_cookie=15d133685bcb44e in_seq=0 out_seq=0 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.037+0000 7f6c8a000700 10 client.25994 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.038+0000 7f6c8a000700 1 -- 172.21.15.67:0/704712332 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c48753cf0 con 0x555c489cd840 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.038+0000 7f6c8a000700 10 client.25994 handle_command_reply: tid=0 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.038+0000 7f6cbefe8b80 1 client.25994 shutdown 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.038+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/704712332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 msgr2=0x555c489babd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.038+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489babd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c900a1f10 tx=0x7f6c900cd450 comp rx=0 tx=0).stop 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/704712332 shutdown_connections 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555c489cd840 0x555c489cfc60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489bd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489bb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/704712332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489babd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6c8a000700 0 client.25994 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:01.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/704712332 >> 172.21.15.67:0/704712332 conn(0x555c489126a0 msgr2=0x555c489b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:01.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/704712332 shutdown_connections 2024-03-30T02:38:01.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/704712332 wait complete. 2024-03-30T02:38:01.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.039+0000 7f6cbefe8b80 20 client.25994 trim_cache size 0 max 16384 2024-03-30T02:38:01.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.072+0000 7f6cbefe8b80 1 Processor -- start 2024-03-30T02:38:01.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.072+0000 7f6cbefe8b80 1 -- start start 2024-03-30T02:38:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.072+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b27a0 0x555c489c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b30c0 0x555c489ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c489c5210 con 0x555c489b30c0 2024-03-30T02:38:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c489d02c0 con 0x555c489b27a0 2024-03-30T02:38:01.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c489bba80 con 0x555c489bdfc0 2024-03-30T02:38:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6ca8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b27a0 0x555c489c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6ca8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b27a0 0x555c489c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:37494/0 (socket says 172.21.15.67:37494) 2024-03-30T02:38:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6ca8a00700 1 -- 172.21.15.67:0/2977348577 learned_addr learned my addr 172.21.15.67:0/2977348577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/2977348577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b30c0 0x555c489ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.073+0000 7f6ca8a00700 1 -- 172.21.15.67:0/2977348577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 msgr2=0x555c489aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.074+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/2977348577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.074+0000 7f6ca8a00700 1 -- 172.21.15.67:0/2977348577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b30c0 msgr2=0x555c489ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.074+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/2977348577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b30c0 0x555c489ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.074+0000 7f6ca8a00700 1 -- 172.21.15.67:0/2977348577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c488caea0 con 0x555c489b27a0 2024-03-30T02:38:01.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.074+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/2977348577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b27a0 0x555c489c4cd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009990 tx=0x7f6c98005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=404d3c7a1c4940 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.074+0000 7f6c96600700 1 -- 172.21.15.67:0/2977348577 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98005400 con 0x555c489b27a0 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.075+0000 7f6c96600700 1 -- 172.21.15.67:0/2977348577 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9801e040 con 0x555c489b27a0 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.075+0000 7f6c96600700 1 -- 172.21.15.67:0/2977348577 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800b750 con 0x555c489b27a0 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2977348577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b27a0 msgr2=0x555c489c4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2977348577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b27a0 0x555c489c4cd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009990 tx=0x7f6c98005c00 comp rx=0 tx=0).stop 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2977348577 shutdown_connections 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2977348577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2977348577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b27a0 0x555c489c4cd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2977348577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b30c0 0x555c489ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2977348577 >> 172.21.15.67:0/2977348577 conn(0x555c489126a0 msgr2=0x555c489b94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2977348577 shutdown_connections 2024-03-30T02:38:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2977348577 wait complete. 2024-03-30T02:38:01.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.076+0000 7f6cbefe8b80 1 -- start start 2024-03-30T02:38:01.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.077+0000 7f6cbefe8b80 1 Processor -- start 2024-03-30T02:38:01.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.077+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.077+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6cbefe8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c489c3c80 con 0x555c489b27a0 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c489c3df0 con 0x555c489b30c0 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6cbefe8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c48a21640 con 0x555c489bdfc0 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6cbefe8b80 10 client.0 _ll_register_callbacks cb 0x555c488aa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40868/0 (socket says 172.21.15.67:40868) 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca8a00700 1 -- 172.21.15.67:0/2743145719 learned_addr learned my addr 172.21.15.67:0/2743145719 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca9400700 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca8a00700 1 -- 172.21.15.67:0/2743145719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 msgr2=0x555c489c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:01.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca8a00700 1 -- 172.21.15.67:0/2743145719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 msgr2=0x555c489c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca8a00700 1 -- 172.21.15.67:0/2743145719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca410e290 con 0x555c489b27a0 2024-03-30T02:38:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca9400700 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.078+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.079+0000 7f6ca8a00700 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489c4b70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6c980098e0 tx=0x7f6c98005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17af09396a8be8f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:01.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.079+0000 7f6c8a000700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:01.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.079+0000 7f6c8a000700 1 -- 172.21.15.67:0/2743145719 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c980282a0 con 0x555c489b27a0 2024-03-30T02:38:01.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.079+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2743145719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c488c5380 con 0x555c489b27a0 2024-03-30T02:38:01.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.079+0000 7f6c8a000700 1 -- 172.21.15.67:0/2743145719 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f6c98028400 con 0x555c489b27a0 2024-03-30T02:38:01.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6c8a000700 1 -- 172.21.15.67:0/2743145719 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c980285c0 con 0x555c489b27a0 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6cbefe8b80 10 client.15948 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2743145719 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555c4878acd0 con 0x555c489b27a0 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6c8a000700 1 -- 172.21.15.67:0/2743145719 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c488c5380 con 0x555c489b27a0 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6c8a000700 1 -- 172.21.15.67:0/2743145719 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c98004500 con 0x555c489b27a0 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6cbefe8b80 10 client.15948 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6cbefe8b80 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6cbefe8b80 20 client.15948 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555c48a22a70 0x555c48a24e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6cbefe8b80 4 client.15948 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.080+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2743145719 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555c48788ce0 con 0x555c48a22a70 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.081+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555c48a22a70 0x555c48a24e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:01.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.081+0000 7f6ca3e00700 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555c48a22a70 0x555c48a24e90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e1cbc5548dbe52 server_cookie=8e2700ec8cc1a738 in_seq=0 out_seq=0 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.081+0000 7f6c8a000700 10 client.15948 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6c8a000700 1 -- 172.21.15.67:0/2743145719 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555c48788ce0 con 0x555c48a22a70 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6c8a000700 10 client.15948 handle_command_reply: tid=0 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 client.15948 shutdown 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2743145719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 msgr2=0x555c489c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489c4b70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6c980098e0 tx=0x7f6c98005a80 comp rx=0 tx=0).stop 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2743145719 shutdown_connections 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555c48a22a70 0x555c48a24e90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555c489b27a0 0x555c489c4b70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c489bdfc0 0x555c489c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 --2- 172.21.15.67:0/2743145719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555c489b30c0 0x555c489c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2743145719 >> 172.21.15.67:0/2743145719 conn(0x555c489126a0 msgr2=0x555c489ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:01.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6c8a000700 0 client.15948 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:01.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2743145719 shutdown_connections 2024-03-30T02:38:01.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 1 -- 172.21.15.67:0/2743145719 wait complete. 2024-03-30T02:38:01.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.082+0000 7f6cbefe8b80 20 client.15948 trim_cache size 0 max 16384 2024-03-30T02:38:01.085 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:38:01.085 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T02:38:01.085 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:38:01.085 INFO:teuthology.orchestra.run.smithi067.stdout: "cfcfeede-0437-4cb0-bc99-55ac6a5d63e9": { 2024-03-30T02:38:01.085 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:38:01.085 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "cfcfeede-0437-4cb0-bc99-55ac6a5d63e9", 2024-03-30T02:38:01.086 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:38:01.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:38:01.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:38:01.086 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:38:01.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 -- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c8c0a13a0 msgr2=0x7f6c8c0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c8c0a13a0 0x7f6c8c0a3860 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0127f0 tx=0x7f6c9c00b040 comp rx=0 tx=0).stop 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 -- 172.21.15.67:0/4229217644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ca41026b0 msgr2=0x7f6ca410a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ca41026b0 0x7f6ca410a420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000e9a0 tx=0x7f6c9000ecb0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 -- 172.21.15.67:0/4229217644 shutdown_connections 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c8c0a13a0 0x7f6c8c0a3860 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ca410b160 0x7f6ca41a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca4101d90 0x7f6ca4109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 --2- 172.21.15.67:0/4229217644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ca41026b0 0x7f6ca410a420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.083+0000 7f6ca9e00700 1 -- 172.21.15.67:0/4229217644 >> 172.21.15.67:0/4229217644 conn(0x7f6ca40fb510 msgr2=0x7f6ca4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.084+0000 7f6ca9e00700 1 -- 172.21.15.67:0/4229217644 shutdown_connections 2024-03-30T02:38:01.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:01.084+0000 7f6ca9e00700 1 -- 172.21.15.67:0/4229217644 wait complete. 2024-03-30T02:38:01.105 INFO:tasks.cephfs.filesystem:scrub status for tag:cfcfeede-0437-4cb0-bc99-55ac6a5d63e9 - {'path': '/', 'tag': 'cfcfeede-0437-4cb0-bc99-55ac6a5d63e9', 'options': 'recursive,force'} 2024-03-30T02:38:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:02 smithi067 ceph-mon[37808]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:02 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:38:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:38:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:02 smithi067 ceph-mon[37808]: mgrmap e27: x(active, since 58s), standbys: z, y 2024-03-30T02:38:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:38:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:02 smithi170 ceph-mon[78798]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:02 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:38:02.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:02.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:38:02.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:02.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:02 smithi170 ceph-mon[78798]: mgrmap e27: x(active, since 58s), standbys: z, y 2024-03-30T02:38:02.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:38:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:02 smithi082 ceph-mon[79176]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:02 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:38:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:02.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:38:02.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:02.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:02 smithi082 ceph-mon[79176]: mgrmap e27: x(active, since 58s), standbys: z, y 2024-03-30T02:38:02.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:38:04.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:04 smithi067 ceph-mon[37808]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:04 smithi170 ceph-mon[78798]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:04.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:04 smithi082 ceph-mon[79176]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:06 smithi067 ceph-mon[37808]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-30T02:38:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:06 smithi170 ceph-mon[78798]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-30T02:38:06.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:06 smithi082 ceph-mon[79176]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-30T02:38:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:08 smithi067 ceph-mon[37808]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-30T02:38:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:08 smithi170 ceph-mon[78798]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-30T02:38:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:08 smithi082 ceph-mon[79176]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-30T02:38:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:09 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:38:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:09 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:38:09.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:09 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:38:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:10 smithi067 ceph-mon[37808]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-30T02:38:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:10 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:38:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:10 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:38:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:10 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:38:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:10 smithi170 ceph-mon[78798]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-30T02:38:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:10 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:38:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:10 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:38:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:10 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:38:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:10 smithi082 ceph-mon[79176]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-30T02:38:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:10 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:38:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:10 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:38:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:10 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:38:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:12 smithi067 ceph-mon[37808]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-30T02:38:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:12 smithi170 ceph-mon[78798]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-30T02:38:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:12 smithi082 ceph-mon[79176]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-30T02:38:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:14 smithi067 ceph-mon[37808]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-30T02:38:14.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:14 smithi170 ceph-mon[78798]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-30T02:38:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:14 smithi082 ceph-mon[79176]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-30T02:38:16.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:16 smithi067 ceph-mon[37808]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-30T02:38:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:16 smithi170 ceph-mon[78798]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-30T02:38:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:16 smithi082 ceph-mon[79176]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-03-30T02:38:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:18 smithi067 ceph-mon[37808]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-30T02:38:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:18 smithi170 ceph-mon[78798]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-30T02:38:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:18 smithi082 ceph-mon[79176]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-03-30T02:38:20.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:20 smithi067 ceph-mon[37808]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-30T02:38:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:20 smithi170 ceph-mon[78798]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-30T02:38:20.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:20 smithi082 ceph-mon[79176]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-03-30T02:38:22.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:22 smithi067 ceph-mon[37808]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:22 smithi170 ceph-mon[78798]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:22 smithi082 ceph-mon[79176]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:24 smithi067 ceph-mon[37808]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:24.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:24 smithi170 ceph-mon[78798]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:24.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:24 smithi082 ceph-mon[79176]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-30T02:38:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:26 smithi067 ceph-mon[37808]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-30T02:38:26.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:26 smithi170 ceph-mon[78798]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-30T02:38:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:26 smithi082 ceph-mon[79176]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-30T02:38:28.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:28 smithi067 ceph-mon[37808]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-30T02:38:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:28 smithi170 ceph-mon[78798]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-30T02:38:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:28 smithi082 ceph-mon[79176]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-30T02:38:29.781 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 61% (7591/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-30T02:38:30.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:30 smithi067 ceph-mon[37808]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-30T02:38:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:30 smithi170 ceph-mon[78798]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-30T02:38:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:30 smithi082 ceph-mon[79176]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-30T02:38:30.953 INFO:tasks.workunit.client.0.smithi067.stderr:Note: switching to '470100a4ccbcf7769052bf09a5413881ccf1b97f'. 2024-03-30T02:38:30.953 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:30.953 INFO:tasks.workunit.client.0.smithi067.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-30T02:38:30.953 INFO:tasks.workunit.client.0.smithi067.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-30T02:38:30.953 INFO:tasks.workunit.client.0.smithi067.stderr:state without impacting any branches by switching back to a branch. 2024-03-30T02:38:30.953 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:30.953 INFO:tasks.workunit.client.0.smithi067.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr: git switch -c 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr:Or undo this operation with: 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr: git switch - 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:30.954 INFO:tasks.workunit.client.0.smithi067.stderr:HEAD is now at 470100a4ccb DNM: trigger builds again 2024-03-30T02:38:30.969 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-30T02:38:31.017 INFO:tasks.workunit.client.0.smithi067.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-30T02:38:31.022 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-30T02:38:31.022 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-30T02:38:31.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:38:31.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.312+0000 7f9b25200700 1 -- 172.21.15.67:0/439571578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b20069260 msgr2=0x7f9b20069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.312+0000 7f9b25200700 1 --2- 172.21.15.67:0/439571578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b20069260 0x7f9b20069640 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9b100099c0 tx=0x7f9b10009cd0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.312+0000 7f9b25200700 1 -- 172.21.15.67:0/439571578 shutdown_connections 2024-03-30T02:38:31.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.312+0000 7f9b25200700 1 --2- 172.21.15.67:0/439571578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b20069260 0x7f9b20069640 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.312+0000 7f9b25200700 1 --2- 172.21.15.67:0/439571578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b201060b0 0x7f9b2010dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.312+0000 7f9b25200700 1 --2- 172.21.15.67:0/439571578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b20069c10 0x7f9b20105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.312+0000 7f9b25200700 1 -- 172.21.15.67:0/439571578 >> 172.21.15.67:0/439571578 conn(0x7f9b20076d80 msgr2=0x7f9b20077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.312+0000 7f9b25200700 1 -- 172.21.15.67:0/439571578 shutdown_connections 2024-03-30T02:38:31.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.312+0000 7f9b25200700 1 -- 172.21.15.67:0/439571578 wait complete. 2024-03-30T02:38:31.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.313+0000 7f9b25200700 1 Processor -- start 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.314+0000 7f9b25200700 1 -- start start 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.314+0000 7f9b25200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b20069260 0x7f9b20197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.314+0000 7f9b25200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201060b0 0x7f9b20197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.314+0000 7f9b25200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2019bae0 0x7f9b2019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.314+0000 7f9b25200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2019c560 con 0x7f9b201060b0 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.314+0000 7f9b25200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2019c6d0 con 0x7f9b2019bae0 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.314+0000 7f9b25200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2019c890 con 0x7f9b20069260 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201060b0 0x7f9b20197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b20069260 0x7f9b20197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b20069260 0x7f9b20197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60552/0 (socket says 172.21.15.67:60552) 2024-03-30T02:38:31.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201060b0 0x7f9b20197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50202/0 (socket says 172.21.15.67:50202) 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1de00700 1 -- 172.21.15.67:0/1096957501 learned_addr learned my addr 172.21.15.67:0/1096957501 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1e800700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2019bae0 0x7f9b2019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1de00700 1 -- 172.21.15.67:0/1096957501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2019bae0 msgr2=0x7f9b2019bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1de00700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2019bae0 0x7f9b2019bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1de00700 1 -- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201060b0 msgr2=0x7f9b20197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1de00700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201060b0 0x7f9b20197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1de00700 1 -- 172.21.15.67:0/1096957501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100095f0 con 0x7f9b20069260 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1d400700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201060b0 0x7f9b20197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b1de00700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b20069260 0x7f9b20197200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000b580 tx=0x7f9b1000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.315+0000 7f9b16a00700 1 -- 172.21.15.67:0/1096957501 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1001c070 con 0x7f9b20069260 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.316+0000 7f9b16a00700 1 -- 172.21.15.67:0/1096957501 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b1000fd40 con 0x7f9b20069260 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.316+0000 7f9b25200700 1 -- 172.21.15.67:0/1096957501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b201a3d90 con 0x7f9b20069260 2024-03-30T02:38:31.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.316+0000 7f9b16a00700 1 -- 172.21.15.67:0/1096957501 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10022b80 con 0x7f9b20069260 2024-03-30T02:38:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.316+0000 7f9b25200700 1 -- 172.21.15.67:0/1096957501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b201a41c0 con 0x7f9b20069260 2024-03-30T02:38:31.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.319+0000 7f9b16a00700 1 -- 172.21.15.67:0/1096957501 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f9b10020070 con 0x7f9b20069260 2024-03-30T02:38:31.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.320+0000 7f9b38d8eb80 1 Processor -- start 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.320+0000 7f9b38d8eb80 1 -- start start 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.320+0000 7f9b16a00700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9b040a1430 0x7f9b040a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.320+0000 7f9b16a00700 1 -- 172.21.15.67:0/1096957501 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9b100cdff0 con 0x7f9b20069260 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b7c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 0x555583b878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b8a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583ad5990 con 0x555583b7c5d0 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583ad3350 con 0x555583b87df0 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583acec40 con 0x555583b7cef0 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1e800700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9b040a1430 0x7f9b040a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b7c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b7c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50210/0 (socket says 172.21.15.67:50210) 2024-03-30T02:38:31.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1de00700 1 -- 172.21.15.67:0/3769731869 learned_addr learned my addr 172.21.15.67:0/3769731869 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1d400700 1 --2- 172.21.15.67:0/3769731869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 0x555583b878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1e800700 1 --2- 172.21.15.67:0/3769731869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b8a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1de00700 1 -- 172.21.15.67:0/3769731869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 msgr2=0x555583b878b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1de00700 1 --2- 172.21.15.67:0/3769731869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 0x555583b878b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1de00700 1 -- 172.21.15.67:0/3769731869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 msgr2=0x555583b8a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1de00700 1 --2- 172.21.15.67:0/3769731869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b8a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.321+0000 7f9b1de00700 1 -- 172.21.15.67:0/3769731869 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555839503b0 con 0x555583b7c5d0 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.322+0000 7f9b1de00700 1 --2- 172.21.15.67:0/3769731869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b7c9b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009990 tx=0x7f9b1009a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d392b5e33b94b0a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.322+0000 7f9b1e800700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9b040a1430 0x7f9b040a38f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800f870 tx=0x7f9b1800fc30 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.322+0000 7f9b0b400700 1 -- 172.21.15.67:0/3769731869 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1009abb0 con 0x555583b7c5d0 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.322+0000 7f9b0b400700 1 -- 172.21.15.67:0/3769731869 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b1009e9e0 con 0x555583b7c5d0 2024-03-30T02:38:31.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.322+0000 7f9b0b400700 1 -- 172.21.15.67:0/3769731869 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b100a2b10 con 0x555583b7c5d0 2024-03-30T02:38:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.322+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3769731869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 msgr2=0x555583b7c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.322+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3769731869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b7c9b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9b10009990 tx=0x7f9b1009a6f0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.323+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3769731869 shutdown_connections 2024-03-30T02:38:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.323+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3769731869 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b7c9b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.323+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3769731869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b8a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.323+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3769731869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 0x555583b878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.323+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3769731869 >> 172.21.15.67:0/3769731869 conn(0x555583adc6a0 msgr2=0x555583b78a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.323+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3769731869 shutdown_connections 2024-03-30T02:38:31.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.323+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3769731869 wait complete. 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.323+0000 7f9b38d8eb80 1 -- start start 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.323+0000 7f9b38d8eb80 1 Processor -- start 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b87720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 0x555583b84ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b85010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583a6ad80 con 0x555583b7c5d0 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583a6aa20 con 0x555583b87df0 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555839526a0 con 0x555583b7cef0 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b38d8eb80 10 client.0 _ll_register_callbacks cb 0x555583a74470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b1de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b87720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b1de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b87720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50214/0 (socket says 172.21.15.67:50214) 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b1de00700 1 -- 172.21.15.67:0/1656362280 learned_addr learned my addr 172.21.15.67:0/1656362280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:31.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b1d400700 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 0x555583b84ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.324+0000 7f9b1e800700 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b85010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9b1de00700 1 -- 172.21.15.67:0/1656362280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 msgr2=0x555583b84ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9b1de00700 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 0x555583b84ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9b1de00700 1 -- 172.21.15.67:0/1656362280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 msgr2=0x555583b85010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9b1de00700 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b85010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9b1de00700 1 -- 172.21.15.67:0/1656362280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b201a41c0 con 0x555583b7c5d0 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9b1de00700 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b87720 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9b10000c00 tx=0x7f9b10009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a468fe3540875df server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9affe00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9affe00700 1 -- 172.21.15.67:0/1656362280 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b100a2d10 con 0x555583b7c5d0 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9affe00700 1 -- 172.21.15.67:0/1656362280 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9b100a2fa0 con 0x555583b7c5d0 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/1656362280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555583a8f380 con 0x555583b7c5d0 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9affe00700 1 -- 172.21.15.67:0/1656362280 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b1009abb0 con 0x555583b7c5d0 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9b38d8eb80 10 client.15960 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:31.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.325+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/1656362280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5555838fcb60 con 0x555583b7c5d0 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.326+0000 7f9affe00700 1 -- 172.21.15.67:0/1656362280 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555583a8f380 con 0x555583b7c5d0 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.326+0000 7f9affe00700 1 -- 172.21.15.67:0/1656362280 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9b100cb510 con 0x555583b7c5d0 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.326+0000 7f9b38d8eb80 10 client.15960 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.326+0000 7f9b38d8eb80 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.326+0000 7f9b38d8eb80 20 client.15960 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.326+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555583b97180 0x555583b995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.326+0000 7f9b38d8eb80 4 client.15960 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.326+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/1656362280 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555583ac7700 con 0x555583b97180 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.326+0000 7f9b1d400700 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555583b97180 0x555583b995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.327+0000 7f9b1d400700 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555583b97180 0x555583b995a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f3dcba6473b01a server_cookie=aaa47dd37ff2979 in_seq=0 out_seq=0 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.327+0000 7f9affe00700 10 client.15960 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9affe00700 1 -- 172.21.15.67:0/1656362280 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555583ac7700 con 0x555583b97180 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9affe00700 10 client.15960 handle_command_reply: tid=0 2024-03-30T02:38:31.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 client.15960 shutdown 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/1656362280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 msgr2=0x555583b87720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b87720 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9b10000c00 tx=0x7f9b10009310 comp rx=0 tx=0).stop 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/1656362280 shutdown_connections 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555583b97180 0x555583b995a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7c5d0 0x555583b87720 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b85010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9affe00700 0 client.15960 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/1656362280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7cef0 0x555583b84ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/1656362280 >> 172.21.15.67:0/1656362280 conn(0x555583adc6a0 msgr2=0x555583b7e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/1656362280 shutdown_connections 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/1656362280 wait complete. 2024-03-30T02:38:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.328+0000 7f9b38d8eb80 20 client.15960 trim_cache size 0 max 16384 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b38d8eb80 1 Processor -- start 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b38d8eb80 1 -- start start 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 0x555583b85010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7cef0 0x555583b8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b84340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583b94110 con 0x555583b7cef0 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583b86010 con 0x555583b87df0 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583b7d880 con 0x555583b7c5d0 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b1d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7cef0 0x555583b8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b1d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7cef0 0x555583b8e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50216/0 (socket says 172.21.15.67:50216) 2024-03-30T02:38:31.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b1d400700 1 -- 172.21.15.67:0/2736789182 learned_addr learned my addr 172.21.15.67:0/2736789182 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b1e800700 1 --2- 172.21.15.67:0/2736789182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b84340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.346+0000 7f9b1de00700 1 --2- 172.21.15.67:0/2736789182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 0x555583b85010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.347+0000 7f9b1d400700 1 -- 172.21.15.67:0/2736789182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 msgr2=0x555583b85010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.347+0000 7f9b1d400700 1 --2- 172.21.15.67:0/2736789182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 0x555583b85010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.347+0000 7f9b1d400700 1 -- 172.21.15.67:0/2736789182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 msgr2=0x555583b84340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.347+0000 7f9b1d400700 1 --2- 172.21.15.67:0/2736789182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b84340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.347+0000 7f9b1d400700 1 -- 172.21.15.67:0/2736789182 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555583a94ea0 con 0x555583b7cef0 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.347+0000 7f9b1d400700 1 --2- 172.21.15.67:0/2736789182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7cef0 0x555583b8e7e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x555583b85fe0 tx=0x7f9b0c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a052a958a70164 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.347+0000 7f9b0be00700 1 -- 172.21.15.67:0/2736789182 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c017040 con 0x555583b7cef0 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.347+0000 7f9b0be00700 1 -- 172.21.15.67:0/2736789182 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b0c023070 con 0x555583b7cef0 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.347+0000 7f9b0be00700 1 -- 172.21.15.67:0/2736789182 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c004bd0 con 0x555583b7cef0 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/2736789182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7cef0 msgr2=0x555583b8e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/2736789182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7cef0 0x555583b8e7e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x555583b85fe0 tx=0x7f9b0c009260 comp rx=0 tx=0).stop 2024-03-30T02:38:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/2736789182 shutdown_connections 2024-03-30T02:38:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/2736789182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b87df0 0x555583b84340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/2736789182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 0x555583b85010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/2736789182 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b7cef0 0x555583b8e7e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/2736789182 >> 172.21.15.67:0/2736789182 conn(0x555583adc6a0 msgr2=0x555583b83380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/2736789182 shutdown_connections 2024-03-30T02:38:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/2736789182 wait complete. 2024-03-30T02:38:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.348+0000 7f9b38d8eb80 1 -- start start 2024-03-30T02:38:31.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b38d8eb80 1 Processor -- start 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 0x555583b8e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b7cef0 0x555583b8caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b38d8eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b87df0 0x555583b8d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583b8d940 con 0x555583b87df0 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583bef390 con 0x555583b7cef0 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b38d8eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555583bef4d0 con 0x555583b7c5d0 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b38d8eb80 10 client.0 _ll_register_callbacks cb 0x555583a74470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b1d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b7cef0 0x555583b8caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b1e800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b87df0 0x555583b8d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b1d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b7cef0 0x555583b8caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38070/0 (socket says 172.21.15.67:38070) 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b1de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 0x555583b8e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.349+0000 7f9b1d400700 1 -- 172.21.15.67:0/3882899388 learned_addr learned my addr 172.21.15.67:0/3882899388 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9b1de00700 1 -- 172.21.15.67:0/3882899388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b7cef0 msgr2=0x555583b8caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9b1de00700 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b7cef0 0x555583b8caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9b1de00700 1 -- 172.21.15.67:0/3882899388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b87df0 msgr2=0x555583b8d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9b1de00700 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b87df0 0x555583b8d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9b1de00700 1 -- 172.21.15.67:0/3882899388 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0c012040 con 0x555583b7c5d0 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9b1de00700 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 0x555583b8e750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b100cb870 tx=0x7f9b100b3440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73507a2983c84b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9affe00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9affe00700 1 -- 172.21.15.67:0/3882899388 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10096cf0 con 0x555583b7c5d0 2024-03-30T02:38:31.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9affe00700 1 -- 172.21.15.67:0/3882899388 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9b100a2fa0 con 0x555583b7c5d0 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3882899388 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b1009e9e0 con 0x555583b7c5d0 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9affe00700 1 -- 172.21.15.67:0/3882899388 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b1009a6f0 con 0x555583b7c5d0 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.350+0000 7f9b38d8eb80 10 client.26002 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3882899388 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555583954cd0 con 0x555583b7c5d0 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9affe00700 1 -- 172.21.15.67:0/3882899388 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b1009e9e0 con 0x555583b7c5d0 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9affe00700 1 -- 172.21.15.67:0/3882899388 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9b100a2b10 con 0x555583b7c5d0 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9b38d8eb80 10 client.26002 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9b38d8eb80 10 client.26002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9b38d8eb80 20 client.26002 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555583bf0780 0x555583bf2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9b38d8eb80 4 client.26002 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3882899388 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55558391dcf0 con 0x555583bf0780 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.351+0000 7f9b1d400700 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555583bf0780 0x555583bf2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b1d400700 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555583bf0780 0x555583bf2b70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6e99e4a2b8b4df9 server_cookie=2a0540805586d366 in_seq=0 out_seq=0 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9affe00700 10 client.26002 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9affe00700 1 -- 172.21.15.67:0/3882899388 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55558391dcf0 con 0x555583bf0780 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9affe00700 10 client.26002 handle_command_reply: tid=0 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 client.26002 shutdown 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3882899388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 msgr2=0x555583b8e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 0x555583b8e750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9b100cb870 tx=0x7f9b100b3440 comp rx=0 tx=0).stop 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3882899388 shutdown_connections 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555583bf0780 0x555583bf2b70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555583b87df0 0x555583b8d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555583b7c5d0 0x555583b8e750 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9affe00700 0 client.26002 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 --2- 172.21.15.67:0/3882899388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555583b7cef0 0x555583b8caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3882899388 >> 172.21.15.67:0/3882899388 conn(0x555583adc6a0 msgr2=0x555583b92f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3882899388 shutdown_connections 2024-03-30T02:38:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.352+0000 7f9b38d8eb80 1 -- 172.21.15.67:0/3882899388 wait complete. 2024-03-30T02:38:31.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b38d8eb80 20 client.26002 trim_cache size 0 max 16384 2024-03-30T02:38:31.359 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:38:31.359 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:38:31.359 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:38:31.359 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:38:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 -- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9b040a1430 msgr2=0x7f9b040a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9b040a1430 0x7f9b040a38f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800f870 tx=0x7f9b1800fc30 comp rx=0 tx=0).stop 2024-03-30T02:38:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 -- 172.21.15.67:0/1096957501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b20069260 msgr2=0x7f9b20197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b20069260 0x7f9b20197200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000b580 tx=0x7f9b1000bfd0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 -- 172.21.15.67:0/1096957501 shutdown_connections 2024-03-30T02:38:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9b040a1430 0x7f9b040a38f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2019bae0 0x7f9b2019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b20069260 0x7f9b20197200 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 --2- 172.21.15.67:0/1096957501 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b201060b0 0x7f9b20197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 -- 172.21.15.67:0/1096957501 >> 172.21.15.67:0/1096957501 conn(0x7f9b20076d80 msgr2=0x7f9b201071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 -- 172.21.15.67:0/1096957501 shutdown_connections 2024-03-30T02:38:31.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.353+0000 7f9b25200700 1 -- 172.21.15.67:0/1096957501 wait complete. 2024-03-30T02:38:31.373 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:38:31.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:38:31.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.576+0000 7f1872000700 1 -- 172.21.15.67:0/541390931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 msgr2=0x7f186c101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.576+0000 7f1872000700 1 --2- 172.21.15.67:0/541390931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 0x7f186c101240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f185c009990 tx=0x7f185c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.576+0000 7f1872000700 1 -- 172.21.15.67:0/541390931 shutdown_connections 2024-03-30T02:38:31.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.576+0000 7f1872000700 1 --2- 172.21.15.67:0/541390931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 0x7f186c101240 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.576+0000 7f1872000700 1 --2- 172.21.15.67:0/541390931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f186c105f80 0x7f186c10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.576+0000 7f1872000700 1 --2- 172.21.15.67:0/541390931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f186c101810 0x7f186c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.576+0000 7f1872000700 1 -- 172.21.15.67:0/541390931 >> 172.21.15.67:0/541390931 conn(0x7f186c0fc700 msgr2=0x7f186c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:31.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.576+0000 7f1872000700 1 -- 172.21.15.67:0/541390931 shutdown_connections 2024-03-30T02:38:31.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.576+0000 7f1872000700 1 -- 172.21.15.67:0/541390931 wait complete. 2024-03-30T02:38:31.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.577+0000 7f1872000700 1 Processor -- start 2024-03-30T02:38:31.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.577+0000 7f1872000700 1 -- start start 2024-03-30T02:38:31.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f1872000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 0x7f186c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f1872000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f186c105f80 0x7f186c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f1872000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f186c19ba40 0x7f186c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f1872000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c19c4c0 con 0x7f186c100e60 2024-03-30T02:38:31.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f1872000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c19c630 con 0x7f186c105f80 2024-03-30T02:38:31.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f1872000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c19c7f0 con 0x7f186c19ba40 2024-03-30T02:38:31.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f186b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 0x7f186c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f186b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 0x7f186c197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50236/0 (socket says 172.21.15.67:50236) 2024-03-30T02:38:31.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f186b400700 1 -- 172.21.15.67:0/731246877 learned_addr learned my addr 172.21.15.67:0/731246877 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f186be00700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f186c19ba40 0x7f186c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.578+0000 7f186aa00700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f186c105f80 0x7f186c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f186b400700 1 -- 172.21.15.67:0/731246877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f186c19ba40 msgr2=0x7f186c19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f186b400700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f186c19ba40 0x7f186c19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f186b400700 1 -- 172.21.15.67:0/731246877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f186c105f80 msgr2=0x7f186c1976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f186b400700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f186c105f80 0x7f186c1976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f186b400700 1 -- 172.21.15.67:0/731246877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c0095f0 con 0x7f186c100e60 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f186b400700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 0x7f186c197160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f185c005910 tx=0x7f185c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f1863e00700 1 -- 172.21.15.67:0/731246877 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c01d070 con 0x7f186c100e60 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f1863e00700 1 -- 172.21.15.67:0/731246877 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185c022470 con 0x7f186c100e60 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c1a3cf0 con 0x7f186c100e60 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f1863e00700 1 -- 172.21.15.67:0/731246877 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c017650 con 0x7f186c100e60 2024-03-30T02:38:31.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.579+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c1a4110 con 0x7f186c100e60 2024-03-30T02:38:31.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.580+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c1091a0 con 0x7f186c100e60 2024-03-30T02:38:31.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.582+0000 7f1863e00700 1 -- 172.21.15.67:0/731246877 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f185c0225e0 con 0x7f186c100e60 2024-03-30T02:38:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.584+0000 7f1863e00700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f18500a13b0 0x7f18500a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.584+0000 7f1863e00700 1 -- 172.21.15.67:0/731246877 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f185c0ce660 con 0x7f186c100e60 2024-03-30T02:38:31.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.584+0000 7f186aa00700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f18500a13b0 0x7f18500a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:31.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.585+0000 7f186aa00700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f18500a13b0 0x7f18500a3870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f186c1986b0 tx=0x7f1858009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:31.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.587+0000 7f1863e00700 1 -- 172.21.15.67:0/731246877 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f185c097660 con 0x7f186c100e60 2024-03-30T02:38:31.768 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-30T02:38:31.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.856+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f186c102330 con 0x7f186c100e60 2024-03-30T02:38:31.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.857+0000 7f1863e00700 1 -- 172.21.15.67:0/731246877 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f185c09ac80 con 0x7f186c100e60 2024-03-30T02:38:31.857 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:38:31.857 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:36:42.080059+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:38:31.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f18500a13b0 msgr2=0x7f18500a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f18500a13b0 0x7f18500a3870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f186c1986b0 tx=0x7f1858009260 comp rx=0 tx=0).stop 2024-03-30T02:38:31.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 msgr2=0x7f186c197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:31.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 0x7f186c197160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f185c005910 tx=0x7f185c004730 comp rx=0 tx=0).stop 2024-03-30T02:38:31.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 shutdown_connections 2024-03-30T02:38:31.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f18500a13b0 0x7f18500a3870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f186c100e60 0x7f186c197160 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f186c19ba40 0x7f186c19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 --2- 172.21.15.67:0/731246877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f186c105f80 0x7f186c1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 >> 172.21.15.67:0/731246877 conn(0x7f186c0fc700 msgr2=0x7f186c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 shutdown_connections 2024-03-30T02:38:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:31.860+0000 7f1872000700 1 -- 172.21.15.67:0/731246877 wait complete. 2024-03-30T02:38:31.861 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-03-30T02:38:31.879 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:38:31.886 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-30T02:38:31.953 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-30T02:38:31.957 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-30T02:38:31.957 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-30T02:38:32.021 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-30T02:38:32.028 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:38:32.028 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-30T02:38:32.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.085+0000 7feb1e600700 1 -- 172.21.15.67:0/2374157812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb18073150 msgr2=0x7feb18073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.085+0000 7feb1e600700 1 --2- 172.21.15.67:0/2374157812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb18073150 0x7feb18073530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feb08009990 tx=0x7feb08009ca0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.086+0000 7feb1e600700 1 -- 172.21.15.67:0/2374157812 shutdown_connections 2024-03-30T02:38:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.086+0000 7feb1e600700 1 --2- 172.21.15.67:0/2374157812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb18111d10 0x7feb18114100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.086+0000 7feb1e600700 1 --2- 172.21.15.67:0/2374157812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb18073150 0x7feb18073530 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.086+0000 7feb1e600700 1 --2- 172.21.15.67:0/2374157812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb18073a70 0x7feb181117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.086+0000 7feb1e600700 1 -- 172.21.15.67:0/2374157812 >> 172.21.15.67:0/2374157812 conn(0x7feb180fc7f0 msgr2=0x7feb180fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.086+0000 7feb1e600700 1 -- 172.21.15.67:0/2374157812 shutdown_connections 2024-03-30T02:38:32.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.086+0000 7feb1e600700 1 -- 172.21.15.67:0/2374157812 wait complete. 2024-03-30T02:38:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.087+0000 7feb1e600700 1 Processor -- start 2024-03-30T02:38:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.087+0000 7feb1e600700 1 -- start start 2024-03-30T02:38:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb1e600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb18073150 0x7feb181a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb1e600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb18073a70 0x7feb181a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb1e600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb18111d10 0x7feb1819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb1e600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1819dcd0 con 0x7feb18073150 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb1e600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1819de40 con 0x7feb18073a70 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb1e600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1819e000 con 0x7feb18111d10 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb17400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb18073150 0x7feb181a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb17400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb18073150 0x7feb181a3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50254/0 (socket says 172.21.15.67:50254) 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb16a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb18073a70 0x7feb181a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb17400700 1 -- 172.21.15.67:0/2504946444 learned_addr learned my addr 172.21.15.67:0/2504946444 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb17e00700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb18111d10 0x7feb1819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb17400700 1 -- 172.21.15.67:0/2504946444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb18111d10 msgr2=0x7feb1819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb17400700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb18111d10 0x7feb1819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb17400700 1 -- 172.21.15.67:0/2504946444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb18073a70 msgr2=0x7feb181a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb17400700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb18073a70 0x7feb181a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb17400700 1 -- 172.21.15.67:0/2504946444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb080095f0 con 0x7feb18073150 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.088+0000 7feb16a00700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb18073a70 0x7feb181a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:32.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.089+0000 7feb17400700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb18073150 0x7feb181a3540 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feb08000c00 tx=0x7feb08005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.089+0000 7feb0fe00700 1 -- 172.21.15.67:0/2504946444 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0801d070 con 0x7feb18073150 2024-03-30T02:38:32.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.089+0000 7feb0fe00700 1 -- 172.21.15.67:0/2504946444 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb08022470 con 0x7feb18073150 2024-03-30T02:38:32.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.089+0000 7feb0fe00700 1 -- 172.21.15.67:0/2504946444 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08017650 con 0x7feb18073150 2024-03-30T02:38:32.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.089+0000 7feb1e600700 1 -- 172.21.15.67:0/2504946444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb1819e280 con 0x7feb18073150 2024-03-30T02:38:32.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.089+0000 7feb1e600700 1 -- 172.21.15.67:0/2504946444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb181aa530 con 0x7feb18073150 2024-03-30T02:38:32.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.092+0000 7feb0fe00700 1 -- 172.21.15.67:0/2504946444 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) v1 ==== 136098+0+0 (secure 0 0 0) 0x7feb080225e0 con 0x7feb18073150 2024-03-30T02:38:32.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.094+0000 7feb0fe00700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7feafc0a1430 0x7feafc0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.094+0000 7feb0fe00700 1 -- 172.21.15.67:0/2504946444 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7feb080ce620 con 0x7feb18073150 2024-03-30T02:38:32.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.094+0000 7feb16a00700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7feafc0a1430 0x7feafc0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.094+0000 7feb32092b80 1 Processor -- start 2024-03-30T02:38:32.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb32092b80 1 -- start start 2024-03-30T02:38:32.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb16a00700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7feafc0a1430 0x7feafc0a38f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb1819ed80 tx=0x7feb04009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb32092b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cdf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb32092b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 0x563c42cea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb32092b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 0x563c42ced1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb32092b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42c38990 con 0x563c42cdf5d0 2024-03-30T02:38:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb32092b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42c36350 con 0x563c42ceadf0 2024-03-30T02:38:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb32092b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42c31c40 con 0x563c42cdfef0 2024-03-30T02:38:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb17400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cdf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb17400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cdf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50260/0 (socket says 172.21.15.67:50260) 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.095+0000 7feb17400700 1 -- 172.21.15.67:0/1977746301 learned_addr learned my addr 172.21.15.67:0/1977746301 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb17e00700 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 0x563c42ced1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb16a00700 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 0x563c42cea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb17400700 1 -- 172.21.15.67:0/1977746301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 msgr2=0x563c42cea8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb17400700 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 0x563c42cea8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb17400700 1 -- 172.21.15.67:0/1977746301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 msgr2=0x563c42ced1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb17400700 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 0x563c42ced1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb17400700 1 -- 172.21.15.67:0/1977746301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c42ab33b0 con 0x563c42cdf5d0 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb17e00700 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 0x563c42ced1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb17400700 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cdf9b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feb0809b270 tx=0x7feb08097740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a231e023681f66ad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb0cc00700 1 -- 172.21.15.67:0/1977746301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0809ac40 con 0x563c42cdf5d0 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb0cc00700 1 -- 172.21.15.67:0/1977746301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb0809b9e0 con 0x563c42cdf5d0 2024-03-30T02:38:32.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.096+0000 7feb0cc00700 1 -- 172.21.15.67:0/1977746301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08099750 con 0x563c42cdf5d0 2024-03-30T02:38:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.097+0000 7feb32092b80 1 -- 172.21.15.67:0/1977746301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 msgr2=0x563c42cdf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.097+0000 7feb32092b80 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cdf9b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feb0809b270 tx=0x7feb08097740 comp rx=0 tx=0).stop 2024-03-30T02:38:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.097+0000 7feb32092b80 1 -- 172.21.15.67:0/1977746301 shutdown_connections 2024-03-30T02:38:32.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.097+0000 7feb32092b80 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cdf9b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.097+0000 7feb32092b80 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 0x563c42ced1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.097+0000 7feb32092b80 1 --2- 172.21.15.67:0/1977746301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 0x563c42cea8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.097+0000 7feb32092b80 1 -- 172.21.15.67:0/1977746301 >> 172.21.15.67:0/1977746301 conn(0x563c42c3f6a0 msgr2=0x563c42cdba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.098+0000 7feb32092b80 1 -- 172.21.15.67:0/1977746301 shutdown_connections 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.098+0000 7feb32092b80 1 -- 172.21.15.67:0/1977746301 wait complete. 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.098+0000 7feb32092b80 1 -- start start 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.098+0000 7feb32092b80 1 Processor -- start 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb32092b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cea820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb32092b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdfef0 0x563c42ce7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb32092b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42ceadf0 0x563c42ce8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb32092b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42bcdd80 con 0x563c42cdf5d0 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb32092b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42bcda20 con 0x563c42cdfef0 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb32092b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42ab56a0 con 0x563c42ceadf0 2024-03-30T02:38:32.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb32092b80 10 client.0 _ll_register_callbacks cb 0x563c42bd7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb17400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb17400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cea820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50274/0 (socket says 172.21.15.67:50274) 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb17400700 1 -- 172.21.15.67:0/973901522 learned_addr learned my addr 172.21.15.67:0/973901522 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb16a00700 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdfef0 0x563c42ce7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.099+0000 7feb17e00700 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42ceadf0 0x563c42ce8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb17400700 1 -- 172.21.15.67:0/973901522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42ceadf0 msgr2=0x563c42ce8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb17400700 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42ceadf0 0x563c42ce8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb17400700 1 -- 172.21.15.67:0/973901522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdfef0 msgr2=0x563c42ce7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb17400700 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdfef0 0x563c42ce7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb17400700 1 -- 172.21.15.67:0/973901522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb10011230 con 0x563c42cdf5d0 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb16a00700 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdfef0 0x563c42ce7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb17400700 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cea820 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feb080977e0 tx=0x7feb08005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9845ed0ccc86947f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb01600700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb01600700 1 -- 172.21.15.67:0/973901522 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0807e070 con 0x563c42cdf5d0 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb01600700 1 -- 172.21.15.67:0/973901522 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7feb0809b2e0 con 0x563c42cdf5d0 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb32092b80 1 -- 172.21.15.67:0/973901522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c42bf2380 con 0x563c42cdf5d0 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb01600700 1 -- 172.21.15.67:0/973901522 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb080a3240 con 0x563c42cdf5d0 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb32092b80 10 client.15984 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.100+0000 7feb32092b80 1 -- 172.21.15.67:0/973901522 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7feb181aa530 con 0x563c42cdf5d0 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.101+0000 7feb01600700 1 -- 172.21.15.67:0/973901522 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb080a7020 con 0x563c42cdf5d0 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.101+0000 7feb01600700 1 -- 172.21.15.67:0/973901522 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb08081030 con 0x563c42cdf5d0 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.101+0000 7feb32092b80 10 client.15984 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:32.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.101+0000 7feb32092b80 10 client.15984 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.101+0000 7feb32092b80 20 client.15984 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.101+0000 7feb32092b80 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x563c42cfa220 0x563c42cfc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.101+0000 7feb32092b80 4 client.15984 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.101+0000 7feb32092b80 1 -- 172.21.15.67:0/973901522 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c42c2a700 con 0x563c42cfa220 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.101+0000 7feb16a00700 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x563c42cfa220 0x563c42cfc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.102+0000 7feb16a00700 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x563c42cfa220 0x563c42cfc640 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8fda7eb464015cc7 server_cookie=45b032614eb7811e in_seq=0 out_seq=0 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.102+0000 7feb01600700 10 client.15984 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb01600700 1 -- 172.21.15.67:0/973901522 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563c42c2a700 con 0x563c42cfa220 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb01600700 10 client.15984 handle_command_reply: tid=0 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 client.15984 shutdown 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 -- 172.21.15.67:0/973901522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 msgr2=0x563c42cea820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cea820 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feb080977e0 tx=0x7feb08005c00 comp rx=0 tx=0).stop 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 -- 172.21.15.67:0/973901522 shutdown_connections 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdf5d0 0x563c42cea820 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x563c42cfa220 0x563c42cfc640 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42ceadf0 0x563c42ce8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 --2- 172.21.15.67:0/973901522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdfef0 0x563c42ce7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb01600700 0 client.15984 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:38:32.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 -- 172.21.15.67:0/973901522 >> 172.21.15.67:0/973901522 conn(0x563c42c3f6a0 msgr2=0x563c42ce16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 -- 172.21.15.67:0/973901522 shutdown_connections 2024-03-30T02:38:32.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.103+0000 7feb32092b80 1 -- 172.21.15.67:0/973901522 wait complete. 2024-03-30T02:38:32.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.104+0000 7feb32092b80 20 client.15984 trim_cache size 0 max 16384 2024-03-30T02:38:32.116 INFO:tasks.workunit:Running workunits matching kernel_untar_build.sh on client.0... 2024-03-30T02:38:32.117 INFO:tasks.workunit:Running workunit kernel_untar_build.sh... 2024-03-30T02:38:32.117 DEBUG:teuthology.orchestra.run.smithi067:workunit test kernel_untar_build.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=470100a4ccbcf7769052bf09a5413881ccf1b97f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh 2024-03-30T02:38:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb32092b80 1 Processor -- start 2024-03-30T02:38:32.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb32092b80 1 -- start start 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb32092b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdf5d0 0x563c42ce8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb32092b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 0x563c42cf17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb32092b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42ceadf0 0x563c42ce2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb32092b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42ce89b0 con 0x563c42ceadf0 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb32092b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42cf1d20 con 0x563c42cdf5d0 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb32092b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42ce61c0 con 0x563c42cdfef0 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb17e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42ceadf0 0x563c42ce2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb17e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42ceadf0 0x563c42ce2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50290/0 (socket says 172.21.15.67:50290) 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb17e00700 1 -- 172.21.15.67:0/3256863709 learned_addr learned my addr 172.21.15.67:0/3256863709 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb16a00700 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 0x563c42cf17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.119+0000 7feb17400700 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdf5d0 0x563c42ce8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb17e00700 1 -- 172.21.15.67:0/3256863709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 msgr2=0x563c42cf17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb17e00700 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 0x563c42cf17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb17e00700 1 -- 172.21.15.67:0/3256863709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdf5d0 msgr2=0x563c42ce8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb17e00700 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdf5d0 0x563c42ce8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb17e00700 1 -- 172.21.15.67:0/3256863709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c42a5fb60 con 0x563c42ceadf0 2024-03-30T02:38:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb17400700 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdf5d0 0x563c42ce8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb17e00700 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42ceadf0 0x563c42ce2f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feb10007660 tx=0x7feb10009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff115f809ee87610 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb0d600700 1 -- 172.21.15.67:0/3256863709 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000e040 con 0x563c42ceadf0 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb0d600700 1 -- 172.21.15.67:0/3256863709 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb1000f040 con 0x563c42ceadf0 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.120+0000 7feb0d600700 1 -- 172.21.15.67:0/3256863709 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10005280 con 0x563c42ceadf0 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.121+0000 7feb32092b80 1 -- 172.21.15.67:0/3256863709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42ceadf0 msgr2=0x563c42ce2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.121+0000 7feb32092b80 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42ceadf0 0x563c42ce2f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feb10007660 tx=0x7feb10009920 comp rx=0 tx=0).stop 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.121+0000 7feb32092b80 1 -- 172.21.15.67:0/3256863709 shutdown_connections 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.121+0000 7feb32092b80 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42ceadf0 0x563c42ce2f90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.121+0000 7feb32092b80 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42cdf5d0 0x563c42ce8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.121+0000 7feb32092b80 1 --2- 172.21.15.67:0/3256863709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdfef0 0x563c42cf17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.121+0000 7feb32092b80 1 -- 172.21.15.67:0/3256863709 >> 172.21.15.67:0/3256863709 conn(0x563c42c3f6a0 msgr2=0x563c42cd45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.121+0000 7feb32092b80 1 -- 172.21.15.67:0/3256863709 shutdown_connections 2024-03-30T02:38:32.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.121+0000 7feb32092b80 1 -- 172.21.15.67:0/3256863709 wait complete. 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.122+0000 7feb32092b80 1 -- start start 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.122+0000 7feb32092b80 1 Processor -- start 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb32092b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdf5d0 0x563c42cef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb32092b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdfef0 0x563c42ceff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb32092b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 0x563c42d47020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb32092b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42d47560 con 0x563c42cdfef0 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb32092b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42d476d0 con 0x563c42ceadf0 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb32092b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c42d47890 con 0x563c42cdf5d0 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb32092b80 10 client.0 _ll_register_callbacks cb 0x563c42bd7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb16a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdfef0 0x563c42ceff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb16a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdfef0 0x563c42ceff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50300/0 (socket says 172.21.15.67:50300) 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb16a00700 1 -- 172.21.15.67:0/1890585450 learned_addr learned my addr 172.21.15.67:0/1890585450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb17400700 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdf5d0 0x563c42cef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb17e00700 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 0x563c42d47020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb16a00700 1 -- 172.21.15.67:0/1890585450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdf5d0 msgr2=0x563c42cef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb16a00700 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdf5d0 0x563c42cef9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb16a00700 1 -- 172.21.15.67:0/1890585450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 msgr2=0x563c42d47020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb16a00700 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 0x563c42d47020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.123+0000 7feb16a00700 1 -- 172.21.15.67:0/1890585450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb10009270 con 0x563c42cdfef0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.124+0000 7feb16a00700 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdfef0 0x563c42ceff10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x563c42ce90e0 tx=0x7feb040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f386c74a60dca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.124+0000 7feb01600700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.124+0000 7feb01600700 1 -- 172.21.15.67:0/1890585450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb04024040 con 0x563c42cdfef0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.124+0000 7feb01600700 1 -- 172.21.15.67:0/1890585450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7feb04003b00 con 0x563c42cdfef0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.124+0000 7feb32092b80 1 -- 172.21.15.67:0/1890585450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb0809b9e0 con 0x563c42cdfef0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.124+0000 7feb01600700 1 -- 172.21.15.67:0/1890585450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb04029580 con 0x563c42cdfef0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.124+0000 7feb32092b80 10 client.15990 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.124+0000 7feb32092b80 1 -- 172.21.15.67:0/1890585450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563c42bf7ea0 con 0x563c42cdfef0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.125+0000 7feb01600700 1 -- 172.21.15.67:0/1890585450 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb0809b9e0 con 0x563c42cdfef0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.125+0000 7feb01600700 1 -- 172.21.15.67:0/1890585450 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb040313c0 con 0x563c42cdfef0 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.125+0000 7feb32092b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.125+0000 7feb32092b80 10 client.15990 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:38:32.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.125+0000 7feb32092b80 20 client.15990 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.125+0000 7feb32092b80 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x563c42d4f1a0 0x563c42d515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.125+0000 7feb32092b80 4 client.15990 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:38:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.125+0000 7feb32092b80 1 -- 172.21.15.67:0/1890585450 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c42a80cf0 con 0x563c42d4f1a0 2024-03-30T02:38:32.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.125+0000 7feb17400700 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x563c42d4f1a0 0x563c42d515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb01600700 10 client.15990 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:38:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb17400700 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x563c42d4f1a0 0x563c42d515c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4cd8bf21a076b99b server_cookie=8da25fb407eb22db in_seq=0 out_seq=0 2024-03-30T02:38:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb01600700 1 -- 172.21.15.67:0/1890585450 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c42a80cf0 con 0x563c42d4f1a0 2024-03-30T02:38:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb01600700 10 client.15990 handle_command_reply: tid=0 2024-03-30T02:38:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb32092b80 1 client.15990 shutdown 2024-03-30T02:38:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb32092b80 1 -- 172.21.15.67:0/1890585450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdfef0 msgr2=0x563c42ceff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb32092b80 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdfef0 0x563c42ceff10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x563c42ce90e0 tx=0x7feb040047c0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb32092b80 1 -- 172.21.15.67:0/1890585450 shutdown_connections 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb32092b80 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x563c42d4f1a0 0x563c42d515c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb32092b80 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c42ceadf0 0x563c42d47020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb32092b80 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563c42cdf5d0 0x563c42cef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb32092b80 1 --2- 172.21.15.67:0/1890585450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563c42cdfef0 0x563c42ceff10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb32092b80 1 -- 172.21.15.67:0/1890585450 >> 172.21.15.67:0/1890585450 conn(0x563c42c3f6a0 msgr2=0x563c42ce2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.126+0000 7feb01600700 0 client.15990 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb32092b80 1 -- 172.21.15.67:0/1890585450 shutdown_connections 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb32092b80 1 -- 172.21.15.67:0/1890585450 wait complete. 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb32092b80 20 client.15990 trim_cache size 0 max 16384 2024-03-30T02:38:32.131 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 -- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7feafc0a1430 msgr2=0x7feafc0a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7feafc0a1430 0x7feafc0a38f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feb1819ed80 tx=0x7feb04009260 comp rx=0 tx=0).stop 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 -- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb18073150 msgr2=0x7feb181a3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb18073150 0x7feb181a3540 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feb08000c00 tx=0x7feb08005740 comp rx=0 tx=0).stop 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 -- 172.21.15.67:0/2504946444 shutdown_connections 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7feafc0a1430 0x7feafc0a38f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7feb18073150 0x7feb181a3540 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb18111d10 0x7feb1819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 --2- 172.21.15.67:0/2504946444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb18073a70 0x7feb181a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.127+0000 7feb1e600700 1 -- 172.21.15.67:0/2504946444 >> 172.21.15.67:0/2504946444 conn(0x7feb180fc7f0 msgr2=0x7feb180fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.128+0000 7feb1e600700 1 -- 172.21.15.67:0/2504946444 shutdown_connections 2024-03-30T02:38:32.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.128+0000 7feb1e600700 1 -- 172.21.15.67:0/2504946444 wait complete. 2024-03-30T02:38:32.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:38:32.218 INFO:tasks.workunit.client.0.smithi067.stderr:--2024-03-30 02:38:32-- http://download.ceph.com/qa/linux-6.5.11.tar.xz 2024-03-30T02:38:32.229 INFO:tasks.workunit.client.0.smithi067.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-30T02:38:32.265 INFO:tasks.workunit.client.0.smithi067.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-30T02:38:32.302 INFO:tasks.workunit.client.0.smithi067.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-30T02:38:32.302 INFO:tasks.workunit.client.0.smithi067.stderr:Length: 139010916 (133M) [application/octet-stream] 2024-03-30T02:38:32.302 INFO:tasks.workunit.client.0.smithi067.stderr:Saving to: ‘linux.tar.xz’ 2024-03-30T02:38:32.307 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:32.376 INFO:tasks.workunit.client.0.smithi067.stderr: 0K .......... .......... .......... .......... .......... 0% 703K 3m13s 2024-03-30T02:38:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.392+0000 7f1925c00700 1 -- 172.21.15.67:0/278485647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 msgr2=0x7f192010dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.392+0000 7f1925c00700 1 --2- 172.21.15.67:0/278485647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 0x7f192010dc00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1918009f90 tx=0x7f191800b330 comp rx=0 tx=0).stop 2024-03-30T02:38:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.392+0000 7f1925c00700 1 -- 172.21.15.67:0/278485647 shutdown_connections 2024-03-30T02:38:32.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.392+0000 7f1925c00700 1 --2- 172.21.15.67:0/278485647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 0x7f192010dc00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.392+0000 7f1925c00700 1 --2- 172.21.15.67:0/278485647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1920100f60 0x7f1920101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.392+0000 7f1925c00700 1 --2- 172.21.15.67:0/278485647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1920101910 0x7f1920105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.392+0000 7f1925c00700 1 -- 172.21.15.67:0/278485647 >> 172.21.15.67:0/278485647 conn(0x7f19200fc7c0 msgr2=0x7f19200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.393+0000 7f1925c00700 1 -- 172.21.15.67:0/278485647 shutdown_connections 2024-03-30T02:38:32.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.393+0000 7f1925c00700 1 -- 172.21.15.67:0/278485647 wait complete. 2024-03-30T02:38:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.394+0000 7f1925c00700 1 Processor -- start 2024-03-30T02:38:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.394+0000 7f1925c00700 1 -- start start 2024-03-30T02:38:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.394+0000 7f1925c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1920100f60 0x7f1920199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.394+0000 7f1925c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 0x7f1920199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.394+0000 7f1925c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f192019de30 0x7f192019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.394+0000 7f1925c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192019e8b0 con 0x7f1920106080 2024-03-30T02:38:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.394+0000 7f1925c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192019ea20 con 0x7f1920100f60 2024-03-30T02:38:32.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.394+0000 7f1925c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192019ebe0 con 0x7f192019de30 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.394+0000 7f191e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 0x7f1920199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 0x7f1920199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50310/0 (socket says 172.21.15.67:50310) 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f192019de30 0x7f192019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191e000700 1 -- 172.21.15.67:0/2684570765 learned_addr learned my addr 172.21.15.67:0/2684570765 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191ea00700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1920100f60 0x7f1920199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191e000700 1 -- 172.21.15.67:0/2684570765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f192019de30 msgr2=0x7f192019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191e000700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f192019de30 0x7f192019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191e000700 1 -- 172.21.15.67:0/2684570765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1920100f60 msgr2=0x7f1920199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191e000700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1920100f60 0x7f1920199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191e000700 1 -- 172.21.15.67:0/2684570765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100095f0 con 0x7f1920106080 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191f400700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f192019de30 0x7f192019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f191e000700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 0x7f1920199a90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f190c00e970 tx=0x7f190c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f1917400700 1 -- 172.21.15.67:0/2684570765 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c009790 con 0x7f1920106080 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f1925c00700 1 -- 172.21.15.67:0/2684570765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1918009c40 con 0x7f1920106080 2024-03-30T02:38:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.395+0000 7f1917400700 1 -- 172.21.15.67:0/2684570765 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f190c00c7e0 con 0x7f1920106080 2024-03-30T02:38:32.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.396+0000 7f1917400700 1 -- 172.21.15.67:0/2684570765 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0054c0 con 0x7f1920106080 2024-03-30T02:38:32.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.396+0000 7f1925c00700 1 -- 172.21.15.67:0/2684570765 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19201a6490 con 0x7f1920106080 2024-03-30T02:38:32.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.399+0000 7f1917400700 1 -- 172.21.15.67:0/2684570765 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f190c00c950 con 0x7f1920106080 2024-03-30T02:38:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f19396cab80 1 Processor -- start 2024-03-30T02:38:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f19396cab80 1 -- start start 2024-03-30T02:38:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f1917400700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19040a13b0 0x7f19040a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f19396cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d1247c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f19396cab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d124878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f1917400700 1 -- 172.21.15.67:0/2684570765 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f190c014070 con 0x7f1920106080 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f19396cab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d12487df0 0x557d1248a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f19396cab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d123d5990 con 0x557d1247cef0 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f19396cab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d123d3350 con 0x557d12487df0 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.400+0000 7f19396cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d123cec40 con 0x557d1247c5d0 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191f400700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19040a13b0 0x7f19040a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d1247c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d1247c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60666/0 (socket says 172.21.15.67:60666) 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191ea00700 1 -- 172.21.15.67:0/829538262 learned_addr learned my addr 172.21.15.67:0/829538262 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191e000700 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d124878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191f400700 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d12487df0 0x557d1248a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191ea00700 1 -- 172.21.15.67:0/829538262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d12487df0 msgr2=0x557d1248a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191ea00700 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d12487df0 0x557d1248a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191ea00700 1 -- 172.21.15.67:0/829538262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 msgr2=0x557d124878b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191ea00700 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d124878b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191ea00700 1 -- 172.21.15.67:0/829538262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d122503b0 con 0x557d1247c5d0 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191e000700 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d124878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.401+0000 7f191ea00700 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d1247c9b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1910009fc0 tx=0x7f1910011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c011b63915612cfb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.402+0000 7f191f400700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19040a13b0 0x7f19040a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19180079a0 tx=0x7f1918003c50 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.402+0000 7f190be00700 1 -- 172.21.15.67:0/829538262 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191000e670 con 0x557d1247c5d0 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.402+0000 7f190be00700 1 -- 172.21.15.67:0/829538262 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f191000ec90 con 0x557d1247c5d0 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.402+0000 7f190be00700 1 -- 172.21.15.67:0/829538262 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910019c70 con 0x557d1247c5d0 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.402+0000 7f19396cab80 1 -- 172.21.15.67:0/829538262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 msgr2=0x557d1247c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.402+0000 7f19396cab80 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d1247c9b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1910009fc0 tx=0x7f1910011720 comp rx=0 tx=0).stop 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.403+0000 7f19396cab80 1 -- 172.21.15.67:0/829538262 shutdown_connections 2024-03-30T02:38:32.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.403+0000 7f19396cab80 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d12487df0 0x557d1248a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.403+0000 7f19396cab80 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d1247c9b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.403+0000 7f19396cab80 1 --2- 172.21.15.67:0/829538262 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d124878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.403+0000 7f19396cab80 1 -- 172.21.15.67:0/829538262 >> 172.21.15.67:0/829538262 conn(0x557d123dc6a0 msgr2=0x557d12478a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.403+0000 7f19396cab80 1 -- 172.21.15.67:0/829538262 shutdown_connections 2024-03-30T02:38:32.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.403+0000 7f19396cab80 1 -- 172.21.15.67:0/829538262 wait complete. 2024-03-30T02:38:32.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.403+0000 7f19396cab80 1 -- start start 2024-03-30T02:38:32.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.403+0000 7f19396cab80 1 Processor -- start 2024-03-30T02:38:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f19396cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d12487700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f19396cab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 0x557d12484a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f19396cab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d12487df0 0x557d12485000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f19396cab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1236ad80 con 0x557d12487df0 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f19396cab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1236aa20 con 0x557d1247cef0 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f19396cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d122526a0 con 0x557d1247c5d0 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f19396cab80 10 client.0 _ll_register_callbacks cb 0x557d12374470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f191ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d12487700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f191f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d12487df0 0x557d12485000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f191ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d12487700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60670/0 (socket says 172.21.15.67:60670) 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f191ea00700 1 -- 172.21.15.67:0/328872901 learned_addr learned my addr 172.21.15.67:0/328872901 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.404+0000 7f191e000700 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 0x557d12484a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f191ea00700 1 -- 172.21.15.67:0/328872901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 msgr2=0x557d12484a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f191ea00700 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 0x557d12484a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f191ea00700 1 -- 172.21.15.67:0/328872901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d12487df0 msgr2=0x557d12485000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f191ea00700 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d12487df0 0x557d12485000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f191ea00700 1 -- 172.21.15.67:0/328872901 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1910009c20 con 0x557d1247c5d0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f191ea00700 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d12487700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f191001c040 tx=0x7f1910005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da895565f2695223 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f1908c00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f1908c00700 1 -- 172.21.15.67:0/328872901 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191000e470 con 0x557d1247c5d0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f1908c00700 1 -- 172.21.15.67:0/328872901 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f191000e5d0 con 0x557d1247c5d0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.405+0000 7f19396cab80 1 -- 172.21.15.67:0/328872901 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d1238f380 con 0x557d1247c5d0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f1908c00700 1 -- 172.21.15.67:0/328872901 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f191000e790 con 0x557d1247c5d0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f19396cab80 10 client.26038 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f19396cab80 1 -- 172.21.15.67:0/328872901 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557d121fcb60 con 0x557d1247c5d0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f1908c00700 1 -- 172.21.15.67:0/328872901 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d1238f380 con 0x557d1247c5d0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f1908c00700 1 -- 172.21.15.67:0/328872901 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1910004870 con 0x557d1247c5d0 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f19396cab80 10 client.26038 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f19396cab80 10 client.26038 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f19396cab80 20 client.26038 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:32.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f19396cab80 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557d12497750 0x557d12499b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f19396cab80 4 client.26038 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f19396cab80 1 -- 172.21.15.67:0/328872901 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d123c7700 con 0x557d12497750 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.406+0000 7f191e000700 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557d12497750 0x557d12499b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.407+0000 7f191e000700 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557d12497750 0x557d12499b70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9bc12c2f5f757f8a server_cookie=9ab45087fd57498b in_seq=0 out_seq=0 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.407+0000 7f1908c00700 10 client.26038 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f1908c00700 1 -- 172.21.15.67:0/328872901 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d123c7700 con 0x557d12497750 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f1908c00700 10 client.26038 handle_command_reply: tid=0 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 client.26038 shutdown 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 -- 172.21.15.67:0/328872901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 msgr2=0x557d12487700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d12487700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f191001c040 tx=0x7f1910005b00 comp rx=0 tx=0).stop 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 -- 172.21.15.67:0/328872901 shutdown_connections 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557d12497750 0x557d12499b70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d12487df0 0x557d12485000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d1247c5d0 0x557d12487700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 --2- 172.21.15.67:0/328872901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 0x557d12484a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f1908c00700 0 client.26038 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:38:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 -- 172.21.15.67:0/328872901 >> 172.21.15.67:0/328872901 conn(0x557d123dc6a0 msgr2=0x557d1247e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 -- 172.21.15.67:0/328872901 shutdown_connections 2024-03-30T02:38:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 1 -- 172.21.15.67:0/328872901 wait complete. 2024-03-30T02:38:32.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.408+0000 7f19396cab80 20 client.26038 trim_cache size 0 max 16384 2024-03-30T02:38:32.416 INFO:tasks.workunit.client.0.smithi067.stderr: 50K .......... .......... .......... .......... .......... 0% 1.38M 2m25s 2024-03-30T02:38:32.416 INFO:tasks.workunit.client.0.smithi067.stderr: 100K .......... .......... .......... .......... .......... 0% 66.3M 97s 2024-03-30T02:38:32.416 INFO:tasks.workunit.client.0.smithi067.stderr: 150K .......... .......... .......... .......... .......... 0% 106M 73s 2024-03-30T02:38:32.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:32 smithi067 ceph-mon[37808]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-30T02:38:32.417 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:32 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/731246877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:38:32.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.423+0000 7f19396cab80 1 Processor -- start 2024-03-30T02:38:32.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.423+0000 7f19396cab80 1 -- start start 2024-03-30T02:38:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.423+0000 7f19396cab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247c5d0 0x557d12484fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.423+0000 7f19396cab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 0x557d1248eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.423+0000 7f19396cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d124715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.423+0000 7f19396cab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1248efe0 con 0x557d1247c5d0 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.423+0000 7f19396cab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d12482fb0 con 0x557d1247cef0 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.423+0000 7f19396cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d12471ae0 con 0x557d12487df0 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247c5d0 0x557d12484fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247c5d0 0x557d12484fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50344/0 (socket says 172.21.15.67:50344) 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191ea00700 1 -- 172.21.15.67:0/3887180817 learned_addr learned my addr 172.21.15.67:0/3887180817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191f400700 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d124715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191e000700 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 0x557d1248eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191ea00700 1 -- 172.21.15.67:0/3887180817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 msgr2=0x557d124715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191ea00700 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d124715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191ea00700 1 -- 172.21.15.67:0/3887180817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 msgr2=0x557d1248eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191ea00700 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 0x557d1248eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191ea00700 1 -- 172.21.15.67:0/3887180817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d12394ea0 con 0x557d1247c5d0 2024-03-30T02:38:32.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191f400700 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d124715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191e000700 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 0x557d1248eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f191ea00700 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247c5d0 0x557d12484fc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f191001c010 tx=0x7f1910005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9263983a47b2521d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.424+0000 7f1914c00700 1 -- 172.21.15.67:0/3887180817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910011bd0 con 0x557d1247c5d0 2024-03-30T02:38:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f1914c00700 1 -- 172.21.15.67:0/3887180817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f191000ee00 con 0x557d1247c5d0 2024-03-30T02:38:32.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f1914c00700 1 -- 172.21.15.67:0/3887180817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910005520 con 0x557d1247c5d0 2024-03-30T02:38:32.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f19396cab80 1 -- 172.21.15.67:0/3887180817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247c5d0 msgr2=0x557d12484fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f19396cab80 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247c5d0 0x557d12484fc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f191001c010 tx=0x7f1910005a00 comp rx=0 tx=0).stop 2024-03-30T02:38:32.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f19396cab80 1 -- 172.21.15.67:0/3887180817 shutdown_connections 2024-03-30T02:38:32.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f19396cab80 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247c5d0 0x557d12484fc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f19396cab80 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d124715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f19396cab80 1 --2- 172.21.15.67:0/3887180817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247cef0 0x557d1248eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f19396cab80 1 -- 172.21.15.67:0/3887180817 >> 172.21.15.67:0/3887180817 conn(0x557d123dc6a0 msgr2=0x557d1249e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f19396cab80 1 -- 172.21.15.67:0/3887180817 shutdown_connections 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.425+0000 7f19396cab80 1 -- 172.21.15.67:0/3887180817 wait complete. 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.426+0000 7f19396cab80 1 -- start start 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.426+0000 7f19396cab80 1 Processor -- start 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.426+0000 7f19396cab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247c5d0 0x557d1248e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f19396cab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d1248ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f19396cab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d1248caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f19396cab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1248d190 con 0x557d1247cef0 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f19396cab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1248d300 con 0x557d1247c5d0 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f19396cab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1248d4c0 con 0x557d12487df0 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f19396cab80 10 client.0 _ll_register_callbacks cb 0x557d12374470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d1248ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d1248ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50352/0 (socket says 172.21.15.67:50352) 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191e000700 1 -- 172.21.15.67:0/795428460 learned_addr learned my addr 172.21.15.67:0/795428460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191ea00700 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247c5d0 0x557d1248e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191f400700 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d1248caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191e000700 1 -- 172.21.15.67:0/795428460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 msgr2=0x557d1248caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191e000700 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d1248caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191e000700 1 -- 172.21.15.67:0/795428460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247c5d0 msgr2=0x557d1248e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191e000700 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247c5d0 0x557d1248e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191e000700 1 -- 172.21.15.67:0/795428460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d121fcb60 con 0x557d1247cef0 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191f400700 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d1248caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191ea00700 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247c5d0 0x557d1248e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f191e000700 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d1248ea40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f190c005950 tx=0x7f190c09e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dda8ec8983e91ca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.427+0000 7f1908c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f1908c00700 1 -- 172.21.15.67:0/795428460 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c005620 con 0x557d1247cef0 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f1908c00700 1 -- 172.21.15.67:0/795428460 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f190c01f340 con 0x557d1247cef0 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f19396cab80 1 -- 172.21.15.67:0/795428460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f191000ec90 con 0x557d1247cef0 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f1908c00700 1 -- 172.21.15.67:0/795428460 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f190c005780 con 0x557d1247cef0 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f19396cab80 10 client.16008 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:32.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f19396cab80 1 -- 172.21.15.67:0/795428460 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557d12254cd0 con 0x557d1247cef0 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f1908c00700 1 -- 172.21.15.67:0/795428460 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f190c0a6080 con 0x557d1247cef0 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f1908c00700 1 -- 172.21.15.67:0/795428460 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f190c065020 con 0x557d1247cef0 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f19396cab80 10 client.16008 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f19396cab80 10 client.16008 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f19396cab80 20 client.16008 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f19396cab80 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557d124ec720 0x557d124eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f19396cab80 4 client.16008 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.428+0000 7f19396cab80 1 -- 172.21.15.67:0/795428460 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d1221dcf0 con 0x557d124ec720 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.429+0000 7f191ea00700 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557d124ec720 0x557d124eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.429+0000 7f1908c00700 10 client.16008 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.429+0000 7f191ea00700 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557d124ec720 0x557d124eeb40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=43a476ccb6564898 server_cookie=a193bb57cd94a0b8 in_seq=0 out_seq=0 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f1908c00700 1 -- 172.21.15.67:0/795428460 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d1221dcf0 con 0x557d124ec720 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f1908c00700 10 client.16008 handle_command_reply: tid=0 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 client.16008 shutdown 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 -- 172.21.15.67:0/795428460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 msgr2=0x557d1248ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d1248ea40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f190c005950 tx=0x7f190c09e0a0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 -- 172.21.15.67:0/795428460 shutdown_connections 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x557d124ec720 0x557d124eeb40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d12487df0 0x557d1248caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d1247c5d0 0x557d1248e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f1908c00700 0 client.16008 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 --2- 172.21.15.67:0/795428460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557d1247cef0 0x557d1248ea40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 -- 172.21.15.67:0/795428460 >> 172.21.15.67:0/795428460 conn(0x557d123dc6a0 msgr2=0x557d124832b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 -- 172.21.15.67:0/795428460 shutdown_connections 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 1 -- 172.21.15.67:0/795428460 wait complete. 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.430+0000 7f19396cab80 20 client.16008 trim_cache size 0 max 16384 2024-03-30T02:38:32.434 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:38:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 -- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19040a13b0 msgr2=0x7f19040a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19040a13b0 0x7f19040a3870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f19180079a0 tx=0x7f1918003c50 comp rx=0 tx=0).stop 2024-03-30T02:38:32.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 -- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 msgr2=0x7f1920199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 0x7f1920199a90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f190c00e970 tx=0x7f190c00ec80 comp rx=0 tx=0).stop 2024-03-30T02:38:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 -- 172.21.15.67:0/2684570765 shutdown_connections 2024-03-30T02:38:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19040a13b0 0x7f19040a3870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f192019de30 0x7f192019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1920100f60 0x7f1920199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 --2- 172.21.15.67:0/2684570765 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1920106080 0x7f1920199a90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 -- 172.21.15.67:0/2684570765 >> 172.21.15.67:0/2684570765 conn(0x7f19200fc7c0 msgr2=0x7f19200feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 -- 172.21.15.67:0/2684570765 shutdown_connections 2024-03-30T02:38:32.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.431+0000 7f1925c00700 1 -- 172.21.15.67:0/2684570765 wait complete. 2024-03-30T02:38:32.449 INFO:tasks.workunit.client.0.smithi067.stderr: 200K .......... .......... .......... .......... .......... 0% 1.39M 77s 2024-03-30T02:38:32.449 INFO:tasks.workunit.client.0.smithi067.stderr: 250K .......... .......... .......... .......... .......... 0% 34.8M 65s 2024-03-30T02:38:32.449 INFO:tasks.workunit.client.0.smithi067.stderr: 300K .......... .......... .......... .......... .......... 0% 164M 56s 2024-03-30T02:38:32.449 INFO:tasks.workunit.client.0.smithi067.stderr: 350K .......... .......... .......... .......... .......... 0% 235M 49s 2024-03-30T02:38:32.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:38:32.481 INFO:tasks.workunit.client.0.smithi067.stderr: 400K .......... .......... .......... .......... .......... 0% 1.47M 54s 2024-03-30T02:38:32.483 INFO:tasks.workunit.client.0.smithi067.stderr: 450K .......... .......... .......... .......... .......... 0% 56.4M 48s 2024-03-30T02:38:32.483 INFO:tasks.workunit.client.0.smithi067.stderr: 500K .......... .......... .......... .......... .......... 0% 77.7M 44s 2024-03-30T02:38:32.483 INFO:tasks.workunit.client.0.smithi067.stderr: 550K .......... .......... .......... .......... .......... 0% 115M 41s 2024-03-30T02:38:32.483 INFO:tasks.workunit.client.0.smithi067.stderr: 600K .......... .......... .......... .......... .......... 0% 74.9M 38s 2024-03-30T02:38:32.483 INFO:tasks.workunit.client.0.smithi067.stderr: 650K .......... .......... .......... .......... .......... 0% 234M 35s 2024-03-30T02:38:32.526 INFO:tasks.workunit.client.0.smithi067.stderr: 700K .......... .......... .......... .......... .......... 0% 1.45M 39s 2024-03-30T02:38:32.526 INFO:tasks.workunit.client.0.smithi067.stderr: 750K .......... .......... .......... .......... .......... 0% 62.5M 36s 2024-03-30T02:38:32.526 INFO:tasks.workunit.client.0.smithi067.stderr: 800K .......... .......... .......... .......... .......... 0% 52.0M 34s 2024-03-30T02:38:32.526 INFO:tasks.workunit.client.0.smithi067.stderr: 850K .......... .......... .......... .......... .......... 0% 31.5M 33s 2024-03-30T02:38:32.526 INFO:tasks.workunit.client.0.smithi067.stderr: 900K .......... .......... .......... .......... .......... 0% 83.5M 31s 2024-03-30T02:38:32.526 INFO:tasks.workunit.client.0.smithi067.stderr: 950K .......... .......... .......... .......... .......... 0% 33.5M 30s 2024-03-30T02:38:32.526 INFO:tasks.workunit.client.0.smithi067.stderr: 1000K .......... .......... .......... .......... .......... 0% 108M 28s 2024-03-30T02:38:32.562 INFO:tasks.workunit.client.0.smithi067.stderr: 1050K .......... .......... .......... .......... .......... 0% 1.63M 31s 2024-03-30T02:38:32.562 INFO:tasks.workunit.client.0.smithi067.stderr: 1100K .......... .......... .......... .......... .......... 0% 30.5M 29s 2024-03-30T02:38:32.562 INFO:tasks.workunit.client.0.smithi067.stderr: 1150K .......... .......... .......... .......... .......... 0% 31.3M 28s 2024-03-30T02:38:32.562 INFO:tasks.workunit.client.0.smithi067.stderr: 1200K .......... .......... .......... .......... .......... 0% 43.9M 27s 2024-03-30T02:38:32.562 INFO:tasks.workunit.client.0.smithi067.stderr: 1250K .......... .......... .......... .......... .......... 0% 88.2M 26s 2024-03-30T02:38:32.562 INFO:tasks.workunit.client.0.smithi067.stderr: 1300K .......... .......... .......... .......... .......... 0% 40.3M 25s 2024-03-30T02:38:32.599 INFO:tasks.workunit.client.0.smithi067.stderr: 1350K .......... .......... .......... .......... .......... 1% 1.68M 27s 2024-03-30T02:38:32.600 INFO:tasks.workunit.client.0.smithi067.stderr: 1400K .......... .......... .......... .......... .......... 1% 37.2M 27s 2024-03-30T02:38:32.600 INFO:tasks.workunit.client.0.smithi067.stderr: 1450K .......... .......... .......... .......... .......... 1% 61.7M 26s 2024-03-30T02:38:32.600 INFO:tasks.workunit.client.0.smithi067.stderr: 1500K .......... .......... .......... .......... .......... 1% 30.2M 25s 2024-03-30T02:38:32.600 INFO:tasks.workunit.client.0.smithi067.stderr: 1550K .......... .......... .......... .......... .......... 1% 51.5M 24s 2024-03-30T02:38:32.600 INFO:tasks.workunit.client.0.smithi067.stderr: 1600K .......... .......... .......... .......... .......... 1% 81.4M 24s 2024-03-30T02:38:32.600 INFO:tasks.workunit.client.0.smithi067.stderr: 1650K .......... .......... .......... .......... .......... 1% 44.6M 23s 2024-03-30T02:38:32.630 INFO:tasks.workunit.client.0.smithi067.stderr: 1700K .......... .......... .......... .......... .......... 1% 1.67M 25s 2024-03-30T02:38:32.631 INFO:tasks.workunit.client.0.smithi067.stderr: 1750K .......... .......... .......... .......... .......... 1% 133M 24s 2024-03-30T02:38:32.631 INFO:tasks.workunit.client.0.smithi067.stderr: 1800K .......... .......... .......... .......... .......... 1% 28.5M 23s 2024-03-30T02:38:32.631 INFO:tasks.workunit.client.0.smithi067.stderr: 1850K .......... .......... .......... .......... .......... 1% 31.0M 23s 2024-03-30T02:38:32.631 INFO:tasks.workunit.client.0.smithi067.stderr: 1900K .......... .......... .......... .......... .......... 1% 58.4M 22s 2024-03-30T02:38:32.631 INFO:tasks.workunit.client.0.smithi067.stderr: 1950K .......... .......... .......... .......... .......... 1% 100M 22s 2024-03-30T02:38:32.631 INFO:tasks.workunit.client.0.smithi067.stderr: 2000K .......... .......... .......... .......... .......... 1% 42.2M 21s 2024-03-30T02:38:32.631 INFO:tasks.workunit.client.0.smithi067.stderr: 2050K .......... .......... .......... .......... .......... 1% 43.5M 21s 2024-03-30T02:38:32.666 INFO:tasks.workunit.client.0.smithi067.stderr: 2100K .......... .......... .......... .......... .......... 1% 1.71M 22s 2024-03-30T02:38:32.666 INFO:tasks.workunit.client.0.smithi067.stderr: 2150K .......... .......... .......... .......... .......... 1% 44.3M 22s 2024-03-30T02:38:32.667 INFO:tasks.workunit.client.0.smithi067.stderr: 2200K .......... .......... .......... .......... .......... 1% 56.7M 21s 2024-03-30T02:38:32.667 INFO:tasks.workunit.client.0.smithi067.stderr: 2250K .......... .......... .......... .......... .......... 1% 32.3M 21s 2024-03-30T02:38:32.667 INFO:tasks.workunit.client.0.smithi067.stderr: 2300K .......... .......... .......... .......... .......... 1% 51.7M 20s 2024-03-30T02:38:32.667 INFO:tasks.workunit.client.0.smithi067.stderr: 2350K .......... .......... .......... .......... .......... 1% 110M 20s 2024-03-30T02:38:32.667 INFO:tasks.workunit.client.0.smithi067.stderr: 2400K .......... .......... .......... .......... .......... 1% 28.7M 20s 2024-03-30T02:38:32.667 INFO:tasks.workunit.client.0.smithi067.stderr: 2450K .......... .......... .......... .......... .......... 1% 76.4M 19s 2024-03-30T02:38:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.685+0000 7fcabde00700 1 -- 172.21.15.67:0/31305117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 msgr2=0x7fcab810db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.685+0000 7fcabde00700 1 --2- 172.21.15.67:0/31305117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 0x7fcab810db30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcab000b0a0 tx=0x7fcab000b3b0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.685+0000 7fcabde00700 1 -- 172.21.15.67:0/31305117 shutdown_connections 2024-03-30T02:38:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.685+0000 7fcabde00700 1 --2- 172.21.15.67:0/31305117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 0x7fcab810db30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.685+0000 7fcabde00700 1 --2- 172.21.15.67:0/31305117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab8100e90 0x7fcab8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.685+0000 7fcabde00700 1 --2- 172.21.15.67:0/31305117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcab8101840 0x7fcab8105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.685+0000 7fcabde00700 1 -- 172.21.15.67:0/31305117 >> 172.21.15.67:0/31305117 conn(0x7fcab80fc710 msgr2=0x7fcab80feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.685+0000 7fcabde00700 1 -- 172.21.15.67:0/31305117 shutdown_connections 2024-03-30T02:38:32.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.685+0000 7fcabde00700 1 -- 172.21.15.67:0/31305117 wait complete. 2024-03-30T02:38:32.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.686+0000 7fcabde00700 1 Processor -- start 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.686+0000 7fcabde00700 1 -- start start 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcabde00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab8100e90 0x7fcab8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcabde00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 0x7fcab8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcabde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcab819dd30 0x7fcab819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcabde00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab819e7b0 con 0x7fcab8105fb0 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcabde00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab819e920 con 0x7fcab8100e90 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcabde00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab819eae0 con 0x7fcab819dd30 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcaafe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 0x7fcab8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcaafe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 0x7fcab8199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50366/0 (socket says 172.21.15.67:50366) 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcaafe00700 1 -- 172.21.15.67:0/1086471180 learned_addr learned my addr 172.21.15.67:0/1086471180 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcab6a00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab8100e90 0x7fcab8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcab7400700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcab819dd30 0x7fcab819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcaafe00700 1 -- 172.21.15.67:0/1086471180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcab819dd30 msgr2=0x7fcab819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcaafe00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcab819dd30 0x7fcab819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcaafe00700 1 -- 172.21.15.67:0/1086471180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab8100e90 msgr2=0x7fcab8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcaafe00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab8100e90 0x7fcab8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.687+0000 7fcaafe00700 1 -- 172.21.15.67:0/1086471180 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0009d00 con 0x7fcab8105fb0 2024-03-30T02:38:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.688+0000 7fcaafe00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 0x7fcab8199990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000e970 tx=0x7fcaa000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.688+0000 7fcabca00700 1 -- 172.21.15.67:0/1086471180 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0009790 con 0x7fcab8105fb0 2024-03-30T02:38:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.688+0000 7fcabca00700 1 -- 172.21.15.67:0/1086471180 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa000c7e0 con 0x7fcab8105fb0 2024-03-30T02:38:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.688+0000 7fcabca00700 1 -- 172.21.15.67:0/1086471180 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00054e0 con 0x7fcab8105fb0 2024-03-30T02:38:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.688+0000 7fcabde00700 1 -- 172.21.15.67:0/1086471180 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab81a5fe0 con 0x7fcab8105fb0 2024-03-30T02:38:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.688+0000 7fcabde00700 1 -- 172.21.15.67:0/1086471180 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab81a6470 con 0x7fcab8105fb0 2024-03-30T02:38:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:32 smithi170 ceph-mon[78798]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-30T02:38:32.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:32 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/731246877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:38:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.691+0000 7fcabca00700 1 -- 172.21.15.67:0/1086471180 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fcaa000c950 con 0x7fcab8105fb0 2024-03-30T02:38:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.692+0000 7fcad18a9b80 1 Processor -- start 2024-03-30T02:38:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.692+0000 7fcabca00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fca9c0a1420 0x7fca9c0a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.692+0000 7fcabca00700 1 -- 172.21.15.67:0/1086471180 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fcaa0014070 con 0x7fcab8105fb0 2024-03-30T02:38:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcad18a9b80 1 -- start start 2024-03-30T02:38:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcab6a00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fca9c0a1420 0x7fca9c0a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c4a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51c581e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51ba3990 con 0x557c51c4aef0 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51ba1350 con 0x557c51c55df0 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51b9cc40 con 0x557c51c4a5d0 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c4a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcaafe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c4a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60726/0 (socket says 172.21.15.67:60726) 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.693+0000 7fcab6a00700 1 -- 172.21.15.67:0/130323650 learned_addr learned my addr 172.21.15.67:0/130323650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcab7400700 1 --2- 172.21.15.67:0/130323650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51c581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcab6a00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fca9c0a1420 0x7fca9c0a38e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009cb0 tx=0x7fcaa8009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcaafe00700 1 -- 172.21.15.67:0/130323650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 msgr2=0x557c51c4a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcaafe00700 1 --2- 172.21.15.67:0/130323650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c4a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcaafe00700 1 -- 172.21.15.67:0/130323650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 msgr2=0x557c51c581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcaafe00700 1 --2- 172.21.15.67:0/130323650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51c581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcaafe00700 1 -- 172.21.15.67:0/130323650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c51a1e3b0 con 0x557c51c4aef0 2024-03-30T02:38:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcab7400700 1 --2- 172.21.15.67:0/130323650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51c581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:32.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcaafe00700 1 --2- 172.21.15.67:0/130323650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c558b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ca680 tx=0x7fcaa0099bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d8ef02a10413a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcaa7e00700 1 -- 172.21.15.67:0/130323650 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0099f70 con 0x557c51c4aef0 2024-03-30T02:38:32.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcaa7e00700 1 -- 172.21.15.67:0/130323650 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa009dda0 con 0x557c51c4aef0 2024-03-30T02:38:32.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.694+0000 7fcaa7e00700 1 -- 172.21.15.67:0/130323650 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa009a3f0 con 0x557c51c4aef0 2024-03-30T02:38:32.698 INFO:tasks.workunit.client.0.smithi067.stderr: 2500K .......... .......... .......... .......... .......... 1% 1.71M 20s 2024-03-30T02:38:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.695+0000 7fcad18a9b80 1 -- 172.21.15.67:0/130323650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 msgr2=0x557c51c558b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.695+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/130323650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c558b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ca680 tx=0x7fcaa0099bc0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.695+0000 7fcad18a9b80 1 -- 172.21.15.67:0/130323650 shutdown_connections 2024-03-30T02:38:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.695+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/130323650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51c581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.695+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/130323650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c4a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.695+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/130323650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c558b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.695+0000 7fcad18a9b80 1 -- 172.21.15.67:0/130323650 >> 172.21.15.67:0/130323650 conn(0x557c51baa6a0 msgr2=0x557c51c46a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.695+0000 7fcad18a9b80 1 -- 172.21.15.67:0/130323650 shutdown_connections 2024-03-30T02:38:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.695+0000 7fcad18a9b80 1 -- 172.21.15.67:0/130323650 wait complete. 2024-03-30T02:38:32.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.696+0000 7fcad18a9b80 1 -- start start 2024-03-30T02:38:32.701 INFO:tasks.workunit.client.0.smithi067.stderr: 2550K .......... .......... .......... .......... .......... 1% 34.3M 20s 2024-03-30T02:38:32.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.696+0000 7fcad18a9b80 1 Processor -- start 2024-03-30T02:38:32.702 INFO:tasks.workunit.client.0.smithi067.stderr: 2600K .......... .......... .......... .......... .......... 1% 88.9M 20s 2024-03-30T02:38:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4a5d0 0x557c51c55800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4aef0 0x557c51c52b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51c53150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51b38d80 con 0x557c51c4a5d0 2024-03-30T02:38:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51b38a20 con 0x557c51c55df0 2024-03-30T02:38:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51a206a0 con 0x557c51c4aef0 2024-03-30T02:38:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcad18a9b80 10 client.0 _ll_register_callbacks cb 0x557c51b42470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4a5d0 0x557c51c55800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4a5d0 0x557c51c55800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50382/0 (socket says 172.21.15.67:50382) 2024-03-30T02:38:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcab6a00700 1 -- 172.21.15.67:0/1963736312 learned_addr learned my addr 172.21.15.67:0/1963736312 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcab7400700 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51c53150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.697+0000 7fcaafe00700 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4aef0 0x557c51c52b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcab6a00700 1 -- 172.21.15.67:0/1963736312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4aef0 msgr2=0x557c51c52b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcab6a00700 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4aef0 0x557c51c52b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcab6a00700 1 -- 172.21.15.67:0/1963736312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 msgr2=0x557c51c53150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcab6a00700 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51c53150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcab6a00700 1 -- 172.21.15.67:0/1963736312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab81a6470 con 0x557c51c4a5d0 2024-03-30T02:38:32.706 INFO:tasks.workunit.client.0.smithi067.stderr: 2650K .......... .......... .......... .......... .......... 1% 34.8M 19s 2024-03-30T02:38:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcab6a00700 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4a5d0 0x557c51c55800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80042c0 tx=0x7fcaa80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0ba7f65e53034e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcaa4c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcaa4c00700 1 -- 172.21.15.67:0/1963736312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8003d20 con 0x557c51c4a5d0 2024-03-30T02:38:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcaa4c00700 1 -- 172.21.15.67:0/1963736312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fcaa8025410 con 0x557c51c4a5d0 2024-03-30T02:38:32.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcad18a9b80 1 -- 172.21.15.67:0/1963736312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c51b5d380 con 0x557c51c4a5d0 2024-03-30T02:38:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.698+0000 7fcaa4c00700 1 -- 172.21.15.67:0/1963736312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaa8003e80 con 0x557c51c4a5d0 2024-03-30T02:38:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcad18a9b80 10 client.16020 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcad18a9b80 1 -- 172.21.15.67:0/1963736312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557c519cab60 con 0x557c51c4a5d0 2024-03-30T02:38:32.709 INFO:tasks.workunit.client.0.smithi067.stderr: 2700K .......... .......... .......... .......... .......... 2% 53.2M 19s 2024-03-30T02:38:32.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcaa4c00700 1 -- 172.21.15.67:0/1963736312 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c51b5d380 con 0x557c51c4a5d0 2024-03-30T02:38:32.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcaa4c00700 1 -- 172.21.15.67:0/1963736312 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcaa801c500 con 0x557c51c4a5d0 2024-03-30T02:38:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcad18a9b80 10 client.16020 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcad18a9b80 10 client.16020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:38:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcad18a9b80 20 client.16020 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:32.711 INFO:tasks.workunit.client.0.smithi067.stderr: 2750K .......... .......... .......... .......... .......... 2% 108M 19s 2024-03-30T02:38:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557c51c65a40 0x557c51c67e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcad18a9b80 4 client.16020 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.699+0000 7fcad18a9b80 1 -- 172.21.15.67:0/1963736312 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c51b95700 con 0x557c51c65a40 2024-03-30T02:38:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.700+0000 7fcaafe00700 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557c51c65a40 0x557c51c67e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.700+0000 7fcaa4c00700 10 client.16020 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:38:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.700+0000 7fcaafe00700 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557c51c65a40 0x557c51c67e60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebcac7a66ccbada1 server_cookie=acf215f3df38f31f in_seq=0 out_seq=0 2024-03-30T02:38:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcaa4c00700 1 -- 172.21.15.67:0/1963736312 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557c51b95700 con 0x557c51c65a40 2024-03-30T02:38:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcaa4c00700 10 client.16020 handle_command_reply: tid=0 2024-03-30T02:38:32.715 INFO:tasks.workunit.client.0.smithi067.stderr: 2800K .......... .......... .......... .......... .......... 2% 32.1M 19s 2024-03-30T02:38:32.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 client.16020 shutdown 2024-03-30T02:38:32.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 -- 172.21.15.67:0/1963736312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4a5d0 msgr2=0x557c51c55800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4a5d0 0x557c51c55800 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80042c0 tx=0x7fcaa80047c0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 -- 172.21.15.67:0/1963736312 shutdown_connections 2024-03-30T02:38:32.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557c51c65a40 0x557c51c67e60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcaa4c00700 0 client.16020 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:38:32.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4a5d0 0x557c51c55800 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51c53150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/1963736312 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4aef0 0x557c51c52b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 -- 172.21.15.67:0/1963736312 >> 172.21.15.67:0/1963736312 conn(0x557c51baa6a0 msgr2=0x557c51c4c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.718 INFO:tasks.workunit.client.0.smithi067.stderr: 2850K .......... .......... .......... .......... .......... 2% 79.9M 18s 2024-03-30T02:38:32.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.701+0000 7fcad18a9b80 1 -- 172.21.15.67:0/1963736312 shutdown_connections 2024-03-30T02:38:32.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.702+0000 7fcad18a9b80 1 -- 172.21.15.67:0/1963736312 wait complete. 2024-03-30T02:38:32.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.702+0000 7fcad18a9b80 20 client.16020 trim_cache size 0 max 16384 2024-03-30T02:38:32.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.715+0000 7fcad18a9b80 1 Processor -- start 2024-03-30T02:38:32.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.715+0000 7fcad18a9b80 1 -- start start 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.715+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c4a5d0 0x557c51c45f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c55df0 0x557c51c4de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51c681f0 con 0x557c51c4aef0 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51c53a30 con 0x557c51c4a5d0 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51c5cfb0 con 0x557c51c55df0 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcaafe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcaafe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50394/0 (socket says 172.21.15.67:50394) 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcaafe00700 1 -- 172.21.15.67:0/4206261305 learned_addr learned my addr 172.21.15.67:0/4206261305 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcab6a00700 1 --2- 172.21.15.67:0/4206261305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c4a5d0 0x557c51c45f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcab7400700 1 --2- 172.21.15.67:0/4206261305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c55df0 0x557c51c4de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcaafe00700 1 -- 172.21.15.67:0/4206261305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c55df0 msgr2=0x557c51c4de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcaafe00700 1 --2- 172.21.15.67:0/4206261305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c55df0 0x557c51c4de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcaafe00700 1 -- 172.21.15.67:0/4206261305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c4a5d0 msgr2=0x557c51c45f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcaafe00700 1 --2- 172.21.15.67:0/4206261305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c4a5d0 0x557c51c45f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcaafe00700 1 -- 172.21.15.67:0/4206261305 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c51b62ea0 con 0x557c51c4aef0 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.716+0000 7fcaafe00700 1 --2- 172.21.15.67:0/4206261305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5ca70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ca6b0 tx=0x7fcaa000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92551fb5dc0d46d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcaad200700 1 -- 172.21.15.67:0/4206261305 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00b8040 con 0x557c51c4aef0 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcaad200700 1 -- 172.21.15.67:0/4206261305 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa0099f70 con 0x557c51c4aef0 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcaad200700 1 -- 172.21.15.67:0/4206261305 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0099460 con 0x557c51c4aef0 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcad18a9b80 1 -- 172.21.15.67:0/4206261305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 msgr2=0x557c51c5ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/4206261305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5ca70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ca6b0 tx=0x7fcaa000cf50 comp rx=0 tx=0).stop 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcad18a9b80 1 -- 172.21.15.67:0/4206261305 shutdown_connections 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/4206261305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c55df0 0x557c51c4de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/4206261305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c4a5d0 0x557c51c45f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/4206261305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5ca70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcad18a9b80 1 -- 172.21.15.67:0/4206261305 >> 172.21.15.67:0/4206261305 conn(0x557c51baa6a0 msgr2=0x557c51c3f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcad18a9b80 1 -- 172.21.15.67:0/4206261305 shutdown_connections 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.717+0000 7fcad18a9b80 1 -- 172.21.15.67:0/4206261305 wait complete. 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.718+0000 7fcad18a9b80 1 -- start start 2024-03-30T02:38:32.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.718+0000 7fcad18a9b80 1 Processor -- start 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.718+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c5ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.718+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.718+0000 7fcad18a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51cb28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.718+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51cb2df0 con 0x557c51c4aef0 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.718+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51cb2f60 con 0x557c51c55df0 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.718+0000 7fcad18a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c51cb3120 con 0x557c51c4a5d0 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.718+0000 7fcad18a9b80 10 client.0 _ll_register_callbacks cb 0x557c51b42470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcaafe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c5ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcab6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c5ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60770/0 (socket says 172.21.15.67:60770) 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcab6a00700 1 -- 172.21.15.67:0/2720746032 learned_addr learned my addr 172.21.15.67:0/2720746032 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcab7400700 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51cb28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcaafe00700 1 -- 172.21.15.67:0/2720746032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 msgr2=0x557c51c5ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcaafe00700 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c5ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcaafe00700 1 -- 172.21.15.67:0/2720746032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 msgr2=0x557c51cb28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcaafe00700 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51cb28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcaafe00700 1 -- 172.21.15.67:0/2720746032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c51a1e3b0 con 0x557c51c4aef0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcab6a00700 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c5ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcaafe00700 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5b1a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ca680 tx=0x7fcaa0016dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc581cd98855448 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.719+0000 7fcaa4c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcaa4c00700 1 -- 172.21.15.67:0/2720746032 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0016ee0 con 0x557c51c4aef0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcaa4c00700 1 -- 172.21.15.67:0/2720746032 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fcaa00ade40 con 0x557c51c4aef0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcad18a9b80 1 -- 172.21.15.67:0/2720746032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaa009dda0 con 0x557c51c4aef0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcaa4c00700 1 -- 172.21.15.67:0/2720746032 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaa00b8290 con 0x557c51c4aef0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcad18a9b80 10 client.16026 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcad18a9b80 1 -- 172.21.15.67:0/2720746032 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557c51a22cd0 con 0x557c51c4aef0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcaa4c00700 1 -- 172.21.15.67:0/2720746032 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa009dda0 con 0x557c51c4aef0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcaa4c00700 1 -- 172.21.15.67:0/2720746032 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcaa0099580 con 0x557c51c4aef0 2024-03-30T02:38:32.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcad18a9b80 10 client.16026 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcad18a9b80 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcad18a9b80 20 client.16026 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557c51cba4c0 0x557c51cbc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcad18a9b80 4 client.16026 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.720+0000 7fcad18a9b80 1 -- 172.21.15.67:0/2720746032 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c519ebcf0 con 0x557c51cba4c0 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.721+0000 7fcab6a00700 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557c51cba4c0 0x557c51cbc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.721+0000 7fcab6a00700 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557c51cba4c0 0x557c51cbc8b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ee7d01c4e828828 server_cookie=b6ecfb53251f2977 in_seq=0 out_seq=0 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.721+0000 7fcaa4c00700 10 client.16026 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcaa4c00700 1 -- 172.21.15.67:0/2720746032 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c519ebcf0 con 0x557c51cba4c0 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcaa4c00700 10 client.16026 handle_command_reply: tid=0 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 client.16026 shutdown 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 -- 172.21.15.67:0/2720746032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 msgr2=0x557c51c5b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5b1a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00ca680 tx=0x7fcaa0016dd0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 -- 172.21.15.67:0/2720746032 shutdown_connections 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557c51cba4c0 0x557c51cbc8b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c51c55df0 0x557c51cb28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557c51c4a5d0 0x557c51c5ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcaa4c00700 0 client.16026 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:38:32.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 --2- 172.21.15.67:0/2720746032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557c51c4aef0 0x557c51c5b1a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 -- 172.21.15.67:0/2720746032 >> 172.21.15.67:0/2720746032 conn(0x557c51baa6a0 msgr2=0x557c51c4de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 -- 172.21.15.67:0/2720746032 shutdown_connections 2024-03-30T02:38:32.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 1 -- 172.21.15.67:0/2720746032 wait complete. 2024-03-30T02:38:32.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.722+0000 7fcad18a9b80 20 client.16026 trim_cache size 0 max 16384 2024-03-30T02:38:32.729 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:38:32.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 -- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fca9c0a1420 msgr2=0x7fca9c0a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fca9c0a1420 0x7fca9c0a38e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009cb0 tx=0x7fcaa8009260 comp rx=0 tx=0).stop 2024-03-30T02:38:32.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 -- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 msgr2=0x7fcab8199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 0x7fcab8199990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000e970 tx=0x7fcaa000ec80 comp rx=0 tx=0).stop 2024-03-30T02:38:32.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 -- 172.21.15.67:0/1086471180 shutdown_connections 2024-03-30T02:38:32.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fca9c0a1420 0x7fca9c0a38e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcab819dd30 0x7fcab819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab8100e90 0x7fcab8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 --2- 172.21.15.67:0/1086471180 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcab8105fb0 0x7fcab8199990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 -- 172.21.15.67:0/1086471180 >> 172.21.15.67:0/1086471180 conn(0x7fcab80fc710 msgr2=0x7fcab80fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 -- 172.21.15.67:0/1086471180 shutdown_connections 2024-03-30T02:38:32.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.723+0000 7fcabde00700 1 -- 172.21.15.67:0/1086471180 wait complete. 2024-03-30T02:38:32.742 INFO:tasks.workunit.client.0.smithi067.stderr: 2900K .......... .......... .......... .......... .......... 2% 1.72M 19s 2024-03-30T02:38:32.742 INFO:tasks.workunit.client.0.smithi067.stderr: 2950K .......... .......... .......... .......... .......... 2% 46.3M 19s 2024-03-30T02:38:32.742 INFO:tasks.workunit.client.0.smithi067.stderr: 3000K .......... .......... .......... .......... .......... 2% 53.8M 19s 2024-03-30T02:38:32.743 INFO:tasks.workunit.client.0.smithi067.stderr: 3050K .......... .......... .......... .......... .......... 2% 34.3M 18s 2024-03-30T02:38:32.743 INFO:tasks.workunit.client.0.smithi067.stderr: 3100K .......... .......... .......... .......... .......... 2% 52.6M 18s 2024-03-30T02:38:32.743 INFO:tasks.workunit.client.0.smithi067.stderr: 3150K .......... .......... .......... .......... .......... 2% 107M 18s 2024-03-30T02:38:32.743 INFO:tasks.workunit.client.0.smithi067.stderr: 3200K .......... .......... .......... .......... .......... 2% 68.8M 18s 2024-03-30T02:38:32.743 INFO:tasks.workunit.client.0.smithi067.stderr: 3250K .......... .......... .......... .......... .......... 2% 40.4M 17s 2024-03-30T02:38:32.743 INFO:tasks.workunit.client.0.smithi067.stderr: 3300K .......... .......... .......... .......... .......... 2% 98.1M 17s 2024-03-30T02:38:32.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:38:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:32 smithi082 ceph-mon[79176]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-30T02:38:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:32 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/731246877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:38:32.767 INFO:tasks.workunit.client.0.smithi067.stderr: 3350K .......... .......... .......... .......... .......... 2% 1.72M 18s 2024-03-30T02:38:32.778 INFO:tasks.workunit.client.0.smithi067.stderr: 3400K .......... .......... .......... .......... .......... 2% 36.0M 18s 2024-03-30T02:38:32.778 INFO:tasks.workunit.client.0.smithi067.stderr: 3450K .......... .......... .......... .......... .......... 2% 85.4M 18s 2024-03-30T02:38:32.778 INFO:tasks.workunit.client.0.smithi067.stderr: 3500K .......... .......... .......... .......... .......... 2% 34.6M 17s 2024-03-30T02:38:32.779 INFO:tasks.workunit.client.0.smithi067.stderr: 3550K .......... .......... .......... .......... .......... 2% 48.5M 17s 2024-03-30T02:38:32.779 INFO:tasks.workunit.client.0.smithi067.stderr: 3600K .......... .......... .......... .......... .......... 2% 112M 17s 2024-03-30T02:38:32.779 INFO:tasks.workunit.client.0.smithi067.stderr: 3650K .......... .......... .......... .......... .......... 2% 71.5M 17s 2024-03-30T02:38:32.779 INFO:tasks.workunit.client.0.smithi067.stderr: 3700K .......... .......... .......... .......... .......... 2% 45.3M 17s 2024-03-30T02:38:32.779 INFO:tasks.workunit.client.0.smithi067.stderr: 3750K .......... .......... .......... .......... .......... 2% 97.2M 16s 2024-03-30T02:38:32.814 INFO:tasks.workunit.client.0.smithi067.stderr: 3800K .......... .......... .......... .......... .......... 2% 1.72M 17s 2024-03-30T02:38:32.814 INFO:tasks.workunit.client.0.smithi067.stderr: 3850K .......... .......... .......... .......... .......... 2% 37.2M 17s 2024-03-30T02:38:32.814 INFO:tasks.workunit.client.0.smithi067.stderr: 3900K .......... .......... .......... .......... .......... 2% 93.4M 17s 2024-03-30T02:38:32.814 INFO:tasks.workunit.client.0.smithi067.stderr: 3950K .......... .......... .......... .......... .......... 2% 34.9M 17s 2024-03-30T02:38:32.814 INFO:tasks.workunit.client.0.smithi067.stderr: 4000K .......... .......... .......... .......... .......... 2% 30.7M 16s 2024-03-30T02:38:32.815 INFO:tasks.workunit.client.0.smithi067.stderr: 4050K .......... .......... .......... .......... .......... 3% 106M 16s 2024-03-30T02:38:32.815 INFO:tasks.workunit.client.0.smithi067.stderr: 4100K .......... .......... .......... .......... .......... 3% 48.6M 16s 2024-03-30T02:38:32.815 INFO:tasks.workunit.client.0.smithi067.stderr: 4150K .......... .......... .......... .......... .......... 3% 48.2M 16s 2024-03-30T02:38:32.815 INFO:tasks.workunit.client.0.smithi067.stderr: 4200K .......... .......... .......... .......... .......... 3% 51.1M 16s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4250K .......... .......... .......... .......... .......... 3% 1.81M 16s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4300K .......... .......... .......... .......... .......... 3% 43.4M 16s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4350K .......... .......... .......... .......... .......... 3% 87.0M 16s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4400K .......... .......... .......... .......... .......... 3% 80.0M 16s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4450K .......... .......... .......... .......... .......... 3% 26.0M 16s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4500K .......... .......... .......... .......... .......... 3% 31.8M 16s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4550K .......... .......... .......... .......... .......... 3% 47.9M 15s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4600K .......... .......... .......... .......... .......... 3% 49.0M 15s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4650K .......... .......... .......... .......... .......... 3% 34.2M 15s 2024-03-30T02:38:32.854 INFO:tasks.workunit.client.0.smithi067.stderr: 4700K .......... .......... .......... .......... .......... 3% 85.2M 15s 2024-03-30T02:38:32.888 INFO:tasks.workunit.client.0.smithi067.stderr: 4750K .......... .......... .......... .......... .......... 3% 1.86M 16s 2024-03-30T02:38:32.888 INFO:tasks.workunit.client.0.smithi067.stderr: 4800K .......... .......... .......... .......... .......... 3% 43.9M 15s 2024-03-30T02:38:32.888 INFO:tasks.workunit.client.0.smithi067.stderr: 4850K .......... .......... .......... .......... .......... 3% 51.8M 15s 2024-03-30T02:38:32.888 INFO:tasks.workunit.client.0.smithi067.stderr: 4900K .......... .......... .......... .......... .......... 3% 47.6M 15s 2024-03-30T02:38:32.888 INFO:tasks.workunit.client.0.smithi067.stderr: 4950K .......... .......... .......... .......... .......... 3% 21.6M 15s 2024-03-30T02:38:32.888 INFO:tasks.workunit.client.0.smithi067.stderr: 5000K .......... .......... .......... .......... .......... 3% 23.3M 15s 2024-03-30T02:38:32.888 INFO:tasks.workunit.client.0.smithi067.stderr: 5050K .......... .......... .......... .......... .......... 3% 114M 15s 2024-03-30T02:38:32.888 INFO:tasks.workunit.client.0.smithi067.stderr: 5100K .......... .......... .......... .......... .......... 3% 30.4M 15s 2024-03-30T02:38:32.888 INFO:tasks.workunit.client.0.smithi067.stderr: 5150K .......... .......... .......... .......... .......... 3% 26.1M 15s 2024-03-30T02:38:32.889 INFO:tasks.workunit.client.0.smithi067.stderr: 5200K .......... .......... .......... .......... .......... 3% 30.9M 14s 2024-03-30T02:38:32.889 INFO:tasks.workunit.client.0.smithi067.stderr: 5250K .......... .......... .......... .......... .......... 3% 37.5M 14s 2024-03-30T02:38:32.911 INFO:tasks.workunit.client.0.smithi067.stderr: 5300K .......... .......... .......... .......... .......... 3% 2.01M 15s 2024-03-30T02:38:32.914 INFO:tasks.workunit.client.0.smithi067.stderr: 5350K .......... .......... .......... .......... .......... 3% 25.3M 15s 2024-03-30T02:38:32.914 INFO:tasks.workunit.client.0.smithi067.stderr: 5400K .......... .......... .......... .......... .......... 4% 87.8M 15s 2024-03-30T02:38:32.916 INFO:tasks.workunit.client.0.smithi067.stderr: 5450K .......... .......... .......... .......... .......... 4% 24.2M 15s 2024-03-30T02:38:32.921 INFO:tasks.workunit.client.0.smithi067.stderr: 5500K .......... .......... .......... .......... .......... 4% 29.8M 14s 2024-03-30T02:38:32.921 INFO:tasks.workunit.client.0.smithi067.stderr: 5550K .......... .......... .......... .......... .......... 4% 45.8M 14s 2024-03-30T02:38:32.922 INFO:tasks.workunit.client.0.smithi067.stderr: 5600K .......... .......... .......... .......... .......... 4% 40.2M 14s 2024-03-30T02:38:32.922 INFO:tasks.workunit.client.0.smithi067.stderr: 5650K .......... .......... .......... .......... .......... 4% 41.2M 14s 2024-03-30T02:38:32.922 INFO:tasks.workunit.client.0.smithi067.stderr: 5700K .......... .......... .......... .......... .......... 4% 33.9M 14s 2024-03-30T02:38:32.925 INFO:tasks.workunit.client.0.smithi067.stderr: 5750K .......... .......... .......... .......... .......... 4% 40.2M 14s 2024-03-30T02:38:32.925 INFO:tasks.workunit.client.0.smithi067.stderr: 5800K .......... .......... .......... .......... .......... 4% 32.6M 14s 2024-03-30T02:38:32.946 INFO:tasks.workunit.client.0.smithi067.stderr: 5850K .......... .......... .......... .......... .......... 4% 2.25M 14s 2024-03-30T02:38:32.952 INFO:tasks.workunit.client.0.smithi067.stderr: 5900K .......... .......... .......... .......... .......... 4% 22.4M 14s 2024-03-30T02:38:32.952 INFO:tasks.workunit.client.0.smithi067.stderr: 5950K .......... .......... .......... .......... .......... 4% 106M 14s 2024-03-30T02:38:32.952 INFO:tasks.workunit.client.0.smithi067.stderr: 6000K .......... .......... .......... .......... .......... 4% 29.9M 14s 2024-03-30T02:38:32.952 INFO:tasks.workunit.client.0.smithi067.stderr: 6050K .......... .......... .......... .......... .......... 4% 39.6M 14s 2024-03-30T02:38:32.955 INFO:tasks.workunit.client.0.smithi067.stderr: 6100K .......... .......... .......... .......... .......... 4% 30.8M 14s 2024-03-30T02:38:32.955 INFO:tasks.workunit.client.0.smithi067.stderr: 6150K .......... .......... .......... .......... .......... 4% 73.2M 14s 2024-03-30T02:38:32.960 INFO:tasks.workunit.client.0.smithi067.stderr: 6200K .......... .......... .......... .......... .......... 4% 27.1M 14s 2024-03-30T02:38:32.961 INFO:tasks.workunit.client.0.smithi067.stderr: 6250K .......... .......... .......... .......... .......... 4% 35.3M 13s 2024-03-30T02:38:32.961 INFO:tasks.workunit.client.0.smithi067.stderr: 6300K .......... .......... .......... .......... .......... 4% 45.7M 13s 2024-03-30T02:38:32.961 INFO:tasks.workunit.client.0.smithi067.stderr: 6350K .......... .......... .......... .......... .......... 4% 76.6M 13s 2024-03-30T02:38:32.961 INFO:tasks.workunit.client.0.smithi067.stderr: 6400K .......... .......... .......... .......... .......... 4% 32.5M 13s 2024-03-30T02:38:32.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.972+0000 7fd422600700 1 -- 172.21.15.67:0/2415165530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c069260 msgr2=0x7fd41c069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.972+0000 7fd422600700 1 --2- 172.21.15.67:0/2415165530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c069260 0x7fd41c069640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0099c0 tx=0x7fd40c009cd0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.972+0000 7fd422600700 1 -- 172.21.15.67:0/2415165530 shutdown_connections 2024-03-30T02:38:32.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.972+0000 7fd422600700 1 --2- 172.21.15.67:0/2415165530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1060b0 0x7fd41c10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.972+0000 7fd422600700 1 --2- 172.21.15.67:0/2415165530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c069260 0x7fd41c069640 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.972+0000 7fd422600700 1 --2- 172.21.15.67:0/2415165530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c069c10 0x7fd41c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.972+0000 7fd422600700 1 -- 172.21.15.67:0/2415165530 >> 172.21.15.67:0/2415165530 conn(0x7fd41c076d80 msgr2=0x7fd41c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.972+0000 7fd422600700 1 -- 172.21.15.67:0/2415165530 shutdown_connections 2024-03-30T02:38:32.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.972+0000 7fd422600700 1 -- 172.21.15.67:0/2415165530 wait complete. 2024-03-30T02:38:32.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.973+0000 7fd422600700 1 Processor -- start 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.973+0000 7fd422600700 1 -- start start 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.973+0000 7fd422600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c069260 0x7fd41c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd422600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1060b0 0x7fd41c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd422600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c19dd30 0x7fd41c19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd422600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19e7b0 con 0x7fd41c1060b0 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd422600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19e920 con 0x7fd41c19dd30 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd422600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19eae0 con 0x7fd41c069260 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1060b0 0x7fd41c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1060b0 0x7fd41c199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50408/0 (socket says 172.21.15.67:50408) 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41aa00700 1 -- 172.21.15.67:0/393359358 learned_addr learned my addr 172.21.15.67:0/393359358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41b400700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c069260 0x7fd41c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41be00700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c19dd30 0x7fd41c19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41aa00700 1 -- 172.21.15.67:0/393359358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c069260 msgr2=0x7fd41c199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41aa00700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c069260 0x7fd41c199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41aa00700 1 -- 172.21.15.67:0/393359358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c19dd30 msgr2=0x7fd41c19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41aa00700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c19dd30 0x7fd41c19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41aa00700 1 -- 172.21.15.67:0/393359358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0095f0 con 0x7fd41c1060b0 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.974+0000 7fd41be00700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c19dd30 0x7fd41c19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.975+0000 7fd41aa00700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1060b0 0x7fd41c199990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e970 tx=0x7fd40800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.975+0000 7fd413e00700 1 -- 172.21.15.67:0/393359358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408009790 con 0x7fd41c1060b0 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.975+0000 7fd422600700 1 -- 172.21.15.67:0/393359358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c1a5fe0 con 0x7fd41c1060b0 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.975+0000 7fd413e00700 1 -- 172.21.15.67:0/393359358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40800eec0 con 0x7fd41c1060b0 2024-03-30T02:38:32.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.975+0000 7fd413e00700 1 -- 172.21.15.67:0/393359358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408005590 con 0x7fd41c1060b0 2024-03-30T02:38:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.975+0000 7fd422600700 1 -- 172.21.15.67:0/393359358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c1a6510 con 0x7fd41c1060b0 2024-03-30T02:38:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.978+0000 7fd413e00700 1 -- 172.21.15.67:0/393359358 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fd40800cbd0 con 0x7fd41c1060b0 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.979+0000 7fd4360bbb80 1 Processor -- start 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.979+0000 7fd4360bbb80 1 -- start start 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.979+0000 7fd413e00700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4000a1430 0x7fd4000a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb269b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 0x55daceb341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacea7f990 con 0x55daceb26ef0 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacea7d350 con 0x55daceb265d0 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacea78c40 con 0x55daceb31df0 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd413e00700 1 -- 172.21.15.67:0/393359358 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd408014070 con 0x7fd41c1060b0 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41be00700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4000a1430 0x7fd4000a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb269b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38268/0 (socket says 172.21.15.67:38268) 2024-03-30T02:38:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41b400700 1 -- 172.21.15.67:0/1244706682 learned_addr learned my addr 172.21.15.67:0/1244706682 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41be00700 1 --2- 172.21.15.67:0/1244706682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 0x55daceb341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41b400700 1 -- 172.21.15.67:0/1244706682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 msgr2=0x55daceb341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41b400700 1 --2- 172.21.15.67:0/1244706682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 0x55daceb341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41b400700 1 -- 172.21.15.67:0/1244706682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 msgr2=0x55daceb318b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41b400700 1 --2- 172.21.15.67:0/1244706682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb318b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41b400700 1 -- 172.21.15.67:0/1244706682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dace8fa3b0 con 0x55daceb265d0 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.980+0000 7fd41aa00700 1 --2- 172.21.15.67:0/1244706682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.981+0000 7fd41be00700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4000a1430 0x7fd4000a38f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd414015e20 tx=0x7fd414015420 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.981+0000 7fd41b400700 1 --2- 172.21.15.67:0/1244706682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb269b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0099c0 tx=0x7fd40c005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60af7ed9c0df141 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.981+0000 7fd410c00700 1 -- 172.21.15.67:0/1244706682 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01c050 con 0x55daceb265d0 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.981+0000 7fd410c00700 1 -- 172.21.15.67:0/1244706682 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c01d040 con 0x55daceb265d0 2024-03-30T02:38:32.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.981+0000 7fd410c00700 1 -- 172.21.15.67:0/1244706682 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0036c0 con 0x55daceb265d0 2024-03-30T02:38:32.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.982+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1244706682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 msgr2=0x55daceb269b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:32.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.982+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/1244706682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb269b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0099c0 tx=0x7fd40c005fb0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.982+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1244706682 shutdown_connections 2024-03-30T02:38:32.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.982+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/1244706682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 0x55daceb341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.982+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/1244706682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb269b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.982+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/1244706682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb318b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.982+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1244706682 >> 172.21.15.67:0/1244706682 conn(0x55dacea866a0 msgr2=0x55daceb22a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:32.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.982+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1244706682 shutdown_connections 2024-03-30T02:38:32.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.982+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1244706682 wait complete. 2024-03-30T02:38:32.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.983+0000 7fd4360bbb80 1 -- start start 2024-03-30T02:38:32.986 INFO:tasks.workunit.client.0.smithi067.stderr: 6450K .......... .......... .......... .......... .......... 4% 2.19M 14s 2024-03-30T02:38:32.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.983+0000 7fd4360bbb80 1 Processor -- start 2024-03-30T02:38:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb265d0 0x55daceb316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb2ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb31df0 0x55daceb2ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacea14d80 con 0x55daceb26ef0 2024-03-30T02:38:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dacea14a20 con 0x55daceb31df0 2024-03-30T02:38:32.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dace8fc6a0 con 0x55daceb265d0 2024-03-30T02:38:32.989 INFO:tasks.workunit.client.0.smithi067.stderr: 6500K .......... .......... .......... .......... .......... 4% 47.5M 13s 2024-03-30T02:38:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd4360bbb80 10 client.0 _ll_register_callbacks cb 0x55dacea1e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:32.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd41aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb2ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd41be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb31df0 0x55daceb2ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd41b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb265d0 0x55daceb316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd41aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb2ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50426/0 (socket says 172.21.15.67:50426) 2024-03-30T02:38:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd41be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb31df0 0x55daceb2ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38272/0 (socket says 172.21.15.67:38272) 2024-03-30T02:38:32.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.984+0000 7fd41aa00700 1 -- 172.21.15.67:0/2628745052 learned_addr learned my addr 172.21.15.67:0/2628745052 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:32.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd41be00700 1 -- 172.21.15.67:0/2628745052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb265d0 msgr2=0x55daceb316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd41be00700 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb265d0 0x55daceb316c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd41be00700 1 -- 172.21.15.67:0/2628745052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 msgr2=0x55daceb2ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:32.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd41be00700 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb2ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:32.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd41be00700 1 -- 172.21.15.67:0/2628745052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c1a6510 con 0x55daceb31df0 2024-03-30T02:38:32.993 INFO:tasks.workunit.client.0.smithi067.stderr: 6550K .......... .......... .......... .......... .......... 4% 55.8M 13s 2024-03-30T02:38:32.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd41aa00700 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb2ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:32.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd41b400700 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb265d0 0x55daceb316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:32.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd41be00700 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb31df0 0x55daceb2ef60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd414015940 tx=0x7fd41400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4738d1caaa01c35 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:32.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd405600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:38:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd405600700 1 -- 172.21.15.67:0/2628745052 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400e040 con 0x55daceb31df0 2024-03-30T02:38:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd4360bbb80 1 -- 172.21.15.67:0/2628745052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dacea39380 con 0x55daceb31df0 2024-03-30T02:38:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd405600700 1 -- 172.21.15.67:0/2628745052 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd414020030 con 0x55daceb31df0 2024-03-30T02:38:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd405600700 1 -- 172.21.15.67:0/2628745052 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd414019ed0 con 0x55daceb31df0 2024-03-30T02:38:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd4360bbb80 10 client.26090 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.985+0000 7fd4360bbb80 1 -- 172.21.15.67:0/2628745052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fd41c10e470 con 0x55daceb31df0 2024-03-30T02:38:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.986+0000 7fd405600700 1 -- 172.21.15.67:0/2628745052 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dacea39380 con 0x55daceb31df0 2024-03-30T02:38:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.986+0000 7fd405600700 1 -- 172.21.15.67:0/2628745052 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd41402e070 con 0x55daceb31df0 2024-03-30T02:38:32.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.986+0000 7fd4360bbb80 10 client.26090 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:32.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.986+0000 7fd4360bbb80 10 client.26090 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:38:32.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.986+0000 7fd4360bbb80 20 client.26090 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:32.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.986+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55daceb41820 0x55daceb43c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:32.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.986+0000 7fd4360bbb80 4 client.26090 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:32.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.986+0000 7fd4360bbb80 1 -- 172.21.15.67:0/2628745052 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dacea71700 con 0x55daceb41820 2024-03-30T02:38:32.998 INFO:tasks.workunit.client.0.smithi067.stderr: 6600K .......... .......... .......... .......... .......... 4% 29.7M 13s 2024-03-30T02:38:32.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.986+0000 7fd41b400700 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55daceb41820 0x55daceb43c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:32.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.987+0000 7fd405600700 10 client.26090 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:38:32.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.987+0000 7fd41b400700 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55daceb41820 0x55daceb43c40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5c75bc5ef999079 server_cookie=f02a9194b4d0dae9 in_seq=0 out_seq=0 2024-03-30T02:38:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.987+0000 7fd405600700 1 -- 172.21.15.67:0/2628745052 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dacea71700 con 0x55daceb41820 2024-03-30T02:38:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.987+0000 7fd405600700 10 client.26090 handle_command_reply: tid=0 2024-03-30T02:38:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 client.26090 shutdown 2024-03-30T02:38:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 -- 172.21.15.67:0/2628745052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb31df0 msgr2=0x55daceb2ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb31df0 0x55daceb2ef60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd414015940 tx=0x7fd41400b040 comp rx=0 tx=0).stop 2024-03-30T02:38:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 -- 172.21.15.67:0/2628745052 shutdown_connections 2024-03-30T02:38:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb31df0 0x55daceb2ef60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb265d0 0x55daceb316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55daceb41820 0x55daceb43c40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/2628745052 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb26ef0 0x55daceb2ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 -- 172.21.15.67:0/2628745052 >> 172.21.15.67:0/2628745052 conn(0x55dacea866a0 msgr2=0x55daceb28510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd405600700 0 client.26090 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:38:33.002 INFO:tasks.workunit.client.0.smithi067.stderr: 6650K .......... .......... .......... .......... .......... 4% 28.9M 13s 2024-03-30T02:38:33.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 -- 172.21.15.67:0/2628745052 shutdown_connections 2024-03-30T02:38:33.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 1 -- 172.21.15.67:0/2628745052 wait complete. 2024-03-30T02:38:33.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:32.988+0000 7fd4360bbb80 20 client.26090 trim_cache size 0 max 16384 2024-03-30T02:38:33.012 INFO:tasks.workunit.client.0.smithi067.stderr: 6700K .......... .......... .......... .......... .......... 4% 46.3M 13s 2024-03-30T02:38:33.012 INFO:tasks.workunit.client.0.smithi067.stderr: 6750K .......... .......... .......... .......... .......... 5% 31.8M 13s 2024-03-30T02:38:33.012 INFO:tasks.workunit.client.0.smithi067.stderr: 6800K .......... .......... .......... .......... .......... 5% 63.8M 13s 2024-03-30T02:38:33.012 INFO:tasks.workunit.client.0.smithi067.stderr: 6850K .......... .......... .......... .......... .......... 5% 38.7M 13s 2024-03-30T02:38:33.012 INFO:tasks.workunit.client.0.smithi067.stderr: 6900K .......... .......... .......... .......... .......... 5% 36.5M 13s 2024-03-30T02:38:33.012 INFO:tasks.workunit.client.0.smithi067.stderr: 6950K .......... .......... .......... .......... .......... 5% 78.7M 13s 2024-03-30T02:38:33.012 INFO:tasks.workunit.client.0.smithi067.stderr: 7000K .......... .......... .......... .......... .......... 5% 38.2M 13s 2024-03-30T02:38:33.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.002+0000 7fd4360bbb80 1 Processor -- start 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.002+0000 7fd4360bbb80 1 -- start start 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb38a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb26ef0 0x55daceb2d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb31df0 0x55daceb2d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daceb27850 con 0x55daceb31df0 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daceb43fd0 con 0x55daceb265d0 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daceb2dd70 con 0x55daceb26ef0 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb38a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb38a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38288/0 (socket says 172.21.15.67:38288) 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41b400700 1 -- 172.21.15.67:0/1397967926 learned_addr learned my addr 172.21.15.67:0/1397967926 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41be00700 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb31df0 0x55daceb2d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41aa00700 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb26ef0 0x55daceb2d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41b400700 1 -- 172.21.15.67:0/1397967926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb26ef0 msgr2=0x55daceb2d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41b400700 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb26ef0 0x55daceb2d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41b400700 1 -- 172.21.15.67:0/1397967926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb31df0 msgr2=0x55daceb2d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41b400700 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb31df0 0x55daceb2d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41b400700 1 -- 172.21.15.67:0/1397967926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dace8a6b60 con 0x55daceb265d0 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.003+0000 7fd41be00700 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb31df0 0x55daceb2d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd41b400700 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb38a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd40c000c00 tx=0x7fd40c005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af23695dbf338b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd411600700 1 -- 172.21.15.67:0/1397967926 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01c040 con 0x55daceb265d0 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd411600700 1 -- 172.21.15.67:0/1397967926 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c005940 con 0x55daceb265d0 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd411600700 1 -- 172.21.15.67:0/1397967926 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c032a70 con 0x55daceb265d0 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1397967926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 msgr2=0x55daceb38a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb38a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd40c000c00 tx=0x7fd40c005ad0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1397967926 shutdown_connections 2024-03-30T02:38:33.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb31df0 0x55daceb2d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb265d0 0x55daceb38a10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/1397967926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb26ef0 0x55daceb2d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.004+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1397967926 >> 172.21.15.67:0/1397967926 conn(0x55dacea866a0 msgr2=0x55daceb31360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.005+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1397967926 shutdown_connections 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.005+0000 7fd4360bbb80 1 -- 172.21.15.67:0/1397967926 wait complete. 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.005+0000 7fd4360bbb80 1 -- start start 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.005+0000 7fd4360bbb80 1 Processor -- start 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb265d0 0x55daceb369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb26ef0 0x55daceb36f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd4360bbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 0x55daceb37470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daceb946c0 con 0x55daceb265d0 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daceb94830 con 0x55daceb26ef0 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd4360bbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daceb949f0 con 0x55daceb31df0 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd4360bbb80 10 client.0 _ll_register_callbacks cb 0x55dacea1e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:33.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd41b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb265d0 0x55daceb369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd41b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb265d0 0x55daceb369f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50450/0 (socket says 172.21.15.67:50450) 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd41b400700 1 -- 172.21.15.67:0/784960587 learned_addr learned my addr 172.21.15.67:0/784960587 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd41aa00700 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb26ef0 0x55daceb36f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.006+0000 7fd41be00700 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 0x55daceb37470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd41aa00700 1 -- 172.21.15.67:0/784960587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 msgr2=0x55daceb37470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd41aa00700 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 0x55daceb37470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd41aa00700 1 -- 172.21.15.67:0/784960587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb265d0 msgr2=0x55daceb369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd41aa00700 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb265d0 0x55daceb369f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd41aa00700 1 -- 172.21.15.67:0/784960587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dace8fa3b0 con 0x55daceb26ef0 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd41b400700 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb265d0 0x55daceb369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd41aa00700 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb26ef0 0x55daceb36f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd4080ca840 tx=0x7fd40809a720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31533ecfdb354e05 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd405600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd405600700 1 -- 172.21.15.67:0/784960587 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40809a790 con 0x55daceb26ef0 2024-03-30T02:38:33.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd405600700 1 -- 172.21.15.67:0/784960587 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd40800b8b0 con 0x55daceb26ef0 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd4360bbb80 1 -- 172.21.15.67:0/784960587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd40c01d040 con 0x55daceb26ef0 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd405600700 1 -- 172.21.15.67:0/784960587 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd40809a020 con 0x55daceb26ef0 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd4360bbb80 10 client.26096 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.007+0000 7fd4360bbb80 1 -- 172.21.15.67:0/784960587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dacea3eea0 con 0x55daceb26ef0 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.008+0000 7fd405600700 1 -- 172.21.15.67:0/784960587 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c01d040 con 0x55daceb26ef0 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.008+0000 7fd405600700 1 -- 172.21.15.67:0/784960587 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4080cd1a0 con 0x55daceb26ef0 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.008+0000 7fd4360bbb80 10 client.26096 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.008+0000 7fd4360bbb80 10 client.26096 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.008+0000 7fd4360bbb80 20 client.26096 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.008+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55daceb96180 0x55daceb985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.008+0000 7fd4360bbb80 4 client.26096 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.008+0000 7fd4360bbb80 1 -- 172.21.15.67:0/784960587 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dace8c7cf0 con 0x55daceb96180 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.008+0000 7fd41b400700 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55daceb96180 0x55daceb985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd41b400700 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55daceb96180 0x55daceb985a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4a884a6549a0012 server_cookie=ea20ec24065e70b3 in_seq=0 out_seq=0 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd405600700 10 client.26096 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd405600700 1 -- 172.21.15.67:0/784960587 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dace8c7cf0 con 0x55daceb96180 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd405600700 10 client.26096 handle_command_reply: tid=0 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 client.26096 shutdown 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 -- 172.21.15.67:0/784960587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb26ef0 msgr2=0x55daceb36f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb26ef0 0x55daceb36f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd4080ca840 tx=0x7fd40809a720 comp rx=0 tx=0).stop 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 -- 172.21.15.67:0/784960587 shutdown_connections 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55daceb96180 0x55daceb985a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55daceb265d0 0x55daceb369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55daceb31df0 0x55daceb37470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 --2- 172.21.15.67:0/784960587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55daceb26ef0 0x55daceb36f30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 -- 172.21.15.67:0/784960587 >> 172.21.15.67:0/784960587 conn(0x55dacea866a0 msgr2=0x55daceb2d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd405600700 0 client.26096 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:38:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 -- 172.21.15.67:0/784960587 shutdown_connections 2024-03-30T02:38:33.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.009+0000 7fd4360bbb80 1 -- 172.21.15.67:0/784960587 wait complete. 2024-03-30T02:38:33.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd4360bbb80 20 client.26096 trim_cache size 0 max 16384 2024-03-30T02:38:33.024 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 -- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4000a1430 msgr2=0x7fd4000a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4000a1430 0x7fd4000a38f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd414015e20 tx=0x7fd414015420 comp rx=0 tx=0).stop 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 -- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1060b0 msgr2=0x7fd41c199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1060b0 0x7fd41c199990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e970 tx=0x7fd40800ec80 comp rx=0 tx=0).stop 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 -- 172.21.15.67:0/393359358 shutdown_connections 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4000a1430 0x7fd4000a38f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c19dd30 0x7fd41c19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c069260 0x7fd41c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 --2- 172.21.15.67:0/393359358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1060b0 0x7fd41c199990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 -- 172.21.15.67:0/393359358 >> 172.21.15.67:0/393359358 conn(0x7fd41c076d80 msgr2=0x7fd41c10bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.010+0000 7fd422600700 1 -- 172.21.15.67:0/393359358 shutdown_connections 2024-03-30T02:38:33.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.011+0000 7fd422600700 1 -- 172.21.15.67:0/393359358 wait complete. 2024-03-30T02:38:33.041 INFO:tasks.workunit.client.0.smithi067.stderr: 7050K .......... .......... .......... .......... .......... 5% 2.19M 13s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7100K .......... .......... .......... .......... .......... 5% 47.6M 13s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7150K .......... .......... .......... .......... .......... 5% 52.0M 13s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7200K .......... .......... .......... .......... .......... 5% 35.0M 13s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7250K .......... .......... .......... .......... .......... 5% 27.8M 13s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7300K .......... .......... .......... .......... .......... 5% 47.2M 13s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7350K .......... .......... .......... .......... .......... 5% 79.6M 13s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7400K .......... .......... .......... .......... .......... 5% 28.3M 12s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7450K .......... .......... .......... .......... .......... 5% 83.5M 12s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7500K .......... .......... .......... .......... .......... 5% 28.3M 12s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7550K .......... .......... .......... .......... .......... 5% 72.3M 12s 2024-03-30T02:38:33.042 INFO:tasks.workunit.client.0.smithi067.stderr: 7600K .......... .......... .......... .......... .......... 5% 65.2M 12s 2024-03-30T02:38:33.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:38:33.045 INFO:tasks.workunit.client.0.smithi067.stderr: 7650K .......... .......... .......... .......... .......... 5% 37.3M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 7700K .......... .......... .......... .......... .......... 5% 2.13M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 7750K .......... .......... .......... .......... .......... 5% 64.9M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 7800K .......... .......... .......... .......... .......... 5% 76.1M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 7850K .......... .......... .......... .......... .......... 5% 41.1M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 7900K .......... .......... .......... .......... .......... 5% 26.2M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 7950K .......... .......... .......... .......... .......... 5% 57.5M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 8000K .......... .......... .......... .......... .......... 5% 56.8M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 8050K .......... .......... .......... .......... .......... 5% 31.1M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 8100K .......... .......... .......... .......... .......... 6% 62.6M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 8150K .......... .......... .......... .......... .......... 6% 27.6M 12s 2024-03-30T02:38:33.070 INFO:tasks.workunit.client.0.smithi067.stderr: 8200K .......... .......... .......... .......... .......... 6% 54.0M 12s 2024-03-30T02:38:33.071 INFO:tasks.workunit.client.0.smithi067.stderr: 8250K .......... .......... .......... .......... .......... 6% 102M 12s 2024-03-30T02:38:33.071 INFO:tasks.workunit.client.0.smithi067.stderr: 8300K .......... .......... .......... .......... .......... 6% 66.2M 12s 2024-03-30T02:38:33.107 INFO:tasks.workunit.client.0.smithi067.stderr: 8350K .......... .......... .......... .......... .......... 6% 2.19M 12s 2024-03-30T02:38:33.107 INFO:tasks.workunit.client.0.smithi067.stderr: 8400K .......... .......... .......... .......... .......... 6% 48.4M 12s 2024-03-30T02:38:33.107 INFO:tasks.workunit.client.0.smithi067.stderr: 8450K .......... .......... .......... .......... .......... 6% 88.3M 12s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8500K .......... .......... .......... .......... .......... 6% 29.5M 12s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8550K .......... .......... .......... .......... .......... 6% 36.1M 12s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8600K .......... .......... .......... .......... .......... 6% 52.5M 12s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8650K .......... .......... .......... .......... .......... 6% 51.5M 12s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8700K .......... .......... .......... .......... .......... 6% 81.1M 12s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8750K .......... .......... .......... .......... .......... 6% 28.9M 11s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8800K .......... .......... .......... .......... .......... 6% 76.6M 11s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8850K .......... .......... .......... .......... .......... 6% 30.9M 11s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8900K .......... .......... .......... .......... .......... 6% 86.7M 11s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 8950K .......... .......... .......... .......... .......... 6% 30.5M 11s 2024-03-30T02:38:33.108 INFO:tasks.workunit.client.0.smithi067.stderr: 9000K .......... .......... .......... .......... .......... 6% 24.7M 11s 2024-03-30T02:38:33.148 INFO:tasks.workunit.client.0.smithi067.stderr: 9050K .......... .......... .......... .......... .......... 6% 2.32M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9100K .......... .......... .......... .......... .......... 6% 97.7M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9150K .......... .......... .......... .......... .......... 6% 101M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9200K .......... .......... .......... .......... .......... 6% 73.7M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9250K .......... .......... .......... .......... .......... 6% 21.4M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9300K .......... .......... .......... .......... .......... 6% 56.2M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9350K .......... .......... .......... .......... .......... 6% 55.4M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9400K .......... .......... .......... .......... .......... 6% 88.7M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9450K .......... .......... .......... .......... .......... 6% 28.3M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9500K .......... .......... .......... .......... .......... 7% 74.6M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9550K .......... .......... .......... .......... .......... 7% 38.8M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9600K .......... .......... .......... .......... .......... 7% 38.1M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9650K .......... .......... .......... .......... .......... 7% 40.8M 11s 2024-03-30T02:38:33.149 INFO:tasks.workunit.client.0.smithi067.stderr: 9700K .......... .......... .......... .......... .......... 7% 28.8M 11s 2024-03-30T02:38:33.150 INFO:tasks.workunit.client.0.smithi067.stderr: 9750K .......... .......... .......... .......... .......... 7% 16.9M 11s 2024-03-30T02:38:33.166 INFO:tasks.workunit.client.0.smithi067.stderr: 9800K .......... .......... .......... .......... .......... 7% 2.35M 11s 2024-03-30T02:38:33.166 INFO:tasks.workunit.client.0.smithi067.stderr: 9850K .......... .......... .......... .......... .......... 7% 49.4M 11s 2024-03-30T02:38:33.166 INFO:tasks.workunit.client.0.smithi067.stderr: 9900K .......... .......... .......... .......... .......... 7% 97.5M 11s 2024-03-30T02:38:33.167 INFO:tasks.workunit.client.0.smithi067.stderr: 9950K .......... .......... .......... .......... .......... 7% 21.6M 11s 2024-03-30T02:38:33.170 INFO:tasks.workunit.client.0.smithi067.stderr: 10000K .......... .......... .......... .......... .......... 7% 28.2M 11s 2024-03-30T02:38:33.197 INFO:tasks.workunit.client.0.smithi067.stderr: 10050K .......... .......... .......... .......... .......... 7% 15.9M 11s 2024-03-30T02:38:33.197 INFO:tasks.workunit.client.0.smithi067.stderr: 10100K .......... .......... .......... .......... .......... 7% 109M 11s 2024-03-30T02:38:33.197 INFO:tasks.workunit.client.0.smithi067.stderr: 10150K .......... .......... .......... .......... .......... 7% 116M 11s 2024-03-30T02:38:33.197 INFO:tasks.workunit.client.0.smithi067.stderr: 10200K .......... .......... .......... .......... .......... 7% 87.5M 11s 2024-03-30T02:38:33.197 INFO:tasks.workunit.client.0.smithi067.stderr: 10250K .......... .......... .......... .......... .......... 7% 94.7M 11s 2024-03-30T02:38:33.197 INFO:tasks.workunit.client.0.smithi067.stderr: 10300K .......... .......... .......... .......... .......... 7% 82.6M 11s 2024-03-30T02:38:33.197 INFO:tasks.workunit.client.0.smithi067.stderr: 10350K .......... .......... .......... .......... .......... 7% 84.4M 11s 2024-03-30T02:38:33.197 INFO:tasks.workunit.client.0.smithi067.stderr: 10400K .......... .......... .......... .......... .......... 7% 74.3M 10s 2024-03-30T02:38:33.198 INFO:tasks.workunit.client.0.smithi067.stderr: 10450K .......... .......... .......... .......... .......... 7% 71.9M 10s 2024-03-30T02:38:33.198 INFO:tasks.workunit.client.0.smithi067.stderr: 10500K .......... .......... .......... .......... .......... 7% 53.3M 10s 2024-03-30T02:38:33.198 INFO:tasks.workunit.client.0.smithi067.stderr: 10550K .......... .......... .......... .......... .......... 7% 5.94M 10s 2024-03-30T02:38:33.203 INFO:tasks.workunit.client.0.smithi067.stderr: 10600K .......... .......... .......... .......... .......... 7% 3.58M 11s 2024-03-30T02:38:33.203 INFO:tasks.workunit.client.0.smithi067.stderr: 10650K .......... .......... .......... .......... .......... 7% 52.3M 10s 2024-03-30T02:38:33.203 INFO:tasks.workunit.client.0.smithi067.stderr: 10700K .......... .......... .......... .......... .......... 7% 45.6M 10s 2024-03-30T02:38:33.205 INFO:tasks.workunit.client.0.smithi067.stderr: 10750K .......... .......... .......... .......... .......... 7% 27.8M 10s 2024-03-30T02:38:33.205 INFO:tasks.workunit.client.0.smithi067.stderr: 10800K .......... .......... .......... .......... .......... 7% 24.8M 10s 2024-03-30T02:38:33.223 INFO:tasks.workunit.client.0.smithi067.stderr: 10850K .......... .......... .......... .......... .......... 8% 17.3M 10s 2024-03-30T02:38:33.223 INFO:tasks.workunit.client.0.smithi067.stderr: 10900K .......... .......... .......... .......... .......... 8% 63.9M 10s 2024-03-30T02:38:33.223 INFO:tasks.workunit.client.0.smithi067.stderr: 10950K .......... .......... .......... .......... .......... 8% 105M 10s 2024-03-30T02:38:33.223 INFO:tasks.workunit.client.0.smithi067.stderr: 11000K .......... .......... .......... .......... .......... 8% 113M 10s 2024-03-30T02:38:33.223 INFO:tasks.workunit.client.0.smithi067.stderr: 11050K .......... .......... .......... .......... .......... 8% 105M 10s 2024-03-30T02:38:33.223 INFO:tasks.workunit.client.0.smithi067.stderr: 11100K .......... .......... .......... .......... .......... 8% 111M 10s 2024-03-30T02:38:33.223 INFO:tasks.workunit.client.0.smithi067.stderr: 11150K .......... .......... .......... .......... .......... 8% 77.2M 10s 2024-03-30T02:38:33.223 INFO:tasks.workunit.client.0.smithi067.stderr: 11200K .......... .......... .......... .......... .......... 8% 115M 10s 2024-03-30T02:38:33.224 INFO:tasks.workunit.client.0.smithi067.stderr: 11250K .......... .......... .......... .......... .......... 8% 51.3M 10s 2024-03-30T02:38:33.224 INFO:tasks.workunit.client.0.smithi067.stderr: 11300K .......... .......... .......... .......... .......... 8% 23.8M 10s 2024-03-30T02:38:33.224 INFO:tasks.workunit.client.0.smithi067.stderr: 11350K .......... .......... .......... .......... .......... 8% 41.1M 10s 2024-03-30T02:38:33.240 INFO:tasks.workunit.client.0.smithi067.stderr: 11400K .......... .......... .......... .......... .......... 8% 2.55M 10s 2024-03-30T02:38:33.240 INFO:tasks.workunit.client.0.smithi067.stderr: 11450K .......... .......... .......... .......... .......... 8% 58.3M 10s 2024-03-30T02:38:33.240 INFO:tasks.workunit.client.0.smithi067.stderr: 11500K .......... .......... .......... .......... .......... 8% 93.4M 10s 2024-03-30T02:38:33.240 INFO:tasks.workunit.client.0.smithi067.stderr: 11550K .......... .......... .......... .......... .......... 8% 21.0M 10s 2024-03-30T02:38:33.240 INFO:tasks.workunit.client.0.smithi067.stderr: 11600K .......... .......... .......... .......... .......... 8% 24.7M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 11650K .......... .......... .......... .......... .......... 8% 14.5M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 11700K .......... .......... .......... .......... .......... 8% 45.3M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 11750K .......... .......... .......... .......... .......... 8% 89.4M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 11800K .......... .......... .......... .......... .......... 8% 45.7M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 11850K .......... .......... .......... .......... .......... 8% 50.3M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 11900K .......... .......... .......... .......... .......... 8% 91.0M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 11950K .......... .......... .......... .......... .......... 8% 47.6M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 12000K .......... .......... .......... .......... .......... 8% 45.2M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 12050K .......... .......... .......... .......... .......... 8% 27.8M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 12100K .......... .......... .......... .......... .......... 8% 21.5M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 12150K .......... .......... .......... .......... .......... 8% 28.6M 10s 2024-03-30T02:38:33.272 INFO:tasks.workunit.client.0.smithi067.stderr: 12200K .......... .......... .......... .......... .......... 9% 20.0M 10s 2024-03-30T02:38:33.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.261+0000 7fc082800700 1 -- 172.21.15.67:0/3732686653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 msgr2=0x7fc07c105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.261+0000 7fc082800700 1 --2- 172.21.15.67:0/3732686653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 0x7fc07c105e30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009990 tx=0x7fc06c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.261+0000 7fc082800700 1 -- 172.21.15.67:0/3732686653 shutdown_connections 2024-03-30T02:38:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.261+0000 7fc082800700 1 --2- 172.21.15.67:0/3732686653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 0x7fc07c105e30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.261+0000 7fc082800700 1 --2- 172.21.15.67:0/3732686653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc07c100400 0x7fc07c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.261+0000 7fc082800700 1 --2- 172.21.15.67:0/3732686653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07c0ffa40 0x7fc07c0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.261+0000 7fc082800700 1 -- 172.21.15.67:0/3732686653 >> 172.21.15.67:0/3732686653 conn(0x7fc07c0fb550 msgr2=0x7fc07c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.261+0000 7fc082800700 1 -- 172.21.15.67:0/3732686653 shutdown_connections 2024-03-30T02:38:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.261+0000 7fc082800700 1 -- 172.21.15.67:0/3732686653 wait complete. 2024-03-30T02:38:33.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.262+0000 7fc082800700 1 Processor -- start 2024-03-30T02:38:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.262+0000 7fc082800700 1 -- start start 2024-03-30T02:38:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.262+0000 7fc082800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07c0ffa40 0x7fc07c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.262+0000 7fc082800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc07c100400 0x7fc07c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.262+0000 7fc082800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 0x7fc07c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.262+0000 7fc082800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c19e0e0 con 0x7fc07c105a50 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.262+0000 7fc082800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c19e250 con 0x7fc07c0ffa40 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.262+0000 7fc082800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c19e410 con 0x7fc07c100400 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 0x7fc07c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 0x7fc07c19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50460/0 (socket says 172.21.15.67:50460) 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07be00700 1 -- 172.21.15.67:0/1249737871 learned_addr learned my addr 172.21.15.67:0/1249737871 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07aa00700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc07c100400 0x7fc07c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07b400700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07c0ffa40 0x7fc07c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07be00700 1 -- 172.21.15.67:0/1249737871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc07c100400 msgr2=0x7fc07c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07be00700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc07c100400 0x7fc07c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07be00700 1 -- 172.21.15.67:0/1249737871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07c0ffa40 msgr2=0x7fc07c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07be00700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07c0ffa40 0x7fc07c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07be00700 1 -- 172.21.15.67:0/1249737871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c0095f0 con 0x7fc07c105a50 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.263+0000 7fc07be00700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 0x7fc07c19daa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc074007ee0 tx=0x7fc07400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.264+0000 7fc073e00700 1 -- 172.21.15.67:0/1249737871 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074017070 con 0x7fc07c105a50 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.264+0000 7fc073e00700 1 -- 172.21.15.67:0/1249737871 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07400f040 con 0x7fc07c105a50 2024-03-30T02:38:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.264+0000 7fc082800700 1 -- 172.21.15.67:0/1249737871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c19e6f0 con 0x7fc07c105a50 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.264+0000 7fc073e00700 1 -- 172.21.15.67:0/1249737871 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074005550 con 0x7fc07c105a50 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.264+0000 7fc082800700 1 -- 172.21.15.67:0/1249737871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c1a5f30 con 0x7fc07c105a50 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.267+0000 7fc073e00700 1 -- 172.21.15.67:0/1249737871 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) v1 ==== 136098+0+0 (secure 0 0 0) 0x7fc07400db20 con 0x7fc07c105a50 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.268+0000 7fc0963d4b80 1 Processor -- start 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.268+0000 7fc073e00700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0600a13b0 0x7fc0600a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.268+0000 7fc0963d4b80 1 -- start start 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.268+0000 7fc073e00700 1 -- 172.21.15.67:0/1249737871 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fc0740cc9e0 con 0x7fc07c105a50 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483d50c0 0x561c483dfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483dffc0 0x561c483e23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c4832d990 con 0x561c483d47a0 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c4832b350 con 0x561c483dffc0 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c48326c40 con 0x561c483d50c0 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07aa00700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0600a13b0 0x7fc0600a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483d4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50476/0 (socket says 172.21.15.67:50476) 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07b400700 1 -- 172.21.15.67:0/1934703945 learned_addr learned my addr 172.21.15.67:0/1934703945 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07be00700 1 --2- 172.21.15.67:0/1934703945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483dffc0 0x561c483e23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07aa00700 1 --2- 172.21.15.67:0/1934703945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483d50c0 0x561c483dfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07b400700 1 -- 172.21.15.67:0/1934703945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483d50c0 msgr2=0x561c483dfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07b400700 1 --2- 172.21.15.67:0/1934703945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483d50c0 0x561c483dfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07b400700 1 -- 172.21.15.67:0/1934703945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483dffc0 msgr2=0x561c483e23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07b400700 1 --2- 172.21.15.67:0/1934703945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483dffc0 0x561c483e23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.269+0000 7fc07b400700 1 -- 172.21.15.67:0/1934703945 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c481a83b0 con 0x561c483d47a0 2024-03-30T02:38:33.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.270+0000 7fc07b400700 1 --2- 172.21.15.67:0/1934703945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483d4b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b610 tx=0x7fc06c00b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84c27b19e5607a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.270+0000 7fc07aa00700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0600a13b0 0x7fc0600a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc07c19a710 tx=0x7fc068009f90 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.270+0000 7fc070c00700 1 -- 172.21.15.67:0/1934703945 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c01d070 con 0x561c483d47a0 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.270+0000 7fc070c00700 1 -- 172.21.15.67:0/1934703945 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc06c01e040 con 0x561c483d47a0 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.270+0000 7fc070c00700 1 -- 172.21.15.67:0/1934703945 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c004860 con 0x561c483d47a0 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1934703945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 msgr2=0x561c483d4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1934703945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483d4b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b610 tx=0x7fc06c00b640 comp rx=0 tx=0).stop 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1934703945 shutdown_connections 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1934703945 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483d4b80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1934703945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483dffc0 0x561c483e23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1934703945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483d50c0 0x561c483dfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1934703945 >> 172.21.15.67:0/1934703945 conn(0x561c483346a0 msgr2=0x561c483d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1934703945 shutdown_connections 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1934703945 wait complete. 2024-03-30T02:38:33.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 -- start start 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.271+0000 7fc0963d4b80 1 Processor -- start 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483df930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483dcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c483dd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c482c2d80 con 0x561c483d47a0 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c482c2a20 con 0x561c483d50c0 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c481aa6a0 con 0x561c483dffc0 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc0963d4b80 10 client.0 _ll_register_callbacks cb 0x561c482cc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc07b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483df930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc07aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483dcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc07b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483df930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50488/0 (socket says 172.21.15.67:50488) 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc07b400700 1 -- 172.21.15.67:0/555564158 learned_addr learned my addr 172.21.15.67:0/555564158 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.272+0000 7fc07be00700 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c483dd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc07b400700 1 -- 172.21.15.67:0/555564158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 msgr2=0x561c483dd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc07b400700 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c483dd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc07b400700 1 -- 172.21.15.67:0/555564158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 msgr2=0x561c483dcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc07b400700 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483dcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc07b400700 1 -- 172.21.15.67:0/555564158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c10d210 con 0x561c483d47a0 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc07aa00700 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483dcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc07b400700 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483df930 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b6f0 tx=0x7fc06c00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426a40b411d41ac1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc065600700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc065600700 1 -- 172.21.15.67:0/555564158 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c01d030 con 0x561c483d47a0 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc0963d4b80 1 -- 172.21.15.67:0/555564158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c482e7380 con 0x561c483d47a0 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc065600700 1 -- 172.21.15.67:0/555564158 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fc06c004910 con 0x561c483d47a0 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc065600700 1 -- 172.21.15.67:0/555564158 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc06c004ad0 con 0x561c483d47a0 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc0963d4b80 10 client.16056 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:33.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.273+0000 7fc0963d4b80 1 -- 172.21.15.67:0/555564158 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561c48154b60 con 0x561c483d47a0 2024-03-30T02:38:33.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.274+0000 7fc065600700 1 -- 172.21.15.67:0/555564158 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c482e7380 con 0x561c483d47a0 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.274+0000 7fc065600700 1 -- 172.21.15.67:0/555564158 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc06c00fa40 con 0x561c483d47a0 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.274+0000 7fc0963d4b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.274+0000 7fc0963d4b80 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.274+0000 7fc0963d4b80 20 client.16056 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.274+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561c483ef6f0 0x561c483f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.274+0000 7fc0963d4b80 4 client.16056 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.274+0000 7fc0963d4b80 1 -- 172.21.15.67:0/555564158 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c4831f700 con 0x561c483ef6f0 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.274+0000 7fc07aa00700 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561c483ef6f0 0x561c483f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.275+0000 7fc07aa00700 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561c483ef6f0 0x561c483f1ae0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66caf3e7917ebcf server_cookie=1c195fa11b48bb8 in_seq=0 out_seq=0 2024-03-30T02:38:33.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.275+0000 7fc065600700 10 client.16056 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc065600700 1 -- 172.21.15.67:0/555564158 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561c4831f700 con 0x561c483ef6f0 2024-03-30T02:38:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc065600700 10 client.16056 handle_command_reply: tid=0 2024-03-30T02:38:33.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 client.16056 shutdown 2024-03-30T02:38:33.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 -- 172.21.15.67:0/555564158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 msgr2=0x561c483df930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483df930 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b6f0 tx=0x7fc06c00b9e0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.291 INFO:tasks.workunit.client.0.smithi067.stderr: 12250K .......... .......... .......... .......... .......... 9% 2.72M 10s 2024-03-30T02:38:33.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 -- 172.21.15.67:0/555564158 shutdown_connections 2024-03-30T02:38:33.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561c483ef6f0 0x561c483f1ae0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483df930 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c483dd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/555564158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483dcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 -- 172.21.15.67:0/555564158 >> 172.21.15.67:0/555564158 conn(0x561c483346a0 msgr2=0x561c483d6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc065600700 0 client.16056 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:33.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 -- 172.21.15.67:0/555564158 shutdown_connections 2024-03-30T02:38:33.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 1 -- 172.21.15.67:0/555564158 wait complete. 2024-03-30T02:38:33.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.276+0000 7fc0963d4b80 20 client.16056 trim_cache size 0 max 16384 2024-03-30T02:38:33.302 INFO:tasks.workunit.client.0.smithi067.stderr: 12300K .......... .......... .......... .......... .......... 9% 48.2M 10s 2024-03-30T02:38:33.302 INFO:tasks.workunit.client.0.smithi067.stderr: 12350K .......... .......... .......... .......... .......... 9% 29.2M 10s 2024-03-30T02:38:33.302 INFO:tasks.workunit.client.0.smithi067.stderr: 12400K .......... .......... .......... .......... .......... 9% 20.4M 10s 2024-03-30T02:38:33.302 INFO:tasks.workunit.client.0.smithi067.stderr: 12450K .......... .......... .......... .......... .......... 9% 22.2M 10s 2024-03-30T02:38:33.302 INFO:tasks.workunit.client.0.smithi067.stderr: 12500K .......... .......... .......... .......... .......... 9% 10.7M 10s 2024-03-30T02:38:33.302 INFO:tasks.workunit.client.0.smithi067.stderr: 12550K .......... .......... .......... .......... .......... 9% 43.1M 10s 2024-03-30T02:38:33.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.290+0000 7fc0963d4b80 1 Processor -- start 2024-03-30T02:38:33.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.290+0000 7fc0963d4b80 1 -- start start 2024-03-30T02:38:33.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.290+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d47a0 0x561c483e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.290+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d50c0 0x561c483db570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.290+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c483dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.290+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c483dbff0 con 0x561c483d50c0 2024-03-30T02:38:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.290+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c483e7120 con 0x561c483d47a0 2024-03-30T02:38:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.290+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c483ddb20 con 0x561c483dffc0 2024-03-30T02:38:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d47a0 0x561c483e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d47a0 0x561c483e6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38338/0 (socket says 172.21.15.67:38338) 2024-03-30T02:38:33.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07b400700 1 -- 172.21.15.67:0/2350188299 learned_addr learned my addr 172.21.15.67:0/2350188299 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07aa00700 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d50c0 0x561c483db570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07be00700 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c483dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07b400700 1 -- 172.21.15.67:0/2350188299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 msgr2=0x561c483dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07b400700 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c483dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07b400700 1 -- 172.21.15.67:0/2350188299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d50c0 msgr2=0x561c483db570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07b400700 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d50c0 0x561c483db570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07b400700 1 -- 172.21.15.67:0/2350188299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c482ecea0 con 0x561c483d47a0 2024-03-30T02:38:33.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07be00700 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c483dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:33.310 INFO:tasks.workunit.client.0.smithi067.stderr: 12600K .......... .......... .......... .......... .......... 9% 22.8M 10s 2024-03-30T02:38:33.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc07b400700 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d47a0 0x561c483e6be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009990 tx=0x7fc06c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ec56d6a4c17a5e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:33.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.291+0000 7fc071600700 1 -- 172.21.15.67:0/2350188299 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c01d040 con 0x561c483d47a0 2024-03-30T02:38:33.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc071600700 1 -- 172.21.15.67:0/2350188299 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc06c00bea0 con 0x561c483d47a0 2024-03-30T02:38:33.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc071600700 1 -- 172.21.15.67:0/2350188299 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00f7e0 con 0x561c483d47a0 2024-03-30T02:38:33.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc0963d4b80 1 -- 172.21.15.67:0/2350188299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d47a0 msgr2=0x561c483e6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d47a0 0x561c483e6be0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009990 tx=0x7fc06c009310 comp rx=0 tx=0).stop 2024-03-30T02:38:33.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc0963d4b80 1 -- 172.21.15.67:0/2350188299 shutdown_connections 2024-03-30T02:38:33.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c483dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d47a0 0x561c483e6be0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/2350188299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d50c0 0x561c483db570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc0963d4b80 1 -- 172.21.15.67:0/2350188299 >> 172.21.15.67:0/2350188299 conn(0x561c483346a0 msgr2=0x561c483d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.316 INFO:tasks.workunit.client.0.smithi067.stderr: 12650K .......... .......... .......... .......... .......... 9% 43.7M 10s 2024-03-30T02:38:33.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc0963d4b80 1 -- 172.21.15.67:0/2350188299 shutdown_connections 2024-03-30T02:38:33.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.292+0000 7fc0963d4b80 1 -- 172.21.15.67:0/2350188299 wait complete. 2024-03-30T02:38:33.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.293+0000 7fc0963d4b80 1 -- start start 2024-03-30T02:38:33.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.293+0000 7fc0963d4b80 1 Processor -- start 2024-03-30T02:38:33.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.293+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.293+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.293+0000 7fc0963d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c48442510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.293+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c483e5490 con 0x561c483d47a0 2024-03-30T02:38:33.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.293+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c48442a50 con 0x561c483d50c0 2024-03-30T02:38:33.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.293+0000 7fc0963d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c48442b90 con 0x561c483dffc0 2024-03-30T02:38:33.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.293+0000 7fc0963d4b80 10 client.0 _ll_register_callbacks cb 0x561c482cc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:33.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483e4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50514/0 (socket says 172.21.15.67:50514) 2024-03-30T02:38:33.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07b400700 1 -- 172.21.15.67:0/1182860322 learned_addr learned my addr 172.21.15.67:0/1182860322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:33.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07aa00700 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07be00700 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c48442510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.320 INFO:tasks.workunit.client.0.smithi067.stderr: 12700K .......... .......... .......... .......... .......... 9% 26.2M 10s 2024-03-30T02:38:33.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07aa00700 1 -- 172.21.15.67:0/1182860322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 msgr2=0x561c48442510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07aa00700 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c48442510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07aa00700 1 -- 172.21.15.67:0/1182860322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 msgr2=0x561c483e4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:33.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07aa00700 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483e4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07aa00700 1 -- 172.21.15.67:0/1182860322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c48154b60 con 0x561c483d50c0 2024-03-30T02:38:33.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07b400700 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:33.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc07aa00700 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483e4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x561c483f23e0 tx=0x7fc06801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=675fa27c51cc1e1f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:33.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.294+0000 7fc065600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:38:33.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.295+0000 7fc065600700 1 -- 172.21.15.67:0/1182860322 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068026040 con 0x561c483d50c0 2024-03-30T02:38:33.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.295+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1182860322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc06c01e040 con 0x561c483d50c0 2024-03-30T02:38:33.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.295+0000 7fc065600700 1 -- 172.21.15.67:0/1182860322 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fc068003ae0 con 0x561c483d50c0 2024-03-30T02:38:33.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.295+0000 7fc065600700 1 -- 172.21.15.67:0/1182860322 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc068003ca0 con 0x561c483d50c0 2024-03-30T02:38:33.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.295+0000 7fc0963d4b80 10 client.26114 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:33.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.295+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1182860322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561c481accd0 con 0x561c483d50c0 2024-03-30T02:38:33.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.295+0000 7fc065600700 1 -- 172.21.15.67:0/1182860322 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc06c01e040 con 0x561c483d50c0 2024-03-30T02:38:33.324 INFO:tasks.workunit.client.0.smithi067.stderr: 12750K .......... .......... .......... .......... .......... 9% 36.0M 10s 2024-03-30T02:38:33.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.295+0000 7fc065600700 1 -- 172.21.15.67:0/1182860322 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc068005320 con 0x561c483d50c0 2024-03-30T02:38:33.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.295+0000 7fc0963d4b80 10 client.26114 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:33.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.296+0000 7fc0963d4b80 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:33.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.296+0000 7fc0963d4b80 20 client.26114 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:33.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.296+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561c48444300 0x561c48446720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:33.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.296+0000 7fc0963d4b80 4 client.26114 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:38:33.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.296+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1182860322 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561c48175cf0 con 0x561c48444300 2024-03-30T02:38:33.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.296+0000 7fc07b400700 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561c48444300 0x561c48446720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:33.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.296+0000 7fc07b400700 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561c48444300 0x561c48446720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b196407d3875fc3 server_cookie=3541a4f47c63861a in_seq=0 out_seq=0 2024-03-30T02:38:33.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.296+0000 7fc065600700 10 client.26114 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:33.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc065600700 1 -- 172.21.15.67:0/1182860322 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561c48175cf0 con 0x561c48444300 2024-03-30T02:38:33.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc065600700 10 client.26114 handle_command_reply: tid=0 2024-03-30T02:38:33.328 INFO:tasks.workunit.client.0.smithi067.stderr: 12800K .......... .......... .......... .......... .......... 9% 38.7M 10s 2024-03-30T02:38:33.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 client.26114 shutdown 2024-03-30T02:38:33.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1182860322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 msgr2=0x561c483e4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483e4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x561c483f23e0 tx=0x7fc06801b000 comp rx=0 tx=0).stop 2024-03-30T02:38:33.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1182860322 shutdown_connections 2024-03-30T02:38:33.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561c48444300 0x561c48446720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561c483d47a0 0x561c483e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c483dffc0 0x561c48442510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc065600700 0 client.26114 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:33.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 --2- 172.21.15.67:0/1182860322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561c483d50c0 0x561c483e4dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1182860322 >> 172.21.15.67:0/1182860322 conn(0x561c483346a0 msgr2=0x561c483db490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1182860322 shutdown_connections 2024-03-30T02:38:33.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 1 -- 172.21.15.67:0/1182860322 wait complete. 2024-03-30T02:38:33.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.297+0000 7fc0963d4b80 20 client.26114 trim_cache size 0 max 16384 2024-03-30T02:38:33.332 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:38:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 -- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0600a13b0 msgr2=0x7fc0600a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0600a13b0 0x7fc0600a3870 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc07c19a710 tx=0x7fc068009f90 comp rx=0 tx=0).stop 2024-03-30T02:38:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 -- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 msgr2=0x7fc07c19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 0x7fc07c19daa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc074007ee0 tx=0x7fc07400d3b0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 -- 172.21.15.67:0/1249737871 shutdown_connections 2024-03-30T02:38:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0600a13b0 0x7fc0600a3870 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc07c105a50 0x7fc07c19daa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc07c0ffa40 0x7fc07c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 --2- 172.21.15.67:0/1249737871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc07c100400 0x7fc07c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:33.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 -- 172.21.15.67:0/1249737871 >> 172.21.15.67:0/1249737871 conn(0x7fc07c0fb550 msgr2=0x7fc07c10b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:33.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 -- 172.21.15.67:0/1249737871 shutdown_connections 2024-03-30T02:38:33.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:33.298+0000 7fc082800700 1 -- 172.21.15.67:0/1249737871 wait complete. 2024-03-30T02:38:33.343 INFO:tasks.workunit.client.0.smithi067.stderr: 12850K .......... .......... .......... .......... .......... 9% 25.4M 9s 2024-03-30T02:38:33.343 INFO:tasks.workunit.client.0.smithi067.stderr: 12900K .......... .......... .......... .......... .......... 9% 58.3M 9s 2024-03-30T02:38:33.343 INFO:tasks.workunit.client.0.smithi067.stderr: 12950K .......... .......... .......... .......... .......... 9% 17.3M 9s 2024-03-30T02:38:33.343 INFO:tasks.workunit.client.0.smithi067.stderr: 13000K .......... .......... .......... .......... .......... 9% 34.5M 9s 2024-03-30T02:38:33.343 INFO:tasks.workunit.client.0.smithi067.stderr: 13050K .......... .......... .......... .......... .......... 9% 32.3M 9s 2024-03-30T02:38:33.343 INFO:tasks.workunit.client.0.smithi067.stderr: 13100K .......... .......... .......... .......... .......... 9% 8.78M 9s 2024-03-30T02:38:33.399 INFO:tasks.workunit.client.0.smithi067.stderr: 13150K .......... .......... .......... .......... .......... 9% 4.74M 9s 2024-03-30T02:38:33.399 INFO:tasks.workunit.client.0.smithi067.stderr: 13200K .......... .......... .......... .......... .......... 9% 28.3M 9s 2024-03-30T02:38:33.399 INFO:tasks.workunit.client.0.smithi067.stderr: 13250K .......... .......... .......... .......... .......... 9% 130M 9s 2024-03-30T02:38:33.399 INFO:tasks.workunit.client.0.smithi067.stderr: 13300K .......... .......... .......... .......... .......... 9% 59.6M 9s 2024-03-30T02:38:33.399 INFO:tasks.workunit.client.0.smithi067.stderr: 13350K .......... .......... .......... .......... .......... 9% 85.7M 9s 2024-03-30T02:38:33.399 INFO:tasks.workunit.client.0.smithi067.stderr: 13400K .......... .......... .......... .......... .......... 9% 58.8M 9s 2024-03-30T02:38:33.399 INFO:tasks.workunit.client.0.smithi067.stderr: 13450K .......... .......... .......... .......... .......... 9% 83.2M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13500K .......... .......... .......... .......... .......... 9% 112M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13550K .......... .......... .......... .......... .......... 10% 32.4M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13600K .......... .......... .......... .......... .......... 10% 32.4M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13650K .......... .......... .......... .......... .......... 10% 64.0M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13700K .......... .......... .......... .......... .......... 10% 40.4M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13750K .......... .......... .......... .......... .......... 10% 29.1M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13800K .......... .......... .......... .......... .......... 10% 35.1M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13850K .......... .......... .......... .......... .......... 10% 24.9M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13900K .......... .......... .......... .......... .......... 10% 32.1M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 13950K .......... .......... .......... .......... .......... 10% 32.7M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 14000K .......... .......... .......... .......... .......... 10% 22.7M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 14050K .......... .......... .......... .......... .......... 10% 3.20M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 14100K .......... .......... .......... .......... .......... 10% 32.8M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 14150K .......... .......... .......... .......... .......... 10% 43.2M 9s 2024-03-30T02:38:33.400 INFO:tasks.workunit.client.0.smithi067.stderr: 14200K .......... .......... .......... .......... .......... 10% 54.9M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14250K .......... .......... .......... .......... .......... 10% 38.7M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14300K .......... .......... .......... .......... .......... 10% 39.9M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14350K .......... .......... .......... .......... .......... 10% 43.3M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14400K .......... .......... .......... .......... .......... 10% 48.3M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14450K .......... .......... .......... .......... .......... 10% 55.7M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14500K .......... .......... .......... .......... .......... 10% 23.8M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14550K .......... .......... .......... .......... .......... 10% 45.2M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14600K .......... .......... .......... .......... .......... 10% 22.3M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14650K .......... .......... .......... .......... .......... 10% 38.6M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14700K .......... .......... .......... .......... .......... 10% 19.4M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14750K .......... .......... .......... .......... .......... 10% 19.9M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14800K .......... .......... .......... .......... .......... 10% 25.4M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14850K .......... .......... .......... .......... .......... 10% 28.6M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14900K .......... .......... .......... .......... .......... 11% 22.8M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 14950K .......... .......... .......... .......... .......... 11% 21.1M 9s 2024-03-30T02:38:33.401 INFO:tasks.workunit.client.0.smithi067.stderr: 15000K .......... .......... .......... .......... .......... 11% 11.2M 9s 2024-03-30T02:38:33.423 INFO:tasks.workunit.client.0.smithi067.stderr: 15050K .......... .......... .......... .......... .......... 11% 4.89M 9s 2024-03-30T02:38:33.423 INFO:tasks.workunit.client.0.smithi067.stderr: 15100K .......... .......... .......... .......... .......... 11% 25.7M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15150K .......... .......... .......... .......... .......... 11% 40.6M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15200K .......... .......... .......... .......... .......... 11% 35.0M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15250K .......... .......... .......... .......... .......... 11% 31.0M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15300K .......... .......... .......... .......... .......... 11% 29.7M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15350K .......... .......... .......... .......... .......... 11% 38.4M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15400K .......... .......... .......... .......... .......... 11% 43.0M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15450K .......... .......... .......... .......... .......... 11% 27.4M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15500K .......... .......... .......... .......... .......... 11% 18.2M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15550K .......... .......... .......... .......... .......... 11% 43.4M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15600K .......... .......... .......... .......... .......... 11% 19.9M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15650K .......... .......... .......... .......... .......... 11% 34.0M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15700K .......... .......... .......... .......... .......... 11% 24.8M 9s 2024-03-30T02:38:33.424 INFO:tasks.workunit.client.0.smithi067.stderr: 15750K .......... .......... .......... .......... .......... 11% 18.9M 9s 2024-03-30T02:38:33.426 INFO:tasks.workunit.client.0.smithi067.stderr: 15800K .......... .......... .......... .......... .......... 11% 25.1M 8s 2024-03-30T02:38:33.429 INFO:tasks.workunit.client.0.smithi067.stderr: 15850K .......... .......... .......... .......... .......... 11% 16.2M 8s 2024-03-30T02:38:33.431 INFO:tasks.workunit.client.0.smithi067.stderr: 15900K .......... .......... .......... .......... .......... 11% 27.7M 8s 2024-03-30T02:38:33.433 INFO:tasks.workunit.client.0.smithi067.stderr: 15950K .......... .......... .......... .......... .......... 11% 19.0M 8s 2024-03-30T02:38:33.438 INFO:tasks.workunit.client.0.smithi067.stderr: 16000K .......... .......... .......... .......... .......... 11% 9.70M 8s 2024-03-30T02:38:33.440 INFO:tasks.workunit.client.0.smithi067.stderr: 16050K .......... .......... .......... .......... .......... 11% 28.6M 8s 2024-03-30T02:38:33.449 INFO:tasks.workunit.client.0.smithi067.stderr: 16100K .......... .......... .......... .......... .......... 11% 5.29M 8s 2024-03-30T02:38:33.451 INFO:tasks.workunit.client.0.smithi067.stderr: 16150K .......... .......... .......... .......... .......... 11% 27.4M 8s 2024-03-30T02:38:33.454 INFO:tasks.workunit.client.0.smithi067.stderr: 16200K .......... .......... .......... .......... .......... 11% 21.0M 8s 2024-03-30T02:38:33.456 INFO:tasks.workunit.client.0.smithi067.stderr: 16250K .......... .......... .......... .......... .......... 12% 24.2M 8s 2024-03-30T02:38:33.457 INFO:tasks.workunit.client.0.smithi067.stderr: 16300K .......... .......... .......... .......... .......... 12% 54.8M 8s 2024-03-30T02:38:33.479 INFO:tasks.workunit.client.0.smithi067.stderr: 16350K .......... .......... .......... .......... .......... 12% 25.8M 8s 2024-03-30T02:38:33.479 INFO:tasks.workunit.client.0.smithi067.stderr: 16400K .......... .......... .......... .......... .......... 12% 37.0M 8s 2024-03-30T02:38:33.479 INFO:tasks.workunit.client.0.smithi067.stderr: 16450K .......... .......... .......... .......... .......... 12% 78.4M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16500K .......... .......... .......... .......... .......... 12% 61.5M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16550K .......... .......... .......... .......... .......... 12% 78.4M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16600K .......... .......... .......... .......... .......... 12% 38.6M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16650K .......... .......... .......... .......... .......... 12% 92.5M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16700K .......... .......... .......... .......... .......... 12% 72.8M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16750K .......... .......... .......... .......... .......... 12% 55.1M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16800K .......... .......... .......... .......... .......... 12% 86.0M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16850K .......... .......... .......... .......... .......... 12% 43.5M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16900K .......... .......... .......... .......... .......... 12% 46.3M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 16950K .......... .......... .......... .......... .......... 12% 74.3M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 17000K .......... .......... .......... .......... .......... 12% 70.9M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 17050K .......... .......... .......... .......... .......... 12% 75.9M 8s 2024-03-30T02:38:33.480 INFO:tasks.workunit.client.0.smithi067.stderr: 17100K .......... .......... .......... .......... .......... 12% 11.5M 8s 2024-03-30T02:38:33.485 INFO:tasks.workunit.client.0.smithi067.stderr: 17150K .......... .......... .......... .......... .......... 12% 4.73M 8s 2024-03-30T02:38:33.485 INFO:tasks.workunit.client.0.smithi067.stderr: 17200K .......... .......... .......... .......... .......... 12% 66.2M 8s 2024-03-30T02:38:33.487 INFO:tasks.workunit.client.0.smithi067.stderr: 17250K .......... .......... .......... .......... .......... 12% 19.4M 8s 2024-03-30T02:38:33.490 INFO:tasks.workunit.client.0.smithi067.stderr: 17300K .......... .......... .......... .......... .......... 12% 30.6M 8s 2024-03-30T02:38:33.491 INFO:tasks.workunit.client.0.smithi067.stderr: 17350K .......... .......... .......... .......... .......... 12% 28.4M 8s 2024-03-30T02:38:33.493 INFO:tasks.workunit.client.0.smithi067.stderr: 17400K .......... .......... .......... .......... .......... 12% 40.3M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17450K .......... .......... .......... .......... .......... 12% 26.8M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17500K .......... .......... .......... .......... .......... 12% 38.8M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17550K .......... .......... .......... .......... .......... 12% 67.4M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17600K .......... .......... .......... .......... .......... 13% 65.0M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17650K .......... .......... .......... .......... .......... 13% 42.1M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17700K .......... .......... .......... .......... .......... 13% 49.6M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17750K .......... .......... .......... .......... .......... 13% 81.0M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17800K .......... .......... .......... .......... .......... 13% 105M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17850K .......... .......... .......... .......... .......... 13% 78.7M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17900K .......... .......... .......... .......... .......... 13% 87.8M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 17950K .......... .......... .......... .......... .......... 13% 45.7M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 18000K .......... .......... .......... .......... .......... 13% 23.9M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 18050K .......... .......... .......... .......... .......... 13% 35.6M 8s 2024-03-30T02:38:33.512 INFO:tasks.workunit.client.0.smithi067.stderr: 18100K .......... .......... .......... .......... .......... 13% 24.8M 8s 2024-03-30T02:38:33.513 INFO:tasks.workunit.client.0.smithi067.stderr: 18150K .......... .......... .......... .......... .......... 13% 24.2M 8s 2024-03-30T02:38:33.513 INFO:tasks.workunit.client.0.smithi067.stderr: 18200K .......... .......... .......... .......... .......... 13% 49.0M 8s 2024-03-30T02:38:33.513 INFO:tasks.workunit.client.0.smithi067.stderr: 18250K .......... .......... .......... .......... .......... 13% 35.4M 8s 2024-03-30T02:38:33.520 INFO:tasks.workunit.client.0.smithi067.stderr: 18300K .......... .......... .......... .......... .......... 13% 6.10M 8s 2024-03-30T02:38:33.522 INFO:tasks.workunit.client.0.smithi067.stderr: 18350K .......... .......... .......... .......... .......... 13% 25.3M 8s 2024-03-30T02:38:33.524 INFO:tasks.workunit.client.0.smithi067.stderr: 18400K .......... .......... .......... .......... .......... 13% 21.2M 8s 2024-03-30T02:38:33.524 INFO:tasks.workunit.client.0.smithi067.stderr: 18450K .......... .......... .......... .......... .......... 13% 79.0M 8s 2024-03-30T02:38:33.528 INFO:tasks.workunit.client.0.smithi067.stderr: 18500K .......... .......... .......... .......... .......... 13% 19.7M 8s 2024-03-30T02:38:33.528 INFO:tasks.workunit.client.0.smithi067.stderr: 18550K .......... .......... .......... .......... .......... 13% 112M 8s 2024-03-30T02:38:33.546 INFO:tasks.workunit.client.0.smithi067.stderr: 18600K .......... .......... .......... .......... .......... 13% 27.4M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 18650K .......... .......... .......... .......... .......... 13% 37.2M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 18700K .......... .......... .......... .......... .......... 13% 83.2M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 18750K .......... .......... .......... .......... .......... 13% 71.0M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 18800K .......... .......... .......... .......... .......... 13% 77.1M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 18850K .......... .......... .......... .......... .......... 13% 29.1M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 18900K .......... .......... .......... .......... .......... 13% 87.0M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 18950K .......... .......... .......... .......... .......... 13% 59.4M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 19000K .......... .......... .......... .......... .......... 14% 24.2M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 19050K .......... .......... .......... .......... .......... 14% 33.7M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 19100K .......... .......... .......... .......... .......... 14% 24.4M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 19150K .......... .......... .......... .......... .......... 14% 38.6M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 19200K .......... .......... .......... .......... .......... 14% 31.1M 8s 2024-03-30T02:38:33.547 INFO:tasks.workunit.client.0.smithi067.stderr: 19250K .......... .......... .......... .......... .......... 14% 23.4M 8s 2024-03-30T02:38:33.548 INFO:tasks.workunit.client.0.smithi067.stderr: 19300K .......... .......... .......... .......... .......... 14% 18.6M 7s 2024-03-30T02:38:33.550 INFO:tasks.workunit.client.0.smithi067.stderr: 19350K .......... .......... .......... .......... .......... 14% 27.2M 7s 2024-03-30T02:38:33.554 INFO:tasks.workunit.client.0.smithi067.stderr: 19400K .......... .......... .......... .......... .......... 14% 20.1M 7s 2024-03-30T02:38:33.554 INFO:tasks.workunit.client.0.smithi067.stderr: 19450K .......... .......... .......... .......... .......... 14% 41.0M 7s 2024-03-30T02:38:33.558 INFO:tasks.workunit.client.0.smithi067.stderr: 19500K .......... .......... .......... .......... .......... 14% 13.2M 7s 2024-03-30T02:38:33.564 INFO:tasks.workunit.client.0.smithi067.stderr: 19550K .......... .......... .......... .......... .......... 14% 28.6M 7s 2024-03-30T02:38:33.564 INFO:tasks.workunit.client.0.smithi067.stderr: 19600K .......... .......... .......... .......... .......... 14% 66.9M 7s 2024-03-30T02:38:33.564 INFO:tasks.workunit.client.0.smithi067.stderr: 19650K .......... .......... .......... .......... .......... 14% 35.2M 7s 2024-03-30T02:38:33.564 INFO:tasks.workunit.client.0.smithi067.stderr: 19700K .......... .......... .......... .......... .......... 14% 98.2M 7s 2024-03-30T02:38:33.565 INFO:tasks.workunit.client.0.smithi067.stderr: 19750K .......... .......... .......... .......... .......... 14% 23.9M 7s 2024-03-30T02:38:33.567 INFO:tasks.workunit.client.0.smithi067.stderr: 19800K .......... .......... .......... .......... .......... 14% 20.9M 7s 2024-03-30T02:38:33.571 INFO:tasks.workunit.client.0.smithi067.stderr: 19850K .......... .......... .......... .......... .......... 14% 23.5M 7s 2024-03-30T02:38:33.571 INFO:tasks.workunit.client.0.smithi067.stderr: 19900K .......... .......... .......... .......... .......... 14% 25.4M 7s 2024-03-30T02:38:33.571 INFO:tasks.workunit.client.0.smithi067.stderr: 19950K .......... .......... .......... .......... .......... 14% 58.8M 7s 2024-03-30T02:38:33.573 INFO:tasks.workunit.client.0.smithi067.stderr: 20000K .......... .......... .......... .......... .......... 14% 23.8M 7s 2024-03-30T02:38:33.575 INFO:tasks.workunit.client.0.smithi067.stderr: 20050K .......... .......... .......... .......... .......... 14% 34.2M 7s 2024-03-30T02:38:33.577 INFO:tasks.workunit.client.0.smithi067.stderr: 20100K .......... .......... .......... .......... .......... 14% 24.2M 7s 2024-03-30T02:38:33.580 INFO:tasks.workunit.client.0.smithi067.stderr: 20150K .......... .......... .......... .......... .......... 14% 23.4M 7s 2024-03-30T02:38:33.580 INFO:tasks.workunit.client.0.smithi067.stderr: 20200K .......... .......... .......... .......... .......... 14% 79.1M 7s 2024-03-30T02:38:33.582 INFO:tasks.workunit.client.0.smithi067.stderr: 20250K .......... .......... .......... .......... .......... 14% 22.4M 7s 2024-03-30T02:38:33.583 INFO:tasks.workunit.client.0.smithi067.stderr: 20300K .......... .......... .......... .......... .......... 14% 33.0M 7s 2024-03-30T02:38:33.585 INFO:tasks.workunit.client.0.smithi067.stderr: 20350K .......... .......... .......... .......... .......... 15% 23.9M 7s 2024-03-30T02:38:33.586 INFO:tasks.workunit.client.0.smithi067.stderr: 20400K .......... .......... .......... .......... .......... 15% 33.5M 7s 2024-03-30T02:38:33.588 INFO:tasks.workunit.client.0.smithi067.stderr: 20450K .......... .......... .......... .......... .......... 15% 33.7M 7s 2024-03-30T02:38:33.590 INFO:tasks.workunit.client.0.smithi067.stderr: 20500K .......... .......... .......... .......... .......... 15% 28.1M 7s 2024-03-30T02:38:33.592 INFO:tasks.workunit.client.0.smithi067.stderr: 20550K .......... .......... .......... .......... .......... 15% 20.9M 7s 2024-03-30T02:38:33.595 INFO:tasks.workunit.client.0.smithi067.stderr: 20600K .......... .......... .......... .......... .......... 15% 26.5M 7s 2024-03-30T02:38:33.595 INFO:tasks.workunit.client.0.smithi067.stderr: 20650K .......... .......... .......... .......... .......... 15% 66.6M 7s 2024-03-30T02:38:33.596 INFO:tasks.workunit.client.0.smithi067.stderr: 20700K .......... .......... .......... .......... .......... 15% 31.6M 7s 2024-03-30T02:38:33.598 INFO:tasks.workunit.client.0.smithi067.stderr: 20750K .......... .......... .......... .......... .......... 15% 22.8M 7s 2024-03-30T02:38:33.600 INFO:tasks.workunit.client.0.smithi067.stderr: 20800K .......... .......... .......... .......... .......... 15% 22.3M 7s 2024-03-30T02:38:33.602 INFO:tasks.workunit.client.0.smithi067.stderr: 20850K .......... .......... .......... .......... .......... 15% 35.6M 7s 2024-03-30T02:38:33.603 INFO:tasks.workunit.client.0.smithi067.stderr: 20900K .......... .......... .......... .......... .......... 15% 28.7M 7s 2024-03-30T02:38:33.605 INFO:tasks.workunit.client.0.smithi067.stderr: 20950K .......... .......... .......... .......... .......... 15% 34.6M 7s 2024-03-30T02:38:33.615 INFO:tasks.workunit.client.0.smithi067.stderr: 21000K .......... .......... .......... .......... .......... 15% 26.8M 7s 2024-03-30T02:38:33.615 INFO:tasks.workunit.client.0.smithi067.stderr: 21050K .......... .......... .......... .......... .......... 15% 25.3M 7s 2024-03-30T02:38:33.615 INFO:tasks.workunit.client.0.smithi067.stderr: 21100K .......... .......... .......... .......... .......... 15% 32.5M 7s 2024-03-30T02:38:33.615 INFO:tasks.workunit.client.0.smithi067.stderr: 21150K .......... .......... .......... .......... .......... 15% 25.7M 7s 2024-03-30T02:38:33.615 INFO:tasks.workunit.client.0.smithi067.stderr: 21200K .......... .......... .......... .......... .......... 15% 50.7M 7s 2024-03-30T02:38:33.615 INFO:tasks.workunit.client.0.smithi067.stderr: 21250K .......... .......... .......... .......... .......... 15% 25.7M 7s 2024-03-30T02:38:33.617 INFO:tasks.workunit.client.0.smithi067.stderr: 21300K .......... .......... .......... .......... .......... 15% 32.9M 7s 2024-03-30T02:38:33.619 INFO:tasks.workunit.client.0.smithi067.stderr: 21350K .......... .......... .......... .......... .......... 15% 24.3M 7s 2024-03-30T02:38:33.621 INFO:tasks.workunit.client.0.smithi067.stderr: 21400K .......... .......... .......... .......... .......... 15% 25.5M 7s 2024-03-30T02:38:33.621 INFO:tasks.workunit.client.0.smithi067.stderr: 21450K .......... .......... .......... .......... .......... 15% 56.4M 7s 2024-03-30T02:38:33.623 INFO:tasks.workunit.client.0.smithi067.stderr: 21500K .......... .......... .......... .......... .......... 15% 23.4M 7s 2024-03-30T02:38:33.625 INFO:tasks.workunit.client.0.smithi067.stderr: 21550K .......... .......... .......... .......... .......... 15% 22.7M 7s 2024-03-30T02:38:33.627 INFO:tasks.workunit.client.0.smithi067.stderr: 21600K .......... .......... .......... .......... .......... 15% 36.5M 7s 2024-03-30T02:38:33.629 INFO:tasks.workunit.client.0.smithi067.stderr: 21650K .......... .......... .......... .......... .......... 15% 20.7M 7s 2024-03-30T02:38:33.632 INFO:tasks.workunit.client.0.smithi067.stderr: 21700K .......... .......... .......... .......... .......... 16% 33.4M 7s 2024-03-30T02:38:33.633 INFO:tasks.workunit.client.0.smithi067.stderr: 21750K .......... .......... .......... .......... .......... 16% 20.1M 7s 2024-03-30T02:38:33.635 INFO:tasks.workunit.client.0.smithi067.stderr: 21800K .......... .......... .......... .......... .......... 16% 27.0M 7s 2024-03-30T02:38:33.636 INFO:tasks.workunit.client.0.smithi067.stderr: 21850K .......... .......... .......... .......... .......... 16% 26.0M 7s 2024-03-30T02:38:33.639 INFO:tasks.workunit.client.0.smithi067.stderr: 21900K .......... .......... .......... .......... .......... 16% 19.7M 7s 2024-03-30T02:38:33.642 INFO:tasks.workunit.client.0.smithi067.stderr: 21950K .......... .......... .......... .......... .......... 16% 48.2M 7s 2024-03-30T02:38:33.643 INFO:tasks.workunit.client.0.smithi067.stderr: 22000K .......... .......... .......... .......... .......... 16% 20.6M 7s 2024-03-30T02:38:33.645 INFO:tasks.workunit.client.0.smithi067.stderr: 22050K .......... .......... .......... .......... .......... 16% 20.1M 7s 2024-03-30T02:38:33.649 INFO:tasks.workunit.client.0.smithi067.stderr: 22100K .......... .......... .......... .......... .......... 16% 28.2M 7s 2024-03-30T02:38:33.649 INFO:tasks.workunit.client.0.smithi067.stderr: 22150K .......... .......... .......... .......... .......... 16% 27.3M 7s 2024-03-30T02:38:33.651 INFO:tasks.workunit.client.0.smithi067.stderr: 22200K .......... .......... .......... .......... .......... 16% 19.7M 7s 2024-03-30T02:38:33.652 INFO:tasks.workunit.client.0.smithi067.stderr: 22250K .......... .......... .......... .......... .......... 16% 31.5M 7s 2024-03-30T02:38:33.654 INFO:tasks.workunit.client.0.smithi067.stderr: 22300K .......... .......... .......... .......... .......... 16% 24.6M 7s 2024-03-30T02:38:33.656 INFO:tasks.workunit.client.0.smithi067.stderr: 22350K .......... .......... .......... .......... .......... 16% 39.4M 7s 2024-03-30T02:38:33.658 INFO:tasks.workunit.client.0.smithi067.stderr: 22400K .......... .......... .......... .......... .......... 16% 31.3M 7s 2024-03-30T02:38:33.661 INFO:tasks.workunit.client.0.smithi067.stderr: 22450K .......... .......... .......... .......... .......... 16% 15.4M 7s 2024-03-30T02:38:33.661 INFO:tasks.workunit.client.0.smithi067.stderr: 22500K .......... .......... .......... .......... .......... 16% 52.1M 7s 2024-03-30T02:38:33.664 INFO:tasks.workunit.client.0.smithi067.stderr: 22550K .......... .......... .......... .......... .......... 16% 26.3M 7s 2024-03-30T02:38:33.667 INFO:tasks.workunit.client.0.smithi067.stderr: 22600K .......... .......... .......... .......... .......... 16% 21.6M 7s 2024-03-30T02:38:33.669 INFO:tasks.workunit.client.0.smithi067.stderr: 22650K .......... .......... .......... .......... .......... 16% 15.8M 7s 2024-03-30T02:38:33.674 INFO:tasks.workunit.client.0.smithi067.stderr: 22700K .......... .......... .......... .......... .......... 16% 13.5M 7s 2024-03-30T02:38:33.674 INFO:tasks.workunit.client.0.smithi067.stderr: 22750K .......... .......... .......... .......... .......... 16% 30.3M 7s 2024-03-30T02:38:33.678 INFO:tasks.workunit.client.0.smithi067.stderr: 22800K .......... .......... .......... .......... .......... 16% 12.1M 7s 2024-03-30T02:38:33.680 INFO:tasks.workunit.client.0.smithi067.stderr: 22850K .......... .......... .......... .......... .......... 16% 18.5M 7s 2024-03-30T02:38:33.683 INFO:tasks.workunit.client.0.smithi067.stderr: 22900K .......... .......... .......... .......... .......... 16% 20.8M 7s 2024-03-30T02:38:33.686 INFO:tasks.workunit.client.0.smithi067.stderr: 22950K .......... .......... .......... .......... .......... 16% 14.6M 7s 2024-03-30T02:38:33.687 INFO:tasks.workunit.client.0.smithi067.stderr: 23000K .......... .......... .......... .......... .......... 16% 52.8M 7s 2024-03-30T02:38:33.690 INFO:tasks.workunit.client.0.smithi067.stderr: 23050K .......... .......... .......... .......... .......... 17% 19.0M 7s 2024-03-30T02:38:33.692 INFO:tasks.workunit.client.0.smithi067.stderr: 23100K .......... .......... .......... .......... .......... 17% 39.0M 7s 2024-03-30T02:38:33.693 INFO:tasks.workunit.client.0.smithi067.stderr: 23150K .......... .......... .......... .......... .......... 17% 25.6M 7s 2024-03-30T02:38:33.694 INFO:tasks.workunit.client.0.smithi067.stderr: 23200K .......... .......... .......... .......... .......... 17% 32.9M 7s 2024-03-30T02:38:33.697 INFO:tasks.workunit.client.0.smithi067.stderr: 23250K .......... .......... .......... .......... .......... 17% 24.1M 7s 2024-03-30T02:38:33.699 INFO:tasks.workunit.client.0.smithi067.stderr: 23300K .......... .......... .......... .......... .......... 17% 29.7M 7s 2024-03-30T02:38:33.701 INFO:tasks.workunit.client.0.smithi067.stderr: 23350K .......... .......... .......... .......... .......... 17% 23.3M 7s 2024-03-30T02:38:33.702 INFO:tasks.workunit.client.0.smithi067.stderr: 23400K .......... .......... .......... .......... .......... 17% 25.2M 7s 2024-03-30T02:38:33.704 INFO:tasks.workunit.client.0.smithi067.stderr: 23450K .......... .......... .......... .......... .......... 17% 27.4M 7s 2024-03-30T02:38:33.706 INFO:tasks.workunit.client.0.smithi067.stderr: 23500K .......... .......... .......... .......... .......... 17% 28.7M 7s 2024-03-30T02:38:33.708 INFO:tasks.workunit.client.0.smithi067.stderr: 23550K .......... .......... .......... .......... .......... 17% 23.9M 7s 2024-03-30T02:38:33.709 INFO:tasks.workunit.client.0.smithi067.stderr: 23600K .......... .......... .......... .......... .......... 17% 35.2M 7s 2024-03-30T02:38:33.711 INFO:tasks.workunit.client.0.smithi067.stderr: 23650K .......... .......... .......... .......... .......... 17% 25.2M 7s 2024-03-30T02:38:33.763 INFO:tasks.workunit.client.0.smithi067.stderr: 23700K .......... .......... .......... .......... .......... 17% 33.5M 7s 2024-03-30T02:38:33.763 INFO:tasks.workunit.client.0.smithi067.stderr: 23750K .......... .......... .......... .......... .......... 17% 55.2M 7s 2024-03-30T02:38:33.763 INFO:tasks.workunit.client.0.smithi067.stderr: 23800K .......... .......... .......... .......... .......... 17% 68.1M 7s 2024-03-30T02:38:33.763 INFO:tasks.workunit.client.0.smithi067.stderr: 23850K .......... .......... .......... .......... .......... 17% 60.1M 7s 2024-03-30T02:38:33.763 INFO:tasks.workunit.client.0.smithi067.stderr: 23900K .......... .......... .......... .......... .......... 17% 79.1M 7s 2024-03-30T02:38:33.763 INFO:tasks.workunit.client.0.smithi067.stderr: 23950K .......... .......... .......... .......... .......... 17% 70.4M 7s 2024-03-30T02:38:33.763 INFO:tasks.workunit.client.0.smithi067.stderr: 24000K .......... .......... .......... .......... .......... 17% 83.6M 7s 2024-03-30T02:38:33.763 INFO:tasks.workunit.client.0.smithi067.stderr: 24050K .......... .......... .......... .......... .......... 17% 61.0M 7s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24100K .......... .......... .......... .......... .......... 17% 86.2M 7s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24150K .......... .......... .......... .......... .......... 17% 65.5M 7s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24200K .......... .......... .......... .......... .......... 17% 92.8M 7s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24250K .......... .......... .......... .......... .......... 17% 74.3M 7s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24300K .......... .......... .......... .......... .......... 17% 127M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24350K .......... .......... .......... .......... .......... 17% 62.8M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24400K .......... .......... .......... .......... .......... 18% 22.1M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24450K .......... .......... .......... .......... .......... 18% 43.4M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24500K .......... .......... .......... .......... .......... 18% 29.7M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24550K .......... .......... .......... .......... .......... 18% 27.1M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24600K .......... .......... .......... .......... .......... 18% 22.9M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24650K .......... .......... .......... .......... .......... 18% 38.3M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24700K .......... .......... .......... .......... .......... 18% 24.3M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24750K .......... .......... .......... .......... .......... 18% 35.0M 6s 2024-03-30T02:38:33.764 INFO:tasks.workunit.client.0.smithi067.stderr: 24800K .......... .......... .......... .......... .......... 18% 20.9M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 24850K .......... .......... .......... .......... .......... 18% 22.0M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 24900K .......... .......... .......... .......... .......... 18% 41.9M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 24950K .......... .......... .......... .......... .......... 18% 21.1M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25000K .......... .......... .......... .......... .......... 18% 30.9M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25050K .......... .......... .......... .......... .......... 18% 30.6M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25100K .......... .......... .......... .......... .......... 18% 21.1M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25150K .......... .......... .......... .......... .......... 18% 29.2M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25200K .......... .......... .......... .......... .......... 18% 20.4M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25250K .......... .......... .......... .......... .......... 18% 43.7M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25300K .......... .......... .......... .......... .......... 18% 22.6M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25350K .......... .......... .......... .......... .......... 18% 18.5M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25400K .......... .......... .......... .......... .......... 18% 44.0M 6s 2024-03-30T02:38:33.765 INFO:tasks.workunit.client.0.smithi067.stderr: 25450K .......... .......... .......... .......... .......... 18% 18.2M 6s 2024-03-30T02:38:33.774 INFO:tasks.workunit.client.0.smithi067.stderr: 25500K .......... .......... .......... .......... .......... 18% 26.2M 6s 2024-03-30T02:38:33.774 INFO:tasks.workunit.client.0.smithi067.stderr: 25550K .......... .......... .......... .......... .......... 18% 21.2M 6s 2024-03-30T02:38:33.774 INFO:tasks.workunit.client.0.smithi067.stderr: 25600K .......... .......... .......... .......... .......... 18% 40.4M 6s 2024-03-30T02:38:33.774 INFO:tasks.workunit.client.0.smithi067.stderr: 25650K .......... .......... .......... .......... .......... 18% 20.9M 6s 2024-03-30T02:38:33.774 INFO:tasks.workunit.client.0.smithi067.stderr: 25700K .......... .......... .......... .......... .......... 18% 16.0M 6s 2024-03-30T02:38:33.774 INFO:tasks.workunit.client.0.smithi067.stderr: 25750K .......... .......... .......... .......... .......... 19% 43.9M 6s 2024-03-30T02:38:33.776 INFO:tasks.workunit.client.0.smithi067.stderr: 25800K .......... .......... .......... .......... .......... 19% 19.1M 6s 2024-03-30T02:38:33.779 INFO:tasks.workunit.client.0.smithi067.stderr: 25850K .......... .......... .......... .......... .......... 19% 30.1M 6s 2024-03-30T02:38:33.780 INFO:tasks.workunit.client.0.smithi067.stderr: 25900K .......... .......... .......... .......... .......... 19% 20.7M 6s 2024-03-30T02:38:33.786 INFO:tasks.workunit.client.0.smithi067.stderr: 25950K .......... .......... .......... .......... .......... 19% 13.6M 6s 2024-03-30T02:38:33.786 INFO:tasks.workunit.client.0.smithi067.stderr: 26000K .......... .......... .......... .......... .......... 19% 96.6M 6s 2024-03-30T02:38:33.789 INFO:tasks.workunit.client.0.smithi067.stderr: 26050K .......... .......... .......... .......... .......... 19% 21.2M 6s 2024-03-30T02:38:33.790 INFO:tasks.workunit.client.0.smithi067.stderr: 26100K .......... .......... .......... .......... .......... 19% 19.9M 6s 2024-03-30T02:38:33.792 INFO:tasks.workunit.client.0.smithi067.stderr: 26150K .......... .......... .......... .......... .......... 19% 29.1M 6s 2024-03-30T02:38:33.793 INFO:tasks.workunit.client.0.smithi067.stderr: 26200K .......... .......... .......... .......... .......... 19% 19.7M 6s 2024-03-30T02:38:33.796 INFO:tasks.workunit.client.0.smithi067.stderr: 26250K .......... .......... .......... .......... .......... 19% 46.9M 6s 2024-03-30T02:38:33.797 INFO:tasks.workunit.client.0.smithi067.stderr: 26300K .......... .......... .......... .......... .......... 19% 19.8M 6s 2024-03-30T02:38:33.800 INFO:tasks.workunit.client.0.smithi067.stderr: 26350K .......... .......... .......... .......... .......... 19% 27.2M 6s 2024-03-30T02:38:33.803 INFO:tasks.workunit.client.0.smithi067.stderr: 26400K .......... .......... .......... .......... .......... 19% 20.8M 6s 2024-03-30T02:38:33.805 INFO:tasks.workunit.client.0.smithi067.stderr: 26450K .......... .......... .......... .......... .......... 19% 18.8M 6s 2024-03-30T02:38:33.805 INFO:tasks.workunit.client.0.smithi067.stderr: 26500K .......... .......... .......... .......... .......... 19% 91.3M 6s 2024-03-30T02:38:33.807 INFO:tasks.workunit.client.0.smithi067.stderr: 26550K .......... .......... .......... .......... .......... 19% 18.5M 6s 2024-03-30T02:38:33.819 INFO:tasks.workunit.client.0.smithi067.stderr: 26600K .......... .......... .......... .......... .......... 19% 25.9M 6s 2024-03-30T02:38:33.819 INFO:tasks.workunit.client.0.smithi067.stderr: 26650K .......... .......... .......... .......... .......... 19% 26.2M 6s 2024-03-30T02:38:33.819 INFO:tasks.workunit.client.0.smithi067.stderr: 26700K .......... .......... .......... .......... .......... 19% 14.3M 6s 2024-03-30T02:38:33.819 INFO:tasks.workunit.client.0.smithi067.stderr: 26750K .......... .......... .......... .......... .......... 19% 49.7M 6s 2024-03-30T02:38:33.819 INFO:tasks.workunit.client.0.smithi067.stderr: 26800K .......... .......... .......... .......... .......... 19% 24.3M 6s 2024-03-30T02:38:33.863 INFO:tasks.workunit.client.0.smithi067.stderr: 26850K .......... .......... .......... .......... .......... 19% 18.6M 6s 2024-03-30T02:38:33.863 INFO:tasks.workunit.client.0.smithi067.stderr: 26900K .......... .......... .......... .......... .......... 19% 41.1M 6s 2024-03-30T02:38:33.863 INFO:tasks.workunit.client.0.smithi067.stderr: 26950K .......... .......... .......... .......... .......... 19% 56.2M 6s 2024-03-30T02:38:33.863 INFO:tasks.workunit.client.0.smithi067.stderr: 27000K .......... .......... .......... .......... .......... 19% 63.9M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27050K .......... .......... .......... .......... .......... 19% 47.4M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27100K .......... .......... .......... .......... .......... 19% 87.7M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27150K .......... .......... .......... .......... .......... 20% 51.0M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27200K .......... .......... .......... .......... .......... 20% 52.1M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27250K .......... .......... .......... .......... .......... 20% 71.3M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27300K .......... .......... .......... .......... .......... 20% 60.0M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27350K .......... .......... .......... .......... .......... 20% 53.3M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27400K .......... .......... .......... .......... .......... 20% 73.9M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27450K .......... .......... .......... .......... .......... 20% 45.7M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27500K .......... .......... .......... .......... .......... 20% 52.5M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27550K .......... .......... .......... .......... .......... 20% 85.7M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27600K .......... .......... .......... .......... .......... 20% 48.7M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27650K .......... .......... .......... .......... .......... 20% 35.8M 6s 2024-03-30T02:38:33.864 INFO:tasks.workunit.client.0.smithi067.stderr: 27700K .......... .......... .......... .......... .......... 20% 25.0M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 27750K .......... .......... .......... .......... .......... 20% 24.7M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 27800K .......... .......... .......... .......... .......... 20% 42.6M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 27850K .......... .......... .......... .......... .......... 20% 25.0M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 27900K .......... .......... .......... .......... .......... 20% 35.4M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 27950K .......... .......... .......... .......... .......... 20% 25.9M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28000K .......... .......... .......... .......... .......... 20% 24.7M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28050K .......... .......... .......... .......... .......... 20% 40.8M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28100K .......... .......... .......... .......... .......... 20% 23.2M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28150K .......... .......... .......... .......... .......... 20% 38.1M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28200K .......... .......... .......... .......... .......... 20% 25.6M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28250K .......... .......... .......... .......... .......... 20% 34.1M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28300K .......... .......... .......... .......... .......... 20% 24.3M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28350K .......... .......... .......... .......... .......... 20% 28.0M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28400K .......... .......... .......... .......... .......... 20% 30.9M 6s 2024-03-30T02:38:33.865 INFO:tasks.workunit.client.0.smithi067.stderr: 28450K .......... .......... .......... .......... .......... 20% 35.0M 6s 2024-03-30T02:38:33.866 INFO:tasks.workunit.client.0.smithi067.stderr: 28500K .......... .......... .......... .......... .......... 21% 20.7M 6s 2024-03-30T02:38:33.871 INFO:tasks.workunit.client.0.smithi067.stderr: 28550K .......... .......... .......... .......... .......... 21% 44.2M 6s 2024-03-30T02:38:33.871 INFO:tasks.workunit.client.0.smithi067.stderr: 28600K .......... .......... .......... .......... .......... 21% 24.0M 6s 2024-03-30T02:38:33.871 INFO:tasks.workunit.client.0.smithi067.stderr: 28650K .......... .......... .......... .......... .......... 21% 32.8M 6s 2024-03-30T02:38:33.871 INFO:tasks.workunit.client.0.smithi067.stderr: 28700K .......... .......... .......... .......... .......... 21% 19.4M 6s 2024-03-30T02:38:33.873 INFO:tasks.workunit.client.0.smithi067.stderr: 28750K .......... .......... .......... .......... .......... 21% 17.6M 6s 2024-03-30T02:38:33.875 INFO:tasks.workunit.client.0.smithi067.stderr: 28800K .......... .......... .......... .......... .......... 21% 29.0M 6s 2024-03-30T02:38:33.877 INFO:tasks.workunit.client.0.smithi067.stderr: 28850K .......... .......... .......... .......... .......... 21% 22.6M 6s 2024-03-30T02:38:33.879 INFO:tasks.workunit.client.0.smithi067.stderr: 28900K .......... .......... .......... .......... .......... 21% 28.7M 6s 2024-03-30T02:38:33.881 INFO:tasks.workunit.client.0.smithi067.stderr: 28950K .......... .......... .......... .......... .......... 21% 21.8M 6s 2024-03-30T02:38:33.889 INFO:tasks.workunit.client.0.smithi067.stderr: 29000K .......... .......... .......... .......... .......... 21% 18.6M 6s 2024-03-30T02:38:33.889 INFO:tasks.workunit.client.0.smithi067.stderr: 29050K .......... .......... .......... .......... .......... 21% 47.9M 6s 2024-03-30T02:38:33.889 INFO:tasks.workunit.client.0.smithi067.stderr: 29100K .......... .......... .......... .......... .......... 21% 50.3M 6s 2024-03-30T02:38:33.890 INFO:tasks.workunit.client.0.smithi067.stderr: 29150K .......... .......... .......... .......... .......... 21% 58.5M 6s 2024-03-30T02:38:33.890 INFO:tasks.workunit.client.0.smithi067.stderr: 29200K .......... .......... .......... .......... .......... 21% 26.6M 6s 2024-03-30T02:38:33.893 INFO:tasks.workunit.client.0.smithi067.stderr: 29250K .......... .......... .......... .......... .......... 21% 21.5M 6s 2024-03-30T02:38:33.893 INFO:tasks.workunit.client.0.smithi067.stderr: 29300K .......... .......... .......... .......... .......... 21% 59.0M 6s 2024-03-30T02:38:33.893 INFO:tasks.workunit.client.0.smithi067.stderr: 29350K .......... .......... .......... .......... .......... 21% 22.5M 6s 2024-03-30T02:38:33.896 INFO:tasks.workunit.client.0.smithi067.stderr: 29400K .......... .......... .......... .......... .......... 21% 30.9M 6s 2024-03-30T02:38:33.898 INFO:tasks.workunit.client.0.smithi067.stderr: 29450K .......... .......... .......... .......... .......... 21% 24.4M 6s 2024-03-30T02:38:33.901 INFO:tasks.workunit.client.0.smithi067.stderr: 29500K .......... .......... .......... .......... .......... 21% 24.2M 6s 2024-03-30T02:38:33.901 INFO:tasks.workunit.client.0.smithi067.stderr: 29550K .......... .......... .......... .......... .......... 21% 69.2M 6s 2024-03-30T02:38:33.902 INFO:tasks.workunit.client.0.smithi067.stderr: 29600K .......... .......... .......... .......... .......... 21% 22.1M 6s 2024-03-30T02:38:33.909 INFO:tasks.workunit.client.0.smithi067.stderr: 29650K .......... .......... .......... .......... .......... 21% 18.5M 6s 2024-03-30T02:38:33.909 INFO:tasks.workunit.client.0.smithi067.stderr: 29700K .......... .......... .......... .......... .......... 21% 46.8M 6s 2024-03-30T02:38:33.909 INFO:tasks.workunit.client.0.smithi067.stderr: 29750K .......... .......... .......... .......... .......... 21% 53.0M 6s 2024-03-30T02:38:33.909 INFO:tasks.workunit.client.0.smithi067.stderr: 29800K .......... .......... .......... .......... .......... 21% 29.7M 6s 2024-03-30T02:38:33.911 INFO:tasks.workunit.client.0.smithi067.stderr: 29850K .......... .......... .......... .......... .......... 22% 24.1M 6s 2024-03-30T02:38:33.912 INFO:tasks.workunit.client.0.smithi067.stderr: 29900K .......... .......... .......... .......... .......... 22% 23.8M 6s 2024-03-30T02:38:33.916 INFO:tasks.workunit.client.0.smithi067.stderr: 29950K .......... .......... .......... .......... .......... 22% 32.6M 6s 2024-03-30T02:38:33.917 INFO:tasks.workunit.client.0.smithi067.stderr: 30000K .......... .......... .......... .......... .......... 22% 26.0M 6s 2024-03-30T02:38:33.917 INFO:tasks.workunit.client.0.smithi067.stderr: 30050K .......... .......... .......... .......... .......... 22% 54.9M 6s 2024-03-30T02:38:33.919 INFO:tasks.workunit.client.0.smithi067.stderr: 30100K .......... .......... .......... .......... .......... 22% 24.8M 6s 2024-03-30T02:38:33.921 INFO:tasks.workunit.client.0.smithi067.stderr: 30150K .......... .......... .......... .......... .......... 22% 32.2M 6s 2024-03-30T02:38:33.923 INFO:tasks.workunit.client.0.smithi067.stderr: 30200K .......... .......... .......... .......... .......... 22% 22.9M 6s 2024-03-30T02:38:33.925 INFO:tasks.workunit.client.0.smithi067.stderr: 30250K .......... .......... .......... .......... .......... 22% 27.0M 6s 2024-03-30T02:38:33.925 INFO:tasks.workunit.client.0.smithi067.stderr: 30300K .......... .......... .......... .......... .......... 22% 50.8M 6s 2024-03-30T02:38:33.927 INFO:tasks.workunit.client.0.smithi067.stderr: 30350K .......... .......... .......... .......... .......... 22% 23.9M 6s 2024-03-30T02:38:33.929 INFO:tasks.workunit.client.0.smithi067.stderr: 30400K .......... .......... .......... .......... .......... 22% 30.0M 6s 2024-03-30T02:38:33.932 INFO:tasks.workunit.client.0.smithi067.stderr: 30450K .......... .......... .......... .......... .......... 22% 22.2M 6s 2024-03-30T02:38:33.932 INFO:tasks.workunit.client.0.smithi067.stderr: 30500K .......... .......... .......... .......... .......... 22% 49.5M 6s 2024-03-30T02:38:33.934 INFO:tasks.workunit.client.0.smithi067.stderr: 30550K .......... .......... .......... .......... .......... 22% 28.8M 6s 2024-03-30T02:38:33.936 INFO:tasks.workunit.client.0.smithi067.stderr: 30600K .......... .......... .......... .......... .......... 22% 25.5M 6s 2024-03-30T02:38:33.938 INFO:tasks.workunit.client.0.smithi067.stderr: 30650K .......... .......... .......... .......... .......... 22% 24.7M 6s 2024-03-30T02:38:33.939 INFO:tasks.workunit.client.0.smithi067.stderr: 30700K .......... .......... .......... .......... .......... 22% 37.4M 6s 2024-03-30T02:38:33.942 INFO:tasks.workunit.client.0.smithi067.stderr: 30750K .......... .......... .......... .......... .......... 22% 22.3M 6s 2024-03-30T02:38:33.942 INFO:tasks.workunit.client.0.smithi067.stderr: 30800K .......... .......... .......... .......... .......... 22% 49.2M 6s 2024-03-30T02:38:33.944 INFO:tasks.workunit.client.0.smithi067.stderr: 30850K .......... .......... .......... .......... .......... 22% 23.7M 6s 2024-03-30T02:38:33.945 INFO:tasks.workunit.client.0.smithi067.stderr: 30900K .......... .......... .......... .......... .......... 22% 36.1M 6s 2024-03-30T02:38:33.948 INFO:tasks.workunit.client.0.smithi067.stderr: 30950K .......... .......... .......... .......... .......... 22% 22.9M 6s 2024-03-30T02:38:33.948 INFO:tasks.workunit.client.0.smithi067.stderr: 31000K .......... .......... .......... .......... .......... 22% 50.7M 6s 2024-03-30T02:38:33.951 INFO:tasks.workunit.client.0.smithi067.stderr: 31050K .......... .......... .......... .......... .......... 22% 25.9M 6s 2024-03-30T02:38:33.953 INFO:tasks.workunit.client.0.smithi067.stderr: 31100K .......... .......... .......... .......... .......... 22% 27.8M 6s 2024-03-30T02:38:33.955 INFO:tasks.workunit.client.0.smithi067.stderr: 31150K .......... .......... .......... .......... .......... 22% 22.3M 6s 2024-03-30T02:38:33.957 INFO:tasks.workunit.client.0.smithi067.stderr: 31200K .......... .......... .......... .......... .......... 23% 24.8M 6s 2024-03-30T02:38:33.958 INFO:tasks.workunit.client.0.smithi067.stderr: 31250K .......... .......... .......... .......... .......... 23% 44.1M 6s 2024-03-30T02:38:33.961 INFO:tasks.workunit.client.0.smithi067.stderr: 31300K .......... .......... .......... .......... .......... 23% 33.8M 6s 2024-03-30T02:38:33.961 INFO:tasks.workunit.client.0.smithi067.stderr: 31350K .......... .......... .......... .......... .......... 23% 31.5M 6s 2024-03-30T02:38:33.963 INFO:tasks.workunit.client.0.smithi067.stderr: 31400K .......... .......... .......... .......... .......... 23% 24.4M 6s 2024-03-30T02:38:33.965 INFO:tasks.workunit.client.0.smithi067.stderr: 31450K .......... .......... .......... .......... .......... 23% 27.4M 6s 2024-03-30T02:38:33.967 INFO:tasks.workunit.client.0.smithi067.stderr: 31500K .......... .......... .......... .......... .......... 23% 24.0M 5s 2024-03-30T02:38:33.967 INFO:tasks.workunit.client.0.smithi067.stderr: 31550K .......... .......... .......... .......... .......... 23% 60.0M 5s 2024-03-30T02:38:33.969 INFO:tasks.workunit.client.0.smithi067.stderr: 31600K .......... .......... .......... .......... .......... 23% 23.9M 5s 2024-03-30T02:38:33.971 INFO:tasks.workunit.client.0.smithi067.stderr: 31650K .......... .......... .......... .......... .......... 23% 28.9M 5s 2024-03-30T02:38:33.973 INFO:tasks.workunit.client.0.smithi067.stderr: 31700K .......... .......... .......... .......... .......... 23% 23.2M 5s 2024-03-30T02:38:33.975 INFO:tasks.workunit.client.0.smithi067.stderr: 31750K .......... .......... .......... .......... .......... 23% 22.7M 5s 2024-03-30T02:38:33.976 INFO:tasks.workunit.client.0.smithi067.stderr: 31800K .......... .......... .......... .......... .......... 23% 38.7M 5s 2024-03-30T02:38:33.983 INFO:tasks.workunit.client.0.smithi067.stderr: 31850K .......... .......... .......... .......... .......... 23% 23.3M 5s 2024-03-30T02:38:33.983 INFO:tasks.workunit.client.0.smithi067.stderr: 31900K .......... .......... .......... .......... .......... 23% 38.5M 5s 2024-03-30T02:38:33.983 INFO:tasks.workunit.client.0.smithi067.stderr: 31950K .......... .......... .......... .......... .......... 23% 35.5M 5s 2024-03-30T02:38:33.983 INFO:tasks.workunit.client.0.smithi067.stderr: 32000K .......... .......... .......... .......... .......... 23% 23.6M 5s 2024-03-30T02:38:33.984 INFO:tasks.workunit.client.0.smithi067.stderr: 32050K .......... .......... .......... .......... .......... 23% 37.8M 5s 2024-03-30T02:38:33.986 INFO:tasks.workunit.client.0.smithi067.stderr: 32100K .......... .......... .......... .......... .......... 23% 23.7M 5s 2024-03-30T02:38:33.988 INFO:tasks.workunit.client.0.smithi067.stderr: 32150K .......... .......... .......... .......... .......... 23% 24.7M 5s 2024-03-30T02:38:33.992 INFO:tasks.workunit.client.0.smithi067.stderr: 32200K .......... .......... .......... .......... .......... 23% 22.3M 5s 2024-03-30T02:38:33.992 INFO:tasks.workunit.client.0.smithi067.stderr: 32250K .......... .......... .......... .......... .......... 23% 43.1M 5s 2024-03-30T02:38:33.994 INFO:tasks.workunit.client.0.smithi067.stderr: 32300K .......... .......... .......... .......... .......... 23% 33.4M 5s 2024-03-30T02:38:33.995 INFO:tasks.workunit.client.0.smithi067.stderr: 32350K .......... .......... .......... .......... .......... 23% 24.2M 5s 2024-03-30T02:38:33.998 INFO:tasks.workunit.client.0.smithi067.stderr: 32400K .......... .......... .......... .......... .......... 23% 24.2M 5s 2024-03-30T02:38:34.000 INFO:tasks.workunit.client.0.smithi067.stderr: 32450K .......... .......... .......... .......... .......... 23% 25.7M 5s 2024-03-30T02:38:34.000 INFO:tasks.workunit.client.0.smithi067.stderr: 32500K .......... .......... .......... .......... .......... 23% 51.9M 5s 2024-03-30T02:38:34.002 INFO:tasks.workunit.client.0.smithi067.stderr: 32550K .......... .......... .......... .......... .......... 24% 31.3M 5s 2024-03-30T02:38:34.005 INFO:tasks.workunit.client.0.smithi067.stderr: 32600K .......... .......... .......... .......... .......... 24% 19.9M 5s 2024-03-30T02:38:34.008 INFO:tasks.workunit.client.0.smithi067.stderr: 32650K .......... .......... .......... .......... .......... 24% 19.8M 5s 2024-03-30T02:38:34.008 INFO:tasks.workunit.client.0.smithi067.stderr: 32700K .......... .......... .......... .......... .......... 24% 41.0M 5s 2024-03-30T02:38:34.010 INFO:tasks.workunit.client.0.smithi067.stderr: 32750K .......... .......... .......... .......... .......... 24% 21.5M 5s 2024-03-30T02:38:34.014 INFO:tasks.workunit.client.0.smithi067.stderr: 32800K .......... .......... .......... .......... .......... 24% 16.0M 5s 2024-03-30T02:38:34.014 INFO:tasks.workunit.client.0.smithi067.stderr: 32850K .......... .......... .......... .......... .......... 24% 42.2M 5s 2024-03-30T02:38:34.017 INFO:tasks.workunit.client.0.smithi067.stderr: 32900K .......... .......... .......... .......... .......... 24% 22.7M 5s 2024-03-30T02:38:34.021 INFO:tasks.workunit.client.0.smithi067.stderr: 32950K .......... .......... .......... .......... .......... 24% 20.9M 5s 2024-03-30T02:38:34.038 INFO:tasks.workunit.client.0.smithi067.stderr: 33000K .......... .......... .......... .......... .......... 24% 25.2M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33050K .......... .......... .......... .......... .......... 24% 18.6M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33100K .......... .......... .......... .......... .......... 24% 41.8M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33150K .......... .......... .......... .......... .......... 24% 42.3M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33200K .......... .......... .......... .......... .......... 24% 51.8M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33250K .......... .......... .......... .......... .......... 24% 43.0M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33300K .......... .......... .......... .......... .......... 24% 35.8M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33350K .......... .......... .......... .......... .......... 24% 53.5M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33400K .......... .......... .......... .......... .......... 24% 24.0M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33450K .......... .......... .......... .......... .......... 24% 34.2M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33500K .......... .......... .......... .......... .......... 24% 35.8M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33550K .......... .......... .......... .......... .......... 24% 19.3M 5s 2024-03-30T02:38:34.039 INFO:tasks.workunit.client.0.smithi067.stderr: 33600K .......... .......... .......... .......... .......... 24% 38.2M 5s 2024-03-30T02:38:34.042 INFO:tasks.workunit.client.0.smithi067.stderr: 33650K .......... .......... .......... .......... .......... 24% 28.7M 5s 2024-03-30T02:38:34.043 INFO:tasks.workunit.client.0.smithi067.stderr: 33700K .......... .......... .......... .......... .......... 24% 30.7M 5s 2024-03-30T02:38:34.045 INFO:tasks.workunit.client.0.smithi067.stderr: 33750K .......... .......... .......... .......... .......... 24% 25.0M 5s 2024-03-30T02:38:34.045 INFO:tasks.workunit.client.0.smithi067.stderr: 33800K .......... .......... .......... .......... .......... 24% 33.6M 5s 2024-03-30T02:38:34.047 INFO:tasks.workunit.client.0.smithi067.stderr: 33850K .......... .......... .......... .......... .......... 24% 31.8M 5s 2024-03-30T02:38:34.049 INFO:tasks.workunit.client.0.smithi067.stderr: 33900K .......... .......... .......... .......... .......... 25% 24.2M 5s 2024-03-30T02:38:34.051 INFO:tasks.workunit.client.0.smithi067.stderr: 33950K .......... .......... .......... .......... .......... 25% 24.1M 5s 2024-03-30T02:38:34.052 INFO:tasks.workunit.client.0.smithi067.stderr: 34000K .......... .......... .......... .......... .......... 25% 47.2M 5s 2024-03-30T02:38:34.055 INFO:tasks.workunit.client.0.smithi067.stderr: 34050K .......... .......... .......... .......... .......... 25% 22.9M 5s 2024-03-30T02:38:34.055 INFO:tasks.workunit.client.0.smithi067.stderr: 34100K .......... .......... .......... .......... .......... 25% 50.3M 5s 2024-03-30T02:38:34.057 INFO:tasks.workunit.client.0.smithi067.stderr: 34150K .......... .......... .......... .......... .......... 25% 24.9M 5s 2024-03-30T02:38:34.059 INFO:tasks.workunit.client.0.smithi067.stderr: 34200K .......... .......... .......... .......... .......... 25% 29.3M 5s 2024-03-30T02:38:34.064 INFO:tasks.workunit.client.0.smithi067.stderr: 34250K .......... .......... .......... .......... .......... 25% 22.0M 5s 2024-03-30T02:38:34.065 INFO:tasks.workunit.client.0.smithi067.stderr: 34300K .......... .......... .......... .......... .......... 25% 28.6M 5s 2024-03-30T02:38:34.065 INFO:tasks.workunit.client.0.smithi067.stderr: 34350K .......... .......... .......... .......... .......... 25% 53.3M 5s 2024-03-30T02:38:34.066 INFO:tasks.workunit.client.0.smithi067.stderr: 34400K .......... .......... .......... .......... .......... 25% 23.3M 5s 2024-03-30T02:38:34.073 INFO:tasks.workunit.client.0.smithi067.stderr: 34450K .......... .......... .......... .......... .......... 25% 31.5M 5s 2024-03-30T02:38:34.073 INFO:tasks.workunit.client.0.smithi067.stderr: 34500K .......... .......... .......... .......... .......... 25% 23.4M 5s 2024-03-30T02:38:34.073 INFO:tasks.workunit.client.0.smithi067.stderr: 34550K .......... .......... .......... .......... .......... 25% 35.2M 5s 2024-03-30T02:38:34.073 INFO:tasks.workunit.client.0.smithi067.stderr: 34600K .......... .......... .......... .......... .......... 25% 37.5M 5s 2024-03-30T02:38:34.074 INFO:tasks.workunit.client.0.smithi067.stderr: 34650K .......... .......... .......... .......... .......... 25% 22.8M 5s 2024-03-30T02:38:34.078 INFO:tasks.workunit.client.0.smithi067.stderr: 34700K .......... .......... .......... .......... .......... 25% 35.7M 5s 2024-03-30T02:38:34.079 INFO:tasks.workunit.client.0.smithi067.stderr: 34750K .......... .......... .......... .......... .......... 25% 25.8M 5s 2024-03-30T02:38:34.080 INFO:tasks.workunit.client.0.smithi067.stderr: 34800K .......... .......... .......... .......... .......... 25% 44.3M 5s 2024-03-30T02:38:34.081 INFO:tasks.workunit.client.0.smithi067.stderr: 34850K .......... .......... .......... .......... .......... 25% 25.6M 5s 2024-03-30T02:38:34.088 INFO:tasks.workunit.client.0.smithi067.stderr: 34900K .......... .......... .......... .......... .......... 25% 10.4M 5s 2024-03-30T02:38:34.088 INFO:tasks.workunit.client.0.smithi067.stderr: 34950K .......... .......... .......... .......... .......... 25% 104M 5s 2024-03-30T02:38:34.088 INFO:tasks.workunit.client.0.smithi067.stderr: 35000K .......... .......... .......... .......... .......... 25% 80.0M 5s 2024-03-30T02:38:34.088 INFO:tasks.workunit.client.0.smithi067.stderr: 35050K .......... .......... .......... .......... .......... 25% 31.2M 5s 2024-03-30T02:38:34.089 INFO:tasks.workunit.client.0.smithi067.stderr: 35100K .......... .......... .......... .......... .......... 25% 42.3M 5s 2024-03-30T02:38:34.091 INFO:tasks.workunit.client.0.smithi067.stderr: 35150K .......... .......... .......... .......... .......... 25% 22.8M 5s 2024-03-30T02:38:34.092 INFO:tasks.workunit.client.0.smithi067.stderr: 35200K .......... .......... .......... .......... .......... 25% 44.3M 5s 2024-03-30T02:38:34.095 INFO:tasks.workunit.client.0.smithi067.stderr: 35250K .......... .......... .......... .......... .......... 26% 23.9M 5s 2024-03-30T02:38:34.096 INFO:tasks.workunit.client.0.smithi067.stderr: 35300K .......... .......... .......... .......... .......... 26% 25.6M 5s 2024-03-30T02:38:34.098 INFO:tasks.workunit.client.0.smithi067.stderr: 35350K .......... .......... .......... .......... .......... 26% 28.4M 5s 2024-03-30T02:38:34.100 INFO:tasks.workunit.client.0.smithi067.stderr: 35400K .......... .......... .......... .......... .......... 26% 23.5M 5s 2024-03-30T02:38:34.102 INFO:tasks.workunit.client.0.smithi067.stderr: 35450K .......... .......... .......... .......... .......... 26% 24.3M 5s 2024-03-30T02:38:34.104 INFO:tasks.workunit.client.0.smithi067.stderr: 35500K .......... .......... .......... .......... .......... 26% 21.4M 5s 2024-03-30T02:38:34.113 INFO:tasks.workunit.client.0.smithi067.stderr: 35550K .......... .......... .......... .......... .......... 26% 28.5M 5s 2024-03-30T02:38:34.114 INFO:tasks.workunit.client.0.smithi067.stderr: 35600K .......... .......... .......... .......... .......... 26% 88.6M 5s 2024-03-30T02:38:34.114 INFO:tasks.workunit.client.0.smithi067.stderr: 35650K .......... .......... .......... .......... .......... 26% 37.8M 5s 2024-03-30T02:38:34.114 INFO:tasks.workunit.client.0.smithi067.stderr: 35700K .......... .......... .......... .......... .......... 26% 27.3M 5s 2024-03-30T02:38:34.114 INFO:tasks.workunit.client.0.smithi067.stderr: 35750K .......... .......... .......... .......... .......... 26% 25.1M 5s 2024-03-30T02:38:34.114 INFO:tasks.workunit.client.0.smithi067.stderr: 35800K .......... .......... .......... .......... .......... 26% 23.3M 5s 2024-03-30T02:38:34.117 INFO:tasks.workunit.client.0.smithi067.stderr: 35850K .......... .......... .......... .......... .......... 26% 36.1M 5s 2024-03-30T02:38:34.118 INFO:tasks.workunit.client.0.smithi067.stderr: 35900K .......... .......... .......... .......... .......... 26% 28.7M 5s 2024-03-30T02:38:34.119 INFO:tasks.workunit.client.0.smithi067.stderr: 35950K .......... .......... .......... .......... .......... 26% 22.7M 5s 2024-03-30T02:38:34.120 INFO:tasks.workunit.client.0.smithi067.stderr: 36000K .......... .......... .......... .......... .......... 26% 28.8M 5s 2024-03-30T02:38:34.123 INFO:tasks.workunit.client.0.smithi067.stderr: 36050K .......... .......... .......... .......... .......... 26% 23.7M 5s 2024-03-30T02:38:34.123 INFO:tasks.workunit.client.0.smithi067.stderr: 36100K .......... .......... .......... .......... .......... 26% 63.0M 5s 2024-03-30T02:38:34.125 INFO:tasks.workunit.client.0.smithi067.stderr: 36150K .......... .......... .......... .......... .......... 26% 25.2M 5s 2024-03-30T02:38:34.128 INFO:tasks.workunit.client.0.smithi067.stderr: 36200K .......... .......... .......... .......... .......... 26% 22.9M 5s 2024-03-30T02:38:34.130 INFO:tasks.workunit.client.0.smithi067.stderr: 36250K .......... .......... .......... .......... .......... 26% 24.2M 5s 2024-03-30T02:38:34.130 INFO:tasks.workunit.client.0.smithi067.stderr: 36300K .......... .......... .......... .......... .......... 26% 37.4M 5s 2024-03-30T02:38:34.133 INFO:tasks.workunit.client.0.smithi067.stderr: 36350K .......... .......... .......... .......... .......... 26% 28.9M 5s 2024-03-30T02:38:34.135 INFO:tasks.workunit.client.0.smithi067.stderr: 36400K .......... .......... .......... .......... .......... 26% 23.9M 5s 2024-03-30T02:38:34.137 INFO:tasks.workunit.client.0.smithi067.stderr: 36450K .......... .......... .......... .......... .......... 26% 19.0M 5s 2024-03-30T02:38:34.140 INFO:tasks.workunit.client.0.smithi067.stderr: 36500K .......... .......... .......... .......... .......... 26% 33.8M 5s 2024-03-30T02:38:34.140 INFO:tasks.workunit.client.0.smithi067.stderr: 36550K .......... .......... .......... .......... .......... 26% 32.5M 5s 2024-03-30T02:38:34.141 INFO:tasks.workunit.client.0.smithi067.stderr: 36600K .......... .......... .......... .......... .......... 26% 38.8M 5s 2024-03-30T02:38:34.143 INFO:tasks.workunit.client.0.smithi067.stderr: 36650K .......... .......... .......... .......... .......... 27% 26.0M 5s 2024-03-30T02:38:34.145 INFO:tasks.workunit.client.0.smithi067.stderr: 36700K .......... .......... .......... .......... .......... 27% 21.4M 5s 2024-03-30T02:38:34.146 INFO:tasks.workunit.client.0.smithi067.stderr: 36750K .......... .......... .......... .......... .......... 27% 36.1M 5s 2024-03-30T02:38:34.150 INFO:tasks.workunit.client.0.smithi067.stderr: 36800K .......... .......... .......... .......... .......... 27% 24.0M 5s 2024-03-30T02:38:34.150 INFO:tasks.workunit.client.0.smithi067.stderr: 36850K .......... .......... .......... .......... .......... 27% 46.4M 5s 2024-03-30T02:38:34.156 INFO:tasks.workunit.client.0.smithi067.stderr: 36900K .......... .......... .......... .......... .......... 27% 33.0M 5s 2024-03-30T02:38:34.156 INFO:tasks.workunit.client.0.smithi067.stderr: 36950K .......... .......... .......... .......... .......... 27% 25.6M 5s 2024-03-30T02:38:34.156 INFO:tasks.workunit.client.0.smithi067.stderr: 37000K .......... .......... .......... .......... .......... 27% 24.9M 5s 2024-03-30T02:38:34.158 INFO:tasks.workunit.client.0.smithi067.stderr: 37050K .......... .......... .......... .......... .......... 27% 24.3M 5s 2024-03-30T02:38:34.158 INFO:tasks.workunit.client.0.smithi067.stderr: 37100K .......... .......... .......... .......... .......... 27% 59.8M 5s 2024-03-30T02:38:34.161 INFO:tasks.workunit.client.0.smithi067.stderr: 37150K .......... .......... .......... .......... .......... 27% 27.5M 5s 2024-03-30T02:38:34.162 INFO:tasks.workunit.client.0.smithi067.stderr: 37200K .......... .......... .......... .......... .......... 27% 20.7M 5s 2024-03-30T02:38:34.163 INFO:tasks.workunit.client.0.smithi067.stderr: 37250K .......... .......... .......... .......... .......... 27% 41.0M 5s 2024-03-30T02:38:34.166 INFO:tasks.workunit.client.0.smithi067.stderr: 37300K .......... .......... .......... .......... .......... 27% 22.1M 5s 2024-03-30T02:38:34.166 INFO:tasks.workunit.client.0.smithi067.stderr: 37350K .......... .......... .......... .......... .......... 27% 50.7M 5s 2024-03-30T02:38:34.168 INFO:tasks.workunit.client.0.smithi067.stderr: 37400K .......... .......... .......... .......... .......... 27% 24.1M 5s 2024-03-30T02:38:34.171 INFO:tasks.workunit.client.0.smithi067.stderr: 37450K .......... .......... .......... .......... .......... 27% 31.4M 5s 2024-03-30T02:38:34.173 INFO:tasks.workunit.client.0.smithi067.stderr: 37500K .......... .......... .......... .......... .......... 27% 24.5M 5s 2024-03-30T02:38:34.173 INFO:tasks.workunit.client.0.smithi067.stderr: 37550K .......... .......... .......... .......... .......... 27% 42.1M 5s 2024-03-30T02:38:34.176 INFO:tasks.workunit.client.0.smithi067.stderr: 37600K .......... .......... .......... .......... .......... 27% 32.0M 5s 2024-03-30T02:38:34.177 INFO:tasks.workunit.client.0.smithi067.stderr: 37650K .......... .......... .......... .......... .......... 27% 23.5M 5s 2024-03-30T02:38:34.179 INFO:tasks.workunit.client.0.smithi067.stderr: 37700K .......... .......... .......... .......... .......... 27% 23.3M 5s 2024-03-30T02:38:34.181 INFO:tasks.workunit.client.0.smithi067.stderr: 37750K .......... .......... .......... .......... .......... 27% 27.1M 5s 2024-03-30T02:38:34.181 INFO:tasks.workunit.client.0.smithi067.stderr: 37800K .......... .......... .......... .......... .......... 27% 45.3M 5s 2024-03-30T02:38:34.185 INFO:tasks.workunit.client.0.smithi067.stderr: 37850K .......... .......... .......... .......... .......... 27% 23.9M 5s 2024-03-30T02:38:34.185 INFO:tasks.workunit.client.0.smithi067.stderr: 37900K .......... .......... .......... .......... .......... 27% 40.4M 5s 2024-03-30T02:38:34.189 INFO:tasks.workunit.client.0.smithi067.stderr: 37950K .......... .......... .......... .......... .......... 27% 28.0M 5s 2024-03-30T02:38:34.189 INFO:tasks.workunit.client.0.smithi067.stderr: 38000K .......... .......... .......... .......... .......... 28% 27.5M 5s 2024-03-30T02:38:34.191 INFO:tasks.workunit.client.0.smithi067.stderr: 38050K .......... .......... .......... .......... .......... 28% 24.4M 5s 2024-03-30T02:38:34.193 INFO:tasks.workunit.client.0.smithi067.stderr: 38100K .......... .......... .......... .......... .......... 28% 24.4M 5s 2024-03-30T02:38:34.193 INFO:tasks.workunit.client.0.smithi067.stderr: 38150K .......... .......... .......... .......... .......... 28% 43.9M 5s 2024-03-30T02:38:34.196 INFO:tasks.workunit.client.0.smithi067.stderr: 38200K .......... .......... .......... .......... .......... 28% 35.4M 5s 2024-03-30T02:38:34.197 INFO:tasks.workunit.client.0.smithi067.stderr: 38250K .......... .......... .......... .......... .......... 28% 26.2M 5s 2024-03-30T02:38:34.200 INFO:tasks.workunit.client.0.smithi067.stderr: 38300K .......... .......... .......... .......... .......... 28% 24.2M 5s 2024-03-30T02:38:34.202 INFO:tasks.workunit.client.0.smithi067.stderr: 38350K .......... .......... .......... .......... .......... 28% 23.5M 5s 2024-03-30T02:38:34.202 INFO:tasks.workunit.client.0.smithi067.stderr: 38400K .......... .......... .......... .......... .......... 28% 47.0M 5s 2024-03-30T02:38:34.204 INFO:tasks.workunit.client.0.smithi067.stderr: 38450K .......... .......... .......... .......... .......... 28% 25.1M 5s 2024-03-30T02:38:34.206 INFO:tasks.workunit.client.0.smithi067.stderr: 38500K .......... .......... .......... .......... .......... 28% 27.9M 5s 2024-03-30T02:38:34.207 INFO:tasks.workunit.client.0.smithi067.stderr: 38550K .......... .......... .......... .......... .......... 28% 37.7M 5s 2024-03-30T02:38:34.210 INFO:tasks.workunit.client.0.smithi067.stderr: 38600K .......... .......... .......... .......... .......... 28% 19.4M 5s 2024-03-30T02:38:34.210 INFO:tasks.workunit.client.0.smithi067.stderr: 38650K .......... .......... .......... .......... .......... 28% 47.2M 5s 2024-03-30T02:38:34.212 INFO:tasks.workunit.client.0.smithi067.stderr: 38700K .......... .......... .......... .......... .......... 28% 24.5M 5s 2024-03-30T02:38:34.217 INFO:tasks.workunit.client.0.smithi067.stderr: 38750K .......... .......... .......... .......... .......... 28% 29.6M 5s 2024-03-30T02:38:34.217 INFO:tasks.workunit.client.0.smithi067.stderr: 38800K .......... .......... .......... .......... .......... 28% 25.1M 5s 2024-03-30T02:38:34.217 INFO:tasks.workunit.client.0.smithi067.stderr: 38850K .......... .......... .......... .......... .......... 28% 50.0M 5s 2024-03-30T02:38:34.219 INFO:tasks.workunit.client.0.smithi067.stderr: 38900K .......... .......... .......... .......... .......... 28% 28.6M 5s 2024-03-30T02:38:34.222 INFO:tasks.workunit.client.0.smithi067.stderr: 38950K .......... .......... .......... .......... .......... 28% 24.7M 5s 2024-03-30T02:38:34.224 INFO:tasks.workunit.client.0.smithi067.stderr: 39000K .......... .......... .......... .......... .......... 28% 25.3M 5s 2024-03-30T02:38:34.224 INFO:tasks.workunit.client.0.smithi067.stderr: 39050K .......... .......... .......... .......... .......... 28% 49.2M 5s 2024-03-30T02:38:34.226 INFO:tasks.workunit.client.0.smithi067.stderr: 39100K .......... .......... .......... .......... .......... 28% 21.9M 5s 2024-03-30T02:38:34.227 INFO:tasks.workunit.client.0.smithi067.stderr: 39150K .......... .......... .......... .......... .......... 28% 35.0M 5s 2024-03-30T02:38:34.229 INFO:tasks.workunit.client.0.smithi067.stderr: 39200K .......... .......... .......... .......... .......... 28% 23.2M 5s 2024-03-30T02:38:34.231 INFO:tasks.workunit.client.0.smithi067.stderr: 39250K .......... .......... .......... .......... .......... 28% 45.5M 5s 2024-03-30T02:38:34.233 INFO:tasks.workunit.client.0.smithi067.stderr: 39300K .......... .......... .......... .......... .......... 28% 21.7M 5s 2024-03-30T02:38:34.235 INFO:tasks.workunit.client.0.smithi067.stderr: 39350K .......... .......... .......... .......... .......... 29% 23.6M 5s 2024-03-30T02:38:34.239 INFO:tasks.workunit.client.0.smithi067.stderr: 39400K .......... .......... .......... .......... .......... 29% 29.2M 5s 2024-03-30T02:38:34.239 INFO:tasks.workunit.client.0.smithi067.stderr: 39450K .......... .......... .......... .......... .......... 29% 24.6M 5s 2024-03-30T02:38:34.239 INFO:tasks.workunit.client.0.smithi067.stderr: 39500K .......... .......... .......... .......... .......... 29% 58.7M 5s 2024-03-30T02:38:34.242 INFO:tasks.workunit.client.0.smithi067.stderr: 39550K .......... .......... .......... .......... .......... 29% 20.4M 5s 2024-03-30T02:38:34.244 INFO:tasks.workunit.client.0.smithi067.stderr: 39600K .......... .......... .......... .......... .......... 29% 25.2M 5s 2024-03-30T02:38:34.245 INFO:tasks.workunit.client.0.smithi067.stderr: 39650K .......... .......... .......... .......... .......... 29% 28.7M 5s 2024-03-30T02:38:34.247 INFO:tasks.workunit.client.0.smithi067.stderr: 39700K .......... .......... .......... .......... .......... 29% 35.4M 5s 2024-03-30T02:38:34.249 INFO:tasks.workunit.client.0.smithi067.stderr: 39750K .......... .......... .......... .......... .......... 29% 26.3M 5s 2024-03-30T02:38:34.251 INFO:tasks.workunit.client.0.smithi067.stderr: 39800K .......... .......... .......... .......... .......... 29% 32.5M 5s 2024-03-30T02:38:34.253 INFO:tasks.workunit.client.0.smithi067.stderr: 39850K .......... .......... .......... .......... .......... 29% 23.7M 5s 2024-03-30T02:38:34.254 INFO:tasks.workunit.client.0.smithi067.stderr: 39900K .......... .......... .......... .......... .......... 29% 39.8M 5s 2024-03-30T02:38:34.258 INFO:tasks.workunit.client.0.smithi067.stderr: 39950K .......... .......... .......... .......... .......... 29% 17.4M 5s 2024-03-30T02:38:34.258 INFO:tasks.workunit.client.0.smithi067.stderr: 40000K .......... .......... .......... .......... .......... 29% 36.6M 5s 2024-03-30T02:38:34.260 INFO:tasks.workunit.client.0.smithi067.stderr: 40050K .......... .......... .......... .......... .......... 29% 20.9M 5s 2024-03-30T02:38:34.261 INFO:tasks.workunit.client.0.smithi067.stderr: 40100K .......... .......... .......... .......... .......... 29% 37.2M 5s 2024-03-30T02:38:34.263 INFO:tasks.workunit.client.0.smithi067.stderr: 40150K .......... .......... .......... .......... .......... 29% 28.5M 5s 2024-03-30T02:38:34.265 INFO:tasks.workunit.client.0.smithi067.stderr: 40200K .......... .......... .......... .......... .......... 29% 27.8M 5s 2024-03-30T02:38:34.267 INFO:tasks.workunit.client.0.smithi067.stderr: 40250K .......... .......... .......... .......... .......... 29% 23.9M 5s 2024-03-30T02:38:34.270 INFO:tasks.workunit.client.0.smithi067.stderr: 40300K .......... .......... .......... .......... .......... 29% 22.1M 5s 2024-03-30T02:38:34.270 INFO:tasks.workunit.client.0.smithi067.stderr: 40350K .......... .......... .......... .......... .......... 29% 45.9M 5s 2024-03-30T02:38:34.272 INFO:tasks.workunit.client.0.smithi067.stderr: 40400K .......... .......... .......... .......... .......... 29% 23.5M 5s 2024-03-30T02:38:34.274 INFO:tasks.workunit.client.0.smithi067.stderr: 40450K .......... .......... .......... .......... .......... 29% 34.2M 5s 2024-03-30T02:38:34.276 INFO:tasks.workunit.client.0.smithi067.stderr: 40500K .......... .......... .......... .......... .......... 29% 24.6M 5s 2024-03-30T02:38:34.276 INFO:tasks.workunit.client.0.smithi067.stderr: 40550K .......... .......... .......... .......... .......... 29% 49.3M 5s 2024-03-30T02:38:34.279 INFO:tasks.workunit.client.0.smithi067.stderr: 40600K .......... .......... .......... .......... .......... 29% 21.0M 5s 2024-03-30T02:38:34.280 INFO:tasks.workunit.client.0.smithi067.stderr: 40650K .......... .......... .......... .......... .......... 29% 35.8M 5s 2024-03-30T02:38:34.282 INFO:tasks.workunit.client.0.smithi067.stderr: 40700K .......... .......... .......... .......... .......... 30% 26.7M 5s 2024-03-30T02:38:34.285 INFO:tasks.workunit.client.0.smithi067.stderr: 40750K .......... .......... .......... .......... .......... 30% 23.7M 5s 2024-03-30T02:38:34.285 INFO:tasks.workunit.client.0.smithi067.stderr: 40800K .......... .......... .......... .......... .......... 30% 49.3M 5s 2024-03-30T02:38:34.287 INFO:tasks.workunit.client.0.smithi067.stderr: 40850K .......... .......... .......... .......... .......... 30% 23.0M 5s 2024-03-30T02:38:34.289 INFO:tasks.workunit.client.0.smithi067.stderr: 40900K .......... .......... .......... .......... .......... 30% 34.5M 5s 2024-03-30T02:38:34.291 INFO:tasks.workunit.client.0.smithi067.stderr: 40950K .......... .......... .......... .......... .......... 30% 21.9M 5s 2024-03-30T02:38:34.292 INFO:tasks.workunit.client.0.smithi067.stderr: 41000K .......... .......... .......... .......... .......... 30% 39.2M 5s 2024-03-30T02:38:34.294 INFO:tasks.workunit.client.0.smithi067.stderr: 41050K .......... .......... .......... .......... .......... 30% 29.7M 5s 2024-03-30T02:38:34.295 INFO:tasks.workunit.client.0.smithi067.stderr: 41100K .......... .......... .......... .......... .......... 30% 27.5M 5s 2024-03-30T02:38:34.298 INFO:tasks.workunit.client.0.smithi067.stderr: 41150K .......... .......... .......... .......... .......... 30% 26.1M 5s 2024-03-30T02:38:34.299 INFO:tasks.workunit.client.0.smithi067.stderr: 41200K .......... .......... .......... .......... .......... 30% 24.5M 5s 2024-03-30T02:38:34.300 INFO:tasks.workunit.client.0.smithi067.stderr: 41250K .......... .......... .......... .......... .......... 30% 36.3M 5s 2024-03-30T02:38:34.302 INFO:tasks.workunit.client.0.smithi067.stderr: 41300K .......... .......... .......... .......... .......... 30% 33.8M 5s 2024-03-30T02:38:34.303 INFO:tasks.workunit.client.0.smithi067.stderr: 41350K .......... .......... .......... .......... .......... 30% 28.2M 5s 2024-03-30T02:38:34.306 INFO:tasks.workunit.client.0.smithi067.stderr: 41400K .......... .......... .......... .......... .......... 30% 22.7M 5s 2024-03-30T02:38:34.308 INFO:tasks.workunit.client.0.smithi067.stderr: 41450K .......... .......... .......... .......... .......... 30% 27.3M 5s 2024-03-30T02:38:34.309 INFO:tasks.workunit.client.0.smithi067.stderr: 41500K .......... .......... .......... .......... .......... 30% 35.9M 5s 2024-03-30T02:38:34.311 INFO:tasks.workunit.client.0.smithi067.stderr: 41550K .......... .......... .......... .......... .......... 30% 27.3M 5s 2024-03-30T02:38:34.312 INFO:tasks.workunit.client.0.smithi067.stderr: 41600K .......... .......... .......... .......... .......... 30% 33.7M 5s 2024-03-30T02:38:34.315 INFO:tasks.workunit.client.0.smithi067.stderr: 41650K .......... .......... .......... .......... .......... 30% 23.0M 5s 2024-03-30T02:38:34.315 INFO:tasks.workunit.client.0.smithi067.stderr: 41700K .......... .......... .......... .......... .......... 30% 57.1M 5s 2024-03-30T02:38:34.317 INFO:tasks.workunit.client.0.smithi067.stderr: 41750K .......... .......... .......... .......... .......... 30% 21.0M 5s 2024-03-30T02:38:34.319 INFO:tasks.workunit.client.0.smithi067.stderr: 41800K .......... .......... .......... .......... .......... 30% 37.0M 5s 2024-03-30T02:38:34.321 INFO:tasks.workunit.client.0.smithi067.stderr: 41850K .......... .......... .......... .......... .......... 30% 25.7M 5s 2024-03-30T02:38:34.323 INFO:tasks.workunit.client.0.smithi067.stderr: 41900K .......... .......... .......... .......... .......... 30% 24.8M 5s 2024-03-30T02:38:34.324 INFO:tasks.workunit.client.0.smithi067.stderr: 41950K .......... .......... .......... .......... .......... 30% 46.1M 5s 2024-03-30T02:38:34.326 INFO:tasks.workunit.client.0.smithi067.stderr: 42000K .......... .......... .......... .......... .......... 30% 20.9M 5s 2024-03-30T02:38:34.328 INFO:tasks.workunit.client.0.smithi067.stderr: 42050K .......... .......... .......... .......... .......... 31% 38.9M 5s 2024-03-30T02:38:34.329 INFO:tasks.workunit.client.0.smithi067.stderr: 42100K .......... .......... .......... .......... .......... 31% 25.3M 5s 2024-03-30T02:38:34.330 INFO:tasks.workunit.client.0.smithi067.stderr: 42150K .......... .......... .......... .......... .......... 31% 37.8M 4s 2024-03-30T02:38:34.332 INFO:tasks.workunit.client.0.smithi067.stderr: 42200K .......... .......... .......... .......... .......... 31% 28.5M 4s 2024-03-30T02:38:34.334 INFO:tasks.workunit.client.0.smithi067.stderr: 42250K .......... .......... .......... .......... .......... 31% 32.7M 4s 2024-03-30T02:38:34.337 INFO:tasks.workunit.client.0.smithi067.stderr: 42300K .......... .......... .......... .......... .......... 31% 21.1M 4s 2024-03-30T02:38:34.338 INFO:tasks.workunit.client.0.smithi067.stderr: 42350K .......... .......... .......... .......... .......... 31% 25.7M 4s 2024-03-30T02:38:34.339 INFO:tasks.workunit.client.0.smithi067.stderr: 42400K .......... .......... .......... .......... .......... 31% 32.4M 4s 2024-03-30T02:38:34.341 INFO:tasks.workunit.client.0.smithi067.stderr: 42450K .......... .......... .......... .......... .......... 31% 36.9M 4s 2024-03-30T02:38:34.343 INFO:tasks.workunit.client.0.smithi067.stderr: 42500K .......... .......... .......... .......... .......... 31% 26.4M 4s 2024-03-30T02:38:34.345 INFO:tasks.workunit.client.0.smithi067.stderr: 42550K .......... .......... .......... .......... .......... 31% 26.1M 4s 2024-03-30T02:38:34.345 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:38:34.346 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:38:34.347 INFO:tasks.workunit.client.0.smithi067.stderr: 42600K .......... .......... .......... .......... .......... 31% 56.2M 4s 2024-03-30T02:38:34.350 INFO:tasks.workunit.client.0.smithi067.stderr: 42650K .......... .......... .......... .......... .......... 31% 16.0M 4s 2024-03-30T02:38:34.350 INFO:tasks.workunit.client.0.smithi067.stderr: 42700K .......... .......... .......... .......... .......... 31% 36.3M 4s 2024-03-30T02:38:34.351 INFO:tasks.workunit.client.0.smithi067.stderr: 42750K .......... .......... .......... .......... .......... 31% 30.4M 4s 2024-03-30T02:38:34.353 INFO:tasks.workunit.client.0.smithi067.stderr: 42800K .......... .......... .......... .......... .......... 31% 34.3M 4s 2024-03-30T02:38:34.354 INFO:tasks.workunit.client.0.smithi067.stderr: 42850K .......... .......... .......... .......... .......... 31% 31.8M 4s 2024-03-30T02:38:34.357 INFO:tasks.workunit.client.0.smithi067.stderr: 42900K .......... .......... .......... .......... .......... 31% 17.2M 4s 2024-03-30T02:38:34.359 INFO:tasks.workunit.client.0.smithi067.stderr: 42950K .......... .......... .......... .......... .......... 31% 34.1M 4s 2024-03-30T02:38:34.360 INFO:tasks.workunit.client.0.smithi067.stderr: 43000K .......... .......... .......... .......... .......... 31% 26.7M 4s 2024-03-30T02:38:34.362 INFO:tasks.workunit.client.0.smithi067.stderr: 43050K .......... .......... .......... .......... .......... 31% 33.2M 4s 2024-03-30T02:38:34.363 INFO:tasks.workunit.client.0.smithi067.stderr: 43100K .......... .......... .......... .......... .......... 31% 37.0M 4s 2024-03-30T02:38:34.365 INFO:tasks.workunit.client.0.smithi067.stderr: 43150K .......... .......... .......... .......... .......... 31% 23.5M 4s 2024-03-30T02:38:34.368 INFO:tasks.workunit.client.0.smithi067.stderr: 43200K .......... .......... .......... .......... .......... 31% 20.9M 4s 2024-03-30T02:38:34.371 INFO:tasks.workunit.client.0.smithi067.stderr: 43250K .......... .......... .......... .......... .......... 31% 23.5M 4s 2024-03-30T02:38:34.371 INFO:tasks.workunit.client.0.smithi067.stderr: 43300K .......... .......... .......... .......... .......... 31% 47.8M 4s 2024-03-30T02:38:34.373 INFO:tasks.workunit.client.0.smithi067.stderr: 43350K .......... .......... .......... .......... .......... 31% 25.3M 4s 2024-03-30T02:38:34.375 INFO:tasks.workunit.client.0.smithi067.stderr: 43400K .......... .......... .......... .......... .......... 32% 25.2M 4s 2024-03-30T02:38:34.378 INFO:tasks.workunit.client.0.smithi067.stderr: 43450K .......... .......... .......... .......... .......... 32% 33.8M 4s 2024-03-30T02:38:34.380 INFO:tasks.workunit.client.0.smithi067.stderr: 43500K .......... .......... .......... .......... .......... 32% 17.2M 4s 2024-03-30T02:38:34.380 INFO:tasks.workunit.client.0.smithi067.stderr: 43550K .......... .......... .......... .......... .......... 32% 63.6M 4s 2024-03-30T02:38:34.382 INFO:tasks.workunit.client.0.smithi067.stderr: 43600K .......... .......... .......... .......... .......... 32% 29.6M 4s 2024-03-30T02:38:34.384 INFO:tasks.workunit.client.0.smithi067.stderr: 43650K .......... .......... .......... .......... .......... 32% 24.1M 4s 2024-03-30T02:38:34.386 INFO:tasks.workunit.client.0.smithi067.stderr: 43700K .......... .......... .......... .......... .......... 32% 24.0M 4s 2024-03-30T02:38:34.386 INFO:tasks.workunit.client.0.smithi067.stderr: 43750K .......... .......... .......... .......... .......... 32% 65.2M 4s 2024-03-30T02:38:34.389 INFO:tasks.workunit.client.0.smithi067.stderr: 43800K .......... .......... .......... .......... .......... 32% 24.8M 4s 2024-03-30T02:38:34.391 INFO:tasks.workunit.client.0.smithi067.stderr: 43850K .......... .......... .......... .......... .......... 32% 22.6M 4s 2024-03-30T02:38:34.392 INFO:tasks.workunit.client.0.smithi067.stderr: 43900K .......... .......... .......... .......... .......... 32% 27.9M 4s 2024-03-30T02:38:34.396 INFO:tasks.workunit.client.0.smithi067.stderr: 43950K .......... .......... .......... .......... .......... 32% 27.7M 4s 2024-03-30T02:38:34.397 INFO:tasks.workunit.client.0.smithi067.stderr: 44000K .......... .......... .......... .......... .......... 32% 49.8M 4s 2024-03-30T02:38:34.397 INFO:tasks.workunit.client.0.smithi067.stderr: 44050K .......... .......... .......... .......... .......... 32% 28.7M 4s 2024-03-30T02:38:34.399 INFO:tasks.workunit.client.0.smithi067.stderr: 44100K .......... .......... .......... .......... .......... 32% 32.1M 4s 2024-03-30T02:38:34.401 INFO:tasks.workunit.client.0.smithi067.stderr: 44150K .......... .......... .......... .......... .......... 32% 18.7M 4s 2024-03-30T02:38:34.403 INFO:tasks.workunit.client.0.smithi067.stderr: 44200K .......... .......... .......... .......... .......... 32% 30.9M 4s 2024-03-30T02:38:34.403 INFO:tasks.workunit.client.0.smithi067.stderr: 44250K .......... .......... .......... .......... .......... 32% 43.1M 4s 2024-03-30T02:38:34.405 INFO:tasks.workunit.client.0.smithi067.stderr: 44300K .......... .......... .......... .......... .......... 32% 28.9M 4s 2024-03-30T02:38:34.407 INFO:tasks.workunit.client.0.smithi067.stderr: 44350K .......... .......... .......... .......... .......... 32% 30.8M 4s 2024-03-30T02:38:34.410 INFO:tasks.workunit.client.0.smithi067.stderr: 44400K .......... .......... .......... .......... .......... 32% 19.3M 4s 2024-03-30T02:38:34.412 INFO:tasks.workunit.client.0.smithi067.stderr: 44450K .......... .......... .......... .......... .......... 32% 29.2M 4s 2024-03-30T02:38:34.412 INFO:tasks.workunit.client.0.smithi067.stderr: 44500K .......... .......... .......... .......... .......... 32% 47.0M 4s 2024-03-30T02:38:34.414 INFO:tasks.workunit.client.0.smithi067.stderr: 44550K .......... .......... .......... .......... .......... 32% 25.1M 4s 2024-03-30T02:38:34.416 INFO:tasks.workunit.client.0.smithi067.stderr: 44600K .......... .......... .......... .......... .......... 32% 29.4M 4s 2024-03-30T02:38:34.419 INFO:tasks.workunit.client.0.smithi067.stderr: 44650K .......... .......... .......... .......... .......... 32% 20.9M 4s 2024-03-30T02:38:34.421 INFO:tasks.workunit.client.0.smithi067.stderr: 44700K .......... .......... .......... .......... .......... 32% 88.8M 4s 2024-03-30T02:38:34.421 INFO:tasks.workunit.client.0.smithi067.stderr: 44750K .......... .......... .......... .......... .......... 33% 28.8M 4s 2024-03-30T02:38:34.423 INFO:tasks.workunit.client.0.smithi067.stderr: 44800K .......... .......... .......... .......... .......... 33% 21.7M 4s 2024-03-30T02:38:34.425 INFO:tasks.workunit.client.0.smithi067.stderr: 44850K .......... .......... .......... .......... .......... 33% 25.4M 4s 2024-03-30T02:38:34.425 INFO:tasks.workunit.client.0.smithi067.stderr: 44900K .......... .......... .......... .......... .......... 33% 73.1M 4s 2024-03-30T02:38:34.427 INFO:tasks.workunit.client.0.smithi067.stderr: 44950K .......... .......... .......... .......... .......... 33% 28.4M 4s 2024-03-30T02:38:34.430 INFO:tasks.workunit.client.0.smithi067.stderr: 45000K .......... .......... .......... .......... .......... 33% 26.4M 4s 2024-03-30T02:38:34.439 INFO:tasks.workunit.client.0.smithi067.stderr: 45050K .......... .......... .......... .......... .......... 33% 21.1M 4s 2024-03-30T02:38:34.439 INFO:tasks.workunit.client.0.smithi067.stderr: 45100K .......... .......... .......... .......... .......... 33% 32.4M 4s 2024-03-30T02:38:34.439 INFO:tasks.workunit.client.0.smithi067.stderr: 45150K .......... .......... .......... .......... .......... 33% 45.0M 4s 2024-03-30T02:38:34.439 INFO:tasks.workunit.client.0.smithi067.stderr: 45200K .......... .......... .......... .......... .......... 33% 23.7M 4s 2024-03-30T02:38:34.439 INFO:tasks.workunit.client.0.smithi067.stderr: 45250K .......... .......... .......... .......... .......... 33% 25.2M 4s 2024-03-30T02:38:34.442 INFO:tasks.workunit.client.0.smithi067.stderr: 45300K .......... .......... .......... .......... .......... 33% 23.7M 4s 2024-03-30T02:38:34.443 INFO:tasks.workunit.client.0.smithi067.stderr: 45350K .......... .......... .......... .......... .......... 33% 101M 4s 2024-03-30T02:38:34.443 INFO:tasks.workunit.client.0.smithi067.stderr: 45400K .......... .......... .......... .......... .......... 33% 24.6M 4s 2024-03-30T02:38:34.445 INFO:tasks.workunit.client.0.smithi067.stderr: 45450K .......... .......... .......... .......... .......... 33% 21.8M 4s 2024-03-30T02:38:34.447 INFO:tasks.workunit.client.0.smithi067.stderr: 45500K .......... .......... .......... .......... .......... 33% 22.1M 4s 2024-03-30T02:38:34.449 INFO:tasks.workunit.client.0.smithi067.stderr: 45550K .......... .......... .......... .......... .......... 33% 22.9M 4s 2024-03-30T02:38:34.451 INFO:tasks.workunit.client.0.smithi067.stderr: 45600K .......... .......... .......... .......... .......... 33% 118M 4s 2024-03-30T02:38:34.451 INFO:tasks.workunit.client.0.smithi067.stderr: 45650K .......... .......... .......... .......... .......... 33% 27.2M 4s 2024-03-30T02:38:34.454 INFO:tasks.workunit.client.0.smithi067.stderr: 45700K .......... .......... .......... .......... .......... 33% 18.7M 4s 2024-03-30T02:38:34.456 INFO:tasks.workunit.client.0.smithi067.stderr: 45750K .......... .......... .......... .......... .......... 33% 26.3M 4s 2024-03-30T02:38:34.456 INFO:tasks.workunit.client.0.smithi067.stderr: 45800K .......... .......... .......... .......... .......... 33% 68.5M 4s 2024-03-30T02:38:34.459 INFO:tasks.workunit.client.0.smithi067.stderr: 45850K .......... .......... .......... .......... .......... 33% 23.6M 4s 2024-03-30T02:38:34.460 INFO:tasks.workunit.client.0.smithi067.stderr: 45900K .......... .......... .......... .......... .......... 33% 28.5M 4s 2024-03-30T02:38:34.463 INFO:tasks.workunit.client.0.smithi067.stderr: 45950K .......... .......... .......... .......... .......... 33% 18.0M 4s 2024-03-30T02:38:34.466 INFO:tasks.workunit.client.0.smithi067.stderr: 46000K .......... .......... .......... .......... .......... 33% 22.1M 4s 2024-03-30T02:38:34.466 INFO:tasks.workunit.client.0.smithi067.stderr: 46050K .......... .......... .......... .......... .......... 33% 117M 4s 2024-03-30T02:38:34.468 INFO:tasks.workunit.client.0.smithi067.stderr: 46100K .......... .......... .......... .......... .......... 33% 21.0M 4s 2024-03-30T02:38:34.470 INFO:tasks.workunit.client.0.smithi067.stderr: 46150K .......... .......... .......... .......... .......... 34% 24.1M 4s 2024-03-30T02:38:34.472 INFO:tasks.workunit.client.0.smithi067.stderr: 46200K .......... .......... .......... .......... .......... 34% 24.4M 4s 2024-03-30T02:38:34.475 INFO:tasks.workunit.client.0.smithi067.stderr: 46250K .......... .......... .......... .......... .......... 34% 20.2M 4s 2024-03-30T02:38:34.475 INFO:tasks.workunit.client.0.smithi067.stderr: 46300K .......... .......... .......... .......... .......... 34% 121M 4s 2024-03-30T02:38:34.477 INFO:tasks.workunit.client.0.smithi067.stderr: 46350K .......... .......... .......... .......... .......... 34% 22.7M 4s 2024-03-30T02:38:34.481 INFO:tasks.workunit.client.0.smithi067.stderr: 46400K .......... .......... .......... .......... .......... 34% 23.1M 4s 2024-03-30T02:38:34.481 INFO:tasks.workunit.client.0.smithi067.stderr: 46450K .......... .......... .......... .......... .......... 34% 24.0M 4s 2024-03-30T02:38:34.481 INFO:tasks.workunit.client.0.smithi067.stderr: 46500K .......... .......... .......... .......... .......... 34% 124M 4s 2024-03-30T02:38:34.484 INFO:tasks.workunit.client.0.smithi067.stderr: 46550K .......... .......... .......... .......... .......... 34% 23.6M 4s 2024-03-30T02:38:34.486 INFO:tasks.workunit.client.0.smithi067.stderr: 46600K .......... .......... .......... .......... .......... 34% 24.2M 4s 2024-03-30T02:38:34.488 INFO:tasks.workunit.client.0.smithi067.stderr: 46650K .......... .......... .......... .......... .......... 34% 24.6M 4s 2024-03-30T02:38:34.490 INFO:tasks.workunit.client.0.smithi067.stderr: 46700K .......... .......... .......... .......... .......... 34% 24.0M 4s 2024-03-30T02:38:34.490 INFO:tasks.workunit.client.0.smithi067.stderr: 46750K .......... .......... .......... .......... .......... 34% 115M 4s 2024-03-30T02:38:34.492 INFO:tasks.workunit.client.0.smithi067.stderr: 46800K .......... .......... .......... .......... .......... 34% 24.1M 4s 2024-03-30T02:38:34.494 INFO:tasks.workunit.client.0.smithi067.stderr: 46850K .......... .......... .......... .......... .......... 34% 23.2M 4s 2024-03-30T02:38:34.497 INFO:tasks.workunit.client.0.smithi067.stderr: 46900K .......... .......... .......... .......... .......... 34% 24.6M 4s 2024-03-30T02:38:34.497 INFO:tasks.workunit.client.0.smithi067.stderr: 46950K .......... .......... .......... .......... .......... 34% 113M 4s 2024-03-30T02:38:34.536 INFO:tasks.workunit.client.0.smithi067.stderr: 47000K .......... .......... .......... .......... .......... 34% 3.64M 4s 2024-03-30T02:38:34.536 INFO:tasks.workunit.client.0.smithi067.stderr: 47050K .......... .......... .......... .......... .......... 34% 117M 4s 2024-03-30T02:38:34.536 INFO:tasks.workunit.client.0.smithi067.stderr: 47100K .......... .......... .......... .......... .......... 34% 104M 4s 2024-03-30T02:38:34.536 INFO:tasks.workunit.client.0.smithi067.stderr: 47150K .......... .......... .......... .......... .......... 34% 115M 4s 2024-03-30T02:38:34.536 INFO:tasks.workunit.client.0.smithi067.stderr: 47200K .......... .......... .......... .......... .......... 34% 119M 4s 2024-03-30T02:38:34.536 INFO:tasks.workunit.client.0.smithi067.stderr: 47250K .......... .......... .......... .......... .......... 34% 52.7M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47300K .......... .......... .......... .......... .......... 34% 59.4M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47350K .......... .......... .......... .......... .......... 34% 36.0M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47400K .......... .......... .......... .......... .......... 34% 30.5M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47450K .......... .......... .......... .......... .......... 34% 101M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47500K .......... .......... .......... .......... .......... 35% 23.6M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47550K .......... .......... .......... .......... .......... 35% 25.8M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47600K .......... .......... .......... .......... .......... 35% 22.4M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47650K .......... .......... .......... .......... .......... 35% 119M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47700K .......... .......... .......... .......... .......... 35% 22.6M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47750K .......... .......... .......... .......... .......... 35% 43.5M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47800K .......... .......... .......... .......... .......... 35% 22.9M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47850K .......... .......... .......... .......... .......... 35% 18.1M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47900K .......... .......... .......... .......... .......... 35% 124M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 47950K .......... .......... .......... .......... .......... 35% 22.8M 4s 2024-03-30T02:38:34.537 INFO:tasks.workunit.client.0.smithi067.stderr: 48000K .......... .......... .......... .......... .......... 35% 24.5M 4s 2024-03-30T02:38:34.543 INFO:tasks.workunit.client.0.smithi067.stderr: 48050K .......... .......... .......... .......... .......... 35% 25.1M 4s 2024-03-30T02:38:34.543 INFO:tasks.workunit.client.0.smithi067.stderr: 48100K .......... .......... .......... .......... .......... 35% 119M 4s 2024-03-30T02:38:34.543 INFO:tasks.workunit.client.0.smithi067.stderr: 48150K .......... .......... .......... .......... .......... 35% 26.0M 4s 2024-03-30T02:38:34.543 INFO:tasks.workunit.client.0.smithi067.stderr: 48200K .......... .......... .......... .......... .......... 35% 26.5M 4s 2024-03-30T02:38:34.545 INFO:tasks.workunit.client.0.smithi067.stderr: 48250K .......... .......... .......... .......... .......... 35% 19.4M 4s 2024-03-30T02:38:34.547 INFO:tasks.workunit.client.0.smithi067.stderr: 48300K .......... .......... .......... .......... .......... 35% 25.4M 4s 2024-03-30T02:38:34.547 INFO:tasks.workunit.client.0.smithi067.stderr: 48350K .......... .......... .......... .......... .......... 35% 101M 4s 2024-03-30T02:38:34.549 INFO:tasks.workunit.client.0.smithi067.stderr: 48400K .......... .......... .......... .......... .......... 35% 22.1M 4s 2024-03-30T02:38:34.552 INFO:tasks.workunit.client.0.smithi067.stderr: 48450K .......... .......... .......... .......... .......... 35% 23.5M 4s 2024-03-30T02:38:34.553 INFO:tasks.workunit.client.0.smithi067.stderr: 48500K .......... .......... .......... .......... .......... 35% 22.8M 4s 2024-03-30T02:38:34.555 INFO:tasks.workunit.client.0.smithi067.stderr: 48550K .......... .......... .......... .......... .......... 35% 25.2M 4s 2024-03-30T02:38:34.555 INFO:tasks.workunit.client.0.smithi067.stderr: 48600K .......... .......... .......... .......... .......... 35% 123M 4s 2024-03-30T02:38:34.560 INFO:tasks.workunit.client.0.smithi067.stderr: 48650K .......... .......... .......... .......... .......... 35% 19.8M 4s 2024-03-30T02:38:34.560 INFO:tasks.workunit.client.0.smithi067.stderr: 48700K .......... .......... .......... .......... .......... 35% 25.2M 4s 2024-03-30T02:38:34.565 INFO:tasks.workunit.client.0.smithi067.stderr: 48750K .......... .......... .......... .......... .......... 35% 20.9M 4s 2024-03-30T02:38:34.565 INFO:tasks.workunit.client.0.smithi067.stderr: 48800K .......... .......... .......... .......... .......... 35% 84.5M 4s 2024-03-30T02:38:34.565 INFO:tasks.workunit.client.0.smithi067.stderr: 48850K .......... .......... .......... .......... .......... 36% 24.0M 4s 2024-03-30T02:38:34.567 INFO:tasks.workunit.client.0.smithi067.stderr: 48900K .......... .......... .......... .......... .......... 36% 23.5M 4s 2024-03-30T02:38:34.569 INFO:tasks.workunit.client.0.smithi067.stderr: 48950K .......... .......... .......... .......... .......... 36% 24.3M 4s 2024-03-30T02:38:34.569 INFO:tasks.workunit.client.0.smithi067.stderr: 49000K .......... .......... .......... .......... .......... 36% 122M 4s 2024-03-30T02:38:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.570+0000 7f2dd0a00700 1 -- 172.21.15.67:0/1051213862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dcc100db0 msgr2=0x7f2dcc101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.570+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/1051213862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dcc100db0 0x7f2dcc101190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0098f0 tx=0x7f2dbc009c00 comp rx=0 tx=0).stop 2024-03-30T02:38:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.570+0000 7f2dd0a00700 1 -- 172.21.15.67:0/1051213862 shutdown_connections 2024-03-30T02:38:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.570+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/1051213862 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dcc100db0 0x7f2dcc101190 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.570+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/1051213862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dcc105ed0 0x7f2dcc10da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.570+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/1051213862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dcc101760 0x7f2dcc1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.570+0000 7f2dd0a00700 1 -- 172.21.15.67:0/1051213862 >> 172.21.15.67:0/1051213862 conn(0x7f2dcc0fc650 msgr2=0x7f2dcc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.570+0000 7f2dd0a00700 1 -- 172.21.15.67:0/1051213862 shutdown_connections 2024-03-30T02:38:34.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.570+0000 7f2dd0a00700 1 -- 172.21.15.67:0/1051213862 wait complete. 2024-03-30T02:38:34.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.571+0000 7f2dd0a00700 1 Processor -- start 2024-03-30T02:38:34.572 INFO:tasks.workunit.client.0.smithi067.stderr: 49050K .......... .......... .......... .......... .......... 36% 19.4M 4s 2024-03-30T02:38:34.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.572+0000 7f2dd0a00700 1 -- start start 2024-03-30T02:38:34.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.572+0000 7f2dd0a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dcc100db0 0x7f2dcc1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.572+0000 7f2dd0a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dcc105ed0 0x7f2dcc199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.572+0000 7f2dd0a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dcc19dba0 0x7f2dcc19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.572+0000 7f2dd0a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19e620 con 0x7f2dcc19dba0 2024-03-30T02:38:34.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.572+0000 7f2dd0a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19e790 con 0x7f2dcc100db0 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.572+0000 7f2dd0a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc19e950 con 0x7f2dcc105ed0 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.572+0000 7f2dc9e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dcc105ed0 0x7f2dcc199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.572+0000 7f2dcb200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dcc19dba0 0x7f2dcc19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dc9e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dcc105ed0 0x7f2dcc199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60878/0 (socket says 172.21.15.67:60878) 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dc9e00700 1 -- 172.21.15.67:0/3411955483 learned_addr learned my addr 172.21.15.67:0/3411955483 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dca800700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dcc100db0 0x7f2dcc1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dc9e00700 1 -- 172.21.15.67:0/3411955483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dcc100db0 msgr2=0x7f2dcc1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dcc100db0 0x7f2dcc1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dc9e00700 1 -- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dcc19dba0 msgr2=0x7f2dcc19e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dcc19dba0 0x7f2dcc19e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dc9e00700 1 -- 172.21.15.67:0/3411955483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc009520 con 0x7f2dcc105ed0 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dcb200700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dcc19dba0 0x7f2dcc19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:34.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dcc105ed0 0x7f2dcc199800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2db800e9f0 tx=0x7f2db800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.573+0000 7f2dc3400700 1 -- 172.21.15.67:0/3411955483 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8009690 con 0x7f2dcc105ed0 2024-03-30T02:38:34.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.574+0000 7f2dc3400700 1 -- 172.21.15.67:0/3411955483 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db800ce80 con 0x7f2dcc105ed0 2024-03-30T02:38:34.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.574+0000 7f2dd0a00700 1 -- 172.21.15.67:0/3411955483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc1a5eb0 con 0x7f2dcc105ed0 2024-03-30T02:38:34.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.574+0000 7f2dc3400700 1 -- 172.21.15.67:0/3411955483 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db8018420 con 0x7f2dcc105ed0 2024-03-30T02:38:34.577 INFO:tasks.workunit.client.0.smithi067.stderr: 49100K .......... .......... .......... .......... .......... 36% 23.7M 4s 2024-03-30T02:38:34.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.574+0000 7f2dd0a00700 1 -- 172.21.15.67:0/3411955483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc1a6380 con 0x7f2dcc105ed0 2024-03-30T02:38:34.582 INFO:tasks.workunit.client.0.smithi067.stderr: 49150K .......... .......... .......... .......... .......... 36% 21.7M 4s 2024-03-30T02:38:34.582 INFO:tasks.workunit.client.0.smithi067.stderr: 49200K .......... .......... .......... .......... .......... 36% 24.4M 4s 2024-03-30T02:38:34.582 INFO:tasks.workunit.client.0.smithi067.stderr: 49250K .......... .......... .......... .......... .......... 36% 84.7M 4s 2024-03-30T02:38:34.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.579+0000 7f2dc3400700 1 -- 172.21.15.67:0/3411955483 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 27) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f2db800c9c0 con 0x7f2dcc105ed0 2024-03-30T02:38:34.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.580+0000 7f2de4f94b80 1 Processor -- start 2024-03-30T02:38:34.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.580+0000 7f2de4f94b80 1 -- start start 2024-03-30T02:38:34.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.580+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cbec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.580+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.580+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67ccc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.580+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67c17a10 con 0x55ff67cca040 2024-03-30T02:38:34.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.580+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67c153d0 con 0x55ff67cbf140 2024-03-30T02:38:34.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.580+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67c10cc0 con 0x55ff67cbe820 2024-03-30T02:38:34.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.580+0000 7f2dc9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67ccc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67ccc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50546/0 (socket says 172.21.15.67:50546) 2024-03-30T02:38:34.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc9e00700 1 -- 172.21.15.67:0/1002811698 learned_addr learned my addr 172.21.15.67:0/1002811698 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dca800700 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dcb200700 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cbec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.586 INFO:tasks.workunit.client.0.smithi067.stderr: 49300K .......... .......... .......... .......... .......... 36% 20.6M 4s 2024-03-30T02:38:34.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc9e00700 1 -- 172.21.15.67:0/1002811698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 msgr2=0x55ff67cbec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cbec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc9e00700 1 -- 172.21.15.67:0/1002811698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 msgr2=0x55ff67cc9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc9e00700 1 -- 172.21.15.67:0/1002811698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff67a92430 con 0x55ff67cca040 2024-03-30T02:38:34.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dca800700 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:34.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc3400700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2db00a1430 0x7f2db00a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67ccc430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2db80ca520 tx=0x7f2db80d5490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b01f939711c2d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dc3400700 1 -- 172.21.15.67:0/3411955483 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f2db8014070 con 0x7f2dcc105ed0 2024-03-30T02:38:34.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.581+0000 7f2dcb200700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2db00a1430 0x7f2db00a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2db7400700 1 -- 172.21.15.67:0/1002811698 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db80961b0 con 0x55ff67cca040 2024-03-30T02:38:34.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2db7400700 1 -- 172.21.15.67:0/1002811698 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db8099c40 con 0x55ff67cca040 2024-03-30T02:38:34.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2db7400700 1 -- 172.21.15.67:0/1002811698 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db809dca0 con 0x55ff67cca040 2024-03-30T02:38:34.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1002811698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 msgr2=0x55ff67ccc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2dcb200700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2db00a1430 0x7f2db00a38f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4004090 tx=0x7f2dc400b040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67ccc430 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2db80ca520 tx=0x7f2db80d5490 comp rx=0 tx=0).stop 2024-03-30T02:38:34.591 INFO:tasks.workunit.client.0.smithi067.stderr: 49350K .......... .......... .......... .......... .......... 36% 33.5M 4s 2024-03-30T02:38:34.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1002811698 shutdown_connections 2024-03-30T02:38:34.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67ccc430 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cbec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1002811698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.582+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1002811698 >> 172.21.15.67:0/1002811698 conn(0x55ff67c1e720 msgr2=0x55ff67cbace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.583+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1002811698 shutdown_connections 2024-03-30T02:38:34.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.583+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1002811698 wait complete. 2024-03-30T02:38:34.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.583+0000 7f2de4f94b80 1 -- start start 2024-03-30T02:38:34.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.583+0000 7f2de4f94b80 1 Processor -- start 2024-03-30T02:38:34.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cc9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cc71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67bace00 con 0x55ff67cca040 2024-03-30T02:38:34.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67bacaa0 con 0x55ff67cbf140 2024-03-30T02:38:34.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67a94720 con 0x55ff67cbe820 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2de4f94b80 10 client.0 _ll_register_callbacks cb 0x55ff67bb64f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2dcb200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cc71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2dcb200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cc71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50560/0 (socket says 172.21.15.67:50560) 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2dcb200700 1 -- 172.21.15.67:0/1314657578 learned_addr learned my addr 172.21.15.67:0/1314657578 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.584+0000 7f2dca800700 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cc9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dcb200700 1 -- 172.21.15.67:0/1314657578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 msgr2=0x55ff67cc9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dcb200700 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cc9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dcb200700 1 -- 172.21.15.67:0/1314657578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 msgr2=0x55ff67cc6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dcb200700 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dcb200700 1 -- 172.21.15.67:0/1314657578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc10e2c0 con 0x55ff67cca040 2024-03-30T02:38:34.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:34.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dca800700 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cc9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:34.596 INFO:tasks.workunit.client.0.smithi067.stderr: 49400K .......... .......... .......... .......... .......... 36% 19.1M 4s 2024-03-30T02:38:34.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dcb200700 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cc71a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4008ac0 tx=0x7f2dc4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905f64d98c30d301 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dabe00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:34.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dabe00700 1 -- 172.21.15.67:0/1314657578 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400e900 con 0x55ff67cca040 2024-03-30T02:38:34.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2dabe00700 1 -- 172.21.15.67:0/1314657578 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f2dc4003c60 con 0x55ff67cca040 2024-03-30T02:38:34.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.585+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1314657578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff67bd1400 con 0x55ff67cca040 2024-03-30T02:38:34.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2dabe00700 1 -- 172.21.15.67:0/1314657578 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dc400ea60 con 0x55ff67cca040 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2de4f94b80 10 client.16074 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1314657578 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f2dcc1a6380 con 0x55ff67cca040 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2dabe00700 1 -- 172.21.15.67:0/1314657578 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff67bd1400 con 0x55ff67cca040 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2dabe00700 1 -- 172.21.15.67:0/1314657578 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dc4023c60 con 0x55ff67cca040 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2de4f94b80 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2de4f94b80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2de4f94b80 20 client.16074 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ff67cd9c10 0x55ff67cdc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2de4f94b80 4 client.16074 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:34.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.586+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1314657578 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff67c09780 con 0x55ff67cd9c10 2024-03-30T02:38:34.600 INFO:tasks.workunit.client.0.smithi067.stderr: 49450K .......... .......... .......... .......... .......... 36% 27.8M 4s 2024-03-30T02:38:34.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.587+0000 7f2dca800700 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ff67cd9c10 0x55ff67cdc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.587+0000 7f2dca800700 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ff67cd9c10 0x55ff67cdc030 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6c71d63f9c7aff server_cookie=2335ed95462a937c in_seq=0 out_seq=0 2024-03-30T02:38:34.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.587+0000 7f2dabe00700 10 client.16074 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:34.601 INFO:tasks.workunit.client.0.smithi067.stderr: 49500K .......... .......... .......... .......... .......... 36% 58.5M 4s 2024-03-30T02:38:34.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2dabe00700 1 -- 172.21.15.67:0/1314657578 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ff67c09780 con 0x55ff67cd9c10 2024-03-30T02:38:34.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2dabe00700 10 client.16074 handle_command_reply: tid=0 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2de4f94b80 1 client.16074 shutdown 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1314657578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 msgr2=0x55ff67cc71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cc71a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4008ac0 tx=0x7f2dc4004b40 comp rx=0 tx=0).stop 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1314657578 shutdown_connections 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ff67cd9c10 0x55ff67cdc030 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cc71a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cc9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1314657578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cc6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2dabe00700 0 client.16074 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.588+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1314657578 >> 172.21.15.67:0/1314657578 conn(0x55ff67c1e720 msgr2=0x55ff67cc0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.589+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1314657578 shutdown_connections 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.589+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1314657578 wait complete. 2024-03-30T02:38:34.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.589+0000 7f2de4f94b80 20 client.16074 trim_cache size 0 max 16384 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 49550K .......... .......... .......... .......... .......... 36% 26.1M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 49600K .......... .......... .......... .......... .......... 36% 29.5M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 49650K .......... .......... .......... .......... .......... 36% 20.7M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 49700K .......... .......... .......... .......... .......... 36% 28.0M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 49750K .......... .......... .......... .......... .......... 36% 56.2M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 49800K .......... .......... .......... .......... .......... 36% 25.9M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 49850K .......... .......... .......... .......... .......... 36% 27.9M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 49900K .......... .......... .......... .......... .......... 36% 21.6M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 49950K .......... .......... .......... .......... .......... 36% 83.3M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 50000K .......... .......... .......... .......... .......... 36% 24.7M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 50050K .......... .......... .......... .......... .......... 36% 25.1M 4s 2024-03-30T02:38:34.625 INFO:tasks.workunit.client.0.smithi067.stderr: 50100K .......... .......... .......... .......... .......... 36% 24.0M 4s 2024-03-30T02:38:34.626 INFO:tasks.workunit.client.0.smithi067.stderr: 50150K .......... .......... .......... .......... .......... 36% 92.5M 4s 2024-03-30T02:38:34.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.611+0000 7f2de4f94b80 1 Processor -- start 2024-03-30T02:38:34.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.611+0000 7f2de4f94b80 1 -- start start 2024-03-30T02:38:34.628 INFO:tasks.workunit.client.0.smithi067.stderr: 50200K .......... .......... .......... .......... .......... 37% 26.3M 4s 2024-03-30T02:38:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.611+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cbacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.611+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cd5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.611+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.611+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67c09780 con 0x55ff67cca040 2024-03-30T02:38:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.611+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67cd7340 con 0x55ff67cbf140 2024-03-30T02:38:34.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.611+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67cd6730 con 0x55ff67cbe820 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.611+0000 7f2dcb200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dca800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cbacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dcb200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50576/0 (socket says 172.21.15.67:50576) 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dcb200700 1 -- 172.21.15.67:0/1388049681 learned_addr learned my addr 172.21.15.67:0/1388049681 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/1388049681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cd5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dcb200700 1 -- 172.21.15.67:0/1388049681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 msgr2=0x55ff67cbacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dcb200700 1 --2- 172.21.15.67:0/1388049681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cbacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dcb200700 1 -- 172.21.15.67:0/1388049681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 msgr2=0x55ff67cd5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dcb200700 1 --2- 172.21.15.67:0/1388049681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cd5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dcb200700 1 -- 172.21.15.67:0/1388049681 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff67a3ebe0 con 0x55ff67cca040 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2dcb200700 1 --2- 172.21.15.67:0/1388049681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0d50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4006000 tx=0x7f2dc4006030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e545f389159e2c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.612+0000 7f2db7e00700 1 -- 172.21.15.67:0/1388049681 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4009920 con 0x55ff67cca040 2024-03-30T02:38:34.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2db7e00700 1 -- 172.21.15.67:0/1388049681 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dc400f040 con 0x55ff67cca040 2024-03-30T02:38:34.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2db7e00700 1 -- 172.21.15.67:0/1388049681 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400e960 con 0x55ff67cca040 2024-03-30T02:38:34.632 INFO:tasks.workunit.client.0.smithi067.stderr: 50250K .......... .......... .......... .......... .......... 37% 24.1M 4s 2024-03-30T02:38:34.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1388049681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 msgr2=0x55ff67cd0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1388049681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0d50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4006000 tx=0x7f2dc4006030 comp rx=0 tx=0).stop 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1388049681 shutdown_connections 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1388049681 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0d50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1388049681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbe820 0x55ff67cbacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/1388049681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbf140 0x55ff67cd5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1388049681 >> 172.21.15.67:0/1388049681 conn(0x55ff67c1e720 msgr2=0x55ff67cd54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1388049681 shutdown_connections 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.613+0000 7f2de4f94b80 1 -- 172.21.15.67:0/1388049681 wait complete. 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.614+0000 7f2de4f94b80 1 -- start start 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.614+0000 7f2de4f94b80 1 Processor -- start 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.614+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbe820 0x55ff67d27040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.614+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbf140 0x55ff67d27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.614+0000 7f2de4f94b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.614+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67cd06f0 con 0x55ff67cca040 2024-03-30T02:38:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.614+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67cd0830 con 0x55ff67cbe820 2024-03-30T02:38:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.614+0000 7f2de4f94b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff67cd09f0 con 0x55ff67cbf140 2024-03-30T02:38:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.614+0000 7f2de4f94b80 10 client.0 _ll_register_callbacks cb 0x55ff67bb64f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dcb200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dcb200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50578/0 (socket says 172.21.15.67:50578) 2024-03-30T02:38:34.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dcb200700 1 -- 172.21.15.67:0/868540140 learned_addr learned my addr 172.21.15.67:0/868540140 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dc9e00700 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbf140 0x55ff67d27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dca800700 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbe820 0x55ff67d27040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.637 INFO:tasks.workunit.client.0.smithi067.stderr: 50300K .......... .......... .......... .......... .......... 37% 25.0M 4s 2024-03-30T02:38:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dcb200700 1 -- 172.21.15.67:0/868540140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbf140 msgr2=0x55ff67d27580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dcb200700 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbf140 0x55ff67d27580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dcb200700 1 -- 172.21.15.67:0/868540140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbe820 msgr2=0x55ff67d27040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dcb200700 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbe820 0x55ff67d27040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dcb200700 1 -- 172.21.15.67:0/868540140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc1a6380 con 0x55ff67cca040 2024-03-30T02:38:34.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dca800700 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbe820 0x55ff67d27040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:34.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dcb200700 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40164c0 tx=0x7f2dc4006120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b9ddbf64336fcb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.615+0000 7f2dabe00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2dabe00700 1 -- 172.21.15.67:0/868540140 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40166f0 con 0x55ff67cca040 2024-03-30T02:38:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2dabe00700 1 -- 172.21.15.67:0/868540140 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f2dc40168d0 con 0x55ff67cca040 2024-03-30T02:38:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2de4f94b80 1 -- 172.21.15.67:0/868540140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2db8099c40 con 0x55ff67cca040 2024-03-30T02:38:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2dabe00700 1 -- 172.21.15.67:0/868540140 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dc4004e60 con 0x55ff67cca040 2024-03-30T02:38:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2de4f94b80 10 client.16080 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2de4f94b80 1 -- 172.21.15.67:0/868540140 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ff67bd6f20 con 0x55ff67cca040 2024-03-30T02:38:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2dabe00700 1 -- 172.21.15.67:0/868540140 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2db8099c40 con 0x55ff67cca040 2024-03-30T02:38:34.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2dabe00700 1 -- 172.21.15.67:0/868540140 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2dc4005040 con 0x55ff67cca040 2024-03-30T02:38:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2de4f94b80 10 client.16080 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2de4f94b80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2de4f94b80 20 client.16080 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ff67d2fbd0 0x55ff67d31fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2de4f94b80 4 client.16080 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:38:34.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.616+0000 7f2de4f94b80 1 -- 172.21.15.67:0/868540140 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ff67a5fd70 con 0x55ff67d2fbd0 2024-03-30T02:38:34.645 INFO:tasks.workunit.client.0.smithi067.stderr: 50350K .......... .......... .......... .......... .......... 37% 29.5M 4s 2024-03-30T02:38:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.617+0000 7f2dca800700 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ff67d2fbd0 0x55ff67d31fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.617+0000 7f2dca800700 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ff67d2fbd0 0x55ff67d31fc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d045e16f76929edb server_cookie=852c24f815afebd5 in_seq=0 out_seq=0 2024-03-30T02:38:34.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.617+0000 7f2dabe00700 10 client.16080 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:34.647 INFO:tasks.workunit.client.0.smithi067.stderr: 50400K .......... .......... .......... .......... .......... 37% 50.5M 4s 2024-03-30T02:38:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.618+0000 7f2dabe00700 1 -- 172.21.15.67:0/868540140 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ff67a5fd70 con 0x55ff67d2fbd0 2024-03-30T02:38:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.618+0000 7f2dabe00700 10 client.16080 handle_command_reply: tid=0 2024-03-30T02:38:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.618+0000 7f2de4f94b80 1 client.16080 shutdown 2024-03-30T02:38:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 -- 172.21.15.67:0/868540140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 msgr2=0x55ff67cd0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40164c0 tx=0x7f2dc4006120 comp rx=0 tx=0).stop 2024-03-30T02:38:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 -- 172.21.15.67:0/868540140 shutdown_connections 2024-03-30T02:38:34.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ff67d2fbd0 0x55ff67d31fc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ff67cca040 0x55ff67cd0120 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff67cbe820 0x55ff67d27040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 --2- 172.21.15.67:0/868540140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff67cbf140 0x55ff67d27580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 -- 172.21.15.67:0/868540140 >> 172.21.15.67:0/868540140 conn(0x55ff67c1e720 msgr2=0x55ff67cc0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2dabe00700 0 client.16080 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 -- 172.21.15.67:0/868540140 shutdown_connections 2024-03-30T02:38:34.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 1 -- 172.21.15.67:0/868540140 wait complete. 2024-03-30T02:38:34.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.619+0000 7f2de4f94b80 20 client.16080 trim_cache size 0 max 16384 2024-03-30T02:38:34.651 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:38:34.651 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:38:34.651 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "747c466b-0894-422a-862a-a9130ddd9209", 2024-03-30T02:38:34.651 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:38:34.651 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:38:34.652 INFO:tasks.workunit.client.0.smithi067.stderr: 50450K .......... .......... .......... .......... .......... 37% 23.3M 4s 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 -- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2db00a1430 msgr2=0x7f2db00a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2db00a1430 0x7f2db00a38f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4004090 tx=0x7f2dc400b040 comp rx=0 tx=0).stop 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 -- 172.21.15.67:0/3411955483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dcc105ed0 msgr2=0x7f2dcc199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dcc105ed0 0x7f2dcc199800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2db800e9f0 tx=0x7f2db800ed00 comp rx=0 tx=0).stop 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 -- 172.21.15.67:0/3411955483 shutdown_connections 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2db00a1430 0x7f2db00a38f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dcc19dba0 0x7f2dcc19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dcc100db0 0x7f2dcc1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 --2- 172.21.15.67:0/3411955483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dcc105ed0 0x7f2dcc199800 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 -- 172.21.15.67:0/3411955483 >> 172.21.15.67:0/3411955483 conn(0x7f2dcc0fc650 msgr2=0x7f2dcc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 -- 172.21.15.67:0/3411955483 shutdown_connections 2024-03-30T02:38:34.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.620+0000 7f2dd0a00700 1 -- 172.21.15.67:0/3411955483 wait complete. 2024-03-30T02:38:34.667 INFO:tasks.workunit.client.0.smithi067.stderr: 50500K .......... .......... .......... .......... .......... 37% 32.6M 4s 2024-03-30T02:38:34.667 INFO:tasks.workunit.client.0.smithi067.stderr: 50550K .......... .......... .......... .......... .......... 37% 16.8M 4s 2024-03-30T02:38:34.667 INFO:tasks.workunit.client.0.smithi067.stderr: 50600K .......... .......... .......... .......... .......... 37% 18.3M 4s 2024-03-30T02:38:34.667 INFO:tasks.workunit.client.0.smithi067.stderr: 50650K .......... .......... .......... .......... .......... 37% 49.8M 4s 2024-03-30T02:38:34.667 INFO:tasks.workunit.client.0.smithi067.stderr: 50700K .......... .......... .......... .......... .......... 37% 33.0M 4s 2024-03-30T02:38:34.667 INFO:tasks.workunit.client.0.smithi067.stderr: 50750K .......... .......... .......... .......... .......... 37% 37.6M 4s 2024-03-30T02:38:34.667 INFO:tasks.workunit.client.0.smithi067.stderr: 50800K .......... .......... .......... .......... .......... 37% 26.2M 4s 2024-03-30T02:38:34.668 INFO:tasks.workunit.client.0.smithi067.stderr: 50850K .......... .......... .......... .......... .......... 37% 34.6M 4s 2024-03-30T02:38:34.668 INFO:tasks.workunit.client.0.smithi067.stderr: 50900K .......... .......... .......... .......... .......... 37% 43.7M 4s 2024-03-30T02:38:34.668 INFO:tasks.workunit.client.0.smithi067.stderr: 50950K .......... .......... .......... .......... .......... 37% 28.1M 4s 2024-03-30T02:38:34.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:34 smithi067 ceph-mon[37808]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-30T02:38:34.670 INFO:tasks.workunit.client.0.smithi067.stderr: 51000K .......... .......... .......... .......... .......... 37% 28.6M 4s 2024-03-30T02:38:34.671 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:38:34.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:34 smithi170 ceph-mon[78798]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-30T02:38:34.709 INFO:tasks.workunit.client.0.smithi067.stderr: 51050K .......... .......... .......... .......... .......... 37% 21.0M 4s 2024-03-30T02:38:34.709 INFO:tasks.workunit.client.0.smithi067.stderr: 51100K .......... .......... .......... .......... .......... 37% 123M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51150K .......... .......... .......... .......... .......... 37% 28.1M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51200K .......... .......... .......... .......... .......... 37% 21.3M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51250K .......... .......... .......... .......... .......... 37% 25.6M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51300K .......... .......... .......... .......... .......... 37% 79.8M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51350K .......... .......... .......... .......... .......... 37% 23.9M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51400K .......... .......... .......... .......... .......... 37% 28.8M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51450K .......... .......... .......... .......... .......... 37% 19.3M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51500K .......... .......... .......... .......... .......... 37% 31.4M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51550K .......... .......... .......... .......... .......... 38% 59.6M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51600K .......... .......... .......... .......... .......... 38% 21.4M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51650K .......... .......... .......... .......... .......... 38% 30.1M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51700K .......... .......... .......... .......... .......... 38% 17.7M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51750K .......... .......... .......... .......... .......... 38% 26.9M 4s 2024-03-30T02:38:34.710 INFO:tasks.workunit.client.0.smithi067.stderr: 51800K .......... .......... .......... .......... .......... 38% 123M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 51850K .......... .......... .......... .......... .......... 38% 19.1M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 51900K .......... .......... .......... .......... .......... 38% 28.3M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 51950K .......... .......... .......... .......... .......... 38% 17.6M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52000K .......... .......... .......... .......... .......... 38% 28.7M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52050K .......... .......... .......... .......... .......... 38% 59.0M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52100K .......... .......... .......... .......... .......... 38% 21.2M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52150K .......... .......... .......... .......... .......... 38% 26.2M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52200K .......... .......... .......... .......... .......... 38% 125M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52250K .......... .......... .......... .......... .......... 38% 21.7M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52300K .......... .......... .......... .......... .......... 38% 23.3M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52350K .......... .......... .......... .......... .......... 38% 20.5M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52400K .......... .......... .......... .......... .......... 38% 26.4M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52450K .......... .......... .......... .......... .......... 38% 47.7M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52500K .......... .......... .......... .......... .......... 38% 26.3M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52550K .......... .......... .......... .......... .......... 38% 16.0M 4s 2024-03-30T02:38:34.711 INFO:tasks.workunit.client.0.smithi067.stderr: 52600K .......... .......... .......... .......... .......... 38% 18.6M 4s 2024-03-30T02:38:34.733 INFO:tasks.workunit.client.0.smithi067.stderr: 52650K .......... .......... .......... .......... .......... 38% 50.2M 4s 2024-03-30T02:38:34.733 INFO:tasks.workunit.client.0.smithi067.stderr: 52700K .......... .......... .......... .......... .......... 38% 83.3M 4s 2024-03-30T02:38:34.733 INFO:tasks.workunit.client.0.smithi067.stderr: 52750K .......... .......... .......... .......... .......... 38% 47.9M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 52800K .......... .......... .......... .......... .......... 38% 27.8M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 52850K .......... .......... .......... .......... .......... 38% 21.2M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 52900K .......... .......... .......... .......... .......... 39% 96.8M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 52950K .......... .......... .......... .......... .......... 39% 25.8M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 53000K .......... .......... .......... .......... .......... 39% 21.9M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 53050K .......... .......... .......... .......... .......... 39% 28.9M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 53100K .......... .......... .......... .......... .......... 39% 20.4M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 53150K .......... .......... .......... .......... .......... 39% 91.1M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 53200K .......... .......... .......... .......... .......... 39% 27.5M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 53250K .......... .......... .......... .......... .......... 39% 21.7M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 53300K .......... .......... .......... .......... .......... 39% 26.4M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 53350K .......... .......... .......... .......... .......... 39% 89.2M 4s 2024-03-30T02:38:34.734 INFO:tasks.workunit.client.0.smithi067.stderr: 53400K .......... .......... .......... .......... .......... 39% 22.9M 4s 2024-03-30T02:38:34.735 INFO:tasks.workunit.client.0.smithi067.stderr: 53450K .......... .......... .......... .......... .......... 39% 27.4M 4s 2024-03-30T02:38:34.735 INFO:tasks.workunit.client.0.smithi067.stderr: 53500K .......... .......... .......... .......... .......... 39% 22.5M 4s 2024-03-30T02:38:34.735 INFO:tasks.workunit.client.0.smithi067.stderr: 53550K .......... .......... .......... .......... .......... 39% 24.1M 4s 2024-03-30T02:38:34.735 INFO:tasks.workunit.client.0.smithi067.stderr: 53600K .......... .......... .......... .......... .......... 39% 100M 4s 2024-03-30T02:38:34.735 INFO:tasks.workunit.client.0.smithi067.stderr: 53650K .......... .......... .......... .......... .......... 39% 21.9M 4s 2024-03-30T02:38:34.735 INFO:tasks.workunit.client.0.smithi067.stderr: 53700K .......... .......... .......... .......... .......... 39% 26.3M 4s 2024-03-30T02:38:34.735 INFO:tasks.workunit.client.0.smithi067.stderr: 53750K .......... .......... .......... .......... .......... 39% 24.7M 4s 2024-03-30T02:38:34.738 INFO:tasks.workunit.client.0.smithi067.stderr: 53800K .......... .......... .......... .......... .......... 39% 24.9M 4s 2024-03-30T02:38:34.738 INFO:tasks.workunit.client.0.smithi067.stderr: 53850K .......... .......... .......... .......... .......... 39% 101M 4s 2024-03-30T02:38:34.738 INFO:tasks.workunit.client.0.smithi067.stderr: 53900K .......... .......... .......... .......... .......... 39% 24.2M 4s 2024-03-30T02:38:34.741 INFO:tasks.workunit.client.0.smithi067.stderr: 53950K .......... .......... .......... .......... .......... 39% 26.2M 4s 2024-03-30T02:38:34.742 INFO:tasks.workunit.client.0.smithi067.stderr: 54000K .......... .......... .......... .......... .......... 39% 22.4M 4s 2024-03-30T02:38:34.745 INFO:tasks.workunit.client.0.smithi067.stderr: 54050K .......... .......... .......... .......... .......... 39% 123M 4s 2024-03-30T02:38:34.745 INFO:tasks.workunit.client.0.smithi067.stderr: 54100K .......... .......... .......... .......... .......... 39% 24.8M 4s 2024-03-30T02:38:34.747 INFO:tasks.workunit.client.0.smithi067.stderr: 54150K .......... .......... .......... .......... .......... 39% 21.1M 4s 2024-03-30T02:38:34.749 INFO:tasks.workunit.client.0.smithi067.stderr: 54200K .......... .......... .......... .......... .......... 39% 23.7M 4s 2024-03-30T02:38:34.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:34 smithi082 ceph-mon[79176]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-30T02:38:34.751 INFO:tasks.workunit.client.0.smithi067.stderr: 54250K .......... .......... .......... .......... .......... 39% 23.7M 4s 2024-03-30T02:38:34.752 INFO:tasks.workunit.client.0.smithi067.stderr: 54300K .......... .......... .......... .......... .......... 40% 124M 4s 2024-03-30T02:38:34.754 INFO:tasks.workunit.client.0.smithi067.stderr: 54350K .......... .......... .......... .......... .......... 40% 21.0M 4s 2024-03-30T02:38:34.756 INFO:tasks.workunit.client.0.smithi067.stderr: 54400K .......... .......... .......... .......... .......... 40% 21.9M 4s 2024-03-30T02:38:34.758 INFO:tasks.workunit.client.0.smithi067.stderr: 54450K .......... .......... .......... .......... .......... 40% 26.1M 4s 2024-03-30T02:38:34.758 INFO:tasks.workunit.client.0.smithi067.stderr: 54500K .......... .......... .......... .......... .......... 40% 80.6M 4s 2024-03-30T02:38:34.761 INFO:tasks.workunit.client.0.smithi067.stderr: 54550K .......... .......... .......... .......... .......... 40% 19.5M 4s 2024-03-30T02:38:34.763 INFO:tasks.workunit.client.0.smithi067.stderr: 54600K .......... .......... .......... .......... .......... 40% 25.1M 4s 2024-03-30T02:38:34.765 INFO:tasks.workunit.client.0.smithi067.stderr: 54650K .......... .......... .......... .......... .......... 40% 21.7M 4s 2024-03-30T02:38:34.768 INFO:tasks.workunit.client.0.smithi067.stderr: 54700K .......... .......... .......... .......... .......... 40% 24.6M 4s 2024-03-30T02:38:34.768 INFO:tasks.workunit.client.0.smithi067.stderr: 54750K .......... .......... .......... .......... .......... 40% 88.8M 4s 2024-03-30T02:38:34.770 INFO:tasks.workunit.client.0.smithi067.stderr: 54800K .......... .......... .......... .......... .......... 40% 21.9M 4s 2024-03-30T02:38:34.772 INFO:tasks.workunit.client.0.smithi067.stderr: 54850K .......... .......... .......... .......... .......... 40% 23.1M 4s 2024-03-30T02:38:34.774 INFO:tasks.workunit.client.0.smithi067.stderr: 54900K .......... .......... .......... .......... .......... 40% 22.4M 4s 2024-03-30T02:38:34.774 INFO:tasks.workunit.client.0.smithi067.stderr: 54950K .......... .......... .......... .......... .......... 40% 87.4M 4s 2024-03-30T02:38:34.776 INFO:tasks.workunit.client.0.smithi067.stderr: 55000K .......... .......... .......... .......... .......... 40% 23.9M 4s 2024-03-30T02:38:34.779 INFO:tasks.workunit.client.0.smithi067.stderr: 55050K .......... .......... .......... .......... .......... 40% 22.9M 4s 2024-03-30T02:38:34.781 INFO:tasks.workunit.client.0.smithi067.stderr: 55100K .......... .......... .......... .......... .......... 40% 25.3M 4s 2024-03-30T02:38:34.801 INFO:tasks.workunit.client.0.smithi067.stderr: 55150K .......... .......... .......... .......... .......... 40% 5.85M 4s 2024-03-30T02:38:34.801 INFO:tasks.workunit.client.0.smithi067.stderr: 55200K .......... .......... .......... .......... .......... 40% 63.8M 4s 2024-03-30T02:38:34.801 INFO:tasks.workunit.client.0.smithi067.stderr: 55250K .......... .......... .......... .......... .......... 40% 93.7M 4s 2024-03-30T02:38:34.802 INFO:tasks.workunit.client.0.smithi067.stderr: 55300K .......... .......... .......... .......... .......... 40% 26.0M 4s 2024-03-30T02:38:34.802 INFO:tasks.workunit.client.0.smithi067.stderr: 55350K .......... .......... .......... .......... .......... 40% 88.1M 4s 2024-03-30T02:38:34.802 INFO:tasks.workunit.client.0.smithi067.stderr: 55400K .......... .......... .......... .......... .......... 40% 116M 4s 2024-03-30T02:38:34.802 INFO:tasks.workunit.client.0.smithi067.stderr: 55450K .......... .......... .......... .......... .......... 40% 20.2M 4s 2024-03-30T02:38:34.802 INFO:tasks.workunit.client.0.smithi067.stderr: 55500K .......... .......... .......... .......... .......... 40% 99.2M 4s 2024-03-30T02:38:34.802 INFO:tasks.workunit.client.0.smithi067.stderr: 55550K .......... .......... .......... .......... .......... 40% 19.8M 4s 2024-03-30T02:38:34.802 INFO:tasks.workunit.client.0.smithi067.stderr: 55600K .......... .......... .......... .......... .......... 40% 86.9M 4s 2024-03-30T02:38:34.802 INFO:tasks.workunit.client.0.smithi067.stderr: 55650K .......... .......... .......... .......... .......... 41% 104M 4s 2024-03-30T02:38:34.805 INFO:tasks.workunit.client.0.smithi067.stderr: 55700K .......... .......... .......... .......... .......... 41% 13.2M 4s 2024-03-30T02:38:34.805 INFO:tasks.workunit.client.0.smithi067.stderr: 55750K .......... .......... .......... .......... .......... 41% 65.5M 4s 2024-03-30T02:38:34.809 INFO:tasks.workunit.client.0.smithi067.stderr: 55800K .......... .......... .......... .......... .......... 41% 17.7M 4s 2024-03-30T02:38:34.809 INFO:tasks.workunit.client.0.smithi067.stderr: 55850K .......... .......... .......... .......... .......... 41% 89.9M 4s 2024-03-30T02:38:34.809 INFO:tasks.workunit.client.0.smithi067.stderr: 55900K .......... .......... .......... .......... .......... 41% 121M 4s 2024-03-30T02:38:34.815 INFO:tasks.workunit.client.0.smithi067.stderr: 55950K .......... .......... .......... .......... .......... 41% 13.5M 4s 2024-03-30T02:38:34.815 INFO:tasks.workunit.client.0.smithi067.stderr: 56000K .......... .......... .......... .......... .......... 41% 115M 4s 2024-03-30T02:38:34.815 INFO:tasks.workunit.client.0.smithi067.stderr: 56050K .......... .......... .......... .......... .......... 41% 16.3M 4s 2024-03-30T02:38:34.816 INFO:tasks.workunit.client.0.smithi067.stderr: 56100K .......... .......... .......... .......... .......... 41% 107M 4s 2024-03-30T02:38:34.819 INFO:tasks.workunit.client.0.smithi067.stderr: 56150K .......... .......... .......... .......... .......... 41% 15.3M 4s 2024-03-30T02:38:34.825 INFO:tasks.workunit.client.0.smithi067.stderr: 56200K .......... .......... .......... .......... .......... 41% 10.9M 4s 2024-03-30T02:38:34.825 INFO:tasks.workunit.client.0.smithi067.stderr: 56250K .......... .......... .......... .......... .......... 41% 113M 4s 2024-03-30T02:38:34.832 INFO:tasks.workunit.client.0.smithi067.stderr: 56300K .......... .......... .......... .......... .......... 41% 15.3M 4s 2024-03-30T02:38:34.832 INFO:tasks.workunit.client.0.smithi067.stderr: 56350K .......... .......... .......... .......... .......... 41% 102M 4s 2024-03-30T02:38:34.832 INFO:tasks.workunit.client.0.smithi067.stderr: 56400K .......... .......... .......... .......... .......... 41% 116M 4s 2024-03-30T02:38:34.832 INFO:tasks.workunit.client.0.smithi067.stderr: 56450K .......... .......... .......... .......... .......... 41% 92.3M 4s 2024-03-30T02:38:34.832 INFO:tasks.workunit.client.0.smithi067.stderr: 56500K .......... .......... .......... .......... .......... 41% 17.6M 4s 2024-03-30T02:38:34.832 INFO:tasks.workunit.client.0.smithi067.stderr: 56550K .......... .......... .......... .......... .......... 41% 105M 4s 2024-03-30T02:38:34.837 INFO:tasks.workunit.client.0.smithi067.stderr: 56600K .......... .......... .......... .......... .......... 41% 12.0M 4s 2024-03-30T02:38:34.837 INFO:tasks.workunit.client.0.smithi067.stderr: 56650K .......... .......... .......... .......... .......... 41% 93.6M 4s 2024-03-30T02:38:34.840 INFO:tasks.workunit.client.0.smithi067.stderr: 56700K .......... .......... .......... .......... .......... 41% 14.8M 4s 2024-03-30T02:38:34.843 INFO:tasks.workunit.client.0.smithi067.stderr: 56750K .......... .......... .......... .......... .......... 41% 16.0M 4s 2024-03-30T02:38:34.843 INFO:tasks.workunit.client.0.smithi067.stderr: 56800K .......... .......... .......... .......... .......... 41% 116M 4s 2024-03-30T02:38:34.849 INFO:tasks.workunit.client.0.smithi067.stderr: 56850K .......... .......... .......... .......... .......... 41% 11.2M 4s 2024-03-30T02:38:34.849 INFO:tasks.workunit.client.0.smithi067.stderr: 56900K .......... .......... .......... .......... .......... 41% 105M 4s 2024-03-30T02:38:34.853 INFO:tasks.workunit.client.0.smithi067.stderr: 56950K .......... .......... .......... .......... .......... 41% 14.9M 4s 2024-03-30T02:38:34.853 INFO:tasks.workunit.client.0.smithi067.stderr: 57000K .......... .......... .......... .......... .......... 42% 121M 4s 2024-03-30T02:38:34.853 INFO:tasks.workunit.client.0.smithi067.stderr: 57050K .......... .......... .......... .......... .......... 42% 108M 4s 2024-03-30T02:38:34.873 INFO:tasks.workunit.client.0.smithi067.stderr: 57100K .......... .......... .......... .......... .......... 42% 8.85M 4s 2024-03-30T02:38:34.873 INFO:tasks.workunit.client.0.smithi067.stderr: 57150K .......... .......... .......... .......... .......... 42% 116M 4s 2024-03-30T02:38:34.873 INFO:tasks.workunit.client.0.smithi067.stderr: 57200K .......... .......... .......... .......... .......... 42% 29.4M 4s 2024-03-30T02:38:34.873 INFO:tasks.workunit.client.0.smithi067.stderr: 57250K .......... .......... .......... .......... .......... 42% 116M 4s 2024-03-30T02:38:34.873 INFO:tasks.workunit.client.0.smithi067.stderr: 57300K .......... .......... .......... .......... .......... 42% 78.1M 3s 2024-03-30T02:38:34.873 INFO:tasks.workunit.client.0.smithi067.stderr: 57350K .......... .......... .......... .......... .......... 42% 20.3M 3s 2024-03-30T02:38:34.873 INFO:tasks.workunit.client.0.smithi067.stderr: 57400K .......... .......... .......... .......... .......... 42% 112M 3s 2024-03-30T02:38:34.873 INFO:tasks.workunit.client.0.smithi067.stderr: 57450K .......... .......... .......... .......... .......... 42% 14.5M 3s 2024-03-30T02:38:34.874 INFO:tasks.workunit.client.0.smithi067.stderr: 57500K .......... .......... .......... .......... .......... 42% 131M 3s 2024-03-30T02:38:34.874 INFO:tasks.workunit.client.0.smithi067.stderr: 57550K .......... .......... .......... .......... .......... 42% 109M 3s 2024-03-30T02:38:34.874 INFO:tasks.workunit.client.0.smithi067.stderr: 57600K .......... .......... .......... .......... .......... 42% 111M 3s 2024-03-30T02:38:34.874 INFO:tasks.workunit.client.0.smithi067.stderr: 57650K .......... .......... .......... .......... .......... 42% 16.9M 3s 2024-03-30T02:38:34.874 INFO:tasks.workunit.client.0.smithi067.stderr: 57700K .......... .......... .......... .......... .......... 42% 100M 3s 2024-03-30T02:38:34.878 INFO:tasks.workunit.client.0.smithi067.stderr: 57750K .......... .......... .......... .......... .......... 42% 10.7M 3s 2024-03-30T02:38:34.878 INFO:tasks.workunit.client.0.smithi067.stderr: 57800K .......... .......... .......... .......... .......... 42% 115M 3s 2024-03-30T02:38:34.884 INFO:tasks.workunit.client.0.smithi067.stderr: 57850K .......... .......... .......... .......... .......... 42% 15.8M 3s 2024-03-30T02:38:34.884 INFO:tasks.workunit.client.0.smithi067.stderr: 57900K .......... .......... .......... .......... .......... 42% 112M 3s 2024-03-30T02:38:34.884 INFO:tasks.workunit.client.0.smithi067.stderr: 57950K .......... .......... .......... .......... .......... 42% 117M 3s 2024-03-30T02:38:34.884 INFO:tasks.workunit.client.0.smithi067.stderr: 58000K .......... .......... .......... .......... .......... 42% 23.6M 3s 2024-03-30T02:38:34.887 INFO:tasks.workunit.client.0.smithi067.stderr: 58050K .......... .......... .......... .......... .......... 42% 11.0M 3s 2024-03-30T02:38:34.893 INFO:tasks.workunit.client.0.smithi067.stderr: 58100K .......... .......... .......... .......... .......... 42% 10.4M 3s 2024-03-30T02:38:34.893 INFO:tasks.workunit.client.0.smithi067.stderr: 58150K .......... .......... .......... .......... .......... 42% 122M 3s 2024-03-30T02:38:34.894 INFO:tasks.workunit.client.0.smithi067.stderr: 58200K .......... .......... .......... .......... .......... 42% 110M 3s 2024-03-30T02:38:34.897 INFO:tasks.workunit.client.0.smithi067.stderr: 58250K .......... .......... .......... .......... .......... 42% 15.9M 3s 2024-03-30T02:38:34.897 INFO:tasks.workunit.client.0.smithi067.stderr: 58300K .......... .......... .......... .......... .......... 42% 112M 3s 2024-03-30T02:38:34.902 INFO:tasks.workunit.client.0.smithi067.stderr: 58350K .......... .......... .......... .......... .......... 43% 13.6M 3s 2024-03-30T02:38:34.902 INFO:tasks.workunit.client.0.smithi067.stderr: 58400K .......... .......... .......... .......... .......... 43% 117M 3s 2024-03-30T02:38:34.902 INFO:tasks.workunit.client.0.smithi067.stderr: 58450K .......... .......... .......... .......... .......... 43% 106M 3s 2024-03-30T02:38:34.902 INFO:tasks.workunit.client.0.smithi067.stderr: 58500K .......... .......... .......... .......... .......... 43% 21.5M 3s 2024-03-30T02:38:34.907 INFO:tasks.workunit.client.0.smithi067.stderr: 58550K .......... .......... .......... .......... .......... 43% 11.8M 3s 2024-03-30T02:38:34.912 INFO:tasks.workunit.client.0.smithi067.stderr: 58600K .......... .......... .......... .......... .......... 43% 80.9M 3s 2024-03-30T02:38:34.913 INFO:tasks.workunit.client.0.smithi067.stderr: 58650K .......... .......... .......... .......... .......... 43% 10.5M 3s 2024-03-30T02:38:34.913 INFO:tasks.workunit.client.0.smithi067.stderr: 58700K .......... .......... .......... .......... .......... 43% 111M 3s 2024-03-30T02:38:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.913+0000 7f38b8a00700 1 -- 172.21.15.67:0/2888249983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b4106c10 msgr2=0x7f38b4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.913+0000 7f38b8a00700 1 --2- 172.21.15.67:0/2888249983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b4106c10 0x7f38b4106ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f38a40098f0 tx=0x7f38a4009c00 comp rx=0 tx=0).stop 2024-03-30T02:38:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.913+0000 7f38b8a00700 1 -- 172.21.15.67:0/2888249983 shutdown_connections 2024-03-30T02:38:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.913+0000 7f38b8a00700 1 --2- 172.21.15.67:0/2888249983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b4101510 0x7f38b410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.913+0000 7f38b8a00700 1 --2- 172.21.15.67:0/2888249983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b4106c10 0x7f38b4106ff0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.913+0000 7f38b8a00700 1 --2- 172.21.15.67:0/2888249983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b4100b50 0x7f38b4100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.913+0000 7f38b8a00700 1 -- 172.21.15.67:0/2888249983 >> 172.21.15.67:0/2888249983 conn(0x7f38b40fc620 msgr2=0x7f38b40fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.913+0000 7f38b8a00700 1 -- 172.21.15.67:0/2888249983 shutdown_connections 2024-03-30T02:38:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.913+0000 7f38b8a00700 1 -- 172.21.15.67:0/2888249983 wait complete. 2024-03-30T02:38:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.914+0000 7f38b8a00700 1 Processor -- start 2024-03-30T02:38:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.914+0000 7f38b8a00700 1 -- start start 2024-03-30T02:38:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b4100b50 0x7f38b4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b4101510 0x7f38b41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.916 INFO:tasks.workunit.client.0.smithi067.stderr: 58750K .......... .......... .......... .......... .......... 43% 15.5M 3s 2024-03-30T02:38:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b8a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b4106c10 0x7f38b419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b8a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b419beb0 con 0x7f38b4100b50 2024-03-30T02:38:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b8a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b419c020 con 0x7f38b4101510 2024-03-30T02:38:34.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b8a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b419c1e0 con 0x7f38b4106c10 2024-03-30T02:38:34.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b4100b50 0x7f38b4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b4100b50 0x7f38b4196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50592/0 (socket says 172.21.15.67:50592) 2024-03-30T02:38:34.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b2800700 1 -- 172.21.15.67:0/4165932288 learned_addr learned my addr 172.21.15.67:0/4165932288 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b1e00700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b4101510 0x7f38b41974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.915+0000 7f38b3200700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b4106c10 0x7f38b419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.920 INFO:tasks.workunit.client.0.smithi067.stderr: 58800K .......... .......... .......... .......... .......... 43% 106M 3s 2024-03-30T02:38:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38b2800700 1 -- 172.21.15.67:0/4165932288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b4106c10 msgr2=0x7f38b419b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38b2800700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b4106c10 0x7f38b419b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38b2800700 1 -- 172.21.15.67:0/4165932288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b4101510 msgr2=0x7f38b41974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38b2800700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b4101510 0x7f38b41974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38b2800700 1 -- 172.21.15.67:0/4165932288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00095f0 con 0x7f38b4100b50 2024-03-30T02:38:34.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38b3200700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b4106c10 0x7f38b419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:34.921 INFO:tasks.workunit.client.0.smithi067.stderr: 58850K .......... .......... .......... .......... .......... 43% 106M 3s 2024-03-30T02:38:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38b2800700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b4100b50 0x7f38b4196f90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f38a400fa00 tx=0x7f38a400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38ab400700 1 -- 172.21.15.67:0/4165932288 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a401c070 con 0x7f38b4100b50 2024-03-30T02:38:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38b8a00700 1 -- 172.21.15.67:0/4165932288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ac009c40 con 0x7f38b4100b50 2024-03-30T02:38:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38ab400700 1 -- 172.21.15.67:0/4165932288 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a400bba0 con 0x7f38b4100b50 2024-03-30T02:38:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38b8a00700 1 -- 172.21.15.67:0/4165932288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a4009520 con 0x7f38b4100b50 2024-03-30T02:38:34.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.916+0000 7f38ab400700 1 -- 172.21.15.67:0/4165932288 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40048d0 con 0x7f38b4100b50 2024-03-30T02:38:34.924 INFO:tasks.workunit.client.0.smithi067.stderr: 58900K .......... .......... .......... .......... .......... 43% 13.6M 3s 2024-03-30T02:38:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.920+0000 7f38ab400700 1 -- 172.21.15.67:0/4165932288 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 27) v1 ==== 136098+0+0 (secure 0 0 0) 0x7f38a4020070 con 0x7f38b4100b50 2024-03-30T02:38:34.927 INFO:tasks.workunit.client.0.smithi067.stderr: 58950K .......... .......... .......... .......... .......... 43% 79.9M 3s 2024-03-30T02:38:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.921+0000 7f38ab400700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f38980a13b0 0x7f38980a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.921+0000 7f38ab400700 1 -- 172.21.15.67:0/4165932288 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f38a40cde20 con 0x7f38b4100b50 2024-03-30T02:38:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.921+0000 7f38b1e00700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f38980a13b0 0x7f38980a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.921+0000 7f38ccf8fb80 1 Processor -- start 2024-03-30T02:38:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.922+0000 7f38ccf8fb80 1 -- start start 2024-03-30T02:38:34.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.922+0000 7f38b1e00700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f38980a13b0 0x7f38980a3870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38b41984e0 tx=0x7f38a0009310 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.931 INFO:tasks.workunit.client.0.smithi067.stderr: 59000K .......... .......... .......... .......... .......... 43% 24.9M 3s 2024-03-30T02:38:34.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.922+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e0efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.922+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e19e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.922+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 0x55c438e1c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.922+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438d63990 con 0x55c438e1a3c0 2024-03-30T02:38:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.922+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438d61350 con 0x55c438e0f520 2024-03-30T02:38:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.922+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438d5cc40 con 0x55c438e0ec00 2024-03-30T02:38:34.934 INFO:tasks.workunit.client.0.smithi067.stderr: 59050K .......... .......... .......... .......... .......... 43% 107M 3s 2024-03-30T02:38:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b2800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e0efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b3200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 0x55c438e1c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b2800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e0efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60946/0 (socket says 172.21.15.67:60946) 2024-03-30T02:38:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b2800700 1 -- 172.21.15.67:0/78073225 learned_addr learned my addr 172.21.15.67:0/78073225 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b1e00700 1 --2- 172.21.15.67:0/78073225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e19e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.936 INFO:tasks.workunit.client.0.smithi067.stderr: 59100K .......... .......... .......... .......... .......... 43% 114M 3s 2024-03-30T02:38:34.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b2800700 1 -- 172.21.15.67:0/78073225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 msgr2=0x55c438e19e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b2800700 1 --2- 172.21.15.67:0/78073225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e19e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b2800700 1 -- 172.21.15.67:0/78073225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 msgr2=0x55c438e1c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b2800700 1 --2- 172.21.15.67:0/78073225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 0x55c438e1c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b2800700 1 -- 172.21.15.67:0/78073225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c438bde3b0 con 0x55c438e0ec00 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b3200700 1 --2- 172.21.15.67:0/78073225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 0x55c438e1c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.923+0000 7f38b2800700 1 --2- 172.21.15.67:0/78073225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e0efe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f38a40098c0 tx=0x7f38a400faa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=424ab05761d85fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.924+0000 7f389f400700 1 -- 172.21.15.67:0/78073225 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a409a9c0 con 0x55c438e0ec00 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.924+0000 7f389f400700 1 -- 172.21.15.67:0/78073225 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a409e7f0 con 0x55c438e0ec00 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.924+0000 7f389f400700 1 -- 172.21.15.67:0/78073225 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4099eb0 con 0x55c438e0ec00 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.924+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/78073225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 msgr2=0x55c438e0efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.924+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/78073225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e0efe0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f38a40098c0 tx=0x7f38a400faa0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.925+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/78073225 shutdown_connections 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.925+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/78073225 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 0x55c438e1c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.925+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/78073225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e0efe0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.925+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/78073225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e19e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.925+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/78073225 >> 172.21.15.67:0/78073225 conn(0x55c438d6a6a0 msgr2=0x55c438e06c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.925+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/78073225 shutdown_connections 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.925+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/78073225 wait complete. 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.925+0000 7f38ccf8fb80 1 -- start start 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.925+0000 7f38ccf8fb80 1 Processor -- start 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e19df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 0x55c438e17920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438cf8d80 con 0x55c438e1a3c0 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438cf8a20 con 0x55c438e0f520 2024-03-30T02:38:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438be06a0 con 0x55c438e0ec00 2024-03-30T02:38:34.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38ccf8fb80 10 client.0 _ll_register_callbacks cb 0x55c438d02470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:34.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38b1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38b3200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 0x55c438e17920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38b1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e171e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38446/0 (socket says 172.21.15.67:38446) 2024-03-30T02:38:34.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38b1e00700 1 -- 172.21.15.67:0/1664035917 learned_addr learned my addr 172.21.15.67:0/1664035917 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.926+0000 7f38b2800700 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e19df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.942 INFO:tasks.workunit.client.0.smithi067.stderr: 59150K .......... .......... .......... .......... .......... 43% 13.5M 3s 2024-03-30T02:38:34.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f38b1e00700 1 -- 172.21.15.67:0/1664035917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 msgr2=0x55c438e19df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f38b1e00700 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e19df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f38b1e00700 1 -- 172.21.15.67:0/1664035917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 msgr2=0x55c438e17920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f38b1e00700 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 0x55c438e17920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f38b1e00700 1 -- 172.21.15.67:0/1664035917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ac0091e0 con 0x55c438e0f520 2024-03-30T02:38:34.945 INFO:tasks.workunit.client.0.smithi067.stderr: 59200K .......... .......... .......... .......... .......... 43% 120M 3s 2024-03-30T02:38:34.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f38b1e00700 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e171e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f38a0005d10 tx=0x7f38a00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b68e564f9f39c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f3893e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:38:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f3893e00700 1 -- 172.21.15.67:0/1664035917 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0022070 con 0x55c438e0f520 2024-03-30T02:38:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/1664035917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c438d1d380 con 0x55c438e0f520 2024-03-30T02:38:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.927+0000 7f3893e00700 1 -- 172.21.15.67:0/1664035917 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f38a001d750 con 0x55c438e0f520 2024-03-30T02:38:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.928+0000 7f3893e00700 1 -- 172.21.15.67:0/1664035917 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a001d910 con 0x55c438e0f520 2024-03-30T02:38:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.928+0000 7f38ccf8fb80 10 client.26144 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.928+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/1664035917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f38a4009520 con 0x55c438e0f520 2024-03-30T02:38:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.928+0000 7f3893e00700 1 -- 172.21.15.67:0/1664035917 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c438d1d380 con 0x55c438e0f520 2024-03-30T02:38:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.928+0000 7f3893e00700 1 -- 172.21.15.67:0/1664035917 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f38a0026c70 con 0x55c438e0f520 2024-03-30T02:38:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.928+0000 7f38ccf8fb80 10 client.26144 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.928+0000 7f38ccf8fb80 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.928+0000 7f38ccf8fb80 20 client.26144 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:34.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.928+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c438e29d00 0x55c438e2c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.929+0000 7f38ccf8fb80 4 client.26144 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:38:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.929+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/1664035917 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c438d55700 con 0x55c438e29d00 2024-03-30T02:38:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.929+0000 7f38b2800700 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c438e29d00 0x55c438e2c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.929+0000 7f38b2800700 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c438e29d00 0x55c438e2c120 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e154f6df1ce732bb server_cookie=6fad07db530b52bf in_seq=0 out_seq=0 2024-03-30T02:38:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.929+0000 7f3893e00700 10 client.26144 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f3893e00700 1 -- 172.21.15.67:0/1664035917 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c438d55700 con 0x55c438e29d00 2024-03-30T02:38:34.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f3893e00700 10 client.26144 handle_command_reply: tid=0 2024-03-30T02:38:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 client.26144 shutdown 2024-03-30T02:38:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/1664035917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 msgr2=0x55c438e171e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e171e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f38a0005d10 tx=0x7f38a00047c0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/1664035917 shutdown_connections 2024-03-30T02:38:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c438e29d00 0x55c438e2c120 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e1a3c0 0x55c438e17920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0ec00 0x55c438e19df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/1664035917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e171e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f3893e00700 0 client.26144 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/1664035917 >> 172.21.15.67:0/1664035917 conn(0x55c438d6a6a0 msgr2=0x55c438e10cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.956 INFO:tasks.workunit.client.0.smithi067.stderr: 59250K .......... .......... .......... .......... .......... 43% 13.5M 3s 2024-03-30T02:38:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.930+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/1664035917 shutdown_connections 2024-03-30T02:38:34.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.931+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/1664035917 wait complete. 2024-03-30T02:38:34.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.931+0000 7f38ccf8fb80 20 client.26144 trim_cache size 0 max 16384 2024-03-30T02:38:34.970 INFO:tasks.workunit.client.0.smithi067.stderr: 59300K .......... .......... .......... .......... .......... 43% 106M 3s 2024-03-30T02:38:34.970 INFO:tasks.workunit.client.0.smithi067.stderr: 59350K .......... .......... .......... .......... .......... 43% 104M 3s 2024-03-30T02:38:34.970 INFO:tasks.workunit.client.0.smithi067.stderr: 59400K .......... .......... .......... .......... .......... 43% 6.69M 3s 2024-03-30T02:38:34.970 INFO:tasks.workunit.client.0.smithi067.stderr: 59450K .......... .......... .......... .......... .......... 43% 90.6M 3s 2024-03-30T02:38:34.971 INFO:tasks.workunit.client.0.smithi067.stderr: 59500K .......... .......... .......... .......... .......... 43% 10.2M 3s 2024-03-30T02:38:34.971 INFO:tasks.workunit.client.0.smithi067.stderr: 59550K .......... .......... .......... .......... .......... 43% 114M 3s 2024-03-30T02:38:34.971 INFO:tasks.workunit.client.0.smithi067.stderr: 59600K .......... .......... .......... .......... .......... 43% 112M 3s 2024-03-30T02:38:34.971 INFO:tasks.workunit.client.0.smithi067.stderr: 59650K .......... .......... .......... .......... .......... 43% 22.4M 3s 2024-03-30T02:38:34.971 INFO:tasks.workunit.client.0.smithi067.stderr: 59700K .......... .......... .......... .......... .......... 44% 113M 3s 2024-03-30T02:38:34.971 INFO:tasks.workunit.client.0.smithi067.stderr: 59750K .......... .......... .......... .......... .......... 44% 101M 3s 2024-03-30T02:38:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.949+0000 7f38ccf8fb80 1 Processor -- start 2024-03-30T02:38:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.949+0000 7f38ccf8fb80 1 -- start start 2024-03-30T02:38:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e06160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0f520 0x55c438e178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e1a3c0 0x55c438e20db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438e2c4b0 con 0x55c438e0ec00 2024-03-30T02:38:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438e0ea20 con 0x55c438e1a3c0 2024-03-30T02:38:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438e212f0 con 0x55c438e0f520 2024-03-30T02:38:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e06160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e06160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50620/0 (socket says 172.21.15.67:50620) 2024-03-30T02:38:34.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b2800700 1 -- 172.21.15.67:0/3501296121 learned_addr learned my addr 172.21.15.67:0/3501296121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b3200700 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e1a3c0 0x55c438e20db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b1e00700 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0f520 0x55c438e178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b2800700 1 -- 172.21.15.67:0/3501296121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0f520 msgr2=0x55c438e178a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b2800700 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0f520 0x55c438e178a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b2800700 1 -- 172.21.15.67:0/3501296121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e1a3c0 msgr2=0x55c438e20db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b2800700 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e1a3c0 0x55c438e20db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b2800700 1 -- 172.21.15.67:0/3501296121 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c438b8ab60 con 0x55c438e0ec00 2024-03-30T02:38:34.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.950+0000 7f38b1e00700 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0f520 0x55c438e178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:34.976 INFO:tasks.workunit.client.0.smithi067.stderr: 59800K .......... .......... .......... .......... .......... 44% 19.9M 3s 2024-03-30T02:38:34.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.951+0000 7f38b2800700 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e06160 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38a40caf80 tx=0x7f38a4005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d511ea3fc222db9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.977 INFO:tasks.workunit.client.0.smithi067.stderr: 59850K .......... .......... .......... .......... .......... 44% 98.6M 3s 2024-03-30T02:38:34.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.951+0000 7f389fe00700 1 -- 172.21.15.67:0/3501296121 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40a2db0 con 0x55c438e0ec00 2024-03-30T02:38:34.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.951+0000 7f389fe00700 1 -- 172.21.15.67:0/3501296121 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a40b2070 con 0x55c438e0ec00 2024-03-30T02:38:34.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.951+0000 7f389fe00700 1 -- 172.21.15.67:0/3501296121 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4099eb0 con 0x55c438e0ec00 2024-03-30T02:38:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.951+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/3501296121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 msgr2=0x55c438e06160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.951+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e06160 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38a40caf80 tx=0x7f38a4005960 comp rx=0 tx=0).stop 2024-03-30T02:38:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.952+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/3501296121 shutdown_connections 2024-03-30T02:38:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.952+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e06160 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.952+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e1a3c0 0x55c438e20db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.952+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/3501296121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e0f520 0x55c438e178a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.952+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/3501296121 >> 172.21.15.67:0/3501296121 conn(0x55c438d6a6a0 msgr2=0x55c438dff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.952+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/3501296121 shutdown_connections 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.952+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/3501296121 wait complete. 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.952+0000 7f38ccf8fb80 1 -- start start 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.952+0000 7f38ccf8fb80 1 Processor -- start 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e20c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e1f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38ccf8fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e1a3c0 0x55c438e1f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438e1fd90 con 0x55c438e0ec00 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438e1ff00 con 0x55c438e0f520 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38ccf8fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c438e7db40 con 0x55c438e1a3c0 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38ccf8fb80 10 client.0 _ll_register_callbacks cb 0x55c438d02470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38b2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38b2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e20c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50632/0 (socket says 172.21.15.67:50632) 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.953+0000 7f38b2800700 1 -- 172.21.15.67:0/2881083557 learned_addr learned my addr 172.21.15.67:0/2881083557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38b1e00700 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e1f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38b3200700 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e1a3c0 0x55c438e1f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38b2800700 1 -- 172.21.15.67:0/2881083557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e1a3c0 msgr2=0x55c438e1f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38b2800700 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e1a3c0 0x55c438e1f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38b2800700 1 -- 172.21.15.67:0/2881083557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 msgr2=0x55c438e1f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:38:34.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38b2800700 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e1f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38b2800700 1 -- 172.21.15.67:0/2881083557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c438bde3b0 con 0x55c438e0ec00 2024-03-30T02:38:34.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38b1e00700 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e1f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:38:34.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38b2800700 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e20c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f38a400fae0 tx=0x7f38a4096e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcc0bb0baf766d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:38:34.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f3893e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:38:34.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f3893e00700 1 -- 172.21.15.67:0/2881083557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4005340 con 0x55c438e0ec00 2024-03-30T02:38:34.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f3893e00700 1 -- 172.21.15.67:0/2881083557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f38a409ac20 con 0x55c438e0ec00 2024-03-30T02:38:34.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.954+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/2881083557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38a409e7f0 con 0x55c438e0ec00 2024-03-30T02:38:34.984 INFO:tasks.workunit.client.0.smithi067.stderr: 59900K .......... .......... .......... .......... .......... 44% 14.4M 3s 2024-03-30T02:38:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f3893e00700 1 -- 172.21.15.67:0/2881083557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a409ade0 con 0x55c438e0ec00 2024-03-30T02:38:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f38ccf8fb80 10 client.16098 fetch_fsmap learned FSMap version 29 2024-03-30T02:38:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/2881083557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c438d22ea0 con 0x55c438e0ec00 2024-03-30T02:38:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f3893e00700 1 -- 172.21.15.67:0/2881083557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a409e7f0 con 0x55c438e0ec00 2024-03-30T02:38:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f3893e00700 1 -- 172.21.15.67:0/2881083557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f38a40cb320 con 0x55c438e0ec00 2024-03-30T02:38:34.987 INFO:tasks.workunit.client.0.smithi067.stderr: 59950K .......... .......... .......... .......... .......... 44% 72.6M 3s 2024-03-30T02:38:34.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f38ccf8fb80 10 client.16098 fetch_fsmap finished waiting for FSMap version 29 2024-03-30T02:38:34.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f38ccf8fb80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:38:34.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f38ccf8fb80 20 client.16098 populate_metadata read hostname 'smithi067' 2024-03-30T02:38:34.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c438e7edc0 0x55c438e811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:38:34.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f38ccf8fb80 4 client.16098 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:38:34.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/2881083557 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c438babcf0 con 0x55c438e7edc0 2024-03-30T02:38:34.989 INFO:tasks.workunit.client.0.smithi067.stderr: 60000K .......... .......... .......... .......... .......... 44% 147M 3s 2024-03-30T02:38:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.955+0000 7f38b1e00700 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c438e7edc0 0x55c438e811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:38:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38b1e00700 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c438e7edc0 0x55c438e811b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba164b9733232c5d server_cookie=5f08d9417fbcb983 in_seq=0 out_seq=0 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f3893e00700 10 client.16098 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f3893e00700 1 -- 172.21.15.67:0/2881083557 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c438babcf0 con 0x55c438e7edc0 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f3893e00700 10 client.16098 handle_command_reply: tid=0 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38ccf8fb80 1 client.16098 shutdown 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/2881083557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 msgr2=0x55c438e20c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e20c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f38a400fae0 tx=0x7f38a4096e30 comp rx=0 tx=0).stop 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/2881083557 shutdown_connections 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c438e7edc0 0x55c438e811b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c438e0ec00 0x55c438e20c00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c438e1a3c0 0x55c438e1f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f3893e00700 0 client.16098 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:38:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38ccf8fb80 1 --2- 172.21.15.67:0/2881083557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c438e0f520 0x55c438e1f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.956+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/2881083557 >> 172.21.15.67:0/2881083557 conn(0x55c438d6a6a0 msgr2=0x55c438e00010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/2881083557 shutdown_connections 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38ccf8fb80 1 -- 172.21.15.67:0/2881083557 wait complete. 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38ccf8fb80 20 client.16098 trim_cache size 0 max 16384 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stdout: "747c466b-0894-422a-862a-a9130ddd9209": { 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:38:34.992 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "747c466b-0894-422a-862a-a9130ddd9209", 2024-03-30T02:38:34.993 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:38:34.993 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:38:34.993 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:38:34.993 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:38:34.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 -- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f38980a13b0 msgr2=0x7f38980a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f38980a13b0 0x7f38980a3870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38b41984e0 tx=0x7f38a0009310 comp rx=0 tx=0).stop 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 -- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b4100b50 msgr2=0x7f38b4196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b4100b50 0x7f38b4196f90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f38a400fa00 tx=0x7f38a400fa30 comp rx=0 tx=0).stop 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 -- 172.21.15.67:0/4165932288 shutdown_connections 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f38980a13b0 0x7f38980a3870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b4100b50 0x7f38b4196f90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b4106c10 0x7f38b419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 --2- 172.21.15.67:0/4165932288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b4101510 0x7f38b41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.957+0000 7f38b8a00700 1 -- 172.21.15.67:0/4165932288 >> 172.21.15.67:0/4165932288 conn(0x7f38b40fc620 msgr2=0x7f38b40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.958+0000 7f38b8a00700 1 -- 172.21.15.67:0/4165932288 shutdown_connections 2024-03-30T02:38:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:38:34.958+0000 7f38b8a00700 1 -- 172.21.15.67:0/4165932288 wait complete. 2024-03-30T02:38:35.006 INFO:tasks.workunit.client.0.smithi067.stderr: 60050K .......... .......... .......... .......... .......... 44% 14.4M 3s 2024-03-30T02:38:35.007 INFO:tasks.workunit.client.0.smithi067.stderr: 60100K .......... .......... .......... .......... .......... 44% 116M 3s 2024-03-30T02:38:35.007 INFO:tasks.workunit.client.0.smithi067.stderr: 60150K .......... .......... .......... .......... .......... 44% 15.2M 3s 2024-03-30T02:38:35.007 INFO:tasks.workunit.client.0.smithi067.stderr: 60200K .......... .......... .......... .......... .......... 44% 103M 3s 2024-03-30T02:38:35.007 INFO:tasks.workunit.client.0.smithi067.stderr: 60250K .......... .......... .......... .......... .......... 44% 90.2M 3s 2024-03-30T02:38:35.007 INFO:tasks.workunit.client.0.smithi067.stderr: 60300K .......... .......... .......... .......... .......... 44% 14.4M 3s 2024-03-30T02:38:35.007 INFO:tasks.workunit.client.0.smithi067.stderr: 60350K .......... .......... .......... .......... .......... 44% 104M 3s 2024-03-30T02:38:35.007 INFO:tasks.workunit.client.0.smithi067.stderr: 60400K .......... .......... .......... .......... .......... 44% 7.16M 3s 2024-03-30T02:38:35.007 INFO:tasks.workunit.client.0.smithi067.stderr: 60450K .......... .......... .......... .......... .......... 44% 130M 3s 2024-03-30T02:38:35.007 INFO:tasks.workunit.client.0.smithi067.stderr: 60500K .......... .......... .......... .......... .......... 44% 91.7M 3s 2024-03-30T02:38:35.008 INFO:tasks.cephfs.filesystem:scrub status for tag:747c466b-0894-422a-862a-a9130ddd9209 - {'path': '/', 'tag': '747c466b-0894-422a-862a-a9130ddd9209', 'options': 'recursive,force'} 2024-03-30T02:38:35.089 INFO:tasks.workunit.client.0.smithi067.stderr: 60550K .......... .......... .......... .......... .......... 44% 14.0M 3s 2024-03-30T02:38:35.089 INFO:tasks.workunit.client.0.smithi067.stderr: 60600K .......... .......... .......... .......... .......... 44% 101M 3s 2024-03-30T02:38:35.089 INFO:tasks.workunit.client.0.smithi067.stderr: 60650K .......... .......... .......... .......... .......... 44% 14.4M 3s 2024-03-30T02:38:35.089 INFO:tasks.workunit.client.0.smithi067.stderr: 60700K .......... .......... .......... .......... .......... 44% 101M 3s 2024-03-30T02:38:35.089 INFO:tasks.workunit.client.0.smithi067.stderr: 60750K .......... .......... .......... .......... .......... 44% 143M 3s 2024-03-30T02:38:35.089 INFO:tasks.workunit.client.0.smithi067.stderr: 60800K .......... .......... .......... .......... .......... 44% 13.4M 3s 2024-03-30T02:38:35.089 INFO:tasks.workunit.client.0.smithi067.stderr: 60850K .......... .......... .......... .......... .......... 44% 137M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 60900K .......... .......... .......... .......... .......... 44% 93.6M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 60950K .......... .......... .......... .......... .......... 44% 15.6M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61000K .......... .......... .......... .......... .......... 44% 12.8M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61050K .......... .......... .......... .......... .......... 45% 112M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61100K .......... .......... .......... .......... .......... 45% 19.3M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61150K .......... .......... .......... .......... .......... 45% 87.7M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61200K .......... .......... .......... .......... .......... 45% 11.3M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61250K .......... .......... .......... .......... .......... 45% 106M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61300K .......... .......... .......... .......... .......... 45% 6.43M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61350K .......... .......... .......... .......... .......... 45% 90.7M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61400K .......... .......... .......... .......... .......... 45% 92.7M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61450K .......... .......... .......... .......... .......... 45% 15.7M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61500K .......... .......... .......... .......... .......... 45% 119M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61550K .......... .......... .......... .......... .......... 45% 12.4M 3s 2024-03-30T02:38:35.090 INFO:tasks.workunit.client.0.smithi067.stderr: 61600K .......... .......... .......... .......... .......... 45% 113M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 61650K .......... .......... .......... .......... .......... 45% 144M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 61700K .......... .......... .......... .......... .......... 45% 13.7M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 61750K .......... .......... .......... .......... .......... 45% 157M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 61800K .......... .......... .......... .......... .......... 45% 108M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 61850K .......... .......... .......... .......... .......... 45% 18.3M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 61900K .......... .......... .......... .......... .......... 45% 125M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 61950K .......... .......... .......... .......... .......... 45% 15.0M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 62000K .......... .......... .......... .......... .......... 45% 180M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 62050K .......... .......... .......... .......... .......... 45% 107M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 62100K .......... .......... .......... .......... .......... 45% 13.7M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 62150K .......... .......... .......... .......... .......... 45% 116M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 62200K .......... .......... .......... .......... .......... 45% 14.0M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 62250K .......... .......... .......... .......... .......... 45% 125M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 62300K .......... .......... .......... .......... .......... 45% 110M 3s 2024-03-30T02:38:35.091 INFO:tasks.workunit.client.0.smithi067.stderr: 62350K .......... .......... .......... .......... .......... 45% 14.4M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62400K .......... .......... .......... .......... .......... 46% 117M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62450K .......... .......... .......... .......... .......... 46% 14.0M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62500K .......... .......... .......... .......... .......... 46% 143M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62550K .......... .......... .......... .......... .......... 46% 130M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62600K .......... .......... .......... .......... .......... 46% 15.3M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62650K .......... .......... .......... .......... .......... 46% 168M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62700K .......... .......... .......... .......... .......... 46% 11.7M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62750K .......... .......... .......... .......... .......... 46% 115M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62800K .......... .......... .......... .......... .......... 46% 123M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62850K .......... .......... .......... .......... .......... 46% 9.16M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62900K .......... .......... .......... .......... .......... 46% 188M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 62950K .......... .......... .......... .......... .......... 46% 88.0M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 63000K .......... .......... .......... .......... .......... 46% 40.7M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 63050K .......... .......... .......... .......... .......... 46% 102M 3s 2024-03-30T02:38:35.092 INFO:tasks.workunit.client.0.smithi067.stderr: 63100K .......... .......... .......... .......... .......... 46% 13.4M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63150K .......... .......... .......... .......... .......... 46% 182M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63200K .......... .......... .......... .......... .......... 46% 111M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63250K .......... .......... .......... .......... .......... 46% 19.5M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63300K .......... .......... .......... .......... .......... 46% 61.3M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63350K .......... .......... .......... .......... .......... 46% 16.1M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63400K .......... .......... .......... .......... .......... 46% 47.5M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63450K .......... .......... .......... .......... .......... 46% 118M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63500K .......... .......... .......... .......... .......... 46% 16.9M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63550K .......... .......... .......... .......... .......... 46% 8.33M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63600K .......... .......... .......... .......... .......... 46% 49.9M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63650K .......... .......... .......... .......... .......... 46% 113M 3s 2024-03-30T02:38:35.093 INFO:tasks.workunit.client.0.smithi067.stderr: 63700K .......... .......... .......... .......... .......... 46% 108M 3s 2024-03-30T02:38:35.112 INFO:tasks.workunit.client.0.smithi067.stderr: 63750K .......... .......... .......... .......... .......... 46% 13.8M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 63800K .......... .......... .......... .......... .......... 47% 102M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 63850K .......... .......... .......... .......... .......... 47% 13.1M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 63900K .......... .......... .......... .......... .......... 47% 128M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 63950K .......... .......... .......... .......... .......... 47% 112M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64000K .......... .......... .......... .......... .......... 47% 15.2M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64050K .......... .......... .......... .......... .......... 47% 113M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64100K .......... .......... .......... .......... .......... 47% 109M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64150K .......... .......... .......... .......... .......... 47% 13.8M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64200K .......... .......... .......... .......... .......... 47% 124M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64250K .......... .......... .......... .......... .......... 47% 13.0M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64300K .......... .......... .......... .......... .......... 47% 105M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64350K .......... .......... .......... .......... .......... 47% 104M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64400K .......... .......... .......... .......... .......... 47% 21.0M 3s 2024-03-30T02:38:35.113 INFO:tasks.workunit.client.0.smithi067.stderr: 64450K .......... .......... .......... .......... .......... 47% 90.2M 3s 2024-03-30T02:38:35.117 INFO:tasks.workunit.client.0.smithi067.stderr: 64500K .......... .......... .......... .......... .......... 47% 13.8M 3s 2024-03-30T02:38:35.118 INFO:tasks.workunit.client.0.smithi067.stderr: 64550K .......... .......... .......... .......... .......... 47% 107M 3s 2024-03-30T02:38:35.118 INFO:tasks.workunit.client.0.smithi067.stderr: 64600K .......... .......... .......... .......... .......... 47% 87.4M 3s 2024-03-30T02:38:35.121 INFO:tasks.workunit.client.0.smithi067.stderr: 64650K .......... .......... .......... .......... .......... 47% 13.9M 3s 2024-03-30T02:38:35.121 INFO:tasks.workunit.client.0.smithi067.stderr: 64700K .......... .......... .......... .......... .......... 47% 77.7M 3s 2024-03-30T02:38:35.125 INFO:tasks.workunit.client.0.smithi067.stderr: 64750K .......... .......... .......... .......... .......... 47% 11.1M 3s 2024-03-30T02:38:35.126 INFO:tasks.workunit.client.0.smithi067.stderr: 64800K .......... .......... .......... .......... .......... 47% 72.6M 3s 2024-03-30T02:38:35.126 INFO:tasks.workunit.client.0.smithi067.stderr: 64850K .......... .......... .......... .......... .......... 47% 68.7M 3s 2024-03-30T02:38:35.129 INFO:tasks.workunit.client.0.smithi067.stderr: 64900K .......... .......... .......... .......... .......... 47% 24.2M 3s 2024-03-30T02:38:35.129 INFO:tasks.workunit.client.0.smithi067.stderr: 64950K .......... .......... .......... .......... .......... 47% 45.3M 3s 2024-03-30T02:38:35.129 INFO:tasks.workunit.client.0.smithi067.stderr: 65000K .......... .......... .......... .......... .......... 47% 99.6M 3s 2024-03-30T02:38:35.134 INFO:tasks.workunit.client.0.smithi067.stderr: 65050K .......... .......... .......... .......... .......... 47% 17.7M 3s 2024-03-30T02:38:35.134 INFO:tasks.workunit.client.0.smithi067.stderr: 65100K .......... .......... .......... .......... .......... 47% 38.9M 3s 2024-03-30T02:38:35.137 INFO:tasks.workunit.client.0.smithi067.stderr: 65150K .......... .......... .......... .......... .......... 48% 10.8M 3s 2024-03-30T02:38:35.140 INFO:tasks.workunit.client.0.smithi067.stderr: 65200K .......... .......... .......... .......... .......... 48% 18.5M 3s 2024-03-30T02:38:35.141 INFO:tasks.workunit.client.0.smithi067.stderr: 65250K .......... .......... .......... .......... .......... 48% 41.3M 3s 2024-03-30T02:38:35.144 INFO:tasks.workunit.client.0.smithi067.stderr: 65300K .......... .......... .......... .......... .......... 48% 28.9M 3s 2024-03-30T02:38:35.145 INFO:tasks.workunit.client.0.smithi067.stderr: 65350K .......... .......... .......... .......... .......... 48% 34.4M 3s 2024-03-30T02:38:35.153 INFO:tasks.workunit.client.0.smithi067.stderr: 65400K .......... .......... .......... .......... .......... 48% 7.06M 3s 2024-03-30T02:38:35.153 INFO:tasks.workunit.client.0.smithi067.stderr: 65450K .......... .......... .......... .......... .......... 48% 76.0M 3s 2024-03-30T02:38:35.153 INFO:tasks.workunit.client.0.smithi067.stderr: 65500K .......... .......... .......... .......... .......... 48% 65.8M 3s 2024-03-30T02:38:35.159 INFO:tasks.workunit.client.0.smithi067.stderr: 65550K .......... .......... .......... .......... .......... 48% 8.88M 3s 2024-03-30T02:38:35.159 INFO:tasks.workunit.client.0.smithi067.stderr: 65600K .......... .......... .......... .......... .......... 48% 101M 3s 2024-03-30T02:38:35.165 INFO:tasks.workunit.client.0.smithi067.stderr: 65650K .......... .......... .......... .......... .......... 48% 13.0M 3s 2024-03-30T02:38:35.165 INFO:tasks.workunit.client.0.smithi067.stderr: 65700K .......... .......... .......... .......... .......... 48% 94.9M 3s 2024-03-30T02:38:35.165 INFO:tasks.workunit.client.0.smithi067.stderr: 65750K .......... .......... .......... .......... .......... 48% 101M 3s 2024-03-30T02:38:35.165 INFO:tasks.workunit.client.0.smithi067.stderr: 65800K .......... .......... .......... .......... .......... 48% 98.0M 3s 2024-03-30T02:38:35.167 INFO:tasks.workunit.client.0.smithi067.stderr: 65850K .......... .......... .......... .......... .......... 48% 16.5M 3s 2024-03-30T02:38:35.172 INFO:tasks.workunit.client.0.smithi067.stderr: 65900K .......... .......... .......... .......... .......... 48% 15.8M 3s 2024-03-30T02:38:35.172 INFO:tasks.workunit.client.0.smithi067.stderr: 65950K .......... .......... .......... .......... .......... 48% 71.5M 3s 2024-03-30T02:38:35.172 INFO:tasks.workunit.client.0.smithi067.stderr: 66000K .......... .......... .......... .......... .......... 48% 76.9M 3s 2024-03-30T02:38:35.176 INFO:tasks.workunit.client.0.smithi067.stderr: 66050K .......... .......... .......... .......... .......... 48% 15.4M 3s 2024-03-30T02:38:35.176 INFO:tasks.workunit.client.0.smithi067.stderr: 66100K .......... .......... .......... .......... .......... 48% 96.4M 3s 2024-03-30T02:38:35.176 INFO:tasks.workunit.client.0.smithi067.stderr: 66150K .......... .......... .......... .......... .......... 48% 105M 3s 2024-03-30T02:38:35.180 INFO:tasks.workunit.client.0.smithi067.stderr: 66200K .......... .......... .......... .......... .......... 48% 15.3M 3s 2024-03-30T02:38:35.180 INFO:tasks.workunit.client.0.smithi067.stderr: 66250K .......... .......... .......... .......... .......... 48% 95.9M 3s 2024-03-30T02:38:35.183 INFO:tasks.workunit.client.0.smithi067.stderr: 66300K .......... .......... .......... .......... .......... 48% 16.6M 3s 2024-03-30T02:38:35.183 INFO:tasks.workunit.client.0.smithi067.stderr: 66350K .......... .......... .......... .......... .......... 48% 93.6M 3s 2024-03-30T02:38:35.184 INFO:tasks.workunit.client.0.smithi067.stderr: 66400K .......... .......... .......... .......... .......... 48% 89.0M 3s 2024-03-30T02:38:35.188 INFO:tasks.workunit.client.0.smithi067.stderr: 66450K .......... .......... .......... .......... .......... 48% 10.5M 3s 2024-03-30T02:38:35.189 INFO:tasks.workunit.client.0.smithi067.stderr: 66500K .......... .......... .......... .......... .......... 49% 102M 3s 2024-03-30T02:38:35.193 INFO:tasks.workunit.client.0.smithi067.stderr: 66550K .......... .......... .......... .......... .......... 49% 11.9M 3s 2024-03-30T02:38:35.193 INFO:tasks.workunit.client.0.smithi067.stderr: 66600K .......... .......... .......... .......... .......... 49% 90.5M 3s 2024-03-30T02:38:35.193 INFO:tasks.workunit.client.0.smithi067.stderr: 66650K .......... .......... .......... .......... .......... 49% 94.0M 3s 2024-03-30T02:38:35.201 INFO:tasks.workunit.client.0.smithi067.stderr: 66700K .......... .......... .......... .......... .......... 49% 12.5M 3s 2024-03-30T02:38:35.202 INFO:tasks.workunit.client.0.smithi067.stderr: 66750K .......... .......... .......... .......... .......... 49% 83.7M 3s 2024-03-30T02:38:35.206 INFO:tasks.workunit.client.0.smithi067.stderr: 66800K .......... .......... .......... .......... .......... 49% 17.5M 3s 2024-03-30T02:38:35.206 INFO:tasks.workunit.client.0.smithi067.stderr: 66850K .......... .......... .......... .......... .......... 49% 88.8M 3s 2024-03-30T02:38:35.206 INFO:tasks.workunit.client.0.smithi067.stderr: 66900K .......... .......... .......... .......... .......... 49% 99.7M 3s 2024-03-30T02:38:35.206 INFO:tasks.workunit.client.0.smithi067.stderr: 66950K .......... .......... .......... .......... .......... 49% 15.8M 3s 2024-03-30T02:38:35.207 INFO:tasks.workunit.client.0.smithi067.stderr: 67000K .......... .......... .......... .......... .......... 49% 101M 3s 2024-03-30T02:38:35.209 INFO:tasks.workunit.client.0.smithi067.stderr: 67050K .......... .......... .......... .......... .......... 49% 22.9M 3s 2024-03-30T02:38:35.209 INFO:tasks.workunit.client.0.smithi067.stderr: 67100K .......... .......... .......... .......... .......... 49% 80.5M 3s 2024-03-30T02:38:35.210 INFO:tasks.workunit.client.0.smithi067.stderr: 67150K .......... .......... .......... .......... .......... 49% 59.2M 3s 2024-03-30T02:38:35.212 INFO:tasks.workunit.client.0.smithi067.stderr: 67200K .......... .......... .......... .......... .......... 49% 15.5M 3s 2024-03-30T02:38:35.215 INFO:tasks.workunit.client.0.smithi067.stderr: 67250K .......... .......... .......... .......... .......... 49% 16.1M 3s 2024-03-30T02:38:35.215 INFO:tasks.workunit.client.0.smithi067.stderr: 67300K .......... .......... .......... .......... .......... 49% 100M 3s 2024-03-30T02:38:35.221 INFO:tasks.workunit.client.0.smithi067.stderr: 67350K .......... .......... .......... .......... .......... 49% 11.1M 3s 2024-03-30T02:38:35.221 INFO:tasks.workunit.client.0.smithi067.stderr: 67400K .......... .......... .......... .......... .......... 49% 100M 3s 2024-03-30T02:38:35.229 INFO:tasks.workunit.client.0.smithi067.stderr: 67450K .......... .......... .......... .......... .......... 49% 6.44M 3s 2024-03-30T02:38:35.229 INFO:tasks.workunit.client.0.smithi067.stderr: 67500K .......... .......... .......... .......... .......... 49% 99.8M 3s 2024-03-30T02:38:35.229 INFO:tasks.workunit.client.0.smithi067.stderr: 67550K .......... .......... .......... .......... .......... 49% 98.5M 3s 2024-03-30T02:38:35.232 INFO:tasks.workunit.client.0.smithi067.stderr: 67600K .......... .......... .......... .......... .......... 49% 30.4M 3s 2024-03-30T02:38:35.232 INFO:tasks.workunit.client.0.smithi067.stderr: 67650K .......... .......... .......... .......... .......... 49% 99.7M 3s 2024-03-30T02:38:35.236 INFO:tasks.workunit.client.0.smithi067.stderr: 67700K .......... .......... .......... .......... .......... 49% 14.0M 3s 2024-03-30T02:38:35.236 INFO:tasks.workunit.client.0.smithi067.stderr: 67750K .......... .......... .......... .......... .......... 49% 83.4M 3s 2024-03-30T02:38:35.236 INFO:tasks.workunit.client.0.smithi067.stderr: 67800K .......... .......... .......... .......... .......... 49% 106M 3s 2024-03-30T02:38:35.241 INFO:tasks.workunit.client.0.smithi067.stderr: 67850K .......... .......... .......... .......... .......... 50% 13.2M 3s 2024-03-30T02:38:35.241 INFO:tasks.workunit.client.0.smithi067.stderr: 67900K .......... .......... .......... .......... .......... 50% 98.5M 3s 2024-03-30T02:38:35.249 INFO:tasks.workunit.client.0.smithi067.stderr: 67950K .......... .......... .......... .......... .......... 50% 5.83M 3s 2024-03-30T02:38:35.249 INFO:tasks.workunit.client.0.smithi067.stderr: 68000K .......... .......... .......... .......... .......... 50% 95.0M 3s 2024-03-30T02:38:35.249 INFO:tasks.workunit.client.0.smithi067.stderr: 68050K .......... .......... .......... .......... .......... 50% 86.7M 3s 2024-03-30T02:38:35.252 INFO:tasks.workunit.client.0.smithi067.stderr: 68100K .......... .......... .......... .......... .......... 50% 23.1M 3s 2024-03-30T02:38:35.252 INFO:tasks.workunit.client.0.smithi067.stderr: 68150K .......... .......... .......... .......... .......... 50% 98.8M 3s 2024-03-30T02:38:35.252 INFO:tasks.workunit.client.0.smithi067.stderr: 68200K .......... .......... .......... .......... .......... 50% 105M 3s 2024-03-30T02:38:35.257 INFO:tasks.workunit.client.0.smithi067.stderr: 68250K .......... .......... .......... .......... .......... 50% 13.3M 3s 2024-03-30T02:38:35.257 INFO:tasks.workunit.client.0.smithi067.stderr: 68300K .......... .......... .......... .......... .......... 50% 84.9M 3s 2024-03-30T02:38:35.261 INFO:tasks.workunit.client.0.smithi067.stderr: 68350K .......... .......... .......... .......... .......... 50% 15.7M 3s 2024-03-30T02:38:35.261 INFO:tasks.workunit.client.0.smithi067.stderr: 68400K .......... .......... .......... .......... .......... 50% 99.8M 3s 2024-03-30T02:38:35.261 INFO:tasks.workunit.client.0.smithi067.stderr: 68450K .......... .......... .......... .......... .......... 50% 88.9M 3s 2024-03-30T02:38:35.267 INFO:tasks.workunit.client.0.smithi067.stderr: 68500K .......... .......... .......... .......... .......... 50% 8.77M 3s 2024-03-30T02:38:35.267 INFO:tasks.workunit.client.0.smithi067.stderr: 68550K .......... .......... .......... .......... .......... 50% 99.1M 3s 2024-03-30T02:38:35.271 INFO:tasks.workunit.client.0.smithi067.stderr: 68600K .......... .......... .......... .......... .......... 50% 16.0M 3s 2024-03-30T02:38:35.271 INFO:tasks.workunit.client.0.smithi067.stderr: 68650K .......... .......... .......... .......... .......... 50% 85.7M 3s 2024-03-30T02:38:35.271 INFO:tasks.workunit.client.0.smithi067.stderr: 68700K .......... .......... .......... .......... .......... 50% 98.3M 3s 2024-03-30T02:38:35.274 INFO:tasks.workunit.client.0.smithi067.stderr: 68750K .......... .......... .......... .......... .......... 50% 22.2M 3s 2024-03-30T02:38:35.274 INFO:tasks.workunit.client.0.smithi067.stderr: 68800K .......... .......... .......... .......... .......... 50% 83.9M 3s 2024-03-30T02:38:35.282 INFO:tasks.workunit.client.0.smithi067.stderr: 68850K .......... .......... .......... .......... .......... 50% 6.82M 3s 2024-03-30T02:38:35.282 INFO:tasks.workunit.client.0.smithi067.stderr: 68900K .......... .......... .......... .......... .......... 50% 98.5M 3s 2024-03-30T02:38:35.282 INFO:tasks.workunit.client.0.smithi067.stderr: 68950K .......... .......... .......... .......... .......... 50% 101M 3s 2024-03-30T02:38:35.286 INFO:tasks.workunit.client.0.smithi067.stderr: 69000K .......... .......... .......... .......... .......... 50% 16.1M 3s 2024-03-30T02:38:35.286 INFO:tasks.workunit.client.0.smithi067.stderr: 69050K .......... .......... .......... .......... .......... 50% 72.9M 3s 2024-03-30T02:38:35.289 INFO:tasks.workunit.client.0.smithi067.stderr: 69100K .......... .......... .......... .......... .......... 50% 19.7M 3s 2024-03-30T02:38:35.290 INFO:tasks.workunit.client.0.smithi067.stderr: 69150K .......... .......... .......... .......... .......... 50% 98.4M 3s 2024-03-30T02:38:35.290 INFO:tasks.workunit.client.0.smithi067.stderr: 69200K .......... .......... .......... .......... .......... 51% 95.9M 3s 2024-03-30T02:38:35.292 INFO:tasks.workunit.client.0.smithi067.stderr: 69250K .......... .......... .......... .......... .......... 51% 25.1M 3s 2024-03-30T02:38:35.292 INFO:tasks.workunit.client.0.smithi067.stderr: 69300K .......... .......... .......... .......... .......... 51% 101M 3s 2024-03-30T02:38:35.292 INFO:tasks.workunit.client.0.smithi067.stderr: 69350K .......... .......... .......... .......... .......... 51% 90.2M 3s 2024-03-30T02:38:35.296 INFO:tasks.workunit.client.0.smithi067.stderr: 69400K .......... .......... .......... .......... .......... 51% 16.1M 3s 2024-03-30T02:38:35.296 INFO:tasks.workunit.client.0.smithi067.stderr: 69450K .......... .......... .......... .......... .......... 51% 86.0M 3s 2024-03-30T02:38:35.300 INFO:tasks.workunit.client.0.smithi067.stderr: 69500K .......... .......... .......... .......... .......... 51% 24.1M 3s 2024-03-30T02:38:35.300 INFO:tasks.workunit.client.0.smithi067.stderr: 69550K .......... .......... .......... .......... .......... 51% 96.4M 3s 2024-03-30T02:38:35.300 INFO:tasks.workunit.client.0.smithi067.stderr: 69600K .......... .......... .......... .......... .......... 51% 68.2M 3s 2024-03-30T02:38:35.303 INFO:tasks.workunit.client.0.smithi067.stderr: 69650K .......... .......... .......... .......... .......... 51% 14.6M 3s 2024-03-30T02:38:35.303 INFO:tasks.workunit.client.0.smithi067.stderr: 69700K .......... .......... .......... .......... .......... 51% 83.3M 3s 2024-03-30T02:38:35.308 INFO:tasks.workunit.client.0.smithi067.stderr: 69750K .......... .......... .......... .......... .......... 51% 10.7M 3s 2024-03-30T02:38:35.308 INFO:tasks.workunit.client.0.smithi067.stderr: 69800K .......... .......... .......... .......... .......... 51% 98.2M 3s 2024-03-30T02:38:35.308 INFO:tasks.workunit.client.0.smithi067.stderr: 69850K .......... .......... .......... .......... .......... 51% 88.4M 3s 2024-03-30T02:38:35.311 INFO:tasks.workunit.client.0.smithi067.stderr: 69900K .......... .......... .......... .......... .......... 51% 18.4M 3s 2024-03-30T02:38:35.313 INFO:tasks.workunit.client.0.smithi067.stderr: 69950K .......... .......... .......... .......... .......... 51% 23.8M 3s 2024-03-30T02:38:35.318 INFO:tasks.workunit.client.0.smithi067.stderr: 70000K .......... .......... .......... .......... .......... 51% 11.4M 3s 2024-03-30T02:38:35.319 INFO:tasks.workunit.client.0.smithi067.stderr: 70050K .......... .......... .......... .......... .......... 51% 76.7M 3s 2024-03-30T02:38:35.319 INFO:tasks.workunit.client.0.smithi067.stderr: 70100K .......... .......... .......... .......... .......... 51% 103M 3s 2024-03-30T02:38:35.323 INFO:tasks.workunit.client.0.smithi067.stderr: 70150K .......... .......... .......... .......... .......... 51% 13.6M 3s 2024-03-30T02:38:35.323 INFO:tasks.workunit.client.0.smithi067.stderr: 70200K .......... .......... .......... .......... .......... 51% 103M 3s 2024-03-30T02:38:35.327 INFO:tasks.workunit.client.0.smithi067.stderr: 70250K .......... .......... .......... .......... .......... 51% 15.6M 3s 2024-03-30T02:38:35.327 INFO:tasks.workunit.client.0.smithi067.stderr: 70300K .......... .......... .......... .......... .......... 51% 95.8M 3s 2024-03-30T02:38:35.327 INFO:tasks.workunit.client.0.smithi067.stderr: 70350K .......... .......... .......... .......... .......... 51% 99.6M 3s 2024-03-30T02:38:35.334 INFO:tasks.workunit.client.0.smithi067.stderr: 70400K .......... .......... .......... .......... .......... 51% 7.23M 3s 2024-03-30T02:38:35.334 INFO:tasks.workunit.client.0.smithi067.stderr: 70450K .......... .......... .......... .......... .......... 51% 83.2M 3s 2024-03-30T02:38:35.334 INFO:tasks.workunit.client.0.smithi067.stderr: 70500K .......... .......... .......... .......... .......... 51% 111M 3s 2024-03-30T02:38:35.341 INFO:tasks.workunit.client.0.smithi067.stderr: 70550K .......... .......... .......... .......... .......... 52% 8.84M 3s 2024-03-30T02:38:35.341 INFO:tasks.workunit.client.0.smithi067.stderr: 70600K .......... .......... .......... .......... .......... 52% 106M 3s 2024-03-30T02:38:35.344 INFO:tasks.workunit.client.0.smithi067.stderr: 70650K .......... .......... .......... .......... .......... 52% 17.8M 3s 2024-03-30T02:38:35.344 INFO:tasks.workunit.client.0.smithi067.stderr: 70700K .......... .......... .......... .......... .......... 52% 99.5M 3s 2024-03-30T02:38:35.344 INFO:tasks.workunit.client.0.smithi067.stderr: 70750K .......... .......... .......... .......... .......... 52% 87.0M 3s 2024-03-30T02:38:35.349 INFO:tasks.workunit.client.0.smithi067.stderr: 70800K .......... .......... .......... .......... .......... 52% 12.4M 3s 2024-03-30T02:38:35.349 INFO:tasks.workunit.client.0.smithi067.stderr: 70850K .......... .......... .......... .......... .......... 52% 104M 3s 2024-03-30T02:38:35.355 INFO:tasks.workunit.client.0.smithi067.stderr: 70900K .......... .......... .......... .......... .......... 52% 9.17M 3s 2024-03-30T02:38:35.355 INFO:tasks.workunit.client.0.smithi067.stderr: 70950K .......... .......... .......... .......... .......... 52% 98.6M 3s 2024-03-30T02:38:35.355 INFO:tasks.workunit.client.0.smithi067.stderr: 71000K .......... .......... .......... .......... .......... 52% 85.0M 3s 2024-03-30T02:38:35.357 INFO:tasks.workunit.client.0.smithi067.stderr: 71050K .......... .......... .......... .......... .......... 52% 24.5M 3s 2024-03-30T02:38:35.361 INFO:tasks.workunit.client.0.smithi067.stderr: 71100K .......... .......... .......... .......... .......... 52% 12.4M 3s 2024-03-30T02:38:35.366 INFO:tasks.workunit.client.0.smithi067.stderr: 71150K .......... .......... .......... .......... .......... 52% 15.8M 3s 2024-03-30T02:38:35.366 INFO:tasks.workunit.client.0.smithi067.stderr: 71200K .......... .......... .......... .......... .......... 52% 56.7M 3s 2024-03-30T02:38:35.366 INFO:tasks.workunit.client.0.smithi067.stderr: 71250K .......... .......... .......... .......... .......... 52% 112M 3s 2024-03-30T02:38:35.369 INFO:tasks.workunit.client.0.smithi067.stderr: 71300K .......... .......... .......... .......... .......... 52% 17.0M 3s 2024-03-30T02:38:35.369 INFO:tasks.workunit.client.0.smithi067.stderr: 71350K .......... .......... .......... .......... .......... 52% 84.5M 3s 2024-03-30T02:38:35.369 INFO:tasks.workunit.client.0.smithi067.stderr: 71400K .......... .......... .......... .......... .......... 52% 106M 3s 2024-03-30T02:38:35.375 INFO:tasks.workunit.client.0.smithi067.stderr: 71450K .......... .......... .......... .......... .......... 52% 11.4M 3s 2024-03-30T02:38:35.375 INFO:tasks.workunit.client.0.smithi067.stderr: 71500K .......... .......... .......... .......... .......... 52% 83.7M 3s 2024-03-30T02:38:35.376 INFO:tasks.workunit.client.0.smithi067.stderr: 71550K .......... .......... .......... .......... .......... 52% 25.1M 3s 2024-03-30T02:38:35.377 INFO:tasks.workunit.client.0.smithi067.stderr: 71600K .......... .......... .......... .......... .......... 52% 43.1M 3s 2024-03-30T02:38:35.377 INFO:tasks.workunit.client.0.smithi067.stderr: 71650K .......... .......... .......... .......... .......... 52% 118M 3s 2024-03-30T02:38:35.382 INFO:tasks.workunit.client.0.smithi067.stderr: 71700K .......... .......... .......... .......... .......... 52% 12.4M 3s 2024-03-30T02:38:35.382 INFO:tasks.workunit.client.0.smithi067.stderr: 71750K .......... .......... .......... .......... .......... 52% 98.8M 3s 2024-03-30T02:38:35.387 INFO:tasks.workunit.client.0.smithi067.stderr: 71800K .......... .......... .......... .......... .......... 52% 14.9M 3s 2024-03-30T02:38:35.388 INFO:tasks.workunit.client.0.smithi067.stderr: 71850K .......... .......... .......... .......... .......... 52% 108M 3s 2024-03-30T02:38:35.388 INFO:tasks.workunit.client.0.smithi067.stderr: 71900K .......... .......... .......... .......... .......... 53% 109M 3s 2024-03-30T02:38:35.392 INFO:tasks.workunit.client.0.smithi067.stderr: 71950K .......... .......... .......... .......... .......... 53% 14.5M 3s 2024-03-30T02:38:35.392 INFO:tasks.workunit.client.0.smithi067.stderr: 72000K .......... .......... .......... .......... .......... 53% 72.5M 3s 2024-03-30T02:38:35.399 INFO:tasks.workunit.client.0.smithi067.stderr: 72050K .......... .......... .......... .......... .......... 53% 7.12M 3s 2024-03-30T02:38:35.400 INFO:tasks.workunit.client.0.smithi067.stderr: 72100K .......... .......... .......... .......... .......... 53% 111M 3s 2024-03-30T02:38:35.400 INFO:tasks.workunit.client.0.smithi067.stderr: 72150K .......... .......... .......... .......... .......... 53% 94.5M 3s 2024-03-30T02:38:35.403 INFO:tasks.workunit.client.0.smithi067.stderr: 72200K .......... .......... .......... .......... .......... 53% 13.8M 3s 2024-03-30T02:38:35.403 INFO:tasks.workunit.client.0.smithi067.stderr: 72250K .......... .......... .......... .......... .......... 53% 136M 3s 2024-03-30T02:38:35.409 INFO:tasks.workunit.client.0.smithi067.stderr: 72300K .......... .......... .......... .......... .......... 53% 10.8M 3s 2024-03-30T02:38:35.409 INFO:tasks.workunit.client.0.smithi067.stderr: 72350K .......... .......... .......... .......... .......... 53% 111M 3s 2024-03-30T02:38:35.409 INFO:tasks.workunit.client.0.smithi067.stderr: 72400K .......... .......... .......... .......... .......... 53% 103M 3s 2024-03-30T02:38:35.412 INFO:tasks.workunit.client.0.smithi067.stderr: 72450K .......... .......... .......... .......... .......... 53% 11.2M 3s 2024-03-30T02:38:35.412 INFO:tasks.workunit.client.0.smithi067.stderr: 72500K .......... .......... .......... .......... .......... 53% 170M 3s 2024-03-30T02:38:35.412 INFO:tasks.workunit.client.0.smithi067.stderr: 72550K .......... .......... .......... .......... .......... 53% 177M 3s 2024-03-30T02:38:35.417 INFO:tasks.workunit.client.0.smithi067.stderr: 72600K .......... .......... .......... .......... .......... 53% 10.9M 3s 2024-03-30T02:38:35.417 INFO:tasks.workunit.client.0.smithi067.stderr: 72650K .......... .......... .......... .......... .......... 53% 142M 3s 2024-03-30T02:38:35.422 INFO:tasks.workunit.client.0.smithi067.stderr: 72700K .......... .......... .......... .......... .......... 53% 16.5M 3s 2024-03-30T02:38:35.422 INFO:tasks.workunit.client.0.smithi067.stderr: 72750K .......... .......... .......... .......... .......... 53% 101M 3s 2024-03-30T02:38:35.422 INFO:tasks.workunit.client.0.smithi067.stderr: 72800K .......... .......... .......... .......... .......... 53% 80.4M 3s 2024-03-30T02:38:35.424 INFO:tasks.workunit.client.0.smithi067.stderr: 72850K .......... .......... .......... .......... .......... 53% 20.7M 3s 2024-03-30T02:38:35.428 INFO:tasks.workunit.client.0.smithi067.stderr: 72900K .......... .......... .......... .......... .......... 53% 11.7M 3s 2024-03-30T02:38:35.430 INFO:tasks.workunit.client.0.smithi067.stderr: 72950K .......... .......... .......... .......... .......... 53% 32.8M 3s 2024-03-30T02:38:35.430 INFO:tasks.workunit.client.0.smithi067.stderr: 73000K .......... .......... .......... .......... .......... 53% 116M 3s 2024-03-30T02:38:35.430 INFO:tasks.workunit.client.0.smithi067.stderr: 73050K .......... .......... .......... .......... .......... 53% 156M 3s 2024-03-30T02:38:35.435 INFO:tasks.workunit.client.0.smithi067.stderr: 73100K .......... .......... .......... .......... .......... 53% 14.2M 3s 2024-03-30T02:38:35.435 INFO:tasks.workunit.client.0.smithi067.stderr: 73150K .......... .......... .......... .......... .......... 53% 110M 3s 2024-03-30T02:38:35.439 INFO:tasks.workunit.client.0.smithi067.stderr: 73200K .......... .......... .......... .......... .......... 53% 14.5M 3s 2024-03-30T02:38:35.440 INFO:tasks.workunit.client.0.smithi067.stderr: 73250K .......... .......... .......... .......... .......... 53% 70.0M 3s 2024-03-30T02:38:35.440 INFO:tasks.workunit.client.0.smithi067.stderr: 73300K .......... .......... .......... .......... .......... 54% 105M 3s 2024-03-30T02:38:35.443 INFO:tasks.workunit.client.0.smithi067.stderr: 73350K .......... .......... .......... .......... .......... 54% 14.8M 3s 2024-03-30T02:38:35.443 INFO:tasks.workunit.client.0.smithi067.stderr: 73400K .......... .......... .......... .......... .......... 54% 160M 3s 2024-03-30T02:38:35.449 INFO:tasks.workunit.client.0.smithi067.stderr: 73450K .......... .......... .......... .......... .......... 54% 8.61M 3s 2024-03-30T02:38:35.449 INFO:tasks.workunit.client.0.smithi067.stderr: 73500K .......... .......... .......... .......... .......... 54% 144M 3s 2024-03-30T02:38:35.449 INFO:tasks.workunit.client.0.smithi067.stderr: 73550K .......... .......... .......... .......... .......... 54% 104M 3s 2024-03-30T02:38:35.453 INFO:tasks.workunit.client.0.smithi067.stderr: 73600K .......... .......... .......... .......... .......... 54% 16.3M 3s 2024-03-30T02:38:35.453 INFO:tasks.workunit.client.0.smithi067.stderr: 73650K .......... .......... .......... .......... .......... 54% 49.6M 3s 2024-03-30T02:38:35.454 INFO:tasks.workunit.client.0.smithi067.stderr: 73700K .......... .......... .......... .......... .......... 54% 109M 3s 2024-03-30T02:38:35.458 INFO:tasks.workunit.client.0.smithi067.stderr: 73750K .......... .......... .......... .......... .......... 54% 12.4M 3s 2024-03-30T02:38:35.458 INFO:tasks.workunit.client.0.smithi067.stderr: 73800K .......... .......... .......... .......... .......... 54% 112M 3s 2024-03-30T02:38:35.464 INFO:tasks.workunit.client.0.smithi067.stderr: 73850K .......... .......... .......... .......... .......... 54% 16.5M 3s 2024-03-30T02:38:35.464 INFO:tasks.workunit.client.0.smithi067.stderr: 73900K .......... .......... .......... .......... .......... 54% 60.6M 3s 2024-03-30T02:38:35.464 INFO:tasks.workunit.client.0.smithi067.stderr: 73950K .......... .......... .......... .......... .......... 54% 80.1M 3s 2024-03-30T02:38:35.469 INFO:tasks.workunit.client.0.smithi067.stderr: 74000K .......... .......... .......... .......... .......... 54% 9.87M 3s 2024-03-30T02:38:35.469 INFO:tasks.workunit.client.0.smithi067.stderr: 74050K .......... .......... .......... .......... .......... 54% 108M 3s 2024-03-30T02:38:35.473 INFO:tasks.workunit.client.0.smithi067.stderr: 74100K .......... .......... .......... .......... .......... 54% 15.9M 3s 2024-03-30T02:38:35.473 INFO:tasks.workunit.client.0.smithi067.stderr: 74150K .......... .......... .......... .......... .......... 54% 58.4M 3s 2024-03-30T02:38:35.473 INFO:tasks.workunit.client.0.smithi067.stderr: 74200K .......... .......... .......... .......... .......... 54% 110M 3s 2024-03-30T02:38:35.476 INFO:tasks.workunit.client.0.smithi067.stderr: 74250K .......... .......... .......... .......... .......... 54% 16.1M 3s 2024-03-30T02:38:35.476 INFO:tasks.workunit.client.0.smithi067.stderr: 74300K .......... .......... .......... .......... .......... 54% 109M 3s 2024-03-30T02:38:35.481 INFO:tasks.workunit.client.0.smithi067.stderr: 74350K .......... .......... .......... .......... .......... 54% 13.9M 3s 2024-03-30T02:38:35.481 INFO:tasks.workunit.client.0.smithi067.stderr: 74400K .......... .......... .......... .......... .......... 54% 53.2M 3s 2024-03-30T02:38:35.481 INFO:tasks.workunit.client.0.smithi067.stderr: 74450K .......... .......... .......... .......... .......... 54% 101M 3s 2024-03-30T02:38:35.484 INFO:tasks.workunit.client.0.smithi067.stderr: 74500K .......... .......... .......... .......... .......... 54% 17.4M 3s 2024-03-30T02:38:35.484 INFO:tasks.workunit.client.0.smithi067.stderr: 74550K .......... .......... .......... .......... .......... 54% 115M 3s 2024-03-30T02:38:35.489 INFO:tasks.workunit.client.0.smithi067.stderr: 74600K .......... .......... .......... .......... .......... 54% 105M 3s 2024-03-30T02:38:35.489 INFO:tasks.workunit.client.0.smithi067.stderr: 74650K .......... .......... .......... .......... .......... 55% 13.2M 3s 2024-03-30T02:38:35.489 INFO:tasks.workunit.client.0.smithi067.stderr: 74700K .......... .......... .......... .......... .......... 55% 104M 3s 2024-03-30T02:38:35.493 INFO:tasks.workunit.client.0.smithi067.stderr: 74750K .......... .......... .......... .......... .......... 55% 13.4M 3s 2024-03-30T02:38:35.493 INFO:tasks.workunit.client.0.smithi067.stderr: 74800K .......... .......... .......... .......... .......... 55% 153M 3s 2024-03-30T02:38:35.493 INFO:tasks.workunit.client.0.smithi067.stderr: 74850K .......... .......... .......... .......... .......... 55% 103M 3s 2024-03-30T02:38:35.502 INFO:tasks.workunit.client.0.smithi067.stderr: 74900K .......... .......... .......... .......... .......... 55% 18.6M 3s 2024-03-30T02:38:35.502 INFO:tasks.workunit.client.0.smithi067.stderr: 74950K .......... .......... .......... .......... .......... 55% 112M 3s 2024-03-30T02:38:35.502 INFO:tasks.workunit.client.0.smithi067.stderr: 75000K .......... .......... .......... .......... .......... 55% 31.3M 3s 2024-03-30T02:38:35.502 INFO:tasks.workunit.client.0.smithi067.stderr: 75050K .......... .......... .......... .......... .......... 55% 104M 3s 2024-03-30T02:38:35.502 INFO:tasks.workunit.client.0.smithi067.stderr: 75100K .......... .......... .......... .......... .......... 55% 110M 3s 2024-03-30T02:38:35.502 INFO:tasks.workunit.client.0.smithi067.stderr: 75150K .......... .......... .......... .......... .......... 55% 42.9M 3s 2024-03-30T02:38:35.502 INFO:tasks.workunit.client.0.smithi067.stderr: 75200K .......... .......... .......... .......... .......... 55% 115M 3s 2024-03-30T02:38:35.504 INFO:tasks.workunit.client.0.smithi067.stderr: 75250K .......... .......... .......... .......... .......... 55% 13.9M 3s 2024-03-30T02:38:35.504 INFO:tasks.workunit.client.0.smithi067.stderr: 75300K .......... .......... .......... .......... .......... 55% 109M 3s 2024-03-30T02:38:35.505 INFO:tasks.workunit.client.0.smithi067.stderr: 75350K .......... .......... .......... .......... .......... 55% 169M 3s 2024-03-30T02:38:35.521 INFO:tasks.workunit.client.0.smithi067.stderr: 75400K .......... .......... .......... .......... .......... 55% 3.81M 3s 2024-03-30T02:38:35.521 INFO:tasks.workunit.client.0.smithi067.stderr: 75450K .......... .......... .......... .......... .......... 55% 114M 3s 2024-03-30T02:38:35.521 INFO:tasks.workunit.client.0.smithi067.stderr: 75500K .......... .......... .......... .......... .......... 55% 26.2M 3s 2024-03-30T02:38:35.521 INFO:tasks.workunit.client.0.smithi067.stderr: 75550K .......... .......... .......... .......... .......... 55% 114M 3s 2024-03-30T02:38:35.521 INFO:tasks.workunit.client.0.smithi067.stderr: 75600K .......... .......... .......... .......... .......... 55% 107M 3s 2024-03-30T02:38:35.528 INFO:tasks.workunit.client.0.smithi067.stderr: 75650K .......... .......... .......... .......... .......... 55% 6.83M 3s 2024-03-30T02:38:35.528 INFO:tasks.workunit.client.0.smithi067.stderr: 75700K .......... .......... .......... .......... .......... 55% 119M 3s 2024-03-30T02:38:35.528 INFO:tasks.workunit.client.0.smithi067.stderr: 75750K .......... .......... .......... .......... .......... 55% 106M 3s 2024-03-30T02:38:35.532 INFO:tasks.workunit.client.0.smithi067.stderr: 75800K .......... .......... .......... .......... .......... 55% 15.3M 3s 2024-03-30T02:38:35.532 INFO:tasks.workunit.client.0.smithi067.stderr: 75850K .......... .......... .......... .......... .......... 55% 115M 3s 2024-03-30T02:38:35.536 INFO:tasks.workunit.client.0.smithi067.stderr: 75900K .......... .......... .......... .......... .......... 55% 12.5M 3s 2024-03-30T02:38:35.536 INFO:tasks.workunit.client.0.smithi067.stderr: 75950K .......... .......... .......... .......... .......... 55% 171M 3s 2024-03-30T02:38:35.536 INFO:tasks.workunit.client.0.smithi067.stderr: 76000K .......... .......... .......... .......... .......... 56% 146M 3s 2024-03-30T02:38:35.540 INFO:tasks.workunit.client.0.smithi067.stderr: 76050K .......... .......... .......... .......... .......... 56% 13.6M 3s 2024-03-30T02:38:35.540 INFO:tasks.workunit.client.0.smithi067.stderr: 76100K .......... .......... .......... .......... .......... 56% 163M 3s 2024-03-30T02:38:35.548 INFO:tasks.workunit.client.0.smithi067.stderr: 76150K .......... .......... .......... .......... .......... 56% 14.2M 3s 2024-03-30T02:38:35.548 INFO:tasks.workunit.client.0.smithi067.stderr: 76200K .......... .......... .......... .......... .......... 56% 113M 3s 2024-03-30T02:38:35.548 INFO:tasks.workunit.client.0.smithi067.stderr: 76250K .......... .......... .......... .......... .......... 56% 110M 3s 2024-03-30T02:38:35.548 INFO:tasks.workunit.client.0.smithi067.stderr: 76300K .......... .......... .......... .......... .......... 56% 21.1M 3s 2024-03-30T02:38:35.552 INFO:tasks.workunit.client.0.smithi067.stderr: 76350K .......... .......... .......... .......... .......... 56% 11.2M 3s 2024-03-30T02:38:35.555 INFO:tasks.workunit.client.0.smithi067.stderr: 76400K .......... .......... .......... .......... .......... 56% 11.6M 3s 2024-03-30T02:38:35.555 INFO:tasks.workunit.client.0.smithi067.stderr: 76450K .......... .......... .......... .......... .......... 56% 109M 3s 2024-03-30T02:38:35.560 INFO:tasks.workunit.client.0.smithi067.stderr: 76500K .......... .......... .......... .......... .......... 56% 12.5M 3s 2024-03-30T02:38:35.569 INFO:tasks.workunit.client.0.smithi067.stderr: 76550K .......... .......... .......... .......... .......... 56% 12.2M 3s 2024-03-30T02:38:35.569 INFO:tasks.workunit.client.0.smithi067.stderr: 76600K .......... .......... .......... .......... .......... 56% 97.8M 3s 2024-03-30T02:38:35.569 INFO:tasks.workunit.client.0.smithi067.stderr: 76650K .......... .......... .......... .......... .......... 56% 18.6M 3s 2024-03-30T02:38:35.569 INFO:tasks.workunit.client.0.smithi067.stderr: 76700K .......... .......... .......... .......... .......... 56% 106M 3s 2024-03-30T02:38:35.569 INFO:tasks.workunit.client.0.smithi067.stderr: 76750K .......... .......... .......... .......... .......... 56% 177M 3s 2024-03-30T02:38:35.581 INFO:tasks.workunit.client.0.smithi067.stderr: 76800K .......... .......... .......... .......... .......... 56% 7.47M 3s 2024-03-30T02:38:35.581 INFO:tasks.workunit.client.0.smithi067.stderr: 76850K .......... .......... .......... .......... .......... 56% 154M 3s 2024-03-30T02:38:35.582 INFO:tasks.workunit.client.0.smithi067.stderr: 76900K .......... .......... .......... .......... .......... 56% 153M 2s 2024-03-30T02:38:35.582 INFO:tasks.workunit.client.0.smithi067.stderr: 76950K .......... .......... .......... .......... .......... 56% 39.9M 2s 2024-03-30T02:38:35.582 INFO:tasks.workunit.client.0.smithi067.stderr: 77000K .......... .......... .......... .......... .......... 56% 182M 2s 2024-03-30T02:38:35.582 INFO:tasks.workunit.client.0.smithi067.stderr: 77050K .......... .......... .......... .......... .......... 56% 16.6M 2s 2024-03-30T02:38:35.582 INFO:tasks.workunit.client.0.smithi067.stderr: 77100K .......... .......... .......... .......... .......... 56% 116M 2s 2024-03-30T02:38:35.582 INFO:tasks.workunit.client.0.smithi067.stderr: 77150K .......... .......... .......... .......... .......... 56% 115M 2s 2024-03-30T02:38:35.586 INFO:tasks.workunit.client.0.smithi067.stderr: 77200K .......... .......... .......... .......... .......... 56% 12.0M 2s 2024-03-30T02:38:35.586 INFO:tasks.workunit.client.0.smithi067.stderr: 77250K .......... .......... .......... .......... .......... 56% 101M 2s 2024-03-30T02:38:35.588 INFO:tasks.workunit.client.0.smithi067.stderr: 77300K .......... .......... .......... .......... .......... 56% 14.1M 2s 2024-03-30T02:38:35.589 INFO:tasks.workunit.client.0.smithi067.stderr: 77350K .......... .......... .......... .......... .......... 57% 119M 2s 2024-03-30T02:38:35.589 INFO:tasks.workunit.client.0.smithi067.stderr: 77400K .......... .......... .......... .......... .......... 57% 168M 2s 2024-03-30T02:38:35.592 INFO:tasks.workunit.client.0.smithi067.stderr: 77450K .......... .......... .......... .......... .......... 57% 21.1M 2s 2024-03-30T02:38:35.592 INFO:tasks.workunit.client.0.smithi067.stderr: 77500K .......... .......... .......... .......... .......... 57% 118M 2s 2024-03-30T02:38:35.597 INFO:tasks.workunit.client.0.smithi067.stderr: 77550K .......... .......... .......... .......... .......... 57% 12.5M 2s 2024-03-30T02:38:35.597 INFO:tasks.workunit.client.0.smithi067.stderr: 77600K .......... .......... .......... .......... .......... 57% 124M 2s 2024-03-30T02:38:35.597 INFO:tasks.workunit.client.0.smithi067.stderr: 77650K .......... .......... .......... .......... .......... 57% 145M 2s 2024-03-30T02:38:35.602 INFO:tasks.workunit.client.0.smithi067.stderr: 77700K .......... .......... .......... .......... .......... 57% 14.6M 2s 2024-03-30T02:38:35.602 INFO:tasks.workunit.client.0.smithi067.stderr: 77750K .......... .......... .......... .......... .......... 57% 142M 2s 2024-03-30T02:38:35.602 INFO:tasks.workunit.client.0.smithi067.stderr: 77800K .......... .......... .......... .......... .......... 57% 44.0M 2s 2024-03-30T02:38:35.602 INFO:tasks.workunit.client.0.smithi067.stderr: 77850K .......... .......... .......... .......... .......... 57% 164M 2s 2024-03-30T02:38:35.605 INFO:tasks.workunit.client.0.smithi067.stderr: 77900K .......... .......... .......... .......... .......... 57% 12.6M 2s 2024-03-30T02:38:35.609 INFO:tasks.workunit.client.0.smithi067.stderr: 77950K .......... .......... .......... .......... .......... 57% 14.5M 2s 2024-03-30T02:38:35.609 INFO:tasks.workunit.client.0.smithi067.stderr: 78000K .......... .......... .......... .......... .......... 57% 171M 2s 2024-03-30T02:38:35.609 INFO:tasks.workunit.client.0.smithi067.stderr: 78050K .......... .......... .......... .......... .......... 57% 156M 2s 2024-03-30T02:38:35.613 INFO:tasks.workunit.client.0.smithi067.stderr: 78100K .......... .......... .......... .......... .......... 57% 14.6M 2s 2024-03-30T02:38:35.613 INFO:tasks.workunit.client.0.smithi067.stderr: 78150K .......... .......... .......... .......... .......... 57% 177M 2s 2024-03-30T02:38:35.616 INFO:tasks.workunit.client.0.smithi067.stderr: 78200K .......... .......... .......... .......... .......... 57% 19.9M 2s 2024-03-30T02:38:35.616 INFO:tasks.workunit.client.0.smithi067.stderr: 78250K .......... .......... .......... .......... .......... 57% 107M 2s 2024-03-30T02:38:35.616 INFO:tasks.workunit.client.0.smithi067.stderr: 78300K .......... .......... .......... .......... .......... 57% 89.4M 2s 2024-03-30T02:38:35.626 INFO:tasks.workunit.client.0.smithi067.stderr: 78350K .......... .......... .......... .......... .......... 57% 6.38M 2s 2024-03-30T02:38:35.626 INFO:tasks.workunit.client.0.smithi067.stderr: 78400K .......... .......... .......... .......... .......... 57% 109M 2s 2024-03-30T02:38:35.634 INFO:tasks.workunit.client.0.smithi067.stderr: 78450K .......... .......... .......... .......... .......... 57% 12.4M 2s 2024-03-30T02:38:35.634 INFO:tasks.workunit.client.0.smithi067.stderr: 78500K .......... .......... .......... .......... .......... 57% 108M 2s 2024-03-30T02:38:35.634 INFO:tasks.workunit.client.0.smithi067.stderr: 78550K .......... .......... .......... .......... .......... 57% 82.9M 2s 2024-03-30T02:38:35.635 INFO:tasks.workunit.client.0.smithi067.stderr: 78600K .......... .......... .......... .......... .......... 57% 14.1M 2s 2024-03-30T02:38:35.635 INFO:tasks.workunit.client.0.smithi067.stderr: 78650K .......... .......... .......... .......... .......... 57% 98.3M 2s 2024-03-30T02:38:35.651 INFO:tasks.workunit.client.0.smithi067.stderr: 78700K .......... .......... .......... .......... .......... 58% 10.8M 2s 2024-03-30T02:38:35.651 INFO:tasks.workunit.client.0.smithi067.stderr: 78750K .......... .......... .......... .......... .......... 58% 113M 2s 2024-03-30T02:38:35.651 INFO:tasks.workunit.client.0.smithi067.stderr: 78800K .......... .......... .......... .......... .......... 58% 111M 2s 2024-03-30T02:38:35.651 INFO:tasks.workunit.client.0.smithi067.stderr: 78850K .......... .......... .......... .......... .......... 58% 58.5M 2s 2024-03-30T02:38:35.651 INFO:tasks.workunit.client.0.smithi067.stderr: 78900K .......... .......... .......... .......... .......... 58% 106M 2s 2024-03-30T02:38:35.652 INFO:tasks.workunit.client.0.smithi067.stderr: 78950K .......... .......... .......... .......... .......... 58% 80.6M 2s 2024-03-30T02:38:35.652 INFO:tasks.workunit.client.0.smithi067.stderr: 79000K .......... .......... .......... .......... .......... 58% 8.81M 2s 2024-03-30T02:38:35.652 INFO:tasks.workunit.client.0.smithi067.stderr: 79050K .......... .......... .......... .......... .......... 58% 111M 2s 2024-03-30T02:38:35.652 INFO:tasks.workunit.client.0.smithi067.stderr: 79100K .......... .......... .......... .......... .......... 58% 40.5M 2s 2024-03-30T02:38:35.652 INFO:tasks.workunit.client.0.smithi067.stderr: 79150K .......... .......... .......... .......... .......... 58% 15.8M 2s 2024-03-30T02:38:35.652 INFO:tasks.workunit.client.0.smithi067.stderr: 79200K .......... .......... .......... .......... .......... 58% 125M 2s 2024-03-30T02:38:35.656 INFO:tasks.workunit.client.0.smithi067.stderr: 79250K .......... .......... .......... .......... .......... 58% 8.04M 2s 2024-03-30T02:38:35.659 INFO:tasks.workunit.client.0.smithi067.stderr: 79300K .......... .......... .......... .......... .......... 58% 25.1M 2s 2024-03-30T02:38:35.700 INFO:tasks.workunit.client.0.smithi067.stderr: 79350K .......... .......... .......... .......... .......... 58% 10.7M 2s 2024-03-30T02:38:35.700 INFO:tasks.workunit.client.0.smithi067.stderr: 79400K .......... .......... .......... .......... .......... 58% 114M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79450K .......... .......... .......... .......... .......... 58% 108M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79500K .......... .......... .......... .......... .......... 58% 15.1M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79550K .......... .......... .......... .......... .......... 58% 99.0M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79600K .......... .......... .......... .......... .......... 58% 12.8M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79650K .......... .......... .......... .......... .......... 58% 119M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79700K .......... .......... .......... .......... .......... 58% 106M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79750K .......... .......... .......... .......... .......... 58% 19.2M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79800K .......... .......... .......... .......... .......... 58% 116M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79850K .......... .......... .......... .......... .......... 58% 15.1M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79900K .......... .......... .......... .......... .......... 58% 118M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 79950K .......... .......... .......... .......... .......... 58% 107M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 80000K .......... .......... .......... .......... .......... 58% 14.4M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 80050K .......... .......... .......... .......... .......... 59% 115M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 80100K .......... .......... .......... .......... .......... 59% 112M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 80150K .......... .......... .......... .......... .......... 59% 12.8M 2s 2024-03-30T02:38:35.701 INFO:tasks.workunit.client.0.smithi067.stderr: 80200K .......... .......... .......... .......... .......... 59% 105M 2s 2024-03-30T02:38:35.702 INFO:tasks.workunit.client.0.smithi067.stderr: 80250K .......... .......... .......... .......... .......... 59% 14.6M 2s 2024-03-30T02:38:35.702 INFO:tasks.workunit.client.0.smithi067.stderr: 80300K .......... .......... .......... .......... .......... 59% 116M 2s 2024-03-30T02:38:35.702 INFO:tasks.workunit.client.0.smithi067.stderr: 80350K .......... .......... .......... .......... .......... 59% 113M 2s 2024-03-30T02:38:35.702 INFO:tasks.workunit.client.0.smithi067.stderr: 80400K .......... .......... .......... .......... .......... 59% 20.9M 2s 2024-03-30T02:38:35.702 INFO:tasks.workunit.client.0.smithi067.stderr: 80450K .......... .......... .......... .......... .......... 59% 112M 2s 2024-03-30T02:38:35.702 INFO:tasks.workunit.client.0.smithi067.stderr: 80500K .......... .......... .......... .......... .......... 59% 13.3M 2s 2024-03-30T02:38:35.702 INFO:tasks.workunit.client.0.smithi067.stderr: 80550K .......... .......... .......... .......... .......... 59% 108M 2s 2024-03-30T02:38:35.702 INFO:tasks.workunit.client.0.smithi067.stderr: 80600K .......... .......... .......... .......... .......... 59% 86.6M 2s 2024-03-30T02:38:35.709 INFO:tasks.workunit.client.0.smithi067.stderr: 80650K .......... .......... .......... .......... .......... 59% 14.1M 2s 2024-03-30T02:38:35.709 INFO:tasks.workunit.client.0.smithi067.stderr: 80700K .......... .......... .......... .......... .......... 59% 108M 2s 2024-03-30T02:38:35.709 INFO:tasks.workunit.client.0.smithi067.stderr: 80750K .......... .......... .......... .......... .......... 59% 15.9M 2s 2024-03-30T02:38:35.709 INFO:tasks.workunit.client.0.smithi067.stderr: 80800K .......... .......... .......... .......... .......... 59% 116M 2s 2024-03-30T02:38:35.709 INFO:tasks.workunit.client.0.smithi067.stderr: 80850K .......... .......... .......... .......... .......... 59% 111M 2s 2024-03-30T02:38:35.720 INFO:tasks.workunit.client.0.smithi067.stderr: 80900K .......... .......... .......... .......... .......... 59% 6.52M 2s 2024-03-30T02:38:35.720 INFO:tasks.workunit.client.0.smithi067.stderr: 80950K .......... .......... .......... .......... .......... 59% 105M 2s 2024-03-30T02:38:35.720 INFO:tasks.workunit.client.0.smithi067.stderr: 81000K .......... .......... .......... .......... .......... 59% 112M 2s 2024-03-30T02:38:35.720 INFO:tasks.workunit.client.0.smithi067.stderr: 81050K .......... .......... .......... .......... .......... 59% 16.9M 2s 2024-03-30T02:38:35.723 INFO:tasks.workunit.client.0.smithi067.stderr: 81100K .......... .......... .......... .......... .......... 59% 12.2M 2s 2024-03-30T02:38:35.729 INFO:tasks.workunit.client.0.smithi067.stderr: 81150K .......... .......... .......... .......... .......... 59% 11.6M 2s 2024-03-30T02:38:35.729 INFO:tasks.workunit.client.0.smithi067.stderr: 81200K .......... .......... .......... .......... .......... 59% 100M 2s 2024-03-30T02:38:35.729 INFO:tasks.workunit.client.0.smithi067.stderr: 81250K .......... .......... .......... .......... .......... 59% 110M 2s 2024-03-30T02:38:35.741 INFO:tasks.workunit.client.0.smithi067.stderr: 81300K .......... .......... .......... .......... .......... 59% 8.73M 2s 2024-03-30T02:38:35.741 INFO:tasks.workunit.client.0.smithi067.stderr: 81350K .......... .......... .......... .......... .......... 59% 117M 2s 2024-03-30T02:38:35.741 INFO:tasks.workunit.client.0.smithi067.stderr: 81400K .......... .......... .......... .......... .......... 59% 29.7M 2s 2024-03-30T02:38:35.741 INFO:tasks.workunit.client.0.smithi067.stderr: 81450K .......... .......... .......... .......... .......... 60% 158M 2s 2024-03-30T02:38:35.741 INFO:tasks.workunit.client.0.smithi067.stderr: 81500K .......... .......... .......... .......... .......... 60% 131M 2s 2024-03-30T02:38:35.741 INFO:tasks.workunit.client.0.smithi067.stderr: 81550K .......... .......... .......... .......... .......... 60% 17.6M 2s 2024-03-30T02:38:35.741 INFO:tasks.workunit.client.0.smithi067.stderr: 81600K .......... .......... .......... .......... .......... 60% 114M 2s 2024-03-30T02:38:35.745 INFO:tasks.workunit.client.0.smithi067.stderr: 81650K .......... .......... .......... .......... .......... 60% 14.6M 2s 2024-03-30T02:38:35.745 INFO:tasks.workunit.client.0.smithi067.stderr: 81700K .......... .......... .......... .......... .......... 60% 106M 2s 2024-03-30T02:38:35.745 INFO:tasks.workunit.client.0.smithi067.stderr: 81750K .......... .......... .......... .......... .......... 60% 116M 2s 2024-03-30T02:38:35.748 INFO:tasks.workunit.client.0.smithi067.stderr: 81800K .......... .......... .......... .......... .......... 60% 13.9M 2s 2024-03-30T02:38:35.749 INFO:tasks.workunit.client.0.smithi067.stderr: 81850K .......... .......... .......... .......... .......... 60% 166M 2s 2024-03-30T02:38:35.753 INFO:tasks.workunit.client.0.smithi067.stderr: 81900K .......... .......... .......... .......... .......... 60% 13.8M 2s 2024-03-30T02:38:35.753 INFO:tasks.workunit.client.0.smithi067.stderr: 81950K .......... .......... .......... .......... .......... 60% 157M 2s 2024-03-30T02:38:35.753 INFO:tasks.workunit.client.0.smithi067.stderr: 82000K .......... .......... .......... .......... .......... 60% 112M 2s 2024-03-30T02:38:35.761 INFO:tasks.workunit.client.0.smithi067.stderr: 82050K .......... .......... .......... .......... .......... 60% 7.13M 2s 2024-03-30T02:38:35.761 INFO:tasks.workunit.client.0.smithi067.stderr: 82100K .......... .......... .......... .......... .......... 60% 95.8M 2s 2024-03-30T02:38:35.761 INFO:tasks.workunit.client.0.smithi067.stderr: 82150K .......... .......... .......... .......... .......... 60% 99.4M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82200K .......... .......... .......... .......... .......... 60% 7.33M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82250K .......... .......... .......... .......... .......... 60% 166M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82300K .......... .......... .......... .......... .......... 60% 147M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82350K .......... .......... .......... .......... .......... 60% 61.1M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82400K .......... .......... .......... .......... .......... 60% 176M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82450K .......... .......... .......... .......... .......... 60% 14.5M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82500K .......... .......... .......... .......... .......... 60% 109M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82550K .......... .......... .......... .......... .......... 60% 13.6M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82600K .......... .......... .......... .......... .......... 60% 96.7M 2s 2024-03-30T02:38:35.778 INFO:tasks.workunit.client.0.smithi067.stderr: 82650K .......... .......... .......... .......... .......... 60% 133M 2s 2024-03-30T02:38:35.789 INFO:tasks.workunit.client.0.smithi067.stderr: 82700K .......... .......... .......... .......... .......... 60% 14.3M 2s 2024-03-30T02:38:35.789 INFO:tasks.workunit.client.0.smithi067.stderr: 82750K .......... .......... .......... .......... .......... 60% 112M 2s 2024-03-30T02:38:35.789 INFO:tasks.workunit.client.0.smithi067.stderr: 82800K .......... .......... .......... .......... .......... 61% 19.0M 2s 2024-03-30T02:38:35.789 INFO:tasks.workunit.client.0.smithi067.stderr: 82850K .......... .......... .......... .......... .......... 61% 115M 2s 2024-03-30T02:38:35.789 INFO:tasks.workunit.client.0.smithi067.stderr: 82900K .......... .......... .......... .......... .......... 61% 111M 2s 2024-03-30T02:38:35.789 INFO:tasks.workunit.client.0.smithi067.stderr: 82950K .......... .......... .......... .......... .......... 61% 15.3M 2s 2024-03-30T02:38:35.789 INFO:tasks.workunit.client.0.smithi067.stderr: 83000K .......... .......... .......... .......... .......... 61% 117M 2s 2024-03-30T02:38:35.797 INFO:tasks.workunit.client.0.smithi067.stderr: 83050K .......... .......... .......... .......... .......... 61% 6.23M 2s 2024-03-30T02:38:35.797 INFO:tasks.workunit.client.0.smithi067.stderr: 83100K .......... .......... .......... .......... .......... 61% 145M 2s 2024-03-30T02:38:35.797 INFO:tasks.workunit.client.0.smithi067.stderr: 83150K .......... .......... .......... .......... .......... 61% 152M 2s 2024-03-30T02:38:35.801 INFO:tasks.workunit.client.0.smithi067.stderr: 83200K .......... .......... .......... .......... .......... 61% 12.6M 2s 2024-03-30T02:38:35.801 INFO:tasks.workunit.client.0.smithi067.stderr: 83250K .......... .......... .......... .......... .......... 61% 164M 2s 2024-03-30T02:38:35.801 INFO:tasks.workunit.client.0.smithi067.stderr: 83300K .......... .......... .......... .......... .......... 61% 110M 2s 2024-03-30T02:38:35.810 INFO:tasks.workunit.client.0.smithi067.stderr: 83350K .......... .......... .......... .......... .......... 61% 9.38M 2s 2024-03-30T02:38:35.811 INFO:tasks.workunit.client.0.smithi067.stderr: 83400K .......... .......... .......... .......... .......... 61% 105M 2s 2024-03-30T02:38:35.811 INFO:tasks.workunit.client.0.smithi067.stderr: 83450K .......... .......... .......... .......... .......... 61% 21.8M 2s 2024-03-30T02:38:35.811 INFO:tasks.workunit.client.0.smithi067.stderr: 83500K .......... .......... .......... .......... .......... 61% 86.0M 2s 2024-03-30T02:38:35.811 INFO:tasks.workunit.client.0.smithi067.stderr: 83550K .......... .......... .......... .......... .......... 61% 121M 2s 2024-03-30T02:38:35.848 INFO:tasks.workunit.client.0.smithi067.stderr: 83600K .......... .......... .......... .......... .......... 61% 5.89M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 83650K .......... .......... .......... .......... .......... 61% 109M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 83700K .......... .......... .......... .......... .......... 61% 113M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 83750K .......... .......... .......... .......... .......... 61% 111M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 83800K .......... .......... .......... .......... .......... 61% 77.3M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 83850K .......... .......... .......... .......... .......... 61% 57.1M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 83900K .......... .......... .......... .......... .......... 61% 50.0M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 83950K .......... .......... .......... .......... .......... 61% 23.3M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 84000K .......... .......... .......... .......... .......... 61% 78.3M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 84050K .......... .......... .......... .......... .......... 61% 67.5M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 84100K .......... .......... .......... .......... .......... 61% 24.9M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 84150K .......... .......... .......... .......... .......... 62% 24.2M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 84200K .......... .......... .......... .......... .......... 62% 110M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 84250K .......... .......... .......... .......... .......... 62% 21.4M 2s 2024-03-30T02:38:35.849 INFO:tasks.workunit.client.0.smithi067.stderr: 84300K .......... .......... .......... .......... .......... 62% 37.0M 2s 2024-03-30T02:38:35.850 INFO:tasks.workunit.client.0.smithi067.stderr: 84350K .......... .......... .......... .......... .......... 62% 19.5M 2s 2024-03-30T02:38:35.850 INFO:tasks.workunit.client.0.smithi067.stderr: 84400K .......... .......... .......... .......... .......... 62% 24.4M 2s 2024-03-30T02:38:35.850 INFO:tasks.workunit.client.0.smithi067.stderr: 84450K .......... .......... .......... .......... .......... 62% 114M 2s 2024-03-30T02:38:35.850 INFO:tasks.workunit.client.0.smithi067.stderr: 84500K .......... .......... .......... .......... .......... 62% 11.8M 2s 2024-03-30T02:38:35.850 INFO:tasks.workunit.client.0.smithi067.stderr: 84550K .......... .......... .......... .......... .......... 62% 24.5M 2s 2024-03-30T02:38:35.850 INFO:tasks.workunit.client.0.smithi067.stderr: 84600K .......... .......... .......... .......... .......... 62% 24.1M 2s 2024-03-30T02:38:35.850 INFO:tasks.workunit.client.0.smithi067.stderr: 84650K .......... .......... .......... .......... .......... 62% 23.7M 2s 2024-03-30T02:38:35.850 INFO:tasks.workunit.client.0.smithi067.stderr: 84700K .......... .......... .......... .......... .......... 62% 126M 2s 2024-03-30T02:38:35.859 INFO:tasks.workunit.client.0.smithi067.stderr: 84750K .......... .......... .......... .......... .......... 62% 23.5M 2s 2024-03-30T02:38:35.859 INFO:tasks.workunit.client.0.smithi067.stderr: 84800K .......... .......... .......... .......... .......... 62% 24.4M 2s 2024-03-30T02:38:35.859 INFO:tasks.workunit.client.0.smithi067.stderr: 84850K .......... .......... .......... .......... .......... 62% 38.0M 2s 2024-03-30T02:38:35.859 INFO:tasks.workunit.client.0.smithi067.stderr: 84900K .......... .......... .......... .......... .......... 62% 23.9M 2s 2024-03-30T02:38:35.859 INFO:tasks.workunit.client.0.smithi067.stderr: 84950K .......... .......... .......... .......... .......... 62% 48.1M 2s 2024-03-30T02:38:35.859 INFO:tasks.workunit.client.0.smithi067.stderr: 85000K .......... .......... .......... .......... .......... 62% 21.9M 2s 2024-03-30T02:38:35.861 INFO:tasks.workunit.client.0.smithi067.stderr: 85050K .......... .......... .......... .......... .......... 62% 22.7M 2s 2024-03-30T02:38:35.863 INFO:tasks.workunit.client.0.smithi067.stderr: 85100K .......... .......... .......... .......... .......... 62% 24.4M 2s 2024-03-30T02:38:35.863 INFO:tasks.workunit.client.0.smithi067.stderr: 85150K .......... .......... .......... .......... .......... 62% 108M 2s 2024-03-30T02:38:35.866 INFO:tasks.workunit.client.0.smithi067.stderr: 85200K .......... .......... .......... .......... .......... 62% 23.0M 2s 2024-03-30T02:38:35.870 INFO:tasks.workunit.client.0.smithi067.stderr: 85250K .......... .......... .......... .......... .......... 62% 24.2M 2s 2024-03-30T02:38:35.870 INFO:tasks.workunit.client.0.smithi067.stderr: 85300K .......... .......... .......... .......... .......... 62% 24.0M 2s 2024-03-30T02:38:35.871 INFO:tasks.workunit.client.0.smithi067.stderr: 85350K .......... .......... .......... .......... .......... 62% 118M 2s 2024-03-30T02:38:35.923 INFO:tasks.workunit.client.0.smithi067.stderr: 85400K .......... .......... .......... .......... .......... 62% 2.73M 2s 2024-03-30T02:38:35.923 INFO:tasks.workunit.client.0.smithi067.stderr: 85450K .......... .......... .......... .......... .......... 62% 115M 2s 2024-03-30T02:38:35.923 INFO:tasks.workunit.client.0.smithi067.stderr: 85500K .......... .......... .......... .......... .......... 63% 115M 2s 2024-03-30T02:38:35.923 INFO:tasks.workunit.client.0.smithi067.stderr: 85550K .......... .......... .......... .......... .......... 63% 108M 2s 2024-03-30T02:38:35.923 INFO:tasks.workunit.client.0.smithi067.stderr: 85600K .......... .......... .......... .......... .......... 63% 111M 2s 2024-03-30T02:38:35.923 INFO:tasks.workunit.client.0.smithi067.stderr: 85650K .......... .......... .......... .......... .......... 63% 18.7M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 85700K .......... .......... .......... .......... .......... 63% 116M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 85750K .......... .......... .......... .......... .......... 63% 46.2M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 85800K .......... .......... .......... .......... .......... 63% 30.9M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 85850K .......... .......... .......... .......... .......... 63% 119M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 85900K .......... .......... .......... .......... .......... 63% 28.5M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 85950K .......... .......... .......... .......... .......... 63% 23.2M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86000K .......... .......... .......... .......... .......... 63% 26.4M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86050K .......... .......... .......... .......... .......... 63% 65.5M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86100K .......... .......... .......... .......... .......... 63% 34.2M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86150K .......... .......... .......... .......... .......... 63% 23.5M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86200K .......... .......... .......... .......... .......... 63% 20.5M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86250K .......... .......... .......... .......... .......... 63% 26.3M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86300K .......... .......... .......... .......... .......... 63% 112M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86350K .......... .......... .......... .......... .......... 63% 24.6M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86400K .......... .......... .......... .......... .......... 63% 24.7M 2s 2024-03-30T02:38:35.924 INFO:tasks.workunit.client.0.smithi067.stderr: 86450K .......... .......... .......... .......... .......... 63% 24.4M 2s 2024-03-30T02:38:35.925 INFO:tasks.workunit.client.0.smithi067.stderr: 86500K .......... .......... .......... .......... .......... 63% 112M 2s 2024-03-30T02:38:35.925 INFO:tasks.workunit.client.0.smithi067.stderr: 86550K .......... .......... .......... .......... .......... 63% 24.2M 2s 2024-03-30T02:38:35.925 INFO:tasks.workunit.client.0.smithi067.stderr: 86600K .......... .......... .......... .......... .......... 63% 23.0M 2s 2024-03-30T02:38:35.925 INFO:tasks.workunit.client.0.smithi067.stderr: 86650K .......... .......... .......... .......... .......... 63% 24.8M 2s 2024-03-30T02:38:35.934 INFO:tasks.workunit.client.0.smithi067.stderr: 86700K .......... .......... .......... .......... .......... 63% 24.8M 2s 2024-03-30T02:38:35.934 INFO:tasks.workunit.client.0.smithi067.stderr: 86750K .......... .......... .......... .......... .......... 63% 122M 2s 2024-03-30T02:38:35.934 INFO:tasks.workunit.client.0.smithi067.stderr: 86800K .......... .......... .......... .......... .......... 63% 23.5M 2s 2024-03-30T02:38:35.934 INFO:tasks.workunit.client.0.smithi067.stderr: 86850K .......... .......... .......... .......... .......... 64% 23.3M 2s 2024-03-30T02:38:35.934 INFO:tasks.workunit.client.0.smithi067.stderr: 86900K .......... .......... .......... .......... .......... 64% 25.3M 2s 2024-03-30T02:38:35.934 INFO:tasks.workunit.client.0.smithi067.stderr: 86950K .......... .......... .......... .......... .......... 64% 112M 2s 2024-03-30T02:38:35.934 INFO:tasks.workunit.client.0.smithi067.stderr: 87000K .......... .......... .......... .......... .......... 64% 22.5M 2s 2024-03-30T02:38:35.936 INFO:tasks.workunit.client.0.smithi067.stderr: 87050K .......... .......... .......... .......... .......... 64% 25.9M 2s 2024-03-30T02:38:35.938 INFO:tasks.workunit.client.0.smithi067.stderr: 87100K .......... .......... .......... .......... .......... 64% 24.1M 2s 2024-03-30T02:38:35.949 INFO:tasks.workunit.client.0.smithi067.stderr: 87150K .......... .......... .......... .......... .......... 64% 8.17M 2s 2024-03-30T02:38:35.949 INFO:tasks.workunit.client.0.smithi067.stderr: 87200K .......... .......... .......... .......... .......... 64% 137M 2s 2024-03-30T02:38:35.949 INFO:tasks.workunit.client.0.smithi067.stderr: 87250K .......... .......... .......... .......... .......... 64% 115M 2s 2024-03-30T02:38:35.949 INFO:tasks.workunit.client.0.smithi067.stderr: 87300K .......... .......... .......... .......... .......... 64% 27.6M 2s 2024-03-30T02:38:35.950 INFO:tasks.workunit.client.0.smithi067.stderr: 87350K .......... .......... .......... .......... .......... 64% 24.4M 2s 2024-03-30T02:38:35.950 INFO:tasks.workunit.client.0.smithi067.stderr: 87400K .......... .......... .......... .......... .......... 64% 105M 2s 2024-03-30T02:38:35.951 INFO:tasks.workunit.client.0.smithi067.stderr: 87450K .......... .......... .......... .......... .......... 64% 20.6M 2s 2024-03-30T02:38:35.953 INFO:tasks.workunit.client.0.smithi067.stderr: 87500K .......... .......... .......... .......... .......... 64% 28.1M 2s 2024-03-30T02:38:35.956 INFO:tasks.workunit.client.0.smithi067.stderr: 87550K .......... .......... .......... .......... .......... 64% 23.4M 2s 2024-03-30T02:38:35.958 INFO:tasks.workunit.client.0.smithi067.stderr: 87600K .......... .......... .......... .......... .......... 64% 24.5M 2s 2024-03-30T02:38:35.958 INFO:tasks.workunit.client.0.smithi067.stderr: 87650K .......... .......... .......... .......... .......... 64% 113M 2s 2024-03-30T02:38:35.960 INFO:tasks.workunit.client.0.smithi067.stderr: 87700K .......... .......... .......... .......... .......... 64% 22.5M 2s 2024-03-30T02:38:35.964 INFO:tasks.workunit.client.0.smithi067.stderr: 87750K .......... .......... .......... .......... .......... 64% 25.6M 2s 2024-03-30T02:38:35.964 INFO:tasks.workunit.client.0.smithi067.stderr: 87800K .......... .......... .......... .......... .......... 64% 21.8M 2s 2024-03-30T02:38:35.966 INFO:tasks.workunit.client.0.smithi067.stderr: 87850K .......... .......... .......... .......... .......... 64% 24.3M 2s 2024-03-30T02:38:35.966 INFO:tasks.workunit.client.0.smithi067.stderr: 87900K .......... .......... .......... .......... .......... 64% 128M 2s 2024-03-30T02:38:35.968 INFO:tasks.workunit.client.0.smithi067.stderr: 87950K .......... .......... .......... .......... .......... 64% 26.0M 2s 2024-03-30T02:38:35.971 INFO:tasks.workunit.client.0.smithi067.stderr: 88000K .......... .......... .......... .......... .......... 64% 21.7M 2s 2024-03-30T02:38:35.972 INFO:tasks.workunit.client.0.smithi067.stderr: 88050K .......... .......... .......... .......... .......... 64% 25.0M 2s 2024-03-30T02:38:35.972 INFO:tasks.workunit.client.0.smithi067.stderr: 88100K .......... .......... .......... .......... .......... 64% 164M 2s 2024-03-30T02:38:35.974 INFO:tasks.workunit.client.0.smithi067.stderr: 88150K .......... .......... .......... .......... .......... 64% 24.8M 2s 2024-03-30T02:38:35.977 INFO:tasks.workunit.client.0.smithi067.stderr: 88200K .......... .......... .......... .......... .......... 65% 22.9M 2s 2024-03-30T02:38:35.979 INFO:tasks.workunit.client.0.smithi067.stderr: 88250K .......... .......... .......... .......... .......... 65% 22.7M 2s 2024-03-30T02:38:35.981 INFO:tasks.workunit.client.0.smithi067.stderr: 88300K .......... .......... .......... .......... .......... 65% 23.5M 2s 2024-03-30T02:38:35.981 INFO:tasks.workunit.client.0.smithi067.stderr: 88350K .......... .......... .......... .......... .......... 65% 114M 2s 2024-03-30T02:38:35.983 INFO:tasks.workunit.client.0.smithi067.stderr: 88400K .......... .......... .......... .......... .......... 65% 31.1M 2s 2024-03-30T02:38:35.985 INFO:tasks.workunit.client.0.smithi067.stderr: 88450K .......... .......... .......... .......... .......... 65% 22.4M 2s 2024-03-30T02:38:35.987 INFO:tasks.workunit.client.0.smithi067.stderr: 88500K .......... .......... .......... .......... .......... 65% 24.1M 2s 2024-03-30T02:38:35.987 INFO:tasks.workunit.client.0.smithi067.stderr: 88550K .......... .......... .......... .......... .......... 65% 116M 2s 2024-03-30T02:38:35.989 INFO:tasks.workunit.client.0.smithi067.stderr: 88600K .......... .......... .......... .......... .......... 65% 20.4M 2s 2024-03-30T02:38:35.992 INFO:tasks.workunit.client.0.smithi067.stderr: 88650K .......... .......... .......... .......... .......... 65% 23.1M 2s 2024-03-30T02:38:35.994 INFO:tasks.workunit.client.0.smithi067.stderr: 88700K .......... .......... .......... .......... .......... 65% 27.1M 2s 2024-03-30T02:38:35.996 INFO:tasks.workunit.client.0.smithi067.stderr: 88750K .......... .......... .......... .......... .......... 65% 24.3M 2s 2024-03-30T02:38:35.996 INFO:tasks.workunit.client.0.smithi067.stderr: 88800K .......... .......... .......... .......... .......... 65% 125M 2s 2024-03-30T02:38:35.998 INFO:tasks.workunit.client.0.smithi067.stderr: 88850K .......... .......... .......... .......... .......... 65% 24.2M 2s 2024-03-30T02:38:36.000 INFO:tasks.workunit.client.0.smithi067.stderr: 88900K .......... .......... .......... .......... .......... 65% 24.7M 2s 2024-03-30T02:38:36.002 INFO:tasks.workunit.client.0.smithi067.stderr: 88950K .......... .......... .......... .......... .......... 65% 21.1M 2s 2024-03-30T02:38:36.004 INFO:tasks.workunit.client.0.smithi067.stderr: 89000K .......... .......... .......... .......... .......... 65% 27.0M 2s 2024-03-30T02:38:36.005 INFO:tasks.workunit.client.0.smithi067.stderr: 89050K .......... .......... .......... .......... .......... 65% 124M 2s 2024-03-30T02:38:36.007 INFO:tasks.workunit.client.0.smithi067.stderr: 89100K .......... .......... .......... .......... .......... 65% 24.1M 2s 2024-03-30T02:38:36.009 INFO:tasks.workunit.client.0.smithi067.stderr: 89150K .......... .......... .......... .......... .......... 65% 21.9M 2s 2024-03-30T02:38:36.011 INFO:tasks.workunit.client.0.smithi067.stderr: 89200K .......... .......... .......... .......... .......... 65% 26.1M 2s 2024-03-30T02:38:36.011 INFO:tasks.workunit.client.0.smithi067.stderr: 89250K .......... .......... .......... .......... .......... 65% 121M 2s 2024-03-30T02:38:36.013 INFO:tasks.workunit.client.0.smithi067.stderr: 89300K .......... .......... .......... .......... .......... 65% 23.8M 2s 2024-03-30T02:38:36.016 INFO:tasks.workunit.client.0.smithi067.stderr: 89350K .......... .......... .......... .......... .......... 65% 24.6M 2s 2024-03-30T02:38:36.017 INFO:tasks.workunit.client.0.smithi067.stderr: 89400K .......... .......... .......... .......... .......... 65% 24.5M 2s 2024-03-30T02:38:36.019 INFO:tasks.workunit.client.0.smithi067.stderr: 89450K .......... .......... .......... .......... .......... 65% 23.8M 2s 2024-03-30T02:38:36.019 INFO:tasks.workunit.client.0.smithi067.stderr: 89500K .......... .......... .......... .......... .......... 65% 121M 2s 2024-03-30T02:38:36.022 INFO:tasks.workunit.client.0.smithi067.stderr: 89550K .......... .......... .......... .......... .......... 66% 25.3M 2s 2024-03-30T02:38:36.024 INFO:tasks.workunit.client.0.smithi067.stderr: 89600K .......... .......... .......... .......... .......... 66% 22.8M 2s 2024-03-30T02:38:36.026 INFO:tasks.workunit.client.0.smithi067.stderr: 89650K .......... .......... .......... .......... .......... 66% 24.3M 2s 2024-03-30T02:38:36.027 INFO:tasks.workunit.client.0.smithi067.stderr: 89700K .......... .......... .......... .......... .......... 66% 89.3M 2s 2024-03-30T02:38:36.028 INFO:tasks.workunit.client.0.smithi067.stderr: 89750K .......... .......... .......... .......... .......... 66% 25.2M 2s 2024-03-30T02:38:36.030 INFO:tasks.workunit.client.0.smithi067.stderr: 89800K .......... .......... .......... .......... .......... 66% 24.6M 2s 2024-03-30T02:38:36.034 INFO:tasks.workunit.client.0.smithi067.stderr: 89850K .......... .......... .......... .......... .......... 66% 11.9M 2s 2024-03-30T02:38:36.037 INFO:tasks.workunit.client.0.smithi067.stderr: 89900K .......... .......... .......... .......... .......... 66% 24.5M 2s 2024-03-30T02:38:36.037 INFO:tasks.workunit.client.0.smithi067.stderr: 89950K .......... .......... .......... .......... .......... 66% 119M 2s 2024-03-30T02:38:36.039 INFO:tasks.workunit.client.0.smithi067.stderr: 90000K .......... .......... .......... .......... .......... 66% 21.7M 2s 2024-03-30T02:38:36.041 INFO:tasks.workunit.client.0.smithi067.stderr: 90050K .......... .......... .......... .......... .......... 66% 27.7M 2s 2024-03-30T02:38:36.043 INFO:tasks.workunit.client.0.smithi067.stderr: 90100K .......... .......... .......... .......... .......... 66% 23.2M 2s 2024-03-30T02:38:36.045 INFO:tasks.workunit.client.0.smithi067.stderr: 90150K .......... .......... .......... .......... .......... 66% 112M 2s 2024-03-30T02:38:36.045 INFO:tasks.workunit.client.0.smithi067.stderr: 90200K .......... .......... .......... .......... .......... 66% 25.3M 2s 2024-03-30T02:38:36.047 INFO:tasks.workunit.client.0.smithi067.stderr: 90250K .......... .......... .......... .......... .......... 66% 22.3M 2s 2024-03-30T02:38:36.049 INFO:tasks.workunit.client.0.smithi067.stderr: 90300K .......... .......... .......... .......... .......... 66% 24.9M 2s 2024-03-30T02:38:36.052 INFO:tasks.workunit.client.0.smithi067.stderr: 90350K .......... .......... .......... .......... .......... 66% 22.0M 2s 2024-03-30T02:38:36.052 INFO:tasks.workunit.client.0.smithi067.stderr: 90400K .......... .......... .......... .......... .......... 66% 100M 2s 2024-03-30T02:38:36.054 INFO:tasks.workunit.client.0.smithi067.stderr: 90450K .......... .......... .......... .......... .......... 66% 26.7M 2s 2024-03-30T02:38:36.056 INFO:tasks.workunit.client.0.smithi067.stderr: 90500K .......... .......... .......... .......... .......... 66% 23.5M 2s 2024-03-30T02:38:36.058 INFO:tasks.workunit.client.0.smithi067.stderr: 90550K .......... .......... .......... .......... .......... 66% 22.5M 2s 2024-03-30T02:38:36.060 INFO:tasks.workunit.client.0.smithi067.stderr: 90600K .......... .......... .......... .......... .......... 66% 166M 2s 2024-03-30T02:38:36.060 INFO:tasks.workunit.client.0.smithi067.stderr: 90650K .......... .......... .......... .......... .......... 66% 24.9M 2s 2024-03-30T02:38:36.062 INFO:tasks.workunit.client.0.smithi067.stderr: 90700K .......... .......... .......... .......... .......... 66% 25.8M 2s 2024-03-30T02:38:36.064 INFO:tasks.workunit.client.0.smithi067.stderr: 90750K .......... .......... .......... .......... .......... 66% 23.3M 2s 2024-03-30T02:38:36.067 INFO:tasks.workunit.client.0.smithi067.stderr: 90800K .......... .......... .......... .......... .......... 66% 27.2M 2s 2024-03-30T02:38:36.067 INFO:tasks.workunit.client.0.smithi067.stderr: 90850K .......... .......... .......... .......... .......... 66% 105M 2s 2024-03-30T02:38:36.068 INFO:tasks.workunit.client.0.smithi067.stderr: 90900K .......... .......... .......... .......... .......... 66% 22.7M 2s 2024-03-30T02:38:36.080 INFO:tasks.workunit.client.0.smithi067.stderr: 90950K .......... .......... .......... .......... .......... 67% 27.0M 2s 2024-03-30T02:38:36.080 INFO:tasks.workunit.client.0.smithi067.stderr: 91000K .......... .......... .......... .......... .......... 67% 19.6M 2s 2024-03-30T02:38:36.080 INFO:tasks.workunit.client.0.smithi067.stderr: 91050K .......... .......... .......... .......... .......... 67% 25.0M 2s 2024-03-30T02:38:36.080 INFO:tasks.workunit.client.0.smithi067.stderr: 91100K .......... .......... .......... .......... .......... 67% 110M 2s 2024-03-30T02:38:36.080 INFO:tasks.workunit.client.0.smithi067.stderr: 91150K .......... .......... .......... .......... .......... 67% 23.3M 2s 2024-03-30T02:38:36.080 INFO:tasks.workunit.client.0.smithi067.stderr: 91200K .......... .......... .......... .......... .......... 67% 25.0M 2s 2024-03-30T02:38:36.088 INFO:tasks.workunit.client.0.smithi067.stderr: 91250K .......... .......... .......... .......... .......... 67% 10.5M 2s 2024-03-30T02:38:36.089 INFO:tasks.workunit.client.0.smithi067.stderr: 91300K .......... .......... .......... .......... .......... 67% 64.3M 2s 2024-03-30T02:38:36.089 INFO:tasks.workunit.client.0.smithi067.stderr: 91350K .......... .......... .......... .......... .......... 67% 124M 2s 2024-03-30T02:38:36.089 INFO:tasks.workunit.client.0.smithi067.stderr: 91400K .......... .......... .......... .......... .......... 67% 69.5M 2s 2024-03-30T02:38:36.089 INFO:tasks.workunit.client.0.smithi067.stderr: 91450K .......... .......... .......... .......... .......... 67% 24.7M 2s 2024-03-30T02:38:36.091 INFO:tasks.workunit.client.0.smithi067.stderr: 91500K .......... .......... .......... .......... .......... 67% 23.7M 2s 2024-03-30T02:38:36.091 INFO:tasks.workunit.client.0.smithi067.stderr: 91550K .......... .......... .......... .......... .......... 67% 106M 2s 2024-03-30T02:38:36.092 INFO:tasks.workunit.client.0.smithi067.stderr: 91600K .......... .......... .......... .......... .......... 67% 22.6M 2s 2024-03-30T02:38:36.095 INFO:tasks.workunit.client.0.smithi067.stderr: 91650K .......... .......... .......... .......... .......... 67% 26.2M 2s 2024-03-30T02:38:36.097 INFO:tasks.workunit.client.0.smithi067.stderr: 91700K .......... .......... .......... .......... .......... 67% 24.5M 2s 2024-03-30T02:38:36.097 INFO:tasks.workunit.client.0.smithi067.stderr: 91750K .......... .......... .......... .......... .......... 67% 117M 2s 2024-03-30T02:38:36.099 INFO:tasks.workunit.client.0.smithi067.stderr: 91800K .......... .......... .......... .......... .......... 67% 23.5M 2s 2024-03-30T02:38:36.101 INFO:tasks.workunit.client.0.smithi067.stderr: 91850K .......... .......... .......... .......... .......... 67% 24.3M 2s 2024-03-30T02:38:36.104 INFO:tasks.workunit.client.0.smithi067.stderr: 91900K .......... .......... .......... .......... .......... 67% 23.7M 2s 2024-03-30T02:38:36.106 INFO:tasks.workunit.client.0.smithi067.stderr: 91950K .......... .......... .......... .......... .......... 67% 24.5M 2s 2024-03-30T02:38:36.106 INFO:tasks.workunit.client.0.smithi067.stderr: 92000K .......... .......... .......... .......... .......... 67% 111M 2s 2024-03-30T02:38:36.108 INFO:tasks.workunit.client.0.smithi067.stderr: 92050K .......... .......... .......... .......... .......... 67% 24.5M 2s 2024-03-30T02:38:36.109 INFO:tasks.workunit.client.0.smithi067.stderr: 92100K .......... .......... .......... .......... .......... 67% 23.5M 2s 2024-03-30T02:38:36.112 INFO:tasks.workunit.client.0.smithi067.stderr: 92150K .......... .......... .......... .......... .......... 67% 25.0M 2s 2024-03-30T02:38:36.112 INFO:tasks.workunit.client.0.smithi067.stderr: 92200K .......... .......... .......... .......... .......... 67% 114M 2s 2024-03-30T02:38:36.114 INFO:tasks.workunit.client.0.smithi067.stderr: 92250K .......... .......... .......... .......... .......... 67% 24.0M 2s 2024-03-30T02:38:36.116 INFO:tasks.workunit.client.0.smithi067.stderr: 92300K .......... .......... .......... .......... .......... 68% 23.9M 2s 2024-03-30T02:38:36.118 INFO:tasks.workunit.client.0.smithi067.stderr: 92350K .......... .......... .......... .......... .......... 68% 24.2M 2s 2024-03-30T02:38:36.120 INFO:tasks.workunit.client.0.smithi067.stderr: 92400K .......... .......... .......... .......... .......... 68% 24.1M 2s 2024-03-30T02:38:36.120 INFO:tasks.workunit.client.0.smithi067.stderr: 92450K .......... .......... .......... .......... .......... 68% 107M 2s 2024-03-30T02:38:36.123 INFO:tasks.workunit.client.0.smithi067.stderr: 92500K .......... .......... .......... .......... .......... 68% 22.4M 2s 2024-03-30T02:38:36.125 INFO:tasks.workunit.client.0.smithi067.stderr: 92550K .......... .......... .......... .......... .......... 68% 20.5M 2s 2024-03-30T02:38:36.127 INFO:tasks.workunit.client.0.smithi067.stderr: 92600K .......... .......... .......... .......... .......... 68% 24.8M 2s 2024-03-30T02:38:36.130 INFO:tasks.workunit.client.0.smithi067.stderr: 92650K .......... .......... .......... .......... .......... 68% 20.1M 2s 2024-03-30T02:38:36.130 INFO:tasks.workunit.client.0.smithi067.stderr: 92700K .......... .......... .......... .......... .......... 68% 104M 2s 2024-03-30T02:38:36.134 INFO:tasks.workunit.client.0.smithi067.stderr: 92750K .......... .......... .......... .......... .......... 68% 16.6M 2s 2024-03-30T02:38:36.134 INFO:tasks.workunit.client.0.smithi067.stderr: 92800K .......... .......... .......... .......... .......... 68% 65.0M 2s 2024-03-30T02:38:36.138 INFO:tasks.workunit.client.0.smithi067.stderr: 92850K .......... .......... .......... .......... .......... 68% 12.1M 2s 2024-03-30T02:38:36.138 INFO:tasks.workunit.client.0.smithi067.stderr: 92900K .......... .......... .......... .......... .......... 68% 108M 2s 2024-03-30T02:38:36.140 INFO:tasks.workunit.client.0.smithi067.stderr: 92950K .......... .......... .......... .......... .......... 68% 24.4M 2s 2024-03-30T02:38:36.142 INFO:tasks.workunit.client.0.smithi067.stderr: 93000K .......... .......... .......... .......... .......... 68% 24.3M 2s 2024-03-30T02:38:36.144 INFO:tasks.workunit.client.0.smithi067.stderr: 93050K .......... .......... .......... .......... .......... 68% 20.7M 2s 2024-03-30T02:38:36.147 INFO:tasks.workunit.client.0.smithi067.stderr: 93100K .......... .......... .......... .......... .......... 68% 28.6M 2s 2024-03-30T02:38:36.147 INFO:tasks.workunit.client.0.smithi067.stderr: 93150K .......... .......... .......... .......... .......... 68% 106M 2s 2024-03-30T02:38:36.149 INFO:tasks.workunit.client.0.smithi067.stderr: 93200K .......... .......... .......... .......... .......... 68% 25.1M 2s 2024-03-30T02:38:36.151 INFO:tasks.workunit.client.0.smithi067.stderr: 93250K .......... .......... .......... .......... .......... 68% 21.2M 2s 2024-03-30T02:38:36.153 INFO:tasks.workunit.client.0.smithi067.stderr: 93300K .......... .......... .......... .......... .......... 68% 24.1M 2s 2024-03-30T02:38:36.153 INFO:tasks.workunit.client.0.smithi067.stderr: 93350K .......... .......... .......... .......... .......... 68% 120M 2s 2024-03-30T02:38:36.156 INFO:tasks.workunit.client.0.smithi067.stderr: 93400K .......... .......... .......... .......... .......... 68% 23.7M 2s 2024-03-30T02:38:36.158 INFO:tasks.workunit.client.0.smithi067.stderr: 93450K .......... .......... .......... .......... .......... 68% 24.4M 2s 2024-03-30T02:38:36.159 INFO:tasks.workunit.client.0.smithi067.stderr: 93500K .......... .......... .......... .......... .......... 68% 22.7M 2s 2024-03-30T02:38:36.162 INFO:tasks.workunit.client.0.smithi067.stderr: 93550K .......... .......... .......... .......... .......... 68% 26.0M 2s 2024-03-30T02:38:36.162 INFO:tasks.workunit.client.0.smithi067.stderr: 93600K .......... .......... .......... .......... .......... 68% 107M 2s 2024-03-30T02:38:36.164 INFO:tasks.workunit.client.0.smithi067.stderr: 93650K .......... .......... .......... .......... .......... 69% 24.6M 2s 2024-03-30T02:38:36.166 INFO:tasks.workunit.client.0.smithi067.stderr: 93700K .......... .......... .......... .......... .......... 69% 23.1M 2s 2024-03-30T02:38:36.168 INFO:tasks.workunit.client.0.smithi067.stderr: 93750K .......... .......... .......... .......... .......... 69% 22.9M 2s 2024-03-30T02:38:36.168 INFO:tasks.workunit.client.0.smithi067.stderr: 93800K .......... .......... .......... .......... .......... 69% 139M 2s 2024-03-30T02:38:36.171 INFO:tasks.workunit.client.0.smithi067.stderr: 93850K .......... .......... .......... .......... .......... 69% 25.1M 2s 2024-03-30T02:38:36.175 INFO:tasks.workunit.client.0.smithi067.stderr: 93900K .......... .......... .......... .......... .......... 69% 24.5M 2s 2024-03-30T02:38:36.175 INFO:tasks.workunit.client.0.smithi067.stderr: 93950K .......... .......... .......... .......... .......... 69% 23.1M 2s 2024-03-30T02:38:36.177 INFO:tasks.workunit.client.0.smithi067.stderr: 94000K .......... .......... .......... .......... .......... 69% 23.2M 2s 2024-03-30T02:38:36.177 INFO:tasks.workunit.client.0.smithi067.stderr: 94050K .......... .......... .......... .......... .......... 69% 114M 2s 2024-03-30T02:38:36.179 INFO:tasks.workunit.client.0.smithi067.stderr: 94100K .......... .......... .......... .......... .......... 69% 24.3M 2s 2024-03-30T02:38:36.181 INFO:tasks.workunit.client.0.smithi067.stderr: 94150K .......... .......... .......... .......... .......... 69% 24.2M 2s 2024-03-30T02:38:36.183 INFO:tasks.workunit.client.0.smithi067.stderr: 94200K .......... .......... .......... .......... .......... 69% 24.1M 2s 2024-03-30T02:38:36.185 INFO:tasks.workunit.client.0.smithi067.stderr: 94250K .......... .......... .......... .......... .......... 69% 28.7M 2s 2024-03-30T02:38:36.186 INFO:tasks.workunit.client.0.smithi067.stderr: 94300K .......... .......... .......... .......... .......... 69% 109M 2s 2024-03-30T02:38:36.187 INFO:tasks.workunit.client.0.smithi067.stderr: 94350K .......... .......... .......... .......... .......... 69% 21.7M 2s 2024-03-30T02:38:36.190 INFO:tasks.workunit.client.0.smithi067.stderr: 94400K .......... .......... .......... .......... .......... 69% 28.3M 2s 2024-03-30T02:38:36.192 INFO:tasks.workunit.client.0.smithi067.stderr: 94450K .......... .......... .......... .......... .......... 69% 21.6M 2s 2024-03-30T02:38:36.192 INFO:tasks.workunit.client.0.smithi067.stderr: 94500K .......... .......... .......... .......... .......... 69% 114M 2s 2024-03-30T02:38:36.194 INFO:tasks.workunit.client.0.smithi067.stderr: 94550K .......... .......... .......... .......... .......... 69% 24.8M 2s 2024-03-30T02:38:36.196 INFO:tasks.workunit.client.0.smithi067.stderr: 94600K .......... .......... .......... .......... .......... 69% 23.2M 2s 2024-03-30T02:38:36.198 INFO:tasks.workunit.client.0.smithi067.stderr: 94650K .......... .......... .......... .......... .......... 69% 25.8M 2s 2024-03-30T02:38:36.200 INFO:tasks.workunit.client.0.smithi067.stderr: 94700K .......... .......... .......... .......... .......... 69% 23.0M 2s 2024-03-30T02:38:36.200 INFO:tasks.workunit.client.0.smithi067.stderr: 94750K .......... .......... .......... .......... .......... 69% 114M 2s 2024-03-30T02:38:36.203 INFO:tasks.workunit.client.0.smithi067.stderr: 94800K .......... .......... .......... .......... .......... 69% 26.7M 2s 2024-03-30T02:38:36.204 INFO:tasks.workunit.client.0.smithi067.stderr: 94850K .......... .......... .......... .......... .......... 69% 21.9M 2s 2024-03-30T02:38:36.207 INFO:tasks.workunit.client.0.smithi067.stderr: 94900K .......... .......... .......... .......... .......... 69% 26.8M 2s 2024-03-30T02:38:36.207 INFO:tasks.workunit.client.0.smithi067.stderr: 94950K .......... .......... .......... .......... .......... 69% 112M 2s 2024-03-30T02:38:36.209 INFO:tasks.workunit.client.0.smithi067.stderr: 95000K .......... .......... .......... .......... .......... 70% 22.7M 2s 2024-03-30T02:38:36.211 INFO:tasks.workunit.client.0.smithi067.stderr: 95050K .......... .......... .......... .......... .......... 70% 27.0M 2s 2024-03-30T02:38:36.213 INFO:tasks.workunit.client.0.smithi067.stderr: 95100K .......... .......... .......... .......... .......... 70% 20.2M 2s 2024-03-30T02:38:36.216 INFO:tasks.workunit.client.0.smithi067.stderr: 95150K .......... .......... .......... .......... .......... 70% 26.6M 2s 2024-03-30T02:38:36.216 INFO:tasks.workunit.client.0.smithi067.stderr: 95200K .......... .......... .......... .......... .......... 70% 119M 2s 2024-03-30T02:38:36.218 INFO:tasks.workunit.client.0.smithi067.stderr: 95250K .......... .......... .......... .......... .......... 70% 22.3M 2s 2024-03-30T02:38:36.220 INFO:tasks.workunit.client.0.smithi067.stderr: 95300K .......... .......... .......... .......... .......... 70% 23.9M 2s 2024-03-30T02:38:36.222 INFO:tasks.workunit.client.0.smithi067.stderr: 95350K .......... .......... .......... .......... .......... 70% 24.9M 2s 2024-03-30T02:38:36.222 INFO:tasks.workunit.client.0.smithi067.stderr: 95400K .......... .......... .......... .......... .......... 70% 113M 2s 2024-03-30T02:38:36.224 INFO:tasks.workunit.client.0.smithi067.stderr: 95450K .......... .......... .......... .......... .......... 70% 24.1M 2s 2024-03-30T02:38:36.226 INFO:tasks.workunit.client.0.smithi067.stderr: 95500K .......... .......... .......... .......... .......... 70% 24.5M 2s 2024-03-30T02:38:36.228 INFO:tasks.workunit.client.0.smithi067.stderr: 95550K .......... .......... .......... .......... .......... 70% 27.4M 2s 2024-03-30T02:38:36.230 INFO:tasks.workunit.client.0.smithi067.stderr: 95600K .......... .......... .......... .......... .......... 70% 20.2M 2s 2024-03-30T02:38:36.233 INFO:tasks.workunit.client.0.smithi067.stderr: 95650K .......... .......... .......... .......... .......... 70% 26.9M 2s 2024-03-30T02:38:36.233 INFO:tasks.workunit.client.0.smithi067.stderr: 95700K .......... .......... .......... .......... .......... 70% 102M 2s 2024-03-30T02:38:36.235 INFO:tasks.workunit.client.0.smithi067.stderr: 95750K .......... .......... .......... .......... .......... 70% 20.8M 2s 2024-03-30T02:38:36.237 INFO:tasks.workunit.client.0.smithi067.stderr: 95800K .......... .......... .......... .......... .......... 70% 25.4M 2s 2024-03-30T02:38:36.239 INFO:tasks.workunit.client.0.smithi067.stderr: 95850K .......... .......... .......... .......... .......... 70% 26.3M 2s 2024-03-30T02:38:36.239 INFO:tasks.workunit.client.0.smithi067.stderr: 95900K .......... .......... .......... .......... .......... 70% 99.8M 2s 2024-03-30T02:38:36.243 INFO:tasks.workunit.client.0.smithi067.stderr: 95950K .......... .......... .......... .......... .......... 70% 11.1M 2s 2024-03-30T02:38:36.245 INFO:tasks.workunit.client.0.smithi067.stderr: 96000K .......... .......... .......... .......... .......... 70% 24.0M 2s 2024-03-30T02:38:36.248 INFO:tasks.workunit.client.0.smithi067.stderr: 96050K .......... .......... .......... .......... .......... 70% 23.7M 2s 2024-03-30T02:38:36.248 INFO:tasks.workunit.client.0.smithi067.stderr: 96100K .......... .......... .......... .......... .......... 70% 143M 2s 2024-03-30T02:38:36.250 INFO:tasks.workunit.client.0.smithi067.stderr: 96150K .......... .......... .......... .......... .......... 70% 25.3M 2s 2024-03-30T02:38:36.252 INFO:tasks.workunit.client.0.smithi067.stderr: 96200K .......... .......... .......... .......... .......... 70% 25.2M 2s 2024-03-30T02:38:36.254 INFO:tasks.workunit.client.0.smithi067.stderr: 96250K .......... .......... .......... .......... .......... 70% 22.6M 2s 2024-03-30T02:38:36.257 INFO:tasks.workunit.client.0.smithi067.stderr: 96300K .......... .......... .......... .......... .......... 70% 24.3M 2s 2024-03-30T02:38:36.257 INFO:tasks.workunit.client.0.smithi067.stderr: 96350K .......... .......... .......... .......... .......... 71% 111M 2s 2024-03-30T02:38:36.259 INFO:tasks.workunit.client.0.smithi067.stderr: 96400K .......... .......... .......... .......... .......... 71% 23.7M 2s 2024-03-30T02:38:36.261 INFO:tasks.workunit.client.0.smithi067.stderr: 96450K .......... .......... .......... .......... .......... 71% 24.6M 2s 2024-03-30T02:38:36.263 INFO:tasks.workunit.client.0.smithi067.stderr: 96500K .......... .......... .......... .......... .......... 71% 23.6M 2s 2024-03-30T02:38:36.263 INFO:tasks.workunit.client.0.smithi067.stderr: 96550K .......... .......... .......... .......... .......... 71% 120M 2s 2024-03-30T02:38:36.265 INFO:tasks.workunit.client.0.smithi067.stderr: 96600K .......... .......... .......... .......... .......... 71% 24.8M 2s 2024-03-30T02:38:36.267 INFO:tasks.workunit.client.0.smithi067.stderr: 96650K .......... .......... .......... .......... .......... 71% 24.2M 2s 2024-03-30T02:38:36.269 INFO:tasks.workunit.client.0.smithi067.stderr: 96700K .......... .......... .......... .......... .......... 71% 25.5M 2s 2024-03-30T02:38:36.271 INFO:tasks.workunit.client.0.smithi067.stderr: 96750K .......... .......... .......... .......... .......... 71% 21.1M 2s 2024-03-30T02:38:36.271 INFO:tasks.workunit.client.0.smithi067.stderr: 96800K .......... .......... .......... .......... .......... 71% 127M 2s 2024-03-30T02:38:36.274 INFO:tasks.workunit.client.0.smithi067.stderr: 96850K .......... .......... .......... .......... .......... 71% 24.7M 2s 2024-03-30T02:38:36.275 INFO:tasks.workunit.client.0.smithi067.stderr: 96900K .......... .......... .......... .......... .......... 71% 21.5M 2s 2024-03-30T02:38:36.278 INFO:tasks.workunit.client.0.smithi067.stderr: 96950K .......... .......... .......... .......... .......... 71% 26.0M 2s 2024-03-30T02:38:36.278 INFO:tasks.workunit.client.0.smithi067.stderr: 97000K .......... .......... .......... .......... .......... 71% 118M 2s 2024-03-30T02:38:36.285 INFO:tasks.workunit.client.0.smithi067.stderr: 97050K .......... .......... .......... .......... .......... 71% 25.0M 2s 2024-03-30T02:38:36.285 INFO:tasks.workunit.client.0.smithi067.stderr: 97100K .......... .......... .......... .......... .......... 71% 24.8M 2s 2024-03-30T02:38:36.285 INFO:tasks.workunit.client.0.smithi067.stderr: 97150K .......... .......... .......... .......... .......... 71% 23.1M 2s 2024-03-30T02:38:36.286 INFO:tasks.workunit.client.0.smithi067.stderr: 97200K .......... .......... .......... .......... .......... 71% 25.2M 2s 2024-03-30T02:38:36.287 INFO:tasks.workunit.client.0.smithi067.stderr: 97250K .......... .......... .......... .......... .......... 71% 150M 2s 2024-03-30T02:38:36.288 INFO:tasks.workunit.client.0.smithi067.stderr: 97300K .......... .......... .......... .......... .......... 71% 23.9M 2s 2024-03-30T02:38:36.290 INFO:tasks.workunit.client.0.smithi067.stderr: 97350K .......... .......... .......... .......... .......... 71% 26.9M 2s 2024-03-30T02:38:36.292 INFO:tasks.workunit.client.0.smithi067.stderr: 97400K .......... .......... .......... .......... .......... 71% 21.1M 2s 2024-03-30T02:38:36.295 INFO:tasks.workunit.client.0.smithi067.stderr: 97450K .......... .......... .......... .......... .......... 71% 27.8M 2s 2024-03-30T02:38:36.295 INFO:tasks.workunit.client.0.smithi067.stderr: 97500K .......... .......... .......... .......... .......... 71% 127M 2s 2024-03-30T02:38:36.297 INFO:tasks.workunit.client.0.smithi067.stderr: 97550K .......... .......... .......... .......... .......... 71% 21.6M 2s 2024-03-30T02:38:36.299 INFO:tasks.workunit.client.0.smithi067.stderr: 97600K .......... .......... .......... .......... .......... 71% 25.7M 2s 2024-03-30T02:38:36.301 INFO:tasks.workunit.client.0.smithi067.stderr: 97650K .......... .......... .......... .......... .......... 71% 22.0M 2s 2024-03-30T02:38:36.303 INFO:tasks.workunit.client.0.smithi067.stderr: 97700K .......... .......... .......... .......... .......... 72% 26.9M 2s 2024-03-30T02:38:36.303 INFO:tasks.workunit.client.0.smithi067.stderr: 97750K .......... .......... .......... .......... .......... 72% 130M 2s 2024-03-30T02:38:36.305 INFO:tasks.workunit.client.0.smithi067.stderr: 97800K .......... .......... .......... .......... .......... 72% 21.5M 2s 2024-03-30T02:38:36.308 INFO:tasks.workunit.client.0.smithi067.stderr: 97850K .......... .......... .......... .......... .......... 72% 27.0M 2s 2024-03-30T02:38:36.310 INFO:tasks.workunit.client.0.smithi067.stderr: 97900K .......... .......... .......... .......... .......... 72% 21.3M 2s 2024-03-30T02:38:36.310 INFO:tasks.workunit.client.0.smithi067.stderr: 97950K .......... .......... .......... .......... .......... 72% 96.6M 2s 2024-03-30T02:38:36.312 INFO:tasks.workunit.client.0.smithi067.stderr: 98000K .......... .......... .......... .......... .......... 72% 28.6M 2s 2024-03-30T02:38:36.314 INFO:tasks.workunit.client.0.smithi067.stderr: 98050K .......... .......... .......... .......... .......... 72% 21.9M 2s 2024-03-30T02:38:36.316 INFO:tasks.workunit.client.0.smithi067.stderr: 98100K .......... .......... .......... .......... .......... 72% 27.3M 2s 2024-03-30T02:38:36.316 INFO:tasks.workunit.client.0.smithi067.stderr: 98150K .......... .......... .......... .......... .......... 72% 84.5M 2s 2024-03-30T02:38:36.318 INFO:tasks.workunit.client.0.smithi067.stderr: 98200K .......... .......... .......... .......... .......... 72% 22.8M 2s 2024-03-30T02:38:36.321 INFO:tasks.workunit.client.0.smithi067.stderr: 98250K .......... .......... .......... .......... .......... 72% 29.3M 2s 2024-03-30T02:38:36.323 INFO:tasks.workunit.client.0.smithi067.stderr: 98300K .......... .......... .......... .......... .......... 72% 20.6M 2s 2024-03-30T02:38:36.325 INFO:tasks.workunit.client.0.smithi067.stderr: 98350K .......... .......... .......... .......... .......... 72% 24.9M 2s 2024-03-30T02:38:36.325 INFO:tasks.workunit.client.0.smithi067.stderr: 98400K .......... .......... .......... .......... .......... 72% 112M 2s 2024-03-30T02:38:36.327 INFO:tasks.workunit.client.0.smithi067.stderr: 98450K .......... .......... .......... .......... .......... 72% 23.4M 2s 2024-03-30T02:38:36.329 INFO:tasks.workunit.client.0.smithi067.stderr: 98500K .......... .......... .......... .......... .......... 72% 28.8M 2s 2024-03-30T02:38:36.331 INFO:tasks.workunit.client.0.smithi067.stderr: 98550K .......... .......... .......... .......... .......... 72% 19.6M 2s 2024-03-30T02:38:36.334 INFO:tasks.workunit.client.0.smithi067.stderr: 98600K .......... .......... .......... .......... .......... 72% 28.1M 2s 2024-03-30T02:38:36.334 INFO:tasks.workunit.client.0.smithi067.stderr: 98650K .......... .......... .......... .......... .......... 72% 64.5M 2s 2024-03-30T02:38:36.336 INFO:tasks.workunit.client.0.smithi067.stderr: 98700K .......... .......... .......... .......... .......... 72% 25.1M 2s 2024-03-30T02:38:36.337 INFO:tasks.workunit.client.0.smithi067.stderr: 98750K .......... .......... .......... .......... .......... 72% 27.9M 2s 2024-03-30T02:38:36.340 INFO:tasks.workunit.client.0.smithi067.stderr: 98800K .......... .......... .......... .......... .......... 72% 25.6M 2s 2024-03-30T02:38:36.340 INFO:tasks.workunit.client.0.smithi067.stderr: 98850K .......... .......... .......... .......... .......... 72% 42.2M 2s 2024-03-30T02:38:36.342 INFO:tasks.workunit.client.0.smithi067.stderr: 98900K .......... .......... .......... .......... .......... 72% 26.1M 2s 2024-03-30T02:38:36.346 INFO:tasks.workunit.client.0.smithi067.stderr: 98950K .......... .......... .......... .......... .......... 72% 12.3M 2s 2024-03-30T02:38:36.348 INFO:tasks.workunit.client.0.smithi067.stderr: 99000K .......... .......... .......... .......... .......... 72% 22.4M 1s 2024-03-30T02:38:36.351 INFO:tasks.workunit.client.0.smithi067.stderr: 99050K .......... .......... .......... .......... .......... 73% 24.4M 1s 2024-03-30T02:38:36.351 INFO:tasks.workunit.client.0.smithi067.stderr: 99100K .......... .......... .......... .......... .......... 73% 119M 1s 2024-03-30T02:38:36.353 INFO:tasks.workunit.client.0.smithi067.stderr: 99150K .......... .......... .......... .......... .......... 73% 27.0M 1s 2024-03-30T02:38:36.355 INFO:tasks.workunit.client.0.smithi067.stderr: 99200K .......... .......... .......... .......... .......... 73% 21.4M 1s 2024-03-30T02:38:36.357 INFO:tasks.workunit.client.0.smithi067.stderr: 99250K .......... .......... .......... .......... .......... 73% 25.3M 1s 2024-03-30T02:38:36.357 INFO:tasks.workunit.client.0.smithi067.stderr: 99300K .......... .......... .......... .......... .......... 73% 116M 1s 2024-03-30T02:38:36.359 INFO:tasks.workunit.client.0.smithi067.stderr: 99350K .......... .......... .......... .......... .......... 73% 21.7M 1s 2024-03-30T02:38:36.361 INFO:tasks.workunit.client.0.smithi067.stderr: 99400K .......... .......... .......... .......... .......... 73% 25.7M 1s 2024-03-30T02:38:36.363 INFO:tasks.workunit.client.0.smithi067.stderr: 99450K .......... .......... .......... .......... .......... 73% 23.4M 1s 2024-03-30T02:38:36.366 INFO:tasks.workunit.client.0.smithi067.stderr: 99500K .......... .......... .......... .......... .......... 73% 24.4M 1s 2024-03-30T02:38:36.366 INFO:tasks.workunit.client.0.smithi067.stderr: 99550K .......... .......... .......... .......... .......... 73% 142M 1s 2024-03-30T02:38:36.368 INFO:tasks.workunit.client.0.smithi067.stderr: 99600K .......... .......... .......... .......... .......... 73% 23.1M 1s 2024-03-30T02:38:36.370 INFO:tasks.workunit.client.0.smithi067.stderr: 99650K .......... .......... .......... .......... .......... 73% 26.7M 1s 2024-03-30T02:38:36.372 INFO:tasks.workunit.client.0.smithi067.stderr: 99700K .......... .......... .......... .......... .......... 73% 23.7M 1s 2024-03-30T02:38:36.374 INFO:tasks.workunit.client.0.smithi067.stderr: 99750K .......... .......... .......... .......... .......... 73% 26.3M 1s 2024-03-30T02:38:36.374 INFO:tasks.workunit.client.0.smithi067.stderr: 99800K .......... .......... .......... .......... .......... 73% 120M 1s 2024-03-30T02:38:36.376 INFO:tasks.workunit.client.0.smithi067.stderr: 99850K .......... .......... .......... .......... .......... 73% 21.9M 1s 2024-03-30T02:38:36.379 INFO:tasks.workunit.client.0.smithi067.stderr: 99900K .......... .......... .......... .......... .......... 73% 26.3M 1s 2024-03-30T02:38:36.381 INFO:tasks.workunit.client.0.smithi067.stderr: 99950K .......... .......... .......... .......... .......... 73% 22.5M 1s 2024-03-30T02:38:36.381 INFO:tasks.workunit.client.0.smithi067.stderr:100000K .......... .......... .......... .......... .......... 73% 115M 1s 2024-03-30T02:38:36.383 INFO:tasks.workunit.client.0.smithi067.stderr:100050K .......... .......... .......... .......... .......... 73% 25.4M 1s 2024-03-30T02:38:36.385 INFO:tasks.workunit.client.0.smithi067.stderr:100100K .......... .......... .......... .......... .......... 73% 22.0M 1s 2024-03-30T02:38:36.387 INFO:tasks.workunit.client.0.smithi067.stderr:100150K .......... .......... .......... .......... .......... 73% 29.0M 1s 2024-03-30T02:38:36.388 INFO:tasks.workunit.client.0.smithi067.stderr:100200K .......... .......... .......... .......... .......... 73% 91.5M 1s 2024-03-30T02:38:36.389 INFO:tasks.workunit.client.0.smithi067.stderr:100250K .......... .......... .......... .......... .......... 73% 21.4M 1s 2024-03-30T02:38:36.391 INFO:tasks.workunit.client.0.smithi067.stderr:100300K .......... .......... .......... .......... .......... 73% 27.0M 1s 2024-03-30T02:38:36.393 INFO:tasks.workunit.client.0.smithi067.stderr:100350K .......... .......... .......... .......... .......... 73% 25.8M 1s 2024-03-30T02:38:36.396 INFO:tasks.workunit.client.0.smithi067.stderr:100400K .......... .......... .......... .......... .......... 73% 20.0M 1s 2024-03-30T02:38:36.396 INFO:tasks.workunit.client.0.smithi067.stderr:100450K .......... .......... .......... .......... .......... 74% 109M 1s 2024-03-30T02:38:36.399 INFO:tasks.workunit.client.0.smithi067.stderr:100500K .......... .......... .......... .......... .......... 74% 23.6M 1s 2024-03-30T02:38:36.400 INFO:tasks.workunit.client.0.smithi067.stderr:100550K .......... .......... .......... .......... .......... 74% 25.1M 1s 2024-03-30T02:38:36.402 INFO:tasks.workunit.client.0.smithi067.stderr:100600K .......... .......... .......... .......... .......... 74% 23.8M 1s 2024-03-30T02:38:36.405 INFO:tasks.workunit.client.0.smithi067.stderr:100650K .......... .......... .......... .......... .......... 74% 23.1M 1s 2024-03-30T02:38:36.405 INFO:tasks.workunit.client.0.smithi067.stderr:100700K .......... .......... .......... .......... .......... 74% 116M 1s 2024-03-30T02:38:36.407 INFO:tasks.workunit.client.0.smithi067.stderr:100750K .......... .......... .......... .......... .......... 74% 25.0M 1s 2024-03-30T02:38:36.409 INFO:tasks.workunit.client.0.smithi067.stderr:100800K .......... .......... .......... .......... .......... 74% 23.9M 1s 2024-03-30T02:38:36.411 INFO:tasks.workunit.client.0.smithi067.stderr:100850K .......... .......... .......... .......... .......... 74% 23.6M 1s 2024-03-30T02:38:36.411 INFO:tasks.workunit.client.0.smithi067.stderr:100900K .......... .......... .......... .......... .......... 74% 121M 1s 2024-03-30T02:38:36.413 INFO:tasks.workunit.client.0.smithi067.stderr:100950K .......... .......... .......... .......... .......... 74% 24.6M 1s 2024-03-30T02:38:36.415 INFO:tasks.workunit.client.0.smithi067.stderr:101000K .......... .......... .......... .......... .......... 74% 24.1M 1s 2024-03-30T02:38:36.418 INFO:tasks.workunit.client.0.smithi067.stderr:101050K .......... .......... .......... .......... .......... 74% 24.8M 1s 2024-03-30T02:38:36.420 INFO:tasks.workunit.client.0.smithi067.stderr:101100K .......... .......... .......... .......... .......... 74% 22.0M 1s 2024-03-30T02:38:36.420 INFO:tasks.workunit.client.0.smithi067.stderr:101150K .......... .......... .......... .......... .......... 74% 101M 1s 2024-03-30T02:38:36.422 INFO:tasks.workunit.client.0.smithi067.stderr:101200K .......... .......... .......... .......... .......... 74% 25.3M 1s 2024-03-30T02:38:36.424 INFO:tasks.workunit.client.0.smithi067.stderr:101250K .......... .......... .......... .......... .......... 74% 23.4M 1s 2024-03-30T02:38:36.426 INFO:tasks.workunit.client.0.smithi067.stderr:101300K .......... .......... .......... .......... .......... 74% 24.6M 1s 2024-03-30T02:38:36.427 INFO:tasks.workunit.client.0.smithi067.stderr:101350K .......... .......... .......... .......... .......... 74% 119M 1s 2024-03-30T02:38:36.429 INFO:tasks.workunit.client.0.smithi067.stderr:101400K .......... .......... .......... .......... .......... 74% 21.4M 1s 2024-03-30T02:38:36.431 INFO:tasks.workunit.client.0.smithi067.stderr:101450K .......... .......... .......... .......... .......... 74% 25.0M 1s 2024-03-30T02:38:36.433 INFO:tasks.workunit.client.0.smithi067.stderr:101500K .......... .......... .......... .......... .......... 74% 23.9M 1s 2024-03-30T02:38:36.436 INFO:tasks.workunit.client.0.smithi067.stderr:101550K .......... .......... .......... .......... .......... 74% 24.0M 1s 2024-03-30T02:38:36.436 INFO:tasks.workunit.client.0.smithi067.stderr:101600K .......... .......... .......... .......... .......... 74% 124M 1s 2024-03-30T02:38:36.438 INFO:tasks.workunit.client.0.smithi067.stderr:101650K .......... .......... .......... .......... .......... 74% 23.5M 1s 2024-03-30T02:38:36.439 INFO:tasks.workunit.client.0.smithi067.stderr:101700K .......... .......... .......... .......... .......... 74% 24.3M 1s 2024-03-30T02:38:36.442 INFO:tasks.workunit.client.0.smithi067.stderr:101750K .......... .......... .......... .......... .......... 74% 23.8M 1s 2024-03-30T02:38:36.442 INFO:tasks.workunit.client.0.smithi067.stderr:101800K .......... .......... .......... .......... .......... 75% 116M 1s 2024-03-30T02:38:36.445 INFO:tasks.workunit.client.0.smithi067.stderr:101850K .......... .......... .......... .......... .......... 75% 12.5M 1s 2024-03-30T02:38:36.448 INFO:tasks.workunit.client.0.smithi067.stderr:101900K .......... .......... .......... .......... .......... 75% 21.3M 1s 2024-03-30T02:38:36.450 INFO:tasks.workunit.client.0.smithi067.stderr:101950K .......... .......... .......... .......... .......... 75% 22.6M 1s 2024-03-30T02:38:36.452 INFO:tasks.workunit.client.0.smithi067.stderr:102000K .......... .......... .......... .......... .......... 75% 24.4M 1s 2024-03-30T02:38:36.452 INFO:tasks.workunit.client.0.smithi067.stderr:102050K .......... .......... .......... .......... .......... 75% 109M 1s 2024-03-30T02:38:36.455 INFO:tasks.workunit.client.0.smithi067.stderr:102100K .......... .......... .......... .......... .......... 75% 26.3M 1s 2024-03-30T02:38:36.457 INFO:tasks.workunit.client.0.smithi067.stderr:102150K .......... .......... .......... .......... .......... 75% 24.5M 1s 2024-03-30T02:38:36.459 INFO:tasks.workunit.client.0.smithi067.stderr:102200K .......... .......... .......... .......... .......... 75% 24.5M 1s 2024-03-30T02:38:36.461 INFO:tasks.workunit.client.0.smithi067.stderr:102250K .......... .......... .......... .......... .......... 75% 23.1M 1s 2024-03-30T02:38:36.461 INFO:tasks.workunit.client.0.smithi067.stderr:102300K .......... .......... .......... .......... .......... 75% 113M 1s 2024-03-30T02:38:36.463 INFO:tasks.workunit.client.0.smithi067.stderr:102350K .......... .......... .......... .......... .......... 75% 23.9M 1s 2024-03-30T02:38:36.465 INFO:tasks.workunit.client.0.smithi067.stderr:102400K .......... .......... .......... .......... .......... 75% 21.9M 1s 2024-03-30T02:38:36.467 INFO:tasks.workunit.client.0.smithi067.stderr:102450K .......... .......... .......... .......... .......... 75% 24.6M 1s 2024-03-30T02:38:36.467 INFO:tasks.workunit.client.0.smithi067.stderr:102500K .......... .......... .......... .......... .......... 75% 143M 1s 2024-03-30T02:38:36.470 INFO:tasks.workunit.client.0.smithi067.stderr:102550K .......... .......... .......... .......... .......... 75% 25.7M 1s 2024-03-30T02:38:36.472 INFO:tasks.workunit.client.0.smithi067.stderr:102600K .......... .......... .......... .......... .......... 75% 24.1M 1s 2024-03-30T02:38:36.474 INFO:tasks.workunit.client.0.smithi067.stderr:102650K .......... .......... .......... .......... .......... 75% 23.2M 1s 2024-03-30T02:38:36.476 INFO:tasks.workunit.client.0.smithi067.stderr:102700K .......... .......... .......... .......... .......... 75% 24.7M 1s 2024-03-30T02:38:36.476 INFO:tasks.workunit.client.0.smithi067.stderr:102750K .......... .......... .......... .......... .......... 75% 126M 1s 2024-03-30T02:38:36.478 INFO:tasks.workunit.client.0.smithi067.stderr:102800K .......... .......... .......... .......... .......... 75% 24.0M 1s 2024-03-30T02:38:36.480 INFO:tasks.workunit.client.0.smithi067.stderr:102850K .......... .......... .......... .......... .......... 75% 25.2M 1s 2024-03-30T02:38:36.482 INFO:tasks.workunit.client.0.smithi067.stderr:102900K .......... .......... .......... .......... .......... 75% 23.0M 1s 2024-03-30T02:38:36.483 INFO:tasks.workunit.client.0.smithi067.stderr:102950K .......... .......... .......... .......... .......... 75% 126M 1s 2024-03-30T02:38:36.485 INFO:tasks.workunit.client.0.smithi067.stderr:103000K .......... .......... .......... .......... .......... 75% 24.4M 1s 2024-03-30T02:38:36.500 INFO:tasks.workunit.client.0.smithi067.stderr:103050K .......... .......... .......... .......... .......... 75% 22.6M 1s 2024-03-30T02:38:36.500 INFO:tasks.workunit.client.0.smithi067.stderr:103100K .......... .......... .......... .......... .......... 75% 24.5M 1s 2024-03-30T02:38:36.500 INFO:tasks.workunit.client.0.smithi067.stderr:103150K .......... .......... .......... .......... .......... 76% 23.2M 1s 2024-03-30T02:38:36.500 INFO:tasks.workunit.client.0.smithi067.stderr:103200K .......... .......... .......... .......... .......... 76% 123M 1s 2024-03-30T02:38:36.500 INFO:tasks.workunit.client.0.smithi067.stderr:103250K .......... .......... .......... .......... .......... 76% 24.5M 1s 2024-03-30T02:38:36.500 INFO:tasks.workunit.client.0.smithi067.stderr:103300K .......... .......... .......... .......... .......... 76% 24.0M 1s 2024-03-30T02:38:36.501 INFO:tasks.workunit.client.0.smithi067.stderr:103350K .......... .......... .......... .......... .......... 76% 24.7M 1s 2024-03-30T02:38:36.501 INFO:tasks.workunit.client.0.smithi067.stderr:103400K .......... .......... .......... .......... .......... 76% 113M 1s 2024-03-30T02:38:36.501 INFO:tasks.workunit.client.0.smithi067.stderr:103450K .......... .......... .......... .......... .......... 76% 22.3M 1s 2024-03-30T02:38:36.504 INFO:tasks.workunit.client.0.smithi067.stderr:103500K .......... .......... .......... .......... .......... 76% 23.8M 1s 2024-03-30T02:38:36.504 INFO:tasks.workunit.client.0.smithi067.stderr:103550K .......... .......... .......... .......... .......... 76% 24.7M 1s 2024-03-30T02:38:36.506 INFO:tasks.workunit.client.0.smithi067.stderr:103600K .......... .......... .......... .......... .......... 76% 23.6M 1s 2024-03-30T02:38:36.506 INFO:tasks.workunit.client.0.smithi067.stderr:103650K .......... .......... .......... .......... .......... 76% 113M 1s 2024-03-30T02:38:36.508 INFO:tasks.workunit.client.0.smithi067.stderr:103700K .......... .......... .......... .......... .......... 76% 24.8M 1s 2024-03-30T02:38:36.511 INFO:tasks.workunit.client.0.smithi067.stderr:103750K .......... .......... .......... .......... .......... 76% 23.7M 1s 2024-03-30T02:38:36.513 INFO:tasks.workunit.client.0.smithi067.stderr:103800K .......... .......... .......... .......... .......... 76% 24.1M 1s 2024-03-30T02:38:36.515 INFO:tasks.workunit.client.0.smithi067.stderr:103850K .......... .......... .......... .......... .......... 76% 23.7M 1s 2024-03-30T02:38:36.515 INFO:tasks.workunit.client.0.smithi067.stderr:103900K .......... .......... .......... .......... .......... 76% 128M 1s 2024-03-30T02:38:36.517 INFO:tasks.workunit.client.0.smithi067.stderr:103950K .......... .......... .......... .......... .......... 76% 24.4M 1s 2024-03-30T02:38:36.519 INFO:tasks.workunit.client.0.smithi067.stderr:104000K .......... .......... .......... .......... .......... 76% 24.3M 1s 2024-03-30T02:38:36.521 INFO:tasks.workunit.client.0.smithi067.stderr:104050K .......... .......... .......... .......... .......... 76% 23.6M 1s 2024-03-30T02:38:36.521 INFO:tasks.workunit.client.0.smithi067.stderr:104100K .......... .......... .......... .......... .......... 76% 116M 1s 2024-03-30T02:38:36.524 INFO:tasks.workunit.client.0.smithi067.stderr:104150K .......... .......... .......... .......... .......... 76% 24.1M 1s 2024-03-30T02:38:36.526 INFO:tasks.workunit.client.0.smithi067.stderr:104200K .......... .......... .......... .......... .......... 76% 23.2M 1s 2024-03-30T02:38:36.528 INFO:tasks.workunit.client.0.smithi067.stderr:104250K .......... .......... .......... .......... .......... 76% 24.5M 1s 2024-03-30T02:38:36.530 INFO:tasks.workunit.client.0.smithi067.stderr:104300K .......... .......... .......... .......... .......... 76% 22.3M 1s 2024-03-30T02:38:36.530 INFO:tasks.workunit.client.0.smithi067.stderr:104350K .......... .......... .......... .......... .......... 76% 118M 1s 2024-03-30T02:38:36.532 INFO:tasks.workunit.client.0.smithi067.stderr:104400K .......... .......... .......... .......... .......... 76% 24.1M 1s 2024-03-30T02:38:36.537 INFO:tasks.workunit.client.0.smithi067.stderr:104450K .......... .......... .......... .......... .......... 76% 24.1M 1s 2024-03-30T02:38:36.537 INFO:tasks.workunit.client.0.smithi067.stderr:104500K .......... .......... .......... .......... .......... 77% 24.5M 1s 2024-03-30T02:38:36.537 INFO:tasks.workunit.client.0.smithi067.stderr:104550K .......... .......... .......... .......... .......... 77% 115M 1s 2024-03-30T02:38:36.541 INFO:tasks.workunit.client.0.smithi067.stderr:104600K .......... .......... .......... .......... .......... 77% 23.5M 1s 2024-03-30T02:38:36.541 INFO:tasks.workunit.client.0.smithi067.stderr:104650K .......... .......... .......... .......... .......... 77% 24.4M 1s 2024-03-30T02:38:36.545 INFO:tasks.workunit.client.0.smithi067.stderr:104700K .......... .......... .......... .......... .......... 77% 13.8M 1s 2024-03-30T02:38:36.546 INFO:tasks.workunit.client.0.smithi067.stderr:104750K .......... .......... .......... .......... .......... 77% 23.0M 1s 2024-03-30T02:38:36.546 INFO:tasks.workunit.client.0.smithi067.stderr:104800K .......... .......... .......... .......... .......... 77% 120M 1s 2024-03-30T02:38:36.549 INFO:tasks.workunit.client.0.smithi067.stderr:104850K .......... .......... .......... .......... .......... 77% 20.0M 1s 2024-03-30T02:38:36.551 INFO:tasks.workunit.client.0.smithi067.stderr:104900K .......... .......... .......... .......... .......... 77% 23.8M 1s 2024-03-30T02:38:36.553 INFO:tasks.workunit.client.0.smithi067.stderr:104950K .......... .......... .......... .......... .......... 77% 25.0M 1s 2024-03-30T02:38:36.555 INFO:tasks.workunit.client.0.smithi067.stderr:105000K .......... .......... .......... .......... .......... 77% 25.0M 1s 2024-03-30T02:38:36.555 INFO:tasks.workunit.client.0.smithi067.stderr:105050K .......... .......... .......... .......... .......... 77% 121M 1s 2024-03-30T02:38:36.558 INFO:tasks.workunit.client.0.smithi067.stderr:105100K .......... .......... .......... .......... .......... 77% 25.0M 1s 2024-03-30T02:38:36.560 INFO:tasks.workunit.client.0.smithi067.stderr:105150K .......... .......... .......... .......... .......... 77% 22.3M 1s 2024-03-30T02:38:36.564 INFO:tasks.workunit.client.0.smithi067.stderr:105200K .......... .......... .......... .......... .......... 77% 23.5M 1s 2024-03-30T02:38:36.564 INFO:tasks.workunit.client.0.smithi067.stderr:105250K .......... .......... .......... .......... .......... 77% 122M 1s 2024-03-30T02:38:36.564 INFO:tasks.workunit.client.0.smithi067.stderr:105300K .......... .......... .......... .......... .......... 77% 23.8M 1s 2024-03-30T02:38:36.567 INFO:tasks.workunit.client.0.smithi067.stderr:105350K .......... .......... .......... .......... .......... 77% 23.9M 1s 2024-03-30T02:38:36.568 INFO:tasks.workunit.client.0.smithi067.stderr:105400K .......... .......... .......... .......... .......... 77% 25.1M 1s 2024-03-30T02:38:36.571 INFO:tasks.workunit.client.0.smithi067.stderr:105450K .......... .......... .......... .......... .......... 77% 23.0M 1s 2024-03-30T02:38:36.571 INFO:tasks.workunit.client.0.smithi067.stderr:105500K .......... .......... .......... .......... .......... 77% 124M 1s 2024-03-30T02:38:36.573 INFO:tasks.workunit.client.0.smithi067.stderr:105550K .......... .......... .......... .......... .......... 77% 24.6M 1s 2024-03-30T02:38:36.575 INFO:tasks.workunit.client.0.smithi067.stderr:105600K .......... .......... .......... .......... .......... 77% 23.9M 1s 2024-03-30T02:38:36.577 INFO:tasks.workunit.client.0.smithi067.stderr:105650K .......... .......... .......... .......... .......... 77% 24.5M 1s 2024-03-30T02:38:36.577 INFO:tasks.workunit.client.0.smithi067.stderr:105700K .......... .......... .......... .......... .......... 77% 120M 1s 2024-03-30T02:38:36.579 INFO:tasks.workunit.client.0.smithi067.stderr:105750K .......... .......... .......... .......... .......... 77% 23.5M 1s 2024-03-30T02:38:36.581 INFO:tasks.workunit.client.0.smithi067.stderr:105800K .......... .......... .......... .......... .......... 77% 24.9M 1s 2024-03-30T02:38:36.584 INFO:tasks.workunit.client.0.smithi067.stderr:105850K .......... .......... .......... .......... .......... 78% 22.6M 1s 2024-03-30T02:38:36.586 INFO:tasks.workunit.client.0.smithi067.stderr:105900K .......... .......... .......... .......... .......... 78% 24.4M 1s 2024-03-30T02:38:36.586 INFO:tasks.workunit.client.0.smithi067.stderr:105950K .......... .......... .......... .......... .......... 78% 117M 1s 2024-03-30T02:38:36.587 INFO:tasks.workunit.client.0.smithi067.stderr:106000K .......... .......... .......... .......... .......... 78% 23.2M 1s 2024-03-30T02:38:36.590 INFO:tasks.workunit.client.0.smithi067.stderr:106050K .......... .......... .......... .......... .......... 78% 27.3M 1s 2024-03-30T02:38:36.591 INFO:tasks.workunit.client.0.smithi067.stderr:106100K .......... .......... .......... .......... .......... 78% 22.2M 1s 2024-03-30T02:38:36.594 INFO:tasks.workunit.client.0.smithi067.stderr:106150K .......... .......... .......... .......... .......... 78% 26.6M 1s 2024-03-30T02:38:36.594 INFO:tasks.workunit.client.0.smithi067.stderr:106200K .......... .......... .......... .......... .......... 78% 116M 1s 2024-03-30T02:38:36.596 INFO:tasks.workunit.client.0.smithi067.stderr:106250K .......... .......... .......... .......... .......... 78% 23.3M 1s 2024-03-30T02:38:36.599 INFO:tasks.workunit.client.0.smithi067.stderr:106300K .......... .......... .......... .......... .......... 78% 26.6M 1s 2024-03-30T02:38:36.600 INFO:tasks.workunit.client.0.smithi067.stderr:106350K .......... .......... .......... .......... .......... 78% 22.0M 1s 2024-03-30T02:38:36.600 INFO:tasks.workunit.client.0.smithi067.stderr:106400K .......... .......... .......... .......... .......... 78% 113M 1s 2024-03-30T02:38:36.602 INFO:tasks.workunit.client.0.smithi067.stderr:106450K .......... .......... .......... .......... .......... 78% 26.3M 1s 2024-03-30T02:38:36.604 INFO:tasks.workunit.client.0.smithi067.stderr:106500K .......... .......... .......... .......... .......... 78% 23.1M 1s 2024-03-30T02:38:36.607 INFO:tasks.workunit.client.0.smithi067.stderr:106550K .......... .......... .......... .......... .......... 78% 29.2M 1s 2024-03-30T02:38:36.609 INFO:tasks.workunit.client.0.smithi067.stderr:106600K .......... .......... .......... .......... .......... 78% 71.3M 1s 2024-03-30T02:38:36.609 INFO:tasks.workunit.client.0.smithi067.stderr:106650K .......... .......... .......... .......... .......... 78% 22.1M 1s 2024-03-30T02:38:36.611 INFO:tasks.workunit.client.0.smithi067.stderr:106700K .......... .......... .......... .......... .......... 78% 26.5M 1s 2024-03-30T02:38:36.613 INFO:tasks.workunit.client.0.smithi067.stderr:106750K .......... .......... .......... .......... .......... 78% 21.1M 1s 2024-03-30T02:38:36.616 INFO:tasks.workunit.client.0.smithi067.stderr:106800K .......... .......... .......... .......... .......... 78% 25.5M 1s 2024-03-30T02:38:36.616 INFO:tasks.workunit.client.0.smithi067.stderr:106850K .......... .......... .......... .......... .......... 78% 129M 1s 2024-03-30T02:38:36.618 INFO:tasks.workunit.client.0.smithi067.stderr:106900K .......... .......... .......... .......... .......... 78% 21.9M 1s 2024-03-30T02:38:36.620 INFO:tasks.workunit.client.0.smithi067.stderr:106950K .......... .......... .......... .......... .......... 78% 24.4M 1s 2024-03-30T02:38:36.622 INFO:tasks.workunit.client.0.smithi067.stderr:107000K .......... .......... .......... .......... .......... 78% 23.8M 1s 2024-03-30T02:38:36.624 INFO:tasks.workunit.client.0.smithi067.stderr:107050K .......... .......... .......... .......... .......... 78% 25.1M 1s 2024-03-30T02:38:36.624 INFO:tasks.workunit.client.0.smithi067.stderr:107100K .......... .......... .......... .......... .......... 78% 113M 1s 2024-03-30T02:38:36.627 INFO:tasks.workunit.client.0.smithi067.stderr:107150K .......... .......... .......... .......... .......... 78% 22.9M 1s 2024-03-30T02:38:36.629 INFO:tasks.workunit.client.0.smithi067.stderr:107200K .......... .......... .......... .......... .......... 79% 24.4M 1s 2024-03-30T02:38:36.631 INFO:tasks.workunit.client.0.smithi067.stderr:107250K .......... .......... .......... .......... .......... 79% 23.1M 1s 2024-03-30T02:38:36.633 INFO:tasks.workunit.client.0.smithi067.stderr:107300K .......... .......... .......... .......... .......... 79% 113M 1s 2024-03-30T02:38:36.633 INFO:tasks.workunit.client.0.smithi067.stderr:107350K .......... .......... .......... .......... .......... 79% 26.4M 1s 2024-03-30T02:38:36.635 INFO:tasks.workunit.client.0.smithi067.stderr:107400K .......... .......... .......... .......... .......... 79% 22.3M 1s 2024-03-30T02:38:36.638 INFO:tasks.workunit.client.0.smithi067.stderr:107450K .......... .......... .......... .......... .......... 79% 26.0M 1s 2024-03-30T02:38:36.639 INFO:tasks.workunit.client.0.smithi067.stderr:107500K .......... .......... .......... .......... .......... 79% 23.3M 1s 2024-03-30T02:38:36.639 INFO:tasks.workunit.client.0.smithi067.stderr:107550K .......... .......... .......... .......... .......... 79% 120M 1s 2024-03-30T02:38:36.642 INFO:tasks.workunit.client.0.smithi067.stderr:107600K .......... .......... .......... .......... .......... 79% 26.2M 1s 2024-03-30T02:38:36.645 INFO:tasks.workunit.client.0.smithi067.stderr:107650K .......... .......... .......... .......... .......... 79% 12.3M 1s 2024-03-30T02:38:36.648 INFO:tasks.workunit.client.0.smithi067.stderr:107700K .......... .......... .......... .......... .......... 79% 30.4M 1s 2024-03-30T02:38:36.648 INFO:tasks.workunit.client.0.smithi067.stderr:107750K .......... .......... .......... .......... .......... 79% 112M 1s 2024-03-30T02:38:36.649 INFO:tasks.workunit.client.0.smithi067.stderr:107800K .......... .......... .......... .......... .......... 79% 18.5M 1s 2024-03-30T02:38:36.652 INFO:tasks.workunit.client.0.smithi067.stderr:107850K .......... .......... .......... .......... .......... 79% 23.0M 1s 2024-03-30T02:38:36.654 INFO:tasks.workunit.client.0.smithi067.stderr:107900K .......... .......... .......... .......... .......... 79% 25.3M 1s 2024-03-30T02:38:36.656 INFO:tasks.workunit.client.0.smithi067.stderr:107950K .......... .......... .......... .......... .......... 79% 25.4M 1s 2024-03-30T02:38:36.656 INFO:tasks.workunit.client.0.smithi067.stderr:108000K .......... .......... .......... .......... .......... 79% 80.1M 1s 2024-03-30T02:38:36.658 INFO:tasks.workunit.client.0.smithi067.stderr:108050K .......... .......... .......... .......... .......... 79% 21.3M 1s 2024-03-30T02:38:36.661 INFO:tasks.workunit.client.0.smithi067.stderr:108100K .......... .......... .......... .......... .......... 79% 23.7M 1s 2024-03-30T02:38:36.663 INFO:tasks.workunit.client.0.smithi067.stderr:108150K .......... .......... .......... .......... .......... 79% 24.4M 1s 2024-03-30T02:38:36.663 INFO:tasks.workunit.client.0.smithi067.stderr:108200K .......... .......... .......... .......... .......... 79% 128M 1s 2024-03-30T02:38:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:36 smithi067 ceph-mon[37808]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-03-30T02:38:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:36 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:38:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:38:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:36 smithi067 ceph-mon[37808]: mgrmap e28: x(active, since 93s), standbys: z, y 2024-03-30T02:38:36.665 INFO:tasks.workunit.client.0.smithi067.stderr:108250K .......... .......... .......... .......... .......... 79% 25.3M 1s 2024-03-30T02:38:36.667 INFO:tasks.workunit.client.0.smithi067.stderr:108300K .......... .......... .......... .......... .......... 79% 24.8M 1s 2024-03-30T02:38:36.670 INFO:tasks.workunit.client.0.smithi067.stderr:108350K .......... .......... .......... .......... .......... 79% 23.7M 1s 2024-03-30T02:38:36.671 INFO:tasks.workunit.client.0.smithi067.stderr:108400K .......... .......... .......... .......... .......... 79% 22.0M 1s 2024-03-30T02:38:36.671 INFO:tasks.workunit.client.0.smithi067.stderr:108450K .......... .......... .......... .......... .......... 79% 188M 1s 2024-03-30T02:38:36.673 INFO:tasks.workunit.client.0.smithi067.stderr:108500K .......... .......... .......... .......... .......... 79% 24.7M 1s 2024-03-30T02:38:36.676 INFO:tasks.workunit.client.0.smithi067.stderr:108550K .......... .......... .......... .......... .......... 79% 23.2M 1s 2024-03-30T02:38:36.678 INFO:tasks.workunit.client.0.smithi067.stderr:108600K .......... .......... .......... .......... .......... 80% 24.2M 1s 2024-03-30T02:38:36.680 INFO:tasks.workunit.client.0.smithi067.stderr:108650K .......... .......... .......... .......... .......... 80% 24.7M 1s 2024-03-30T02:38:36.680 INFO:tasks.workunit.client.0.smithi067.stderr:108700K .......... .......... .......... .......... .......... 80% 129M 1s 2024-03-30T02:38:36.682 INFO:tasks.workunit.client.0.smithi067.stderr:108750K .......... .......... .......... .......... .......... 80% 25.1M 1s 2024-03-30T02:38:36.684 INFO:tasks.workunit.client.0.smithi067.stderr:108800K .......... .......... .......... .......... .......... 80% 23.5M 1s 2024-03-30T02:38:36.686 INFO:tasks.workunit.client.0.smithi067.stderr:108850K .......... .......... .......... .......... .......... 80% 24.8M 1s 2024-03-30T02:38:36.686 INFO:tasks.workunit.client.0.smithi067.stderr:108900K .......... .......... .......... .......... .......... 80% 120M 1s 2024-03-30T02:38:36.688 INFO:tasks.workunit.client.0.smithi067.stderr:108950K .......... .......... .......... .......... .......... 80% 23.6M 1s 2024-03-30T02:38:36.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:36 smithi170 ceph-mon[78798]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-03-30T02:38:36.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:36 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:38:36.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:36.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:38:36.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:36.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:36 smithi170 ceph-mon[78798]: mgrmap e28: x(active, since 93s), standbys: z, y 2024-03-30T02:38:36.706 INFO:tasks.workunit.client.0.smithi067.stderr:109000K .......... .......... .......... .......... .......... 80% 26.0M 1s 2024-03-30T02:38:36.706 INFO:tasks.workunit.client.0.smithi067.stderr:109050K .......... .......... .......... .......... .......... 80% 22.5M 1s 2024-03-30T02:38:36.706 INFO:tasks.workunit.client.0.smithi067.stderr:109100K .......... .......... .......... .......... .......... 80% 25.9M 1s 2024-03-30T02:38:36.706 INFO:tasks.workunit.client.0.smithi067.stderr:109150K .......... .......... .......... .......... .......... 80% 124M 1s 2024-03-30T02:38:36.707 INFO:tasks.workunit.client.0.smithi067.stderr:109200K .......... .......... .......... .......... .......... 80% 20.3M 1s 2024-03-30T02:38:36.707 INFO:tasks.workunit.client.0.smithi067.stderr:109250K .......... .......... .......... .......... .......... 80% 30.5M 1s 2024-03-30T02:38:36.707 INFO:tasks.workunit.client.0.smithi067.stderr:109300K .......... .......... .......... .......... .......... 80% 21.9M 1s 2024-03-30T02:38:36.707 INFO:tasks.workunit.client.0.smithi067.stderr:109350K .......... .......... .......... .......... .......... 80% 26.0M 1s 2024-03-30T02:38:36.707 INFO:tasks.workunit.client.0.smithi067.stderr:109400K .......... .......... .......... .......... .......... 80% 114M 1s 2024-03-30T02:38:36.707 INFO:tasks.workunit.client.0.smithi067.stderr:109450K .......... .......... .......... .......... .......... 80% 26.7M 1s 2024-03-30T02:38:36.710 INFO:tasks.workunit.client.0.smithi067.stderr:109500K .......... .......... .......... .......... .......... 80% 19.5M 1s 2024-03-30T02:38:36.710 INFO:tasks.workunit.client.0.smithi067.stderr:109550K .......... .......... .......... .......... .......... 80% 24.3M 1s 2024-03-30T02:38:36.710 INFO:tasks.workunit.client.0.smithi067.stderr:109600K .......... .......... .......... .......... .......... 80% 123M 1s 2024-03-30T02:38:36.712 INFO:tasks.workunit.client.0.smithi067.stderr:109650K .......... .......... .......... .......... .......... 80% 24.1M 1s 2024-03-30T02:38:36.714 INFO:tasks.workunit.client.0.smithi067.stderr:109700K .......... .......... .......... .......... .......... 80% 24.1M 1s 2024-03-30T02:38:36.716 INFO:tasks.workunit.client.0.smithi067.stderr:109750K .......... .......... .......... .......... .......... 80% 28.3M 1s 2024-03-30T02:38:36.719 INFO:tasks.workunit.client.0.smithi067.stderr:109800K .......... .......... .......... .......... .......... 80% 78.0M 1s 2024-03-30T02:38:36.719 INFO:tasks.workunit.client.0.smithi067.stderr:109850K .......... .......... .......... .......... .......... 80% 22.8M 1s 2024-03-30T02:38:36.721 INFO:tasks.workunit.client.0.smithi067.stderr:109900K .......... .......... .......... .......... .......... 80% 27.2M 1s 2024-03-30T02:38:36.723 INFO:tasks.workunit.client.0.smithi067.stderr:109950K .......... .......... .......... .......... .......... 81% 21.9M 1s 2024-03-30T02:38:36.725 INFO:tasks.workunit.client.0.smithi067.stderr:110000K .......... .......... .......... .......... .......... 81% 27.4M 1s 2024-03-30T02:38:36.725 INFO:tasks.workunit.client.0.smithi067.stderr:110050K .......... .......... .......... .......... .......... 81% 83.4M 1s 2024-03-30T02:38:36.727 INFO:tasks.workunit.client.0.smithi067.stderr:110100K .......... .......... .......... .......... .......... 81% 23.5M 1s 2024-03-30T02:38:36.729 INFO:tasks.workunit.client.0.smithi067.stderr:110150K .......... .......... .......... .......... .......... 81% 24.0M 1s 2024-03-30T02:38:36.734 INFO:tasks.workunit.client.0.smithi067.stderr:110200K .......... .......... .......... .......... .......... 81% 24.4M 1s 2024-03-30T02:38:36.734 INFO:tasks.workunit.client.0.smithi067.stderr:110250K .......... .......... .......... .......... .......... 81% 28.0M 1s 2024-03-30T02:38:36.734 INFO:tasks.workunit.client.0.smithi067.stderr:110300K .......... .......... .......... .......... .......... 81% 69.8M 1s 2024-03-30T02:38:36.736 INFO:tasks.workunit.client.0.smithi067.stderr:110350K .......... .......... .......... .......... .......... 81% 20.6M 1s 2024-03-30T02:38:36.739 INFO:tasks.workunit.client.0.smithi067.stderr:110400K .......... .......... .......... .......... .......... 81% 26.7M 1s 2024-03-30T02:38:36.740 INFO:tasks.workunit.client.0.smithi067.stderr:110450K .......... .......... .......... .......... .......... 81% 21.9M 1s 2024-03-30T02:38:36.743 INFO:tasks.workunit.client.0.smithi067.stderr:110500K .......... .......... .......... .......... .......... 81% 27.6M 1s 2024-03-30T02:38:36.743 INFO:tasks.workunit.client.0.smithi067.stderr:110550K .......... .......... .......... .......... .......... 81% 67.3M 1s 2024-03-30T02:38:36.744 INFO:tasks.workunit.client.0.smithi067.stderr:110600K .......... .......... .......... .......... .......... 81% 22.5M 1s 2024-03-30T02:38:36.747 INFO:tasks.workunit.client.0.smithi067.stderr:110650K .......... .......... .......... .......... .......... 81% 33.1M 1s 2024-03-30T02:38:36.749 INFO:tasks.workunit.client.0.smithi067.stderr:110700K .......... .......... .......... .......... .......... 81% 21.0M 1s 2024-03-30T02:38:36.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:36 smithi082 ceph-mon[79176]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-03-30T02:38:36.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:36 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:38:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:38:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:38:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:36 smithi082 ceph-mon[79176]: mgrmap e28: x(active, since 93s), standbys: z, y 2024-03-30T02:38:36.751 INFO:tasks.workunit.client.0.smithi067.stderr:110750K .......... .......... .......... .......... .......... 81% 27.9M 1s 2024-03-30T02:38:36.751 INFO:tasks.workunit.client.0.smithi067.stderr:110800K .......... .......... .......... .......... .......... 81% 56.0M 1s 2024-03-30T02:38:36.754 INFO:tasks.workunit.client.0.smithi067.stderr:110850K .......... .......... .......... .......... .......... 81% 14.5M 1s 2024-03-30T02:38:36.757 INFO:tasks.workunit.client.0.smithi067.stderr:110900K .......... .......... .......... .......... .......... 81% 24.7M 1s 2024-03-30T02:38:36.757 INFO:tasks.workunit.client.0.smithi067.stderr:110950K .......... .......... .......... .......... .......... 81% 81.7M 1s 2024-03-30T02:38:36.759 INFO:tasks.workunit.client.0.smithi067.stderr:111000K .......... .......... .......... .......... .......... 81% 21.2M 1s 2024-03-30T02:38:36.762 INFO:tasks.workunit.client.0.smithi067.stderr:111050K .......... .......... .......... .......... .......... 81% 24.0M 1s 2024-03-30T02:38:36.764 INFO:tasks.workunit.client.0.smithi067.stderr:111100K .......... .......... .......... .......... .......... 81% 23.0M 1s 2024-03-30T02:38:36.766 INFO:tasks.workunit.client.0.smithi067.stderr:111150K .......... .......... .......... .......... .......... 81% 28.3M 1s 2024-03-30T02:38:36.766 INFO:tasks.workunit.client.0.smithi067.stderr:111200K .......... .......... .......... .......... .......... 81% 62.9M 1s 2024-03-30T02:38:36.768 INFO:tasks.workunit.client.0.smithi067.stderr:111250K .......... .......... .......... .......... .......... 81% 24.8M 1s 2024-03-30T02:38:36.770 INFO:tasks.workunit.client.0.smithi067.stderr:111300K .......... .......... .......... .......... .......... 82% 29.4M 1s 2024-03-30T02:38:36.772 INFO:tasks.workunit.client.0.smithi067.stderr:111350K .......... .......... .......... .......... .......... 82% 20.5M 1s 2024-03-30T02:38:36.773 INFO:tasks.workunit.client.0.smithi067.stderr:111400K .......... .......... .......... .......... .......... 82% 29.0M 1s 2024-03-30T02:38:36.774 INFO:tasks.workunit.client.0.smithi067.stderr:111450K .......... .......... .......... .......... .......... 82% 40.0M 1s 2024-03-30T02:38:36.776 INFO:tasks.workunit.client.0.smithi067.stderr:111500K .......... .......... .......... .......... .......... 82% 29.4M 1s 2024-03-30T02:38:36.779 INFO:tasks.workunit.client.0.smithi067.stderr:111550K .......... .......... .......... .......... .......... 82% 22.1M 1s 2024-03-30T02:38:36.781 INFO:tasks.workunit.client.0.smithi067.stderr:111600K .......... .......... .......... .......... .......... 82% 26.2M 1s 2024-03-30T02:38:36.781 INFO:tasks.workunit.client.0.smithi067.stderr:111650K .......... .......... .......... .......... .......... 82% 98.3M 1s 2024-03-30T02:38:36.783 INFO:tasks.workunit.client.0.smithi067.stderr:111700K .......... .......... .......... .......... .......... 82% 25.8M 1s 2024-03-30T02:38:36.785 INFO:tasks.workunit.client.0.smithi067.stderr:111750K .......... .......... .......... .......... .......... 82% 24.1M 1s 2024-03-30T02:38:36.787 INFO:tasks.workunit.client.0.smithi067.stderr:111800K .......... .......... .......... .......... .......... 82% 24.2M 1s 2024-03-30T02:38:36.789 INFO:tasks.workunit.client.0.smithi067.stderr:111850K .......... .......... .......... .......... .......... 82% 25.3M 1s 2024-03-30T02:38:36.789 INFO:tasks.workunit.client.0.smithi067.stderr:111900K .......... .......... .......... .......... .......... 82% 89.6M 1s 2024-03-30T02:38:36.791 INFO:tasks.workunit.client.0.smithi067.stderr:111950K .......... .......... .......... .......... .......... 82% 25.7M 1s 2024-03-30T02:38:36.793 INFO:tasks.workunit.client.0.smithi067.stderr:112000K .......... .......... .......... .......... .......... 82% 23.6M 1s 2024-03-30T02:38:36.795 INFO:tasks.workunit.client.0.smithi067.stderr:112050K .......... .......... .......... .......... .......... 82% 29.4M 1s 2024-03-30T02:38:36.796 INFO:tasks.workunit.client.0.smithi067.stderr:112100K .......... .......... .......... .......... .......... 82% 37.1M 1s 2024-03-30T02:38:36.798 INFO:tasks.workunit.client.0.smithi067.stderr:112150K .......... .......... .......... .......... .......... 82% 26.6M 1s 2024-03-30T02:38:36.800 INFO:tasks.workunit.client.0.smithi067.stderr:112200K .......... .......... .......... .......... .......... 82% 27.3M 1s 2024-03-30T02:38:36.802 INFO:tasks.workunit.client.0.smithi067.stderr:112250K .......... .......... .......... .......... .......... 82% 21.8M 1s 2024-03-30T02:38:36.804 INFO:tasks.workunit.client.0.smithi067.stderr:112300K .......... .......... .......... .......... .......... 82% 36.4M 1s 2024-03-30T02:38:36.804 INFO:tasks.workunit.client.0.smithi067.stderr:112350K .......... .......... .......... .......... .......... 82% 50.0M 1s 2024-03-30T02:38:36.806 INFO:tasks.workunit.client.0.smithi067.stderr:112400K .......... .......... .......... .......... .......... 82% 26.4M 1s 2024-03-30T02:38:36.808 INFO:tasks.workunit.client.0.smithi067.stderr:112450K .......... .......... .......... .......... .......... 82% 23.4M 1s 2024-03-30T02:38:36.810 INFO:tasks.workunit.client.0.smithi067.stderr:112500K .......... .......... .......... .......... .......... 82% 23.6M 1s 2024-03-30T02:38:36.812 INFO:tasks.workunit.client.0.smithi067.stderr:112550K .......... .......... .......... .......... .......... 82% 30.7M 1s 2024-03-30T02:38:36.812 INFO:tasks.workunit.client.0.smithi067.stderr:112600K .......... .......... .......... .......... .......... 82% 53.1M 1s 2024-03-30T02:38:36.815 INFO:tasks.workunit.client.0.smithi067.stderr:112650K .......... .......... .......... .......... .......... 83% 24.9M 1s 2024-03-30T02:38:36.817 INFO:tasks.workunit.client.0.smithi067.stderr:112700K .......... .......... .......... .......... .......... 83% 22.9M 1s 2024-03-30T02:38:36.837 INFO:tasks.workunit.client.0.smithi067.stderr:112750K .......... .......... .......... .......... .......... 83% 3.97M 1s 2024-03-30T02:38:36.837 INFO:tasks.workunit.client.0.smithi067.stderr:112800K .......... .......... .......... .......... .......... 83% 94.8M 1s 2024-03-30T02:38:36.837 INFO:tasks.workunit.client.0.smithi067.stderr:112850K .......... .......... .......... .......... .......... 83% 86.3M 1s 2024-03-30T02:38:36.837 INFO:tasks.workunit.client.0.smithi067.stderr:112900K .......... .......... .......... .......... .......... 83% 102M 1s 2024-03-30T02:38:36.837 INFO:tasks.workunit.client.0.smithi067.stderr:112950K .......... .......... .......... .......... .......... 83% 101M 1s 2024-03-30T02:38:36.837 INFO:tasks.workunit.client.0.smithi067.stderr:113000K .......... .......... .......... .......... .......... 83% 97.8M 1s 2024-03-30T02:38:36.837 INFO:tasks.workunit.client.0.smithi067.stderr:113050K .......... .......... .......... .......... .......... 83% 86.9M 1s 2024-03-30T02:38:36.837 INFO:tasks.workunit.client.0.smithi067.stderr:113100K .......... .......... .......... .......... .......... 83% 100M 1s 2024-03-30T02:38:36.838 INFO:tasks.workunit.client.0.smithi067.stderr:113150K .......... .......... .......... .......... .......... 83% 43.7M 1s 2024-03-30T02:38:36.838 INFO:tasks.workunit.client.0.smithi067.stderr:113200K .......... .......... .......... .......... .......... 83% 29.0M 1s 2024-03-30T02:38:36.838 INFO:tasks.workunit.client.0.smithi067.stderr:113250K .......... .......... .......... .......... .......... 83% 44.9M 1s 2024-03-30T02:38:36.840 INFO:tasks.workunit.client.0.smithi067.stderr:113300K .......... .......... .......... .......... .......... 83% 27.2M 1s 2024-03-30T02:38:36.840 INFO:tasks.workunit.client.0.smithi067.stderr:113350K .......... .......... .......... .......... .......... 83% 31.0M 1s 2024-03-30T02:38:36.842 INFO:tasks.workunit.client.0.smithi067.stderr:113400K .......... .......... .......... .......... .......... 83% 20.5M 1s 2024-03-30T02:38:36.845 INFO:tasks.workunit.client.0.smithi067.stderr:113450K .......... .......... .......... .......... .......... 83% 29.2M 1s 2024-03-30T02:38:36.845 INFO:tasks.workunit.client.0.smithi067.stderr:113500K .......... .......... .......... .......... .......... 83% 44.7M 1s 2024-03-30T02:38:36.847 INFO:tasks.workunit.client.0.smithi067.stderr:113550K .......... .......... .......... .......... .......... 83% 27.5M 1s 2024-03-30T02:38:36.849 INFO:tasks.workunit.client.0.smithi067.stderr:113600K .......... .......... .......... .......... .......... 83% 28.7M 1s 2024-03-30T02:38:36.851 INFO:tasks.workunit.client.0.smithi067.stderr:113650K .......... .......... .......... .......... .......... 83% 19.9M 1s 2024-03-30T02:38:36.853 INFO:tasks.workunit.client.0.smithi067.stderr:113700K .......... .......... .......... .......... .......... 83% 31.0M 1s 2024-03-30T02:38:36.853 INFO:tasks.workunit.client.0.smithi067.stderr:113750K .......... .......... .......... .......... .......... 83% 48.4M 1s 2024-03-30T02:38:36.855 INFO:tasks.workunit.client.0.smithi067.stderr:113800K .......... .......... .......... .......... .......... 83% 27.5M 1s 2024-03-30T02:38:36.858 INFO:tasks.workunit.client.0.smithi067.stderr:113850K .......... .......... .......... .......... .......... 83% 26.1M 1s 2024-03-30T02:38:36.862 INFO:tasks.workunit.client.0.smithi067.stderr:113900K .......... .......... .......... .......... .......... 83% 11.3M 1s 2024-03-30T02:38:36.864 INFO:tasks.workunit.client.0.smithi067.stderr:113950K .......... .......... .......... .......... .......... 83% 30.1M 1s 2024-03-30T02:38:36.864 INFO:tasks.workunit.client.0.smithi067.stderr:114000K .......... .......... .......... .......... .......... 84% 57.0M 1s 2024-03-30T02:38:36.866 INFO:tasks.workunit.client.0.smithi067.stderr:114050K .......... .......... .......... .......... .......... 84% 24.5M 1s 2024-03-30T02:38:36.868 INFO:tasks.workunit.client.0.smithi067.stderr:114100K .......... .......... .......... .......... .......... 84% 29.0M 1s 2024-03-30T02:38:36.868 INFO:tasks.workunit.client.0.smithi067.stderr:114150K .......... .......... .......... .......... .......... 84% 55.3M 1s 2024-03-30T02:38:36.870 INFO:tasks.workunit.client.0.smithi067.stderr:114200K .......... .......... .......... .......... .......... 84% 24.0M 1s 2024-03-30T02:38:36.872 INFO:tasks.workunit.client.0.smithi067.stderr:114250K .......... .......... .......... .......... .......... 84% 33.1M 1s 2024-03-30T02:38:36.875 INFO:tasks.workunit.client.0.smithi067.stderr:114300K .......... .......... .......... .......... .......... 84% 20.3M 1s 2024-03-30T02:38:36.877 INFO:tasks.workunit.client.0.smithi067.stderr:114350K .......... .......... .......... .......... .......... 84% 26.3M 1s 2024-03-30T02:38:36.877 INFO:tasks.workunit.client.0.smithi067.stderr:114400K .......... .......... .......... .......... .......... 84% 50.6M 1s 2024-03-30T02:38:36.879 INFO:tasks.workunit.client.0.smithi067.stderr:114450K .......... .......... .......... .......... .......... 84% 26.5M 1s 2024-03-30T02:38:36.881 INFO:tasks.workunit.client.0.smithi067.stderr:114500K .......... .......... .......... .......... .......... 84% 29.2M 1s 2024-03-30T02:38:36.883 INFO:tasks.workunit.client.0.smithi067.stderr:114550K .......... .......... .......... .......... .......... 84% 21.6M 1s 2024-03-30T02:38:36.885 INFO:tasks.workunit.client.0.smithi067.stderr:114600K .......... .......... .......... .......... .......... 84% 29.9M 1s 2024-03-30T02:38:36.885 INFO:tasks.workunit.client.0.smithi067.stderr:114650K .......... .......... .......... .......... .......... 84% 59.5M 1s 2024-03-30T02:38:36.887 INFO:tasks.workunit.client.0.smithi067.stderr:114700K .......... .......... .......... .......... .......... 84% 24.3M 1s 2024-03-30T02:38:36.889 INFO:tasks.workunit.client.0.smithi067.stderr:114750K .......... .......... .......... .......... .......... 84% 29.9M 1s 2024-03-30T02:38:36.891 INFO:tasks.workunit.client.0.smithi067.stderr:114800K .......... .......... .......... .......... .......... 84% 30.2M 1s 2024-03-30T02:38:36.892 INFO:tasks.workunit.client.0.smithi067.stderr:114850K .......... .......... .......... .......... .......... 84% 35.8M 1s 2024-03-30T02:38:36.895 INFO:tasks.workunit.client.0.smithi067.stderr:114900K .......... .......... .......... .......... .......... 84% 20.2M 1s 2024-03-30T02:38:36.897 INFO:tasks.workunit.client.0.smithi067.stderr:114950K .......... .......... .......... .......... .......... 84% 23.6M 1s 2024-03-30T02:38:36.899 INFO:tasks.workunit.client.0.smithi067.stderr:115000K .......... .......... .......... .......... .......... 84% 23.3M 1s 2024-03-30T02:38:36.901 INFO:tasks.workunit.client.0.smithi067.stderr:115050K .......... .......... .......... .......... .......... 84% 26.9M 1s 2024-03-30T02:38:36.901 INFO:tasks.workunit.client.0.smithi067.stderr:115100K .......... .......... .......... .......... .......... 84% 101M 1s 2024-03-30T02:38:36.903 INFO:tasks.workunit.client.0.smithi067.stderr:115150K .......... .......... .......... .......... .......... 84% 24.6M 1s 2024-03-30T02:38:36.904 INFO:tasks.workunit.client.0.smithi067.stderr:115200K .......... .......... .......... .......... .......... 84% 32.1M 1s 2024-03-30T02:38:36.908 INFO:tasks.workunit.client.0.smithi067.stderr:115250K .......... .......... .......... .......... .......... 84% 22.7M 1s 2024-03-30T02:38:36.908 INFO:tasks.workunit.client.0.smithi067.stderr:115300K .......... .......... .......... .......... .......... 84% 101M 1s 2024-03-30T02:38:36.909 INFO:tasks.workunit.client.0.smithi067.stderr:115350K .......... .......... .......... .......... .......... 85% 20.9M 1s 2024-03-30T02:38:36.912 INFO:tasks.workunit.client.0.smithi067.stderr:115400K .......... .......... .......... .......... .......... 85% 23.9M 1s 2024-03-30T02:38:36.914 INFO:tasks.workunit.client.0.smithi067.stderr:115450K .......... .......... .......... .......... .......... 85% 24.7M 1s 2024-03-30T02:38:36.916 INFO:tasks.workunit.client.0.smithi067.stderr:115500K .......... .......... .......... .......... .......... 85% 30.8M 1s 2024-03-30T02:38:36.916 INFO:tasks.workunit.client.0.smithi067.stderr:115550K .......... .......... .......... .......... .......... 85% 82.4M 1s 2024-03-30T02:38:36.917 INFO:tasks.workunit.client.0.smithi067.stderr:115600K .......... .......... .......... .......... .......... 85% 24.9M 1s 2024-03-30T02:38:36.920 INFO:tasks.workunit.client.0.smithi067.stderr:115650K .......... .......... .......... .......... .......... 85% 30.6M 1s 2024-03-30T02:38:36.922 INFO:tasks.workunit.client.0.smithi067.stderr:115700K .......... .......... .......... .......... .......... 85% 20.4M 1s 2024-03-30T02:38:36.923 INFO:tasks.workunit.client.0.smithi067.stderr:115750K .......... .......... .......... .......... .......... 85% 31.1M 1s 2024-03-30T02:38:36.924 INFO:tasks.workunit.client.0.smithi067.stderr:115800K .......... .......... .......... .......... .......... 85% 37.7M 1s 2024-03-30T02:38:36.927 INFO:tasks.workunit.client.0.smithi067.stderr:115850K .......... .......... .......... .......... .......... 85% 25.5M 1s 2024-03-30T02:38:36.929 INFO:tasks.workunit.client.0.smithi067.stderr:115900K .......... .......... .......... .......... .......... 85% 23.5M 1s 2024-03-30T02:38:36.931 INFO:tasks.workunit.client.0.smithi067.stderr:115950K .......... .......... .......... .......... .......... 85% 26.0M 1s 2024-03-30T02:38:36.931 INFO:tasks.workunit.client.0.smithi067.stderr:116000K .......... .......... .......... .......... .......... 85% 98.5M 1s 2024-03-30T02:38:36.933 INFO:tasks.workunit.client.0.smithi067.stderr:116050K .......... .......... .......... .......... .......... 85% 26.5M 1s 2024-03-30T02:38:36.935 INFO:tasks.workunit.client.0.smithi067.stderr:116100K .......... .......... .......... .......... .......... 85% 24.6M 1s 2024-03-30T02:38:36.937 INFO:tasks.workunit.client.0.smithi067.stderr:116150K .......... .......... .......... .......... .......... 85% 27.2M 1s 2024-03-30T02:38:36.937 INFO:tasks.workunit.client.0.smithi067.stderr:116200K .......... .......... .......... .......... .......... 85% 66.0M 1s 2024-03-30T02:38:36.939 INFO:tasks.workunit.client.0.smithi067.stderr:116250K .......... .......... .......... .......... .......... 85% 24.6M 1s 2024-03-30T02:38:36.941 INFO:tasks.workunit.client.0.smithi067.stderr:116300K .......... .......... .......... .......... .......... 85% 25.2M 1s 2024-03-30T02:38:36.944 INFO:tasks.workunit.client.0.smithi067.stderr:116350K .......... .......... .......... .......... .......... 85% 22.1M 1s 2024-03-30T02:38:36.946 INFO:tasks.workunit.client.0.smithi067.stderr:116400K .......... .......... .......... .......... .......... 85% 26.3M 1s 2024-03-30T02:38:36.946 INFO:tasks.workunit.client.0.smithi067.stderr:116450K .......... .......... .......... .......... .......... 85% 83.9M 1s 2024-03-30T02:38:36.948 INFO:tasks.workunit.client.0.smithi067.stderr:116500K .......... .......... .......... .......... .......... 85% 22.9M 1s 2024-03-30T02:38:36.950 INFO:tasks.workunit.client.0.smithi067.stderr:116550K .......... .......... .......... .......... .......... 85% 26.6M 1s 2024-03-30T02:38:36.952 INFO:tasks.workunit.client.0.smithi067.stderr:116600K .......... .......... .......... .......... .......... 85% 26.0M 1s 2024-03-30T02:38:36.953 INFO:tasks.workunit.client.0.smithi067.stderr:116650K .......... .......... .......... .......... .......... 85% 31.8M 1s 2024-03-30T02:38:36.955 INFO:tasks.workunit.client.0.smithi067.stderr:116700K .......... .......... .......... .......... .......... 86% 34.0M 1s 2024-03-30T02:38:36.956 INFO:tasks.workunit.client.0.smithi067.stderr:116750K .......... .......... .......... .......... .......... 86% 27.1M 1s 2024-03-30T02:38:36.958 INFO:tasks.workunit.client.0.smithi067.stderr:116800K .......... .......... .......... .......... .......... 86% 23.6M 1s 2024-03-30T02:38:36.960 INFO:tasks.workunit.client.0.smithi067.stderr:116850K .......... .......... .......... .......... .......... 86% 27.0M 1s 2024-03-30T02:38:36.963 INFO:tasks.workunit.client.0.smithi067.stderr:116900K .......... .......... .......... .......... .......... 86% 33.8M 1s 2024-03-30T02:38:36.963 INFO:tasks.workunit.client.0.smithi067.stderr:116950K .......... .......... .......... .......... .......... 86% 36.0M 1s 2024-03-30T02:38:36.965 INFO:tasks.workunit.client.0.smithi067.stderr:117000K .......... .......... .......... .......... .......... 86% 28.5M 1s 2024-03-30T02:38:36.966 INFO:tasks.workunit.client.0.smithi067.stderr:117050K .......... .......... .......... .......... .......... 86% 37.1M 1s 2024-03-30T02:38:36.971 INFO:tasks.workunit.client.0.smithi067.stderr:117100K .......... .......... .......... .......... .......... 86% 12.8M 1s 2024-03-30T02:38:36.971 INFO:tasks.workunit.client.0.smithi067.stderr:117150K .......... .......... .......... .......... .......... 86% 49.1M 1s 2024-03-30T02:38:36.976 INFO:tasks.workunit.client.0.smithi067.stderr:117200K .......... .......... .......... .......... .......... 86% 15.9M 1s 2024-03-30T02:38:36.976 INFO:tasks.workunit.client.0.smithi067.stderr:117250K .......... .......... .......... .......... .......... 86% 95.5M 1s 2024-03-30T02:38:36.977 INFO:tasks.workunit.client.0.smithi067.stderr:117300K .......... .......... .......... .......... .......... 86% 99.9M 1s 2024-03-30T02:38:36.977 INFO:tasks.workunit.client.0.smithi067.stderr:117350K .......... .......... .......... .......... .......... 86% 41.4M 1s 2024-03-30T02:38:36.984 INFO:tasks.workunit.client.0.smithi067.stderr:117400K .......... .......... .......... .......... .......... 86% 6.89M 1s 2024-03-30T02:38:36.986 INFO:tasks.workunit.client.0.smithi067.stderr:117450K .......... .......... .......... .......... .......... 86% 20.9M 1s 2024-03-30T02:38:36.991 INFO:tasks.workunit.client.0.smithi067.stderr:117500K .......... .......... .......... .......... .......... 86% 23.3M 1s 2024-03-30T02:38:36.991 INFO:tasks.workunit.client.0.smithi067.stderr:117550K .......... .......... .......... .......... .......... 86% 59.4M 1s 2024-03-30T02:38:36.992 INFO:tasks.workunit.client.0.smithi067.stderr:117600K .......... .......... .......... .......... .......... 86% 100M 1s 2024-03-30T02:38:36.992 INFO:tasks.workunit.client.0.smithi067.stderr:117650K .......... .......... .......... .......... .......... 86% 102M 1s 2024-03-30T02:38:36.992 INFO:tasks.workunit.client.0.smithi067.stderr:117700K .......... .......... .......... .......... .......... 86% 77.9M 1s 2024-03-30T02:38:36.994 INFO:tasks.workunit.client.0.smithi067.stderr:117750K .......... .......... .......... .......... .......... 86% 23.4M 1s 2024-03-30T02:38:36.994 INFO:tasks.workunit.client.0.smithi067.stderr:117800K .......... .......... .......... .......... .......... 86% 30.9M 1s 2024-03-30T02:38:36.996 INFO:tasks.workunit.client.0.smithi067.stderr:117850K .......... .......... .......... .......... .......... 86% 27.9M 1s 2024-03-30T02:38:36.997 INFO:tasks.workunit.client.0.smithi067.stderr:117900K .......... .......... .......... .......... .......... 86% 40.3M 1s 2024-03-30T02:38:36.998 INFO:tasks.workunit.client.0.smithi067.stderr:117950K .......... .......... .......... .......... .......... 86% 32.0M 1s 2024-03-30T02:38:37.001 INFO:tasks.workunit.client.0.smithi067.stderr:118000K .......... .......... .......... .......... .......... 86% 18.1M 1s 2024-03-30T02:38:37.004 INFO:tasks.workunit.client.0.smithi067.stderr:118050K .......... .......... .......... .......... .......... 86% 29.9M 1s 2024-03-30T02:38:37.004 INFO:tasks.workunit.client.0.smithi067.stderr:118100K .......... .......... .......... .......... .......... 87% 79.2M 1s 2024-03-30T02:38:37.006 INFO:tasks.workunit.client.0.smithi067.stderr:118150K .......... .......... .......... .......... .......... 87% 23.0M 1s 2024-03-30T02:38:37.008 INFO:tasks.workunit.client.0.smithi067.stderr:118200K .......... .......... .......... .......... .......... 87% 24.4M 1s 2024-03-30T02:38:37.009 INFO:tasks.workunit.client.0.smithi067.stderr:118250K .......... .......... .......... .......... .......... 87% 23.6M 1s 2024-03-30T02:38:37.012 INFO:tasks.workunit.client.0.smithi067.stderr:118300K .......... .......... .......... .......... .......... 87% 30.3M 1s 2024-03-30T02:38:37.012 INFO:tasks.workunit.client.0.smithi067.stderr:118350K .......... .......... .......... .......... .......... 87% 84.0M 1s 2024-03-30T02:38:37.014 INFO:tasks.workunit.client.0.smithi067.stderr:118400K .......... .......... .......... .......... .......... 87% 21.2M 1s 2024-03-30T02:38:37.015 INFO:tasks.workunit.client.0.smithi067.stderr:118450K .......... .......... .......... .......... .......... 87% 32.5M 1s 2024-03-30T02:38:37.017 INFO:tasks.workunit.client.0.smithi067.stderr:118500K .......... .......... .......... .......... .......... 87% 32.4M 1s 2024-03-30T02:38:37.019 INFO:tasks.workunit.client.0.smithi067.stderr:118550K .......... .......... .......... .......... .......... 87% 23.6M 1s 2024-03-30T02:38:37.021 INFO:tasks.workunit.client.0.smithi067.stderr:118600K .......... .......... .......... .......... .......... 87% 29.5M 1s 2024-03-30T02:38:37.023 INFO:tasks.workunit.client.0.smithi067.stderr:118650K .......... .......... .......... .......... .......... 87% 27.2M 1s 2024-03-30T02:38:37.024 INFO:tasks.workunit.client.0.smithi067.stderr:118700K .......... .......... .......... .......... .......... 87% 33.6M 1s 2024-03-30T02:38:37.025 INFO:tasks.workunit.client.0.smithi067.stderr:118750K .......... .......... .......... .......... .......... 87% 32.7M 1s 2024-03-30T02:38:37.027 INFO:tasks.workunit.client.0.smithi067.stderr:118800K .......... .......... .......... .......... .......... 87% 26.4M 1s 2024-03-30T02:38:37.030 INFO:tasks.workunit.client.0.smithi067.stderr:118850K .......... .......... .......... .......... .......... 87% 19.9M 1s 2024-03-30T02:38:37.031 INFO:tasks.workunit.client.0.smithi067.stderr:118900K .......... .......... .......... .......... .......... 87% 31.2M 1s 2024-03-30T02:38:37.032 INFO:tasks.workunit.client.0.smithi067.stderr:118950K .......... .......... .......... .......... .......... 87% 36.2M 1s 2024-03-30T02:38:37.034 INFO:tasks.workunit.client.0.smithi067.stderr:119000K .......... .......... .......... .......... .......... 87% 30.4M 1s 2024-03-30T02:38:37.035 INFO:tasks.workunit.client.0.smithi067.stderr:119050K .......... .......... .......... .......... .......... 87% 34.7M 1s 2024-03-30T02:38:37.039 INFO:tasks.workunit.client.0.smithi067.stderr:119100K .......... .......... .......... .......... .......... 87% 19.2M 1s 2024-03-30T02:38:37.040 INFO:tasks.workunit.client.0.smithi067.stderr:119150K .......... .......... .......... .......... .......... 87% 25.5M 1s 2024-03-30T02:38:37.042 INFO:tasks.workunit.client.0.smithi067.stderr:119200K .......... .......... .......... .......... .......... 87% 36.2M 1s 2024-03-30T02:38:37.044 INFO:tasks.workunit.client.0.smithi067.stderr:119250K .......... .......... .......... .......... .......... 87% 28.1M 1s 2024-03-30T02:38:37.044 INFO:tasks.workunit.client.0.smithi067.stderr:119300K .......... .......... .......... .......... .......... 87% 66.9M 1s 2024-03-30T02:38:37.047 INFO:tasks.workunit.client.0.smithi067.stderr:119350K .......... .......... .......... .......... .......... 87% 19.3M 1s 2024-03-30T02:38:37.047 INFO:tasks.workunit.client.0.smithi067.stderr:119400K .......... .......... .......... .......... .......... 87% 76.8M 1s 2024-03-30T02:38:37.049 INFO:tasks.workunit.client.0.smithi067.stderr:119450K .......... .......... .......... .......... .......... 88% 20.0M 1s 2024-03-30T02:38:37.051 INFO:tasks.workunit.client.0.smithi067.stderr:119500K .......... .......... .......... .......... .......... 88% 38.6M 1s 2024-03-30T02:38:37.053 INFO:tasks.workunit.client.0.smithi067.stderr:119550K .......... .......... .......... .......... .......... 88% 25.0M 1s 2024-03-30T02:38:37.054 INFO:tasks.workunit.client.0.smithi067.stderr:119600K .......... .......... .......... .......... .......... 88% 29.0M 1s 2024-03-30T02:38:37.055 INFO:tasks.workunit.client.0.smithi067.stderr:119650K .......... .......... .......... .......... .......... 88% 37.1M 1s 2024-03-30T02:38:37.057 INFO:tasks.workunit.client.0.smithi067.stderr:119700K .......... .......... .......... .......... .......... 88% 25.8M 1s 2024-03-30T02:38:37.060 INFO:tasks.workunit.client.0.smithi067.stderr:119750K .......... .......... .......... .......... .......... 88% 20.7M 1s 2024-03-30T02:38:37.062 INFO:tasks.workunit.client.0.smithi067.stderr:119800K .......... .......... .......... .......... .......... 88% 28.6M 1s 2024-03-30T02:38:37.063 INFO:tasks.workunit.client.0.smithi067.stderr:119850K .......... .......... .......... .......... .......... 88% 50.3M 1s 2024-03-30T02:38:37.065 INFO:tasks.workunit.client.0.smithi067.stderr:119900K .......... .......... .......... .......... .......... 88% 23.7M 1s 2024-03-30T02:38:37.066 INFO:tasks.workunit.client.0.smithi067.stderr:119950K .......... .......... .......... .......... .......... 88% 28.5M 1s 2024-03-30T02:38:37.068 INFO:tasks.workunit.client.0.smithi067.stderr:120000K .......... .......... .......... .......... .......... 88% 38.7M 1s 2024-03-30T02:38:37.070 INFO:tasks.workunit.client.0.smithi067.stderr:120050K .......... .......... .......... .......... .......... 88% 20.6M 1s 2024-03-30T02:38:37.072 INFO:tasks.workunit.client.0.smithi067.stderr:120100K .......... .......... .......... .......... .......... 88% 31.0M 1s 2024-03-30T02:38:37.073 INFO:tasks.workunit.client.0.smithi067.stderr:120150K .......... .......... .......... .......... .......... 88% 37.1M 1s 2024-03-30T02:38:37.075 INFO:tasks.workunit.client.0.smithi067.stderr:120200K .......... .......... .......... .......... .......... 88% 25.1M 1s 2024-03-30T02:38:37.076 INFO:tasks.workunit.client.0.smithi067.stderr:120250K .......... .......... .......... .......... .......... 88% 37.0M 1s 2024-03-30T02:38:37.079 INFO:tasks.workunit.client.0.smithi067.stderr:120300K .......... .......... .......... .......... .......... 88% 20.2M 1s 2024-03-30T02:38:37.080 INFO:tasks.workunit.client.0.smithi067.stderr:120350K .......... .......... .......... .......... .......... 88% 31.9M 1s 2024-03-30T02:38:37.087 INFO:tasks.workunit.client.0.smithi067.stderr:120400K .......... .......... .......... .......... .......... 88% 11.7M 1s 2024-03-30T02:38:37.087 INFO:tasks.workunit.client.0.smithi067.stderr:120450K .......... .......... .......... .......... .......... 88% 97.5M 1s 2024-03-30T02:38:37.087 INFO:tasks.workunit.client.0.smithi067.stderr:120500K .......... .......... .......... .......... .......... 88% 112M 1s 2024-03-30T02:38:37.087 INFO:tasks.workunit.client.0.smithi067.stderr:120550K .......... .......... .......... .......... .......... 88% 59.8M 1s 2024-03-30T02:38:37.089 INFO:tasks.workunit.client.0.smithi067.stderr:120600K .......... .......... .......... .......... .......... 88% 27.5M 1s 2024-03-30T02:38:37.090 INFO:tasks.workunit.client.0.smithi067.stderr:120650K .......... .......... .......... .......... .......... 88% 22.1M 1s 2024-03-30T02:38:37.092 INFO:tasks.workunit.client.0.smithi067.stderr:120700K .......... .......... .......... .......... .......... 88% 29.0M 1s 2024-03-30T02:38:37.093 INFO:tasks.workunit.client.0.smithi067.stderr:120750K .......... .......... .......... .......... .......... 88% 30.1M 1s 2024-03-30T02:38:37.094 INFO:tasks.workunit.client.0.smithi067.stderr:120800K .......... .......... .......... .......... .......... 89% 40.0M 1s 2024-03-30T02:38:37.098 INFO:tasks.workunit.client.0.smithi067.stderr:120850K .......... .......... .......... .......... .......... 89% 13.2M 1s 2024-03-30T02:38:37.101 INFO:tasks.workunit.client.0.smithi067.stderr:120900K .......... .......... .......... .......... .......... 89% 28.3M 1s 2024-03-30T02:38:37.105 INFO:tasks.workunit.client.0.smithi067.stderr:120950K .......... .......... .......... .......... .......... 89% 19.3M 1s 2024-03-30T02:38:37.105 INFO:tasks.workunit.client.0.smithi067.stderr:121000K .......... .......... .......... .......... .......... 89% 94.2M 1s 2024-03-30T02:38:37.108 INFO:tasks.workunit.client.0.smithi067.stderr:121050K .......... .......... .......... .......... .......... 89% 25.7M 1s 2024-03-30T02:38:37.108 INFO:tasks.workunit.client.0.smithi067.stderr:121100K .......... .......... .......... .......... .......... 89% 22.0M 1s 2024-03-30T02:38:37.109 INFO:tasks.workunit.client.0.smithi067.stderr:121150K .......... .......... .......... .......... .......... 89% 28.8M 1s 2024-03-30T02:38:37.111 INFO:tasks.workunit.client.0.smithi067.stderr:121200K .......... .......... .......... .......... .......... 89% 24.5M 1s 2024-03-30T02:38:37.113 INFO:tasks.workunit.client.0.smithi067.stderr:121250K .......... .......... .......... .......... .......... 89% 31.0M 1s 2024-03-30T02:38:37.114 INFO:tasks.workunit.client.0.smithi067.stderr:121300K .......... .......... .......... .......... .......... 89% 37.3M 1s 2024-03-30T02:38:37.115 INFO:tasks.workunit.client.0.smithi067.stderr:121350K .......... .......... .......... .......... .......... 89% 29.6M 1s 2024-03-30T02:38:37.118 INFO:tasks.workunit.client.0.smithi067.stderr:121400K .......... .......... .......... .......... .......... 89% 22.1M 1s 2024-03-30T02:38:37.120 INFO:tasks.workunit.client.0.smithi067.stderr:121450K .......... .......... .......... .......... .......... 89% 26.4M 1s 2024-03-30T02:38:37.121 INFO:tasks.workunit.client.0.smithi067.stderr:121500K .......... .......... .......... .......... .......... 89% 31.7M 1s 2024-03-30T02:38:37.122 INFO:tasks.workunit.client.0.smithi067.stderr:121550K .......... .......... .......... .......... .......... 89% 36.8M 1s 2024-03-30T02:38:37.124 INFO:tasks.workunit.client.0.smithi067.stderr:121600K .......... .......... .......... .......... .......... 89% 29.9M 1s 2024-03-30T02:38:37.126 INFO:tasks.workunit.client.0.smithi067.stderr:121650K .......... .......... .......... .......... .......... 89% 28.5M 1s 2024-03-30T02:38:37.127 INFO:tasks.workunit.client.0.smithi067.stderr:121700K .......... .......... .......... .......... .......... 89% 38.6M 1s 2024-03-30T02:38:37.130 INFO:tasks.workunit.client.0.smithi067.stderr:121750K .......... .......... .......... .......... .......... 89% 18.6M 1s 2024-03-30T02:38:37.131 INFO:tasks.workunit.client.0.smithi067.stderr:121800K .......... .......... .......... .......... .......... 89% 31.1M 1s 2024-03-30T02:38:37.132 INFO:tasks.workunit.client.0.smithi067.stderr:121850K .......... .......... .......... .......... .......... 89% 36.6M 1s 2024-03-30T02:38:37.134 INFO:tasks.workunit.client.0.smithi067.stderr:121900K .......... .......... .......... .......... .......... 89% 32.4M 1s 2024-03-30T02:38:37.135 INFO:tasks.workunit.client.0.smithi067.stderr:121950K .......... .......... .......... .......... .......... 89% 33.3M 1s 2024-03-30T02:38:37.137 INFO:tasks.workunit.client.0.smithi067.stderr:122000K .......... .......... .......... .......... .......... 89% 31.9M 1s 2024-03-30T02:38:37.140 INFO:tasks.workunit.client.0.smithi067.stderr:122050K .......... .......... .......... .......... .......... 89% 18.8M 1s 2024-03-30T02:38:37.141 INFO:tasks.workunit.client.0.smithi067.stderr:122100K .......... .......... .......... .......... .......... 89% 35.3M 1s 2024-03-30T02:38:37.143 INFO:tasks.workunit.client.0.smithi067.stderr:122150K .......... .......... .......... .......... .......... 90% 32.2M 1s 2024-03-30T02:38:37.144 INFO:tasks.workunit.client.0.smithi067.stderr:122200K .......... .......... .......... .......... .......... 90% 32.1M 1s 2024-03-30T02:38:37.145 INFO:tasks.workunit.client.0.smithi067.stderr:122250K .......... .......... .......... .......... .......... 90% 32.4M 1s 2024-03-30T02:38:37.147 INFO:tasks.workunit.client.0.smithi067.stderr:122300K .......... .......... .......... .......... .......... 90% 33.0M 1s 2024-03-30T02:38:37.150 INFO:tasks.workunit.client.0.smithi067.stderr:122350K .......... .......... .......... .......... .......... 90% 17.4M 1s 2024-03-30T02:38:37.151 INFO:tasks.workunit.client.0.smithi067.stderr:122400K .......... .......... .......... .......... .......... 90% 90.2M 1s 2024-03-30T02:38:37.152 INFO:tasks.workunit.client.0.smithi067.stderr:122450K .......... .......... .......... .......... .......... 90% 29.3M 1s 2024-03-30T02:38:37.154 INFO:tasks.workunit.client.0.smithi067.stderr:122500K .......... .......... .......... .......... .......... 90% 24.9M 1s 2024-03-30T02:38:37.155 INFO:tasks.workunit.client.0.smithi067.stderr:122550K .......... .......... .......... .......... .......... 90% 32.1M 1s 2024-03-30T02:38:37.158 INFO:tasks.workunit.client.0.smithi067.stderr:122600K .......... .......... .......... .......... .......... 90% 33.4M 1s 2024-03-30T02:38:37.160 INFO:tasks.workunit.client.0.smithi067.stderr:122650K .......... .......... .......... .......... .......... 90% 19.6M 1s 2024-03-30T02:38:37.161 INFO:tasks.workunit.client.0.smithi067.stderr:122700K .......... .......... .......... .......... .......... 90% 33.4M 1s 2024-03-30T02:38:37.163 INFO:tasks.workunit.client.0.smithi067.stderr:122750K .......... .......... .......... .......... .......... 90% 27.3M 1s 2024-03-30T02:38:37.164 INFO:tasks.workunit.client.0.smithi067.stderr:122800K .......... .......... .......... .......... .......... 90% 33.7M 1s 2024-03-30T02:38:37.165 INFO:tasks.workunit.client.0.smithi067.stderr:122850K .......... .......... .......... .......... .......... 90% 35.7M 1s 2024-03-30T02:38:37.168 INFO:tasks.workunit.client.0.smithi067.stderr:122900K .......... .......... .......... .......... .......... 90% 32.1M 1s 2024-03-30T02:38:37.169 INFO:tasks.workunit.client.0.smithi067.stderr:122950K .......... .......... .......... .......... .......... 90% 26.4M 1s 2024-03-30T02:38:37.171 INFO:tasks.workunit.client.0.smithi067.stderr:123000K .......... .......... .......... .......... .......... 90% 19.9M 1s 2024-03-30T02:38:37.173 INFO:tasks.workunit.client.0.smithi067.stderr:123050K .......... .......... .......... .......... .......... 90% 36.8M 1s 2024-03-30T02:38:37.174 INFO:tasks.workunit.client.0.smithi067.stderr:123100K .......... .......... .......... .......... .......... 90% 32.8M 0s 2024-03-30T02:38:37.176 INFO:tasks.workunit.client.0.smithi067.stderr:123150K .......... .......... .......... .......... .......... 90% 28.6M 0s 2024-03-30T02:38:37.179 INFO:tasks.workunit.client.0.smithi067.stderr:123200K .......... .......... .......... .......... .......... 90% 20.1M 0s 2024-03-30T02:38:37.180 INFO:tasks.workunit.client.0.smithi067.stderr:123250K .......... .......... .......... .......... .......... 90% 36.4M 0s 2024-03-30T02:38:37.183 INFO:tasks.workunit.client.0.smithi067.stderr:123300K .......... .......... .......... .......... .......... 90% 23.4M 0s 2024-03-30T02:38:37.183 INFO:tasks.workunit.client.0.smithi067.stderr:123350K .......... .......... .......... .......... .......... 90% 45.9M 0s 2024-03-30T02:38:37.186 INFO:tasks.workunit.client.0.smithi067.stderr:123400K .......... .......... .......... .......... .......... 90% 27.3M 0s 2024-03-30T02:38:37.186 INFO:tasks.workunit.client.0.smithi067.stderr:123450K .......... .......... .......... .......... .......... 90% 44.2M 0s 2024-03-30T02:38:37.188 INFO:tasks.workunit.client.0.smithi067.stderr:123500K .......... .......... .......... .......... .......... 91% 18.2M 0s 2024-03-30T02:38:37.190 INFO:tasks.workunit.client.0.smithi067.stderr:123550K .......... .......... .......... .......... .......... 91% 33.1M 0s 2024-03-30T02:38:37.192 INFO:tasks.workunit.client.0.smithi067.stderr:123600K .......... .......... .......... .......... .......... 91% 28.3M 0s 2024-03-30T02:38:37.193 INFO:tasks.workunit.client.0.smithi067.stderr:123650K .......... .......... .......... .......... .......... 91% 35.9M 0s 2024-03-30T02:38:37.195 INFO:tasks.workunit.client.0.smithi067.stderr:123700K .......... .......... .......... .......... .......... 91% 32.1M 0s 2024-03-30T02:38:37.196 INFO:tasks.workunit.client.0.smithi067.stderr:123750K .......... .......... .......... .......... .......... 91% 25.9M 0s 2024-03-30T02:38:37.198 INFO:tasks.workunit.client.0.smithi067.stderr:123800K .......... .......... .......... .......... .......... 91% 37.7M 0s 2024-03-30T02:38:37.200 INFO:tasks.workunit.client.0.smithi067.stderr:123850K .......... .......... .......... .......... .......... 91% 23.5M 0s 2024-03-30T02:38:37.202 INFO:tasks.workunit.client.0.smithi067.stderr:123900K .......... .......... .......... .......... .......... 91% 23.9M 0s 2024-03-30T02:38:37.204 INFO:tasks.workunit.client.0.smithi067.stderr:123950K .......... .......... .......... .......... .......... 91% 28.2M 0s 2024-03-30T02:38:37.204 INFO:tasks.workunit.client.0.smithi067.stderr:124000K .......... .......... .......... .......... .......... 91% 59.5M 0s 2024-03-30T02:38:37.206 INFO:tasks.workunit.client.0.smithi067.stderr:124050K .......... .......... .......... .......... .......... 91% 28.5M 0s 2024-03-30T02:38:37.209 INFO:tasks.workunit.client.0.smithi067.stderr:124100K .......... .......... .......... .......... .......... 91% 19.4M 0s 2024-03-30T02:38:37.210 INFO:tasks.workunit.client.0.smithi067.stderr:124150K .......... .......... .......... .......... .......... 91% 31.8M 0s 2024-03-30T02:38:37.211 INFO:tasks.workunit.client.0.smithi067.stderr:124200K .......... .......... .......... .......... .......... 91% 34.2M 0s 2024-03-30T02:38:37.216 INFO:tasks.workunit.client.0.smithi067.stderr:124250K .......... .......... .......... .......... .......... 91% 24.9M 0s 2024-03-30T02:38:37.216 INFO:tasks.workunit.client.0.smithi067.stderr:124300K .......... .......... .......... .......... .......... 91% 49.1M 0s 2024-03-30T02:38:37.217 INFO:tasks.workunit.client.0.smithi067.stderr:124350K .......... .......... .......... .......... .......... 91% 17.1M 0s 2024-03-30T02:38:37.219 INFO:tasks.workunit.client.0.smithi067.stderr:124400K .......... .......... .......... .......... .......... 91% 36.4M 0s 2024-03-30T02:38:37.220 INFO:tasks.workunit.client.0.smithi067.stderr:124450K .......... .......... .......... .......... .......... 91% 34.2M 0s 2024-03-30T02:38:37.222 INFO:tasks.workunit.client.0.smithi067.stderr:124500K .......... .......... .......... .......... .......... 91% 32.0M 0s 2024-03-30T02:38:37.225 INFO:tasks.workunit.client.0.smithi067.stderr:124550K .......... .......... .......... .......... .......... 91% 13.7M 0s 2024-03-30T02:38:37.226 INFO:tasks.workunit.client.0.smithi067.stderr:124600K .......... .......... .......... .......... .......... 91% 35.7M 0s 2024-03-30T02:38:37.230 INFO:tasks.workunit.client.0.smithi067.stderr:124650K .......... .......... .......... .......... .......... 91% 16.5M 0s 2024-03-30T02:38:37.231 INFO:tasks.workunit.client.0.smithi067.stderr:124700K .......... .......... .......... .......... .......... 91% 47.9M 0s 2024-03-30T02:38:37.232 INFO:tasks.workunit.client.0.smithi067.stderr:124750K .......... .......... .......... .......... .......... 91% 35.3M 0s 2024-03-30T02:38:37.234 INFO:tasks.workunit.client.0.smithi067.stderr:124800K .......... .......... .......... .......... .......... 91% 26.8M 0s 2024-03-30T02:38:37.237 INFO:tasks.workunit.client.0.smithi067.stderr:124850K .......... .......... .......... .......... .......... 92% 31.0M 0s 2024-03-30T02:38:37.237 INFO:tasks.workunit.client.0.smithi067.stderr:124900K .......... .......... .......... .......... .......... 92% 47.4M 0s 2024-03-30T02:38:37.239 INFO:tasks.workunit.client.0.smithi067.stderr:124950K .......... .......... .......... .......... .......... 92% 18.7M 0s 2024-03-30T02:38:37.241 INFO:tasks.workunit.client.0.smithi067.stderr:125000K .......... .......... .......... .......... .......... 92% 28.7M 0s 2024-03-30T02:38:37.242 INFO:tasks.workunit.client.0.smithi067.stderr:125050K .......... .......... .......... .......... .......... 92% 34.2M 0s 2024-03-30T02:38:37.243 INFO:tasks.workunit.client.0.smithi067.stderr:125100K .......... .......... .......... .......... .......... 92% 34.2M 0s 2024-03-30T02:38:37.245 INFO:tasks.workunit.client.0.smithi067.stderr:125150K .......... .......... .......... .......... .......... 92% 28.1M 0s 2024-03-30T02:38:37.248 INFO:tasks.workunit.client.0.smithi067.stderr:125200K .......... .......... .......... .......... .......... 92% 39.6M 0s 2024-03-30T02:38:37.250 INFO:tasks.workunit.client.0.smithi067.stderr:125250K .......... .......... .......... .......... .......... 92% 17.9M 0s 2024-03-30T02:38:37.253 INFO:tasks.workunit.client.0.smithi067.stderr:125300K .......... .......... .......... .......... .......... 92% 36.4M 0s 2024-03-30T02:38:37.253 INFO:tasks.workunit.client.0.smithi067.stderr:125350K .......... .......... .......... .......... .......... 92% 34.2M 0s 2024-03-30T02:38:37.253 INFO:tasks.workunit.client.0.smithi067.stderr:125400K .......... .......... .......... .......... .......... 92% 34.4M 0s 2024-03-30T02:38:37.255 INFO:tasks.workunit.client.0.smithi067.stderr:125450K .......... .......... .......... .......... .......... 92% 30.3M 0s 2024-03-30T02:38:37.258 INFO:tasks.workunit.client.0.smithi067.stderr:125500K .......... .......... .......... .......... .......... 92% 21.3M 0s 2024-03-30T02:38:37.260 INFO:tasks.workunit.client.0.smithi067.stderr:125550K .......... .......... .......... .......... .......... 92% 26.3M 0s 2024-03-30T02:38:37.260 INFO:tasks.workunit.client.0.smithi067.stderr:125600K .......... .......... .......... .......... .......... 92% 45.8M 0s 2024-03-30T02:38:37.262 INFO:tasks.workunit.client.0.smithi067.stderr:125650K .......... .......... .......... .......... .......... 92% 26.5M 0s 2024-03-30T02:38:37.264 INFO:tasks.workunit.client.0.smithi067.stderr:125700K .......... .......... .......... .......... .......... 92% 32.7M 0s 2024-03-30T02:38:37.265 INFO:tasks.workunit.client.0.smithi067.stderr:125750K .......... .......... .......... .......... .......... 92% 33.9M 0s 2024-03-30T02:38:37.268 INFO:tasks.workunit.client.0.smithi067.stderr:125800K .......... .......... .......... .......... .......... 92% 30.6M 0s 2024-03-30T02:38:37.269 INFO:tasks.workunit.client.0.smithi067.stderr:125850K .......... .......... .......... .......... .......... 92% 20.2M 0s 2024-03-30T02:38:37.271 INFO:tasks.workunit.client.0.smithi067.stderr:125900K .......... .......... .......... .......... .......... 92% 34.1M 0s 2024-03-30T02:38:37.272 INFO:tasks.workunit.client.0.smithi067.stderr:125950K .......... .......... .......... .......... .......... 92% 30.2M 0s 2024-03-30T02:38:37.273 INFO:tasks.workunit.client.0.smithi067.stderr:126000K .......... .......... .......... .......... .......... 92% 36.5M 0s 2024-03-30T02:38:37.275 INFO:tasks.workunit.client.0.smithi067.stderr:126050K .......... .......... .......... .......... .......... 92% 32.6M 0s 2024-03-30T02:38:37.277 INFO:tasks.workunit.client.0.smithi067.stderr:126100K .......... .......... .......... .......... .......... 92% 18.5M 0s 2024-03-30T02:38:37.279 INFO:tasks.workunit.client.0.smithi067.stderr:126150K .......... .......... .......... .......... .......... 92% 49.7M 0s 2024-03-30T02:38:37.281 INFO:tasks.workunit.client.0.smithi067.stderr:126200K .......... .......... .......... .......... .......... 92% 22.3M 0s 2024-03-30T02:38:37.284 INFO:tasks.workunit.client.0.smithi067.stderr:126250K .......... .......... .......... .......... .......... 93% 28.1M 0s 2024-03-30T02:38:37.284 INFO:tasks.workunit.client.0.smithi067.stderr:126300K .......... .......... .......... .......... .......... 93% 41.5M 0s 2024-03-30T02:38:37.285 INFO:tasks.workunit.client.0.smithi067.stderr:126350K .......... .......... .......... .......... .......... 93% 34.9M 0s 2024-03-30T02:38:37.288 INFO:tasks.workunit.client.0.smithi067.stderr:126400K .......... .......... .......... .......... .......... 93% 20.3M 0s 2024-03-30T02:38:37.289 INFO:tasks.workunit.client.0.smithi067.stderr:126450K .......... .......... .......... .......... .......... 93% 28.7M 0s 2024-03-30T02:38:37.291 INFO:tasks.workunit.client.0.smithi067.stderr:126500K .......... .......... .......... .......... .......... 93% 34.3M 0s 2024-03-30T02:38:37.292 INFO:tasks.workunit.client.0.smithi067.stderr:126550K .......... .......... .......... .......... .......... 93% 35.6M 0s 2024-03-30T02:38:37.294 INFO:tasks.workunit.client.0.smithi067.stderr:126600K .......... .......... .......... .......... .......... 93% 26.5M 0s 2024-03-30T02:38:37.296 INFO:tasks.workunit.client.0.smithi067.stderr:126650K .......... .......... .......... .......... .......... 93% 33.8M 0s 2024-03-30T02:38:37.298 INFO:tasks.workunit.client.0.smithi067.stderr:126700K .......... .......... .......... .......... .......... 93% 17.2M 0s 2024-03-30T02:38:37.300 INFO:tasks.workunit.client.0.smithi067.stderr:126750K .......... .......... .......... .......... .......... 93% 31.3M 0s 2024-03-30T02:38:37.301 INFO:tasks.workunit.client.0.smithi067.stderr:126800K .......... .......... .......... .......... .......... 93% 35.8M 0s 2024-03-30T02:38:37.303 INFO:tasks.workunit.client.0.smithi067.stderr:126850K .......... .......... .......... .......... .......... 93% 29.3M 0s 2024-03-30T02:38:37.304 INFO:tasks.workunit.client.0.smithi067.stderr:126900K .......... .......... .......... .......... .......... 93% 31.3M 0s 2024-03-30T02:38:37.305 INFO:tasks.workunit.client.0.smithi067.stderr:126950K .......... .......... .......... .......... .......... 93% 34.3M 0s 2024-03-30T02:38:37.310 INFO:tasks.workunit.client.0.smithi067.stderr:127000K .......... .......... .......... .......... .......... 93% 16.3M 0s 2024-03-30T02:38:37.310 INFO:tasks.workunit.client.0.smithi067.stderr:127050K .......... .......... .......... .......... .......... 93% 47.6M 0s 2024-03-30T02:38:37.311 INFO:tasks.workunit.client.0.smithi067.stderr:127100K .......... .......... .......... .......... .......... 93% 29.6M 0s 2024-03-30T02:38:37.313 INFO:tasks.workunit.client.0.smithi067.stderr:127150K .......... .......... .......... .......... .......... 93% 30.6M 0s 2024-03-30T02:38:37.315 INFO:tasks.workunit.client.0.smithi067.stderr:127200K .......... .......... .......... .......... .......... 93% 34.3M 0s 2024-03-30T02:38:37.316 INFO:tasks.workunit.client.0.smithi067.stderr:127250K .......... .......... .......... .......... .......... 93% 33.0M 0s 2024-03-30T02:38:37.317 INFO:tasks.workunit.client.0.smithi067.stderr:127300K .......... .......... .......... .......... .......... 93% 30.6M 0s 2024-03-30T02:38:37.320 INFO:tasks.workunit.client.0.smithi067.stderr:127350K .......... .......... .......... .......... .......... 93% 18.0M 0s 2024-03-30T02:38:37.322 INFO:tasks.workunit.client.0.smithi067.stderr:127400K .......... .......... .......... .......... .......... 93% 32.1M 0s 2024-03-30T02:38:37.323 INFO:tasks.workunit.client.0.smithi067.stderr:127450K .......... .......... .......... .......... .......... 93% 35.0M 0s 2024-03-30T02:38:37.326 INFO:tasks.workunit.client.0.smithi067.stderr:127500K .......... .......... .......... .......... .......... 93% 26.2M 0s 2024-03-30T02:38:37.326 INFO:tasks.workunit.client.0.smithi067.stderr:127550K .......... .......... .......... .......... .......... 93% 35.6M 0s 2024-03-30T02:38:37.329 INFO:tasks.workunit.client.0.smithi067.stderr:127600K .......... .......... .......... .......... .......... 94% 20.5M 0s 2024-03-30T02:38:37.331 INFO:tasks.workunit.client.0.smithi067.stderr:127650K .......... .......... .......... .......... .......... 94% 31.5M 0s 2024-03-30T02:38:37.333 INFO:tasks.workunit.client.0.smithi067.stderr:127700K .......... .......... .......... .......... .......... 94% 28.0M 0s 2024-03-30T02:38:37.333 INFO:tasks.workunit.client.0.smithi067.stderr:127750K .......... .......... .......... .......... .......... 94% 41.8M 0s 2024-03-30T02:38:37.336 INFO:tasks.workunit.client.0.smithi067.stderr:127800K .......... .......... .......... .......... .......... 94% 31.4M 0s 2024-03-30T02:38:37.338 INFO:tasks.workunit.client.0.smithi067.stderr:127850K .......... .......... .......... .......... .......... 94% 17.0M 0s 2024-03-30T02:38:37.340 INFO:tasks.workunit.client.0.smithi067.stderr:127900K .......... .......... .......... .......... .......... 94% 30.3M 0s 2024-03-30T02:38:37.341 INFO:tasks.workunit.client.0.smithi067.stderr:127950K .......... .......... .......... .......... .......... 94% 29.1M 0s 2024-03-30T02:38:37.344 INFO:tasks.workunit.client.0.smithi067.stderr:128000K .......... .......... .......... .......... .......... 94% 16.3M 0s 2024-03-30T02:38:37.346 INFO:tasks.workunit.client.0.smithi067.stderr:128050K .......... .......... .......... .......... .......... 94% 36.7M 0s 2024-03-30T02:38:37.346 INFO:tasks.workunit.client.0.smithi067.stderr:128100K .......... .......... .......... .......... .......... 94% 36.3M 0s 2024-03-30T02:38:37.350 INFO:tasks.workunit.client.0.smithi067.stderr:128150K .......... .......... .......... .......... .......... 94% 17.5M 0s 2024-03-30T02:38:37.353 INFO:tasks.workunit.client.0.smithi067.stderr:128200K .......... .......... .......... .......... .......... 94% 23.8M 0s 2024-03-30T02:38:37.353 INFO:tasks.workunit.client.0.smithi067.stderr:128250K .......... .......... .......... .......... .......... 94% 50.1M 0s 2024-03-30T02:38:37.355 INFO:tasks.workunit.client.0.smithi067.stderr:128300K .......... .......... .......... .......... .......... 94% 26.1M 0s 2024-03-30T02:38:37.356 INFO:tasks.workunit.client.0.smithi067.stderr:128350K .......... .......... .......... .......... .......... 94% 49.4M 0s 2024-03-30T02:38:37.358 INFO:tasks.workunit.client.0.smithi067.stderr:128400K .......... .......... .......... .......... .......... 94% 33.2M 0s 2024-03-30T02:38:37.359 INFO:tasks.workunit.client.0.smithi067.stderr:128450K .......... .......... .......... .......... .......... 94% 23.9M 0s 2024-03-30T02:38:37.361 INFO:tasks.workunit.client.0.smithi067.stderr:128500K .......... .......... .......... .......... .......... 94% 23.0M 0s 2024-03-30T02:38:37.363 INFO:tasks.workunit.client.0.smithi067.stderr:128550K .......... .......... .......... .......... .......... 94% 28.6M 0s 2024-03-30T02:38:37.364 INFO:tasks.workunit.client.0.smithi067.stderr:128600K .......... .......... .......... .......... .......... 94% 28.9M 0s 2024-03-30T02:38:37.365 INFO:tasks.workunit.client.0.smithi067.stderr:128650K .......... .......... .......... .......... .......... 94% 40.8M 0s 2024-03-30T02:38:37.368 INFO:tasks.workunit.client.0.smithi067.stderr:128700K .......... .......... .......... .......... .......... 94% 33.9M 0s 2024-03-30T02:38:37.369 INFO:tasks.workunit.client.0.smithi067.stderr:128750K .......... .......... .......... .......... .......... 94% 19.4M 0s 2024-03-30T02:38:37.371 INFO:tasks.workunit.client.0.smithi067.stderr:128800K .......... .......... .......... .......... .......... 94% 30.0M 0s 2024-03-30T02:38:37.372 INFO:tasks.workunit.client.0.smithi067.stderr:128850K .......... .......... .......... .......... .......... 94% 41.5M 0s 2024-03-30T02:38:37.375 INFO:tasks.workunit.client.0.smithi067.stderr:128900K .......... .......... .......... .......... .......... 94% 30.0M 0s 2024-03-30T02:38:37.376 INFO:tasks.workunit.client.0.smithi067.stderr:128950K .......... .......... .......... .......... .......... 95% 31.4M 0s 2024-03-30T02:38:37.376 INFO:tasks.workunit.client.0.smithi067.stderr:129000K .......... .......... .......... .......... .......... 95% 37.8M 0s 2024-03-30T02:38:37.380 INFO:tasks.workunit.client.0.smithi067.stderr:129050K .......... .......... .......... .......... .......... 95% 18.1M 0s 2024-03-30T02:38:37.381 INFO:tasks.workunit.client.0.smithi067.stderr:129100K .......... .......... .......... .......... .......... 95% 31.5M 0s 2024-03-30T02:38:37.382 INFO:tasks.workunit.client.0.smithi067.stderr:129150K .......... .......... .......... .......... .......... 95% 33.7M 0s 2024-03-30T02:38:37.384 INFO:tasks.workunit.client.0.smithi067.stderr:129200K .......... .......... .......... .......... .......... 95% 31.4M 0s 2024-03-30T02:38:37.385 INFO:tasks.workunit.client.0.smithi067.stderr:129250K .......... .......... .......... .......... .......... 95% 37.6M 0s 2024-03-30T02:38:37.387 INFO:tasks.workunit.client.0.smithi067.stderr:129300K .......... .......... .......... .......... .......... 95% 29.1M 0s 2024-03-30T02:38:37.390 INFO:tasks.workunit.client.0.smithi067.stderr:129350K .......... .......... .......... .......... .......... 95% 18.9M 0s 2024-03-30T02:38:37.391 INFO:tasks.workunit.client.0.smithi067.stderr:129400K .......... .......... .......... .......... .......... 95% 31.5M 0s 2024-03-30T02:38:37.394 INFO:tasks.workunit.client.0.smithi067.stderr:129450K .......... .......... .......... .......... .......... 95% 25.4M 0s 2024-03-30T02:38:37.394 INFO:tasks.workunit.client.0.smithi067.stderr:129500K .......... .......... .......... .......... .......... 95% 80.7M 0s 2024-03-30T02:38:37.396 INFO:tasks.workunit.client.0.smithi067.stderr:129550K .......... .......... .......... .......... .......... 95% 27.2M 0s 2024-03-30T02:38:37.398 INFO:tasks.workunit.client.0.smithi067.stderr:129600K .......... .......... .......... .......... .......... 95% 22.4M 0s 2024-03-30T02:38:37.399 INFO:tasks.workunit.client.0.smithi067.stderr:129650K .......... .......... .......... .......... .......... 95% 26.3M 0s 2024-03-30T02:38:37.401 INFO:tasks.workunit.client.0.smithi067.stderr:129700K .......... .......... .......... .......... .......... 95% 31.9M 0s 2024-03-30T02:38:37.402 INFO:tasks.workunit.client.0.smithi067.stderr:129750K .......... .......... .......... .......... .......... 95% 36.1M 0s 2024-03-30T02:38:37.404 INFO:tasks.workunit.client.0.smithi067.stderr:129800K .......... .......... .......... .......... .......... 95% 31.4M 0s 2024-03-30T02:38:37.405 INFO:tasks.workunit.client.0.smithi067.stderr:129850K .......... .......... .......... .......... .......... 95% 32.4M 0s 2024-03-30T02:38:37.408 INFO:tasks.workunit.client.0.smithi067.stderr:129900K .......... .......... .......... .......... .......... 95% 18.2M 0s 2024-03-30T02:38:37.410 INFO:tasks.workunit.client.0.smithi067.stderr:129950K .......... .......... .......... .......... .......... 95% 34.0M 0s 2024-03-30T02:38:37.411 INFO:tasks.workunit.client.0.smithi067.stderr:130000K .......... .......... .......... .......... .......... 95% 35.9M 0s 2024-03-30T02:38:37.413 INFO:tasks.workunit.client.0.smithi067.stderr:130050K .......... .......... .......... .......... .......... 95% 31.0M 0s 2024-03-30T02:38:37.414 INFO:tasks.workunit.client.0.smithi067.stderr:130100K .......... .......... .......... .......... .......... 95% 30.5M 0s 2024-03-30T02:38:37.417 INFO:tasks.workunit.client.0.smithi067.stderr:130150K .......... .......... .......... .......... .......... 95% 33.6M 0s 2024-03-30T02:38:37.418 INFO:tasks.workunit.client.0.smithi067.stderr:130200K .......... .......... .......... .......... .......... 95% 22.9M 0s 2024-03-30T02:38:37.420 INFO:tasks.workunit.client.0.smithi067.stderr:130250K .......... .......... .......... .......... .......... 95% 30.0M 0s 2024-03-30T02:38:37.424 INFO:tasks.workunit.client.0.smithi067.stderr:130300K .......... .......... .......... .......... .......... 96% 21.3M 0s 2024-03-30T02:38:37.424 INFO:tasks.workunit.client.0.smithi067.stderr:130350K .......... .......... .......... .......... .......... 96% 39.8M 0s 2024-03-30T02:38:37.424 INFO:tasks.workunit.client.0.smithi067.stderr:130400K .......... .......... .......... .......... .......... 96% 38.3M 0s 2024-03-30T02:38:37.426 INFO:tasks.workunit.client.0.smithi067.stderr:130450K .......... .......... .......... .......... .......... 96% 30.5M 0s 2024-03-30T02:38:37.428 INFO:tasks.workunit.client.0.smithi067.stderr:130500K .......... .......... .......... .......... .......... 96% 20.1M 0s 2024-03-30T02:38:37.429 INFO:tasks.workunit.client.0.smithi067.stderr:130550K .......... .......... .......... .......... .......... 96% 31.1M 0s 2024-03-30T02:38:37.431 INFO:tasks.workunit.client.0.smithi067.stderr:130600K .......... .......... .......... .......... .......... 96% 24.6M 0s 2024-03-30T02:38:37.432 INFO:tasks.workunit.client.0.smithi067.stderr:130650K .......... .......... .......... .......... .......... 96% 52.3M 0s 2024-03-30T02:38:37.434 INFO:tasks.workunit.client.0.smithi067.stderr:130700K .......... .......... .......... .......... .......... 96% 25.0M 0s 2024-03-30T02:38:37.436 INFO:tasks.workunit.client.0.smithi067.stderr:130750K .......... .......... .......... .......... .......... 96% 41.1M 0s 2024-03-30T02:38:37.439 INFO:tasks.workunit.client.0.smithi067.stderr:130800K .......... .......... .......... .......... .......... 96% 19.2M 0s 2024-03-30T02:38:37.441 INFO:tasks.workunit.client.0.smithi067.stderr:130850K .......... .......... .......... .......... .......... 96% 26.6M 0s 2024-03-30T02:38:37.442 INFO:tasks.workunit.client.0.smithi067.stderr:130900K .......... .......... .......... .......... .......... 96% 45.0M 0s 2024-03-30T02:38:37.443 INFO:tasks.workunit.client.0.smithi067.stderr:130950K .......... .......... .......... .......... .......... 96% 31.2M 0s 2024-03-30T02:38:37.445 INFO:tasks.workunit.client.0.smithi067.stderr:131000K .......... .......... .......... .......... .......... 96% 32.6M 0s 2024-03-30T02:38:37.447 INFO:tasks.workunit.client.0.smithi067.stderr:131050K .......... .......... .......... .......... .......... 96% 19.0M 0s 2024-03-30T02:38:37.449 INFO:tasks.workunit.client.0.smithi067.stderr:131100K .......... .......... .......... .......... .......... 96% 31.9M 0s 2024-03-30T02:38:37.450 INFO:tasks.workunit.client.0.smithi067.stderr:131150K .......... .......... .......... .......... .......... 96% 38.0M 0s 2024-03-30T02:38:37.451 INFO:tasks.workunit.client.0.smithi067.stderr:131200K .......... .......... .......... .......... .......... 96% 31.2M 0s 2024-03-30T02:38:37.453 INFO:tasks.workunit.client.0.smithi067.stderr:131250K .......... .......... .......... .......... .......... 96% 27.4M 0s 2024-03-30T02:38:37.455 INFO:tasks.workunit.client.0.smithi067.stderr:131300K .......... .......... .......... .......... .......... 96% 40.5M 0s 2024-03-30T02:38:37.456 INFO:tasks.workunit.client.0.smithi067.stderr:131350K .......... .......... .......... .......... .......... 96% 22.1M 0s 2024-03-30T02:38:37.459 INFO:tasks.workunit.client.0.smithi067.stderr:131400K .......... .......... .......... .......... .......... 96% 28.2M 0s 2024-03-30T02:38:37.463 INFO:tasks.workunit.client.0.smithi067.stderr:131450K .......... .......... .......... .......... .......... 96% 13.2M 0s 2024-03-30T02:38:37.463 INFO:tasks.workunit.client.0.smithi067.stderr:131500K .......... .......... .......... .......... .......... 96% 34.7M 0s 2024-03-30T02:38:37.465 INFO:tasks.workunit.client.0.smithi067.stderr:131550K .......... .......... .......... .......... .......... 96% 32.4M 0s 2024-03-30T02:38:37.466 INFO:tasks.workunit.client.0.smithi067.stderr:131600K .......... .......... .......... .......... .......... 96% 34.3M 0s 2024-03-30T02:38:37.469 INFO:tasks.workunit.client.0.smithi067.stderr:131650K .......... .......... .......... .......... .......... 97% 16.9M 0s 2024-03-30T02:38:37.471 INFO:tasks.workunit.client.0.smithi067.stderr:131700K .......... .......... .......... .......... .......... 97% 43.7M 0s 2024-03-30T02:38:37.472 INFO:tasks.workunit.client.0.smithi067.stderr:131750K .......... .......... .......... .......... .......... 97% 29.5M 0s 2024-03-30T02:38:37.473 INFO:tasks.workunit.client.0.smithi067.stderr:131800K .......... .......... .......... .......... .......... 97% 37.4M 0s 2024-03-30T02:38:37.475 INFO:tasks.workunit.client.0.smithi067.stderr:131850K .......... .......... .......... .......... .......... 97% 31.8M 0s 2024-03-30T02:38:37.476 INFO:tasks.workunit.client.0.smithi067.stderr:131900K .......... .......... .......... .......... .......... 97% 28.5M 0s 2024-03-30T02:38:37.480 INFO:tasks.workunit.client.0.smithi067.stderr:131950K .......... .......... .......... .......... .......... 97% 20.6M 0s 2024-03-30T02:38:37.480 INFO:tasks.workunit.client.0.smithi067.stderr:132000K .......... .......... .......... .......... .......... 97% 29.8M 0s 2024-03-30T02:38:37.482 INFO:tasks.workunit.client.0.smithi067.stderr:132050K .......... .......... .......... .......... .......... 97% 33.5M 0s 2024-03-30T02:38:37.484 INFO:tasks.workunit.client.0.smithi067.stderr:132100K .......... .......... .......... .......... .......... 97% 30.6M 0s 2024-03-30T02:38:37.485 INFO:tasks.workunit.client.0.smithi067.stderr:132150K .......... .......... .......... .......... .......... 97% 25.6M 0s 2024-03-30T02:38:37.486 INFO:tasks.workunit.client.0.smithi067.stderr:132200K .......... .......... .......... .......... .......... 97% 36.0M 0s 2024-03-30T02:38:37.488 INFO:tasks.workunit.client.0.smithi067.stderr:132250K .......... .......... .......... .......... .......... 97% 30.7M 0s 2024-03-30T02:38:37.491 INFO:tasks.workunit.client.0.smithi067.stderr:132300K .......... .......... .......... .......... .......... 97% 27.8M 0s 2024-03-30T02:38:37.492 INFO:tasks.workunit.client.0.smithi067.stderr:132350K .......... .......... .......... .......... .......... 97% 26.0M 0s 2024-03-30T02:38:37.493 INFO:tasks.workunit.client.0.smithi067.stderr:132400K .......... .......... .......... .......... .......... 97% 32.4M 0s 2024-03-30T02:38:37.495 INFO:tasks.workunit.client.0.smithi067.stderr:132450K .......... .......... .......... .......... .......... 97% 30.9M 0s 2024-03-30T02:38:37.498 INFO:tasks.workunit.client.0.smithi067.stderr:132500K .......... .......... .......... .......... .......... 97% 19.4M 0s 2024-03-30T02:38:37.499 INFO:tasks.workunit.client.0.smithi067.stderr:132550K .......... .......... .......... .......... .......... 97% 36.9M 0s 2024-03-30T02:38:37.501 INFO:tasks.workunit.client.0.smithi067.stderr:132600K .......... .......... .......... .......... .......... 97% 19.3M 0s 2024-03-30T02:38:37.503 INFO:tasks.workunit.client.0.smithi067.stderr:132650K .......... .......... .......... .......... .......... 97% 35.9M 0s 2024-03-30T02:38:37.503 INFO:tasks.workunit.client.0.smithi067.stderr:132700K .......... .......... .......... .......... .......... 97% 43.4M 0s 2024-03-30T02:38:37.505 INFO:tasks.workunit.client.0.smithi067.stderr:132750K .......... .......... .......... .......... .......... 97% 40.5M 0s 2024-03-30T02:38:37.507 INFO:tasks.workunit.client.0.smithi067.stderr:132800K .......... .......... .......... .......... .......... 97% 24.4M 0s 2024-03-30T02:38:37.510 INFO:tasks.workunit.client.0.smithi067.stderr:132850K .......... .......... .......... .......... .......... 97% 21.6M 0s 2024-03-30T02:38:37.510 INFO:tasks.workunit.client.0.smithi067.stderr:132900K .......... .......... .......... .......... .......... 97% 86.5M 0s 2024-03-30T02:38:37.512 INFO:tasks.workunit.client.0.smithi067.stderr:132950K .......... .......... .......... .......... .......... 97% 25.5M 0s 2024-03-30T02:38:37.513 INFO:tasks.workunit.client.0.smithi067.stderr:133000K .......... .......... .......... .......... .......... 98% 30.8M 0s 2024-03-30T02:38:37.515 INFO:tasks.workunit.client.0.smithi067.stderr:133050K .......... .......... .......... .......... .......... 98% 24.2M 0s 2024-03-30T02:38:37.517 INFO:tasks.workunit.client.0.smithi067.stderr:133100K .......... .......... .......... .......... .......... 98% 22.3M 0s 2024-03-30T02:38:37.519 INFO:tasks.workunit.client.0.smithi067.stderr:133150K .......... .......... .......... .......... .......... 98% 38.3M 0s 2024-03-30T02:38:37.520 INFO:tasks.workunit.client.0.smithi067.stderr:133200K .......... .......... .......... .......... .......... 98% 32.5M 0s 2024-03-30T02:38:37.529 INFO:tasks.workunit.client.0.smithi067.stderr:133250K .......... .......... .......... .......... .......... 98% 8.59M 0s 2024-03-30T02:38:37.529 INFO:tasks.workunit.client.0.smithi067.stderr:133300K .......... .......... .......... .......... .......... 98% 84.9M 0s 2024-03-30T02:38:37.529 INFO:tasks.workunit.client.0.smithi067.stderr:133350K .......... .......... .......... .......... .......... 98% 108M 0s 2024-03-30T02:38:37.529 INFO:tasks.workunit.client.0.smithi067.stderr:133400K .......... .......... .......... .......... .......... 98% 119M 0s 2024-03-30T02:38:37.529 INFO:tasks.workunit.client.0.smithi067.stderr:133450K .......... .......... .......... .......... .......... 98% 44.4M 0s 2024-03-30T02:38:37.530 INFO:tasks.workunit.client.0.smithi067.stderr:133500K .......... .......... .......... .......... .......... 98% 23.5M 0s 2024-03-30T02:38:37.532 INFO:tasks.workunit.client.0.smithi067.stderr:133550K .......... .......... .......... .......... .......... 98% 35.4M 0s 2024-03-30T02:38:37.533 INFO:tasks.workunit.client.0.smithi067.stderr:133600K .......... .......... .......... .......... .......... 98% 29.0M 0s 2024-03-30T02:38:37.535 INFO:tasks.workunit.client.0.smithi067.stderr:133650K .......... .......... .......... .......... .......... 98% 33.7M 0s 2024-03-30T02:38:37.538 INFO:tasks.workunit.client.0.smithi067.stderr:133700K .......... .......... .......... .......... .......... 98% 19.7M 0s 2024-03-30T02:38:37.540 INFO:tasks.workunit.client.0.smithi067.stderr:133750K .......... .......... .......... .......... .......... 98% 22.5M 0s 2024-03-30T02:38:37.541 INFO:tasks.workunit.client.0.smithi067.stderr:133800K .......... .......... .......... .......... .......... 98% 43.9M 0s 2024-03-30T02:38:37.542 INFO:tasks.workunit.client.0.smithi067.stderr:133850K .......... .......... .......... .......... .......... 98% 34.6M 0s 2024-03-30T02:38:37.544 INFO:tasks.workunit.client.0.smithi067.stderr:133900K .......... .......... .......... .......... .......... 98% 25.3M 0s 2024-03-30T02:38:37.545 INFO:tasks.workunit.client.0.smithi067.stderr:133950K .......... .......... .......... .......... .......... 98% 37.7M 0s 2024-03-30T02:38:37.549 INFO:tasks.workunit.client.0.smithi067.stderr:134000K .......... .......... .......... .......... .......... 98% 21.6M 0s 2024-03-30T02:38:37.550 INFO:tasks.workunit.client.0.smithi067.stderr:134050K .......... .......... .......... .......... .......... 98% 34.0M 0s 2024-03-30T02:38:37.551 INFO:tasks.workunit.client.0.smithi067.stderr:134100K .......... .......... .......... .......... .......... 98% 32.6M 0s 2024-03-30T02:38:37.552 INFO:tasks.workunit.client.0.smithi067.stderr:134150K .......... .......... .......... .......... .......... 98% 32.6M 0s 2024-03-30T02:38:37.555 INFO:tasks.workunit.client.0.smithi067.stderr:134200K .......... .......... .......... .......... .......... 98% 20.5M 0s 2024-03-30T02:38:37.556 INFO:tasks.workunit.client.0.smithi067.stderr:134250K .......... .......... .......... .......... .......... 98% 29.0M 0s 2024-03-30T02:38:37.557 INFO:tasks.workunit.client.0.smithi067.stderr:134300K .......... .......... .......... .......... .......... 98% 50.1M 0s 2024-03-30T02:38:37.559 INFO:tasks.workunit.client.0.smithi067.stderr:134350K .......... .......... .......... .......... .......... 99% 26.1M 0s 2024-03-30T02:38:37.561 INFO:tasks.workunit.client.0.smithi067.stderr:134400K .......... .......... .......... .......... .......... 99% 29.7M 0s 2024-03-30T02:38:37.562 INFO:tasks.workunit.client.0.smithi067.stderr:134450K .......... .......... .......... .......... .......... 99% 31.5M 0s 2024-03-30T02:38:37.563 INFO:tasks.workunit.client.0.smithi067.stderr:134500K .......... .......... .......... .......... .......... 99% 35.7M 0s 2024-03-30T02:38:37.567 INFO:tasks.workunit.client.0.smithi067.stderr:134550K .......... .......... .......... .......... .......... 99% 18.3M 0s 2024-03-30T02:38:37.569 INFO:tasks.workunit.client.0.smithi067.stderr:134600K .......... .......... .......... .......... .......... 99% 26.4M 0s 2024-03-30T02:38:37.572 INFO:tasks.workunit.client.0.smithi067.stderr:134650K .......... .......... .......... .......... .......... 99% 15.7M 0s 2024-03-30T02:38:37.574 INFO:tasks.workunit.client.0.smithi067.stderr:134700K .......... .......... .......... .......... .......... 99% 16.9M 0s 2024-03-30T02:38:37.576 INFO:tasks.workunit.client.0.smithi067.stderr:134750K .......... .......... .......... .......... .......... 99% 32.4M 0s 2024-03-30T02:38:37.579 INFO:tasks.workunit.client.0.smithi067.stderr:134800K .......... .......... .......... .......... .......... 99% 22.1M 0s 2024-03-30T02:38:37.580 INFO:tasks.workunit.client.0.smithi067.stderr:134850K .......... .......... .......... .......... .......... 99% 49.6M 0s 2024-03-30T02:38:37.581 INFO:tasks.workunit.client.0.smithi067.stderr:134900K .......... .......... .......... .......... .......... 99% 31.3M 0s 2024-03-30T02:38:37.581 INFO:tasks.workunit.client.0.smithi067.stderr:134950K .......... .......... .......... .......... .......... 99% 44.1M 0s 2024-03-30T02:38:37.583 INFO:tasks.workunit.client.0.smithi067.stderr:135000K .......... .......... .......... .......... .......... 99% 32.2M 0s 2024-03-30T02:38:37.584 INFO:tasks.workunit.client.0.smithi067.stderr:135050K .......... .......... .......... .......... .......... 99% 29.8M 0s 2024-03-30T02:38:37.587 INFO:tasks.workunit.client.0.smithi067.stderr:135100K .......... .......... .......... .......... .......... 99% 20.2M 0s 2024-03-30T02:38:37.590 INFO:tasks.workunit.client.0.smithi067.stderr:135150K .......... .......... .......... .......... .......... 99% 20.3M 0s 2024-03-30T02:38:37.592 INFO:tasks.workunit.client.0.smithi067.stderr:135200K .......... .......... .......... .......... .......... 99% 31.7M 0s 2024-03-30T02:38:37.592 INFO:tasks.workunit.client.0.smithi067.stderr:135250K .......... .......... .......... .......... .......... 99% 47.6M 0s 2024-03-30T02:38:37.599 INFO:tasks.workunit.client.0.smithi067.stderr:135300K .......... .......... .......... .......... .......... 99% 13.0M 0s 2024-03-30T02:38:37.599 INFO:tasks.workunit.client.0.smithi067.stderr:135350K .......... .......... .......... .......... .......... 99% 48.1M 0s 2024-03-30T02:38:37.599 INFO:tasks.workunit.client.0.smithi067.stderr:135400K .......... .......... .......... .......... .......... 99% 22.5M 0s 2024-03-30T02:38:37.601 INFO:tasks.workunit.client.0.smithi067.stderr:135450K .......... .......... .......... .......... .......... 99% 17.9M 0s 2024-03-30T02:38:37.603 INFO:tasks.workunit.client.0.smithi067.stderr:135500K .......... .......... .......... .......... .......... 99% 47.6M 0s 2024-03-30T02:38:37.609 INFO:tasks.workunit.client.0.smithi067.stderr:135550K .......... .......... .......... .......... .......... 99% 30.9M 0s 2024-03-30T02:38:37.609 INFO:tasks.workunit.client.0.smithi067.stderr:135600K .......... .......... .......... .......... .......... 99% 44.7M 0s 2024-03-30T02:38:37.609 INFO:tasks.workunit.client.0.smithi067.stderr:135650K .......... .......... .......... .......... .......... 99% 30.2M 0s 2024-03-30T02:38:37.609 INFO:tasks.workunit.client.0.smithi067.stderr:135700K .......... .......... .......... .......... .......... 99% 57.1M 0s 2024-03-30T02:38:37.609 INFO:tasks.workunit.client.0.smithi067.stderr:135750K .. 100% 5431G=5.3s 2024-03-30T02:38:37.609 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:37.609 INFO:tasks.workunit.client.0.smithi067.stderr:2024-03-30 02:38:37 (25.0 MB/s) - ‘linux.tar.xz’ saved [139010916/139010916] 2024-03-30T02:38:37.609 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-03-30T02:38:37.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:37 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:38:37.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:37 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:38:37.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:37 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:38:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:38 smithi067 ceph-mon[37808]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-03-30T02:38:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:38 smithi170 ceph-mon[78798]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-03-30T02:38:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:38 smithi082 ceph-mon[79176]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2024-03-30T02:38:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:40 smithi067 ceph-mon[37808]: pgmap v51: 129 pgs: 129 active+clean; 53 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 MiB/s wr, 16 op/s 2024-03-30T02:38:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:40 smithi170 ceph-mon[78798]: pgmap v51: 129 pgs: 129 active+clean; 53 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 MiB/s wr, 16 op/s 2024-03-30T02:38:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:40 smithi082 ceph-mon[79176]: pgmap v51: 129 pgs: 129 active+clean; 53 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 MiB/s wr, 16 op/s 2024-03-30T02:38:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:42 smithi067 ceph-mon[37808]: pgmap v52: 129 pgs: 129 active+clean; 53 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-03-30T02:38:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:42 smithi170 ceph-mon[78798]: pgmap v52: 129 pgs: 129 active+clean; 53 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-03-30T02:38:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:42 smithi082 ceph-mon[79176]: pgmap v52: 129 pgs: 129 active+clean; 53 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-03-30T02:38:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:44 smithi067 ceph-mon[37808]: pgmap v53: 129 pgs: 129 active+clean; 53 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-03-30T02:38:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:44 smithi170 ceph-mon[78798]: pgmap v53: 129 pgs: 129 active+clean; 53 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-03-30T02:38:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:44 smithi082 ceph-mon[79176]: pgmap v53: 129 pgs: 129 active+clean; 53 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-03-30T02:38:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:46 smithi067 ceph-mon[37808]: pgmap v54: 129 pgs: 129 active+clean; 136 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 11 MiB/s wr, 20 op/s 2024-03-30T02:38:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:46 smithi170 ceph-mon[78798]: pgmap v54: 129 pgs: 129 active+clean; 136 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 11 MiB/s wr, 20 op/s 2024-03-30T02:38:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:46 smithi082 ceph-mon[79176]: pgmap v54: 129 pgs: 129 active+clean; 136 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 11 MiB/s wr, 20 op/s 2024-03-30T02:38:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:48 smithi067 ceph-mon[37808]: pgmap v55: 129 pgs: 129 active+clean; 136 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 11 MiB/s wr, 15 op/s 2024-03-30T02:38:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:48 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:38:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:48 smithi170 ceph-mon[78798]: pgmap v55: 129 pgs: 129 active+clean; 136 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 11 MiB/s wr, 15 op/s 2024-03-30T02:38:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:48 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:38:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:48 smithi082 ceph-mon[79176]: pgmap v55: 129 pgs: 129 active+clean; 136 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 11 MiB/s wr, 15 op/s 2024-03-30T02:38:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:48 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:38:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:50 smithi067 ceph-mon[37808]: pgmap v56: 129 pgs: 129 active+clean; 145 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 12 MiB/s wr, 37 op/s 2024-03-30T02:38:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:50 smithi170 ceph-mon[78798]: pgmap v56: 129 pgs: 129 active+clean; 145 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 12 MiB/s wr, 37 op/s 2024-03-30T02:38:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:50 smithi082 ceph-mon[79176]: pgmap v56: 129 pgs: 129 active+clean; 145 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 12 MiB/s wr, 37 op/s 2024-03-30T02:38:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:52 smithi067 ceph-mon[37808]: pgmap v57: 129 pgs: 129 active+clean; 145 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 7.8 MiB/s wr, 30 op/s 2024-03-30T02:38:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:52 smithi170 ceph-mon[78798]: pgmap v57: 129 pgs: 129 active+clean; 145 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 7.8 MiB/s wr, 30 op/s 2024-03-30T02:38:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:52 smithi082 ceph-mon[79176]: pgmap v57: 129 pgs: 129 active+clean; 145 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 7.8 MiB/s wr, 30 op/s 2024-03-30T02:38:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:54 smithi067 ceph-mon[37808]: pgmap v58: 129 pgs: 129 active+clean; 145 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 7.8 MiB/s wr, 30 op/s 2024-03-30T02:38:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:54 smithi170 ceph-mon[78798]: pgmap v58: 129 pgs: 129 active+clean; 145 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 7.8 MiB/s wr, 30 op/s 2024-03-30T02:38:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:54 smithi082 ceph-mon[79176]: pgmap v58: 129 pgs: 129 active+clean; 145 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 7.8 MiB/s wr, 30 op/s 2024-03-30T02:38:56.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:56 smithi067 ceph-mon[37808]: pgmap v59: 129 pgs: 129 active+clean; 151 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.3 MiB/s wr, 54 op/s 2024-03-30T02:38:56.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:56 smithi170 ceph-mon[78798]: pgmap v59: 129 pgs: 129 active+clean; 151 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.3 MiB/s wr, 54 op/s 2024-03-30T02:38:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:56 smithi082 ceph-mon[79176]: pgmap v59: 129 pgs: 129 active+clean; 151 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.3 MiB/s wr, 54 op/s 2024-03-30T02:38:58.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:38:58 smithi067 ceph-mon[37808]: pgmap v60: 129 pgs: 129 active+clean; 151 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 46 op/s 2024-03-30T02:38:58.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:38:58 smithi170 ceph-mon[78798]: pgmap v60: 129 pgs: 129 active+clean; 151 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 46 op/s 2024-03-30T02:38:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:38:58 smithi082 ceph-mon[79176]: pgmap v60: 129 pgs: 129 active+clean; 151 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 46 op/s 2024-03-30T02:39:00.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:00 smithi067 ceph-mon[37808]: pgmap v61: 129 pgs: 129 active+clean; 158 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 70 op/s 2024-03-30T02:39:00.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:00 smithi170 ceph-mon[78798]: pgmap v61: 129 pgs: 129 active+clean; 158 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 70 op/s 2024-03-30T02:39:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:00 smithi082 ceph-mon[79176]: pgmap v61: 129 pgs: 129 active+clean; 158 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 70 op/s 2024-03-30T02:39:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:02 smithi067 ceph-mon[37808]: pgmap v62: 129 pgs: 129 active+clean; 158 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-30T02:39:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:02 smithi170 ceph-mon[78798]: pgmap v62: 129 pgs: 129 active+clean; 158 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-30T02:39:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:02 smithi082 ceph-mon[79176]: pgmap v62: 129 pgs: 129 active+clean; 158 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-30T02:39:04.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:04 smithi067 ceph-mon[37808]: pgmap v63: 129 pgs: 129 active+clean; 158 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-30T02:39:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:04 smithi170 ceph-mon[78798]: pgmap v63: 129 pgs: 129 active+clean; 158 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-30T02:39:04.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:04 smithi082 ceph-mon[79176]: pgmap v63: 129 pgs: 129 active+clean; 158 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 47 op/s 2024-03-30T02:39:05.009 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:39:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.218+0000 7f090be00700 1 -- 172.21.15.67:0/3269305633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f090c101510 msgr2=0x7f090c10d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.218+0000 7f090be00700 1 --2- 172.21.15.67:0/3269305633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f090c101510 0x7f090c10d9d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f090400b0d0 tx=0x7f090400b3e0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.218+0000 7f090be00700 1 -- 172.21.15.67:0/3269305633 shutdown_connections 2024-03-30T02:39:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.218+0000 7f090be00700 1 --2- 172.21.15.67:0/3269305633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f090c101510 0x7f090c10d9d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.218+0000 7f090be00700 1 --2- 172.21.15.67:0/3269305633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f090c106c10 0x7f090c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.218+0000 7f090be00700 1 --2- 172.21.15.67:0/3269305633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f090c100b50 0x7f090c100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.218+0000 7f090be00700 1 -- 172.21.15.67:0/3269305633 >> 172.21.15.67:0/3269305633 conn(0x7f090c0fc620 msgr2=0x7f090c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.218+0000 7f090be00700 1 -- 172.21.15.67:0/3269305633 shutdown_connections 2024-03-30T02:39:05.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.218+0000 7f090be00700 1 -- 172.21.15.67:0/3269305633 wait complete. 2024-03-30T02:39:05.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.219+0000 7f090be00700 1 Processor -- start 2024-03-30T02:39:05.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.219+0000 7f090be00700 1 -- start start 2024-03-30T02:39:05.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f090c100b50 0x7f090c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f090c101510 0x7f090c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f090c106c10 0x7f090c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090be00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c19e170 con 0x7f090c100b50 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090be00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c19e2e0 con 0x7f090c101510 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090be00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c19e4a0 con 0x7f090c106c10 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f090c100b50 0x7f090c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f090c101510 0x7f090c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f090c101510 0x7f090c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52972/0 (socket says 172.21.15.67:52972) 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090a000700 1 -- 172.21.15.67:0/4053944039 learned_addr learned my addr 172.21.15.67:0/4053944039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f090c100b50 0x7f090c199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42480/0 (socket says 172.21.15.67:42480) 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.220+0000 7f090b400700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f090c106c10 0x7f090c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f090a000700 1 -- 172.21.15.67:0/4053944039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f090c106c10 msgr2=0x7f090c19db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f090a000700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f090c106c10 0x7f090c19db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f090a000700 1 -- 172.21.15.67:0/4053944039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f090c100b50 msgr2=0x7f090c199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f090a000700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f090c100b50 0x7f090c199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f090a000700 1 -- 172.21.15.67:0/4053944039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904009d00 con 0x7f090c101510 2024-03-30T02:39:05.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f090b400700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f090c106c10 0x7f090c19db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f090a000700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f090c101510 0x7f090c199790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f08f800e9a0 tx=0x7f08f800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f0903400700 1 -- 172.21.15.67:0/4053944039 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8009790 con 0x7f090c101510 2024-03-30T02:39:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f090be00700 1 -- 172.21.15.67:0/4053944039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c19e780 con 0x7f090c101510 2024-03-30T02:39:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f0903400700 1 -- 172.21.15.67:0/4053944039 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08f800cbd0 con 0x7f090c101510 2024-03-30T02:39:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f0903400700 1 -- 172.21.15.67:0/4053944039 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8018680 con 0x7f090c101510 2024-03-30T02:39:05.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.221+0000 7f090be00700 1 -- 172.21.15.67:0/4053944039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c1a5fc0 con 0x7f090c101510 2024-03-30T02:39:05.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.224+0000 7f0903400700 1 -- 172.21.15.67:0/4053944039 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f08f800cd40 con 0x7f090c101510 2024-03-30T02:39:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.226+0000 7f0903400700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08f00a1310 0x7f08f00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.226+0000 7f0925598b80 1 Processor -- start 2024-03-30T02:39:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.226+0000 7f0903400700 1 -- 172.21.15.67:0/4053944039 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f08f8014070 con 0x7f090c101510 2024-03-30T02:39:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.226+0000 7f0925598b80 1 -- start start 2024-03-30T02:39:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.226+0000 7f090aa00700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08f00a1310 0x7f08f00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.226+0000 7f0925598b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7777a0 0x55cd5e777b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.226+0000 7f0925598b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e782a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f0925598b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e7853b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f0925598b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e6d0990 con 0x55cd5e7780c0 2024-03-30T02:39:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f0925598b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e6ce350 con 0x55cd5e782fc0 2024-03-30T02:39:05.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f0925598b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e6c9c40 con 0x55cd5e7777a0 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7777a0 0x55cd5e777b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e7853b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7777a0 0x55cd5e777b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:49142/0 (socket says 172.21.15.67:49142) 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090aa00700 1 -- 172.21.15.67:0/3278015372 learned_addr learned my addr 172.21.15.67:0/3278015372 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090a000700 1 --2- 172.21.15.67:0/3278015372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e782a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090aa00700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08f00a1310 0x7f08f00a37d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009cb0 tx=0x7f08fc009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090aa00700 1 -- 172.21.15.67:0/3278015372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 msgr2=0x55cd5e7853b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090aa00700 1 --2- 172.21.15.67:0/3278015372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e7853b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090aa00700 1 -- 172.21.15.67:0/3278015372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 msgr2=0x55cd5e782a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090aa00700 1 --2- 172.21.15.67:0/3278015372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e782a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090aa00700 1 -- 172.21.15.67:0/3278015372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5e54b3b0 con 0x55cd5e7777a0 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.227+0000 7f090b400700 1 --2- 172.21.15.67:0/3278015372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e7853b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:05.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.228+0000 7f090aa00700 1 --2- 172.21.15.67:0/3278015372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7777a0 0x55cd5e777b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009fd0 tx=0x7f08fc01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c949e6cb009ac9b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.228+0000 7f08f7e00700 1 -- 172.21.15.67:0/3278015372 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc01ce30 con 0x55cd5e7777a0 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.228+0000 7f08f7e00700 1 -- 172.21.15.67:0/3278015372 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08fc004cf0 con 0x55cd5e7777a0 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.228+0000 7f08f7e00700 1 -- 172.21.15.67:0/3278015372 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc024680 con 0x55cd5e7777a0 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 -- 172.21.15.67:0/3278015372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7777a0 msgr2=0x55cd5e777b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 --2- 172.21.15.67:0/3278015372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7777a0 0x55cd5e777b80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009fd0 tx=0x7f08fc01c000 comp rx=0 tx=0).stop 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 -- 172.21.15.67:0/3278015372 shutdown_connections 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 --2- 172.21.15.67:0/3278015372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e7853b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 --2- 172.21.15.67:0/3278015372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7777a0 0x55cd5e777b80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 --2- 172.21.15.67:0/3278015372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e782a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 -- 172.21.15.67:0/3278015372 >> 172.21.15.67:0/3278015372 conn(0x55cd5e6d76a0 msgr2=0x55cd5e773c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 -- 172.21.15.67:0/3278015372 shutdown_connections 2024-03-30T02:39:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 -- 172.21.15.67:0/3278015372 wait complete. 2024-03-30T02:39:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 -- start start 2024-03-30T02:39:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.229+0000 7f0925598b80 1 Processor -- start 2024-03-30T02:39:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f0925598b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e782940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f0925598b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7780c0 0x55cd5e77fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f0925598b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 0x55cd5e780260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f0925598b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e665d80 con 0x55cd5e7777a0 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f0925598b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e665a20 con 0x55cd5e7780c0 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f0925598b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e54d6a0 con 0x55cd5e782fc0 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f0925598b80 10 client.0 _ll_register_callbacks cb 0x55cd5e66f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f090a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7780c0 0x55cd5e77fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f090a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7780c0 0x55cd5e77fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52986/0 (socket says 172.21.15.67:52986) 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f090a000700 1 -- 172.21.15.67:0/2214258466 learned_addr learned my addr 172.21.15.67:0/2214258466 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.230+0000 7f090aa00700 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e782940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f090b400700 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 0x55cd5e780260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f090a000700 1 -- 172.21.15.67:0/2214258466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 msgr2=0x55cd5e780260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f090a000700 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 0x55cd5e780260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f090a000700 1 -- 172.21.15.67:0/2214258466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 msgr2=0x55cd5e782940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f090a000700 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e782940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f090a000700 1 -- 172.21.15.67:0/2214258466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c10e2d0 con 0x55cd5e7780c0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f090a000700 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7780c0 0x55cd5e77fc30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f08f8096a60 tx=0x7f08f80999d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9fec92f649ce654 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f08f4c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f08f4c00700 1 -- 172.21.15.67:0/2214258466 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f809de20 con 0x55cd5e7780c0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f08f4c00700 1 -- 172.21.15.67:0/2214258466 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f08f80a2470 con 0x55cd5e7780c0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f0925598b80 1 -- 172.21.15.67:0/2214258466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd5e68a380 con 0x55cd5e7780c0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f08f4c00700 1 -- 172.21.15.67:0/2214258466 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08f80b81d0 con 0x55cd5e7780c0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f0925598b80 10 client.26162 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.231+0000 7f0925598b80 1 -- 172.21.15.67:0/2214258466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f08fc017050 con 0x55cd5e7780c0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.232+0000 7f08f4c00700 1 -- 172.21.15.67:0/2214258466 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd5e68a380 con 0x55cd5e7780c0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.232+0000 7f08f4c00700 1 -- 172.21.15.67:0/2214258466 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f08f8098fe0 con 0x55cd5e7780c0 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.232+0000 7f0925598b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.232+0000 7f0925598b80 10 client.26162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.232+0000 7f0925598b80 20 client.26162 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.232+0000 7f0925598b80 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55cd5e792b20 0x55cd5e794f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.232+0000 7f0925598b80 4 client.26162 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.232+0000 7f0925598b80 1 -- 172.21.15.67:0/2214258466 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd5e6c2700 con 0x55cd5e792b20 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.232+0000 7f090aa00700 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55cd5e792b20 0x55cd5e794f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.233+0000 7f090aa00700 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55cd5e792b20 0x55cd5e794f40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea68a6fb4fbc5e1 server_cookie=a78906210bc16194 in_seq=0 out_seq=0 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.233+0000 7f08f4c00700 10 client.26162 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.233+0000 7f08f4c00700 1 -- 172.21.15.67:0/2214258466 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cd5e6c2700 con 0x55cd5e792b20 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.233+0000 7f08f4c00700 10 client.26162 handle_command_reply: tid=0 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.233+0000 7f0925598b80 1 client.26162 shutdown 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.233+0000 7f0925598b80 1 -- 172.21.15.67:0/2214258466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7780c0 msgr2=0x55cd5e77fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.233+0000 7f0925598b80 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7780c0 0x55cd5e77fc30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f08f8096a60 tx=0x7f08f80999d0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f0925598b80 1 -- 172.21.15.67:0/2214258466 shutdown_connections 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f0925598b80 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55cd5e792b20 0x55cd5e794f40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f0925598b80 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e782940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f0925598b80 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 0x55cd5e780260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f08f4c00700 0 client.26162 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f0925598b80 1 --2- 172.21.15.67:0/2214258466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7780c0 0x55cd5e77fc30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f0925598b80 1 -- 172.21.15.67:0/2214258466 >> 172.21.15.67:0/2214258466 conn(0x55cd5e6d76a0 msgr2=0x55cd5e779890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f0925598b80 1 -- 172.21.15.67:0/2214258466 shutdown_connections 2024-03-30T02:39:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f0925598b80 1 -- 172.21.15.67:0/2214258466 wait complete. 2024-03-30T02:39:05.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.234+0000 7f0925598b80 20 client.26162 trim_cache size 0 max 16384 2024-03-30T02:39:05.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.251+0000 7f0925598b80 1 Processor -- start 2024-03-30T02:39:05.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f0925598b80 1 -- start start 2024-03-30T02:39:05.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f0925598b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e780180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f0925598b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7780c0 0x55cd5e789c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f0925598b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e77e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f0925598b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e777610 con 0x55cd5e7777a0 2024-03-30T02:39:05.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f0925598b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e790230 con 0x55cd5e782fc0 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f0925598b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e774860 con 0x55cd5e7780c0 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f090aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e780180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f090b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e77e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f090aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e780180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42496/0 (socket says 172.21.15.67:42496) 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f090aa00700 1 -- 172.21.15.67:0/196232963 learned_addr learned my addr 172.21.15.67:0/196232963 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.252+0000 7f090a000700 1 --2- 172.21.15.67:0/196232963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7780c0 0x55cd5e789c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f090aa00700 1 -- 172.21.15.67:0/196232963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7780c0 msgr2=0x55cd5e789c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f090aa00700 1 --2- 172.21.15.67:0/196232963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7780c0 0x55cd5e789c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f090aa00700 1 -- 172.21.15.67:0/196232963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 msgr2=0x55cd5e77e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f090aa00700 1 --2- 172.21.15.67:0/196232963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e77e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f090aa00700 1 -- 172.21.15.67:0/196232963 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5e4f7b60 con 0x55cd5e7777a0 2024-03-30T02:39:05.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f090b400700 1 --2- 172.21.15.67:0/196232963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e77e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:05.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f090aa00700 1 --2- 172.21.15.67:0/196232963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e780180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f08fc011ab0 tx=0x7f08fc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65692eb7a35117cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f0900c00700 1 -- 172.21.15.67:0/196232963 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc005d00 con 0x55cd5e7777a0 2024-03-30T02:39:05.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f0900c00700 1 -- 172.21.15.67:0/196232963 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08fc005320 con 0x55cd5e7777a0 2024-03-30T02:39:05.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.253+0000 7f0900c00700 1 -- 172.21.15.67:0/196232963 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc024680 con 0x55cd5e7777a0 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.254+0000 7f0925598b80 1 -- 172.21.15.67:0/196232963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 msgr2=0x55cd5e780180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.254+0000 7f0925598b80 1 --2- 172.21.15.67:0/196232963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e780180 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f08fc011ab0 tx=0x7f08fc01ca70 comp rx=0 tx=0).stop 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.254+0000 7f0925598b80 1 -- 172.21.15.67:0/196232963 shutdown_connections 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.254+0000 7f0925598b80 1 --2- 172.21.15.67:0/196232963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7777a0 0x55cd5e780180 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.254+0000 7f0925598b80 1 --2- 172.21.15.67:0/196232963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e782fc0 0x55cd5e77e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.254+0000 7f0925598b80 1 --2- 172.21.15.67:0/196232963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e7780c0 0x55cd5e789c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.254+0000 7f0925598b80 1 -- 172.21.15.67:0/196232963 >> 172.21.15.67:0/196232963 conn(0x55cd5e6d76a0 msgr2=0x55cd5e773700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.254+0000 7f0925598b80 1 -- 172.21.15.67:0/196232963 shutdown_connections 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.254+0000 7f0925598b80 1 -- 172.21.15.67:0/196232963 wait complete. 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.255+0000 7f0925598b80 1 -- start start 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.255+0000 7f0925598b80 1 Processor -- start 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.255+0000 7f0925598b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7777a0 0x55cd5e789a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.255+0000 7f0925598b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e787f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.255+0000 7f0925598b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 0x55cd5e788490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.255+0000 7f0925598b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e7889d0 con 0x55cd5e7780c0 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.255+0000 7f0925598b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e788b40 con 0x55cd5e7777a0 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.255+0000 7f0925598b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e788cb0 con 0x55cd5e782fc0 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.255+0000 7f0925598b80 10 client.0 _ll_register_callbacks cb 0x55cd5e66f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e787f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e787f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42498/0 (socket says 172.21.15.67:42498) 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7777a0 0x55cd5e789a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090a000700 1 -- 172.21.15.67:0/2875216946 learned_addr learned my addr 172.21.15.67:0/2875216946 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090b400700 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 0x55cd5e788490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090a000700 1 -- 172.21.15.67:0/2875216946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 msgr2=0x55cd5e788490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090a000700 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 0x55cd5e788490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090a000700 1 -- 172.21.15.67:0/2875216946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7777a0 msgr2=0x55cd5e789a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090a000700 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7777a0 0x55cd5e789a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090a000700 1 -- 172.21.15.67:0/2875216946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040093a0 con 0x55cd5e7780c0 2024-03-30T02:39:05.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f090a000700 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e787f50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f08f8099f80 tx=0x7f08f80cd140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb3bb0a1aaadf04 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f08f4c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.256+0000 7f08f4c00700 1 -- 172.21.15.67:0/2875216946 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f809ed40 con 0x55cd5e7780c0 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f08f4c00700 1 -- 172.21.15.67:0/2875216946 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f08f80a2470 con 0x55cd5e7780c0 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f0925598b80 1 -- 172.21.15.67:0/2875216946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd5e518cf0 con 0x55cd5e7780c0 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f08f4c00700 1 -- 172.21.15.67:0/2875216946 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08f8096330 con 0x55cd5e7780c0 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f0925598b80 10 client.16116 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f0925598b80 1 -- 172.21.15.67:0/2875216946 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cd5e54b3b0 con 0x55cd5e7780c0 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f08f4c00700 1 -- 172.21.15.67:0/2875216946 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd5e518cf0 con 0x55cd5e7780c0 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f08f4c00700 1 -- 172.21.15.67:0/2875216946 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f08f80a2100 con 0x55cd5e7780c0 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f0925598b80 10 client.16116 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f0925598b80 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f0925598b80 20 client.16116 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f0925598b80 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55cd5e7e7bd0 0x55cd5e7e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f0925598b80 4 client.16116 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:39:05.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f0925598b80 1 -- 172.21.15.67:0/2875216946 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd5e54dce0 con 0x55cd5e7e7bd0 2024-03-30T02:39:05.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.257+0000 7f090aa00700 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55cd5e7e7bd0 0x55cd5e7e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.258+0000 7f090aa00700 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55cd5e7e7bd0 0x55cd5e7e9ff0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de3b0802f22d4e4f server_cookie=16513e1c0644a0a3 in_seq=0 out_seq=0 2024-03-30T02:39:05.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.258+0000 7f08f4c00700 10 client.16116 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f08f4c00700 1 -- 172.21.15.67:0/2875216946 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cd5e54dce0 con 0x55cd5e7e7bd0 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f08f4c00700 10 client.16116 handle_command_reply: tid=0 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 client.16116 shutdown 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 -- 172.21.15.67:0/2875216946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 msgr2=0x55cd5e787f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e787f50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f08f8099f80 tx=0x7f08f80cd140 comp rx=0 tx=0).stop 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 -- 172.21.15.67:0/2875216946 shutdown_connections 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5e782fc0 0x55cd5e788490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5e7777a0 0x55cd5e789a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55cd5e7e7bd0 0x55cd5e7e9ff0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 --2- 172.21.15.67:0/2875216946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55cd5e7780c0 0x55cd5e787f50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f08f4c00700 0 client.16116 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 -- 172.21.15.67:0/2875216946 >> 172.21.15.67:0/2875216946 conn(0x55cd5e6d76a0 msgr2=0x55cd5e77e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 -- 172.21.15.67:0/2875216946 shutdown_connections 2024-03-30T02:39:05.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.259+0000 7f0925598b80 1 -- 172.21.15.67:0/2875216946 wait complete. 2024-03-30T02:39:05.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f0925598b80 20 client.16116 trim_cache size 0 max 16384 2024-03-30T02:39:05.264 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:39:05.264 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:39:05.264 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:39:05.264 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 -- 172.21.15.67:0/4053944039 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08f00a1310 msgr2=0x7f08f00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08f00a1310 0x7f08f00a37d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009cb0 tx=0x7f08fc009260 comp rx=0 tx=0).stop 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 -- 172.21.15.67:0/4053944039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f090c101510 msgr2=0x7f090c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f090c101510 0x7f090c199790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f08f800e9a0 tx=0x7f08f800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 -- 172.21.15.67:0/4053944039 shutdown_connections 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f08f00a1310 0x7f08f00a37d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f090c100b50 0x7f090c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f090c106c10 0x7f090c19db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 --2- 172.21.15.67:0/4053944039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f090c101510 0x7f090c199790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 -- 172.21.15.67:0/4053944039 >> 172.21.15.67:0/4053944039 conn(0x7f090c0fc620 msgr2=0x7f090c0fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 -- 172.21.15.67:0/4053944039 shutdown_connections 2024-03-30T02:39:05.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.260+0000 7f090be00700 1 -- 172.21.15.67:0/4053944039 wait complete. 2024-03-30T02:39:05.279 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:39:05.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:39:05.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.482+0000 7fbdeb400700 1 -- 172.21.15.67:0/2321016875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbde4101810 msgr2=0x7fbde4105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.482+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2321016875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbde4101810 0x7fbde4105850 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0009990 tx=0x7fbdd0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.482+0000 7fbdeb400700 1 -- 172.21.15.67:0/2321016875 shutdown_connections 2024-03-30T02:39:05.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.482+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2321016875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde4105f80 0x7fbde410db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.482+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2321016875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbde4100e60 0x7fbde4101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.482+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2321016875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbde4101810 0x7fbde4105850 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.482+0000 7fbdeb400700 1 -- 172.21.15.67:0/2321016875 >> 172.21.15.67:0/2321016875 conn(0x7fbde40fc700 msgr2=0x7fbde40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.483+0000 7fbdeb400700 1 -- 172.21.15.67:0/2321016875 shutdown_connections 2024-03-30T02:39:05.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.483+0000 7fbdeb400700 1 -- 172.21.15.67:0/2321016875 wait complete. 2024-03-30T02:39:05.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbdeb400700 1 Processor -- start 2024-03-30T02:39:05.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbdeb400700 1 -- start start 2024-03-30T02:39:05.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbdeb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde4100e60 0x7fbde40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbdeb400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbde4105f80 0x7fbde40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbdeb400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbde4073b30 0x7fbde41a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbdeb400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde41a58d0 con 0x7fbde4100e60 2024-03-30T02:39:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbdeb400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde41a5a10 con 0x7fbde4073b30 2024-03-30T02:39:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbdeb400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde41a5bd0 con 0x7fbde4105f80 2024-03-30T02:39:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbde8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde4100e60 0x7fbde40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbde8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde4100e60 0x7fbde40730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42524/0 (socket says 172.21.15.67:42524) 2024-03-30T02:39:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbde8a00700 1 -- 172.21.15.67:0/2931845419 learned_addr learned my addr 172.21.15.67:0/2931845419 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbde3e00700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbde4105f80 0x7fbde40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.484+0000 7fbde9400700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbde4073b30 0x7fbde41a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbde8a00700 1 -- 172.21.15.67:0/2931845419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbde4105f80 msgr2=0x7fbde40735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbde8a00700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbde4105f80 0x7fbde40735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbde8a00700 1 -- 172.21.15.67:0/2931845419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbde4073b30 msgr2=0x7fbde41a5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbde8a00700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbde4073b30 0x7fbde41a5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbde8a00700 1 -- 172.21.15.67:0/2931845419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00095f0 con 0x7fbde4100e60 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbde8a00700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde4100e60 0x7fbde40730b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800e9a0 tx=0x7fbdd800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbde1600700 1 -- 172.21.15.67:0/2931845419 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8009790 con 0x7fbde4100e60 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbde1600700 1 -- 172.21.15.67:0/2931845419 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd800cbd0 con 0x7fbde4100e60 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbde1600700 1 -- 172.21.15.67:0/2931845419 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8018680 con 0x7fbde4100e60 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde41a5eb0 con 0x7fbde4100e60 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.485+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde41a6330 con 0x7fbde4100e60 2024-03-30T02:39:05.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.487+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde4109250 con 0x7fbde4100e60 2024-03-30T02:39:05.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.489+0000 7fbde1600700 1 -- 172.21.15.67:0/2931845419 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbdd800cd40 con 0x7fbde4100e60 2024-03-30T02:39:05.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.490+0000 7fbde1600700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbdcc0a1310 0x7fbdcc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.490+0000 7fbde1600700 1 -- 172.21.15.67:0/2931845419 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fbdd8014070 con 0x7fbde4100e60 2024-03-30T02:39:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.490+0000 7fbde3e00700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbdcc0a1310 0x7fbdcc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.491+0000 7fbde3e00700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbdcc0a1310 0x7fbdcc0a37d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000b2e0 tx=0x7fbdd0005fd0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.494+0000 7fbde1600700 1 -- 172.21.15.67:0/2931845419 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fbdd80529f0 con 0x7fbde4100e60 2024-03-30T02:39:05.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.757+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbde4102330 con 0x7fbde4100e60 2024-03-30T02:39:05.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.758+0000 7fbde1600700 1 -- 172.21.15.67:0/2931845419 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10916 (secure 0 0 0) 0x7fbdd8096310 con 0x7fbde4100e60 2024-03-30T02:39:05.759 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:39:05.759 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:38:47.255528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:39:05.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.760+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbdcc0a1310 msgr2=0x7fbdcc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.760+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbdcc0a1310 0x7fbdcc0a37d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000b2e0 tx=0x7fbdd0005fd0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.760+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde4100e60 msgr2=0x7fbde40730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.760+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde4100e60 0x7fbde40730b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800e9a0 tx=0x7fbdd800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.760+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 shutdown_connections 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.760+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbdcc0a1310 0x7fbdcc0a37d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.760+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbde4100e60 0x7fbde40730b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.761+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbde4073b30 0x7fbde41a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.761+0000 7fbdeb400700 1 --2- 172.21.15.67:0/2931845419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbde4105f80 0x7fbde40735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.761+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 >> 172.21.15.67:0/2931845419 conn(0x7fbde40fc700 msgr2=0x7fbde40fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.761+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 shutdown_connections 2024-03-30T02:39:05.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.761+0000 7fbdeb400700 1 -- 172.21.15.67:0/2931845419 wait complete. 2024-03-30T02:39:05.763 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-03-30T02:39:05.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:39:05.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.981+0000 7f064ba00700 1 -- 172.21.15.67:0/574407806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 msgr2=0x7f0644101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.981+0000 7f064ba00700 1 --2- 172.21.15.67:0/574407806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 0x7f0644101340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f06380098f0 tx=0x7f0638009c00 comp rx=0 tx=0).stop 2024-03-30T02:39:05.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.981+0000 7f064ba00700 1 -- 172.21.15.67:0/574407806 shutdown_connections 2024-03-30T02:39:05.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.981+0000 7f064ba00700 1 --2- 172.21.15.67:0/574407806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0644106080 0x7f064410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.981+0000 7f064ba00700 1 --2- 172.21.15.67:0/574407806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 0x7f0644101340 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.981+0000 7f064ba00700 1 --2- 172.21.15.67:0/574407806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0644101910 0x7f0644105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.981+0000 7f064ba00700 1 -- 172.21.15.67:0/574407806 >> 172.21.15.67:0/574407806 conn(0x7f06440fc7c0 msgr2=0x7f06440febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.981+0000 7f064ba00700 1 -- 172.21.15.67:0/574407806 shutdown_connections 2024-03-30T02:39:05.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.981+0000 7f064ba00700 1 -- 172.21.15.67:0/574407806 wait complete. 2024-03-30T02:39:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.982+0000 7f064ba00700 1 Processor -- start 2024-03-30T02:39:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.982+0000 7f064ba00700 1 -- start start 2024-03-30T02:39:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f064ba00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 0x7f06440730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f064ba00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0644106080 0x7f06440735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f064ba00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0644073b30 0x7f06441a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f064ba00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441a59b0 con 0x7f0644106080 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f064ba00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441a5af0 con 0x7f0644100f60 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f064ba00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441a5cb0 con 0x7f0644073b30 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0649000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 0x7f06440730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0649000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 0x7f06440730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34326/0 (socket says 172.21.15.67:34326) 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0643e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0644106080 0x7f06440735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0649000700 1 -- 172.21.15.67:0/3892875683 learned_addr learned my addr 172.21.15.67:0/3892875683 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0649a00700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0644073b30 0x7f06441a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0649000700 1 -- 172.21.15.67:0/3892875683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0644073b30 msgr2=0x7f06441a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0649000700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0644073b30 0x7f06441a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0649000700 1 -- 172.21.15.67:0/3892875683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0644106080 msgr2=0x7f06440735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0649000700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0644106080 0x7f06440735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.983+0000 7f0649000700 1 -- 172.21.15.67:0/3892875683 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300095f0 con 0x7f0644100f60 2024-03-30T02:39:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.984+0000 7f0649000700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 0x7f06440730b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f063800fa00 tx=0x7f063800fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.984+0000 7f0641600700 1 -- 172.21.15.67:0/3892875683 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063801c070 con 0x7f0644100f60 2024-03-30T02:39:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.984+0000 7f0641600700 1 -- 172.21.15.67:0/3892875683 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063800bb20 con 0x7f0644100f60 2024-03-30T02:39:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.984+0000 7f064ba00700 1 -- 172.21.15.67:0/3892875683 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0638009520 con 0x7f0644100f60 2024-03-30T02:39:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.984+0000 7f0641600700 1 -- 172.21.15.67:0/3892875683 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638004930 con 0x7f0644100f60 2024-03-30T02:39:05.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.984+0000 7f064ba00700 1 -- 172.21.15.67:0/3892875683 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441a6210 con 0x7f0644100f60 2024-03-30T02:39:05.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.988+0000 7f0641600700 1 -- 172.21.15.67:0/3892875683 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0638020070 con 0x7f0644100f60 2024-03-30T02:39:05.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.989+0000 7f065f457b80 1 Processor -- start 2024-03-30T02:39:05.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f065f457b80 1 -- start start 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f065f457b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf49269b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f065f457b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49318b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f065f457b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf49341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f065f457b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf487f990 con 0x555cf49265d0 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f065f457b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf487d350 con 0x555cf4926ef0 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f0641600700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f062c0a1310 0x7f062c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f065f457b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4878c40 con 0x555cf4931df0 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f0641600700 1 -- 172.21.15.67:0/3892875683 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f06380cdbe0 con 0x7f0644100f60 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f0649000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49318b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f0649a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf49269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f0649000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49318b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34342/0 (socket says 172.21.15.67:34342) 2024-03-30T02:39:05.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f0649000700 1 -- 172.21.15.67:0/2880089616 learned_addr learned my addr 172.21.15.67:0/2880089616 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f0643e00700 1 --2- 172.21.15.67:0/2880089616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf49341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.990+0000 7f0649a00700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f062c0a1310 0x7f062c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0649000700 1 -- 172.21.15.67:0/2880089616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 msgr2=0x555cf49341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0649000700 1 --2- 172.21.15.67:0/2880089616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf49341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0649000700 1 -- 172.21.15.67:0/2880089616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 msgr2=0x555cf49269b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0649000700 1 --2- 172.21.15.67:0/2880089616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf49269b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0649000700 1 -- 172.21.15.67:0/2880089616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf46fa3b0 con 0x555cf4926ef0 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0649000700 1 --2- 172.21.15.67:0/2880089616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49318b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f06380cafe0 tx=0x7f06380054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a984e12cffa5d915 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0636000700 1 -- 172.21.15.67:0/2880089616 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063809b0d0 con 0x555cf4926ef0 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0636000700 1 -- 172.21.15.67:0/2880089616 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063809a9d0 con 0x555cf4926ef0 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0636000700 1 -- 172.21.15.67:0/2880089616 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380cb200 con 0x555cf4926ef0 2024-03-30T02:39:05.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.991+0000 7f0649a00700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f062c0a1310 0x7f062c0a37d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f063c009fd0 tx=0x7f063c00b040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 -- 172.21.15.67:0/2880089616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 msgr2=0x555cf49318b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:05.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 --2- 172.21.15.67:0/2880089616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49318b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f06380cafe0 tx=0x7f06380054e0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 -- 172.21.15.67:0/2880089616 shutdown_connections 2024-03-30T02:39:05.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 --2- 172.21.15.67:0/2880089616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf49269b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 --2- 172.21.15.67:0/2880089616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf49341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 --2- 172.21.15.67:0/2880089616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49318b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 -- 172.21.15.67:0/2880089616 >> 172.21.15.67:0/2880089616 conn(0x555cf48866a0 msgr2=0x555cf4922a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:05.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 -- 172.21.15.67:0/2880089616 shutdown_connections 2024-03-30T02:39:05.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 -- 172.21.15.67:0/2880089616 wait complete. 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.992+0000 7f065f457b80 1 -- start start 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f065f457b80 1 Processor -- start 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f065f457b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf49265d0 0x555cf49317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f065f457b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf492eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f065f457b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4931df0 0x555cf492f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f065f457b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4814d80 con 0x555cf4931df0 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f065f457b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4814a20 con 0x555cf4926ef0 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f065f457b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf46fc6a0 con 0x555cf49265d0 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f065f457b80 10 client.0 _ll_register_callbacks cb 0x555cf481e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f0649000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf49265d0 0x555cf49317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f0649000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf49265d0 0x555cf49317b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48800/0 (socket says 172.21.15.67:48800) 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f0649a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4931df0 0x555cf492f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.993+0000 7f0649000700 1 -- 172.21.15.67:0/3633963898 learned_addr learned my addr 172.21.15.67:0/3633963898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:05.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f0643e00700 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf492eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f0649000700 1 -- 172.21.15.67:0/3633963898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 msgr2=0x555cf492eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f0649000700 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf492eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f0649000700 1 -- 172.21.15.67:0/3633963898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4931df0 msgr2=0x555cf492f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f0649000700 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4931df0 0x555cf492f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f0649000700 1 -- 172.21.15.67:0/3633963898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441a6210 con 0x555cf49265d0 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f0643e00700 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf492eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f0649000700 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf49265d0 0x555cf49317b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f06380094f0 tx=0x7f063800bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fd7cd83a28c5679 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f062aa00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f062aa00700 1 -- 172.21.15.67:0/3633963898 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380cb410 con 0x555cf49265d0 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f062aa00700 1 -- 172.21.15.67:0/3633963898 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f063809e800 con 0x555cf49265d0 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.994+0000 7f065f457b80 1 -- 172.21.15.67:0/3633963898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cf4839380 con 0x555cf49265d0 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f062aa00700 1 -- 172.21.15.67:0/3633963898 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06380b3200 con 0x555cf49265d0 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f065f457b80 10 client.26167 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f065f457b80 1 -- 172.21.15.67:0/3633963898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555cf46a6b60 con 0x555cf49265d0 2024-03-30T02:39:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f062aa00700 1 -- 172.21.15.67:0/3633963898 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555cf4839380 con 0x555cf49265d0 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f062aa00700 1 -- 172.21.15.67:0/3633963898 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0638099ec0 con 0x555cf49265d0 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f065f457b80 10 client.26167 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f065f457b80 10 client.26167 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f065f457b80 20 client.26167 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f065f457b80 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555cf49411f0 0x555cf4943610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f065f457b80 4 client.26167 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.995+0000 7f065f457b80 1 -- 172.21.15.67:0/3633963898 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cf4871700 con 0x555cf49411f0 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.996+0000 7f0643e00700 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555cf49411f0 0x555cf4943610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.996+0000 7f0643e00700 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555cf49411f0 0x555cf4943610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=54cce52e5c826886 server_cookie=ce23f1b67b17a631 in_seq=0 out_seq=0 2024-03-30T02:39:06.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.996+0000 7f062aa00700 10 client.26167 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:39:06.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f062aa00700 1 -- 172.21.15.67:0/3633963898 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555cf4871700 con 0x555cf49411f0 2024-03-30T02:39:06.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f062aa00700 10 client.26167 handle_command_reply: tid=0 2024-03-30T02:39:06.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 client.26167 shutdown 2024-03-30T02:39:06.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 -- 172.21.15.67:0/3633963898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf49265d0 msgr2=0x555cf49317b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf49265d0 0x555cf49317b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f06380094f0 tx=0x7f063800bc90 comp rx=0 tx=0).stop 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 -- 172.21.15.67:0/3633963898 shutdown_connections 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4931df0 0x555cf492f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555cf49411f0 0x555cf4943610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf49265d0 0x555cf49317b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 --2- 172.21.15.67:0/3633963898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf492eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f062aa00700 0 client.26167 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 -- 172.21.15.67:0/3633963898 >> 172.21.15.67:0/3633963898 conn(0x555cf48866a0 msgr2=0x555cf49286c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 -- 172.21.15.67:0/3633963898 shutdown_connections 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.997+0000 7f065f457b80 1 -- 172.21.15.67:0/3633963898 wait complete. 2024-03-30T02:39:06.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:05.998+0000 7f065f457b80 20 client.26167 trim_cache size 0 max 16384 2024-03-30T02:39:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.011+0000 7f065f457b80 1 Processor -- start 2024-03-30T02:39:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.011+0000 7f065f457b80 1 -- start start 2024-03-30T02:39:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.011+0000 7f065f457b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf492f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.011+0000 7f065f457b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f065f457b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf492e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f065f457b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf49278c0 con 0x555cf49265d0 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f065f457b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4927e50 con 0x555cf4926ef0 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f065f457b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4938d20 con 0x555cf4931df0 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0649000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf492f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0649000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf492f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51812/0 (socket says 172.21.15.67:51812) 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0649000700 1 -- 172.21.15.67:0/1464047671 learned_addr learned my addr 172.21.15.67:0/1464047671 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0643e00700 1 --2- 172.21.15.67:0/1464047671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0649a00700 1 --2- 172.21.15.67:0/1464047671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf492e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0649000700 1 -- 172.21.15.67:0/1464047671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 msgr2=0x555cf492e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0649000700 1 --2- 172.21.15.67:0/1464047671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf492e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0649000700 1 -- 172.21.15.67:0/1464047671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 msgr2=0x555cf49387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0649000700 1 --2- 172.21.15.67:0/1464047671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49387e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.012+0000 7f0649000700 1 -- 172.21.15.67:0/1464047671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf483eea0 con 0x555cf49265d0 2024-03-30T02:39:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f0649000700 1 --2- 172.21.15.67:0/1464047671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf492f070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0638097440 tx=0x7f0638005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8a79033bf6420a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f0636a00700 1 -- 172.21.15.67:0/1464047671 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063809b0d0 con 0x555cf49265d0 2024-03-30T02:39:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f0636a00700 1 -- 172.21.15.67:0/1464047671 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063809e800 con 0x555cf49265d0 2024-03-30T02:39:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f0636a00700 1 -- 172.21.15.67:0/1464047671 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063809ac30 con 0x555cf49265d0 2024-03-30T02:39:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f065f457b80 1 -- 172.21.15.67:0/1464047671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 msgr2=0x555cf492f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f065f457b80 1 --2- 172.21.15.67:0/1464047671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf492f070 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0638097440 tx=0x7f0638005870 comp rx=0 tx=0).stop 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f065f457b80 1 -- 172.21.15.67:0/1464047671 shutdown_connections 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f065f457b80 1 --2- 172.21.15.67:0/1464047671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf49265d0 0x555cf492f070 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f065f457b80 1 --2- 172.21.15.67:0/1464047671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf492e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f065f457b80 1 --2- 172.21.15.67:0/1464047671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf4926ef0 0x555cf49387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f065f457b80 1 -- 172.21.15.67:0/1464047671 >> 172.21.15.67:0/1464047671 conn(0x555cf48866a0 msgr2=0x555cf491b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f065f457b80 1 -- 172.21.15.67:0/1464047671 shutdown_connections 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.013+0000 7f065f457b80 1 -- 172.21.15.67:0/1464047671 wait complete. 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.014+0000 7f065f457b80 1 -- start start 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.014+0000 7f065f457b80 1 Processor -- start 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.014+0000 7f065f457b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf49265d0 0x555cf4935f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.014+0000 7f065f457b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4926ef0 0x555cf4936480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.014+0000 7f065f457b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf49369c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.014+0000 7f065f457b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf49940c0 con 0x555cf4926ef0 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.014+0000 7f065f457b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4994230 con 0x555cf49265d0 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.014+0000 7f065f457b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf49943f0 con 0x555cf4931df0 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f065f457b80 10 client.0 _ll_register_callbacks cb 0x555cf481e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0643e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4926ef0 0x555cf4936480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf49265d0 0x555cf4935f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf49369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf49265d0 0x555cf4935f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34374/0 (socket says 172.21.15.67:34374) 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649000700 1 -- 172.21.15.67:0/1810724853 learned_addr learned my addr 172.21.15.67:0/1810724853 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0643e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4926ef0 0x555cf4936480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51824/0 (socket says 172.21.15.67:51824) 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649000700 1 -- 172.21.15.67:0/1810724853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 msgr2=0x555cf49369c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649000700 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf49369c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649000700 1 -- 172.21.15.67:0/1810724853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4926ef0 msgr2=0x555cf4936480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649000700 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4926ef0 0x555cf4936480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649000700 1 -- 172.21.15.67:0/1810724853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cf46a6b60 con 0x555cf49265d0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f0649000700 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf49265d0 0x555cf4935f40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f063809a910 tx=0x7f0638009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4c56d66a819c9a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.015+0000 7f062aa00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f062aa00700 1 -- 172.21.15.67:0/1810724853 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380a2b90 con 0x555cf49265d0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f062aa00700 1 -- 172.21.15.67:0/1810724853 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f06380a2dc0 con 0x555cf49265d0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f065f457b80 1 -- 172.21.15.67:0/1810724853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f063809a9d0 con 0x555cf49265d0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f062aa00700 1 -- 172.21.15.67:0/1810724853 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f063809ac30 con 0x555cf49265d0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f065f457b80 10 client.26192 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f065f457b80 1 -- 172.21.15.67:0/1810724853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555cf46fecd0 con 0x555cf49265d0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f062aa00700 1 -- 172.21.15.67:0/1810724853 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063809a9d0 con 0x555cf49265d0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f062aa00700 1 -- 172.21.15.67:0/1810724853 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f063800b6e0 con 0x555cf49265d0 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f065f457b80 10 client.26192 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f065f457b80 10 client.26192 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:39:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f065f457b80 20 client.26192 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f065f457b80 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555cf4995680 0x555cf4997a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f065f457b80 4 client.26192 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.016+0000 7f065f457b80 1 -- 172.21.15.67:0/1810724853 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555cf46c7cf0 con 0x555cf4995680 2024-03-30T02:39:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.017+0000 7f0643e00700 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555cf4995680 0x555cf4997a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.017+0000 7f0643e00700 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555cf4995680 0x555cf4997a70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ced48d348ddb426 server_cookie=9bad24f999fb097f in_seq=0 out_seq=0 2024-03-30T02:39:06.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.017+0000 7f062aa00700 10 client.26192 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:39:06.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.017+0000 7f062aa00700 1 -- 172.21.15.67:0/1810724853 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555cf46c7cf0 con 0x555cf4995680 2024-03-30T02:39:06.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.017+0000 7f062aa00700 10 client.26192 handle_command_reply: tid=0 2024-03-30T02:39:06.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.017+0000 7f065f457b80 1 client.26192 shutdown 2024-03-30T02:39:06.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.017+0000 7f065f457b80 1 -- 172.21.15.67:0/1810724853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf49265d0 msgr2=0x555cf4935f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf49265d0 0x555cf4935f40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f063809a910 tx=0x7f0638009c00 comp rx=0 tx=0).stop 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 1 -- 172.21.15.67:0/1810724853 shutdown_connections 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x555cf4995680 0x555cf4997a70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555cf4931df0 0x555cf49369c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555cf49265d0 0x555cf4935f40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f062aa00700 0 client.26192 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 1 --2- 172.21.15.67:0/1810724853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555cf4926ef0 0x555cf4936480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 1 -- 172.21.15.67:0/1810724853 >> 172.21.15.67:0/1810724853 conn(0x555cf48866a0 msgr2=0x555cf493cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 1 -- 172.21.15.67:0/1810724853 shutdown_connections 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 1 -- 172.21.15.67:0/1810724853 wait complete. 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f065f457b80 20 client.26192 trim_cache size 0 max 16384 2024-03-30T02:39:06.023 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:06.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 -- 172.21.15.67:0/3892875683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f062c0a1310 msgr2=0x7f062c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f062c0a1310 0x7f062c0a37d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f063c009fd0 tx=0x7f063c00b040 comp rx=0 tx=0).stop 2024-03-30T02:39:06.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 -- 172.21.15.67:0/3892875683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 msgr2=0x7f06440730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 0x7f06440730b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f063800fa00 tx=0x7f063800fa30 comp rx=0 tx=0).stop 2024-03-30T02:39:06.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 -- 172.21.15.67:0/3892875683 shutdown_connections 2024-03-30T02:39:06.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f062c0a1310 0x7f062c0a37d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0644073b30 0x7f06441a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0644100f60 0x7f06440730b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 --2- 172.21.15.67:0/3892875683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0644106080 0x7f06440735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.018+0000 7f064ba00700 1 -- 172.21.15.67:0/3892875683 >> 172.21.15.67:0/3892875683 conn(0x7f06440fc7c0 msgr2=0x7f06440fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.019+0000 7f064ba00700 1 -- 172.21.15.67:0/3892875683 shutdown_connections 2024-03-30T02:39:06.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.019+0000 7f064ba00700 1 -- 172.21.15.67:0/3892875683 wait complete. 2024-03-30T02:39:06.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:39:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.237+0000 7fe2bd800700 1 -- 172.21.15.67:0/1557552766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8100f60 msgr2=0x7fe2b8101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.237+0000 7fe2bd800700 1 --2- 172.21.15.67:0/1557552766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8100f60 0x7fe2b8101340 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009990 tx=0x7fe2a8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.238+0000 7fe2bd800700 1 -- 172.21.15.67:0/1557552766 shutdown_connections 2024-03-30T02:39:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.238+0000 7fe2bd800700 1 --2- 172.21.15.67:0/1557552766 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8100f60 0x7fe2b8101340 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.238+0000 7fe2bd800700 1 --2- 172.21.15.67:0/1557552766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b8106080 0x7fe2b810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.238+0000 7fe2bd800700 1 --2- 172.21.15.67:0/1557552766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2b8101910 0x7fe2b8105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.238+0000 7fe2bd800700 1 -- 172.21.15.67:0/1557552766 >> 172.21.15.67:0/1557552766 conn(0x7fe2b80fc7c0 msgr2=0x7fe2b80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.238+0000 7fe2bd800700 1 -- 172.21.15.67:0/1557552766 shutdown_connections 2024-03-30T02:39:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.238+0000 7fe2bd800700 1 -- 172.21.15.67:0/1557552766 wait complete. 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.239+0000 7fe2bd800700 1 Processor -- start 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.239+0000 7fe2bd800700 1 -- start start 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.239+0000 7fe2bd800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b8100f60 0x7fe2b8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.239+0000 7fe2bd800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8106080 0x7fe2b81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.239+0000 7fe2bd800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2b819dd70 0x7fe2b819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.239+0000 7fe2bd800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b819e7f0 con 0x7fe2b8106080 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.239+0000 7fe2bd800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b819e960 con 0x7fe2b819dd70 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.239+0000 7fe2bd800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b819eb20 con 0x7fe2b8100f60 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8106080 0x7fe2b81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b8100f60 0x7fe2b8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b8100f60 0x7fe2b8199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48848/0 (socket says 172.21.15.67:48848) 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6a00700 1 -- 172.21.15.67:0/596005432 learned_addr learned my addr 172.21.15.67:0/596005432 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8106080 0x7fe2b81999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51838/0 (socket says 172.21.15.67:51838) 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b7400700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2b819dd70 0x7fe2b819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6a00700 1 -- 172.21.15.67:0/596005432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2b819dd70 msgr2=0x7fe2b819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2b819dd70 0x7fe2b819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6a00700 1 -- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8106080 msgr2=0x7fe2b81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8106080 0x7fe2b81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6a00700 1 -- 172.21.15.67:0/596005432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4009520 con 0x7fe2b8100f60 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.240+0000 7fe2b6000700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8106080 0x7fe2b81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.241+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b8100f60 0x7fe2b8199490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005910 tx=0x7fe2a8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.241+0000 7fe2af400700 1 -- 172.21.15.67:0/596005432 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a801d070 con 0x7fe2b8100f60 2024-03-30T02:39:06.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.241+0000 7fe2af400700 1 -- 172.21.15.67:0/596005432 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2a8022470 con 0x7fe2b8100f60 2024-03-30T02:39:06.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.241+0000 7fe2bd800700 1 -- 172.21.15.67:0/596005432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a80095f0 con 0x7fe2b8100f60 2024-03-30T02:39:06.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.241+0000 7fe2af400700 1 -- 172.21.15.67:0/596005432 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8017650 con 0x7fe2b8100f60 2024-03-30T02:39:06.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.241+0000 7fe2bd800700 1 -- 172.21.15.67:0/596005432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81a62b0 con 0x7fe2b8100f60 2024-03-30T02:39:06.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.244+0000 7fe2af400700 1 -- 172.21.15.67:0/596005432 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe2a80225e0 con 0x7fe2b8100f60 2024-03-30T02:39:06.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.245+0000 7fe2d13c6b80 1 Processor -- start 2024-03-30T02:39:06.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.246+0000 7fe2d13c6b80 1 -- start start 2024-03-30T02:39:06.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.246+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 0x55e1d60b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.246+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b2ef0 0x55e1d60bd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.246+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.246+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d600b990 con 0x55e1d60b2ef0 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.246+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d6009350 con 0x55e1d60bddf0 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.246+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d6004c40 con 0x55e1d60b25d0 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.246+0000 7fe2af400700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe29c0a1290 0x7fe29c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b2ef0 0x55e1d60bd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34394/0 (socket says 172.21.15.67:34394) 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6000700 1 -- 172.21.15.67:0/2125154050 learned_addr learned my addr 172.21.15.67:0/2125154050 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b7400700 1 --2- 172.21.15.67:0/2125154050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 0x55e1d60b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2af400700 1 -- 172.21.15.67:0/596005432 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fe2a80ce2a0 con 0x7fe2b8100f60 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b7400700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe29c0a1290 0x7fe29c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6000700 1 -- 172.21.15.67:0/2125154050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 msgr2=0x55e1d60b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6000700 1 --2- 172.21.15.67:0/2125154050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 0x55e1d60b29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6000700 1 -- 172.21.15.67:0/2125154050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b2ef0 msgr2=0x55e1d60bd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6000700 1 --2- 172.21.15.67:0/2125154050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b2ef0 0x55e1d60bd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6000700 1 -- 172.21.15.67:0/2125154050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1d5e863b0 con 0x55e1d60bddf0 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/2125154050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b2ef0 0x55e1d60bd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b6000700 1 --2- 172.21.15.67:0/2125154050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c01e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4011810 tx=0x7fe2a4011b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d25a60430d6584a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2b7400700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe29c0a1290 0x7fe29c0a3750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009ec0 tx=0x7fe2b000b040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.247+0000 7fe2a3e00700 1 -- 172.21.15.67:0/2125154050 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4004830 con 0x55e1d60bddf0 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2a3e00700 1 -- 172.21.15.67:0/2125154050 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2a4004e50 con 0x55e1d60bddf0 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2a3e00700 1 -- 172.21.15.67:0/2125154050 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400f7e0 con 0x55e1d60bddf0 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/2125154050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 msgr2=0x55e1d60c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/2125154050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c01e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4011810 tx=0x7fe2a4011b20 comp rx=0 tx=0).stop 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/2125154050 shutdown_connections 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/2125154050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c01e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/2125154050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 0x55e1d60b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/2125154050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b2ef0 0x55e1d60bd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/2125154050 >> 172.21.15.67:0/2125154050 conn(0x55e1d60126a0 msgr2=0x55e1d60aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/2125154050 shutdown_connections 2024-03-30T02:39:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.248+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/2125154050 wait complete. 2024-03-30T02:39:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.249+0000 7fe2d13c6b80 1 -- start start 2024-03-30T02:39:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.249+0000 7fe2d13c6b80 1 Processor -- start 2024-03-30T02:39:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.249+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b25d0 0x55e1d60bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.249+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 0x55e1d60bab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 0x55e1d60bb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d5fa0d80 con 0x55e1d60bddf0 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d5fa0a20 con 0x55e1d60b25d0 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d5e886a0 con 0x55e1d60b2ef0 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2d13c6b80 10 client.0 _ll_register_callbacks cb 0x55e1d5faa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b25d0 0x55e1d60bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b25d0 0x55e1d60bd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34402/0 (socket says 172.21.15.67:34402) 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b7400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 0x55e1d60bb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6a00700 1 -- 172.21.15.67:0/184372742 learned_addr learned my addr 172.21.15.67:0/184372742 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6000700 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 0x55e1d60bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6a00700 1 -- 172.21.15.67:0/184372742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 msgr2=0x55e1d60bab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 0x55e1d60bab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6a00700 1 -- 172.21.15.67:0/184372742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 msgr2=0x55e1d60bb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 0x55e1d60bb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6a00700 1 -- 172.21.15.67:0/184372742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a4011440 con 0x55e1d60b25d0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.250+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b25d0 0x55e1d60bd7c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800f740 tx=0x7fe2a802af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a49467a803f3a492 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2a0c00700 1 -- 172.21.15.67:0/184372742 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80973d0 con 0x55e1d60b25d0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2a0c00700 1 -- 172.21.15.67:0/184372742 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fe2a8066b50 con 0x55e1d60b25d0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/184372742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1d5fc5380 con 0x55e1d60b25d0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2a0c00700 1 -- 172.21.15.67:0/184372742 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2a809b790 con 0x55e1d60b25d0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2d13c6b80 10 client.26204 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/184372742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e1d5e32b60 con 0x55e1d60b25d0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2a0c00700 1 -- 172.21.15.67:0/184372742 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1d5fc5380 con 0x55e1d60b25d0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2a0c00700 1 -- 172.21.15.67:0/184372742 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe2a800cc50 con 0x55e1d60b25d0 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2d13c6b80 10 client.26204 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2d13c6b80 10 client.26204 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:39:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2d13c6b80 20 client.26204 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55e1d60cd540 0x55e1d60cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2d13c6b80 4 client.26204 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.251+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/184372742 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1d5ffd700 con 0x55e1d60cd540 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.252+0000 7fe2b6000700 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55e1d60cd540 0x55e1d60cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.252+0000 7fe2b6000700 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55e1d60cd540 0x55e1d60cf960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc579110808ed2c6 server_cookie=4317141fb3f93b05 in_seq=0 out_seq=0 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.252+0000 7fe2a0c00700 10 client.26204 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2a0c00700 1 -- 172.21.15.67:0/184372742 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e1d5ffd700 con 0x55e1d60cd540 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2a0c00700 10 client.26204 handle_command_reply: tid=0 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 client.26204 shutdown 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/184372742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b25d0 msgr2=0x55e1d60bd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b25d0 0x55e1d60bd7c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800f740 tx=0x7fe2a802af90 comp rx=0 tx=0).stop 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/184372742 shutdown_connections 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55e1d60cd540 0x55e1d60cf960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 0x55e1d60bb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b25d0 0x55e1d60bd7c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/184372742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 0x55e1d60bab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2a0c00700 0 client.26204 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:39:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/184372742 >> 172.21.15.67:0/184372742 conn(0x55e1d60126a0 msgr2=0x55e1d60b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.253+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/184372742 shutdown_connections 2024-03-30T02:39:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.254+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/184372742 wait complete. 2024-03-30T02:39:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.254+0000 7fe2d13c6b80 20 client.26204 trim_cache size 0 max 16384 2024-03-30T02:39:06.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.268+0000 7fe2d13c6b80 1 Processor -- start 2024-03-30T02:39:06.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.268+0000 7fe2d13c6b80 1 -- start start 2024-03-30T02:39:06.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.268+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 0x55e1d60adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.268+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b2ef0 0x55e1d60baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.268+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 0x55e1d60c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.268+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d5fc5380 con 0x55e1d60bddf0 2024-03-30T02:39:06.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.268+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d60b2460 con 0x55e1d60b2ef0 2024-03-30T02:39:06.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.268+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d60bc100 con 0x55e1d60b25d0 2024-03-30T02:39:06.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b6000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b2ef0 0x55e1d60baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b7400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 0x55e1d60c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 0x55e1d60adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b6000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b2ef0 0x55e1d60baac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34408/0 (socket says 172.21.15.67:34408) 2024-03-30T02:39:06.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b6000700 1 -- 172.21.15.67:0/3316610740 learned_addr learned my addr 172.21.15.67:0/3316610740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b7400700 1 -- 172.21.15.67:0/3316610740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 msgr2=0x55e1d60adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b7400700 1 --2- 172.21.15.67:0/3316610740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 0x55e1d60adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b7400700 1 -- 172.21.15.67:0/3316610740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b2ef0 msgr2=0x55e1d60baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b7400700 1 --2- 172.21.15.67:0/3316610740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b2ef0 0x55e1d60baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b7400700 1 -- 172.21.15.67:0/3316610740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1d5fcaea0 con 0x55e1d60bddf0 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b6000700 1 --2- 172.21.15.67:0/3316610740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b2ef0 0x55e1d60baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/3316610740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 0x55e1d60adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2b7400700 1 --2- 172.21.15.67:0/3316610740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 0x55e1d60c4aa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0007dc0 tx=0x7fe2b0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6fb2a300b6d0c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.269+0000 7fe2acc00700 1 -- 172.21.15.67:0/3316610740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0028070 con 0x55e1d60bddf0 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2acc00700 1 -- 172.21.15.67:0/3316610740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b000f040 con 0x55e1d60bddf0 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2acc00700 1 -- 172.21.15.67:0/3316610740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000e070 con 0x55e1d60bddf0 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3316610740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 msgr2=0x55e1d60c4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3316610740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 0x55e1d60c4aa0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0007dc0 tx=0x7fe2b0009920 comp rx=0 tx=0).stop 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3316610740 shutdown_connections 2024-03-30T02:39:06.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3316610740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60bddf0 0x55e1d60c4aa0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3316610740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b25d0 0x55e1d60adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3316610740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60b2ef0 0x55e1d60baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3316610740 >> 172.21.15.67:0/3316610740 conn(0x55e1d60126a0 msgr2=0x55e1d60ba390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3316610740 shutdown_connections 2024-03-30T02:39:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.270+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3316610740 wait complete. 2024-03-30T02:39:06.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.271+0000 7fe2d13c6b80 1 -- start start 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.271+0000 7fe2d13c6b80 1 Processor -- start 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.271+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b25d0 0x55e1d60c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.271+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 0x55e1d60c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.271+0000 7fe2d13c6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d60c3cd0 con 0x55e1d60b25d0 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d60c3e40 con 0x55e1d60bddf0 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2d13c6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1d6121380 con 0x55e1d60b2ef0 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2d13c6b80 10 client.0 _ll_register_callbacks cb 0x55e1d5faa470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b25d0 0x55e1d60c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b25d0 0x55e1d60c4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51852/0 (socket says 172.21.15.67:51852) 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6a00700 1 -- 172.21.15.67:0/3229751676 learned_addr learned my addr 172.21.15.67:0/3229751676 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6000700 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 0x55e1d60c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b7400700 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6a00700 1 -- 172.21.15.67:0/3229751676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 msgr2=0x55e1d60c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 0x55e1d60c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6a00700 1 -- 172.21.15.67:0/3229751676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 msgr2=0x55e1d60c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6a00700 1 -- 172.21.15.67:0/3229751676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1d5e32b60 con 0x55e1d60b25d0 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b7400700 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2b6a00700 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b25d0 0x55e1d60c4a70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80997c0 tx=0x7fe2a8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29918b847e77cfbd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.272+0000 7fe2a0c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2a0c00700 1 -- 172.21.15.67:0/3229751676 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a806a1c0 con 0x55e1d60b25d0 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2a0c00700 1 -- 172.21.15.67:0/3229751676 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fe2a8066b50 con 0x55e1d60b25d0 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3229751676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2a4004e50 con 0x55e1d60b25d0 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2a0c00700 1 -- 172.21.15.67:0/3229751676 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2a80973d0 con 0x55e1d60b25d0 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2d13c6b80 10 client.16158 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:06.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3229751676 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e1d5e863b0 con 0x55e1d60b25d0 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2a0c00700 1 -- 172.21.15.67:0/3229751676 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2a4004e50 con 0x55e1d60b25d0 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2a0c00700 1 -- 172.21.15.67:0/3229751676 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe2a809b2f0 con 0x55e1d60b25d0 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2d13c6b80 10 client.16158 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2d13c6b80 10 client.16158 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2d13c6b80 20 client.16158 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55e1d61225f0 0x55e1d6124a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2d13c6b80 4 client.16158 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.273+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3229751676 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e1d5e53cf0 con 0x55e1d61225f0 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.274+0000 7fe2b6000700 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55e1d61225f0 0x55e1d6124a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.274+0000 7fe2b6000700 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55e1d61225f0 0x55e1d6124a10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a3e034b319f316 server_cookie=46baa713f574d6ee in_seq=0 out_seq=0 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.274+0000 7fe2a0c00700 10 client.16158 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.274+0000 7fe2a0c00700 1 -- 172.21.15.67:0/3229751676 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e1d5e53cf0 con 0x55e1d61225f0 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.274+0000 7fe2a0c00700 10 client.16158 handle_command_reply: tid=0 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.274+0000 7fe2d13c6b80 1 client.16158 shutdown 2024-03-30T02:39:06.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.274+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3229751676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b25d0 msgr2=0x55e1d60c4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.274+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b25d0 0x55e1d60c4a70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80997c0 tx=0x7fe2a8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3229751676 shutdown_connections 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55e1d61225f0 0x55e1d6124a10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e1d60b25d0 0x55e1d60c4a70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e1d60bddf0 0x55e1d60c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2d13c6b80 1 --2- 172.21.15.67:0/3229751676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e1d60b2ef0 0x55e1d60c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3229751676 >> 172.21.15.67:0/3229751676 conn(0x55e1d60126a0 msgr2=0x55e1d60ba290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2a0c00700 0 client.16158 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3229751676 shutdown_connections 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2d13c6b80 1 -- 172.21.15.67:0/3229751676 wait complete. 2024-03-30T02:39:06.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2d13c6b80 20 client.16158 trim_cache size 0 max 16384 2024-03-30T02:39:06.281 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:06.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2bd800700 1 -- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe29c0a1290 msgr2=0x7fe29c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2bd800700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe29c0a1290 0x7fe29c0a3750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009ec0 tx=0x7fe2b000b040 comp rx=0 tx=0).stop 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2bd800700 1 -- 172.21.15.67:0/596005432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b8100f60 msgr2=0x7fe2b8199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.275+0000 7fe2bd800700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b8100f60 0x7fe2b8199490 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005910 tx=0x7fe2a8004730 comp rx=0 tx=0).stop 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.276+0000 7fe2bd800700 1 -- 172.21.15.67:0/596005432 shutdown_connections 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.276+0000 7fe2bd800700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe29c0a1290 0x7fe29c0a3750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.276+0000 7fe2bd800700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2b819dd70 0x7fe2b819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.276+0000 7fe2bd800700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b8100f60 0x7fe2b8199490 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.276+0000 7fe2bd800700 1 --2- 172.21.15.67:0/596005432 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2b8106080 0x7fe2b81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.276+0000 7fe2bd800700 1 -- 172.21.15.67:0/596005432 >> 172.21.15.67:0/596005432 conn(0x7fe2b80fc7c0 msgr2=0x7fe2b80fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.276+0000 7fe2bd800700 1 -- 172.21.15.67:0/596005432 shutdown_connections 2024-03-30T02:39:06.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.276+0000 7fe2bd800700 1 -- 172.21.15.67:0/596005432 wait complete. 2024-03-30T02:39:06.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:06 smithi067 ceph-mon[37808]: pgmap v64: 129 pgs: 129 active+clean; 165 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-03-30T02:39:06.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:06 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2931845419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:39:06.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:39:06.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.518+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2027110674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 msgr2=0x7fa4c410dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.518+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2027110674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 0x7fa4c410dc00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009f90 tx=0x7fa4bc00b330 comp rx=0 tx=0).stop 2024-03-30T02:39:06.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.518+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2027110674 shutdown_connections 2024-03-30T02:39:06.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.518+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2027110674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 0x7fa4c410dc00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.518+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2027110674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c4100f60 0x7fa4c4101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.518+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2027110674 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c4101910 0x7fa4c4105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.518+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2027110674 >> 172.21.15.67:0/2027110674 conn(0x7fa4c40fc7c0 msgr2=0x7fa4c40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.518+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2027110674 shutdown_connections 2024-03-30T02:39:06.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.518+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2027110674 wait complete. 2024-03-30T02:39:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.519+0000 7fa4c8a00700 1 Processor -- start 2024-03-30T02:39:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.520+0000 7fa4c8a00700 1 -- start start 2024-03-30T02:39:06.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.520+0000 7fa4c8a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c4100f60 0x7fa4c40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.520+0000 7fa4c8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 0x7fa4c40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.520+0000 7fa4c8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c4073b30 0x7fa4c41a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.520+0000 7fa4c8a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41a59b0 con 0x7fa4c4073b30 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.520+0000 7fa4c8a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41a5b20 con 0x7fa4c4106080 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.520+0000 7fa4c8a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41a5ce0 con 0x7fa4c4100f60 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.520+0000 7fa4c1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 0x7fa4c40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.520+0000 7fa4c2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c4073b30 0x7fa4c41a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 0x7fa4c40735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34434/0 (socket says 172.21.15.67:34434) 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c1400700 1 -- 172.21.15.67:0/2118126918 learned_addr learned my addr 172.21.15.67:0/2118126918 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c4100f60 0x7fa4c40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c1400700 1 -- 172.21.15.67:0/2118126918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c4100f60 msgr2=0x7fa4c40730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c1400700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c4100f60 0x7fa4c40730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c1400700 1 -- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c4073b30 msgr2=0x7fa4c41a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c1400700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c4073b30 0x7fa4c41a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c1400700 1 -- 172.21.15.67:0/2118126918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b00095f0 con 0x7fa4c4106080 2024-03-30T02:39:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c2800700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c4073b30 0x7fa4c41a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4c1400700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 0x7fa4c40735f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000ec70 tx=0x7fa4b000ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4baa00700 1 -- 172.21.15.67:0/2118126918 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00099d0 con 0x7fa4c4106080 2024-03-30T02:39:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4baa00700 1 -- 172.21.15.67:0/2118126918 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b00044e0 con 0x7fa4c4106080 2024-03-30T02:39:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.521+0000 7fa4baa00700 1 -- 172.21.15.67:0/2118126918 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0010430 con 0x7fa4c4106080 2024-03-30T02:39:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.522+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2118126918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4bc009c40 con 0x7fa4c4106080 2024-03-30T02:39:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.522+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2118126918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c41a6260 con 0x7fa4c4106080 2024-03-30T02:39:06.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.525+0000 7fa4baa00700 1 -- 172.21.15.67:0/2118126918 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa4b0003f10 con 0x7fa4c4106080 2024-03-30T02:39:06.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.526+0000 7fa4dc5fcb80 1 Processor -- start 2024-03-30T02:39:06.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.526+0000 7fa4dc5fcb80 1 -- start start 2024-03-30T02:39:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1ab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b0f6990 con 0x564a1b19def0 2024-03-30T02:39:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b0f4350 con 0x564a1b19d5d0 2024-03-30T02:39:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b0efc40 con 0x564a1b1a8df0 2024-03-30T02:39:06.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4baa00700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa4a80a1310 0x7fa4a80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4baa00700 1 -- 172.21.15.67:0/2118126918 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa4b0014070 con 0x7fa4c4106080 2024-03-30T02:39:06.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34444/0 (socket says 172.21.15.67:34444) 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c2800700 1 -- 172.21.15.67:0/4293333580 learned_addr learned my addr 172.21.15.67:0/4293333580 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1a88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51894/0 (socket says 172.21.15.67:51894) 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c1400700 1 --2- 172.21.15.67:0/4293333580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1ab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c2800700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa4a80a1310 0x7fa4a80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c1400700 1 -- 172.21.15.67:0/4293333580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 msgr2=0x564a1b19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c1400700 1 --2- 172.21.15.67:0/4293333580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c1400700 1 -- 172.21.15.67:0/4293333580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 msgr2=0x564a1b1a88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.527+0000 7fa4c1400700 1 --2- 172.21.15.67:0/4293333580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1a88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.528+0000 7fa4c1400700 1 -- 172.21.15.67:0/4293333580 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a1af713b0 con 0x564a1b1a8df0 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.528+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/4293333580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.528+0000 7fa4c1400700 1 --2- 172.21.15.67:0/4293333580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1ab1e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00ca780 tx=0x7fa4b00965b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9dbcef3347338b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.528+0000 7fa4af000700 1 -- 172.21.15.67:0/4293333580 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0003680 con 0x564a1b1a8df0 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.528+0000 7fa4c2800700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa4a80a1310 0x7fa4a80a37d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009040 tx=0x7fa4bc009720 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.528+0000 7fa4af000700 1 -- 172.21.15.67:0/4293333580 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b009a840 con 0x564a1b1a8df0 2024-03-30T02:39:06.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.528+0000 7fa4af000700 1 -- 172.21.15.67:0/4293333580 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b001fde0 con 0x564a1b1a8df0 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/4293333580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 msgr2=0x564a1b1ab1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/4293333580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1ab1e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00ca780 tx=0x7fa4b00965b0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/4293333580 shutdown_connections 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/4293333580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1ab1e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/4293333580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/4293333580 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/4293333580 >> 172.21.15.67:0/4293333580 conn(0x564a1b0fd6a0 msgr2=0x564a1b199a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/4293333580 shutdown_connections 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/4293333580 wait complete. 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.529+0000 7fa4dc5fcb80 1 -- start start 2024-03-30T02:39:06.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.530+0000 7fa4dc5fcb80 1 Processor -- start 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.530+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19d5d0 0x564a1b1a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.530+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.530+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1a6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.530+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b08bd80 con 0x564a1b19d5d0 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.530+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b08ba20 con 0x564a1b19def0 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.530+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1af736a0 con 0x564a1b1a8df0 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.530+0000 7fa4dc5fcb80 10 client.0 _ll_register_callbacks cb 0x564a1b095470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.530+0000 7fa4c1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19d5d0 0x564a1b1a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19d5d0 0x564a1b1a87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51898/0 (socket says 172.21.15.67:51898) 2024-03-30T02:39:06.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1e00700 1 -- 172.21.15.67:0/2017328100 learned_addr learned my addr 172.21.15.67:0/2017328100 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1400700 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c2800700 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1e00700 1 -- 172.21.15.67:0/2017328100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 msgr2=0x564a1b1a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1a6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1e00700 1 -- 172.21.15.67:0/2017328100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 msgr2=0x564a1b1a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1e00700 1 -- 172.21.15.67:0/2017328100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b40098e0 con 0x564a1b19d5d0 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1400700 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19d5d0 0x564a1b1a87c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40119d0 tx=0x7fa4b4011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dca82fbdd9289b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.531+0000 7fa4a3400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4a3400700 1 -- 172.21.15.67:0/2017328100 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4016070 con 0x564a1b19d5d0 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4a3400700 1 -- 172.21.15.67:0/2017328100 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa4b400fa40 con 0x564a1b19d5d0 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/2017328100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a1b0b0380 con 0x564a1b19d5d0 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4a3400700 1 -- 172.21.15.67:0/2017328100 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4b400fc00 con 0x564a1b19d5d0 2024-03-30T02:39:06.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4dc5fcb80 10 client.16170 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/2017328100 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fa4c41a6260 con 0x564a1b19d5d0 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4a3400700 1 -- 172.21.15.67:0/2017328100 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a1b0b0380 con 0x564a1b19d5d0 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4a3400700 1 -- 172.21.15.67:0/2017328100 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa4b4019390 con 0x564a1b19d5d0 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4dc5fcb80 10 client.16170 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4dc5fcb80 10 client.16170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4dc5fcb80 20 client.16170 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x564a1b1b89c0 0x564a1b1bade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4dc5fcb80 4 client.16170 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.532+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/2017328100 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a1b0e8700 con 0x564a1b1b89c0 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.533+0000 7fa4c1400700 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x564a1b1b89c0 0x564a1b1bade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.533+0000 7fa4c1400700 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x564a1b1b89c0 0x564a1b1bade0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36777d9b63d19c9e server_cookie=4d84917fbec7a377 in_seq=0 out_seq=0 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.533+0000 7fa4a3400700 10 client.16170 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4a3400700 1 -- 172.21.15.67:0/2017328100 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a1b0e8700 con 0x564a1b1b89c0 2024-03-30T02:39:06.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4a3400700 10 client.16170 handle_command_reply: tid=0 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 client.16170 shutdown 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/2017328100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19d5d0 msgr2=0x564a1b1a87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19d5d0 0x564a1b1a87c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40119d0 tx=0x7fa4b4011ce0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/2017328100 shutdown_connections 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x564a1b1b89c0 0x564a1b1bade0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19d5d0 0x564a1b1a87c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/2017328100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/2017328100 >> 172.21.15.67:0/2017328100 conn(0x564a1b0fd6a0 msgr2=0x564a1b19f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4a3400700 0 client.16170 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/2017328100 shutdown_connections 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.534+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/2017328100 wait complete. 2024-03-30T02:39:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.535+0000 7fa4dc5fcb80 20 client.16170 trim_cache size 0 max 16384 2024-03-30T02:39:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.548+0000 7fa4dc5fcb80 1 Processor -- start 2024-03-30T02:39:06.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4dc5fcb80 1 -- start start 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b1a6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b199fc0 con 0x564a1b19def0 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b1affb0 con 0x564a1b19d5d0 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b1a40c0 con 0x564a1b1a8df0 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4c1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b1a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4c1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4c1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b1a6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34466/0 (socket says 172.21.15.67:34466) 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4c1e00700 1 -- 172.21.15.67:0/1469645370 learned_addr learned my addr 172.21.15.67:0/1469645370 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4c2800700 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4c1e00700 1 -- 172.21.15.67:0/1469645370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 msgr2=0x564a1b1a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.549+0000 7fa4c1e00700 1 -- 172.21.15.67:0/1469645370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 msgr2=0x564a1b1afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.550+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.550+0000 7fa4c1e00700 1 -- 172.21.15.67:0/1469645370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a1af1db60 con 0x564a1b19d5d0 2024-03-30T02:39:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.550+0000 7fa4c2800700 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.550+0000 7fa4c1400700 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.550+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b1a6160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009340 tx=0x7fa4b400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=743e1ff76452a386 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.550+0000 7fa4afa00700 1 -- 172.21.15.67:0/1469645370 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4016070 con 0x564a1b19d5d0 2024-03-30T02:39:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.550+0000 7fa4afa00700 1 -- 172.21.15.67:0/1469645370 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b4019450 con 0x564a1b19d5d0 2024-03-30T02:39:06.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.550+0000 7fa4afa00700 1 -- 172.21.15.67:0/1469645370 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4005310 con 0x564a1b19d5d0 2024-03-30T02:39:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/1469645370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 msgr2=0x564a1b1a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b1a6160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009340 tx=0x7fa4b400f6a0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/1469645370 shutdown_connections 2024-03-30T02:39:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b1a8df0 0x564a1b1a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19d5d0 0x564a1b1a6160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/1469645370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b19def0 0x564a1b1afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/1469645370 >> 172.21.15.67:0/1469645370 conn(0x564a1b0fd6a0 msgr2=0x564a1b1ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/1469645370 shutdown_connections 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/1469645370 wait complete. 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 -- start start 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.551+0000 7fa4dc5fcb80 1 Processor -- start 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b19d5d0 0x564a1b1ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4dc5fcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b1a8df0 0x564a1b1adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b20b7f0 con 0x564a1b1a8df0 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b20b930 con 0x564a1b19def0 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4dc5fcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1b20ba70 con 0x564a1b19d5d0 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4dc5fcb80 10 client.0 _ll_register_callbacks cb 0x564a1b095470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34482/0 (socket says 172.21.15.67:34482) 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c1400700 1 -- 172.21.15.67:0/3865638451 learned_addr learned my addr 172.21.15.67:0/3865638451 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c2800700 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b1a8df0 0x564a1b1adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b19d5d0 0x564a1b1ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c1400700 1 -- 172.21.15.67:0/3865638451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b19d5d0 msgr2=0x564a1b1ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c1400700 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b19d5d0 0x564a1b1ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c1400700 1 -- 172.21.15.67:0/3865638451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b1a8df0 msgr2=0x564a1b1adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c1400700 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b1a8df0 0x564a1b1adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.552+0000 7fa4c1400700 1 -- 172.21.15.67:0/3865638451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a1af713b0 con 0x564a1b19def0 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4c1400700 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1ad5d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c5b0 tx=0x7fa4b009a9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64e2deccc198bc57 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4a3400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4a3400700 1 -- 172.21.15.67:0/3865638451 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b009df70 con 0x564a1b19def0 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4a3400700 1 -- 172.21.15.67:0/3865638451 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa4b009a140 con 0x564a1b19def0 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/3865638451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4b009a840 con 0x564a1b19def0 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4a3400700 1 -- 172.21.15.67:0/3865638451 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4b00984e0 con 0x564a1b19def0 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4dc5fcb80 10 client.26228 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:06.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/3865638451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fa4c41a6260 con 0x564a1b19def0 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4a3400700 1 -- 172.21.15.67:0/3865638451 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4b009a840 con 0x564a1b19def0 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4a3400700 1 -- 172.21.15.67:0/3865638451 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa4b009a3a0 con 0x564a1b19def0 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4dc5fcb80 10 client.26228 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4dc5fcb80 10 client.26228 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.553+0000 7fa4dc5fcb80 20 client.26228 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.554+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x564a1b20cf50 0x564a1b20f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.554+0000 7fa4dc5fcb80 4 client.26228 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.554+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/3865638451 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a1af3ecf0 con 0x564a1b20cf50 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.554+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x564a1b20cf50 0x564a1b20f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.554+0000 7fa4c1e00700 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x564a1b20cf50 0x564a1b20f370 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9b078a8fac52e1b server_cookie=1547821bbe404793 in_seq=0 out_seq=0 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.554+0000 7fa4a3400700 10 client.26228 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4a3400700 1 -- 172.21.15.67:0/3865638451 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a1af3ecf0 con 0x564a1b20cf50 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4a3400700 10 client.26228 handle_command_reply: tid=0 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 client.26228 shutdown 2024-03-30T02:39:06.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/3865638451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 msgr2=0x564a1b1ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1ad5d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c5b0 tx=0x7fa4b009a9b0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/3865638451 shutdown_connections 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x564a1b20cf50 0x564a1b20f370 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564a1b1a8df0 0x564a1b1adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a1b19d5d0 0x564a1b1ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4a3400700 0 client.26228 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 --2- 172.21.15.67:0/3865638451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a1b19def0 0x564a1b1ad5d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/3865638451 >> 172.21.15.67:0/3865638451 conn(0x564a1b0fd6a0 msgr2=0x564a1b1a0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/3865638451 shutdown_connections 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 1 -- 172.21.15.67:0/3865638451 wait complete. 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.555+0000 7fa4dc5fcb80 20 client.26228 trim_cache size 0 max 16384 2024-03-30T02:39:06.560 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa4a80a1310 msgr2=0x7fa4a80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa4a80a1310 0x7fa4a80a37d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009040 tx=0x7fa4bc009720 comp rx=0 tx=0).stop 2024-03-30T02:39:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2118126918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 msgr2=0x7fa4c40735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 0x7fa4c40735f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000ec70 tx=0x7fa4b000ef80 comp rx=0 tx=0).stop 2024-03-30T02:39:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2118126918 shutdown_connections 2024-03-30T02:39:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa4a80a1310 0x7fa4a80a37d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa4c4073b30 0x7fa4c41a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4c4100f60 0x7fa4c40730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 --2- 172.21.15.67:0/2118126918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4c4106080 0x7fa4c40735f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2118126918 >> 172.21.15.67:0/2118126918 conn(0x7fa4c40fc7c0 msgr2=0x7fa4c40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2118126918 shutdown_connections 2024-03-30T02:39:06.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.556+0000 7fa4c8a00700 1 -- 172.21.15.67:0/2118126918 wait complete. 2024-03-30T02:39:06.575 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:39:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:06 smithi170 ceph-mon[78798]: pgmap v64: 129 pgs: 129 active+clean; 165 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-03-30T02:39:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:06 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2931845419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:39:06.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:06 smithi082 ceph-mon[79176]: pgmap v64: 129 pgs: 129 active+clean; 165 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-03-30T02:39:06.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:06 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2931845419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:39:06.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.789+0000 7f6b19600700 1 -- 172.21.15.67:0/3442741077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14101910 msgr2=0x7f6b14105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.789+0000 7f6b19600700 1 --2- 172.21.15.67:0/3442741077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14101910 0x7f6b14105950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009990 tx=0x7f6b00009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.790+0000 7f6b19600700 1 -- 172.21.15.67:0/3442741077 shutdown_connections 2024-03-30T02:39:06.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.790+0000 7f6b19600700 1 --2- 172.21.15.67:0/3442741077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b14106080 0x7f6b1410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.790+0000 7f6b19600700 1 --2- 172.21.15.67:0/3442741077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14100f60 0x7f6b14101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.790+0000 7f6b19600700 1 --2- 172.21.15.67:0/3442741077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14101910 0x7f6b14105950 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.790+0000 7f6b19600700 1 -- 172.21.15.67:0/3442741077 >> 172.21.15.67:0/3442741077 conn(0x7f6b140fc7c0 msgr2=0x7f6b140febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.790+0000 7f6b19600700 1 -- 172.21.15.67:0/3442741077 shutdown_connections 2024-03-30T02:39:06.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.790+0000 7f6b19600700 1 -- 172.21.15.67:0/3442741077 wait complete. 2024-03-30T02:39:06.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.791+0000 7f6b19600700 1 Processor -- start 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.791+0000 7f6b19600700 1 -- start start 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.791+0000 7f6b19600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14100f60 0x7f6b14197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.791+0000 7f6b19600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14106080 0x7f6b14197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.791+0000 7f6b19600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b1419bae0 0x7f6b1419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b19600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419c560 con 0x7f6b1419bae0 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b19600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419c6d0 con 0x7f6b14106080 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b19600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419c890 con 0x7f6b14100f60 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b12000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14106080 0x7f6b14197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b12000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14106080 0x7f6b14197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34498/0 (socket says 172.21.15.67:34498) 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b13400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b1419bae0 0x7f6b1419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b12000700 1 -- 172.21.15.67:0/3073958438 learned_addr learned my addr 172.21.15.67:0/3073958438 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b12a00700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14100f60 0x7f6b14197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b12000700 1 -- 172.21.15.67:0/3073958438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14100f60 msgr2=0x7f6b14197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b12000700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14100f60 0x7f6b14197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b12000700 1 -- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b1419bae0 msgr2=0x7f6b1419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b12000700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b1419bae0 0x7f6b1419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b12000700 1 -- 172.21.15.67:0/3073958438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c009c40 con 0x7f6b14106080 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.792+0000 7f6b13400700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b1419bae0 0x7f6b1419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.793+0000 7f6b12000700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14106080 0x7f6b14197740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6b00000c00 tx=0x7f6b00004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.793+0000 7f6b0b400700 1 -- 172.21.15.67:0/3073958438 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0001d070 con 0x7f6b14106080 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.793+0000 7f6b0b400700 1 -- 172.21.15.67:0/3073958438 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b00021410 con 0x7f6b14106080 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.793+0000 7f6b0b400700 1 -- 172.21.15.67:0/3073958438 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00017650 con 0x7f6b14106080 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.793+0000 7f6b19600700 1 -- 172.21.15.67:0/3073958438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b000095f0 con 0x7f6b14106080 2024-03-30T02:39:06.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.793+0000 7f6b19600700 1 -- 172.21.15.67:0/3073958438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b141a4030 con 0x7f6b14106080 2024-03-30T02:39:06.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.799+0000 7f6b2d11fb80 1 Processor -- start 2024-03-30T02:39:06.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.799+0000 7f6b0b400700 1 -- 172.21.15.67:0/3073958438 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6b00021580 con 0x7f6b14106080 2024-03-30T02:39:06.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.799+0000 7f6b2d11fb80 1 -- start start 2024-03-30T02:39:06.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db7729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db772ef0 0x5615db781cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db782200 0x5615db7845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db6cb990 con 0x5615db782200 2024-03-30T02:39:06.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db6c9350 con 0x5615db772ef0 2024-03-30T02:39:06.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db6c4c40 con 0x5615db7725d0 2024-03-30T02:39:06.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b12000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db782200 0x5615db7845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b12a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db772ef0 0x5615db781cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b12a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db772ef0 0x5615db781cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34514/0 (socket says 172.21.15.67:34514) 2024-03-30T02:39:06.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b12a00700 1 -- 172.21.15.67:0/2321071221 learned_addr learned my addr 172.21.15.67:0/2321071221 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.800+0000 7f6b13400700 1 --2- 172.21.15.67:0/2321071221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db7729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6b12a00700 1 -- 172.21.15.67:0/2321071221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 msgr2=0x5615db7729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6b12a00700 1 --2- 172.21.15.67:0/2321071221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db7729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6b12a00700 1 -- 172.21.15.67:0/2321071221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db782200 msgr2=0x5615db7845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6b12a00700 1 --2- 172.21.15.67:0/2321071221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db782200 0x5615db7845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6b12a00700 1 -- 172.21.15.67:0/2321071221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615db5463b0 con 0x5615db772ef0 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6b12a00700 1 --2- 172.21.15.67:0/2321071221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db772ef0 0x5615db781cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009d80 tx=0x7f6b04011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d48e4ce7c2805d95 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6aff400700 1 -- 172.21.15.67:0/2321071221 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04004230 con 0x5615db772ef0 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6b0b400700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6af80a1290 0x7f6af80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6aff400700 1 -- 172.21.15.67:0/2321071221 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b04009e10 con 0x5615db772ef0 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6aff400700 1 -- 172.21.15.67:0/2321071221 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b040197e0 con 0x5615db772ef0 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6b0b400700 1 -- 172.21.15.67:0/3073958438 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f6b000ce0c0 con 0x7f6b14106080 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.801+0000 7f6b13400700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6af80a1290 0x7f6af80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/2321071221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db772ef0 msgr2=0x5615db781cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b13400700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6af80a1290 0x7f6af80a3750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009fd0 tx=0x7f6b0c00b040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/2321071221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db772ef0 0x5615db781cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009d80 tx=0x7f6b04011720 comp rx=0 tx=0).stop 2024-03-30T02:39:06.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/2321071221 shutdown_connections 2024-03-30T02:39:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/2321071221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db782200 0x5615db7845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/2321071221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db7729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/2321071221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db772ef0 0x5615db781cc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/2321071221 >> 172.21.15.67:0/2321071221 conn(0x5615db6d26a0 msgr2=0x5615db76ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/2321071221 shutdown_connections 2024-03-30T02:39:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.802+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/2321071221 wait complete. 2024-03-30T02:39:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.803+0000 7f6b2d11fb80 1 -- start start 2024-03-30T02:39:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.803+0000 7f6b2d11fb80 1 Processor -- start 2024-03-30T02:39:06.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.803+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db781c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db772ef0 0x5615db77ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db77f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db660d80 con 0x5615db7725d0 2024-03-30T02:39:06.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db660a20 con 0x5615db782200 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db5486a0 con 0x5615db772ef0 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b2d11fb80 10 client.0 _ll_register_callbacks cb 0x5615db66a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b12a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db781c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b12a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db781c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51956/0 (socket says 172.21.15.67:51956) 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b12a00700 1 -- 172.21.15.67:0/1978665925 learned_addr learned my addr 172.21.15.67:0/1978665925 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.804+0000 7f6b13400700 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db77f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6b12a00700 1 -- 172.21.15.67:0/1978665925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db772ef0 msgr2=0x5615db77ef30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6b12a00700 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db772ef0 0x5615db77ef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6b12a00700 1 -- 172.21.15.67:0/1978665925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 msgr2=0x5615db77f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6b12a00700 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db77f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6b12a00700 1 -- 172.21.15.67:0/1978665925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b040099e0 con 0x5615db7725d0 2024-03-30T02:39:06.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6b12a00700 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db781c40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6b04003c90 tx=0x7f6b04004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c66df52ba0d0d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6af3e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6af3e00700 1 -- 172.21.15.67:0/1978665925 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04005450 con 0x5615db7725d0 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6af3e00700 1 -- 172.21.15.67:0/1978665925 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f6b04005ce0 con 0x5615db7725d0 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.805+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/1978665925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615db685380 con 0x5615db7725d0 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6af3e00700 1 -- 172.21.15.67:0/1978665925 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b04003cf0 con 0x5615db7725d0 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6b2d11fb80 10 client.16188 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/1978665925 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5615db4f2b60 con 0x5615db7725d0 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6af3e00700 1 -- 172.21.15.67:0/1978665925 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615db685380 con 0x5615db7725d0 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6af3e00700 1 -- 172.21.15.67:0/1978665925 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6b04021040 con 0x5615db7725d0 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6b2d11fb80 10 client.16188 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6b2d11fb80 10 client.16188 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6b2d11fb80 20 client.16188 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5615db778050 0x5615db778430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6b2d11fb80 4 client.16188 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.806+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/1978665925 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615db6bd700 con 0x5615db778050 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.807+0000 7f6b12000700 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5615db778050 0x5615db778430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.807+0000 7f6b12000700 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5615db778050 0x5615db778430 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5171eb63d663910c server_cookie=b2d0d4ec8d555679 in_seq=0 out_seq=0 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.807+0000 7f6af3e00700 10 client.16188 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6af3e00700 1 -- 172.21.15.67:0/1978665925 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5615db6bd700 con 0x5615db778050 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6af3e00700 10 client.16188 handle_command_reply: tid=0 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 client.16188 shutdown 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/1978665925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 msgr2=0x5615db781c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db781c40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6b04003c90 tx=0x7f6b04004280 comp rx=0 tx=0).stop 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/1978665925 shutdown_connections 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5615db778050 0x5615db778430 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db781c40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db77f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/1978665925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db772ef0 0x5615db77ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/1978665925 >> 172.21.15.67:0/1978665925 conn(0x5615db6d26a0 msgr2=0x5615db774530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6af3e00700 0 client.16188 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:39:06.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/1978665925 shutdown_connections 2024-03-30T02:39:06.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.808+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/1978665925 wait complete. 2024-03-30T02:39:06.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.809+0000 7f6b2d11fb80 20 client.16188 trim_cache size 0 max 16384 2024-03-30T02:39:06.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b2d11fb80 1 Processor -- start 2024-03-30T02:39:06.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b2d11fb80 1 -- start start 2024-03-30T02:39:06.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db77eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db772ef0 0x5615db77f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db781770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db774140 con 0x5615db772ef0 2024-03-30T02:39:06.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db76f660 con 0x5615db782200 2024-03-30T02:39:06.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db76ef20 con 0x5615db7725d0 2024-03-30T02:39:06.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b12a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db77eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b13400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db781770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b12a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db77eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:48976/0 (socket says 172.21.15.67:48976) 2024-03-30T02:39:06.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b12a00700 1 -- 172.21.15.67:0/3192709463 learned_addr learned my addr 172.21.15.67:0/3192709463 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b13400700 1 --2- 172.21.15.67:0/3192709463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db781770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34526/0 (socket says 172.21.15.67:34526) 2024-03-30T02:39:06.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.823+0000 7f6b12000700 1 --2- 172.21.15.67:0/3192709463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db772ef0 0x5615db77f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.824+0000 7f6b12a00700 1 -- 172.21.15.67:0/3192709463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 msgr2=0x5615db781770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.824+0000 7f6b12a00700 1 --2- 172.21.15.67:0/3192709463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db781770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.824+0000 7f6b12a00700 1 -- 172.21.15.67:0/3192709463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db772ef0 msgr2=0x5615db77f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.824+0000 7f6b12a00700 1 --2- 172.21.15.67:0/3192709463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db772ef0 0x5615db77f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.824+0000 7f6b12a00700 1 -- 172.21.15.67:0/3192709463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615db68aea0 con 0x5615db7725d0 2024-03-30T02:39:06.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.824+0000 7f6b12a00700 1 --2- 172.21.15.67:0/3192709463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db77eef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b04003fa0 tx=0x7f6b04023690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c3dfd49428c3c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.824+0000 7f6affe00700 1 -- 172.21.15.67:0/3192709463 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04011a70 con 0x5615db7725d0 2024-03-30T02:39:06.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.824+0000 7f6affe00700 1 -- 172.21.15.67:0/3192709463 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b04023e60 con 0x5615db7725d0 2024-03-30T02:39:06.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.824+0000 7f6affe00700 1 -- 172.21.15.67:0/3192709463 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b040056e0 con 0x5615db7725d0 2024-03-30T02:39:06.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.825+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/3192709463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 msgr2=0x5615db77eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.825+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/3192709463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db77eef0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6b04003fa0 tx=0x7f6b04023690 comp rx=0 tx=0).stop 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.825+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/3192709463 shutdown_connections 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.825+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/3192709463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db781770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.825+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/3192709463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db7725d0 0x5615db77eef0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.825+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/3192709463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db772ef0 0x5615db77f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.825+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/3192709463 >> 172.21.15.67:0/3192709463 conn(0x5615db6d26a0 msgr2=0x5615db7675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.825+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/3192709463 shutdown_connections 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.825+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/3192709463 wait complete. 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.826+0000 7f6b2d11fb80 1 -- start start 2024-03-30T02:39:06.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.826+0000 7f6b2d11fb80 1 Processor -- start 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.826+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db798b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.826+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db772ef0 0x5615db799090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.826+0000 7f6b2d11fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db7e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.826+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db799770 con 0x5615db7725d0 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.826+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db7e9300 con 0x5615db782200 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.826+0000 7f6b2d11fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615db7e9440 con 0x5615db772ef0 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b2d11fb80 10 client.0 _ll_register_callbacks cb 0x5615db66a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db798b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db798b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51968/0 (socket says 172.21.15.67:51968) 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12a00700 1 -- 172.21.15.67:0/429630740 learned_addr learned my addr 172.21.15.67:0/429630740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12000700 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db772ef0 0x5615db799090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b13400700 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db7e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12a00700 1 -- 172.21.15.67:0/429630740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db772ef0 msgr2=0x5615db799090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12a00700 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db772ef0 0x5615db799090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12a00700 1 -- 172.21.15.67:0/429630740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 msgr2=0x5615db7e8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12a00700 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db7e8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12a00700 1 -- 172.21.15.67:0/429630740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615db4f2b60 con 0x5615db7725d0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6b12a00700 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db798b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009320 tx=0x7f6b04009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8212c6ff357968f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.827+0000 7f6af3e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6af3e00700 1 -- 172.21.15.67:0/429630740 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04023950 con 0x5615db7725d0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6af3e00700 1 -- 172.21.15.67:0/429630740 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f6b04023ab0 con 0x5615db7725d0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/429630740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b04009e10 con 0x5615db7725d0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6af3e00700 1 -- 172.21.15.67:0/429630740 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b04023c70 con 0x5615db7725d0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6b2d11fb80 10 client.16194 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/429630740 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5615db54acd0 con 0x5615db7725d0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6af3e00700 1 -- 172.21.15.67:0/429630740 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b04009e10 con 0x5615db7725d0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6af3e00700 1 -- 172.21.15.67:0/429630740 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6b04021070 con 0x5615db7725d0 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6b2d11fb80 10 client.16194 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:06.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6b2d11fb80 10 client.16194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:39:06.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.828+0000 7f6b2d11fb80 20 client.16194 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:06.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.829+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5615db7ea730 0x5615db7ecb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:06.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.829+0000 7f6b2d11fb80 4 client.16194 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:06.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.829+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/429630740 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615db513cf0 con 0x5615db7ea730 2024-03-30T02:39:06.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.829+0000 7f6b12000700 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5615db7ea730 0x5615db7ecb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:06.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.829+0000 7f6b12000700 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5615db7ea730 0x5615db7ecb20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a82d9c2cf41ce5b5 server_cookie=111ecd99b61d4e15 in_seq=0 out_seq=0 2024-03-30T02:39:06.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.829+0000 7f6af3e00700 10 client.16194 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:39:06.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.829+0000 7f6af3e00700 1 -- 172.21.15.67:0/429630740 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615db513cf0 con 0x5615db7ea730 2024-03-30T02:39:06.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.829+0000 7f6af3e00700 10 client.16194 handle_command_reply: tid=0 2024-03-30T02:39:06.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.829+0000 7f6b2d11fb80 1 client.16194 shutdown 2024-03-30T02:39:06.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/429630740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 msgr2=0x5615db798b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db798b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009320 tx=0x7f6b04009350 comp rx=0 tx=0).stop 2024-03-30T02:39:06.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/429630740 shutdown_connections 2024-03-30T02:39:06.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5615db7ea730 0x5615db7ecb20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5615db7725d0 0x5615db798b50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615db782200 0x5615db7e8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 --2- 172.21.15.67:0/429630740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5615db772ef0 0x5615db799090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/429630740 >> 172.21.15.67:0/429630740 conn(0x5615db6d26a0 msgr2=0x5615db7744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6af3e00700 0 client.16194 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:39:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/429630740 shutdown_connections 2024-03-30T02:39:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 1 -- 172.21.15.67:0/429630740 wait complete. 2024-03-30T02:39:06.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.830+0000 7f6b2d11fb80 20 client.16194 trim_cache size 0 max 16384 2024-03-30T02:39:06.835 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:06.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 -- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6af80a1290 msgr2=0x7f6af80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6af80a1290 0x7f6af80a3750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009fd0 tx=0x7f6b0c00b040 comp rx=0 tx=0).stop 2024-03-30T02:39:06.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 -- 172.21.15.67:0/3073958438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14106080 msgr2=0x7f6b14197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:06.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14106080 0x7f6b14197740 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6b00000c00 tx=0x7f6b00004730 comp rx=0 tx=0).stop 2024-03-30T02:39:06.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 -- 172.21.15.67:0/3073958438 shutdown_connections 2024-03-30T02:39:06.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6af80a1290 0x7f6af80a3750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6b1419bae0 0x7f6b1419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14100f60 0x7f6b14197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 --2- 172.21.15.67:0/3073958438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b14106080 0x7f6b14197740 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 -- 172.21.15.67:0/3073958438 >> 172.21.15.67:0/3073958438 conn(0x7f6b140fc7c0 msgr2=0x7f6b140fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 -- 172.21.15.67:0/3073958438 shutdown_connections 2024-03-30T02:39:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:06.831+0000 7f6b19600700 1 -- 172.21.15.67:0/3073958438 wait complete. 2024-03-30T02:39:06.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:39:07.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.065+0000 7f99ade00700 1 -- 172.21.15.67:0/2905468314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8100710 msgr2=0x7f99a8100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:07.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.065+0000 7f99ade00700 1 --2- 172.21.15.67:0/2905468314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8100710 0x7f99a8100af0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9998009990 tx=0x7f9998009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.065+0000 7f99ade00700 1 -- 172.21.15.67:0/2905468314 shutdown_connections 2024-03-30T02:39:07.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.065+0000 7f99ade00700 1 --2- 172.21.15.67:0/2905468314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99a810d750 0x7f99a810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.065+0000 7f99ade00700 1 --2- 172.21.15.67:0/2905468314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a8100710 0x7f99a8100af0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.065+0000 7f99ade00700 1 --2- 172.21.15.67:0/2905468314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a8101030 0x7f99a810d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.065+0000 7f99ade00700 1 -- 172.21.15.67:0/2905468314 >> 172.21.15.67:0/2905468314 conn(0x7f99a8076d40 msgr2=0x7f99a8077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:07.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.065+0000 7f99ade00700 1 -- 172.21.15.67:0/2905468314 shutdown_connections 2024-03-30T02:39:07.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.065+0000 7f99ade00700 1 -- 172.21.15.67:0/2905468314 wait complete. 2024-03-30T02:39:07.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.066+0000 7f99ade00700 1 Processor -- start 2024-03-30T02:39:07.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99ade00700 1 -- start start 2024-03-30T02:39:07.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99ade00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a8100710 0x7f99a8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99ade00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99a8101030 0x7f99a81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99ade00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a810d750 0x7f99a819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99ade00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a819e3a0 con 0x7f99a8101030 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99ade00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a819e510 con 0x7f99a810d750 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99ade00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a819e6d0 con 0x7f99a8100710 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99a6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a8100710 0x7f99a8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99a6000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99a8101030 0x7f99a81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99a7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a810d750 0x7f99a819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99a6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a8100710 0x7f99a8199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:49002/0 (socket says 172.21.15.67:49002) 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.067+0000 7f99a6a00700 1 -- 172.21.15.67:0/3087533488 learned_addr learned my addr 172.21.15.67:0/3087533488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99a6a00700 1 -- 172.21.15.67:0/3087533488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a810d750 msgr2=0x7f99a819dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:07.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99a6a00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a810d750 0x7f99a819dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99a6a00700 1 -- 172.21.15.67:0/3087533488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99a8101030 msgr2=0x7f99a81999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99a6a00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99a8101030 0x7f99a81999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99a6a00700 1 -- 172.21.15.67:0/3087533488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980095f0 con 0x7f99a8100710 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99a7400700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a810d750 0x7f99a819dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99a6a00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a8100710 0x7f99a8199480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9998000c00 tx=0x7f9998005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99aca00700 1 -- 172.21.15.67:0/3087533488 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999801d070 con 0x7f99a8100710 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99ade00700 1 -- 172.21.15.67:0/3087533488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a819e950 con 0x7f99a8100710 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99aca00700 1 -- 172.21.15.67:0/3087533488 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9998022470 con 0x7f99a8100710 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.068+0000 7f99aca00700 1 -- 172.21.15.67:0/3087533488 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998017650 con 0x7f99a8100710 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.069+0000 7f99ade00700 1 -- 172.21.15.67:0/3087533488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a81a61f0 con 0x7f99a8100710 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.072+0000 7f99aca00700 1 -- 172.21.15.67:0/3087533488 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f99980225e0 con 0x7f99a8100710 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.073+0000 7f99c196eb80 1 Processor -- start 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.073+0000 7f99c196eb80 1 -- start start 2024-03-30T02:39:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.073+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b231a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.073+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231aef0 0x5619b23258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.073+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b2325df0 0x5619b23281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.073+0000 7f99c196eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b2273990 con 0x5619b2325df0 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.073+0000 7f99c196eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b2271350 con 0x5619b231a5d0 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.073+0000 7f99c196eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b226cc40 con 0x5619b231aef0 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.073+0000 7f99aca00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f998c0a1290 0x7f998c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b231a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b231a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34552/0 (socket says 172.21.15.67:34552) 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 -- 172.21.15.67:0/935599273 learned_addr learned my addr 172.21.15.67:0/935599273 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99aca00700 1 -- 172.21.15.67:0/3087533488 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f99980ce430 con 0x7f99a8100710 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a6a00700 1 --2- 172.21.15.67:0/935599273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231aef0 0x5619b23258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f998c0a1290 0x7f998c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a6000700 1 --2- 172.21.15.67:0/935599273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b2325df0 0x5619b23281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 -- 172.21.15.67:0/935599273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231aef0 msgr2=0x5619b23258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 --2- 172.21.15.67:0/935599273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231aef0 0x5619b23258b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 -- 172.21.15.67:0/935599273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b2325df0 msgr2=0x5619b23281e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 --2- 172.21.15.67:0/935599273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b2325df0 0x5619b23281e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 -- 172.21.15.67:0/935599273 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619b20ee3b0 con 0x5619b231a5d0 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.074+0000 7f99a7400700 1 --2- 172.21.15.67:0/935599273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b231a9b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f99a0004a40 tx=0x7f99a0008d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ddc8445b99a6a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.075+0000 7f9993e00700 1 -- 172.21.15.67:0/935599273 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a000ee20 con 0x5619b231a5d0 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.075+0000 7f9993e00700 1 -- 172.21.15.67:0/935599273 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a000f040 con 0x5619b231a5d0 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.075+0000 7f99a7400700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f998c0a1290 0x7f998c0a3750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f99a0011000 tx=0x7f99a001e000 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.075+0000 7f9993e00700 1 -- 172.21.15.67:0/935599273 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0022bb0 con 0x5619b231a5d0 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.075+0000 7f99c196eb80 1 -- 172.21.15.67:0/935599273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 msgr2=0x5619b231a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.075+0000 7f99c196eb80 1 --2- 172.21.15.67:0/935599273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b231a9b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f99a0004a40 tx=0x7f99a0008d30 comp rx=0 tx=0).stop 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.076+0000 7f99c196eb80 1 -- 172.21.15.67:0/935599273 shutdown_connections 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.076+0000 7f99c196eb80 1 --2- 172.21.15.67:0/935599273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b2325df0 0x5619b23281e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.076+0000 7f99c196eb80 1 --2- 172.21.15.67:0/935599273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b231a9b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.076+0000 7f99c196eb80 1 --2- 172.21.15.67:0/935599273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231aef0 0x5619b23258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.076+0000 7f99c196eb80 1 -- 172.21.15.67:0/935599273 >> 172.21.15.67:0/935599273 conn(0x5619b227a6a0 msgr2=0x5619b2316a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.076+0000 7f99c196eb80 1 -- 172.21.15.67:0/935599273 shutdown_connections 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.076+0000 7f99c196eb80 1 -- 172.21.15.67:0/935599273 wait complete. 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.076+0000 7f99c196eb80 1 -- start start 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.077+0000 7f99c196eb80 1 Processor -- start 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.077+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231a5d0 0x5619b23257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.077+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231aef0 0x5619b2322a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.077+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 0x5619b2323080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.077+0000 7f99c196eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b2208d80 con 0x5619b231a5d0 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.077+0000 7f99c196eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b2208a20 con 0x5619b231aef0 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.077+0000 7f99c196eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b20f06a0 con 0x5619b2325df0 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.077+0000 7f99c196eb80 10 client.0 _ll_register_callbacks cb 0x5619b2212470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231a5d0 0x5619b23257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231a5d0 0x5619b23257f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52002/0 (socket says 172.21.15.67:52002) 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6a00700 1 -- 172.21.15.67:0/3456145606 learned_addr learned my addr 172.21.15.67:0/3456145606 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a7400700 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 0x5619b2323080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6000700 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231aef0 0x5619b2322a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6a00700 1 -- 172.21.15.67:0/3456145606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 msgr2=0x5619b2323080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6a00700 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 0x5619b2323080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6a00700 1 -- 172.21.15.67:0/3456145606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231aef0 msgr2=0x5619b2322a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6a00700 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231aef0 0x5619b2322a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6a00700 1 -- 172.21.15.67:0/3456145606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a0008630 con 0x5619b231a5d0 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a7400700 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 0x5619b2323080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f99a6a00700 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231a5d0 0x5619b23257f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f99980cb830 tx=0x7f99980cbd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c8847389bdb8ce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.078+0000 7f9990c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f9990c00700 1 -- 172.21.15.67:0/3456145606 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999809b110 con 0x5619b231a5d0 2024-03-30T02:39:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f9990c00700 1 -- 172.21.15.67:0/3456145606 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f999809ba40 con 0x5619b231a5d0 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f99c196eb80 1 -- 172.21.15.67:0/3456145606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619b222d380 con 0x5619b231a5d0 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f9990c00700 1 -- 172.21.15.67:0/3456145606 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999809ae80 con 0x5619b231a5d0 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f99c196eb80 10 client.16206 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f99c196eb80 1 -- 172.21.15.67:0/3456145606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5619b209ab60 con 0x5619b231a5d0 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f9990c00700 1 -- 172.21.15.67:0/3456145606 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619b222d380 con 0x5619b231a5d0 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f9990c00700 1 -- 172.21.15.67:0/3456145606 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f999809a830 con 0x5619b231a5d0 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f99c196eb80 10 client.16206 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f99c196eb80 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.079+0000 7f99c196eb80 20 client.16206 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.080+0000 7f99c196eb80 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619b23355c0 0x5619b23379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.080+0000 7f99c196eb80 4 client.16206 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.080+0000 7f99c196eb80 1 -- 172.21.15.67:0/3456145606 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619b2265700 con 0x5619b23355c0 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.080+0000 7f99a6000700 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619b23355c0 0x5619b23379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.080+0000 7f99a6000700 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619b23355c0 0x5619b23379e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f0c79d99c0fac9 server_cookie=a7d54102a3ebcb91 in_seq=0 out_seq=0 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.080+0000 7f9990c00700 10 client.16206 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f9990c00700 1 -- 172.21.15.67:0/3456145606 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5619b2265700 con 0x5619b23355c0 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f9990c00700 10 client.16206 handle_command_reply: tid=0 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 client.16206 shutdown 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 -- 172.21.15.67:0/3456145606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231a5d0 msgr2=0x5619b23257f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231a5d0 0x5619b23257f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f99980cb830 tx=0x7f99980cbd30 comp rx=0 tx=0).stop 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 -- 172.21.15.67:0/3456145606 shutdown_connections 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619b23355c0 0x5619b23379e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231a5d0 0x5619b23257f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 0x5619b2323080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 --2- 172.21.15.67:0/3456145606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231aef0 0x5619b2322a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f9990c00700 0 client.16206 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 -- 172.21.15.67:0/3456145606 >> 172.21.15.67:0/3456145606 conn(0x5619b227a6a0 msgr2=0x5619b231c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 -- 172.21.15.67:0/3456145606 shutdown_connections 2024-03-30T02:39:07.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 1 -- 172.21.15.67:0/3456145606 wait complete. 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.082+0000 7f99c196eb80 20 client.16206 trim_cache size 0 max 16384 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99c196eb80 1 Processor -- start 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99c196eb80 1 -- start start 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231a5d0 0x5619b232ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 0x5619b2316530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b2325df0 0x5619b231de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99c196eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b2323f00 con 0x5619b231aef0 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99c196eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b231b640 con 0x5619b2325df0 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99c196eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b23240e0 con 0x5619b231a5d0 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99a7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b2325df0 0x5619b231de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.096+0000 7f99a6000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 0x5619b2316530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b2325df0 0x5619b231de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34568/0 (socket says 172.21.15.67:34568) 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a7400700 1 -- 172.21.15.67:0/450156845 learned_addr learned my addr 172.21.15.67:0/450156845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:07.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a6a00700 1 --2- 172.21.15.67:0/450156845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231a5d0 0x5619b232ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a7400700 1 -- 172.21.15.67:0/450156845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231a5d0 msgr2=0x5619b232ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a7400700 1 --2- 172.21.15.67:0/450156845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231a5d0 0x5619b232ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a7400700 1 -- 172.21.15.67:0/450156845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 msgr2=0x5619b2316530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a7400700 1 --2- 172.21.15.67:0/450156845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 0x5619b2316530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a7400700 1 -- 172.21.15.67:0/450156845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619b2232ea0 con 0x5619b2325df0 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a6a00700 1 --2- 172.21.15.67:0/450156845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231a5d0 0x5619b232ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f99a7400700 1 --2- 172.21.15.67:0/450156845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b2325df0 0x5619b231de10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f99a0022760 tx=0x7f99a0022790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b41f29040301ad1b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f999d200700 1 -- 172.21.15.67:0/450156845 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a000ee20 con 0x5619b2325df0 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f999d200700 1 -- 172.21.15.67:0/450156845 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a0009550 con 0x5619b2325df0 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.097+0000 7f999d200700 1 -- 172.21.15.67:0/450156845 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a001ae00 con 0x5619b2325df0 2024-03-30T02:39:07.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.098+0000 7f99c196eb80 1 -- 172.21.15.67:0/450156845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b2325df0 msgr2=0x5619b231de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.098+0000 7f99c196eb80 1 --2- 172.21.15.67:0/450156845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b2325df0 0x5619b231de10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f99a0022760 tx=0x7f99a0022790 comp rx=0 tx=0).stop 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.098+0000 7f99c196eb80 1 -- 172.21.15.67:0/450156845 shutdown_connections 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.098+0000 7f99c196eb80 1 --2- 172.21.15.67:0/450156845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b2325df0 0x5619b231de10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.098+0000 7f99c196eb80 1 --2- 172.21.15.67:0/450156845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b231a5d0 0x5619b232ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.098+0000 7f99c196eb80 1 --2- 172.21.15.67:0/450156845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 0x5619b2316530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.098+0000 7f99c196eb80 1 -- 172.21.15.67:0/450156845 >> 172.21.15.67:0/450156845 conn(0x5619b227a6a0 msgr2=0x5619b230f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.098+0000 7f99c196eb80 1 -- 172.21.15.67:0/450156845 shutdown_connections 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.098+0000 7f99c196eb80 1 -- 172.21.15.67:0/450156845 wait complete. 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.099+0000 7f99c196eb80 1 -- start start 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.099+0000 7f99c196eb80 1 Processor -- start 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.099+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b232c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.099+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 0x5619b232ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.099+0000 7f99c196eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 0x5619b232b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.099+0000 7f99c196eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b232b920 con 0x5619b231aef0 2024-03-30T02:39:07.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.099+0000 7f99c196eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b232ba90 con 0x5619b231a5d0 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.099+0000 7f99c196eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619b232bc50 con 0x5619b2325df0 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99c196eb80 10 client.0 _ll_register_callbacks cb 0x5619b2212470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b232c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b232c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34572/0 (socket says 172.21.15.67:34572) 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6a00700 1 -- 172.21.15.67:0/271652446 learned_addr learned my addr 172.21.15.67:0/271652446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6000700 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 0x5619b232ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6a00700 1 -- 172.21.15.67:0/271652446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 msgr2=0x5619b232b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6a00700 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 0x5619b232b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6a00700 1 -- 172.21.15.67:0/271652446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 msgr2=0x5619b232ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6a00700 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 0x5619b232ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6a00700 1 -- 172.21.15.67:0/271652446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619b20ee3b0 con 0x5619b231a5d0 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.100+0000 7f99a6000700 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 0x5619b232ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f99a6a00700 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b232c960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f999800f740 tx=0x7f99980ce670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c472116d2801d5f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:07.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f9990c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f9990c00700 1 -- 172.21.15.67:0/271652446 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999809aa30 con 0x5619b231a5d0 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f9990c00700 1 -- 172.21.15.67:0/271652446 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f999809ba40 con 0x5619b231a5d0 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f99c196eb80 1 -- 172.21.15.67:0/271652446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99a000f040 con 0x5619b231a5d0 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f9990c00700 1 -- 172.21.15.67:0/271652446 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999809ae80 con 0x5619b231a5d0 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f99c196eb80 10 client.26264 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f99c196eb80 1 -- 172.21.15.67:0/271652446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5619b20f2cd0 con 0x5619b231a5d0 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f9990c00700 1 -- 172.21.15.67:0/271652446 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a000f040 con 0x5619b231a5d0 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.101+0000 7f9990c00700 1 -- 172.21.15.67:0/271652446 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f99980a1100 con 0x5619b231a5d0 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f99c196eb80 10 client.26264 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f99c196eb80 10 client.26264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f99c196eb80 20 client.26264 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f99c196eb80 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619b238a540 0x5619b238c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f99c196eb80 4 client.26264 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f99c196eb80 1 -- 172.21.15.67:0/271652446 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5619b20bbcf0 con 0x5619b238a540 2024-03-30T02:39:07.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f99a6000700 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619b238a540 0x5619b238c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f9990c00700 10 client.26264 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f99a6000700 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619b238a540 0x5619b238c960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb9157ea98047f1d server_cookie=f5eac2c6b78bf0e9 in_seq=0 out_seq=0 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.102+0000 7f9990c00700 1 -- 172.21.15.67:0/271652446 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5619b20bbcf0 con 0x5619b238a540 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f9990c00700 10 client.26264 handle_command_reply: tid=0 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 client.26264 shutdown 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 -- 172.21.15.67:0/271652446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 msgr2=0x5619b232c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b232c960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f999800f740 tx=0x7f99980ce670 comp rx=0 tx=0).stop 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 -- 172.21.15.67:0/271652446 shutdown_connections 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619b2325df0 0x5619b232b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619b231a5d0 0x5619b232c960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619b238a540 0x5619b238c960 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 --2- 172.21.15.67:0/271652446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619b231aef0 0x5619b232ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f9990c00700 0 client.26264 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:07.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 -- 172.21.15.67:0/271652446 >> 172.21.15.67:0/271652446 conn(0x5619b227a6a0 msgr2=0x5619b231de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:07.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 -- 172.21.15.67:0/271652446 shutdown_connections 2024-03-30T02:39:07.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 1 -- 172.21.15.67:0/271652446 wait complete. 2024-03-30T02:39:07.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.103+0000 7f99c196eb80 20 client.26264 trim_cache size 0 max 16384 2024-03-30T02:39:07.133 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 -- 172.21.15.67:0/3087533488 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f998c0a1290 msgr2=0x7f998c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f998c0a1290 0x7f998c0a3750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f99a0011000 tx=0x7f99a001e000 comp rx=0 tx=0).stop 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 -- 172.21.15.67:0/3087533488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a8100710 msgr2=0x7f99a8199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a8100710 0x7f99a8199480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9998000c00 tx=0x7f9998005740 comp rx=0 tx=0).stop 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 -- 172.21.15.67:0/3087533488 shutdown_connections 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f998c0a1290 0x7f998c0a3750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a810d750 0x7f99a819dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a8100710 0x7f99a8199480 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 --2- 172.21.15.67:0/3087533488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f99a8101030 0x7f99a81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:07.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 -- 172.21.15.67:0/3087533488 >> 172.21.15.67:0/3087533488 conn(0x7f99a8076d40 msgr2=0x7f99a810ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:07.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 -- 172.21.15.67:0/3087533488 shutdown_connections 2024-03-30T02:39:07.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:07.104+0000 7f99ade00700 1 -- 172.21.15.67:0/3087533488 wait complete. 2024-03-30T02:39:08.135 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:39:08.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:39:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.346+0000 7f9a9e800700 1 -- 172.21.15.67:0/2337822538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a98073b00 msgr2=0x7f9a9810d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.346+0000 7f9a9e800700 1 --2- 172.21.15.67:0/2337822538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a98073b00 0x7f9a9810d000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009990 tx=0x7f9a84009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.347+0000 7f9a9e800700 1 -- 172.21.15.67:0/2337822538 shutdown_connections 2024-03-30T02:39:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.347+0000 7f9a9e800700 1 --2- 172.21.15.67:0/2337822538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a9810d540 0x7f9a9810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.347+0000 7f9a9e800700 1 --2- 172.21.15.67:0/2337822538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a980731e0 0x7f9a980735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.347+0000 7f9a9e800700 1 --2- 172.21.15.67:0/2337822538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a98073b00 0x7f9a9810d000 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.347+0000 7f9a9e800700 1 -- 172.21.15.67:0/2337822538 >> 172.21.15.67:0/2337822538 conn(0x7f9a980fc7f0 msgr2=0x7f9a980fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.347+0000 7f9a9e800700 1 -- 172.21.15.67:0/2337822538 shutdown_connections 2024-03-30T02:39:08.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.347+0000 7f9a9e800700 1 -- 172.21.15.67:0/2337822538 wait complete. 2024-03-30T02:39:08.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.348+0000 7f9a9e800700 1 Processor -- start 2024-03-30T02:39:08.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.348+0000 7f9a9e800700 1 -- start start 2024-03-30T02:39:08.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.348+0000 7f9a9e800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a980731e0 0x7f9a98199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.348+0000 7f9a9e800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a98073b00 0x7f9a98199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a9e800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a9810d540 0x7f9a9819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a9e800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9819e330 con 0x7f9a9810d540 2024-03-30T02:39:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a9e800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9819e4a0 con 0x7f9a980731e0 2024-03-30T02:39:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a9e800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9819e660 con 0x7f9a98073b00 2024-03-30T02:39:08.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a980731e0 0x7f9a98199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a980731e0 0x7f9a98199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34580/0 (socket says 172.21.15.67:34580) 2024-03-30T02:39:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97400700 1 -- 172.21.15.67:0/1131011728 learned_addr learned my addr 172.21.15.67:0/1131011728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97e00700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a9810d540 0x7f9a9819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a96a00700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a98073b00 0x7f9a98199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97400700 1 -- 172.21.15.67:0/1131011728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a98073b00 msgr2=0x7f9a98199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97400700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a98073b00 0x7f9a98199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97400700 1 -- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a9810d540 msgr2=0x7f9a9819dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97400700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a9810d540 0x7f9a9819dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97400700 1 -- 172.21.15.67:0/1131011728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a840095f0 con 0x7f9a980731e0 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.349+0000 7f9a97400700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a980731e0 0x7f9a98199410 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e9a0 tx=0x7f9a8800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.350+0000 7f9a8fe00700 1 -- 172.21.15.67:0/1131011728 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88009790 con 0x7f9a980731e0 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.350+0000 7f9a8fe00700 1 -- 172.21.15.67:0/1131011728 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a8800cbd0 con 0x7f9a980731e0 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.350+0000 7f9a9e800700 1 -- 172.21.15.67:0/1131011728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9819e940 con 0x7f9a980731e0 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.350+0000 7f9a8fe00700 1 -- 172.21.15.67:0/1131011728 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88018680 con 0x7f9a980731e0 2024-03-30T02:39:08.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.350+0000 7f9a9e800700 1 -- 172.21.15.67:0/1131011728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a981a6180 con 0x7f9a980731e0 2024-03-30T02:39:08.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.353+0000 7f9a8fe00700 1 -- 172.21.15.67:0/1131011728 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9a8800cd40 con 0x7f9a980731e0 2024-03-30T02:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.355+0000 7f9ab2349b80 1 Processor -- start 2024-03-30T02:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.355+0000 7f9a8fe00700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9a7c0a1290 0x7f9a7c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.355+0000 7f9ab2349b80 1 -- start start 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.355+0000 7f9a8fe00700 1 -- 172.21.15.67:0/1131011728 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9a88014070 con 0x7f9a980731e0 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.355+0000 7f9a97e00700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9a7c0a1290 0x7f9a7c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.355+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585585a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f585593260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5854dea10 con 0x55f585590e70 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5854dc3d0 con 0x55f585585f70 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5854d7cc0 con 0x55f585585650 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a97400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585585a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34590/0 (socket says 172.21.15.67:34590) 2024-03-30T02:39:08.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a96a00700 1 -- 172.21.15.67:0/3426856364 learned_addr learned my addr 172.21.15.67:0/3426856364 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a97e00700 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f585593260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a96a00700 1 -- 172.21.15.67:0/3426856364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 msgr2=0x55f585585a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a96a00700 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585585a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a96a00700 1 -- 172.21.15.67:0/3426856364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 msgr2=0x55f585593260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a96a00700 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f585593260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a96a00700 1 -- 172.21.15.67:0/3426856364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f585359430 con 0x55f585585f70 2024-03-30T02:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a97400700 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585585a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.356+0000 7f9a97e00700 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f585593260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:39:08.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.357+0000 7f9a97e00700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9a7c0a1290 0x7f9a7c0a3750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9a90003e90 tx=0x7f9a90009100 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.357+0000 7f9a96a00700 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009990 tx=0x7f9a84005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74dc0554de55c9af server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.357+0000 7f9a8cc00700 1 -- 172.21.15.67:0/3426856364 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a840056a0 con 0x55f585585f70 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.357+0000 7f9a8cc00700 1 -- 172.21.15.67:0/3426856364 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a8401e040 con 0x55f585585f70 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.357+0000 7f9a8cc00700 1 -- 172.21.15.67:0/3426856364 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004af0 con 0x55f585585f70 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3426856364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 msgr2=0x55f585590930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009990 tx=0x7f9a84005f00 comp rx=0 tx=0).stop 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3426856364 shutdown_connections 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f585593260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585585a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3426856364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590930 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3426856364 >> 172.21.15.67:0/3426856364 conn(0x55f5854e5720 msgr2=0x55f585581b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3426856364 shutdown_connections 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3426856364 wait complete. 2024-03-30T02:39:08.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.358+0000 7f9ab2349b80 1 -- start start 2024-03-30T02:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.359+0000 7f9ab2349b80 1 Processor -- start 2024-03-30T02:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.359+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585650 0x55f585590800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.359+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585f70 0x55f58558daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.359+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 0x55f58558e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.359+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f585473e00 con 0x55f585585f70 2024-03-30T02:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.359+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f585473aa0 con 0x55f585585650 2024-03-30T02:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.359+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58535b720 con 0x55f585590e70 2024-03-30T02:39:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.359+0000 7f9ab2349b80 10 client.0 _ll_register_callbacks cb 0x55f58547d4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.359+0000 7f9a97400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585650 0x55f585590800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a97400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585650 0x55f585590800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34598/0 (socket says 172.21.15.67:34598) 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a97400700 1 -- 172.21.15.67:0/3751831772 learned_addr learned my addr 172.21.15.67:0/3751831772 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a96a00700 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585f70 0x55f58558daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a97e00700 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 0x55f58558e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a97400700 1 -- 172.21.15.67:0/3751831772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 msgr2=0x55f58558e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a97400700 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 0x55f58558e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a97400700 1 -- 172.21.15.67:0/3751831772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585f70 msgr2=0x55f58558daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a97400700 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585f70 0x55f58558daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a97400700 1 -- 172.21.15.67:0/3751831772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a981a6180 con 0x55f585585650 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a97400700 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585650 0x55f585590800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a880ca6f0 tx=0x7f9a88099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97cb1732cd9634fd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a81600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:08.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a81600700 1 -- 172.21.15.67:0/3751831772 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8809a750 con 0x55f585585650 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9a81600700 1 -- 172.21.15.67:0/3751831772 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9a8809a050 con 0x55f585585650 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.360+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3751831772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f585498400 con 0x55f585585650 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9a81600700 1 -- 172.21.15.67:0/3751831772 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a8809de80 con 0x55f585585650 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9ab2349b80 10 client.26276 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3751831772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f9a98110670 con 0x55f585585650 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9a81600700 1 -- 172.21.15.67:0/3751831772 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a88063070 con 0x55f585585650 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9a81600700 1 -- 172.21.15.67:0/3751831772 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9a880b8080 con 0x55f585585650 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9ab2349b80 10 client.26276 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9ab2349b80 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9ab2349b80 20 client.26276 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f5855a0a30 0x55f5855a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9ab2349b80 4 client.26276 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3751831772 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5854d0780 con 0x55f5855a0a30 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.361+0000 7f9a96a00700 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f5855a0a30 0x55f5855a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.362+0000 7f9a96a00700 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f5855a0a30 0x55f5855a2e50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b414128f534f91e3 server_cookie=d8e0f92350889db7 in_seq=0 out_seq=0 2024-03-30T02:39:08.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.362+0000 7f9a81600700 10 client.26276 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.363+0000 7f9a81600700 1 -- 172.21.15.67:0/3751831772 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f5854d0780 con 0x55f5855a0a30 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.363+0000 7f9a81600700 10 client.26276 handle_command_reply: tid=0 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.363+0000 7f9ab2349b80 1 client.26276 shutdown 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3751831772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585650 msgr2=0x55f585590800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585650 0x55f585590800 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a880ca6f0 tx=0x7f9a88099a30 comp rx=0 tx=0).stop 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3751831772 shutdown_connections 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 0x55f58558e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585650 0x55f585590800 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f5855a0a30 0x55f5855a2e50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3751831772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585f70 0x55f58558daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3751831772 >> 172.21.15.67:0/3751831772 conn(0x55f5854e5720 msgr2=0x55f585587740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9a81600700 0 client.26276 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3751831772 shutdown_connections 2024-03-30T02:39:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3751831772 wait complete. 2024-03-30T02:39:08.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.364+0000 7f9ab2349b80 20 client.26276 trim_cache size 0 max 16384 2024-03-30T02:39:08.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.383+0000 7f9ab2349b80 1 Processor -- start 2024-03-30T02:39:08.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.383+0000 7f9ab2349b80 1 -- start start 2024-03-30T02:39:08.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585650 0x55f585597ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 0x55f58558d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5855871c0 con 0x55f585585650 2024-03-30T02:39:08.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58558e690 con 0x55f585585f70 2024-03-30T02:39:08.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f585582850 con 0x55f585590e70 2024-03-30T02:39:08.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a97400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585650 0x55f585597ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34604/0 (socket says 172.21.15.67:34604) 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a96a00700 1 -- 172.21.15.67:0/1271232806 learned_addr learned my addr 172.21.15.67:0/1271232806 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a97e00700 1 --2- 172.21.15.67:0/1271232806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 0x55f58558d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a96a00700 1 -- 172.21.15.67:0/1271232806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 msgr2=0x55f58558d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a96a00700 1 --2- 172.21.15.67:0/1271232806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 0x55f58558d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a96a00700 1 -- 172.21.15.67:0/1271232806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585650 msgr2=0x55f585597ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a96a00700 1 --2- 172.21.15.67:0/1271232806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585650 0x55f585597ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a96a00700 1 -- 172.21.15.67:0/1271232806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5855a7a40 con 0x55f585585f70 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.384+0000 7f9a97400700 1 --2- 172.21.15.67:0/1271232806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585650 0x55f585597ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.385+0000 7f9a96a00700 1 --2- 172.21.15.67:0/1271232806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a84006010 tx=0x7f9a84004630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0632702d9a2e40d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.385+0000 7f9a8d600700 1 -- 172.21.15.67:0/1271232806 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84030cd0 con 0x55f585585f70 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.385+0000 7f9a8d600700 1 -- 172.21.15.67:0/1271232806 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a84030e30 con 0x55f585585f70 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.385+0000 7f9a8d600700 1 -- 172.21.15.67:0/1271232806 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004740 con 0x55f585585f70 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.385+0000 7f9ab2349b80 1 -- 172.21.15.67:0/1271232806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 msgr2=0x55f585590330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.385+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/1271232806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9a84006010 tx=0x7f9a84004630 comp rx=0 tx=0).stop 2024-03-30T02:39:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.386+0000 7f9ab2349b80 1 -- 172.21.15.67:0/1271232806 shutdown_connections 2024-03-30T02:39:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.386+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/1271232806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585585650 0x55f585597ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.386+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/1271232806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585590e70 0x55f58558d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.386+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/1271232806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585590330 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.386+0000 7f9ab2349b80 1 -- 172.21.15.67:0/1271232806 >> 172.21.15.67:0/1271232806 conn(0x55f5854e5720 msgr2=0x55f58557aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.386+0000 7f9ab2349b80 1 -- 172.21.15.67:0/1271232806 shutdown_connections 2024-03-30T02:39:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.386+0000 7f9ab2349b80 1 -- 172.21.15.67:0/1271232806 wait complete. 2024-03-30T02:39:08.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.386+0000 7f9ab2349b80 1 -- start start 2024-03-30T02:39:08.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.386+0000 7f9ab2349b80 1 Processor -- start 2024-03-30T02:39:08.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585596900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585596e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9ab2349b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f5855f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5855975d0 con 0x55f585590e70 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5855f4da0 con 0x55f585585f70 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9ab2349b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5855f4f60 con 0x55f585585650 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9ab2349b80 10 client.0 _ll_register_callbacks cb 0x55f58547d4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a97400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585596900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a96a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585596e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a97e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f5855f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a97400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585596900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:49112/0 (socket says 172.21.15.67:49112) 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a97400700 1 -- 172.21.15.67:0/3497061890 learned_addr learned my addr 172.21.15.67:0/3497061890 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a97400700 1 -- 172.21.15.67:0/3497061890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 msgr2=0x55f585596e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a97400700 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585596e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a97400700 1 -- 172.21.15.67:0/3497061890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 msgr2=0x55f5855f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a97400700 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f5855f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.387+0000 7f9a97400700 1 -- 172.21.15.67:0/3497061890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a98110670 con 0x55f585585650 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9a97e00700 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f5855f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9a96a00700 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585596e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9a97400700 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585596900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a88099b20 tx=0x7f9a8800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a195912e40c69b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9a81600700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9a81600700 1 -- 172.21.15.67:0/3497061890 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8807a070 con 0x55f585585650 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9a81600700 1 -- 172.21.15.67:0/3497061890 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9a8809a750 con 0x55f585585650 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3497061890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a8401e040 con 0x55f585585650 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9a81600700 1 -- 172.21.15.67:0/3497061890 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a88099e20 con 0x55f585585650 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9ab2349b80 10 client.26257 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.388+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3497061890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f585359430 con 0x55f585585650 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9a81600700 1 -- 172.21.15.67:0/3497061890 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a880a6020 con 0x55f585585650 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9a81600700 1 -- 172.21.15.67:0/3497061890 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9a88096070 con 0x55f585585650 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9ab2349b80 10 client.26257 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9ab2349b80 10 client.26257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9ab2349b80 20 client.26257 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:08.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f5855f6630 0x55f5855f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9ab2349b80 4 client.26257 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:39:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3497061890 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f585326d70 con 0x55f5855f6630 2024-03-30T02:39:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9a96a00700 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f5855f6630 0x55f5855f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9a96a00700 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f5855f6630 0x55f5855f8a50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ffdeda0d03b3ff server_cookie=6deddab2fdac89bf in_seq=0 out_seq=0 2024-03-30T02:39:08.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.389+0000 7f9a81600700 10 client.26257 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.462+0000 7f9a81600700 1 -- 172.21.15.67:0/3497061890 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f585326d70 con 0x55f5855f6630 2024-03-30T02:39:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.462+0000 7f9a81600700 10 client.26257 handle_command_reply: tid=0 2024-03-30T02:39:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.462+0000 7f9ab2349b80 1 client.26257 shutdown 2024-03-30T02:39:08.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.462+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3497061890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 msgr2=0x55f585596900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.462+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585596900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a88099b20 tx=0x7f9a8800eef0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.463+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3497061890 shutdown_connections 2024-03-30T02:39:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.463+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f5855f6630 0x55f5855f8a50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.463+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f585590e70 0x55f5855f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.463+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f585585650 0x55f585596900 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.463+0000 7f9a81600700 0 client.26257 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.463+0000 7f9ab2349b80 1 --2- 172.21.15.67:0/3497061890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f585585f70 0x55f585596e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.463+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3497061890 >> 172.21.15.67:0/3497061890 conn(0x55f5854e5720 msgr2=0x55f58558d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.464+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3497061890 shutdown_connections 2024-03-30T02:39:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.465+0000 7f9ab2349b80 1 -- 172.21.15.67:0/3497061890 wait complete. 2024-03-30T02:39:08.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.465+0000 7f9ab2349b80 20 client.26257 trim_cache size 0 max 16384 2024-03-30T02:39:08.466 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:39:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:39:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "b41bed7a-70c8-4f10-8792-8d2296a8b63c", 2024-03-30T02:39:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:39:08.466 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:39:08.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 -- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9a7c0a1290 msgr2=0x7f9a7c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9a7c0a1290 0x7f9a7c0a3750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9a90003e90 tx=0x7f9a90009100 comp rx=0 tx=0).stop 2024-03-30T02:39:08.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 -- 172.21.15.67:0/1131011728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a980731e0 msgr2=0x7f9a98199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a980731e0 0x7f9a98199410 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e9a0 tx=0x7f9a8800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 -- 172.21.15.67:0/1131011728 shutdown_connections 2024-03-30T02:39:08.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9a7c0a1290 0x7f9a7c0a3750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9a9810d540 0x7f9a9819dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a980731e0 0x7f9a98199410 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 --2- 172.21.15.67:0/1131011728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a98073b00 0x7f9a98199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 -- 172.21.15.67:0/1131011728 >> 172.21.15.67:0/1131011728 conn(0x7f9a980fc7f0 msgr2=0x7f9a980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.466+0000 7f9a9e800700 1 -- 172.21.15.67:0/1131011728 shutdown_connections 2024-03-30T02:39:08.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.467+0000 7f9a9e800700 1 -- 172.21.15.67:0/1131011728 wait complete. 2024-03-30T02:39:08.483 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:08 smithi067 ceph-mon[37808]: pgmap v65: 129 pgs: 129 active+clean; 165 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-03-30T02:39:08.486 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:39:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:08 smithi170 ceph-mon[78798]: pgmap v65: 129 pgs: 129 active+clean; 165 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-03-30T02:39:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.742+0000 7f9490c00700 1 -- 172.21.15.67:0/2533723252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f948c1082f0 msgr2=0x7f948c10fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.742+0000 7f9490c00700 1 --2- 172.21.15.67:0/2533723252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f948c1082f0 0x7f948c10fe70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9484009990 tx=0x7f9484009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.742+0000 7f9490c00700 1 -- 172.21.15.67:0/2533723252 shutdown_connections 2024-03-30T02:39:08.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.742+0000 7f9490c00700 1 --2- 172.21.15.67:0/2533723252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f948c1082f0 0x7f948c10fe70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.742+0000 7f9490c00700 1 --2- 172.21.15.67:0/2533723252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c1031d0 0x7f948c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.742+0000 7f9490c00700 1 --2- 172.21.15.67:0/2533723252 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f948c103b80 0x7f948c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.742+0000 7f9490c00700 1 -- 172.21.15.67:0/2533723252 >> 172.21.15.67:0/2533723252 conn(0x7f948c0fea10 msgr2=0x7f948c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.743+0000 7f9490c00700 1 -- 172.21.15.67:0/2533723252 shutdown_connections 2024-03-30T02:39:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.743+0000 7f9490c00700 1 -- 172.21.15.67:0/2533723252 wait complete. 2024-03-30T02:39:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.744+0000 7f9490c00700 1 Processor -- start 2024-03-30T02:39:08.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.744+0000 7f9490c00700 1 -- start start 2024-03-30T02:39:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f9490c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f948c1031d0 0x7f948c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f9490c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f948c1082f0 0x7f948c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f9490c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c19dc80 0x7f948c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f9490c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c19e700 con 0x7f948c1031d0 2024-03-30T02:39:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f9490c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c19e870 con 0x7f948c1082f0 2024-03-30T02:39:08.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f9490c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c19ea30 con 0x7f948c19dc80 2024-03-30T02:39:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f948a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c19dc80 0x7f948c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f948a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c19dc80 0x7f948c19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:49124/0 (socket says 172.21.15.67:49124) 2024-03-30T02:39:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f948a800700 1 -- 172.21.15.67:0/3584787415 learned_addr learned my addr 172.21.15.67:0/3584787415 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f9489e00700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f948c1031d0 0x7f948c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f9481400700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f948c1082f0 0x7f948c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f948a800700 1 -- 172.21.15.67:0/3584787415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f948c1082f0 msgr2=0x7f948c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f948a800700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f948c1082f0 0x7f948c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f948a800700 1 -- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f948c1031d0 msgr2=0x7f948c1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f948a800700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f948c1031d0 0x7f948c1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f948a800700 1 -- 172.21.15.67:0/3584787415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840095f0 con 0x7f948c19dc80 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.745+0000 7f9481400700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f948c1082f0 0x7f948c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.746+0000 7f948a800700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c19dc80 0x7f948c19e100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f94840098e0 tx=0x7f9484003b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.746+0000 7f9483400700 1 -- 172.21.15.67:0/3584787415 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948401d070 con 0x7f948c19dc80 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.746+0000 7f9490c00700 1 -- 172.21.15.67:0/3584787415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c1a5f30 con 0x7f948c19dc80 2024-03-30T02:39:08.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.746+0000 7f9483400700 1 -- 172.21.15.67:0/3584787415 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9484005580 con 0x7f948c19dc80 2024-03-30T02:39:08.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.746+0000 7f9483400700 1 -- 172.21.15.67:0/3584787415 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400fe20 con 0x7f948c19dc80 2024-03-30T02:39:08.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.746+0000 7f9490c00700 1 -- 172.21.15.67:0/3584787415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c1a6360 con 0x7f948c19dc80 2024-03-30T02:39:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:08 smithi082 ceph-mon[79176]: pgmap v65: 129 pgs: 129 active+clean; 165 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-03-30T02:39:08.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.749+0000 7f9483400700 1 -- 172.21.15.67:0/3584787415 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9484005050 con 0x7f948c19dc80 2024-03-30T02:39:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.750+0000 7f9483400700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f94700a1290 0x7f94700a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.751+0000 7f9483400700 1 -- 172.21.15.67:0/3584787415 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f94840ce140 con 0x7f948c19dc80 2024-03-30T02:39:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.751+0000 7f94a462bb80 1 Processor -- start 2024-03-30T02:39:08.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.751+0000 7f9489e00700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f94700a1290 0x7f94700a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.751+0000 7f94a462bb80 1 -- start start 2024-03-30T02:39:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.751+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.751+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f799f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f94a462bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132ebc990 con 0x55d132f63ef0 2024-03-30T02:39:08.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f94a462bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132eba350 con 0x55d132f635d0 2024-03-30T02:39:08.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f94a462bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132eb5c40 con 0x55d132f77600 2024-03-30T02:39:08.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9489e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9489e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f639b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34634/0 (socket says 172.21.15.67:34634) 2024-03-30T02:39:08.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9489e00700 1 -- 172.21.15.67:0/2310315227 learned_addr learned my addr 172.21.15.67:0/2310315227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9489e00700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f94700a1290 0x7f94700a3750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f947c009fd0 tx=0x7f947c00d040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f948a800700 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f799f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9481400700 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9489e00700 1 -- 172.21.15.67:0/2310315227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 msgr2=0x55d132f799f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9489e00700 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f799f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9489e00700 1 -- 172.21.15.67:0/2310315227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 msgr2=0x55d132f770c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9489e00700 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f770c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9489e00700 1 -- 172.21.15.67:0/2310315227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d132d373b0 con 0x55d132f635d0 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f948a800700 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f799f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.752+0000 7f9481400700 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:08.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.753+0000 7f9489e00700 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f639b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f947c016760 tx=0x7f947c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcdd901fb8cf7c4b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.753+0000 7f947b000700 1 -- 172.21.15.67:0/2310315227 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c010040 con 0x55d132f635d0 2024-03-30T02:39:08.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.753+0000 7f947b000700 1 -- 172.21.15.67:0/2310315227 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f947c00ae90 con 0x55d132f635d0 2024-03-30T02:39:08.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.753+0000 7f947b000700 1 -- 172.21.15.67:0/2310315227 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0158f0 con 0x55d132f635d0 2024-03-30T02:39:08.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 -- 172.21.15.67:0/2310315227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 msgr2=0x55d132f639b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f639b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f947c016760 tx=0x7f947c011000 comp rx=0 tx=0).stop 2024-03-30T02:39:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 -- 172.21.15.67:0/2310315227 shutdown_connections 2024-03-30T02:39:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f799f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f639b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2310315227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 -- 172.21.15.67:0/2310315227 >> 172.21.15.67:0/2310315227 conn(0x55d132ec36a0 msgr2=0x55d132f5fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 -- 172.21.15.67:0/2310315227 shutdown_connections 2024-03-30T02:39:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 -- 172.21.15.67:0/2310315227 wait complete. 2024-03-30T02:39:08.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 -- start start 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.754+0000 7f94a462bb80 1 Processor -- start 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f77040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f74310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f94a462bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132e51d80 con 0x55d132f63ef0 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f94a462bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132e51a20 con 0x55d132f635d0 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f94a462bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132d396a0 con 0x55d132f77600 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f94a462bb80 10 client.0 _ll_register_callbacks cb 0x55d132e5b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f948a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f9481400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f74310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f9489e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f77040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f948a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f74a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:49136/0 (socket says 172.21.15.67:49136) 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.755+0000 7f948a800700 1 -- 172.21.15.67:0/2251744127 learned_addr learned my addr 172.21.15.67:0/2251744127 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f948a800700 1 -- 172.21.15.67:0/2251744127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 msgr2=0x55d132f77040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f948a800700 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f77040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f948a800700 1 -- 172.21.15.67:0/2251744127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 msgr2=0x55d132f74310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f948a800700 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f74310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f948a800700 1 -- 172.21.15.67:0/2251744127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c016410 con 0x55d132f77600 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f9481400700 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f74310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f948a800700 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f74a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f948409aec0 tx=0x7f948409ab40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e407d62fece1f444 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f946f400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f946f400700 1 -- 172.21.15.67:0/2251744127 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94840b2070 con 0x55d132f77600 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f946f400700 1 -- 172.21.15.67:0/2251744127 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9484020e50 con 0x55d132f77600 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f94a462bb80 1 -- 172.21.15.67:0/2251744127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d132e76380 con 0x55d132f77600 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f946f400700 1 -- 172.21.15.67:0/2251744127 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f948409b630 con 0x55d132f77600 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f94a462bb80 10 client.26266 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.756+0000 7f94a462bb80 1 -- 172.21.15.67:0/2251744127 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d132ce3b60 con 0x55d132f77600 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.757+0000 7f946f400700 1 -- 172.21.15.67:0/2251744127 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9484064080 con 0x55d132f77600 2024-03-30T02:39:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.757+0000 7f946f400700 1 -- 172.21.15.67:0/2251744127 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f948409b190 con 0x55d132f77600 2024-03-30T02:39:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.757+0000 7f94a462bb80 10 client.26266 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.757+0000 7f94a462bb80 10 client.26266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.757+0000 7f94a462bb80 20 client.26266 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.757+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d132f86e80 0x55d132f892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.757+0000 7f94a462bb80 4 client.26266 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.757+0000 7f94a462bb80 1 -- 172.21.15.67:0/2251744127 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d132eae700 con 0x55d132f86e80 2024-03-30T02:39:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.757+0000 7f9489e00700 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d132f86e80 0x55d132f892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.758+0000 7f946f400700 10 client.26266 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.758+0000 7f9489e00700 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d132f86e80 0x55d132f892a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7126a4f17bc94ef server_cookie=b85ef85d648e6dd6 in_seq=0 out_seq=0 2024-03-30T02:39:08.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.760+0000 7f946f400700 1 -- 172.21.15.67:0/2251744127 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d132eae700 con 0x55d132f86e80 2024-03-30T02:39:08.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.760+0000 7f946f400700 10 client.26266 handle_command_reply: tid=0 2024-03-30T02:39:08.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.760+0000 7f94a462bb80 1 client.26266 shutdown 2024-03-30T02:39:08.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 -- 172.21.15.67:0/2251744127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 msgr2=0x55d132f74a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f74a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f948409aec0 tx=0x7f948409ab40 comp rx=0 tx=0).stop 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 -- 172.21.15.67:0/2251744127 shutdown_connections 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f77600 0x55d132f74a50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f77040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d132f86e80 0x55d132f892a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2251744127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f63ef0 0x55d132f74310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 -- 172.21.15.67:0/2251744127 >> 172.21.15.67:0/2251744127 conn(0x55d132ec36a0 msgr2=0x55d132f656c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f946f400700 0 client.26266 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 -- 172.21.15.67:0/2251744127 shutdown_connections 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 1 -- 172.21.15.67:0/2251744127 wait complete. 2024-03-30T02:39:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.761+0000 7f94a462bb80 20 client.26266 trim_cache size 0 max 16384 2024-03-30T02:39:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f94a462bb80 1 Processor -- start 2024-03-30T02:39:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f94a462bb80 1 -- start start 2024-03-30T02:39:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f635d0 0x55d132f7e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f63ef0 0x55d132f6f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 0x55d132f6fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f94a462bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132f61300 con 0x55d132f77600 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f94a462bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132f75050 con 0x55d132f63ef0 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f94a462bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132f74690 con 0x55d132f635d0 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f9489e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f635d0 0x55d132f7e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f9481400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f63ef0 0x55d132f6f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.779+0000 7f948a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 0x55d132f6fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f9489e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f635d0 0x55d132f7e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:49150/0 (socket says 172.21.15.67:49150) 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f9489e00700 1 -- 172.21.15.67:0/2729344940 learned_addr learned my addr 172.21.15.67:0/2729344940 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f9481400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f63ef0 0x55d132f6f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34652/0 (socket says 172.21.15.67:34652) 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f9489e00700 1 -- 172.21.15.67:0/2729344940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f63ef0 msgr2=0x55d132f6f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f9489e00700 1 --2- 172.21.15.67:0/2729344940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f63ef0 0x55d132f6f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f9489e00700 1 -- 172.21.15.67:0/2729344940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 msgr2=0x55d132f6fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f9489e00700 1 --2- 172.21.15.67:0/2729344940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 0x55d132f6fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f9489e00700 1 -- 172.21.15.67:0/2729344940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d132e7bea0 con 0x55d132f635d0 2024-03-30T02:39:08.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f9489e00700 1 --2- 172.21.15.67:0/2729344940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f635d0 0x55d132f7e280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f947c016760 tx=0x7f947c009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3127aa8481b9cd85 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.780+0000 7f947ba00700 1 -- 172.21.15.67:0/2729344940 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c010040 con 0x55d132f635d0 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f947ba00700 1 -- 172.21.15.67:0/2729344940 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f947c01c040 con 0x55d132f635d0 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f947ba00700 1 -- 172.21.15.67:0/2729344940 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c020830 con 0x55d132f635d0 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f94a462bb80 1 -- 172.21.15.67:0/2729344940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f635d0 msgr2=0x55d132f7e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2729344940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f635d0 0x55d132f7e280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f947c016760 tx=0x7f947c009390 comp rx=0 tx=0).stop 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f94a462bb80 1 -- 172.21.15.67:0/2729344940 shutdown_connections 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2729344940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 0x55d132f6fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2729344940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f635d0 0x55d132f7e280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f94a462bb80 1 --2- 172.21.15.67:0/2729344940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f63ef0 0x55d132f6f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f94a462bb80 1 -- 172.21.15.67:0/2729344940 >> 172.21.15.67:0/2729344940 conn(0x55d132ec36a0 msgr2=0x55d132f585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f94a462bb80 1 -- 172.21.15.67:0/2729344940 shutdown_connections 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.781+0000 7f94a462bb80 1 -- 172.21.15.67:0/2729344940 wait complete. 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.782+0000 7f94a462bb80 1 -- start start 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.782+0000 7f94a462bb80 1 Processor -- start 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.782+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f7c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.782+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f63ef0 0x55d132f7ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.782+0000 7f94a462bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 0x55d132f7d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.782+0000 7f94a462bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132fdad00 con 0x55d132f77600 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.782+0000 7f94a462bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132fdae40 con 0x55d132f635d0 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.782+0000 7f94a462bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d132fdb000 con 0x55d132f63ef0 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.782+0000 7f94a462bb80 10 client.0 _ll_register_callbacks cb 0x55d132e5b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9489e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f7c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9489e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f7c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:34658/0 (socket says 172.21.15.67:34658) 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9489e00700 1 -- 172.21.15.67:0/3131944038 learned_addr learned my addr 172.21.15.67:0/3131944038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f948a800700 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 0x55d132f7d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9481400700 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f63ef0 0x55d132f7ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9489e00700 1 -- 172.21.15.67:0/3131944038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f63ef0 msgr2=0x55d132f7ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9489e00700 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f63ef0 0x55d132f7ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9489e00700 1 -- 172.21.15.67:0/3131944038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 msgr2=0x55d132f7d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9489e00700 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 0x55d132f7d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9489e00700 1 -- 172.21.15.67:0/3131944038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d132d373b0 con 0x55d132f635d0 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f948a800700 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 0x55d132f7d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f9489e00700 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f7c970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f947c018010 tx=0x7f947c02f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121736791b95fe2f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f946f400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f946f400700 1 -- 172.21.15.67:0/3131944038 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c010030 con 0x55d132f635d0 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f94a462bb80 1 -- 172.21.15.67:0/3131944038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f947c00ae90 con 0x55d132f635d0 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.783+0000 7f946f400700 1 -- 172.21.15.67:0/3131944038 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f947c0205d0 con 0x55d132f635d0 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f946f400700 1 -- 172.21.15.67:0/3131944038 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f947c020790 con 0x55d132f635d0 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f94a462bb80 10 client.26300 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f94a462bb80 1 -- 172.21.15.67:0/3131944038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d132d3bcd0 con 0x55d132f635d0 2024-03-30T02:39:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f946f400700 1 -- 172.21.15.67:0/3131944038 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f947c00ae90 con 0x55d132f635d0 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f946f400700 1 -- 172.21.15.67:0/3131944038 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f947c01f070 con 0x55d132f635d0 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f94a462bb80 10 client.26300 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f94a462bb80 10 client.26300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f94a462bb80 20 client.26300 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f94a462bb80 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d132fdc570 0x55d132fde990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f94a462bb80 4 client.26300 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f94a462bb80 1 -- 172.21.15.67:0/3131944038 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d132d04cf0 con 0x55d132fdc570 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.784+0000 7f9481400700 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d132fdc570 0x55d132fde990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.785+0000 7f946f400700 10 client.26300 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.785+0000 7f9481400700 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d132fdc570 0x55d132fde990 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7afbb4085a819f server_cookie=c06457a8f5dfe60a in_seq=0 out_seq=0 2024-03-30T02:39:08.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.787+0000 7f946f400700 1 -- 172.21.15.67:0/3131944038 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55d132d04cf0 con 0x55d132fdc570 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.787+0000 7f946f400700 10 client.26300 handle_command_reply: tid=0 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.787+0000 7f94a462bb80 1 client.26300 shutdown 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 -- 172.21.15.67:0/3131944038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 msgr2=0x55d132f7c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f7c970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f947c018010 tx=0x7f947c02f660 comp rx=0 tx=0).stop 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 -- 172.21.15.67:0/3131944038 shutdown_connections 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d132fdc570 0x55d132fde990 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d132f77600 0x55d132f7d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d132f635d0 0x55d132f7c970 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f946f400700 0 client.26300 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 --2- 172.21.15.67:0/3131944038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d132f63ef0 0x55d132f7ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 -- 172.21.15.67:0/3131944038 >> 172.21.15.67:0/3131944038 conn(0x55d132ec36a0 msgr2=0x55d132f6f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 -- 172.21.15.67:0/3131944038 shutdown_connections 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 1 -- 172.21.15.67:0/3131944038 wait complete. 2024-03-30T02:39:08.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f94a462bb80 20 client.26300 trim_cache size 0 max 16384 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (2031 inodes in the stack)", 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "b41bed7a-70c8-4f10-8792-8d2296a8b63c": { 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "b41bed7a-70c8-4f10-8792-8d2296a8b63c", 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:39:08.792 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:39:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f9490c00700 1 -- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f94700a1290 msgr2=0x7f94700a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f9490c00700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f94700a1290 0x7f94700a3750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f947c009fd0 tx=0x7f947c00d040 comp rx=0 tx=0).stop 2024-03-30T02:39:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f9490c00700 1 -- 172.21.15.67:0/3584787415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c19dc80 msgr2=0x7f948c19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.788+0000 7f9490c00700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c19dc80 0x7f948c19e100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f94840098e0 tx=0x7f9484003b60 comp rx=0 tx=0).stop 2024-03-30T02:39:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.789+0000 7f9490c00700 1 -- 172.21.15.67:0/3584787415 shutdown_connections 2024-03-30T02:39:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.789+0000 7f9490c00700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f94700a1290 0x7f94700a3750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.789+0000 7f9490c00700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f948c1031d0 0x7f948c1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.789+0000 7f9490c00700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f948c19dc80 0x7f948c19e100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.789+0000 7f9490c00700 1 --2- 172.21.15.67:0/3584787415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f948c1082f0 0x7f948c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.789+0000 7f9490c00700 1 -- 172.21.15.67:0/3584787415 >> 172.21.15.67:0/3584787415 conn(0x7f948c0fea10 msgr2=0x7f948c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.789+0000 7f9490c00700 1 -- 172.21.15.67:0/3584787415 shutdown_connections 2024-03-30T02:39:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:08.789+0000 7f9490c00700 1 -- 172.21.15.67:0/3584787415 wait complete. 2024-03-30T02:39:08.807 INFO:tasks.cephfs.filesystem:scrub status for tag:b41bed7a-70c8-4f10-8792-8d2296a8b63c - {'path': '/', 'tag': 'b41bed7a-70c8-4f10-8792-8d2296a8b63c', 'options': 'recursive,force'} 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:09.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:09.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:09.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:09.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:09.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:09.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:09.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:09.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: pgmap v66: 129 pgs: 129 active+clean; 171 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 81 op/s 2024-03-30T02:39:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:39:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:10.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: pgmap v66: 129 pgs: 129 active+clean; 171 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 81 op/s 2024-03-30T02:39:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:39:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:10.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: pgmap v66: 129 pgs: 129 active+clean; 171 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 81 op/s 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:11.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:39:11.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:39:11.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:39:11.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:11.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:39:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:39:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: pgmap v67: 129 pgs: 129 active+clean; 171 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 57 op/s 2024-03-30T02:39:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:12.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: pgmap v67: 129 pgs: 129 active+clean; 171 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 57 op/s 2024-03-30T02:39:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: pgmap v67: 129 pgs: 129 active+clean; 171 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 57 op/s 2024-03-30T02:39:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:13.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:13.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:14 smithi067 ceph-mon[37808]: pgmap v68: 129 pgs: 129 active+clean; 171 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 57 op/s 2024-03-30T02:39:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:14.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:14 smithi170 ceph-mon[78798]: pgmap v68: 129 pgs: 129 active+clean; 171 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 57 op/s 2024-03-30T02:39:14.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:14.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:14.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:14.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:14 smithi082 ceph-mon[79176]: pgmap v68: 129 pgs: 129 active+clean; 171 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 57 op/s 2024-03-30T02:39:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:16.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:16 smithi067 ceph-mon[37808]: pgmap v69: 129 pgs: 129 active+clean; 182 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 102 op/s 2024-03-30T02:39:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:16 smithi170 ceph-mon[78798]: pgmap v69: 129 pgs: 129 active+clean; 182 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 102 op/s 2024-03-30T02:39:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:16 smithi082 ceph-mon[79176]: pgmap v69: 129 pgs: 129 active+clean; 182 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 102 op/s 2024-03-30T02:39:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:18 smithi067 ceph-mon[37808]: pgmap v70: 129 pgs: 129 active+clean; 182 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-03-30T02:39:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:18 smithi170 ceph-mon[78798]: pgmap v70: 129 pgs: 129 active+clean; 182 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-03-30T02:39:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:18 smithi082 ceph-mon[79176]: pgmap v70: 129 pgs: 129 active+clean; 182 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-03-30T02:39:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:20.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:20 smithi067 ceph-mon[37808]: pgmap v71: 129 pgs: 129 active+clean; 208 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-03-30T02:39:20.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:20 smithi170 ceph-mon[78798]: pgmap v71: 129 pgs: 129 active+clean; 208 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-03-30T02:39:20.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:20 smithi082 ceph-mon[79176]: pgmap v71: 129 pgs: 129 active+clean; 208 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-03-30T02:39:22.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:22 smithi067 ceph-mon[37808]: pgmap v72: 129 pgs: 129 active+clean; 208 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-03-30T02:39:22.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:22 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:39:22.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:22 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:39:22.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:22 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:39:22.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:22 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:39:22.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:22 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:39:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:22 smithi170 ceph-mon[78798]: pgmap v72: 129 pgs: 129 active+clean; 208 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-03-30T02:39:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:22 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:39:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:22 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:39:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:22 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:39:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:22 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:39:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:22 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:39:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:22 smithi082 ceph-mon[79176]: pgmap v72: 129 pgs: 129 active+clean; 208 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-03-30T02:39:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:22 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:39:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:22 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:39:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:22 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:39:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:22 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:39:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:22 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:39:24.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:24 smithi067 ceph-mon[37808]: pgmap v73: 129 pgs: 129 active+clean; 208 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-03-30T02:39:24.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:24 smithi170 ceph-mon[78798]: pgmap v73: 129 pgs: 129 active+clean; 208 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-03-30T02:39:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:24 smithi082 ceph-mon[79176]: pgmap v73: 129 pgs: 129 active+clean; 208 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-03-30T02:39:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:26 smithi067 ceph-mon[37808]: pgmap v74: 129 pgs: 129 active+clean; 239 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 548 op/s 2024-03-30T02:39:26.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:26 smithi170 ceph-mon[78798]: pgmap v74: 129 pgs: 129 active+clean; 239 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 548 op/s 2024-03-30T02:39:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:26 smithi082 ceph-mon[79176]: pgmap v74: 129 pgs: 129 active+clean; 239 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.2 MiB/s wr, 548 op/s 2024-03-30T02:39:28.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:28 smithi067 ceph-mon[37808]: pgmap v75: 129 pgs: 129 active+clean; 239 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-03-30T02:39:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:28 smithi170 ceph-mon[78798]: pgmap v75: 129 pgs: 129 active+clean; 239 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-03-30T02:39:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:28 smithi082 ceph-mon[79176]: pgmap v75: 129 pgs: 129 active+clean; 239 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 503 op/s 2024-03-30T02:39:30.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:30 smithi067 ceph-mon[37808]: pgmap v76: 129 pgs: 129 active+clean; 269 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 8.0 MiB/s wr, 751 op/s 2024-03-30T02:39:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:30 smithi170 ceph-mon[78798]: pgmap v76: 129 pgs: 129 active+clean; 269 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 8.0 MiB/s wr, 751 op/s 2024-03-30T02:39:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:30 smithi082 ceph-mon[79176]: pgmap v76: 129 pgs: 129 active+clean; 269 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 8.0 MiB/s wr, 751 op/s 2024-03-30T02:39:32.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:32 smithi067 ceph-mon[37808]: pgmap v77: 129 pgs: 129 active+clean; 269 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-03-30T02:39:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:32 smithi170 ceph-mon[78798]: pgmap v77: 129 pgs: 129 active+clean; 269 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-03-30T02:39:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:32 smithi082 ceph-mon[79176]: pgmap v77: 129 pgs: 129 active+clean; 269 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-03-30T02:39:34.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:34 smithi067 ceph-mon[37808]: pgmap v78: 129 pgs: 129 active+clean; 269 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-03-30T02:39:34.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:34 smithi170 ceph-mon[78798]: pgmap v78: 129 pgs: 129 active+clean; 269 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-03-30T02:39:34.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:34 smithi082 ceph-mon[79176]: pgmap v78: 129 pgs: 129 active+clean; 269 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.6 MiB/s wr, 572 op/s 2024-03-30T02:39:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:36 smithi067 ceph-mon[37808]: pgmap v79: 129 pgs: 129 active+clean; 306 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.0 MiB/s wr, 826 op/s 2024-03-30T02:39:36.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:36 smithi170 ceph-mon[78798]: pgmap v79: 129 pgs: 129 active+clean; 306 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.0 MiB/s wr, 826 op/s 2024-03-30T02:39:36.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:36 smithi082 ceph-mon[79176]: pgmap v79: 129 pgs: 129 active+clean; 306 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 9.0 MiB/s wr, 826 op/s 2024-03-30T02:39:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:38 smithi067 ceph-mon[37808]: pgmap v80: 129 pgs: 129 active+clean; 306 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.2 MiB/s wr, 502 op/s 2024-03-30T02:39:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:38 smithi170 ceph-mon[78798]: pgmap v80: 129 pgs: 129 active+clean; 306 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.2 MiB/s wr, 502 op/s 2024-03-30T02:39:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:38 smithi082 ceph-mon[79176]: pgmap v80: 129 pgs: 129 active+clean; 306 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.2 MiB/s wr, 502 op/s 2024-03-30T02:39:38.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:39:39.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.016+0000 7f2483200700 1 -- 172.21.15.67:0/815536164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 msgr2=0x7f247c073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.016+0000 7f2483200700 1 --2- 172.21.15.67:0/815536164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 0x7f247c073560 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f246c009990 tx=0x7f246c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.016+0000 7f2483200700 1 -- 172.21.15.67:0/815536164 shutdown_connections 2024-03-30T02:39:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.016+0000 7f2483200700 1 --2- 172.21.15.67:0/815536164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f247c10d580 0x7f247c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.016+0000 7f2483200700 1 --2- 172.21.15.67:0/815536164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 0x7f247c073560 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.016+0000 7f2483200700 1 --2- 172.21.15.67:0/815536164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c073aa0 0x7f247c10d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.016+0000 7f2483200700 1 -- 172.21.15.67:0/815536164 >> 172.21.15.67:0/815536164 conn(0x7f247c0fc7f0 msgr2=0x7f247c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.017+0000 7f2483200700 1 -- 172.21.15.67:0/815536164 shutdown_connections 2024-03-30T02:39:39.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.017+0000 7f2483200700 1 -- 172.21.15.67:0/815536164 wait complete. 2024-03-30T02:39:39.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f2483200700 1 Processor -- start 2024-03-30T02:39:39.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f2483200700 1 -- start start 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f2483200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 0x7f247c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f2483200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f247c073aa0 0x7f247c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f2483200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c10d580 0x7f247c19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f2483200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247c19e340 con 0x7f247c073aa0 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f2483200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247c19e4b0 con 0x7f247c073180 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f2483200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247c19e670 con 0x7f247c10d580 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f247be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 0x7f247c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f247be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 0x7f247c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43300/0 (socket says 172.21.15.67:43300) 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f247be00700 1 -- 172.21.15.67:0/1207200460 learned_addr learned my addr 172.21.15.67:0/1207200460 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f247b400700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f247c073aa0 0x7f247c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.018+0000 7f2481200700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c10d580 0x7f247c19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f247be00700 1 -- 172.21.15.67:0/1207200460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c10d580 msgr2=0x7f247c19dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f247be00700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c10d580 0x7f247c19dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f247be00700 1 -- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f247c073aa0 msgr2=0x7f247c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f247be00700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f247c073aa0 0x7f247c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f247be00700 1 -- 172.21.15.67:0/1207200460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c0095f0 con 0x7f247c073180 2024-03-30T02:39:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f247be00700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 0x7f247c199420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f246c005910 tx=0x7f246c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f2478c00700 1 -- 172.21.15.67:0/1207200460 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c01d070 con 0x7f247c073180 2024-03-30T02:39:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f2478c00700 1 -- 172.21.15.67:0/1207200460 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f246c022470 con 0x7f247c073180 2024-03-30T02:39:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f2478c00700 1 -- 172.21.15.67:0/1207200460 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c017650 con 0x7f247c073180 2024-03-30T02:39:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f2483200700 1 -- 172.21.15.67:0/1207200460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247c19e8f0 con 0x7f247c073180 2024-03-30T02:39:39.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.019+0000 7f2483200700 1 -- 172.21.15.67:0/1207200460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247c19eb70 con 0x7f247c073180 2024-03-30T02:39:39.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.024+0000 7f2478c00700 1 -- 172.21.15.67:0/1207200460 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f246c0225e0 con 0x7f247c073180 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2496ca5b80 1 Processor -- start 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2496ca5b80 1 -- start start 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2478c00700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f24600a1310 0x7f24600a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2478c00700 1 -- 172.21.15.67:0/1207200460 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f246c0ce050 con 0x7f247c073180 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c3c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2481200700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f24600a1310 0x7f24600a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 0x5590a2c4a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2b95990 con 0x5590a2c3cef0 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2b93350 con 0x5590a2c3c5d0 2024-03-30T02:39:39.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.025+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2b8ec40 con 0x5590a2c47df0 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c3c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c3c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43304/0 (socket says 172.21.15.67:43304) 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247be00700 1 -- 172.21.15.67:0/1933113872 learned_addr learned my addr 172.21.15.67:0/1933113872 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247b400700 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f2481200700 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 0x5590a2c4a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247be00700 1 -- 172.21.15.67:0/1933113872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 msgr2=0x5590a2c4a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247be00700 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 0x5590a2c4a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247be00700 1 -- 172.21.15.67:0/1933113872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 msgr2=0x5590a2c478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247be00700 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247be00700 1 -- 172.21.15.67:0/1933113872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a2a103b0 con 0x5590a2c3c5d0 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f2481200700 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 0x5590a2c4a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247b400700 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f2481200700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f24600a1310 0x7f24600a37d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f247c0679c0 tx=0x7f2474009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f247be00700 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c3c9b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f246c0974c0 tx=0x7f246c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859996b5368683e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.026+0000 7f2471600700 1 -- 172.21.15.67:0/1933113872 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c0a10c0 con 0x5590a2c3c5d0 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2471600700 1 -- 172.21.15.67:0/1933113872 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f246c09b6c0 con 0x5590a2c3c5d0 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2471600700 1 -- 172.21.15.67:0/1933113872 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c09b220 con 0x5590a2c3c5d0 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1933113872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 msgr2=0x5590a2c3c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c3c9b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f246c0974c0 tx=0x7f246c009310 comp rx=0 tx=0).stop 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1933113872 shutdown_connections 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 0x5590a2c4a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c3c9b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1933113872 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1933113872 >> 172.21.15.67:0/1933113872 conn(0x5590a2b9c6a0 msgr2=0x5590a2c38a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1933113872 shutdown_connections 2024-03-30T02:39:39.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.027+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1933113872 wait complete. 2024-03-30T02:39:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.028+0000 7f2496ca5b80 1 -- start start 2024-03-30T02:39:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.028+0000 7f2496ca5b80 1 Processor -- start 2024-03-30T02:39:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.028+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3c5d0 0x5590a2c477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.028+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c44b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.028+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 0x5590a2c451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.028+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2b2ad80 con 0x5590a2c3cef0 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.028+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2b2aa20 con 0x5590a2c47df0 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2a126a0 con 0x5590a2c3c5d0 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f2496ca5b80 10 client.0 _ll_register_callbacks cb 0x5590a2b34470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f247be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3c5d0 0x5590a2c477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f247b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c44b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f2481200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 0x5590a2c451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f247be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3c5d0 0x5590a2c477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:34432/0 (socket says 172.21.15.67:34432) 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f247be00700 1 -- 172.21.15.67:0/3871936574 learned_addr learned my addr 172.21.15.67:0/3871936574 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f2481200700 1 -- 172.21.15.67:0/3871936574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3c5d0 msgr2=0x5590a2c477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f2481200700 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3c5d0 0x5590a2c477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f2481200700 1 -- 172.21.15.67:0/3871936574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 msgr2=0x5590a2c44b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f2481200700 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c44b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f2481200700 1 -- 172.21.15.67:0/3871936574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247c19e8f0 con 0x5590a2c47df0 2024-03-30T02:39:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f247be00700 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3c5d0 0x5590a2c477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.029+0000 7f247b400700 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c44b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2481200700 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 0x5590a2c451d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2474019e30 tx=0x7f2474004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e416962cfd642c1b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2466000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2466000700 1 -- 172.21.15.67:0/3871936574 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474027070 con 0x5590a2c47df0 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2496ca5b80 1 -- 172.21.15.67:0/3871936574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590a2b4f380 con 0x5590a2c47df0 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2466000700 1 -- 172.21.15.67:0/3871936574 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f247400fda0 con 0x5590a2c47df0 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2466000700 1 -- 172.21.15.67:0/3871936574 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f247400e040 con 0x5590a2c47df0 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2496ca5b80 10 client.26312 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2496ca5b80 1 -- 172.21.15.67:0/3871936574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5590a29bcb60 con 0x5590a2c47df0 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2466000700 1 -- 172.21.15.67:0/3871936574 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590a2b4f380 con 0x5590a2c47df0 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.030+0000 7f2466000700 1 -- 172.21.15.67:0/3871936574 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f247402a020 con 0x5590a2c47df0 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.031+0000 7f2496ca5b80 10 client.26312 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.031+0000 7f2496ca5b80 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.031+0000 7f2496ca5b80 20 client.26312 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:39.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.031+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5590a2c57840 0x5590a2c59c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.031+0000 7f2496ca5b80 4 client.26312 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.031+0000 7f2496ca5b80 1 -- 172.21.15.67:0/3871936574 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590a2b87700 con 0x5590a2c57840 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.031+0000 7f247be00700 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5590a2c57840 0x5590a2c59c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.031+0000 7f247be00700 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5590a2c57840 0x5590a2c59c60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dea9da8747f8f04 server_cookie=4e5cb91499845467 in_seq=0 out_seq=0 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.031+0000 7f2466000700 10 client.26312 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2466000700 1 -- 172.21.15.67:0/3871936574 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5590a2b87700 con 0x5590a2c57840 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2466000700 10 client.26312 handle_command_reply: tid=0 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 client.26312 shutdown 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 -- 172.21.15.67:0/3871936574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 msgr2=0x5590a2c451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 0x5590a2c451d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2474019e30 tx=0x7f2474004b40 comp rx=0 tx=0).stop 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 -- 172.21.15.67:0/3871936574 shutdown_connections 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 0x5590a2c451d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3c5d0 0x5590a2c477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5590a2c57840 0x5590a2c59c60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/3871936574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c44b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 -- 172.21.15.67:0/3871936574 >> 172.21.15.67:0/3871936574 conn(0x5590a2b9c6a0 msgr2=0x5590a2c3e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2466000700 0 client.26312 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.032+0000 7f2496ca5b80 1 -- 172.21.15.67:0/3871936574 shutdown_connections 2024-03-30T02:39:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.033+0000 7f2496ca5b80 1 -- 172.21.15.67:0/3871936574 wait complete. 2024-03-30T02:39:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.033+0000 7f2496ca5b80 20 client.26312 trim_cache size 0 max 16384 2024-03-30T02:39:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.050+0000 7f2496ca5b80 1 Processor -- start 2024-03-30T02:39:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.050+0000 7f2496ca5b80 1 -- start start 2024-03-30T02:39:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c43420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 0x5590a2c43960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2b4f380 con 0x5590a2c3cef0 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2c45fe0 con 0x5590a2c3c5d0 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2c3d850 con 0x5590a2c47df0 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c43420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c43420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35306/0 (socket says 172.21.15.67:35306) 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247b400700 1 -- 172.21.15.67:0/2061476697 learned_addr learned my addr 172.21.15.67:0/2061476697 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f2481200700 1 --2- 172.21.15.67:0/2061476697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 0x5590a2c43960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247b400700 1 -- 172.21.15.67:0/2061476697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 msgr2=0x5590a2c43960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247b400700 1 --2- 172.21.15.67:0/2061476697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 0x5590a2c43960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247b400700 1 -- 172.21.15.67:0/2061476697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 msgr2=0x5590a2c450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247b400700 1 --2- 172.21.15.67:0/2061476697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247b400700 1 -- 172.21.15.67:0/2061476697 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a2b54ea0 con 0x5590a2c3cef0 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.051+0000 7f247be00700 1 --2- 172.21.15.67:0/2061476697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.052+0000 7f247b400700 1 --2- 172.21.15.67:0/2061476697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c43420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2468009fa0 tx=0x7f2468009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418869683f5ff9e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.052+0000 7f2472000700 1 -- 172.21.15.67:0/2061476697 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468020070 con 0x5590a2c3cef0 2024-03-30T02:39:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.052+0000 7f2472000700 1 -- 172.21.15.67:0/2061476697 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2468018040 con 0x5590a2c3cef0 2024-03-30T02:39:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.052+0000 7f2472000700 1 -- 172.21.15.67:0/2061476697 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468017050 con 0x5590a2c3cef0 2024-03-30T02:39:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.053+0000 7f2496ca5b80 1 -- 172.21.15.67:0/2061476697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 msgr2=0x5590a2c43420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.053+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/2061476697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c43420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2468009fa0 tx=0x7f2468009260 comp rx=0 tx=0).stop 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.053+0000 7f2496ca5b80 1 -- 172.21.15.67:0/2061476697 shutdown_connections 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.053+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/2061476697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c47df0 0x5590a2c43960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.053+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/2061476697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c3c5d0 0x5590a2c450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.053+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/2061476697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3cef0 0x5590a2c43420 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.053+0000 7f2496ca5b80 1 -- 172.21.15.67:0/2061476697 >> 172.21.15.67:0/2061476697 conn(0x5590a2b9c6a0 msgr2=0x5590a2c3fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.053+0000 7f2496ca5b80 1 -- 172.21.15.67:0/2061476697 shutdown_connections 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.053+0000 7f2496ca5b80 1 -- 172.21.15.67:0/2061476697 wait complete. 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.054+0000 7f2496ca5b80 1 -- start start 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.054+0000 7f2496ca5b80 1 Processor -- start 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.054+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3c5d0 0x5590a2c4c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.054+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3cef0 0x5590a2c4c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.054+0000 7f2496ca5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 0x5590a2caa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.054+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2c4d200 con 0x5590a2c3c5d0 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.054+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2caabb0 con 0x5590a2c47df0 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.054+0000 7f2496ca5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a2caacf0 con 0x5590a2c3cef0 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.054+0000 7f2496ca5b80 10 client.0 _ll_register_callbacks cb 0x5590a2b34470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3c5d0 0x5590a2c4c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3cef0 0x5590a2c4c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3c5d0 0x5590a2c4c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35320/0 (socket says 172.21.15.67:35320) 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f2481200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 0x5590a2caa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247be00700 1 -- 172.21.15.67:0/1218529310 learned_addr learned my addr 172.21.15.67:0/1218529310 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247b400700 1 -- 172.21.15.67:0/1218529310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 msgr2=0x5590a2caa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247b400700 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 0x5590a2caa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247b400700 1 -- 172.21.15.67:0/1218529310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3c5d0 msgr2=0x5590a2c4c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247b400700 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3c5d0 0x5590a2c4c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247b400700 1 -- 172.21.15.67:0/1218529310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590a2a103b0 con 0x5590a2c3cef0 2024-03-30T02:39:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f247b400700 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3cef0 0x5590a2c4c8f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5590a2c43330 tx=0x7f24680058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b2bf8451ac21496 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f2466000700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.055+0000 7f2466000700 1 -- 172.21.15.67:0/1218529310 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468020040 con 0x5590a2c3cef0 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2466000700 1 -- 172.21.15.67:0/1218529310 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f2468005220 con 0x5590a2c3cef0 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1218529310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f246c09b6c0 con 0x5590a2c3cef0 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2466000700 1 -- 172.21.15.67:0/1218529310 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2468017020 con 0x5590a2c3cef0 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2496ca5b80 10 client.26287 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1218529310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f2468009c50 con 0x5590a2c3cef0 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2466000700 1 -- 172.21.15.67:0/1218529310 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2468034020 con 0x5590a2c3cef0 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2466000700 1 -- 172.21.15.67:0/1218529310 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2468026c10 con 0x5590a2c3cef0 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2496ca5b80 10 client.26287 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2496ca5b80 10 client.26287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.056+0000 7f2496ca5b80 20 client.26287 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.057+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5590a2cac500 0x5590a2cae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.057+0000 7f2496ca5b80 4 client.26287 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:39:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.057+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1218529310 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590a29ddcf0 con 0x5590a2cac500 2024-03-30T02:39:39.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.057+0000 7f247be00700 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5590a2cac500 0x5590a2cae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.057+0000 7f247be00700 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5590a2cac500 0x5590a2cae920 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b9ff1f5f524d93 server_cookie=f33f7f3e4e7b8f37 in_seq=0 out_seq=0 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.057+0000 7f2466000700 10 client.26287 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.057+0000 7f2466000700 1 -- 172.21.15.67:0/1218529310 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5590a29ddcf0 con 0x5590a2cac500 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.057+0000 7f2466000700 10 client.26287 handle_command_reply: tid=0 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.057+0000 7f2496ca5b80 1 client.26287 shutdown 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1218529310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3cef0 msgr2=0x5590a2c4c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3cef0 0x5590a2c4c8f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x5590a2c43330 tx=0x7f24680058e0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1218529310 shutdown_connections 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5590a2cac500 0x5590a2cae920 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5590a2c3c5d0 0x5590a2c4c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a2c47df0 0x5590a2caa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2466000700 0 client.26287 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 --2- 172.21.15.67:0/1218529310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590a2c3cef0 0x5590a2c4c8f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1218529310 >> 172.21.15.67:0/1218529310 conn(0x5590a2b9c6a0 msgr2=0x5590a2c433e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1218529310 shutdown_connections 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 1 -- 172.21.15.67:0/1218529310 wait complete. 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2496ca5b80 20 client.26287 trim_cache size 0 max 16384 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:39:39.063 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:39:39.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.058+0000 7f2483200700 1 -- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f24600a1310 msgr2=0x7f24600a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f24600a1310 0x7f24600a37d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f247c0679c0 tx=0x7f2474009920 comp rx=0 tx=0).stop 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 -- 172.21.15.67:0/1207200460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 msgr2=0x7f247c199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 0x7f247c199420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f246c005910 tx=0x7f246c004730 comp rx=0 tx=0).stop 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 -- 172.21.15.67:0/1207200460 shutdown_connections 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f24600a1310 0x7f24600a37d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f247c10d580 0x7f247c19dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f247c073180 0x7f247c199420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 --2- 172.21.15.67:0/1207200460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f247c073aa0 0x7f247c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 -- 172.21.15.67:0/1207200460 >> 172.21.15.67:0/1207200460 conn(0x7f247c0fc7f0 msgr2=0x7f247c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 -- 172.21.15.67:0/1207200460 shutdown_connections 2024-03-30T02:39:39.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.059+0000 7f2483200700 1 -- 172.21.15.67:0/1207200460 wait complete. 2024-03-30T02:39:39.079 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:39:39.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:39:39.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.289+0000 7f9924200700 1 -- 172.21.15.67:0/3693696518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c101820 msgr2=0x7f991c105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.289+0000 7f9924200700 1 --2- 172.21.15.67:0/3693696518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c101820 0x7f991c105860 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9908009990 tx=0x7f9908009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.289+0000 7f9924200700 1 -- 172.21.15.67:0/3693696518 shutdown_connections 2024-03-30T02:39:39.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.289+0000 7f9924200700 1 --2- 172.21.15.67:0/3693696518 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c105f90 0x7f991c10db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.289+0000 7f9924200700 1 --2- 172.21.15.67:0/3693696518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f991c100e70 0x7f991c101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.289+0000 7f9924200700 1 --2- 172.21.15.67:0/3693696518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c101820 0x7f991c105860 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.289+0000 7f9924200700 1 -- 172.21.15.67:0/3693696518 >> 172.21.15.67:0/3693696518 conn(0x7f991c0fc730 msgr2=0x7f991c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.289+0000 7f9924200700 1 -- 172.21.15.67:0/3693696518 shutdown_connections 2024-03-30T02:39:39.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.289+0000 7f9924200700 1 -- 172.21.15.67:0/3693696518 wait complete. 2024-03-30T02:39:39.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.290+0000 7f9924200700 1 Processor -- start 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.290+0000 7f9924200700 1 -- start start 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9924200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f991c100e70 0x7f991c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9924200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c105f90 0x7f991c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9924200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c073b30 0x7f991c1a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9924200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c1a5900 con 0x7f991c105f90 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9924200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c1a5a70 con 0x7f991c073b30 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9924200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c1a5c30 con 0x7f991c100e70 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9920e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c105f90 0x7f991c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9922200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c073b30 0x7f991c1a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9921800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f991c100e70 0x7f991c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9922200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c073b30 0x7f991c1a5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43352/0 (socket says 172.21.15.67:43352) 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9922200700 1 -- 172.21.15.67:0/134973019 learned_addr learned my addr 172.21.15.67:0/134973019 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9922200700 1 -- 172.21.15.67:0/134973019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f991c100e70 msgr2=0x7f991c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9922200700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f991c100e70 0x7f991c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9922200700 1 -- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c105f90 msgr2=0x7f991c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9922200700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c105f90 0x7f991c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9922200700 1 -- 172.21.15.67:0/134973019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9910009520 con 0x7f991c073b30 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.291+0000 7f9920e00700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c105f90 0x7f991c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.292+0000 7f9922200700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c073b30 0x7f991c1a5300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f991400ef30 tx=0x7f991400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.292+0000 7f991a000700 1 -- 172.21.15.67:0/134973019 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914018070 con 0x7f991c073b30 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.292+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99080095f0 con 0x7f991c073b30 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.292+0000 7f991a000700 1 -- 172.21.15.67:0/134973019 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f991400f040 con 0x7f991c073b30 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.292+0000 7f991a000700 1 -- 172.21.15.67:0/134973019 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991400cd00 con 0x7f991c073b30 2024-03-30T02:39:39.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.292+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c1a6250 con 0x7f991c073b30 2024-03-30T02:39:39.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.293+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c109260 con 0x7f991c073b30 2024-03-30T02:39:39.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.295+0000 7f991a000700 1 -- 172.21.15.67:0/134973019 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9914003a00 con 0x7f991c073b30 2024-03-30T02:39:39.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.296+0000 7f991a000700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f99040a1310 0x7f99040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.296+0000 7f991a000700 1 -- 172.21.15.67:0/134973019 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f99140ccac0 con 0x7f991c073b30 2024-03-30T02:39:39.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.296+0000 7f9921800700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f99040a1310 0x7f99040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.297+0000 7f9921800700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f99040a1310 0x7f99040a37d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9910009df0 tx=0x7f9910009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.300+0000 7f991a000700 1 -- 172.21.15.67:0/134973019 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f9914095bf0 con 0x7f991c073b30 2024-03-30T02:39:39.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.585+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f991c067a10 con 0x7f991c073b30 2024-03-30T02:39:39.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.586+0000 7f991a000700 1 -- 172.21.15.67:0/134973019 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10916 (secure 0 0 0) 0x7f9914099210 con 0x7f991c073b30 2024-03-30T02:39:39.587 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:39:39.587 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:38:47.255528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:39:39.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.588+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f99040a1310 msgr2=0x7f99040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.588+0000 7f9924200700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f99040a1310 0x7f99040a37d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9910009df0 tx=0x7f9910009260 comp rx=0 tx=0).stop 2024-03-30T02:39:39.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.588+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c073b30 msgr2=0x7f991c1a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.588+0000 7f9924200700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c073b30 0x7f991c1a5300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f991400ef30 tx=0x7f991400c3a0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.589+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 shutdown_connections 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.589+0000 7f9924200700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f99040a1310 0x7f99040a37d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.589+0000 7f9924200700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f991c073b30 0x7f991c1a5300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.589+0000 7f9924200700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f991c100e70 0x7f991c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.589+0000 7f9924200700 1 --2- 172.21.15.67:0/134973019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f991c105f90 0x7f991c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.589+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 >> 172.21.15.67:0/134973019 conn(0x7f991c0fc730 msgr2=0x7f991c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.589+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 shutdown_connections 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.589+0000 7f9924200700 1 -- 172.21.15.67:0/134973019 wait complete. 2024-03-30T02:39:39.590 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-03-30T02:39:39.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:39:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.827+0000 7f8b8fe00700 1 -- 172.21.15.67:0/2973827955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b88069260 msgr2=0x7f8b88069640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.827+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/2973827955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b88069260 0x7f8b88069640 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009920 tx=0x7f8b7c009c30 comp rx=0 tx=0).stop 2024-03-30T02:39:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.827+0000 7f8b8fe00700 1 -- 172.21.15.67:0/2973827955 shutdown_connections 2024-03-30T02:39:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.827+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/2973827955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b88069260 0x7f8b88069640 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.827+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/2973827955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b881060b0 0x7f8b8810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.827+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/2973827955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88069c10 0x7f8b88105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.827+0000 7f8b8fe00700 1 -- 172.21.15.67:0/2973827955 >> 172.21.15.67:0/2973827955 conn(0x7f8b88076d80 msgr2=0x7f8b88077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.827+0000 7f8b8fe00700 1 -- 172.21.15.67:0/2973827955 shutdown_connections 2024-03-30T02:39:39.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.827+0000 7f8b8fe00700 1 -- 172.21.15.67:0/2973827955 wait complete. 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.828+0000 7f8b8fe00700 1 Processor -- start 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.828+0000 7f8b8fe00700 1 -- start start 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b88069260 0x7f8b880ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b881060b0 0x7f8b881001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b881006f0 0x7f8b881a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8fe00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b881a5aa0 con 0x7f8b88069260 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8fe00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b881a5be0 con 0x7f8b881006f0 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8fe00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b881a5da0 con 0x7f8b881060b0 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b88069260 0x7f8b880ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b881006f0 0x7f8b881a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b881006f0 0x7f8b881a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43380/0 (socket says 172.21.15.67:43380) 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8de00700 1 -- 172.21.15.67:0/3628713221 learned_addr learned my addr 172.21.15.67:0/3628713221 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.829+0000 7f8b8ca00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b881060b0 0x7f8b881001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b8de00700 1 -- 172.21.15.67:0/3628713221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b881060b0 msgr2=0x7f8b881001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b8de00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b881060b0 0x7f8b881001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b8de00700 1 -- 172.21.15.67:0/3628713221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b88069260 msgr2=0x7f8b880ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b8de00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b88069260 0x7f8b880ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b8de00700 1 -- 172.21.15.67:0/3628713221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740095f0 con 0x7f8b881006f0 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b8de00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b881006f0 0x7f8b881a54d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8b80007ee0 tx=0x7f8b8000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b86000700 1 -- 172.21.15.67:0/3628713221 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80017070 con 0x7f8b881006f0 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b86000700 1 -- 172.21.15.67:0/3628713221 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8000f040 con 0x7f8b881006f0 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b8fe00700 1 -- 172.21.15.67:0/3628713221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c009520 con 0x7f8b881006f0 2024-03-30T02:39:39.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b86000700 1 -- 172.21.15.67:0/3628713221 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80005580 con 0x7f8b881006f0 2024-03-30T02:39:39.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.830+0000 7f8b8fe00700 1 -- 172.21.15.67:0/3628713221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b881a6450 con 0x7f8b881006f0 2024-03-30T02:39:39.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.833+0000 7f8b86000700 1 -- 172.21.15.67:0/3628713221 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8b8000db20 con 0x7f8b881006f0 2024-03-30T02:39:39.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.835+0000 7f8b86000700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8b700a1310 0x7f8b700a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.835+0000 7f8b86000700 1 -- 172.21.15.67:0/3628713221 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f8b800cc6b0 con 0x7f8b881006f0 2024-03-30T02:39:39.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.835+0000 7f8b8d400700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8b700a1310 0x7f8b700a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.835+0000 7f8ba382bb80 1 Processor -- start 2024-03-30T02:39:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8ba382bb80 1 -- start start 2024-03-30T02:39:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8b8d400700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8b700a1310 0x7f8b700a37d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c010 tx=0x7f8b7c019040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf415d0 0x55eaedf419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 0x55eaedf550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaede9a990 con 0x55eaedf415d0 2024-03-30T02:39:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaede98350 con 0x55eaedf55600 2024-03-30T02:39:39.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaede93c40 con 0x55eaedf41ef0 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8b8de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.836+0000 7f8b8de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43390/0 (socket says 172.21.15.67:43390) 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8de00700 1 -- 172.21.15.67:0/332519610 learned_addr learned my addr 172.21.15.67:0/332519610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8d400700 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf415d0 0x55eaedf419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8ca00700 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 0x55eaedf550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8de00700 1 -- 172.21.15.67:0/332519610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 msgr2=0x55eaedf550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8de00700 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 0x55eaedf550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8de00700 1 -- 172.21.15.67:0/332519610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf415d0 msgr2=0x55eaedf419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8de00700 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf415d0 0x55eaedf419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8de00700 1 -- 172.21.15.67:0/332519610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaedd153b0 con 0x55eaedf55600 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8ca00700 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 0x55eaedf550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8d400700 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf415d0 0x55eaedf419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b8de00700 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf579f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8b80095e80 tx=0x7f8b80004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ac55039553bf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b7a600700 1 -- 172.21.15.67:0/332519610 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800840c0 con 0x55eaedf55600 2024-03-30T02:39:39.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b7a600700 1 -- 172.21.15.67:0/332519610 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8009d240 con 0x55eaedf55600 2024-03-30T02:39:39.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.837+0000 7f8b7a600700 1 -- 172.21.15.67:0/332519610 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800cc2e0 con 0x55eaedf55600 2024-03-30T02:39:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.838+0000 7f8ba382bb80 1 -- 172.21.15.67:0/332519610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 msgr2=0x55eaedf579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.838+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf579f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8b80095e80 tx=0x7f8b80004f10 comp rx=0 tx=0).stop 2024-03-30T02:39:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.838+0000 7f8ba382bb80 1 -- 172.21.15.67:0/332519610 shutdown_connections 2024-03-30T02:39:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.838+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf415d0 0x55eaedf419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.838+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf579f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.838+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/332519610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 0x55eaedf550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.838+0000 7f8ba382bb80 1 -- 172.21.15.67:0/332519610 >> 172.21.15.67:0/332519610 conn(0x55eaedea16a0 msgr2=0x55eaedf3da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.838+0000 7f8ba382bb80 1 -- 172.21.15.67:0/332519610 shutdown_connections 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.838+0000 7f8ba382bb80 1 -- 172.21.15.67:0/332519610 wait complete. 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.839+0000 7f8ba382bb80 1 -- start start 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.839+0000 7f8ba382bb80 1 Processor -- start 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf415d0 0x55eaedf52350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 0x55eaedf52890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 0x55eaedf52dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaede2fd80 con 0x55eaedf55600 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaede2fa20 con 0x55eaedf415d0 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaedd176a0 con 0x55eaedf41ef0 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8ba382bb80 10 client.0 _ll_register_callbacks cb 0x55eaede39470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:39.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf415d0 0x55eaedf52350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf415d0 0x55eaedf52350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43392/0 (socket says 172.21.15.67:43392) 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8d400700 1 -- 172.21.15.67:0/1109357552 learned_addr learned my addr 172.21.15.67:0/1109357552 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8ca00700 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 0x55eaedf52890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8de00700 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 0x55eaedf52dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8d400700 1 -- 172.21.15.67:0/1109357552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 msgr2=0x55eaedf52890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8d400700 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 0x55eaedf52890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8d400700 1 -- 172.21.15.67:0/1109357552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 msgr2=0x55eaedf52dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8d400700 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 0x55eaedf52dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8d400700 1 -- 172.21.15.67:0/1109357552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b881a6450 con 0x55eaedf415d0 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.840+0000 7f8b8de00700 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 0x55eaedf52dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.841+0000 7f8b8d400700 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf415d0 0x55eaedf52350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c016000 tx=0x7f8b7c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c420ef7e0c160e17 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.841+0000 7f8b6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.841+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1109357552 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c005240 con 0x55eaedf415d0 2024-03-30T02:39:39.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.841+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1109357552 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f8b7c00f910 con 0x55eaedf415d0 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.841+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1109357552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eaede54380 con 0x55eaedf415d0 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.841+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1109357552 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b7c00bc00 con 0x55eaedf415d0 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.841+0000 7f8ba382bb80 10 client.26336 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.841+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1109357552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55eaedcc1b60 con 0x55eaedf415d0 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.841+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1109357552 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eaede54380 con 0x55eaedf415d0 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.842+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1109357552 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8b7c023410 con 0x55eaedf415d0 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.842+0000 7f8ba382bb80 10 client.26336 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.842+0000 7f8ba382bb80 10 client.26336 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.842+0000 7f8ba382bb80 20 client.26336 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.842+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55eaedf652f0 0x55eaedf67710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.842+0000 7f8ba382bb80 4 client.26336 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.842+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1109357552 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eaede8c700 con 0x55eaedf652f0 2024-03-30T02:39:39.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.842+0000 7f8b8ca00700 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55eaedf652f0 0x55eaedf67710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.843+0000 7f8b8ca00700 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55eaedf652f0 0x55eaedf67710 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b34b2d87cc0a866a server_cookie=4acbfb55009478e4 in_seq=0 out_seq=0 2024-03-30T02:39:39.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.843+0000 7f8b6ea00700 10 client.26336 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.843+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1109357552 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eaede8c700 con 0x55eaedf652f0 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.843+0000 7f8b6ea00700 10 client.26336 handle_command_reply: tid=0 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.843+0000 7f8ba382bb80 1 client.26336 shutdown 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1109357552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf415d0 msgr2=0x55eaedf52350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf415d0 0x55eaedf52350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c016000 tx=0x7f8b7c009260 comp rx=0 tx=0).stop 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1109357552 shutdown_connections 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 0x55eaedf52dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55eaedf652f0 0x55eaedf67710 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf415d0 0x55eaedf52350 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1109357552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf41ef0 0x55eaedf52890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1109357552 >> 172.21.15.67:0/1109357552 conn(0x55eaedea16a0 msgr2=0x55eaedf436c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8b6ea00700 0 client.26336 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:39:39.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1109357552 shutdown_connections 2024-03-30T02:39:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1109357552 wait complete. 2024-03-30T02:39:39.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.844+0000 7f8ba382bb80 20 client.26336 trim_cache size 0 max 16384 2024-03-30T02:39:39.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8ba382bb80 1 Processor -- start 2024-03-30T02:39:39.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8ba382bb80 1 -- start start 2024-03-30T02:39:39.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 0x55eaedf5c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf41ef0 0x55eaedf4d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf4dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaede543b0 con 0x55eaedf41ef0 2024-03-30T02:39:39.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaedf445e0 con 0x55eaedf55600 2024-03-30T02:39:39.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaedf3f370 con 0x55eaedf415d0 2024-03-30T02:39:39.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8b8ca00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf41ef0 0x55eaedf4d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8b8d400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 0x55eaedf5c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8b8ca00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf41ef0 0x55eaedf4d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35398/0 (socket says 172.21.15.67:35398) 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8b8ca00700 1 -- 172.21.15.67:0/1997114315 learned_addr learned my addr 172.21.15.67:0/1997114315 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.859+0000 7f8b8de00700 1 --2- 172.21.15.67:0/1997114315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf4dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.860+0000 7f8b8d400700 1 -- 172.21.15.67:0/1997114315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 msgr2=0x55eaedf4dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.860+0000 7f8b8d400700 1 --2- 172.21.15.67:0/1997114315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf4dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.860+0000 7f8b8d400700 1 -- 172.21.15.67:0/1997114315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf41ef0 msgr2=0x55eaedf4d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.860+0000 7f8b8d400700 1 --2- 172.21.15.67:0/1997114315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf41ef0 0x55eaedf4d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.860+0000 7f8b8d400700 1 -- 172.21.15.67:0/1997114315 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaede59ea0 con 0x55eaedf415d0 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.860+0000 7f8b8d400700 1 --2- 172.21.15.67:0/1997114315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 0x55eaedf5c280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c023820 tx=0x7f8b7c00ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1b4617b8c0a19da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.860+0000 7f8b7b000700 1 -- 172.21.15.67:0/1997114315 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c00fd00 con 0x55eaedf415d0 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.860+0000 7f8b7b000700 1 -- 172.21.15.67:0/1997114315 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b7c02b040 con 0x55eaedf415d0 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.860+0000 7f8b7b000700 1 -- 172.21.15.67:0/1997114315 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c02ebc0 con 0x55eaedf415d0 2024-03-30T02:39:39.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1997114315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 msgr2=0x55eaedf5c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1997114315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 0x55eaedf5c280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c023820 tx=0x7f8b7c00ffd0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1997114315 shutdown_connections 2024-03-30T02:39:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1997114315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf55600 0x55eaedf4dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1997114315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 0x55eaedf5c280 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1997114315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf41ef0 0x55eaedf4d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1997114315 >> 172.21.15.67:0/1997114315 conn(0x55eaedea16a0 msgr2=0x55eaedf365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1997114315 shutdown_connections 2024-03-30T02:39:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1997114315 wait complete. 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.861+0000 7f8ba382bb80 1 -- start start 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.862+0000 7f8ba382bb80 1 Processor -- start 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.862+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 0x55eaedfb03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.862+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf41ef0 0x55eaedfb08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.862+0000 7f8ba382bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 0x55eaedfb1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.862+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaedfb1800 con 0x55eaedf55600 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaedfb1970 con 0x55eaedf41ef0 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8ba382bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaedfb1b30 con 0x55eaedf415d0 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8ba382bb80 10 client.0 _ll_register_callbacks cb 0x55eaede39470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf41ef0 0x55eaedfb08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf41ef0 0x55eaedfb08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43414/0 (socket says 172.21.15.67:43414) 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8ca00700 1 -- 172.21.15.67:0/1862756466 learned_addr learned my addr 172.21.15.67:0/1862756466 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8d400700 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 0x55eaedfb03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8de00700 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 0x55eaedfb1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8ca00700 1 -- 172.21.15.67:0/1862756466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 msgr2=0x55eaedfb03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8ca00700 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 0x55eaedfb03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8ca00700 1 -- 172.21.15.67:0/1862756466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 msgr2=0x55eaedfb1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8ca00700 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 0x55eaedfb1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.863+0000 7f8b8ca00700 1 -- 172.21.15.67:0/1862756466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaedcc1b60 con 0x55eaedf41ef0 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.864+0000 7f8b8ca00700 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf41ef0 0x55eaedfb08f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b88101350 tx=0x7f8b74004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127899398d450bb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.864+0000 7f8b6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.864+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1862756466 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74017070 con 0x55eaedf41ef0 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.864+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1862756466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b8009d240 con 0x55eaedf41ef0 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.864+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1862756466 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f8b74003cc0 con 0x55eaedf41ef0 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.864+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1862756466 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b74003e80 con 0x55eaedf41ef0 2024-03-30T02:39:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.864+0000 7f8ba382bb80 10 client.26342 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.864+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1862756466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55eaedd19cd0 con 0x55eaedf41ef0 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.864+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1862756466 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8009d240 con 0x55eaedf41ef0 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1862756466 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8b74012540 con 0x55eaedf41ef0 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8ba382bb80 10 client.26342 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8ba382bb80 10 client.26342 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8ba382bb80 20 client.26342 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55eaedfba520 0x55eaedfbc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8ba382bb80 4 client.26342 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1862756466 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eaedce2cf0 con 0x55eaedfba520 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8b8d400700 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55eaedfba520 0x55eaedfbc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8b8d400700 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55eaedfba520 0x55eaedfbc940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f562411f35bdbf72 server_cookie=abc13e0108992ca8 in_seq=0 out_seq=0 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.865+0000 7f8b6ea00700 10 client.26342 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8b6ea00700 1 -- 172.21.15.67:0/1862756466 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eaedce2cf0 con 0x55eaedfba520 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8b6ea00700 10 client.26342 handle_command_reply: tid=0 2024-03-30T02:39:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 client.26342 shutdown 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1862756466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf41ef0 msgr2=0x55eaedfb08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf41ef0 0x55eaedfb08f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8b88101350 tx=0x7f8b74004530 comp rx=0 tx=0).stop 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1862756466 shutdown_connections 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eaedf55600 0x55eaedfb1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55eaedfba520 0x55eaedfbc940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eaedf415d0 0x55eaedfb03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 --2- 172.21.15.67:0/1862756466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaedf41ef0 0x55eaedfb08f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1862756466 >> 172.21.15.67:0/1862756466 conn(0x55eaedea16a0 msgr2=0x55eaedf4d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8b6ea00700 0 client.26342 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1862756466 shutdown_connections 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 1 -- 172.21.15.67:0/1862756466 wait complete. 2024-03-30T02:39:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.866+0000 7f8ba382bb80 20 client.26342 trim_cache size 0 max 16384 2024-03-30T02:39:39.871 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 -- 172.21.15.67:0/3628713221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8b700a1310 msgr2=0x7f8b700a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8b700a1310 0x7f8b700a37d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c010 tx=0x7f8b7c019040 comp rx=0 tx=0).stop 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 -- 172.21.15.67:0/3628713221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b881006f0 msgr2=0x7f8b881a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b881006f0 0x7f8b881a54d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8b80007ee0 tx=0x7f8b8000d3b0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 -- 172.21.15.67:0/3628713221 shutdown_connections 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8b700a1310 0x7f8b700a37d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8b88069260 0x7f8b880ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b881006f0 0x7f8b881a54d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 --2- 172.21.15.67:0/3628713221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b881060b0 0x7f8b881001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 -- 172.21.15.67:0/3628713221 >> 172.21.15.67:0/3628713221 conn(0x7f8b88076d80 msgr2=0x7f8b880fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 -- 172.21.15.67:0/3628713221 shutdown_connections 2024-03-30T02:39:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:39.867+0000 7f8b8fe00700 1 -- 172.21.15.67:0/3628713221 wait complete. 2024-03-30T02:39:39.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:39:40.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.104+0000 7f2574a00700 1 -- 172.21.15.67:0/3413542062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2570101900 msgr2=0x7f2570105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.104+0000 7f2574a00700 1 --2- 172.21.15.67:0/3413542062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2570101900 0x7f2570105940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f255c009990 tx=0x7f255c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.104+0000 7f2574a00700 1 -- 172.21.15.67:0/3413542062 shutdown_connections 2024-03-30T02:39:40.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.104+0000 7f2574a00700 1 --2- 172.21.15.67:0/3413542062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2570100f50 0x7f2570101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.104+0000 7f2574a00700 1 --2- 172.21.15.67:0/3413542062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2570106070 0x7f257010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.104+0000 7f2574a00700 1 --2- 172.21.15.67:0/3413542062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2570101900 0x7f2570105940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.104+0000 7f2574a00700 1 -- 172.21.15.67:0/3413542062 >> 172.21.15.67:0/3413542062 conn(0x7f25700fc7f0 msgr2=0x7f25700fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.105+0000 7f2574a00700 1 -- 172.21.15.67:0/3413542062 shutdown_connections 2024-03-30T02:39:40.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.105+0000 7f2574a00700 1 -- 172.21.15.67:0/3413542062 wait complete. 2024-03-30T02:39:40.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.106+0000 7f2574a00700 1 Processor -- start 2024-03-30T02:39:40.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.106+0000 7f2574a00700 1 -- start start 2024-03-30T02:39:40.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.106+0000 7f2574a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2570100f50 0x7f2570197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.106+0000 7f2574a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2570106070 0x7f2570197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.106+0000 7f2574a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f257019bab0 0x7f257019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.106+0000 7f2574a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257019c530 con 0x7f2570100f50 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.106+0000 7f2574a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257019c6a0 con 0x7f257019bab0 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.106+0000 7f2574a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257019c860 con 0x7f2570106070 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2570100f50 0x7f2570197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2570100f50 0x7f2570197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35422/0 (socket says 172.21.15.67:35422) 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256de00700 1 -- 172.21.15.67:0/155901826 learned_addr learned my addr 172.21.15.67:0/155901826 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256e800700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f257019bab0 0x7f257019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256d400700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2570106070 0x7f2570197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256de00700 1 -- 172.21.15.67:0/155901826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2570106070 msgr2=0x7f2570197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256de00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2570106070 0x7f2570197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256de00700 1 -- 172.21.15.67:0/155901826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f257019bab0 msgr2=0x7f257019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256de00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f257019bab0 0x7f257019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256de00700 1 -- 172.21.15.67:0/155901826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0095f0 con 0x7f2570100f50 2024-03-30T02:39:40.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256e800700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f257019bab0 0x7f257019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:40.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f256de00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2570100f50 0x7f2570197220 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f256000e9a0 tx=0x7f256000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.107+0000 7f2566a00700 1 -- 172.21.15.67:0/155901826 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560009790 con 0x7f2570100f50 2024-03-30T02:39:40.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.108+0000 7f2566a00700 1 -- 172.21.15.67:0/155901826 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2560010430 con 0x7f2570100f50 2024-03-30T02:39:40.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.108+0000 7f2574a00700 1 -- 172.21.15.67:0/155901826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25701a3dc0 con 0x7f2570100f50 2024-03-30T02:39:40.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.108+0000 7f2566a00700 1 -- 172.21.15.67:0/155901826 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560018680 con 0x7f2570100f50 2024-03-30T02:39:40.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.108+0000 7f2574a00700 1 -- 172.21.15.67:0/155901826 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25701a4240 con 0x7f2570100f50 2024-03-30T02:39:40.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.111+0000 7f2566a00700 1 -- 172.21.15.67:0/155901826 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f25600105a0 con 0x7f2570100f50 2024-03-30T02:39:40.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.112+0000 7f25885cbb80 1 Processor -- start 2024-03-30T02:39:40.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.112+0000 7f25885cbb80 1 -- start start 2024-03-30T02:39:40.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f2566a00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f25540a1310 0x7f25540a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f2566a00700 1 -- 172.21.15.67:0/155901826 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f2560014070 con 0x7f2570100f50 2024-03-30T02:39:40.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626113719e0 0x562611371dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562611372300 0x56261137cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56261137d1e0 0x56261137f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f25885cbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626112c6990 con 0x56261137d1e0 2024-03-30T02:39:40.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f25885cbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626112c4350 con 0x562611372300 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f25885cbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626112bfc40 con 0x5626113719e0 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f256e800700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f25540a1310 0x7f25540a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f256de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626113719e0 0x562611371dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f256d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562611372300 0x56261137cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f256de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626113719e0 0x562611371dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:34576/0 (socket says 172.21.15.67:34576) 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f256d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562611372300 0x56261137cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43428/0 (socket says 172.21.15.67:43428) 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.113+0000 7f256de00700 1 -- 172.21.15.67:0/4227941059 learned_addr learned my addr 172.21.15.67:0/4227941059 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f256e800700 1 --2- 172.21.15.67:0/4227941059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56261137d1e0 0x56261137f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f256de00700 1 -- 172.21.15.67:0/4227941059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562611372300 msgr2=0x56261137cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f256de00700 1 --2- 172.21.15.67:0/4227941059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562611372300 0x56261137cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f256de00700 1 -- 172.21.15.67:0/4227941059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56261137d1e0 msgr2=0x56261137f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f256de00700 1 --2- 172.21.15.67:0/4227941059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56261137d1e0 0x56261137f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f256de00700 1 -- 172.21.15.67:0/4227941059 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626111413b0 con 0x5626113719e0 2024-03-30T02:39:40.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f256e800700 1 --2- 172.21.15.67:0/4227941059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56261137d1e0 0x56261137f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f256de00700 1 --2- 172.21.15.67:0/4227941059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626113719e0 0x562611371dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f25600969f0 tx=0x7f25600999f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2270429566be8f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f256e800700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f25540a1310 0x7f25540a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2568009fd0 tx=0x7f2568009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f255b000700 1 -- 172.21.15.67:0/4227941059 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256009ddb0 con 0x5626113719e0 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f255b000700 1 -- 172.21.15.67:0/4227941059 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25600a2400 con 0x5626113719e0 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.114+0000 7f255b000700 1 -- 172.21.15.67:0/4227941059 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560099470 con 0x5626113719e0 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.115+0000 7f25885cbb80 1 -- 172.21.15.67:0/4227941059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626113719e0 msgr2=0x562611371dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.115+0000 7f25885cbb80 1 --2- 172.21.15.67:0/4227941059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626113719e0 0x562611371dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f25600969f0 tx=0x7f25600999f0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.115+0000 7f25885cbb80 1 -- 172.21.15.67:0/4227941059 shutdown_connections 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.115+0000 7f25885cbb80 1 --2- 172.21.15.67:0/4227941059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56261137d1e0 0x56261137f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.115+0000 7f25885cbb80 1 --2- 172.21.15.67:0/4227941059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626113719e0 0x562611371dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.115+0000 7f25885cbb80 1 --2- 172.21.15.67:0/4227941059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562611372300 0x56261137cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.115+0000 7f25885cbb80 1 -- 172.21.15.67:0/4227941059 >> 172.21.15.67:0/4227941059 conn(0x5626112cd6a0 msgr2=0x562611369a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.115+0000 7f25885cbb80 1 -- 172.21.15.67:0/4227941059 shutdown_connections 2024-03-30T02:39:40.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.115+0000 7f25885cbb80 1 -- 172.21.15.67:0/4227941059 wait complete. 2024-03-30T02:39:40.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.116+0000 7f25885cbb80 1 -- start start 2024-03-30T02:39:40.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.116+0000 7f25885cbb80 1 Processor -- start 2024-03-30T02:39:40.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.116+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x56261137cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.116+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611379f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.116+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 0x56261137a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.116+0000 7f25885cbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261125bd80 con 0x5626113719e0 2024-03-30T02:39:40.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.116+0000 7f25885cbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261125ba20 con 0x56261137d1e0 2024-03-30T02:39:40.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.116+0000 7f25885cbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626111436a0 con 0x562611372300 2024-03-30T02:39:40.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f25885cbb80 10 client.0 _ll_register_callbacks cb 0x562611265470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:40.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256d400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611379f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x56261137cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256d400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611379f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:34590/0 (socket says 172.21.15.67:34590) 2024-03-30T02:39:40.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256d400700 1 -- 172.21.15.67:0/3856121384 learned_addr learned my addr 172.21.15.67:0/3856121384 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256e800700 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 0x56261137a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256d400700 1 -- 172.21.15.67:0/3856121384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 msgr2=0x56261137a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256d400700 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 0x56261137a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256d400700 1 -- 172.21.15.67:0/3856121384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 msgr2=0x56261137cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256d400700 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x56261137cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256d400700 1 -- 172.21.15.67:0/3856121384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25701a4240 con 0x562611372300 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256de00700 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x56261137cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f256d400700 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611379f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f255c009990 tx=0x7f255c005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5418e8170e343a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.117+0000 7f254f400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.118+0000 7f254f400700 1 -- 172.21.15.67:0/3856121384 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c01d070 con 0x562611372300 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.118+0000 7f25885cbb80 1 -- 172.21.15.67:0/3856121384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562611280380 con 0x562611372300 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.118+0000 7f254f400700 1 -- 172.21.15.67:0/3856121384 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f255c004500 con 0x562611372300 2024-03-30T02:39:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.118+0000 7f254f400700 1 -- 172.21.15.67:0/3856121384 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f255c00bee0 con 0x562611372300 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.118+0000 7f25885cbb80 10 client.26323 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.118+0000 7f25885cbb80 1 -- 172.21.15.67:0/3856121384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5626110edb60 con 0x562611372300 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.118+0000 7f254f400700 1 -- 172.21.15.67:0/3856121384 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562611280380 con 0x562611372300 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f254f400700 1 -- 172.21.15.67:0/3856121384 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f255c00f700 con 0x562611372300 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f25885cbb80 10 client.26323 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f25885cbb80 10 client.26323 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f25885cbb80 20 client.26323 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x56261138cd30 0x56261138f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f25885cbb80 4 client.26323 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f25885cbb80 1 -- 172.21.15.67:0/3856121384 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626112b8700 con 0x56261138cd30 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f256de00700 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x56261138cd30 0x56261138f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f256de00700 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x56261138cd30 0x56261138f150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88a6829a34f8e63a server_cookie=309a727a153a3a6b in_seq=0 out_seq=0 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.119+0000 7f254f400700 10 client.26323 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f254f400700 1 -- 172.21.15.67:0/3856121384 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5626112b8700 con 0x56261138cd30 2024-03-30T02:39:40.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f254f400700 10 client.26323 handle_command_reply: tid=0 2024-03-30T02:39:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f25885cbb80 1 client.26323 shutdown 2024-03-30T02:39:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f25885cbb80 1 -- 172.21.15.67:0/3856121384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 msgr2=0x562611379f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611379f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f255c009990 tx=0x7f255c005800 comp rx=0 tx=0).stop 2024-03-30T02:39:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f25885cbb80 1 -- 172.21.15.67:0/3856121384 shutdown_connections 2024-03-30T02:39:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x56261138cd30 0x56261138f150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x56261137cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 0x56261137a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3856121384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611379f20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.120+0000 7f25885cbb80 1 -- 172.21.15.67:0/3856121384 >> 172.21.15.67:0/3856121384 conn(0x5626112cd6a0 msgr2=0x562611373af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.121+0000 7f254f400700 0 client.26323 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:39:40.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.121+0000 7f25885cbb80 1 -- 172.21.15.67:0/3856121384 shutdown_connections 2024-03-30T02:39:40.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.121+0000 7f25885cbb80 1 -- 172.21.15.67:0/3856121384 wait complete. 2024-03-30T02:39:40.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.121+0000 7f25885cbb80 20 client.26323 trim_cache size 0 max 16384 2024-03-30T02:39:40.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.134+0000 7f25885cbb80 1 Processor -- start 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.134+0000 7f25885cbb80 1 -- start start 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x5626113787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611378d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 0x5626113885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f25885cbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562611389940 con 0x5626113719e0 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f25885cbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562611379270 con 0x56261137d1e0 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f25885cbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562611369f20 con 0x562611372300 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f256de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x5626113787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f256de00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x5626113787f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35452/0 (socket says 172.21.15.67:35452) 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f256de00700 1 -- 172.21.15.67:0/3192156512 learned_addr learned my addr 172.21.15.67:0/3192156512 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f256e800700 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 0x5626113885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.135+0000 7f256d400700 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611378d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f256de00700 1 -- 172.21.15.67:0/3192156512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 msgr2=0x562611378d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f256de00700 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611378d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f256de00700 1 -- 172.21.15.67:0/3192156512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 msgr2=0x5626113885b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f256de00700 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 0x5626113885b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f256de00700 1 -- 172.21.15.67:0/3192156512 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562611285ea0 con 0x5626113719e0 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f256d400700 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611378d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f256e800700 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 0x5626113885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f256de00700 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x5626113787f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f256000e970 tx=0x7f256009af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7506117bf217c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f255ba00700 1 -- 172.21.15.67:0/3192156512 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256009ddb0 con 0x5626113719e0 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f255ba00700 1 -- 172.21.15.67:0/3192156512 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2560096710 con 0x5626113719e0 2024-03-30T02:39:40.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.136+0000 7f255ba00700 1 -- 172.21.15.67:0/3192156512 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256009a440 con 0x5626113719e0 2024-03-30T02:39:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 -- 172.21.15.67:0/3192156512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 msgr2=0x5626113787f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x5626113787f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f256000e970 tx=0x7f256009af90 comp rx=0 tx=0).stop 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 -- 172.21.15.67:0/3192156512 shutdown_connections 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626113719e0 0x5626113787f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56261137d1e0 0x5626113885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 --2- 172.21.15.67:0/3192156512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562611372300 0x562611378d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 -- 172.21.15.67:0/3192156512 >> 172.21.15.67:0/3192156512 conn(0x5626112cd6a0 msgr2=0x562611375240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 -- 172.21.15.67:0/3192156512 shutdown_connections 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 -- 172.21.15.67:0/3192156512 wait complete. 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.137+0000 7f25885cbb80 1 -- start start 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f25885cbb80 1 Processor -- start 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626113719e0 0x562611384050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562611372300 0x562611381e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f25885cbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56261137d1e0 0x562611382580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f25885cbb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562611382c60 con 0x562611372300 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f25885cbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626113dfb60 con 0x5626113719e0 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f25885cbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626113dfca0 con 0x56261137d1e0 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f25885cbb80 10 client.0 _ll_register_callbacks cb 0x562611265470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f256de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626113719e0 0x562611384050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f256de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626113719e0 0x562611384050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43440/0 (socket says 172.21.15.67:43440) 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f256de00700 1 -- 172.21.15.67:0/886166904 learned_addr learned my addr 172.21.15.67:0/886166904 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f256d400700 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562611372300 0x562611381e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.138+0000 7f256e800700 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56261137d1e0 0x562611382580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f256de00700 1 -- 172.21.15.67:0/886166904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56261137d1e0 msgr2=0x562611382580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f256de00700 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56261137d1e0 0x562611382580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f256de00700 1 -- 172.21.15.67:0/886166904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562611372300 msgr2=0x562611381e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f256de00700 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562611372300 0x562611381e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f256de00700 1 -- 172.21.15.67:0/886166904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25680090f0 con 0x5626113719e0 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f256d400700 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562611372300 0x562611381e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f256e800700 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56261137d1e0 0x562611382580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f256de00700 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626113719e0 0x562611384050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f256009a750 tx=0x7f256009a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e776db0ebb1a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f254f400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f254f400700 1 -- 172.21.15.67:0/886166904 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560010880 con 0x5626113719e0 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f254f400700 1 -- 172.21.15.67:0/886166904 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f25600b81c0 con 0x5626113719e0 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f25885cbb80 1 -- 172.21.15.67:0/886166904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25600a2400 con 0x5626113719e0 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f254f400700 1 -- 172.21.15.67:0/886166904 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25600b8380 con 0x5626113719e0 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f25885cbb80 10 client.26360 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f25885cbb80 1 -- 172.21.15.67:0/886166904 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5626111413b0 con 0x5626113719e0 2024-03-30T02:39:40.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.139+0000 7f254f400700 1 -- 172.21.15.67:0/886166904 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f256007b070 con 0x5626113719e0 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.140+0000 7f254f400700 1 -- 172.21.15.67:0/886166904 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2560094c20 con 0x5626113719e0 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.140+0000 7f25885cbb80 10 client.26360 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.140+0000 7f25885cbb80 10 client.26360 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.140+0000 7f25885cbb80 20 client.26360 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.140+0000 7f25885cbb80 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5626113e0f60 0x5626113e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.140+0000 7f25885cbb80 4 client.26360 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.140+0000 7f25885cbb80 1 -- 172.21.15.67:0/886166904 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56261110ecf0 con 0x5626113e0f60 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.140+0000 7f256d400700 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5626113e0f60 0x5626113e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f254f400700 10 client.26360 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f256d400700 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5626113e0f60 0x5626113e3380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec269e380624288e server_cookie=d27c428a6dfb83ee in_seq=0 out_seq=0 2024-03-30T02:39:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f254f400700 1 -- 172.21.15.67:0/886166904 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56261110ecf0 con 0x5626113e0f60 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f254f400700 10 client.26360 handle_command_reply: tid=0 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 client.26360 shutdown 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 -- 172.21.15.67:0/886166904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626113719e0 msgr2=0x562611384050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626113719e0 0x562611384050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f256009a750 tx=0x7f256009a100 comp rx=0 tx=0).stop 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 -- 172.21.15.67:0/886166904 shutdown_connections 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5626113e0f60 0x5626113e3380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56261137d1e0 0x562611382580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626113719e0 0x562611384050 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 --2- 172.21.15.67:0/886166904 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562611372300 0x562611381e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 -- 172.21.15.67:0/886166904 >> 172.21.15.67:0/886166904 conn(0x5626112cd6a0 msgr2=0x562611379460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f254f400700 0 client.26360 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 -- 172.21.15.67:0/886166904 shutdown_connections 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 1 -- 172.21.15.67:0/886166904 wait complete. 2024-03-30T02:39:40.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.141+0000 7f25885cbb80 20 client.26360 trim_cache size 0 max 16384 2024-03-30T02:39:40.149 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:40.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 -- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f25540a1310 msgr2=0x7f25540a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f25540a1310 0x7f25540a37d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2568009fd0 tx=0x7f2568009920 comp rx=0 tx=0).stop 2024-03-30T02:39:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 -- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2570100f50 msgr2=0x7f2570197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2570100f50 0x7f2570197220 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f256000e9a0 tx=0x7f256000ecb0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 -- 172.21.15.67:0/155901826 shutdown_connections 2024-03-30T02:39:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f25540a1310 0x7f25540a37d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2570100f50 0x7f2570197220 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f257019bab0 0x7f257019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 --2- 172.21.15.67:0/155901826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2570106070 0x7f2570197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 -- 172.21.15.67:0/155901826 >> 172.21.15.67:0/155901826 conn(0x7f25700fc7f0 msgr2=0x7f25700fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 -- 172.21.15.67:0/155901826 shutdown_connections 2024-03-30T02:39:40.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.142+0000 7f2574a00700 1 -- 172.21.15.67:0/155901826 wait complete. 2024-03-30T02:39:40.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:39:40.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.372+0000 7fe32e600700 1 -- 172.21.15.67:0/826465215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe328101cf0 msgr2=0x7fe3281020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.372+0000 7fe32e600700 1 --2- 172.21.15.67:0/826465215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe328101cf0 0x7fe3281020d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009990 tx=0x7fe31c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.372+0000 7fe32e600700 1 -- 172.21.15.67:0/826465215 shutdown_connections 2024-03-30T02:39:40.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.372+0000 7fe32e600700 1 --2- 172.21.15.67:0/826465215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe32810b110 0x7fe32810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.372+0000 7fe32e600700 1 --2- 172.21.15.67:0/826465215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe328101cf0 0x7fe3281020d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.372+0000 7fe32e600700 1 --2- 172.21.15.67:0/826465215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe328102610 0x7fe32810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.372+0000 7fe32e600700 1 -- 172.21.15.67:0/826465215 >> 172.21.15.67:0/826465215 conn(0x7fe3280fb510 msgr2=0x7fe3280fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.372+0000 7fe32e600700 1 -- 172.21.15.67:0/826465215 shutdown_connections 2024-03-30T02:39:40.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.372+0000 7fe32e600700 1 -- 172.21.15.67:0/826465215 wait complete. 2024-03-30T02:39:40.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.373+0000 7fe32e600700 1 Processor -- start 2024-03-30T02:39:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.373+0000 7fe32e600700 1 -- start start 2024-03-30T02:39:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32e600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe328101cf0 0x7fe3280ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32e600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe328102610 0x7fe3280ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32e600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32810b110 0x7fe3281003c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32e600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281a53e0 con 0x7fe328102610 2024-03-30T02:39:40.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32e600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281a5550 con 0x7fe32810b110 2024-03-30T02:39:40.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32e600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281a5710 con 0x7fe328101cf0 2024-03-30T02:39:40.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe327e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe328102610 0x7fe3280ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32dc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32810b110 0x7fe3281003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe327e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe328102610 0x7fe3280ffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35482/0 (socket says 172.21.15.67:35482) 2024-03-30T02:39:40.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32dc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32810b110 0x7fe3281003c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43462/0 (socket says 172.21.15.67:43462) 2024-03-30T02:39:40.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe327e00700 1 -- 172.21.15.67:0/2759593366 learned_addr learned my addr 172.21.15.67:0/2759593366 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32d200700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe328101cf0 0x7fe3280ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32dc00700 1 -- 172.21.15.67:0/2759593366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe328101cf0 msgr2=0x7fe3280ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32dc00700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe328101cf0 0x7fe3280ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32dc00700 1 -- 172.21.15.67:0/2759593366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe328102610 msgr2=0x7fe3280ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32dc00700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe328102610 0x7fe3280ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32dc00700 1 -- 172.21.15.67:0/2759593366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0095f0 con 0x7fe32810b110 2024-03-30T02:39:40.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.374+0000 7fe32d200700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe328101cf0 0x7fe3280ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:40.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.375+0000 7fe32dc00700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32810b110 0x7fe3281003c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe32000efc0 tx=0x7fe32000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.375+0000 7fe325600700 1 -- 172.21.15.67:0/2759593366 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320018070 con 0x7fe32810b110 2024-03-30T02:39:40.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.375+0000 7fe325600700 1 -- 172.21.15.67:0/2759593366 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe32000f040 con 0x7fe32810b110 2024-03-30T02:39:40.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.375+0000 7fe32e600700 1 -- 172.21.15.67:0/2759593366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3281a5990 con 0x7fe32810b110 2024-03-30T02:39:40.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.375+0000 7fe325600700 1 -- 172.21.15.67:0/2759593366 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32000cdf0 con 0x7fe32810b110 2024-03-30T02:39:40.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.375+0000 7fe32e600700 1 -- 172.21.15.67:0/2759593366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3281a5ea0 con 0x7fe32810b110 2024-03-30T02:39:40.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.380+0000 7fe325600700 1 -- 172.21.15.67:0/2759593366 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe320007500 con 0x7fe32810b110 2024-03-30T02:39:40.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.381+0000 7fe325600700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe3100a13e0 0x7fe3100a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.381+0000 7fe343629b80 1 Processor -- start 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.381+0000 7fe325600700 1 -- 172.21.15.67:0/2759593366 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fe3200ccad0 con 0x7fe32810b110 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.381+0000 7fe32d200700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe3100a13e0 0x7fe3100a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.381+0000 7fe343629b80 1 -- start start 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe343629b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b44ff7a0 0x55f2b44ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe343629b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b45000c0 0x55f2b450aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe343629b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b450d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe343629b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b4458990 con 0x55f2b450afc0 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe343629b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b4456350 con 0x55f2b45000c0 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe343629b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b4451c40 con 0x55f2b44ff7a0 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe327e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b45000c0 0x55f2b450aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe32dc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b450d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe327e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b45000c0 0x55f2b450aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43466/0 (socket says 172.21.15.67:43466) 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe327e00700 1 -- 172.21.15.67:0/1815690547 learned_addr learned my addr 172.21.15.67:0/1815690547 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe32d200700 1 --2- 172.21.15.67:0/1815690547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b44ff7a0 0x55f2b44ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe32d200700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe3100a13e0 0x7fe3100a38a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe31c006010 tx=0x7fe31c00b260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe327e00700 1 -- 172.21.15.67:0/1815690547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b44ff7a0 msgr2=0x55f2b44ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe327e00700 1 --2- 172.21.15.67:0/1815690547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b44ff7a0 0x55f2b44ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe327e00700 1 -- 172.21.15.67:0/1815690547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 msgr2=0x55f2b450d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe327e00700 1 --2- 172.21.15.67:0/1815690547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b450d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe327e00700 1 -- 172.21.15.67:0/1815690547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2b42d33b0 con 0x55f2b45000c0 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.382+0000 7fe32dc00700 1 --2- 172.21.15.67:0/1815690547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b450d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.383+0000 7fe327e00700 1 --2- 172.21.15.67:0/1815690547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b45000c0 0x55f2b450aa80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe314009de0 tx=0x7fe314011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58e4747419a2674f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.383+0000 7fe31a000700 1 -- 172.21.15.67:0/1815690547 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314004230 con 0x55f2b45000c0 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.383+0000 7fe31a000700 1 -- 172.21.15.67:0/1815690547 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe314004cf0 con 0x55f2b45000c0 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.383+0000 7fe31a000700 1 -- 172.21.15.67:0/1815690547 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3140056c0 con 0x55f2b45000c0 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 -- 172.21.15.67:0/1815690547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b45000c0 msgr2=0x55f2b450aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 --2- 172.21.15.67:0/1815690547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b45000c0 0x55f2b450aa80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe314009de0 tx=0x7fe314011720 comp rx=0 tx=0).stop 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 -- 172.21.15.67:0/1815690547 shutdown_connections 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 --2- 172.21.15.67:0/1815690547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b450d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 --2- 172.21.15.67:0/1815690547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b44ff7a0 0x55f2b44ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 --2- 172.21.15.67:0/1815690547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b45000c0 0x55f2b450aa80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 -- 172.21.15.67:0/1815690547 >> 172.21.15.67:0/1815690547 conn(0x55f2b445f6a0 msgr2=0x55f2b44fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 -- 172.21.15.67:0/1815690547 shutdown_connections 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 -- 172.21.15.67:0/1815690547 wait complete. 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.384+0000 7fe343629b80 1 -- start start 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe343629b80 1 Processor -- start 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe343629b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b450a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe343629b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 0x55f2b4507d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe343629b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 0x55f2b4508320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe343629b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b43edd80 con 0x55f2b45000c0 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe343629b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b43eda20 con 0x55f2b44ff7a0 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe343629b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b42d56a0 con 0x55f2b450afc0 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe343629b80 10 client.0 _ll_register_callbacks cb 0x55f2b43f7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe32d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b450a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe32d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b450a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43476/0 (socket says 172.21.15.67:43476) 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe32d200700 1 -- 172.21.15.67:0/843763559 learned_addr learned my addr 172.21.15.67:0/843763559 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe327e00700 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 0x55f2b4507d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.385+0000 7fe32dc00700 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 0x55f2b4508320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe32d200700 1 -- 172.21.15.67:0/843763559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 msgr2=0x55f2b4508320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe32d200700 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 0x55f2b4508320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe32d200700 1 -- 172.21.15.67:0/843763559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 msgr2=0x55f2b4507d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe32d200700 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 0x55f2b4507d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe32d200700 1 -- 172.21.15.67:0/843763559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3281a5ea0 con 0x55f2b44ff7a0 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe327e00700 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 0x55f2b4507d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:40.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe32d200700 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b450a990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00ba00 tx=0x7fe31c01a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce50269726edd0c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe30ea00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe30ea00700 1 -- 172.21.15.67:0/843763559 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c004990 con 0x55f2b44ff7a0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe30ea00700 1 -- 172.21.15.67:0/843763559 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fe31c004af0 con 0x55f2b44ff7a0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.386+0000 7fe343629b80 1 -- 172.21.15.67:0/843763559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2b4412380 con 0x55f2b44ff7a0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe30ea00700 1 -- 172.21.15.67:0/843763559 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe31c004cb0 con 0x55f2b44ff7a0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe343629b80 10 client.26372 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe343629b80 1 -- 172.21.15.67:0/843763559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fe314009a40 con 0x55f2b44ff7a0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe30ea00700 1 -- 172.21.15.67:0/843763559 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f2b4412380 con 0x55f2b44ff7a0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe30ea00700 1 -- 172.21.15.67:0/843763559 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe31c027430 con 0x55f2b44ff7a0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe343629b80 10 client.26372 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe343629b80 10 client.26372 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe343629b80 20 client.26372 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe343629b80 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2b451aba0 0x55f2b451cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe343629b80 4 client.26372 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.387+0000 7fe343629b80 1 -- 172.21.15.67:0/843763559 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2b444a700 con 0x55f2b451aba0 2024-03-30T02:39:40.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.388+0000 7fe327e00700 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2b451aba0 0x55f2b451cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.388+0000 7fe327e00700 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2b451aba0 0x55f2b451cfc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53562b052eeddde server_cookie=9d2b39758ce86adf in_seq=0 out_seq=0 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.388+0000 7fe30ea00700 10 client.26372 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe30ea00700 1 -- 172.21.15.67:0/843763559 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f2b444a700 con 0x55f2b451aba0 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe30ea00700 10 client.26372 handle_command_reply: tid=0 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 client.26372 shutdown 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 -- 172.21.15.67:0/843763559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 msgr2=0x55f2b450a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b450a990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00ba00 tx=0x7fe31c01a050 comp rx=0 tx=0).stop 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 -- 172.21.15.67:0/843763559 shutdown_connections 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2b451aba0 0x55f2b451cfc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 0x55f2b4508320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b450a990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe30ea00700 0 client.26372 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:39:40.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 --2- 172.21.15.67:0/843763559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 0x55f2b4507d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 -- 172.21.15.67:0/843763559 >> 172.21.15.67:0/843763559 conn(0x55f2b445f6a0 msgr2=0x55f2b4501890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 -- 172.21.15.67:0/843763559 shutdown_connections 2024-03-30T02:39:40.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.389+0000 7fe343629b80 1 -- 172.21.15.67:0/843763559 wait complete. 2024-03-30T02:39:40.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.390+0000 7fe343629b80 20 client.26372 trim_cache size 0 max 16384 2024-03-30T02:39:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.403+0000 7fe343629b80 1 Processor -- start 2024-03-30T02:39:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe343629b80 1 -- start start 2024-03-30T02:39:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe343629b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b451cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe343629b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 0x55f2b4502740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe343629b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 0x55f2b45065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe343629b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b4501240 con 0x55f2b45000c0 2024-03-30T02:39:40.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe343629b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b450d600 con 0x55f2b44ff7a0 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe343629b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b450d770 con 0x55f2b450afc0 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe32d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b451cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe32d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b451cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43486/0 (socket says 172.21.15.67:43486) 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe32d200700 1 -- 172.21.15.67:0/689322917 learned_addr learned my addr 172.21.15.67:0/689322917 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe32dc00700 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 0x55f2b45065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe327e00700 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 0x55f2b4502740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe32d200700 1 -- 172.21.15.67:0/689322917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 msgr2=0x55f2b45065c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.404+0000 7fe32d200700 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 0x55f2b45065c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.405+0000 7fe32d200700 1 -- 172.21.15.67:0/689322917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 msgr2=0x55f2b4502740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.405+0000 7fe32d200700 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 0x55f2b4502740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.405+0000 7fe32d200700 1 -- 172.21.15.67:0/689322917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2b427fb60 con 0x55f2b44ff7a0 2024-03-30T02:39:40.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.405+0000 7fe32dc00700 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 0x55f2b45065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.405+0000 7fe32d200700 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b451cf90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0277c0 tx=0x7fe31c027960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=545396b4a6494ac5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe31aa00700 1 -- 172.21.15.67:0/689322917 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c044070 con 0x55f2b44ff7a0 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe31aa00700 1 -- 172.21.15.67:0/689322917 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe31c01e040 con 0x55f2b44ff7a0 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe31aa00700 1 -- 172.21.15.67:0/689322917 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00fea0 con 0x55f2b44ff7a0 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe343629b80 1 -- 172.21.15.67:0/689322917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 msgr2=0x55f2b451cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe343629b80 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b451cf90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0277c0 tx=0x7fe31c027960 comp rx=0 tx=0).stop 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe343629b80 1 -- 172.21.15.67:0/689322917 shutdown_connections 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe343629b80 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b450afc0 0x55f2b45065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe343629b80 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b451cf90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe343629b80 1 --2- 172.21.15.67:0/689322917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b45000c0 0x55f2b4502740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe343629b80 1 -- 172.21.15.67:0/689322917 >> 172.21.15.67:0/689322917 conn(0x55f2b445f6a0 msgr2=0x55f2b45032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe343629b80 1 -- 172.21.15.67:0/689322917 shutdown_connections 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.406+0000 7fe343629b80 1 -- 172.21.15.67:0/689322917 wait complete. 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.407+0000 7fe343629b80 1 -- start start 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.407+0000 7fe343629b80 1 Processor -- start 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe343629b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b4511960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe343629b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b45000c0 0x55f2b450fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe343629b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b45101d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe343629b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b4510710 con 0x55f2b450afc0 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe343629b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b4510880 con 0x55f2b44ff7a0 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe343629b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b4510a40 con 0x55f2b45000c0 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe343629b80 10 client.0 _ll_register_callbacks cb 0x55f2b43f7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe32d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b4511960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe32d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b4511960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43492/0 (socket says 172.21.15.67:43492) 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe32d200700 1 -- 172.21.15.67:0/2186521763 learned_addr learned my addr 172.21.15.67:0/2186521763 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe327e00700 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b45000c0 0x55f2b450fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe32dc00700 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b45101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe32d200700 1 -- 172.21.15.67:0/2186521763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b45000c0 msgr2=0x55f2b450fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe32d200700 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b45000c0 0x55f2b450fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe32d200700 1 -- 172.21.15.67:0/2186521763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 msgr2=0x55f2b45101d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe32d200700 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b45101d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.408+0000 7fe32d200700 1 -- 172.21.15.67:0/2186521763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe314009a40 con 0x55f2b44ff7a0 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.409+0000 7fe32dc00700 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b45101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.409+0000 7fe32d200700 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b4511960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0278e0 tx=0x7fe31c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6062f495ffc52079 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.409+0000 7fe30ea00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.409+0000 7fe30ea00700 1 -- 172.21.15.67:0/2186521763 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c044040 con 0x55f2b44ff7a0 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.409+0000 7fe343629b80 1 -- 172.21.15.67:0/2186521763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe314004cf0 con 0x55f2b44ff7a0 2024-03-30T02:39:40.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.409+0000 7fe30ea00700 1 -- 172.21.15.67:0/2186521763 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fe31c004cb0 con 0x55f2b44ff7a0 2024-03-30T02:39:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.409+0000 7fe30ea00700 1 -- 172.21.15.67:0/2186521763 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe31c004eb0 con 0x55f2b44ff7a0 2024-03-30T02:39:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.409+0000 7fe343629b80 10 client.26378 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.409+0000 7fe343629b80 1 -- 172.21.15.67:0/2186521763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f2b4417ea0 con 0x55f2b44ff7a0 2024-03-30T02:39:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.410+0000 7fe30ea00700 1 -- 172.21.15.67:0/2186521763 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe314004cf0 con 0x55f2b44ff7a0 2024-03-30T02:39:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.410+0000 7fe30ea00700 1 -- 172.21.15.67:0/2186521763 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe31c038070 con 0x55f2b44ff7a0 2024-03-30T02:39:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.410+0000 7fe343629b80 10 client.26378 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.410+0000 7fe343629b80 10 client.26378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:39:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.410+0000 7fe343629b80 20 client.26378 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.410+0000 7fe343629b80 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2b456f920 0x55f2b4571d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.410+0000 7fe343629b80 4 client.26378 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:40.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.410+0000 7fe343629b80 1 -- 172.21.15.67:0/2186521763 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f2b42a0cf0 con 0x55f2b456f920 2024-03-30T02:39:40.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.410+0000 7fe327e00700 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2b456f920 0x55f2b4571d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe327e00700 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2b456f920 0x55f2b4571d40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67065bdf08db5d62 server_cookie=10068c9afbd847e5 in_seq=0 out_seq=0 2024-03-30T02:39:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe30ea00700 10 client.26378 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:39:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe30ea00700 1 -- 172.21.15.67:0/2186521763 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f2b42a0cf0 con 0x55f2b456f920 2024-03-30T02:39:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe30ea00700 10 client.26378 handle_command_reply: tid=0 2024-03-30T02:39:40.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe343629b80 1 client.26378 shutdown 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe343629b80 1 -- 172.21.15.67:0/2186521763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 msgr2=0x55f2b4511960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe343629b80 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b4511960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0278e0 tx=0x7fe31c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe343629b80 1 -- 172.21.15.67:0/2186521763 shutdown_connections 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe343629b80 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2b456f920 0x55f2b4571d40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe343629b80 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2b450afc0 0x55f2b45101d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe343629b80 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2b44ff7a0 0x55f2b4511960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe30ea00700 0 client.26378 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe343629b80 1 --2- 172.21.15.67:0/2186521763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2b45000c0 0x55f2b450fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.411+0000 7fe343629b80 1 -- 172.21.15.67:0/2186521763 >> 172.21.15.67:0/2186521763 conn(0x55f2b445f6a0 msgr2=0x55f2b4506580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.412+0000 7fe343629b80 1 -- 172.21.15.67:0/2186521763 shutdown_connections 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.412+0000 7fe343629b80 1 -- 172.21.15.67:0/2186521763 wait complete. 2024-03-30T02:39:40.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.412+0000 7fe343629b80 20 client.26378 trim_cache size 0 max 16384 2024-03-30T02:39:40.418 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:40.418 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:40 smithi067 ceph-mon[37808]: pgmap v81: 129 pgs: 129 active+clean; 333 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.7 MiB/s wr, 755 op/s 2024-03-30T02:39:40.418 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:40 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/134973019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.412+0000 7fe32e600700 1 -- 172.21.15.67:0/2759593366 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe3100a13e0 msgr2=0x7fe3100a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.412+0000 7fe32e600700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe3100a13e0 0x7fe3100a38a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe31c006010 tx=0x7fe31c00b260 comp rx=0 tx=0).stop 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.412+0000 7fe32e600700 1 -- 172.21.15.67:0/2759593366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32810b110 msgr2=0x7fe3281003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.412+0000 7fe32e600700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32810b110 0x7fe3281003c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe32000efc0 tx=0x7fe32000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.412+0000 7fe32e600700 1 -- 172.21.15.67:0/2759593366 shutdown_connections 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.413+0000 7fe32e600700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe3100a13e0 0x7fe3100a38a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.413+0000 7fe32e600700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32810b110 0x7fe3281003c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.413+0000 7fe32e600700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe328101cf0 0x7fe3280ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.413+0000 7fe32e600700 1 --2- 172.21.15.67:0/2759593366 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe328102610 0x7fe3280ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.413+0000 7fe32e600700 1 -- 172.21.15.67:0/2759593366 >> 172.21.15.67:0/2759593366 conn(0x7fe3280fb510 msgr2=0x7fe3280fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.413+0000 7fe32e600700 1 -- 172.21.15.67:0/2759593366 shutdown_connections 2024-03-30T02:39:40.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.413+0000 7fe32e600700 1 -- 172.21.15.67:0/2759593366 wait complete. 2024-03-30T02:39:40.433 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:39:40.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.656+0000 7f1c15e00700 1 -- 172.21.15.67:0/4014196441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10100f60 msgr2=0x7f1c10101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.656+0000 7f1c15e00700 1 --2- 172.21.15.67:0/4014196441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10100f60 0x7f1c10101340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009990 tx=0x7f1c00009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.656+0000 7f1c15e00700 1 -- 172.21.15.67:0/4014196441 shutdown_connections 2024-03-30T02:39:40.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.656+0000 7f1c15e00700 1 --2- 172.21.15.67:0/4014196441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c10106080 0x7f1c1010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.656+0000 7f1c15e00700 1 --2- 172.21.15.67:0/4014196441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10100f60 0x7f1c10101340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.656+0000 7f1c15e00700 1 --2- 172.21.15.67:0/4014196441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c10101910 0x7f1c10105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.656+0000 7f1c15e00700 1 -- 172.21.15.67:0/4014196441 >> 172.21.15.67:0/4014196441 conn(0x7f1c100fc7c0 msgr2=0x7f1c100febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.656+0000 7f1c15e00700 1 -- 172.21.15.67:0/4014196441 shutdown_connections 2024-03-30T02:39:40.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.656+0000 7f1c15e00700 1 -- 172.21.15.67:0/4014196441 wait complete. 2024-03-30T02:39:40.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.657+0000 7f1c15e00700 1 Processor -- start 2024-03-30T02:39:40.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c15e00700 1 -- start start 2024-03-30T02:39:40.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c15e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c10100f60 0x7f1c100730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c15e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10106080 0x7f1c100735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c15e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c10073b30 0x7f1c101a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c15e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101a59c0 con 0x7f1c10100f60 2024-03-30T02:39:40.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c15e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101a5b30 con 0x7f1c10106080 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c15e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101a5cf0 con 0x7f1c10073b30 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c0e000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10106080 0x7f1c100735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c0f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c10073b30 0x7f1c101a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c0ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c10100f60 0x7f1c100730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c0e000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10106080 0x7f1c100735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43514/0 (socket says 172.21.15.67:43514) 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c0e000700 1 -- 172.21.15.67:0/2337958637 learned_addr learned my addr 172.21.15.67:0/2337958637 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.658+0000 7f1c0f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c10073b30 0x7f1c101a5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:34678/0 (socket says 172.21.15.67:34678) 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c0e000700 1 -- 172.21.15.67:0/2337958637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c10073b30 msgr2=0x7f1c101a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c0e000700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c10073b30 0x7f1c101a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c0e000700 1 -- 172.21.15.67:0/2337958637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c10100f60 msgr2=0x7f1c100730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c0e000700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c10100f60 0x7f1c100730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c0e000700 1 -- 172.21.15.67:0/2337958637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c000095f0 con 0x7f1c10106080 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c0f400700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c10073b30 0x7f1c101a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c0e000700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10106080 0x7f1c100735f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00e970 tx=0x7f1bfc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c14a00700 1 -- 172.21.15.67:0/2337958637 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc009790 con 0x7f1c10106080 2024-03-30T02:39:40.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c14a00700 1 -- 172.21.15.67:0/2337958637 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bfc00c7e0 con 0x7f1c10106080 2024-03-30T02:39:40.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c14a00700 1 -- 172.21.15.67:0/2337958637 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0054e0 con 0x7f1c10106080 2024-03-30T02:39:40.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c15e00700 1 -- 172.21.15.67:0/2337958637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c101a5fd0 con 0x7f1c10106080 2024-03-30T02:39:40.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.659+0000 7f1c15e00700 1 -- 172.21.15.67:0/2337958637 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c101a63a0 con 0x7f1c10106080 2024-03-30T02:39:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.663+0000 7f1c14a00700 1 -- 172.21.15.67:0/2337958637 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1bfc00c950 con 0x7f1c10106080 2024-03-30T02:39:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.664+0000 7f1c2991eb80 1 Processor -- start 2024-03-30T02:39:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c2991eb80 1 -- start start 2024-03-30T02:39:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521e5d0 0x55bcf521e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521eef0 0x55bcf52298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf5229df0 0x55bcf522c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5177990 con 0x55bcf5229df0 2024-03-30T02:39:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5175350 con 0x55bcf521eef0 2024-03-30T02:39:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5170c40 con 0x55bcf521e5d0 2024-03-30T02:39:40.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521eef0 0x55bcf52298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c0f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521e5d0 0x55bcf521e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521eef0 0x55bcf52298b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43520/0 (socket says 172.21.15.67:43520) 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c0ea00700 1 -- 172.21.15.67:0/3352484347 learned_addr learned my addr 172.21.15.67:0/3352484347 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c0e000700 1 --2- 172.21.15.67:0/3352484347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf5229df0 0x55bcf522c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.665+0000 7f1c14a00700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1bf40a1310 0x7f1bf40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1c0ea00700 1 -- 172.21.15.67:0/3352484347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521e5d0 msgr2=0x55bcf521e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/3352484347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521e5d0 0x55bcf521e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1c0ea00700 1 -- 172.21.15.67:0/3352484347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf5229df0 msgr2=0x55bcf522c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/3352484347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf5229df0 0x55bcf522c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1c0ea00700 1 -- 172.21.15.67:0/3352484347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcf4ff23b0 con 0x55bcf521eef0 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1c14a00700 1 -- 172.21.15.67:0/2337958637 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1bfc014070 con 0x7f1c10106080 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1c0f400700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1bf40a1310 0x7f1bf40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/3352484347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521eef0 0x55bcf52298b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005ec0 tx=0x7f1c00005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70b708aabe59be4b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1bfbe00700 1 -- 172.21.15.67:0/3352484347 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0001d070 con 0x55bcf521eef0 2024-03-30T02:39:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1bfbe00700 1 -- 172.21.15.67:0/3352484347 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c0001e040 con 0x55bcf521eef0 2024-03-30T02:39:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.666+0000 7f1bfbe00700 1 -- 172.21.15.67:0/3352484347 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00004880 con 0x55bcf521eef0 2024-03-30T02:39:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3352484347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521eef0 msgr2=0x55bcf52298b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c0f400700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1bf40a1310 0x7f1bf40a37d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009110 tx=0x7f1c08004ad0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3352484347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521eef0 0x55bcf52298b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005ec0 tx=0x7f1c00005ef0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3352484347 shutdown_connections 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3352484347 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf5229df0 0x55bcf522c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3352484347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521e5d0 0x55bcf521e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3352484347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521eef0 0x55bcf52298b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3352484347 >> 172.21.15.67:0/3352484347 conn(0x55bcf517e6a0 msgr2=0x55bcf521aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3352484347 shutdown_connections 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.667+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3352484347 wait complete. 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.668+0000 7f1c2991eb80 1 -- start start 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.668+0000 7f1c2991eb80 1 Processor -- start 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf52297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 0x55bcf5226a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 0x55bcf5227040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf510cd80 con 0x55bcf521eef0 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf510ca20 con 0x55bcf521e5d0 2024-03-30T02:39:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf4ff46a0 con 0x55bcf5229df0 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c2991eb80 10 client.0 _ll_register_callbacks cb 0x55bcf5116470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf52297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf52297d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43528/0 (socket says 172.21.15.67:43528) 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0ea00700 1 -- 172.21.15.67:0/4036202647 learned_addr learned my addr 172.21.15.67:0/4036202647 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0e000700 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 0x55bcf5226a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0f400700 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 0x55bcf5227040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0ea00700 1 -- 172.21.15.67:0/4036202647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 msgr2=0x55bcf5227040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 0x55bcf5227040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0ea00700 1 -- 172.21.15.67:0/4036202647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 msgr2=0x55bcf5226a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 0x55bcf5226a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0ea00700 1 -- 172.21.15.67:0/4036202647 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1010e470 con 0x55bcf521e5d0 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.669+0000 7f1c0f400700 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 0x55bcf5227040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.670+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf52297d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000bf40 tx=0x7f1c0000ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e230ea876a2a9a2c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.670+0000 7f1bf8c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.670+0000 7f1bf8c00700 1 -- 172.21.15.67:0/4036202647 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0001d030 con 0x55bcf521e5d0 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.670+0000 7f1bf8c00700 1 -- 172.21.15.67:0/4036202647 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1c00004700 con 0x55bcf521e5d0 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.670+0000 7f1c2991eb80 1 -- 172.21.15.67:0/4036202647 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcf5131380 con 0x55bcf521e5d0 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.670+0000 7f1bf8c00700 1 -- 172.21.15.67:0/4036202647 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c00005bc0 con 0x55bcf521e5d0 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.670+0000 7f1c2991eb80 10 client.26390 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.670+0000 7f1c2991eb80 1 -- 172.21.15.67:0/4036202647 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bcf4f9eb60 con 0x55bcf521e5d0 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.670+0000 7f1bf8c00700 1 -- 172.21.15.67:0/4036202647 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bcf5131380 con 0x55bcf521e5d0 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1bf8c00700 1 -- 172.21.15.67:0/4036202647 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1c0000fa30 con 0x55bcf521e5d0 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1c2991eb80 10 client.26390 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1c2991eb80 10 client.26390 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1c2991eb80 20 client.26390 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55bcf5239420 0x55bcf523b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1c2991eb80 4 client.26390 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1c2991eb80 1 -- 172.21.15.67:0/4036202647 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcf5169700 con 0x55bcf5239420 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1c0e000700 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55bcf5239420 0x55bcf523b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1c0e000700 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55bcf5239420 0x55bcf523b810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=838c334644c5f6b6 server_cookie=7b456ae227c493aa in_seq=0 out_seq=0 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.671+0000 7f1bf8c00700 10 client.26390 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1bf8c00700 1 -- 172.21.15.67:0/4036202647 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bcf5169700 con 0x55bcf5239420 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1bf8c00700 10 client.26390 handle_command_reply: tid=0 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1c2991eb80 1 client.26390 shutdown 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1c2991eb80 1 -- 172.21.15.67:0/4036202647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 msgr2=0x55bcf52297d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf52297d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000bf40 tx=0x7f1c0000ba30 comp rx=0 tx=0).stop 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1c2991eb80 1 -- 172.21.15.67:0/4036202647 shutdown_connections 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 0x55bcf5227040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf52297d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55bcf5239420 0x55bcf523b810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/4036202647 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 0x55bcf5226a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1c2991eb80 1 -- 172.21.15.67:0/4036202647 >> 172.21.15.67:0/4036202647 conn(0x55bcf517e6a0 msgr2=0x55bcf52206c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.672+0000 7f1bf8c00700 0 client.26390 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.673+0000 7f1c2991eb80 1 -- 172.21.15.67:0/4036202647 shutdown_connections 2024-03-30T02:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.673+0000 7f1c2991eb80 1 -- 172.21.15.67:0/4036202647 wait complete. 2024-03-30T02:39:40.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.673+0000 7f1c2991eb80 20 client.26390 trim_cache size 0 max 16384 2024-03-30T02:39:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:40 smithi170 ceph-mon[78798]: pgmap v81: 129 pgs: 129 active+clean; 333 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.7 MiB/s wr, 755 op/s 2024-03-30T02:39:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:40 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/134973019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:39:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c2991eb80 1 Processor -- start 2024-03-30T02:39:40.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c2991eb80 1 -- start start 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf5227000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 0x55bcf5230a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 0x55bcf52135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5227640 con 0x55bcf521eef0 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5213ae0 con 0x55bcf521e5d0 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf5227fc0 con 0x55bcf5229df0 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c0e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 0x55bcf5230a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.691+0000 7f1c0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf5227000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 0x55bcf52135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf5227000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43542/0 (socket says 172.21.15.67:43542) 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 0x55bcf5230a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35570/0 (socket says 172.21.15.67:35570) 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0ea00700 1 -- 172.21.15.67:0/2714558914 learned_addr learned my addr 172.21.15.67:0/2714558914 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0ea00700 1 -- 172.21.15.67:0/2714558914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 msgr2=0x55bcf52135a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/2714558914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 0x55bcf52135a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0ea00700 1 -- 172.21.15.67:0/2714558914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 msgr2=0x55bcf5230a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/2714558914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 0x55bcf5230a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0ea00700 1 -- 172.21.15.67:0/2714558914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcf5136ea0 con 0x55bcf521e5d0 2024-03-30T02:39:40.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/2714558914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf5227000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098e0 tx=0x7f1c00000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14e0e93b2087432 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c05200700 1 -- 172.21.15.67:0/2714558914 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0001d040 con 0x55bcf521e5d0 2024-03-30T02:39:40.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c05200700 1 -- 172.21.15.67:0/2714558914 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c00005bc0 con 0x55bcf521e5d0 2024-03-30T02:39:40.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.692+0000 7f1c05200700 1 -- 172.21.15.67:0/2714558914 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0002ca90 con 0x55bcf521e5d0 2024-03-30T02:39:40.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.693+0000 7f1c2991eb80 1 -- 172.21.15.67:0/2714558914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 msgr2=0x55bcf5227000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.693+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/2714558914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf5227000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098e0 tx=0x7f1c00000bc0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.693+0000 7f1c2991eb80 1 -- 172.21.15.67:0/2714558914 shutdown_connections 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.693+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/2714558914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf5229df0 0x55bcf52135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.693+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/2714558914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf521e5d0 0x55bcf5227000 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.693+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/2714558914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521eef0 0x55bcf5230a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.693+0000 7f1c2991eb80 1 -- 172.21.15.67:0/2714558914 >> 172.21.15.67:0/2714558914 conn(0x55bcf517e6a0 msgr2=0x55bcf5240060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.693+0000 7f1c2991eb80 1 -- 172.21.15.67:0/2714558914 shutdown_connections 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.693+0000 7f1c2991eb80 1 -- 172.21.15.67:0/2714558914 wait complete. 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c2991eb80 1 -- start start 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c2991eb80 1 Processor -- start 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521e5d0 0x55bcf522e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521eef0 0x55bcf522ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c2991eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf5229df0 0x55bcf528c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf528c650 con 0x55bcf521e5d0 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf528c790 con 0x55bcf5229df0 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c2991eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcf528c8d0 con 0x55bcf521eef0 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c2991eb80 10 client.0 _ll_register_callbacks cb 0x55bcf5116470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c0ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521e5d0 0x55bcf522e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.694+0000 7f1c0e000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521eef0 0x55bcf522ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c0ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521e5d0 0x55bcf522e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35578/0 (socket says 172.21.15.67:35578) 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c0ea00700 1 -- 172.21.15.67:0/3834389861 learned_addr learned my addr 172.21.15.67:0/3834389861 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c0f400700 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf5229df0 0x55bcf528c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c0ea00700 1 -- 172.21.15.67:0/3834389861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521eef0 msgr2=0x55bcf522ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521eef0 0x55bcf522ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c0ea00700 1 -- 172.21.15.67:0/3834389861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf5229df0 msgr2=0x55bcf528c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf5229df0 0x55bcf528c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c0ea00700 1 -- 172.21.15.67:0/3834389861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcf4f9eb60 con 0x55bcf521e5d0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c0ea00700 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521e5d0 0x55bcf522e780 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000ba70 tx=0x7f1c0000fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d9031c9352d2c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1bf8c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1bf8c00700 1 -- 172.21.15.67:0/3834389861 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0001d040 con 0x55bcf521e5d0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1bf8c00700 1 -- 172.21.15.67:0/3834389861 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1c00000e40 con 0x55bcf521e5d0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.695+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3834389861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcf4fbfcf0 con 0x55bcf521e5d0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1bf8c00700 1 -- 172.21.15.67:0/3834389861 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c00005bc0 con 0x55bcf521e5d0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1c2991eb80 10 client.16332 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3834389861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55bcf4ff6cd0 con 0x55bcf521e5d0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1bf8c00700 1 -- 172.21.15.67:0/3834389861 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bcf4fbfcf0 con 0x55bcf521e5d0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1bf8c00700 1 -- 172.21.15.67:0/3834389861 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1c00004910 con 0x55bcf521e5d0 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1c2991eb80 10 client.16332 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1c2991eb80 10 client.16332 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:39:40.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1c2991eb80 20 client.16332 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55bcf528dbc0 0x55bcf528ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1c2991eb80 4 client.16332 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.696+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3834389861 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bcf4ff4ce0 con 0x55bcf528dbc0 2024-03-30T02:39:40.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.697+0000 7f1c0e000700 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55bcf528dbc0 0x55bcf528ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.697+0000 7f1c0e000700 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55bcf528dbc0 0x55bcf528ffb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a43c89e8f414ce02 server_cookie=8816bfe2fac7bcc in_seq=0 out_seq=0 2024-03-30T02:39:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.697+0000 7f1bf8c00700 10 client.16332 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:39:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.697+0000 7f1bf8c00700 1 -- 172.21.15.67:0/3834389861 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bcf4ff4ce0 con 0x55bcf528dbc0 2024-03-30T02:39:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.697+0000 7f1bf8c00700 10 client.16332 handle_command_reply: tid=0 2024-03-30T02:39:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.697+0000 7f1c2991eb80 1 client.16332 shutdown 2024-03-30T02:39:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3834389861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521e5d0 msgr2=0x55bcf522e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521e5d0 0x55bcf522e780 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000ba70 tx=0x7f1c0000fbc0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3834389861 shutdown_connections 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55bcf528dbc0 0x55bcf528ffb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55bcf521e5d0 0x55bcf522e780 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bcf5229df0 0x55bcf528c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 --2- 172.21.15.67:0/3834389861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcf521eef0 0x55bcf522ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1bf8c00700 0 client.16332 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3834389861 >> 172.21.15.67:0/3834389861 conn(0x55bcf517e6a0 msgr2=0x55bcf5240060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3834389861 shutdown_connections 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 1 -- 172.21.15.67:0/3834389861 wait complete. 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.698+0000 7f1c2991eb80 20 client.16332 trim_cache size 0 max 16384 2024-03-30T02:39:40.703 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 -- 172.21.15.67:0/2337958637 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1bf40a1310 msgr2=0x7f1bf40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1bf40a1310 0x7f1bf40a37d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009110 tx=0x7f1c08004ad0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 -- 172.21.15.67:0/2337958637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10106080 msgr2=0x7f1c100735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10106080 0x7f1c100735f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00e970 tx=0x7f1bfc00ec80 comp rx=0 tx=0).stop 2024-03-30T02:39:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 -- 172.21.15.67:0/2337958637 shutdown_connections 2024-03-30T02:39:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1bf40a1310 0x7f1bf40a37d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1c10100f60 0x7f1c100730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c10073b30 0x7f1c101a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 --2- 172.21.15.67:0/2337958637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c10106080 0x7f1c100735f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 -- 172.21.15.67:0/2337958637 >> 172.21.15.67:0/2337958637 conn(0x7f1c100fc7c0 msgr2=0x7f1c100fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 -- 172.21.15.67:0/2337958637 shutdown_connections 2024-03-30T02:39:40.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.699+0000 7f1c15e00700 1 -- 172.21.15.67:0/2337958637 wait complete. 2024-03-30T02:39:40.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:39:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:40 smithi082 ceph-mon[79176]: pgmap v81: 129 pgs: 129 active+clean; 333 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.7 MiB/s wr, 755 op/s 2024-03-30T02:39:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:40 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/134973019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:39:40.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.933+0000 7fcf7b800700 1 -- 172.21.15.67:0/912840149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74106070 msgr2=0x7fcf7410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.933+0000 7fcf7b800700 1 --2- 172.21.15.67:0/912840149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74106070 0x7fcf7410dbf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00b0a0 tx=0x7fcf6c00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.933+0000 7fcf7b800700 1 -- 172.21.15.67:0/912840149 shutdown_connections 2024-03-30T02:39:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.933+0000 7fcf7b800700 1 --2- 172.21.15.67:0/912840149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf74100f50 0x7fcf74101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.933+0000 7fcf7b800700 1 --2- 172.21.15.67:0/912840149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74106070 0x7fcf7410dbf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.933+0000 7fcf7b800700 1 --2- 172.21.15.67:0/912840149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf74101900 0x7fcf74105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.933+0000 7fcf7b800700 1 -- 172.21.15.67:0/912840149 >> 172.21.15.67:0/912840149 conn(0x7fcf740fc7f0 msgr2=0x7fcf740fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.933+0000 7fcf7b800700 1 -- 172.21.15.67:0/912840149 shutdown_connections 2024-03-30T02:39:40.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.933+0000 7fcf7b800700 1 -- 172.21.15.67:0/912840149 wait complete. 2024-03-30T02:39:40.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf7b800700 1 Processor -- start 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf7b800700 1 -- start start 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf7b800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74100f50 0x7fcf741992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf7b800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf74106070 0x7fcf74199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf7b800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7419dbc0 0x7fcf7419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf7b800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7419e640 con 0x7fcf74106070 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf7b800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7419e7b0 con 0x7fcf74100f50 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf7b800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7419e970 con 0x7fcf7419dbc0 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf78e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74100f50 0x7fcf741992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf78e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74100f50 0x7fcf741992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43574/0 (socket says 172.21.15.67:43574) 2024-03-30T02:39:40.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.935+0000 7fcf78e00700 1 -- 172.21.15.67:0/1260009822 learned_addr learned my addr 172.21.15.67:0/1260009822 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf79800700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7419dbc0 0x7fcf7419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf73e00700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf74106070 0x7fcf74199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf78e00700 1 -- 172.21.15.67:0/1260009822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7419dbc0 msgr2=0x7fcf7419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf78e00700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7419dbc0 0x7fcf7419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf78e00700 1 -- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf74106070 msgr2=0x7fcf74199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf78e00700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf74106070 0x7fcf74199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf78e00700 1 -- 172.21.15.67:0/1260009822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009d00 con 0x7fcf74100f50 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf73e00700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf74106070 0x7fcf74199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf78e00700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74100f50 0x7fcf741992e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800e9a0 tx=0x7fcf6800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf71600700 1 -- 172.21.15.67:0/1260009822 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68009790 con 0x7fcf74100f50 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf71600700 1 -- 172.21.15.67:0/1260009822 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf6800cbd0 con 0x7fcf74100f50 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf7b800700 1 -- 172.21.15.67:0/1260009822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf741a5ed0 con 0x7fcf74100f50 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.936+0000 7fcf71600700 1 -- 172.21.15.67:0/1260009822 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68018680 con 0x7fcf74100f50 2024-03-30T02:39:40.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.937+0000 7fcf7b800700 1 -- 172.21.15.67:0/1260009822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf741a6350 con 0x7fcf74100f50 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.940+0000 7fcf71600700 1 -- 172.21.15.67:0/1260009822 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fcf6800cd40 con 0x7fcf74100f50 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.941+0000 7fcf8f237b80 1 Processor -- start 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.941+0000 7fcf71600700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcf5c0a1290 0x7fcf5c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.941+0000 7fcf8f237b80 1 -- start start 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.941+0000 7fcf71600700 1 -- 172.21.15.67:0/1260009822 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fcf68014070 con 0x7fcf74100f50 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.941+0000 7fcf79800700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcf5c0a1290 0x7fcf5c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf45d0 0x55d133cf49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133cff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cffdf0 0x55d133d021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133c4d990 con 0x55d133cffdf0 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133c4b350 con 0x55d133cf4ef0 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133c46c40 con 0x55d133cf45d0 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf73e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133cff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf78e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf45d0 0x55d133cf49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf73e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133cff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43588/0 (socket says 172.21.15.67:43588) 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf78e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf45d0 0x55d133cf49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:34750/0 (socket says 172.21.15.67:34750) 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf73e00700 1 -- 172.21.15.67:0/1317228752 learned_addr learned my addr 172.21.15.67:0/1317228752 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf79800700 1 --2- 172.21.15.67:0/1317228752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cffdf0 0x55d133d021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf78e00700 1 -- 172.21.15.67:0/1317228752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 msgr2=0x55d133cff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf78e00700 1 --2- 172.21.15.67:0/1317228752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133cff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf78e00700 1 -- 172.21.15.67:0/1317228752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cffdf0 msgr2=0x55d133d021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf78e00700 1 --2- 172.21.15.67:0/1317228752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cffdf0 0x55d133d021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf79800700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcf5c0a1290 0x7fcf5c0a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009610 tx=0x7fcf6c006230 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.942+0000 7fcf78e00700 1 -- 172.21.15.67:0/1317228752 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d133ac83b0 con 0x55d133cf45d0 2024-03-30T02:39:40.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.943+0000 7fcf78e00700 1 --2- 172.21.15.67:0/1317228752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf45d0 0x55d133cf49b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf680ca690 tx=0x7fcf680999b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54dbea851095ba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.943+0000 7fcf66000700 1 -- 172.21.15.67:0/1317228752 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6809a050 con 0x55d133cf45d0 2024-03-30T02:39:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.943+0000 7fcf66000700 1 -- 172.21.15.67:0/1317228752 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf6809de80 con 0x55d133cf45d0 2024-03-30T02:39:40.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.943+0000 7fcf66000700 1 -- 172.21.15.67:0/1317228752 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680997f0 con 0x55d133cf45d0 2024-03-30T02:39:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1317228752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf45d0 msgr2=0x55d133cf49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1317228752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf45d0 0x55d133cf49b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf680ca690 tx=0x7fcf680999b0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1317228752 shutdown_connections 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1317228752 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cffdf0 0x55d133d021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1317228752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf45d0 0x55d133cf49b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1317228752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133cff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1317228752 >> 172.21.15.67:0/1317228752 conn(0x55d133c546a0 msgr2=0x55d133cf0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1317228752 shutdown_connections 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1317228752 wait complete. 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.944+0000 7fcf8f237b80 1 -- start start 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf8f237b80 1 Processor -- start 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf45d0 0x55d133cff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf4ef0 0x55d133cfcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133cfd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133be2d80 con 0x55d133cf4ef0 2024-03-30T02:39:40.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133be2a20 con 0x55d133cf45d0 2024-03-30T02:39:40.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133aca6a0 con 0x55d133cffdf0 2024-03-30T02:39:40.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf8f237b80 10 client.0 _ll_register_callbacks cb 0x55d133bec470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:40.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf78e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf45d0 0x55d133cff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.945+0000 7fcf73e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf4ef0 0x55d133cfcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf78e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf45d0 0x55d133cff850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43590/0 (socket says 172.21.15.67:43590) 2024-03-30T02:39:40.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf78e00700 1 -- 172.21.15.67:0/2968852680 learned_addr learned my addr 172.21.15.67:0/2968852680 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf79800700 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133cfd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf78e00700 1 -- 172.21.15.67:0/2968852680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 msgr2=0x55d133cfd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf78e00700 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133cfd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf78e00700 1 -- 172.21.15.67:0/2968852680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf4ef0 msgr2=0x55d133cfcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf78e00700 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf4ef0 0x55d133cfcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf78e00700 1 -- 172.21.15.67:0/2968852680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf741a6350 con 0x55d133cf45d0 2024-03-30T02:39:40.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf78e00700 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf45d0 0x55d133cff850 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c5b0 tx=0x7fcf680997d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34bb0e0af67de9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf5aa00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:40.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf5aa00700 1 -- 172.21.15.67:0/2968852680 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf680b6070 con 0x55d133cf45d0 2024-03-30T02:39:40.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf5aa00700 1 -- 172.21.15.67:0/2968852680 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fcf6809a050 con 0x55d133cf45d0 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.946+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2968852680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d133c07380 con 0x55d133cf45d0 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf5aa00700 1 -- 172.21.15.67:0/2968852680 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf6807a020 con 0x55d133cf45d0 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf8f237b80 10 client.26408 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2968852680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d133a74b60 con 0x55d133cf45d0 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf5aa00700 1 -- 172.21.15.67:0/2968852680 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d133c07380 con 0x55d133cf45d0 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf5aa00700 1 -- 172.21.15.67:0/2968852680 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcf68098fb0 con 0x55d133cf45d0 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf8f237b80 10 client.26408 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf8f237b80 10 client.26408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf8f237b80 20 client.26408 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d133d0f550 0x55d133d11970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf8f237b80 4 client.26408 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:40.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2968852680 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d133c3f700 con 0x55d133d0f550 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.947+0000 7fcf73e00700 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d133d0f550 0x55d133d11970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.948+0000 7fcf73e00700 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d133d0f550 0x55d133d11970 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845393ef04de635d server_cookie=8bbd3d55bb18d618 in_seq=0 out_seq=0 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.948+0000 7fcf5aa00700 10 client.26408 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.948+0000 7fcf5aa00700 1 -- 172.21.15.67:0/2968852680 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d133c3f700 con 0x55d133d0f550 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.948+0000 7fcf5aa00700 10 client.26408 handle_command_reply: tid=0 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 client.26408 shutdown 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2968852680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf45d0 msgr2=0x55d133cff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf45d0 0x55d133cff850 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c5b0 tx=0x7fcf680997d0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2968852680 shutdown_connections 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133cfd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf45d0 0x55d133cff850 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d133d0f550 0x55d133d11970 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2968852680 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf4ef0 0x55d133cfcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2968852680 >> 172.21.15.67:0/2968852680 conn(0x55d133c546a0 msgr2=0x55d133cf66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf5aa00700 0 client.26408 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:40.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2968852680 shutdown_connections 2024-03-30T02:39:40.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2968852680 wait complete. 2024-03-30T02:39:40.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.949+0000 7fcf8f237b80 20 client.26408 trim_cache size 0 max 16384 2024-03-30T02:39:40.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.964+0000 7fcf8f237b80 1 Processor -- start 2024-03-30T02:39:40.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.964+0000 7fcf8f237b80 1 -- start start 2024-03-30T02:39:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.964+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 0x55d133cfd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf4ef0 0x55d133cff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cffdf0 0x55d133cf7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133cfe110 con 0x55d133cf45d0 2024-03-30T02:39:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133cf0fa0 con 0x55d133cffdf0 2024-03-30T02:39:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133cfb120 con 0x55d133cf4ef0 2024-03-30T02:39:40.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf78e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 0x55d133cfd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf79800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cffdf0 0x55d133cf7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf79800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cffdf0 0x55d133cf7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43592/0 (socket says 172.21.15.67:43592) 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf79800700 1 -- 172.21.15.67:0/2812725208 learned_addr learned my addr 172.21.15.67:0/2812725208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf73e00700 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf4ef0 0x55d133cff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf79800700 1 -- 172.21.15.67:0/2812725208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf4ef0 msgr2=0x55d133cff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf79800700 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf4ef0 0x55d133cff380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf79800700 1 -- 172.21.15.67:0/2812725208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 msgr2=0x55d133cfd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf79800700 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 0x55d133cfd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf79800700 1 -- 172.21.15.67:0/2812725208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d133c0cea0 con 0x55d133cffdf0 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf78e00700 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 0x55d133cfd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.965+0000 7fcf73e00700 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf4ef0 0x55d133cff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:40.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.966+0000 7fcf79800700 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cffdf0 0x55d133cf7ef0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00b0a0 tx=0x7fcf6c01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e9f6954ed28f81 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.966+0000 7fcf66a00700 1 -- 172.21.15.67:0/2812725208 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00e040 con 0x55d133cffdf0 2024-03-30T02:39:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.966+0000 7fcf66a00700 1 -- 172.21.15.67:0/2812725208 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf6c00be80 con 0x55d133cffdf0 2024-03-30T02:39:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.966+0000 7fcf66a00700 1 -- 172.21.15.67:0/2812725208 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c005a70 con 0x55d133cffdf0 2024-03-30T02:39:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2812725208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cffdf0 msgr2=0x55d133cf7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cffdf0 0x55d133cf7ef0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00b0a0 tx=0x7fcf6c01c000 comp rx=0 tx=0).stop 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2812725208 shutdown_connections 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 0x55d133cfd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cffdf0 0x55d133cf7ef0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/2812725208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cf4ef0 0x55d133cff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2812725208 >> 172.21.15.67:0/2812725208 conn(0x55d133c546a0 msgr2=0x55d133d0b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2812725208 shutdown_connections 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 -- 172.21.15.67:0/2812725208 wait complete. 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.967+0000 7fcf8f237b80 1 -- start start 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf8f237b80 1 Processor -- start 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 0x55d133d04be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133d05120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf8f237b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133d62380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133d05830 con 0x55d133cf45d0 2024-03-30T02:39:40.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133d628c0 con 0x55d133cf4ef0 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf8f237b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d133d62a30 con 0x55d133cffdf0 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf8f237b80 10 client.0 _ll_register_callbacks cb 0x55d133bec470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf73e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133d05120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf79800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133d62380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf78e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 0x55d133d04be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf73e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133d05120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43606/0 (socket says 172.21.15.67:43606) 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf73e00700 1 -- 172.21.15.67:0/1509363220 learned_addr learned my addr 172.21.15.67:0/1509363220 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.968+0000 7fcf79800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133d62380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:34774/0 (socket says 172.21.15.67:34774) 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf73e00700 1 -- 172.21.15.67:0/1509363220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 msgr2=0x55d133d62380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf73e00700 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133d62380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf73e00700 1 -- 172.21.15.67:0/1509363220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 msgr2=0x55d133d04be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf73e00700 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 0x55d133d04be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf73e00700 1 -- 172.21.15.67:0/1509363220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d133a74b60 con 0x55d133cf4ef0 2024-03-30T02:39:40.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf79800700 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133d62380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf73e00700 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133d05120 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d133d12280 tx=0x7fcf6001f320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba1c4ce04787e67d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf5aa00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf5aa00700 1 -- 172.21.15.67:0/1509363220 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60016070 con 0x55d133cf4ef0 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1509363220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d133a95cf0 con 0x55d133cf4ef0 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.969+0000 7fcf5aa00700 1 -- 172.21.15.67:0/1509363220 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fcf60009ce0 con 0x55d133cf4ef0 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf5aa00700 1 -- 172.21.15.67:0/1509363220 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf60009ea0 con 0x55d133cf4ef0 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf8f237b80 10 client.26414 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1509363220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fcf60004340 con 0x55d133cf4ef0 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf5aa00700 1 -- 172.21.15.67:0/1509363220 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d133a95cf0 con 0x55d133cf4ef0 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf5aa00700 1 -- 172.21.15.67:0/1509363220 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcf600286d0 con 0x55d133cf4ef0 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf8f237b80 10 client.26414 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:40.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf8f237b80 10 client.26414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:40.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf8f237b80 20 client.26414 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:40.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d133d64330 0x55d133d66750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:40.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf8f237b80 4 client.26414 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:39:40.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.970+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1509363220 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d133acace0 con 0x55d133d64330 2024-03-30T02:39:40.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.971+0000 7fcf78e00700 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d133d64330 0x55d133d66750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:40.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.971+0000 7fcf78e00700 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d133d64330 0x55d133d66750 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76da22d3fae3147 server_cookie=7cd63fb1b88847c0 in_seq=0 out_seq=0 2024-03-30T02:39:40.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.971+0000 7fcf5aa00700 10 client.26414 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:40.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf5aa00700 1 -- 172.21.15.67:0/1509363220 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d133acace0 con 0x55d133d64330 2024-03-30T02:39:40.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf5aa00700 10 client.26414 handle_command_reply: tid=0 2024-03-30T02:39:40.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 client.26414 shutdown 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1509363220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 msgr2=0x55d133d05120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133d05120 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d133d12280 tx=0x7fcf6001f320 comp rx=0 tx=0).stop 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1509363220 shutdown_connections 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55d133d64330 0x55d133d66750 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d133cf45d0 0x55d133d04be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d133cffdf0 0x55d133d62380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf5aa00700 0 client.26414 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 --2- 172.21.15.67:0/1509363220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d133cf4ef0 0x55d133d05120 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1509363220 >> 172.21.15.67:0/1509363220 conn(0x55d133c546a0 msgr2=0x55d133cf7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1509363220 shutdown_connections 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.972+0000 7fcf8f237b80 1 -- 172.21.15.67:0/1509363220 wait complete. 2024-03-30T02:39:40.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf8f237b80 20 client.26414 trim_cache size 0 max 16384 2024-03-30T02:39:40.978 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:39:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 -- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcf5c0a1290 msgr2=0x7fcf5c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcf5c0a1290 0x7fcf5c0a3750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009610 tx=0x7fcf6c006230 comp rx=0 tx=0).stop 2024-03-30T02:39:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 -- 172.21.15.67:0/1260009822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74100f50 msgr2=0x7fcf741992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74100f50 0x7fcf741992e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800e9a0 tx=0x7fcf6800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 -- 172.21.15.67:0/1260009822 shutdown_connections 2024-03-30T02:39:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcf5c0a1290 0x7fcf5c0a3750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf7419dbc0 0x7fcf7419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf74100f50 0x7fcf741992e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 --2- 172.21.15.67:0/1260009822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf74106070 0x7fcf74199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 -- 172.21.15.67:0/1260009822 >> 172.21.15.67:0/1260009822 conn(0x7fcf740fc7f0 msgr2=0x7fcf740feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:40.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.973+0000 7fcf7b800700 1 -- 172.21.15.67:0/1260009822 shutdown_connections 2024-03-30T02:39:40.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:40.974+0000 7fcf7b800700 1 -- 172.21.15.67:0/1260009822 wait complete. 2024-03-30T02:39:41.994 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:39:41.995 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:39:42.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:42 smithi067 ceph-mon[37808]: pgmap v82: 129 pgs: 129 active+clean; 333 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 506 op/s 2024-03-30T02:39:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.405+0000 7fddf0600700 1 -- 172.21.15.67:0/3007123179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde8106d80 msgr2=0x7fdde80ff1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.405+0000 7fddf0600700 1 --2- 172.21.15.67:0/3007123179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde8106d80 0x7fdde80ff1d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009990 tx=0x7fdddc009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.406+0000 7fddf0600700 1 -- 172.21.15.67:0/3007123179 shutdown_connections 2024-03-30T02:39:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.406+0000 7fddf0600700 1 --2- 172.21.15.67:0/3007123179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde8111ae0 0x7fdde8113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.406+0000 7fddf0600700 1 --2- 172.21.15.67:0/3007123179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde8106d80 0x7fdde80ff1d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.406+0000 7fddf0600700 1 --2- 172.21.15.67:0/3007123179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde80ff7a0 0x7fdde80ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.406+0000 7fddf0600700 1 -- 172.21.15.67:0/3007123179 >> 172.21.15.67:0/3007123179 conn(0x7fdde8074b20 msgr2=0x7fdde8074f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.407+0000 7fddf0600700 1 -- 172.21.15.67:0/3007123179 shutdown_connections 2024-03-30T02:39:42.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.407+0000 7fddf0600700 1 -- 172.21.15.67:0/3007123179 wait complete. 2024-03-30T02:39:42.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.409+0000 7fddf0600700 1 Processor -- start 2024-03-30T02:39:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.409+0000 7fddf0600700 1 -- start start 2024-03-30T02:39:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.410+0000 7fddf0600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde80ff7a0 0x7fdde80723c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.410+0000 7fddf0600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde8106d80 0x7fdde8072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.411+0000 7fddf0600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde8111ae0 0x7fdde806b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.411+0000 7fddf0600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde806bd80 con 0x7fdde8111ae0 2024-03-30T02:39:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.411+0000 7fddf0600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde806bef0 con 0x7fdde8106d80 2024-03-30T02:39:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.411+0000 7fddf0600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde806c0b0 con 0x7fdde80ff7a0 2024-03-30T02:39:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.411+0000 7fddedc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde80ff7a0 0x7fdde80723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.411+0000 7fddedc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde80ff7a0 0x7fdde80723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:34786/0 (socket says 172.21.15.67:34786) 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.411+0000 7fddedc00700 1 -- 172.21.15.67:0/2704133583 learned_addr learned my addr 172.21.15.67:0/2704133583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.412+0000 7fddee600700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde8111ae0 0x7fdde806b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.412+0000 7fdded200700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde8106d80 0x7fdde8072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.412+0000 7fddedc00700 1 -- 172.21.15.67:0/2704133583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde8106d80 msgr2=0x7fdde8072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.412+0000 7fddedc00700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde8106d80 0x7fdde8072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.412+0000 7fddedc00700 1 -- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde8111ae0 msgr2=0x7fdde806b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.412+0000 7fddedc00700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde8111ae0 0x7fdde806b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.412+0000 7fddedc00700 1 -- 172.21.15.67:0/2704133583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0095f0 con 0x7fdde80ff7a0 2024-03-30T02:39:42.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.412+0000 7fddee600700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde8111ae0 0x7fdde806b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:42.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.413+0000 7fddedc00700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde80ff7a0 0x7fdde80723c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdddc000c00 tx=0x7fdddc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.415+0000 7fdde6000700 1 -- 172.21.15.67:0/2704133583 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc01d070 con 0x7fdde80ff7a0 2024-03-30T02:39:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.415+0000 7fdde6000700 1 -- 172.21.15.67:0/2704133583 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdddc022470 con 0x7fdde80ff7a0 2024-03-30T02:39:42.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.415+0000 7fdde6000700 1 -- 172.21.15.67:0/2704133583 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc017650 con 0x7fdde80ff7a0 2024-03-30T02:39:42.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.415+0000 7fddf0600700 1 -- 172.21.15.67:0/2704133583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde806c330 con 0x7fdde80ff7a0 2024-03-30T02:39:42.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.415+0000 7fddf0600700 1 -- 172.21.15.67:0/2704133583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde8070190 con 0x7fdde80ff7a0 2024-03-30T02:39:42.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.420+0000 7fdde6000700 1 -- 172.21.15.67:0/2704133583 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdddc0225e0 con 0x7fdde80ff7a0 2024-03-30T02:39:42.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.423+0000 7fdde6000700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fddd00a13e0 0x7fddd00a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.424+0000 7fdde6000700 1 -- 172.21.15.67:0/2704133583 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fdddc0ce3a0 con 0x7fdde80ff7a0 2024-03-30T02:39:42.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.425+0000 7fdded200700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fddd00a13e0 0x7fddd00a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.426+0000 7fdded200700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fddd00a13e0 0x7fddd00a38a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005950 tx=0x7fddd40058e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.426+0000 7fde040a5b80 1 Processor -- start 2024-03-30T02:39:42.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.427+0000 7fde040a5b80 1 -- start start 2024-03-30T02:39:42.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.428+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429650 0x560965429a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.428+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 0x560965434930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.428+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965434e70 0x560965437260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.428+0000 7fde040a5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560965382a10 con 0x560965434e70 2024-03-30T02:39:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.428+0000 7fde040a5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609653803d0 con 0x560965429650 2024-03-30T02:39:42.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.428+0000 7fde040a5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096537bcc0 con 0x560965429f70 2024-03-30T02:39:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.428+0000 7fddedc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429650 0x560965429a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.428+0000 7fddedc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429650 0x560965429a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43638/0 (socket says 172.21.15.67:43638) 2024-03-30T02:39:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.428+0000 7fddedc00700 1 -- 172.21.15.67:0/2593641558 learned_addr learned my addr 172.21.15.67:0/2593641558 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.429+0000 7fddee600700 1 --2- 172.21.15.67:0/2593641558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965434e70 0x560965437260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.430+0000 7fdded200700 1 --2- 172.21.15.67:0/2593641558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 0x560965434930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.430+0000 7fddee600700 1 -- 172.21.15.67:0/2593641558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 msgr2=0x560965434930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.430+0000 7fddee600700 1 --2- 172.21.15.67:0/2593641558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 0x560965434930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.430+0000 7fddee600700 1 -- 172.21.15.67:0/2593641558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429650 msgr2=0x560965429a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.430+0000 7fddee600700 1 --2- 172.21.15.67:0/2593641558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429650 0x560965429a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.430+0000 7fddee600700 1 -- 172.21.15.67:0/2593641558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609651fd430 con 0x560965434e70 2024-03-30T02:39:42.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.431+0000 7fddee600700 1 --2- 172.21.15.67:0/2593641558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965434e70 0x560965437260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdde0007650 tx=0x7fdde0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9616eb131d24347 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.432+0000 7fdddaa00700 1 -- 172.21.15.67:0/2593641558 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde000e070 con 0x560965434e70 2024-03-30T02:39:42.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.432+0000 7fdddaa00700 1 -- 172.21.15.67:0/2593641558 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde000f040 con 0x560965434e70 2024-03-30T02:39:42.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.432+0000 7fdddaa00700 1 -- 172.21.15.67:0/2593641558 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0005020 con 0x560965434e70 2024-03-30T02:39:42.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.433+0000 7fde040a5b80 1 -- 172.21.15.67:0/2593641558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965434e70 msgr2=0x560965437260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.433+0000 7fde040a5b80 1 --2- 172.21.15.67:0/2593641558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965434e70 0x560965437260 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdde0007650 tx=0x7fdde0009920 comp rx=0 tx=0).stop 2024-03-30T02:39:42.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.434+0000 7fde040a5b80 1 -- 172.21.15.67:0/2593641558 shutdown_connections 2024-03-30T02:39:42.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.434+0000 7fde040a5b80 1 --2- 172.21.15.67:0/2593641558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965434e70 0x560965437260 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.434+0000 7fde040a5b80 1 --2- 172.21.15.67:0/2593641558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429650 0x560965429a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.434+0000 7fde040a5b80 1 --2- 172.21.15.67:0/2593641558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 0x560965434930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.434+0000 7fde040a5b80 1 -- 172.21.15.67:0/2593641558 >> 172.21.15.67:0/2593641558 conn(0x560965389720 msgr2=0x560965425b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.434+0000 7fde040a5b80 1 -- 172.21.15.67:0/2593641558 shutdown_connections 2024-03-30T02:39:42.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.434+0000 7fde040a5b80 1 -- 172.21.15.67:0/2593641558 wait complete. 2024-03-30T02:39:42.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.435+0000 7fde040a5b80 1 -- start start 2024-03-30T02:39:42.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.435+0000 7fde040a5b80 1 Processor -- start 2024-03-30T02:39:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.436+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 0x5609654348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.436+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 0x560965431c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.436+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x560965432370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.436+0000 7fde040a5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560965317e00 con 0x560965429650 2024-03-30T02:39:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.436+0000 7fde040a5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560965317aa0 con 0x560965434e70 2024-03-30T02:39:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.436+0000 7fde040a5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609651ff720 con 0x560965429f70 2024-03-30T02:39:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.436+0000 7fdded200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 0x560965431c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.437+0000 7fddee600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x560965432370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.437+0000 7fde040a5b80 10 client.0 _ll_register_callbacks cb 0x5609653214f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.437+0000 7fdded200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 0x560965431c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:34808/0 (socket says 172.21.15.67:34808) 2024-03-30T02:39:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.437+0000 7fdded200700 1 -- 172.21.15.67:0/1788026789 learned_addr learned my addr 172.21.15.67:0/1788026789 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.437+0000 7fddee600700 1 -- 172.21.15.67:0/1788026789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 msgr2=0x560965431c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.437+0000 7fddee600700 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 0x560965431c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.437+0000 7fddee600700 1 -- 172.21.15.67:0/1788026789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 msgr2=0x5609654348f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:39:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.438+0000 7fddedc00700 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 0x5609654348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.438+0000 7fddee600700 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 0x5609654348f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.438+0000 7fddee600700 1 -- 172.21.15.67:0/1788026789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde0009050 con 0x560965434e70 2024-03-30T02:39:42.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.438+0000 7fddee600700 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x560965432370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdde00094c0 tx=0x7fdde0015670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca45dbe4b7badd85 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.439+0000 7fddcf400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:39:42.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.441+0000 7fddcf400700 1 -- 172.21.15.67:0/1788026789 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde000e050 con 0x560965434e70 2024-03-30T02:39:42.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.441+0000 7fddcf400700 1 -- 172.21.15.67:0/1788026789 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fdde0003a00 con 0x560965434e70 2024-03-30T02:39:42.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.441+0000 7fde040a5b80 1 -- 172.21.15.67:0/1788026789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56096533c400 con 0x560965434e70 2024-03-30T02:39:42.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.442+0000 7fddcf400700 1 -- 172.21.15.67:0/1788026789 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdde00095e0 con 0x560965434e70 2024-03-30T02:39:42.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.442+0000 7fde040a5b80 10 client.26426 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:42.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.442+0000 7fde040a5b80 1 -- 172.21.15.67:0/1788026789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5609651a9be0 con 0x560965434e70 2024-03-30T02:39:42.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.442+0000 7fddcf400700 1 -- 172.21.15.67:0/1788026789 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56096533c400 con 0x560965434e70 2024-03-30T02:39:42.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.443+0000 7fddcf400700 1 -- 172.21.15.67:0/1788026789 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdde001e880 con 0x560965434e70 2024-03-30T02:39:42.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.443+0000 7fde040a5b80 10 client.26426 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:42.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.443+0000 7fde040a5b80 10 client.26426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:42.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.443+0000 7fde040a5b80 20 client.26426 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:42.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.443+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609654449a0 0x560965446dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.443+0000 7fde040a5b80 4 client.26426 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:42.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.443+0000 7fde040a5b80 1 -- 172.21.15.67:0/1788026789 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560965374780 con 0x5609654449a0 2024-03-30T02:39:42.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.443+0000 7fddedc00700 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609654449a0 0x560965446dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.444+0000 7fddedc00700 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609654449a0 0x560965446dc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ccd20de6b1f13aa server_cookie=dcc498a46af3bffe in_seq=0 out_seq=0 2024-03-30T02:39:42.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.444+0000 7fddcf400700 10 client.26426 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:42.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.445+0000 7fddcf400700 1 -- 172.21.15.67:0/1788026789 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560965374780 con 0x5609654449a0 2024-03-30T02:39:42.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.445+0000 7fddcf400700 10 client.26426 handle_command_reply: tid=0 2024-03-30T02:39:42.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fde040a5b80 1 client.26426 shutdown 2024-03-30T02:39:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fde040a5b80 1 -- 172.21.15.67:0/1788026789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 msgr2=0x560965432370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x560965432370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdde00094c0 tx=0x7fdde0015670 comp rx=0 tx=0).stop 2024-03-30T02:39:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fde040a5b80 1 -- 172.21.15.67:0/1788026789 shutdown_connections 2024-03-30T02:39:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609654449a0 0x560965446dc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 0x5609654348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x560965432370 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1788026789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429f70 0x560965431c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fddcf400700 0 client.26426 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.446+0000 7fde040a5b80 1 -- 172.21.15.67:0/1788026789 >> 172.21.15.67:0/1788026789 conn(0x560965389720 msgr2=0x56096542b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.447+0000 7fde040a5b80 1 -- 172.21.15.67:0/1788026789 shutdown_connections 2024-03-30T02:39:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.447+0000 7fde040a5b80 1 -- 172.21.15.67:0/1788026789 wait complete. 2024-03-30T02:39:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.448+0000 7fde040a5b80 20 client.26426 trim_cache size 0 max 16384 2024-03-30T02:39:42.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.479+0000 7fde040a5b80 1 Processor -- start 2024-03-30T02:39:42.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.479+0000 7fde040a5b80 1 -- start start 2024-03-30T02:39:42.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.480+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 0x56096543baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.480+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429f70 0x560965430550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.480+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965434e70 0x560965430a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.480+0000 7fde040a5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560965374780 con 0x560965429650 2024-03-30T02:39:42.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.480+0000 7fde040a5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096533c400 con 0x560965429f70 2024-03-30T02:39:42.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.480+0000 7fde040a5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560965441650 con 0x560965434e70 2024-03-30T02:39:42.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.480+0000 7fdded200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429f70 0x560965430550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.481+0000 7fdded200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429f70 0x560965430550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43664/0 (socket says 172.21.15.67:43664) 2024-03-30T02:39:42.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.481+0000 7fdded200700 1 -- 172.21.15.67:0/3139758018 learned_addr learned my addr 172.21.15.67:0/3139758018 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.481+0000 7fddedc00700 1 --2- 172.21.15.67:0/3139758018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 0x56096543baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.481+0000 7fddee600700 1 --2- 172.21.15.67:0/3139758018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965434e70 0x560965430a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.481+0000 7fdded200700 1 -- 172.21.15.67:0/3139758018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965434e70 msgr2=0x560965430a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.481+0000 7fdded200700 1 --2- 172.21.15.67:0/3139758018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965434e70 0x560965430a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.481+0000 7fdded200700 1 -- 172.21.15.67:0/3139758018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 msgr2=0x56096543baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.481+0000 7fdded200700 1 --2- 172.21.15.67:0/3139758018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 0x56096543baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.481+0000 7fdded200700 1 -- 172.21.15.67:0/3139758018 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56096544b9b0 con 0x560965429f70 2024-03-30T02:39:42.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.482+0000 7fdded200700 1 --2- 172.21.15.67:0/3139758018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429f70 0x560965430550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fddd400feb0 tx=0x7fddd4012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7c42b970105a856 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.483+0000 7fdddb400700 1 -- 172.21.15.67:0/3139758018 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd400fee0 con 0x560965429f70 2024-03-30T02:39:42.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.483+0000 7fdddb400700 1 -- 172.21.15.67:0/3139758018 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddd4020db0 con 0x560965429f70 2024-03-30T02:39:42.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.483+0000 7fdddb400700 1 -- 172.21.15.67:0/3139758018 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4028740 con 0x560965429f70 2024-03-30T02:39:42.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.484+0000 7fde040a5b80 1 -- 172.21.15.67:0/3139758018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429f70 msgr2=0x560965430550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.484+0000 7fde040a5b80 1 --2- 172.21.15.67:0/3139758018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429f70 0x560965430550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fddd400feb0 tx=0x7fddd4012000 comp rx=0 tx=0).stop 2024-03-30T02:39:42.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.484+0000 7fde040a5b80 1 -- 172.21.15.67:0/3139758018 shutdown_connections 2024-03-30T02:39:42.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.484+0000 7fde040a5b80 1 --2- 172.21.15.67:0/3139758018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429650 0x56096543baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.484+0000 7fde040a5b80 1 --2- 172.21.15.67:0/3139758018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965434e70 0x560965430a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.484+0000 7fde040a5b80 1 --2- 172.21.15.67:0/3139758018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965429f70 0x560965430550 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.484+0000 7fde040a5b80 1 -- 172.21.15.67:0/3139758018 >> 172.21.15.67:0/3139758018 conn(0x560965389720 msgr2=0x56096541eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.484+0000 7fde040a5b80 1 -- 172.21.15.67:0/3139758018 shutdown_connections 2024-03-30T02:39:42.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.484+0000 7fde040a5b80 1 -- 172.21.15.67:0/3139758018 wait complete. 2024-03-30T02:39:42.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.485+0000 7fde040a5b80 1 -- start start 2024-03-30T02:39:42.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.485+0000 7fde040a5b80 1 Processor -- start 2024-03-30T02:39:42.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.486+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429650 0x560965492aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.486+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429f70 0x560965492fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.486+0000 7fde040a5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x56096543a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.486+0000 7fde040a5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096543add0 con 0x560965429f70 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.486+0000 7fde040a5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096543af40 con 0x560965434e70 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.486+0000 7fde040a5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096543b100 con 0x560965429650 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fde040a5b80 10 client.0 _ll_register_callbacks cb 0x7fdddc005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fdded200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429f70 0x560965492fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fdded200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429f70 0x560965492fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35678/0 (socket says 172.21.15.67:35678) 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fdded200700 1 -- 172.21.15.67:0/1156447799 learned_addr learned my addr 172.21.15.67:0/1156447799 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fddedc00700 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429650 0x560965492aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fddee600700 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x56096543a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fdded200700 1 -- 172.21.15.67:0/1156447799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429650 msgr2=0x560965492aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fdded200700 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429650 0x560965492aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fdded200700 1 -- 172.21.15.67:0/1156447799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 msgr2=0x56096543a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.487+0000 7fdded200700 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x56096543a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.488+0000 7fdded200700 1 -- 172.21.15.67:0/1156447799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd400fb90 con 0x560965429f70 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.488+0000 7fddee600700 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x56096543a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.488+0000 7fdded200700 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429f70 0x560965492fe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fddd4011ee0 tx=0x7fddd40117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a935760f4503eba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.488+0000 7fddcf400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.488+0000 7fddcf400700 1 -- 172.21.15.67:0/1156447799 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd400fee0 con 0x560965429f70 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.489+0000 7fddcf400700 1 -- 172.21.15.67:0/1156447799 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fddd4020bb0 con 0x560965429f70 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.489+0000 7fde040a5b80 1 -- 172.21.15.67:0/1156447799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdde000f040 con 0x560965429f70 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.489+0000 7fddcf400700 1 -- 172.21.15.67:0/1156447799 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd4011d00 con 0x560965429f70 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.489+0000 7fde040a5b80 10 client.16362 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:42.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.489+0000 7fde040a5b80 1 -- 172.21.15.67:0/1156447799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560965341f20 con 0x560965429f70 2024-03-30T02:39:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.489+0000 7fddcf400700 1 -- 172.21.15.67:0/1156447799 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde000f040 con 0x560965429f70 2024-03-30T02:39:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.489+0000 7fddcf400700 1 -- 172.21.15.67:0/1156447799 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fddd4030070 con 0x560965429f70 2024-03-30T02:39:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.490+0000 7fde040a5b80 10 client.16362 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:42.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.490+0000 7fde040a5b80 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:42.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.490+0000 7fde040a5b80 20 client.16362 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:42.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.490+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609654ab930 0x5609654add50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.490+0000 7fde040a5b80 4 client.16362 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:39:42.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.490+0000 7fde040a5b80 1 -- 172.21.15.67:0/1156447799 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5609651cad70 con 0x5609654ab930 2024-03-30T02:39:42.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.490+0000 7fddedc00700 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609654ab930 0x5609654add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.491+0000 7fddedc00700 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609654ab930 0x5609654add50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e759fc86ec3fd378 server_cookie=d1320758123b8e82 in_seq=0 out_seq=0 2024-03-30T02:39:42.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.491+0000 7fddcf400700 10 client.16362 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:42.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.493+0000 7fddcf400700 1 -- 172.21.15.67:0/1156447799 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5609651cad70 con 0x5609654ab930 2024-03-30T02:39:42.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.493+0000 7fddcf400700 10 client.16362 handle_command_reply: tid=0 2024-03-30T02:39:42.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.493+0000 7fde040a5b80 1 client.16362 shutdown 2024-03-30T02:39:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.493+0000 7fde040a5b80 1 -- 172.21.15.67:0/1156447799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429f70 msgr2=0x560965492fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.493+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429f70 0x560965492fe0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fddd4011ee0 tx=0x7fddd40117a0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fde040a5b80 1 -- 172.21.15.67:0/1156447799 shutdown_connections 2024-03-30T02:39:42.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560965434e70 0x56096543a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560965429650 0x560965492aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609654ab930 0x5609654add50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fde040a5b80 1 --2- 172.21.15.67:0/1156447799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560965429f70 0x560965492fe0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fddcf400700 0 client.16362 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fde040a5b80 1 -- 172.21.15.67:0/1156447799 >> 172.21.15.67:0/1156447799 conn(0x560965389720 msgr2=0x56096542cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fde040a5b80 1 -- 172.21.15.67:0/1156447799 shutdown_connections 2024-03-30T02:39:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fde040a5b80 1 -- 172.21.15.67:0/1156447799 wait complete. 2024-03-30T02:39:42.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.494+0000 7fde040a5b80 20 client.16362 trim_cache size 0 max 16384 2024-03-30T02:39:42.496 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:39:42.496 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:39:42.496 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "9030f556-da0d-4f43-9af5-90265b5eb7d5", 2024-03-30T02:39:42.496 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:39:42.496 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 -- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fddd00a13e0 msgr2=0x7fddd00a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fddd00a13e0 0x7fddd00a38a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fddd4005950 tx=0x7fddd40058e0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 -- 172.21.15.67:0/2704133583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde80ff7a0 msgr2=0x7fdde80723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde80ff7a0 0x7fdde80723c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdddc000c00 tx=0x7fdddc0057f0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 -- 172.21.15.67:0/2704133583 shutdown_connections 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fddd00a13e0 0x7fddd00a38a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdde8111ae0 0x7fdde806b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdde80ff7a0 0x7fdde80723c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 --2- 172.21.15.67:0/2704133583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdde8106d80 0x7fdde8072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.495+0000 7fddf0600700 1 -- 172.21.15.67:0/2704133583 >> 172.21.15.67:0/2704133583 conn(0x7fdde8074b20 msgr2=0x7fdde81120a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.496+0000 7fddf0600700 1 -- 172.21.15.67:0/2704133583 shutdown_connections 2024-03-30T02:39:42.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.496+0000 7fddf0600700 1 -- 172.21.15.67:0/2704133583 wait complete. 2024-03-30T02:39:42.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:39:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:42 smithi170 ceph-mon[78798]: pgmap v82: 129 pgs: 129 active+clean; 333 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 506 op/s 2024-03-30T02:39:42.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.735+0000 7f71b3400700 1 -- 172.21.15.67:0/3426122513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac100f60 msgr2=0x7f71ac101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.735+0000 7f71b3400700 1 --2- 172.21.15.67:0/3426122513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac100f60 0x7f71ac101340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009990 tx=0x7f71a0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.735+0000 7f71b3400700 1 -- 172.21.15.67:0/3426122513 shutdown_connections 2024-03-30T02:39:42.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.735+0000 7f71b3400700 1 --2- 172.21.15.67:0/3426122513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac106080 0x7f71ac10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.735+0000 7f71b3400700 1 --2- 172.21.15.67:0/3426122513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac100f60 0x7f71ac101340 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.735+0000 7f71b3400700 1 --2- 172.21.15.67:0/3426122513 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac101910 0x7f71ac105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.735+0000 7f71b3400700 1 -- 172.21.15.67:0/3426122513 >> 172.21.15.67:0/3426122513 conn(0x7f71ac0fc7c0 msgr2=0x7f71ac0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.735+0000 7f71b3400700 1 -- 172.21.15.67:0/3426122513 shutdown_connections 2024-03-30T02:39:42.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.735+0000 7f71b3400700 1 -- 172.21.15.67:0/3426122513 wait complete. 2024-03-30T02:39:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.736+0000 7f71b3400700 1 Processor -- start 2024-03-30T02:39:42.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.736+0000 7f71b3400700 1 -- start start 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.736+0000 7f71b3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac100f60 0x7f71ac199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b3400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac106080 0x7f71ac1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac19dd50 0x7f71ac19e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b3400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac19e7d0 con 0x7f71ac100f60 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b3400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac19e940 con 0x7f71ac19dd50 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b3400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac19eb00 con 0x7f71ac106080 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b0a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac100f60 0x7f71ac199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac19dd50 0x7f71ac19e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac19dd50 0x7f71ac19e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43680/0 (socket says 172.21.15.67:43680) 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b1400700 1 -- 172.21.15.67:0/2727689369 learned_addr learned my addr 172.21.15.67:0/2727689369 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71abe00700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac106080 0x7f71ac1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b0a00700 1 -- 172.21.15.67:0/2727689369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac106080 msgr2=0x7f71ac1999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b0a00700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac106080 0x7f71ac1999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b0a00700 1 -- 172.21.15.67:0/2727689369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac19dd50 msgr2=0x7f71ac19e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b0a00700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac19dd50 0x7f71ac19e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b0a00700 1 -- 172.21.15.67:0/2727689369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00095f0 con 0x7f71ac100f60 2024-03-30T02:39:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.737+0000 7f71b1400700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac19dd50 0x7f71ac19e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.738+0000 7f71b0a00700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac100f60 0x7f71ac199470 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000c00 tx=0x7f71a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.738+0000 7f71a9600700 1 -- 172.21.15.67:0/2727689369 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a001d070 con 0x7f71ac100f60 2024-03-30T02:39:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.738+0000 7f71a9600700 1 -- 172.21.15.67:0/2727689369 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a0021410 con 0x7f71ac100f60 2024-03-30T02:39:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.738+0000 7f71b3400700 1 -- 172.21.15.67:0/2727689369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac1a6000 con 0x7f71ac100f60 2024-03-30T02:39:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.738+0000 7f71a9600700 1 -- 172.21.15.67:0/2727689369 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0017650 con 0x7f71ac100f60 2024-03-30T02:39:42.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.738+0000 7f71b3400700 1 -- 172.21.15.67:0/2727689369 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac1a6480 con 0x7f71ac100f60 2024-03-30T02:39:42.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.741+0000 7f71a9600700 1 -- 172.21.15.67:0/2727689369 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f71a0021580 con 0x7f71ac100f60 2024-03-30T02:39:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.743+0000 7f71a9600700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.743+0000 7f71a9600700 1 -- 172.21.15.67:0/2727689369 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f71a00ce1b0 con 0x7f71ac100f60 2024-03-30T02:39:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.743+0000 7f71c6f73b80 1 Processor -- start 2024-03-30T02:39:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.743+0000 7f71abe00700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.743+0000 7f71c6f73b80 1 -- start start 2024-03-30T02:39:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34955d0 0x562fd34959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd3495ef0 0x562fd34a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 0x562fd34a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd33ee990 con 0x562fd34a0df0 2024-03-30T02:39:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd33ec350 con 0x562fd34955d0 2024-03-30T02:39:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd33e7c40 con 0x562fd3495ef0 2024-03-30T02:39:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71b0a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34955d0 0x562fd34959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71b1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 0x562fd34a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71b0a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34955d0 0x562fd34959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43696/0 (socket says 172.21.15.67:43696) 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71b0a00700 1 -- 172.21.15.67:0/206062036 learned_addr learned my addr 172.21.15.67:0/206062036 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.744+0000 7f71abe00700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71ac19a9c0 tx=0x7f7198009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f71abe00700 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd3495ef0 0x562fd34a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f71b0a00700 1 -- 172.21.15.67:0/206062036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd3495ef0 msgr2=0x562fd34a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f71b0a00700 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd3495ef0 0x562fd34a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f71b0a00700 1 -- 172.21.15.67:0/206062036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 msgr2=0x562fd34a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f71b0a00700 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 0x562fd34a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f71b0a00700 1 -- 172.21.15.67:0/206062036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fd32693b0 con 0x562fd34955d0 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f71b1400700 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 0x562fd34a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f71abe00700 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd3495ef0 0x562fd34a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f71b0a00700 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34955d0 0x562fd34959b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f71a0097a10 tx=0x7f71a009ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=451eaa772c1978e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f719dc00700 1 -- 172.21.15.67:0/206062036 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00a10e0 con 0x562fd34955d0 2024-03-30T02:39:42.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f719dc00700 1 -- 172.21.15.67:0/206062036 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a009b6a0 con 0x562fd34955d0 2024-03-30T02:39:42.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.745+0000 7f719dc00700 1 -- 172.21.15.67:0/206062036 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a009b460 con 0x562fd34955d0 2024-03-30T02:39:42.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.746+0000 7f71c6f73b80 1 -- 172.21.15.67:0/206062036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34955d0 msgr2=0x562fd34959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.746+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34955d0 0x562fd34959b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f71a0097a10 tx=0x7f71a009ad70 comp rx=0 tx=0).stop 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.747+0000 7f71c6f73b80 1 -- 172.21.15.67:0/206062036 shutdown_connections 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.747+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 0x562fd34a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.747+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34955d0 0x562fd34959b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.747+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/206062036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd3495ef0 0x562fd34a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.747+0000 7f71c6f73b80 1 -- 172.21.15.67:0/206062036 >> 172.21.15.67:0/206062036 conn(0x562fd33f56a0 msgr2=0x562fd3491a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.747+0000 7f71c6f73b80 1 -- 172.21.15.67:0/206062036 shutdown_connections 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.747+0000 7f71c6f73b80 1 -- 172.21.15.67:0/206062036 wait complete. 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.748+0000 7f71c6f73b80 1 -- start start 2024-03-30T02:39:42.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.748+0000 7f71c6f73b80 1 Processor -- start 2024-03-30T02:39:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:42 smithi082 ceph-mon[79176]: pgmap v82: 129 pgs: 129 active+clean; 333 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 506 op/s 2024-03-30T02:39:42.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.748+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34955d0 0x562fd34a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.748+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34a1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.748+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34a0df0 0x562fd34ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd3383d80 con 0x562fd34955d0 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd3383a20 con 0x562fd3495ef0 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd326b6a0 con 0x562fd34a0df0 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71c6f73b80 10 client.0 _ll_register_callbacks cb 0x562fd338d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71b0a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34955d0 0x562fd34a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71b0a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34955d0 0x562fd34a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35716/0 (socket says 172.21.15.67:35716) 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71b0a00700 1 -- 172.21.15.67:0/748052775 learned_addr learned my addr 172.21.15.67:0/748052775 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71b1400700 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34a0df0 0x562fd34ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71abe00700 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71b1400700 1 -- 172.21.15.67:0/748052775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 msgr2=0x562fd34a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71b1400700 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71b1400700 1 -- 172.21.15.67:0/748052775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34955d0 msgr2=0x562fd34a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71b1400700 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34955d0 0x562fd34a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.749+0000 7f71b1400700 1 -- 172.21.15.67:0/748052775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a4009110 con 0x562fd34a0df0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.750+0000 7f71b1400700 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34a0df0 0x562fd34ac250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007fb0 tx=0x7f71a400e7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c994abf35920608 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.750+0000 7f7192000700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.750+0000 7f7192000700 1 -- 172.21.15.67:0/748052775 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4004db0 con 0x562fd34a0df0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.750+0000 7f7192000700 1 -- 172.21.15.67:0/748052775 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f71a4003a00 con 0x562fd34a0df0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.750+0000 7f71c6f73b80 1 -- 172.21.15.67:0/748052775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fd33a8380 con 0x562fd34a0df0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.750+0000 7f7192000700 1 -- 172.21.15.67:0/748052775 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a400eec0 con 0x562fd34a0df0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.750+0000 7f71c6f73b80 10 client.26410 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f71c6f73b80 1 -- 172.21.15.67:0/748052775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f71ac1a6480 con 0x562fd34a0df0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f7192000700 1 -- 172.21.15.67:0/748052775 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562fd33a8380 con 0x562fd34a0df0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f7192000700 1 -- 172.21.15.67:0/748052775 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f71a401e530 con 0x562fd34a0df0 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f71c6f73b80 10 client.26410 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:42.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f71c6f73b80 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:42.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f71c6f73b80 20 client.26410 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:42.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562fd34b06c0 0x562fd34b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f71c6f73b80 4 client.26410 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:39:42.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f71c6f73b80 1 -- 172.21.15.67:0/748052775 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fd33e0700 con 0x562fd34b06c0 2024-03-30T02:39:42.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.751+0000 7f71b0a00700 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562fd34b06c0 0x562fd34b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.752+0000 7f71b0a00700 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562fd34b06c0 0x562fd34b2ae0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e718a15f67b6434a server_cookie=71e76f5cdae484f7 in_seq=0 out_seq=0 2024-03-30T02:39:42.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.752+0000 7f7192000700 10 client.26410 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:42.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.754+0000 7f7192000700 1 -- 172.21.15.67:0/748052775 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562fd33e0700 con 0x562fd34b06c0 2024-03-30T02:39:42.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.754+0000 7f7192000700 10 client.26410 handle_command_reply: tid=0 2024-03-30T02:39:42.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.754+0000 7f71c6f73b80 1 client.26410 shutdown 2024-03-30T02:39:42.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.754+0000 7f71c6f73b80 1 -- 172.21.15.67:0/748052775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34a0df0 msgr2=0x562fd34ac250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.754+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34a0df0 0x562fd34ac250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007fb0 tx=0x7f71a400e7d0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.754+0000 7f71c6f73b80 1 -- 172.21.15.67:0/748052775 shutdown_connections 2024-03-30T02:39:42.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.754+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562fd34b06c0 0x562fd34b2ae0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.755+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34955d0 0x562fd34a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.755+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34a0df0 0x562fd34ac250 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.755+0000 7f7192000700 0 client.26410 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:42.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.755+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/748052775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.755+0000 7f71c6f73b80 1 -- 172.21.15.67:0/748052775 >> 172.21.15.67:0/748052775 conn(0x562fd33f56a0 msgr2=0x562fd34976c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.755+0000 7f71c6f73b80 1 -- 172.21.15.67:0/748052775 shutdown_connections 2024-03-30T02:39:42.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.755+0000 7f71c6f73b80 1 -- 172.21.15.67:0/748052775 wait complete. 2024-03-30T02:39:42.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.755+0000 7f71c6f73b80 20 client.26410 trim_cache size 0 max 16384 2024-03-30T02:39:42.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71c6f73b80 1 Processor -- start 2024-03-30T02:39:42.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71c6f73b80 1 -- start start 2024-03-30T02:39:42.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd3495ef0 0x562fd349ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34a0df0 0x562fd349ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd34ad190 con 0x562fd3495ef0 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd3497130 con 0x562fd34a0df0 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd349f490 con 0x562fd34955d0 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71b0a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71b0a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34a61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:34874/0 (socket says 172.21.15.67:34874) 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71b0a00700 1 -- 172.21.15.67:0/1109627309 learned_addr learned my addr 172.21.15.67:0/1109627309 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71abe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd3495ef0 0x562fd349ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.773+0000 7f71b1400700 1 --2- 172.21.15.67:0/1109627309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34a0df0 0x562fd349ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.774+0000 7f71b0a00700 1 -- 172.21.15.67:0/1109627309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34a0df0 msgr2=0x562fd349ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.774+0000 7f71b0a00700 1 --2- 172.21.15.67:0/1109627309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34a0df0 0x562fd349ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.774+0000 7f71b0a00700 1 -- 172.21.15.67:0/1109627309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd3495ef0 msgr2=0x562fd349ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.774+0000 7f71b0a00700 1 --2- 172.21.15.67:0/1109627309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd3495ef0 0x562fd349ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.774+0000 7f71b0a00700 1 -- 172.21.15.67:0/1109627309 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fd3215b60 con 0x562fd34955d0 2024-03-30T02:39:42.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.774+0000 7f71b0a00700 1 --2- 172.21.15.67:0/1109627309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71a0097a10 tx=0x7f71a0021c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff64285e13c8e6b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.774+0000 7f719e600700 1 -- 172.21.15.67:0/1109627309 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00a3390 con 0x562fd34955d0 2024-03-30T02:39:42.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.774+0000 7f719e600700 1 -- 172.21.15.67:0/1109627309 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a0009290 con 0x562fd34955d0 2024-03-30T02:39:42.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.774+0000 7f719e600700 1 -- 172.21.15.67:0/1109627309 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a009a490 con 0x562fd34955d0 2024-03-30T02:39:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1109627309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 msgr2=0x562fd34a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1109627309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71a0097a10 tx=0x7f71a0021c00 comp rx=0 tx=0).stop 2024-03-30T02:39:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1109627309 shutdown_connections 2024-03-30T02:39:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1109627309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd34a0df0 0x562fd349ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1109627309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1109627309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd3495ef0 0x562fd349ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1109627309 >> 172.21.15.67:0/1109627309 conn(0x562fd33f56a0 msgr2=0x562fd348a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1109627309 shutdown_connections 2024-03-30T02:39:42.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1109627309 wait complete. 2024-03-30T02:39:42.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.775+0000 7f71c6f73b80 1 -- start start 2024-03-30T02:39:42.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71c6f73b80 1 Processor -- start 2024-03-30T02:39:42.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71c6f73b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 0x562fd34fc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd34fca30 con 0x562fd34a0df0 2024-03-30T02:39:42.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd34fcba0 con 0x562fd3495ef0 2024-03-30T02:39:42.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71c6f73b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fd34fcd60 con 0x562fd34955d0 2024-03-30T02:39:42.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71c6f73b80 10 client.0 _ll_register_callbacks cb 0x562fd338d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:39:42.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71abe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71b0a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71abe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34fdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:43724/0 (socket says 172.21.15.67:43724) 2024-03-30T02:39:42.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71b1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 0x562fd34fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.776+0000 7f71abe00700 1 -- 172.21.15.67:0/1621088566 learned_addr learned my addr 172.21.15.67:0/1621088566 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:39:42.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71b0a00700 1 -- 172.21.15.67:0/1621088566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 msgr2=0x562fd34fdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71b0a00700 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34fdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71b0a00700 1 -- 172.21.15.67:0/1621088566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 msgr2=0x562fd34fc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:39:42.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71b0a00700 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 0x562fd34fc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71b0a00700 1 -- 172.21.15.67:0/1621088566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562fd32693b0 con 0x562fd34955d0 2024-03-30T02:39:42.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71abe00700 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34fdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:39:42.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71b0a00700 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34fd760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f71a009af30 tx=0x7f71a00cb630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad0caa458109476 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:39:42.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f7192000700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:39:42.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f7192000700 1 -- 172.21.15.67:0/1621088566 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a009edd0 con 0x562fd34955d0 2024-03-30T02:39:42.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f7192000700 1 -- 172.21.15.67:0/1621088566 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f71a009afa0 con 0x562fd34955d0 2024-03-30T02:39:42.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1621088566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71a009b6a0 con 0x562fd34955d0 2024-03-30T02:39:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f7192000700 1 -- 172.21.15.67:0/1621088566 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a00b91c0 con 0x562fd34955d0 2024-03-30T02:39:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71c6f73b80 10 client.26416 fetch_fsmap learned FSMap version 30 2024-03-30T02:39:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1621088566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562fd33adea0 con 0x562fd34955d0 2024-03-30T02:39:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f7192000700 1 -- 172.21.15.67:0/1621088566 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a009b6a0 con 0x562fd34955d0 2024-03-30T02:39:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.777+0000 7f7192000700 1 -- 172.21.15.67:0/1621088566 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f71a009a590 con 0x562fd34955d0 2024-03-30T02:39:42.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.778+0000 7f71c6f73b80 10 client.26416 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:39:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.778+0000 7f71c6f73b80 10 client.26416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:39:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.778+0000 7f71c6f73b80 20 client.26416 populate_metadata read hostname 'smithi067' 2024-03-30T02:39:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.778+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562fd3505430 0x562fd3507850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:39:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.778+0000 7f71c6f73b80 4 client.26416 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:39:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.778+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1621088566 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562fd3236cf0 con 0x562fd3505430 2024-03-30T02:39:42.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.778+0000 7f71abe00700 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562fd3505430 0x562fd3507850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:39:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.778+0000 7f71abe00700 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562fd3505430 0x562fd3507850 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75a7374293c7462 server_cookie=f57182c808c13c0 in_seq=0 out_seq=0 2024-03-30T02:39:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.778+0000 7f7192000700 10 client.26416 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.780+0000 7f7192000700 1 -- 172.21.15.67:0/1621088566 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x562fd3236cf0 con 0x562fd3505430 2024-03-30T02:39:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.780+0000 7f7192000700 10 client.26416 handle_command_reply: tid=0 2024-03-30T02:39:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.780+0000 7f71c6f73b80 1 client.26416 shutdown 2024-03-30T02:39:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.780+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1621088566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 msgr2=0x562fd34fd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.780+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34fd760 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f71a009af30 tx=0x7f71a00cb630 comp rx=0 tx=0).stop 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1621088566 shutdown_connections 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562fd3505430 0x562fd3507850 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562fd34a0df0 0x562fd34fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fd34955d0 0x562fd34fd760 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f71c6f73b80 1 --2- 172.21.15.67:0/1621088566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562fd3495ef0 0x562fd34fdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1621088566 >> 172.21.15.67:0/1621088566 conn(0x562fd33f56a0 msgr2=0x562fd349e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f7192000700 0 client.26416 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1621088566 shutdown_connections 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f71c6f73b80 1 -- 172.21.15.67:0/1621088566 wait complete. 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.781+0000 7f71c6f73b80 20 client.26416 trim_cache size 0 max 16384 2024-03-30T02:39:42.791 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:39:42.792 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (299 inodes in the stack)", 2024-03-30T02:39:42.792 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:39:42.792 INFO:teuthology.orchestra.run.smithi067.stdout: "9030f556-da0d-4f43-9af5-90265b5eb7d5": { 2024-03-30T02:39:42.792 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:39:42.792 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "9030f556-da0d-4f43-9af5-90265b5eb7d5", 2024-03-30T02:39:42.792 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:39:42.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:39:42.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:39:42.792 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 -- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 msgr2=0x7f71940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71ac19a9c0 tx=0x7f7198009260 comp rx=0 tx=0).stop 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 -- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac100f60 msgr2=0x7f71ac199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac100f60 0x7f71ac199470 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000c00 tx=0x7f71a0004730 comp rx=0 tx=0).stop 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 -- 172.21.15.67:0/2727689369 shutdown_connections 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac100f60 0x7f71ac199470 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac19dd50 0x7f71ac19e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 --2- 172.21.15.67:0/2727689369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac106080 0x7f71ac1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 -- 172.21.15.67:0/2727689369 >> 172.21.15.67:0/2727689369 conn(0x7f71ac0fc7c0 msgr2=0x7f71ac0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:39:42.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 -- 172.21.15.67:0/2727689369 shutdown_connections 2024-03-30T02:39:42.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:39:42.782+0000 7f71b3400700 1 -- 172.21.15.67:0/2727689369 wait complete. 2024-03-30T02:39:42.803 INFO:tasks.cephfs.filesystem:scrub status for tag:9030f556-da0d-4f43-9af5-90265b5eb7d5 - {'path': '/', 'tag': '9030f556-da0d-4f43-9af5-90265b5eb7d5', 'options': 'recursive,force'} 2024-03-30T02:39:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: pgmap v83: 129 pgs: 129 active+clean; 333 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 506 op/s 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: pgmap v83: 129 pgs: 129 active+clean; 333 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 506 op/s 2024-03-30T02:39:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:39:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: pgmap v83: 129 pgs: 129 active+clean; 333 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 506 op/s 2024-03-30T02:39:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:45.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: pgmap v84: 129 pgs: 129 active+clean; 367 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 818 op/s 2024-03-30T02:39:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: pgmap v84: 129 pgs: 129 active+clean; 367 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 818 op/s 2024-03-30T02:39:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: pgmap v84: 129 pgs: 129 active+clean; 367 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 9.0 MiB/s wr, 818 op/s 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:48 smithi067 ceph-mon[37808]: pgmap v85: 129 pgs: 129 active+clean; 367 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-03-30T02:39:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:48 smithi170 ceph-mon[78798]: pgmap v85: 129 pgs: 129 active+clean; 367 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-03-30T02:39:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:48 smithi082 ceph-mon[79176]: pgmap v85: 129 pgs: 129 active+clean; 367 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-03-30T02:39:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: pgmap v86: 129 pgs: 129 active+clean; 393 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.9 MiB/s wr, 808 op/s 2024-03-30T02:39:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:50.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: pgmap v86: 129 pgs: 129 active+clean; 393 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.9 MiB/s wr, 808 op/s 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: pgmap v86: 129 pgs: 129 active+clean; 393 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.9 MiB/s wr, 808 op/s 2024-03-30T02:39:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:50.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:50.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: pgmap v87: 129 pgs: 129 active+clean; 393 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: pgmap v87: 129 pgs: 129 active+clean; 393 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: pgmap v87: 129 pgs: 129 active+clean; 393 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:52.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:52.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:52.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:52.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: pgmap v88: 129 pgs: 129 active+clean; 393 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-03-30T02:39:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: pgmap v88: 129 pgs: 129 active+clean; 393 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-03-30T02:39:53.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:53.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:53.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: pgmap v88: 129 pgs: 129 active+clean; 393 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 555 op/s 2024-03-30T02:39:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:55 smithi170 ceph-mon[78798]: pgmap v89: 129 pgs: 129 active+clean; 414 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.2 MiB/s wr, 700 op/s 2024-03-30T02:39:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:55 smithi082 ceph-mon[79176]: pgmap v89: 129 pgs: 129 active+clean; 414 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.2 MiB/s wr, 700 op/s 2024-03-30T02:39:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:55 smithi067 ceph-mon[37808]: pgmap v89: 129 pgs: 129 active+clean; 414 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.2 MiB/s wr, 700 op/s 2024-03-30T02:39:56.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:39:56.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:56.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:39:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:56.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:39:56.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:56.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:39:58.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:39:58 smithi067 ceph-mon[37808]: pgmap v90: 129 pgs: 129 active+clean; 414 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 388 op/s 2024-03-30T02:39:58.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:39:58 smithi170 ceph-mon[78798]: pgmap v90: 129 pgs: 129 active+clean; 414 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 388 op/s 2024-03-30T02:39:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:39:58 smithi082 ceph-mon[79176]: pgmap v90: 129 pgs: 129 active+clean; 414 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 388 op/s 2024-03-30T02:40:00.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:00 smithi067 ceph-mon[37808]: pgmap v91: 129 pgs: 129 active+clean; 429 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 496 op/s 2024-03-30T02:40:00.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:00 smithi067 ceph-mon[37808]: overall HEALTH_OK 2024-03-30T02:40:00.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:00 smithi170 ceph-mon[78798]: pgmap v91: 129 pgs: 129 active+clean; 429 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 496 op/s 2024-03-30T02:40:00.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:00 smithi170 ceph-mon[78798]: overall HEALTH_OK 2024-03-30T02:40:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:00 smithi082 ceph-mon[79176]: pgmap v91: 129 pgs: 129 active+clean; 429 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 496 op/s 2024-03-30T02:40:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:00 smithi082 ceph-mon[79176]: overall HEALTH_OK 2024-03-30T02:40:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:02 smithi067 ceph-mon[37808]: pgmap v92: 129 pgs: 129 active+clean; 429 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-03-30T02:40:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:40:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:40:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:40:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:40:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:02 smithi170 ceph-mon[78798]: pgmap v92: 129 pgs: 129 active+clean; 429 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-03-30T02:40:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:40:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:40:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:40:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:40:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:02 smithi082 ceph-mon[79176]: pgmap v92: 129 pgs: 129 active+clean; 429 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-03-30T02:40:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:40:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:40:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:40:02.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:40:03.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:03 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:40:03.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:03 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:40:03.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:03 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:40:04.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:04 smithi067 ceph-mon[37808]: pgmap v93: 129 pgs: 129 active+clean; 429 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-03-30T02:40:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:04 smithi170 ceph-mon[78798]: pgmap v93: 129 pgs: 129 active+clean; 429 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-03-30T02:40:04.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:04 smithi082 ceph-mon[79176]: pgmap v93: 129 pgs: 129 active+clean; 429 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-03-30T02:40:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:06 smithi067 ceph-mon[37808]: pgmap v94: 129 pgs: 129 active+clean; 457 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 429 op/s 2024-03-30T02:40:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:06 smithi170 ceph-mon[78798]: pgmap v94: 129 pgs: 129 active+clean; 457 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 429 op/s 2024-03-30T02:40:06.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:06 smithi082 ceph-mon[79176]: pgmap v94: 129 pgs: 129 active+clean; 457 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 429 op/s 2024-03-30T02:40:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:08 smithi067 ceph-mon[37808]: pgmap v95: 129 pgs: 129 active+clean; 457 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-03-30T02:40:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:08 smithi170 ceph-mon[78798]: pgmap v95: 129 pgs: 129 active+clean; 457 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-03-30T02:40:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:08 smithi082 ceph-mon[79176]: pgmap v95: 129 pgs: 129 active+clean; 457 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-03-30T02:40:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:10 smithi067 ceph-mon[37808]: pgmap v96: 129 pgs: 129 active+clean; 479 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.8 MiB/s wr, 410 op/s 2024-03-30T02:40:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:10 smithi170 ceph-mon[78798]: pgmap v96: 129 pgs: 129 active+clean; 479 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.8 MiB/s wr, 410 op/s 2024-03-30T02:40:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:10 smithi082 ceph-mon[79176]: pgmap v96: 129 pgs: 129 active+clean; 479 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.8 MiB/s wr, 410 op/s 2024-03-30T02:40:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:11 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:40:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:11 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:11 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:11 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:11 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:11 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:40:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:11 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:11 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:11 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:11 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:11 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:40:11.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:11 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:11 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:11 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:11 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:12 smithi067 ceph-mon[37808]: pgmap v97: 129 pgs: 129 active+clean; 480 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 316 op/s 2024-03-30T02:40:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:12 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:12 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:12 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:40:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:12 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:40:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:12 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:12 smithi170 ceph-mon[78798]: pgmap v97: 129 pgs: 129 active+clean; 480 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 316 op/s 2024-03-30T02:40:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:12 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:12 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:12 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:40:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:12 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:40:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:12 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:12 smithi082 ceph-mon[79176]: pgmap v97: 129 pgs: 129 active+clean; 480 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 316 op/s 2024-03-30T02:40:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:12 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:12 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:12 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:40:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:12 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:40:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:12 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:40:12.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:40:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.016+0000 7f91e6c00700 1 -- 172.21.15.67:0/1637213650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e0100b50 msgr2=0x7f91e0100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.016+0000 7f91e6c00700 1 --2- 172.21.15.67:0/1637213650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e0100b50 0x7f91e0100fd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009920 tx=0x7f91cc009c30 comp rx=0 tx=0).stop 2024-03-30T02:40:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.016+0000 7f91e6c00700 1 -- 172.21.15.67:0/1637213650 shutdown_connections 2024-03-30T02:40:13.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.016+0000 7f91e6c00700 1 --2- 172.21.15.67:0/1637213650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e0101510 0x7f91e010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.016+0000 7f91e6c00700 1 --2- 172.21.15.67:0/1637213650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e0106c10 0x7f91e0106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.016+0000 7f91e6c00700 1 --2- 172.21.15.67:0/1637213650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e0100b50 0x7f91e0100fd0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.016+0000 7f91e6c00700 1 -- 172.21.15.67:0/1637213650 >> 172.21.15.67:0/1637213650 conn(0x7f91e00fc620 msgr2=0x7f91e00fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.017+0000 7f91e6c00700 1 -- 172.21.15.67:0/1637213650 shutdown_connections 2024-03-30T02:40:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.017+0000 7f91e6c00700 1 -- 172.21.15.67:0/1637213650 wait complete. 2024-03-30T02:40:13.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.018+0000 7f91e6c00700 1 Processor -- start 2024-03-30T02:40:13.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e6c00700 1 -- start start 2024-03-30T02:40:13.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e6c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e0100b50 0x7f91e0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e6c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e0101510 0x7f91e01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e6c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e0106c10 0x7f91e019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e6c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e019e180 con 0x7f91e0100b50 2024-03-30T02:40:13.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e6c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e019e2f0 con 0x7f91e0106c10 2024-03-30T02:40:13.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e6c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e019e4b0 con 0x7f91e0101510 2024-03-30T02:40:13.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e0101510 0x7f91e01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e5800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e0100b50 0x7f91e0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e6200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e0106c10 0x7f91e019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.019+0000 7f91e5800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e0100b50 0x7f91e0199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44814/0 (socket says 172.21.15.67:44814) 2024-03-30T02:40:13.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.020+0000 7f91e5800700 1 -- 172.21.15.67:0/2001201047 learned_addr learned my addr 172.21.15.67:0/2001201047 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:13.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.020+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e0101510 0x7f91e01997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60056/0 (socket says 172.21.15.67:60056) 2024-03-30T02:40:13.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.020+0000 7f91e4e00700 1 -- 172.21.15.67:0/2001201047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e0106c10 msgr2=0x7f91e019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.020+0000 7f91e4e00700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e0106c10 0x7f91e019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.020+0000 7f91e4e00700 1 -- 172.21.15.67:0/2001201047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e0100b50 msgr2=0x7f91e0199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.020+0000 7f91e4e00700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e0100b50 0x7f91e0199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.020+0000 7f91e4e00700 1 -- 172.21.15.67:0/2001201047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8009d00 con 0x7f91e0101510 2024-03-30T02:40:13.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.020+0000 7f91e4e00700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e0101510 0x7f91e01997a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f91cc000c00 tx=0x7f91cc00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:13.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.020+0000 7f91de000700 1 -- 172.21.15.67:0/2001201047 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc01c070 con 0x7f91e0101510 2024-03-30T02:40:13.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.021+0000 7f91e6c00700 1 -- 172.21.15.67:0/2001201047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc009520 con 0x7f91e0101510 2024-03-30T02:40:13.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.021+0000 7f91de000700 1 -- 172.21.15.67:0/2001201047 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91cc00ba20 con 0x7f91e0101510 2024-03-30T02:40:13.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.021+0000 7f91de000700 1 -- 172.21.15.67:0/2001201047 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc004930 con 0x7f91e0101510 2024-03-30T02:40:13.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.021+0000 7f91e6c00700 1 -- 172.21.15.67:0/2001201047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e019e9c0 con 0x7f91e0101510 2024-03-30T02:40:13.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.024+0000 7f91de000700 1 -- 172.21.15.67:0/2001201047 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f91cc020070 con 0x7f91e0101510 2024-03-30T02:40:13.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.025+0000 7f91fbc72b80 1 Processor -- start 2024-03-30T02:40:13.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.025+0000 7f91fbc72b80 1 -- start start 2024-03-30T02:40:13.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91de000700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f91c80a1290 0x7f91c80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91de000700 1 -- 172.21.15.67:0/2001201047 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f91cc0cdae0 con 0x7f91e0101510 2024-03-30T02:40:13.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f5eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f69a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f69fc0 0x558434f6c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434eb7990 con 0x558434f5e7a0 2024-03-30T02:40:13.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434eb5350 con 0x558434f5f0c0 2024-03-30T02:40:13.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434eb0c40 con 0x558434f69fc0 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91e6200700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f91c80a1290 0x7f91c80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f69a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91e5800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f5eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f69a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57434/0 (socket says 172.21.15.67:57434) 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.026+0000 7f91e4e00700 1 -- 172.21.15.67:0/2541453731 learned_addr learned my addr 172.21.15.67:0/2541453731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.027+0000 7f91e6200700 1 --2- 172.21.15.67:0/2541453731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f69fc0 0x558434f6c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.027+0000 7f91e5800700 1 -- 172.21.15.67:0/2541453731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f69fc0 msgr2=0x558434f6c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.027+0000 7f91e5800700 1 --2- 172.21.15.67:0/2541453731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f69fc0 0x558434f6c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.027+0000 7f91e5800700 1 -- 172.21.15.67:0/2541453731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 msgr2=0x558434f69a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.027+0000 7f91e5800700 1 --2- 172.21.15.67:0/2541453731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f69a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.027+0000 7f91e5800700 1 -- 172.21.15.67:0/2541453731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558434d323b0 con 0x558434f5e7a0 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.027+0000 7f91e6200700 1 --2- 172.21.15.67:0/2541453731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f69fc0 0x558434f6c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.027+0000 7f91e5800700 1 --2- 172.21.15.67:0/2541453731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f5eb80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009d10 tx=0x7f91d4011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0beab8329310d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:13.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.028+0000 7f91e6200700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f91c80a1290 0x7f91c80a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009fd0 tx=0x7f91d80099e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:13.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.028+0000 7f91d2a00700 1 -- 172.21.15.67:0/2541453731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4004250 con 0x558434f5e7a0 2024-03-30T02:40:13.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.028+0000 7f91d2a00700 1 -- 172.21.15.67:0/2541453731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91d4004cf0 con 0x558434f5e7a0 2024-03-30T02:40:13.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.028+0000 7f91d2a00700 1 -- 172.21.15.67:0/2541453731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40055e0 con 0x558434f5e7a0 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.029+0000 7f91fbc72b80 1 -- 172.21.15.67:0/2541453731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 msgr2=0x558434f5eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.029+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/2541453731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f5eb80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009d10 tx=0x7f91d4011720 comp rx=0 tx=0).stop 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.029+0000 7f91fbc72b80 1 -- 172.21.15.67:0/2541453731 shutdown_connections 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.029+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/2541453731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f5eb80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.029+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/2541453731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f69fc0 0x558434f6c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.029+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/2541453731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f69a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.029+0000 7f91fbc72b80 1 -- 172.21.15.67:0/2541453731 >> 172.21.15.67:0/2541453731 conn(0x558434ebe6a0 msgr2=0x558434f5ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.029+0000 7f91fbc72b80 1 -- 172.21.15.67:0/2541453731 shutdown_connections 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.029+0000 7f91fbc72b80 1 -- 172.21.15.67:0/2541453731 wait complete. 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.030+0000 7f91fbc72b80 1 -- start start 2024-03-30T02:40:13.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.030+0000 7f91fbc72b80 1 Processor -- start 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5f0c0 0x558434f66cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 0x558434f672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434e4cd80 con 0x558434f5e7a0 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434e4ca20 con 0x558434f69fc0 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434d346a0 con 0x558434f5f0c0 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91fbc72b80 10 client.0 _ll_register_callbacks cb 0x558434e56470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5f0c0 0x558434f66cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91e5800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5f0c0 0x558434f66cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60066/0 (socket says 172.21.15.67:60066) 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91e4e00700 1 -- 172.21.15.67:0/1551325806 learned_addr learned my addr 172.21.15.67:0/1551325806 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:13.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.031+0000 7f91e6200700 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 0x558434f672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.032+0000 7f91e5800700 1 -- 172.21.15.67:0/1551325806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5f0c0 msgr2=0x558434f66cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.032+0000 7f91e5800700 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5f0c0 0x558434f66cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.032+0000 7f91e5800700 1 -- 172.21.15.67:0/1551325806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 msgr2=0x558434f672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.032+0000 7f91e5800700 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 0x558434f672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.032+0000 7f91e5800700 1 -- 172.21.15.67:0/1551325806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d4009970 con 0x558434f5e7a0 2024-03-30T02:40:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.032+0000 7f91e4e00700 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5f0c0 0x558434f66cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.032+0000 7f91e6200700 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 0x558434f672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.032+0000 7f91e5800700 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f699c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f91d40096b0 tx=0x7f91d4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c0e4e266e4dcb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:13.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.032+0000 7f91c7400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:40:13.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.033+0000 7f91c7400700 1 -- 172.21.15.67:0/1551325806 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4004ee0 con 0x558434f5e7a0 2024-03-30T02:40:13.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.033+0000 7f91c7400700 1 -- 172.21.15.67:0/1551325806 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f91d4004960 con 0x558434f5e7a0 2024-03-30T02:40:13.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.033+0000 7f91fbc72b80 1 -- 172.21.15.67:0/1551325806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558434e71380 con 0x558434f5e7a0 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.033+0000 7f91c7400700 1 -- 172.21.15.67:0/1551325806 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d4004b40 con 0x558434f5e7a0 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.033+0000 7f91fbc72b80 10 client.16392 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.033+0000 7f91fbc72b80 1 -- 172.21.15.67:0/1551325806 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558434cdeb60 con 0x558434f5e7a0 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.033+0000 7f91c7400700 1 -- 172.21.15.67:0/1551325806 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558434e71380 con 0x558434f5e7a0 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.034+0000 7f91c7400700 1 -- 172.21.15.67:0/1551325806 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f91d4003c80 con 0x558434f5e7a0 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.034+0000 7f91fbc72b80 10 client.16392 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.034+0000 7f91fbc72b80 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.034+0000 7f91fbc72b80 20 client.16392 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.034+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558434f79a90 0x558434f7beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.034+0000 7f91fbc72b80 4 client.16392 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.034+0000 7f91fbc72b80 1 -- 172.21.15.67:0/1551325806 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558434ea9700 con 0x558434f79a90 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.034+0000 7f91e4e00700 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558434f79a90 0x558434f7beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.035+0000 7f91e4e00700 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558434f79a90 0x558434f7beb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdbd93f1fb84fd7b server_cookie=e94bac0028e77970 in_seq=0 out_seq=0 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.035+0000 7f91c7400700 10 client.16392 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:13.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.035+0000 7f91c7400700 1 -- 172.21.15.67:0/1551325806 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558434ea9700 con 0x558434f79a90 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.035+0000 7f91c7400700 10 client.16392 handle_command_reply: tid=0 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.035+0000 7f91fbc72b80 1 client.16392 shutdown 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.035+0000 7f91fbc72b80 1 -- 172.21.15.67:0/1551325806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 msgr2=0x558434f699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.035+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f699c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f91d40096b0 tx=0x7f91d4009810 comp rx=0 tx=0).stop 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91fbc72b80 1 -- 172.21.15.67:0/1551325806 shutdown_connections 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558434f79a90 0x558434f7beb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5e7a0 0x558434f699c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 0x558434f672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/1551325806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5f0c0 0x558434f66cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91c7400700 0 client.16392 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91fbc72b80 1 -- 172.21.15.67:0/1551325806 >> 172.21.15.67:0/1551325806 conn(0x558434ebe6a0 msgr2=0x558434f60860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91fbc72b80 1 -- 172.21.15.67:0/1551325806 shutdown_connections 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91fbc72b80 1 -- 172.21.15.67:0/1551325806 wait complete. 2024-03-30T02:40:13.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.036+0000 7f91fbc72b80 20 client.16392 trim_cache size 0 max 16384 2024-03-30T02:40:13.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.056+0000 7f91fbc72b80 1 Processor -- start 2024-03-30T02:40:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.056+0000 7f91fbc72b80 1 -- start start 2024-03-30T02:40:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.056+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f7be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.056+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f67260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.056+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f69fc0 0x558434f66c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.056+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434f65220 con 0x558434f69fc0 2024-03-30T02:40:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.056+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434f6c3b0 con 0x558434f5f0c0 2024-03-30T02:40:13.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.056+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434f62720 con 0x558434f5e7a0 2024-03-30T02:40:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.056+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f67260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f67260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57454/0 (socket says 172.21.15.67:57454) 2024-03-30T02:40:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e5800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f7be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e4e00700 1 -- 172.21.15.67:0/3126864039 learned_addr learned my addr 172.21.15.67:0/3126864039 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e6200700 1 --2- 172.21.15.67:0/3126864039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f69fc0 0x558434f66c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e5800700 1 -- 172.21.15.67:0/3126864039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 msgr2=0x558434f67260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e5800700 1 --2- 172.21.15.67:0/3126864039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f67260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e5800700 1 -- 172.21.15.67:0/3126864039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f69fc0 msgr2=0x558434f66c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:13.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e5800700 1 --2- 172.21.15.67:0/3126864039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f69fc0 0x558434f66c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e5800700 1 -- 172.21.15.67:0/3126864039 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558434e76ea0 con 0x558434f5e7a0 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e4e00700 1 --2- 172.21.15.67:0/3126864039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f67260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91e5800700 1 --2- 172.21.15.67:0/3126864039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f7be80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91d401c040 tx=0x7f91d4005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db241ab4d242b60 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91d3400700 1 -- 172.21.15.67:0/3126864039 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4004410 con 0x558434f5e7a0 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91d3400700 1 -- 172.21.15.67:0/3126864039 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91d40226c0 con 0x558434f5e7a0 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.057+0000 7f91d3400700 1 -- 172.21.15.67:0/3126864039 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d400ea70 con 0x558434f5e7a0 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.058+0000 7f91fbc72b80 1 -- 172.21.15.67:0/3126864039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 msgr2=0x558434f7be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.058+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/3126864039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f7be80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91d401c040 tx=0x7f91d4005d90 comp rx=0 tx=0).stop 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.058+0000 7f91fbc72b80 1 -- 172.21.15.67:0/3126864039 shutdown_connections 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.058+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/3126864039 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f69fc0 0x558434f66c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.058+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/3126864039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f7be80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.058+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/3126864039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f5f0c0 0x558434f67260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.058+0000 7f91fbc72b80 1 -- 172.21.15.67:0/3126864039 >> 172.21.15.67:0/3126864039 conn(0x558434ebe6a0 msgr2=0x558434f66540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:13.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.058+0000 7f91fbc72b80 1 -- 172.21.15.67:0/3126864039 shutdown_connections 2024-03-30T02:40:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.058+0000 7f91fbc72b80 1 -- 172.21.15.67:0/3126864039 wait complete. 2024-03-30T02:40:13.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.059+0000 7f91fbc72b80 1 -- start start 2024-03-30T02:40:13.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.059+0000 7f91fbc72b80 1 Processor -- start 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f6ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5f0c0 0x558434f6efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91fbc72b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 0x558434fcc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434fccdc0 con 0x558434f5f0c0 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434fccf00 con 0x558434f69fc0 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91fbc72b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558434fcd040 con 0x558434f5e7a0 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91fbc72b80 10 client.0 _ll_register_callbacks cb 0x558434e56470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5f0c0 0x558434f6efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91e5800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f6ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91e5800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f6ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60092/0 (socket says 172.21.15.67:60092) 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5f0c0 0x558434f6efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44850/0 (socket says 172.21.15.67:44850) 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91e5800700 1 -- 172.21.15.67:0/819999406 learned_addr learned my addr 172.21.15.67:0/819999406 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.060+0000 7f91e6200700 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 0x558434fcc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91e5800700 1 -- 172.21.15.67:0/819999406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 msgr2=0x558434fcc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91e5800700 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 0x558434fcc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91e5800700 1 -- 172.21.15.67:0/819999406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5f0c0 msgr2=0x558434f6efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91e5800700 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5f0c0 0x558434f6efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91e5800700 1 -- 172.21.15.67:0/819999406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e019e9c0 con 0x558434f5e7a0 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91e5800700 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f6ea70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f91d4014040 tx=0x7f91d4004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fd3c6f6f20a4779 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91c7400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91c7400700 1 -- 172.21.15.67:0/819999406 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4004410 con 0x558434f5e7a0 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91fbc72b80 1 -- 172.21.15.67:0/819999406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91d4004cf0 con 0x558434f5e7a0 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.061+0000 7f91c7400700 1 -- 172.21.15.67:0/819999406 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f91d4022b10 con 0x558434f5e7a0 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91c7400700 1 -- 172.21.15.67:0/819999406 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91d4022cf0 con 0x558434f5e7a0 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91fbc72b80 10 client.26431 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91fbc72b80 1 -- 172.21.15.67:0/819999406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558434cdeb60 con 0x558434f5e7a0 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91c7400700 1 -- 172.21.15.67:0/819999406 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91d4004cf0 con 0x558434f5e7a0 2024-03-30T02:40:13.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91c7400700 1 -- 172.21.15.67:0/819999406 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f91d40037a0 con 0x558434f5e7a0 2024-03-30T02:40:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91fbc72b80 10 client.26431 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91fbc72b80 10 client.26431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91fbc72b80 20 client.26431 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558434fce360 0x558434fd0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91fbc72b80 4 client.26431 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:40:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91fbc72b80 1 -- 172.21.15.67:0/819999406 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558434cffcf0 con 0x558434fce360 2024-03-30T02:40:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.062+0000 7f91e4e00700 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558434fce360 0x558434fd0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.063+0000 7f91e4e00700 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558434fce360 0x558434fd0780 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfff53e767c12dac server_cookie=10ee88e775025c70 in_seq=0 out_seq=0 2024-03-30T02:40:13.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.063+0000 7f91c7400700 10 client.26431 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91c7400700 1 -- 172.21.15.67:0/819999406 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558434cffcf0 con 0x558434fce360 2024-03-30T02:40:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91c7400700 10 client.26431 handle_command_reply: tid=0 2024-03-30T02:40:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 client.26431 shutdown 2024-03-30T02:40:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 -- 172.21.15.67:0/819999406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 msgr2=0x558434f6ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f6ea70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f91d4014040 tx=0x7f91d4004c10 comp rx=0 tx=0).stop 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 -- 172.21.15.67:0/819999406 shutdown_connections 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558434f69fc0 0x558434fcc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558434f5e7a0 0x558434f6ea70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558434fce360 0x558434fd0780 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 --2- 172.21.15.67:0/819999406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558434f5f0c0 0x558434f6efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91c7400700 0 client.26431 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 -- 172.21.15.67:0/819999406 >> 172.21.15.67:0/819999406 conn(0x558434ebe6a0 msgr2=0x558434f66460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 -- 172.21.15.67:0/819999406 shutdown_connections 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 1 -- 172.21.15.67:0/819999406 wait complete. 2024-03-30T02:40:13.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.064+0000 7f91fbc72b80 20 client.26431 trim_cache size 0 max 16384 2024-03-30T02:40:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:40:13.068 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:40:13.068 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:40:13.068 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:40:13.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 -- 172.21.15.67:0/2001201047 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f91c80a1290 msgr2=0x7f91c80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f91c80a1290 0x7f91c80a3750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009fd0 tx=0x7f91d80099e0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 -- 172.21.15.67:0/2001201047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e0101510 msgr2=0x7f91e01997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e0101510 0x7f91e01997a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f91cc000c00 tx=0x7f91cc00fa00 comp rx=0 tx=0).stop 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 -- 172.21.15.67:0/2001201047 shutdown_connections 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f91c80a1290 0x7f91c80a3750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91e0100b50 0x7f91e0199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e0106c10 0x7f91e019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 --2- 172.21.15.67:0/2001201047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e0101510 0x7f91e01997a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 -- 172.21.15.67:0/2001201047 >> 172.21.15.67:0/2001201047 conn(0x7f91e00fc620 msgr2=0x7f91e0073430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 -- 172.21.15.67:0/2001201047 shutdown_connections 2024-03-30T02:40:13.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.065+0000 7f91e6c00700 1 -- 172.21.15.67:0/2001201047 wait complete. 2024-03-30T02:40:13.084 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:40:13.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:40:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.399+0000 7f8010c00700 1 -- 172.21.15.67:0/2069373650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f800c111950 msgr2=0x7f800c113d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.399+0000 7f8010c00700 1 --2- 172.21.15.67:0/2069373650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f800c111950 0x7f800c113d60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8004009f90 tx=0x7f800400b330 comp rx=0 tx=0).stop 2024-03-30T02:40:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.400+0000 7f8010c00700 1 -- 172.21.15.67:0/2069373650 shutdown_connections 2024-03-30T02:40:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.400+0000 7f8010c00700 1 --2- 172.21.15.67:0/2069373650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f800c111950 0x7f800c113d60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.400+0000 7f8010c00700 1 --2- 172.21.15.67:0/2069373650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800c106c00 0x7f800c0ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.400+0000 7f8010c00700 1 --2- 172.21.15.67:0/2069373650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f800c0ff620 0x7f800c0ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.400+0000 7f8010c00700 1 -- 172.21.15.67:0/2069373650 >> 172.21.15.67:0/2069373650 conn(0x7f800c075c80 msgr2=0x7f800c076090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.401+0000 7f8010c00700 1 -- 172.21.15.67:0/2069373650 shutdown_connections 2024-03-30T02:40:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.401+0000 7f8010c00700 1 -- 172.21.15.67:0/2069373650 wait complete. 2024-03-30T02:40:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.402+0000 7f8010c00700 1 Processor -- start 2024-03-30T02:40:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.403+0000 7f8010c00700 1 -- start start 2024-03-30T02:40:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.405+0000 7f8010c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f800c0ff620 0x7f800c110cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.405+0000 7f8010c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800c106c00 0x7f800c1111f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.406+0000 7f8010c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f800c111950 0x7f800c10adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.407+0000 7f800b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f800c0ff620 0x7f800c110cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.407+0000 7f800b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f800c0ff620 0x7f800c110cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57474/0 (socket says 172.21.15.67:57474) 2024-03-30T02:40:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.408+0000 7f800aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800c106c00 0x7f800c1111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.409+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c10b5c0 con 0x7f800c111950 2024-03-30T02:40:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.409+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c10b730 con 0x7f800c0ff620 2024-03-30T02:40:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.409+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c10b8f0 con 0x7f800c106c00 2024-03-30T02:40:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.409+0000 7f800b400700 1 -- 172.21.15.67:0/2134832178 learned_addr learned my addr 172.21.15.67:0/2134832178 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:13.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.409+0000 7f800be00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f800c111950 0x7f800c10adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.410+0000 7f800be00700 1 -- 172.21.15.67:0/2134832178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800c106c00 msgr2=0x7f800c1111f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.410+0000 7f800be00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800c106c00 0x7f800c1111f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.410+0000 7f800be00700 1 -- 172.21.15.67:0/2134832178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f800c0ff620 msgr2=0x7f800c110cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.410+0000 7f800be00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f800c0ff620 0x7f800c110cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.410+0000 7f800be00700 1 -- 172.21.15.67:0/2134832178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8004009c40 con 0x7f800c111950 2024-03-30T02:40:13.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.410+0000 7f800be00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f800c111950 0x7f800c10adf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f800c06a420 tx=0x7f80040079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:13.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.411+0000 7f8003e00700 1 -- 172.21.15.67:0/2134832178 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400e040 con 0x7f800c111950 2024-03-30T02:40:13.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.411+0000 7f8003e00700 1 -- 172.21.15.67:0/2134832178 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8004005040 con 0x7f800c111950 2024-03-30T02:40:13.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.411+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c10bb70 con 0x7f800c111950 2024-03-30T02:40:13.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.411+0000 7f8003e00700 1 -- 172.21.15.67:0/2134832178 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800401c660 con 0x7f800c111950 2024-03-30T02:40:13.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.412+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c1b2c40 con 0x7f800c111950 2024-03-30T02:40:13.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.416+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c100300 con 0x7f800c111950 2024-03-30T02:40:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.420+0000 7f8003e00700 1 -- 172.21.15.67:0/2134832178 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f80040051b0 con 0x7f800c111950 2024-03-30T02:40:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.423+0000 7f8003e00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7ff00a13d0 0x7f7ff00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.424+0000 7f800b400700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7ff00a13d0 0x7f7ff00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.427+0000 7f8003e00700 1 -- 172.21.15.67:0/2134832178 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f80040cda20 con 0x7f800c111950 2024-03-30T02:40:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.427+0000 7f800b400700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7ff00a13d0 0x7f7ff00a3890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00bd10 tx=0x7f7ffc00b4e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:13.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:13.438+0000 7f8003e00700 1 -- 172.21.15.67:0/2134832178 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8004096b50 con 0x7f800c111950 2024-03-30T02:40:13.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.000+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f800c10c230 con 0x7f800c111950 2024-03-30T02:40:14.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.004+0000 7f8003e00700 1 -- 172.21.15.67:0/2134832178 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10916 (secure 0 0 0) 0x7f800409a170 con 0x7f800c111950 2024-03-30T02:40:14.006 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:40:14.018 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:38:47.255528+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.010+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7ff00a13d0 msgr2=0x7f7ff00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.010+0000 7f8010c00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7ff00a13d0 0x7f7ff00a3890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00bd10 tx=0x7f7ffc00b4e0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.010+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f800c111950 msgr2=0x7f800c10adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.010+0000 7f8010c00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f800c111950 0x7f800c10adf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f800c06a420 tx=0x7f80040079d0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.010+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 shutdown_connections 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.011+0000 7f8010c00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7ff00a13d0 0x7f7ff00a3890 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.011+0000 7f8010c00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f800c111950 0x7f800c10adf0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.011+0000 7f8010c00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f800c0ff620 0x7f800c110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.011+0000 7f8010c00700 1 --2- 172.21.15.67:0/2134832178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800c106c00 0x7f800c1111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.011+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 >> 172.21.15.67:0/2134832178 conn(0x7f800c075c80 msgr2=0x7f800c112be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:14.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.012+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 shutdown_connections 2024-03-30T02:40:14.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.012+0000 7f8010c00700 1 -- 172.21.15.67:0/2134832178 wait complete. 2024-03-30T02:40:14.022 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-03-30T02:40:14.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:40:14.362 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:14 smithi067 ceph-mon[37808]: pgmap v98: 129 pgs: 129 active+clean; 480 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 316 op/s 2024-03-30T02:40:14.363 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:14 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2134832178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:40:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.483+0000 7f7bd5a00700 1 -- 172.21.15.67:0/279666753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 msgr2=0x7f7bd01150f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.483+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/279666753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 0x7f7bd01150f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009fc0 tx=0x7f7bc800b330 comp rx=0 tx=0).stop 2024-03-30T02:40:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.483+0000 7f7bd5a00700 1 -- 172.21.15.67:0/279666753 shutdown_connections 2024-03-30T02:40:14.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.483+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/279666753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 0x7f7bd01150f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.483+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/279666753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd0109750 0x7f7bd0109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.483+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/279666753 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bd010a070 0x7f7bd01127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.483+0000 7f7bd5a00700 1 -- 172.21.15.67:0/279666753 >> 172.21.15.67:0/279666753 conn(0x7f7bd006c240 msgr2=0x7f7bd006c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:14.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.484+0000 7f7bd5a00700 1 -- 172.21.15.67:0/279666753 shutdown_connections 2024-03-30T02:40:14.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.484+0000 7f7bd5a00700 1 -- 172.21.15.67:0/279666753 wait complete. 2024-03-30T02:40:14.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.485+0000 7f7bd5a00700 1 Processor -- start 2024-03-30T02:40:14.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.486+0000 7f7bd5a00700 1 -- start start 2024-03-30T02:40:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.487+0000 7f7bd5a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bd0109750 0x7f7bd01a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.487+0000 7f7bd5a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd010a070 0x7f7bd01a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.487+0000 7f7bd5a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 0x7f7bd019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.488+0000 7f7bd5a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd019eee0 con 0x7f7bd0109750 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.488+0000 7f7bd5a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd019f050 con 0x7f7bd0112d00 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.488+0000 7f7bd5a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd019f210 con 0x7f7bd010a070 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.488+0000 7f7bcf400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 0x7f7bd019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.488+0000 7f7bcf400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 0x7f7bd019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57488/0 (socket says 172.21.15.67:57488) 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.488+0000 7f7bcf400700 1 -- 172.21.15.67:0/3419632185 learned_addr learned my addr 172.21.15.67:0/3419632185 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.488+0000 7f7bcea00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bd0109750 0x7f7bd01a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.489+0000 7f7bcf400700 1 -- 172.21.15.67:0/3419632185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd010a070 msgr2=0x7f7bd01a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.489+0000 7f7bcf400700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd010a070 0x7f7bd01a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.489+0000 7f7bcf400700 1 -- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bd0109750 msgr2=0x7f7bd01a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.489+0000 7f7bcf400700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bd0109750 0x7f7bd01a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.489+0000 7f7bcf400700 1 -- 172.21.15.67:0/3419632185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc8009c40 con 0x7f7bd0112d00 2024-03-30T02:40:14.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.489+0000 7f7bcf400700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 0x7f7bd019e710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800b6c0 tx=0x7f7bc80075b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:14.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.490+0000 7f7bc7e00700 1 -- 172.21.15.67:0/3419632185 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800e040 con 0x7f7bd0112d00 2024-03-30T02:40:14.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.490+0000 7f7bc7e00700 1 -- 172.21.15.67:0/3419632185 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc800bbf0 con 0x7f7bd0112d00 2024-03-30T02:40:14.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.490+0000 7f7bc7e00700 1 -- 172.21.15.67:0/3419632185 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc801b6c0 con 0x7f7bd0112d00 2024-03-30T02:40:14.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.490+0000 7f7bd5a00700 1 -- 172.21.15.67:0/3419632185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd019f490 con 0x7f7bd0112d00 2024-03-30T02:40:14.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.490+0000 7f7bd5a00700 1 -- 172.21.15.67:0/3419632185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd01ab5f0 con 0x7f7bd0112d00 2024-03-30T02:40:14.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.496+0000 7f7be95dcb80 1 Processor -- start 2024-03-30T02:40:14.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.498+0000 7f7bc7e00700 1 -- 172.21.15.67:0/3419632185 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7bc801d470 con 0x7f7bd0112d00 2024-03-30T02:40:14.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.498+0000 7f7be95dcb80 1 -- start start 2024-03-30T02:40:14.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.499+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a50782e0 0x55f5a50765d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.499+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 0x55f5a5076f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.499+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a50882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.499+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a4fcf990 con 0x55f5a5085eb0 2024-03-30T02:40:14.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.499+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a4fcd350 con 0x55f5a5076b10 2024-03-30T02:40:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.499+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a4fc8c40 con 0x55f5a50782e0 2024-03-30T02:40:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.500+0000 7f7bc6000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a50782e0 0x55f5a50765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.500+0000 7f7bc6000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a50782e0 0x55f5a50765d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60122/0 (socket says 172.21.15.67:60122) 2024-03-30T02:40:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.500+0000 7f7bc6000700 1 -- 172.21.15.67:0/2160763255 learned_addr learned my addr 172.21.15.67:0/2160763255 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.500+0000 7f7bc6000700 1 --2- 172.21.15.67:0/2160763255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a50882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.500+0000 7f7bc6000700 1 -- 172.21.15.67:0/2160763255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 msgr2=0x55f5a5076f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:14.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.500+0000 7f7bc6000700 1 --2- 172.21.15.67:0/2160763255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 0x55f5a5076f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.500+0000 7f7bc6000700 1 -- 172.21.15.67:0/2160763255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 msgr2=0x55f5a50882a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.501+0000 7f7bc6000700 1 --2- 172.21.15.67:0/2160763255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a50882a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.501+0000 7f7bc6000700 1 -- 172.21.15.67:0/2160763255 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a4e4a3b0 con 0x55f5a50782e0 2024-03-30T02:40:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.501+0000 7f7bc7e00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7bb40a1420 0x7f7bb40a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.502+0000 7f7bc7e00700 1 -- 172.21.15.67:0/3419632185 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f7bc80ce110 con 0x7f7bd0112d00 2024-03-30T02:40:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.502+0000 7f7bc6000700 1 --2- 172.21.15.67:0/2160763255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a50782e0 0x55f5a50765d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80103d0 tx=0x7f7bb8009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a0d732bc9db0dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.502+0000 7f7bcea00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7bb40a1420 0x7f7bb40a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.503+0000 7f7bcea00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7bb40a1420 0x7f7bb40a38e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55f5a5077730 tx=0x7f7bc000b2c0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.504+0000 7f7bbfe00700 1 -- 172.21.15.67:0/2160763255 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8010c60 con 0x55f5a50782e0 2024-03-30T02:40:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.504+0000 7f7bbfe00700 1 -- 172.21.15.67:0/2160763255 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb8010dc0 con 0x55f5a50782e0 2024-03-30T02:40:14.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.504+0000 7f7bbfe00700 1 -- 172.21.15.67:0/2160763255 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8011440 con 0x55f5a50782e0 2024-03-30T02:40:14.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.505+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2160763255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a50782e0 msgr2=0x55f5a50765d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.505+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2160763255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a50782e0 0x55f5a50765d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80103d0 tx=0x7f7bb8009740 comp rx=0 tx=0).stop 2024-03-30T02:40:14.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.506+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2160763255 shutdown_connections 2024-03-30T02:40:14.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.506+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2160763255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a50882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.506+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2160763255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a50782e0 0x55f5a50765d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.506+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2160763255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 0x55f5a5076f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.506+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2160763255 >> 172.21.15.67:0/2160763255 conn(0x55f5a4fd66a0 msgr2=0x55f5a5072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:14.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.506+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2160763255 shutdown_connections 2024-03-30T02:40:14.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.507+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2160763255 wait complete. 2024-03-30T02:40:14.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.508+0000 7f7be95dcb80 1 -- start start 2024-03-30T02:40:14.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.508+0000 7f7be95dcb80 1 Processor -- start 2024-03-30T02:40:14.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.510+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 0x55f5a5082d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.510+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 0x55f5a50832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.510+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5085eb0 0x55f5a5083810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.511+0000 7f7bc6000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 0x55f5a50832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.511+0000 7f7bc6000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 0x55f5a50832d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44890/0 (socket says 172.21.15.67:44890) 2024-03-30T02:40:14.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.511+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a4f64d80 con 0x55f5a50782e0 2024-03-30T02:40:14.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.511+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a4f64a20 con 0x55f5a5085eb0 2024-03-30T02:40:14.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.511+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a4e4c6a0 con 0x55f5a5076b10 2024-03-30T02:40:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.512+0000 7f7bc6000700 1 -- 172.21.15.67:0/2512137536 learned_addr learned my addr 172.21.15.67:0/2512137536 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.512+0000 7f7be95dcb80 10 client.0 _ll_register_callbacks cb 0x55f5a4f6e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.512+0000 7f7bc6000700 1 -- 172.21.15.67:0/2512137536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 msgr2=0x55f5a5082d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:14.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.511+0000 7f7bcea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 0x55f5a5082d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.512+0000 7f7bc6000700 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 0x55f5a5082d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.512+0000 7f7bc6000700 1 -- 172.21.15.67:0/2512137536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5085eb0 msgr2=0x55f5a5083810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:14.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.512+0000 7f7bc6000700 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5085eb0 0x55f5a5083810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.512+0000 7f7bc6000700 1 -- 172.21.15.67:0/2512137536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb8009c20 con 0x55f5a50782e0 2024-03-30T02:40:14.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.513+0000 7f7bcea00700 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 0x55f5a5082d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:14.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.513+0000 7f7bc6000700 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 0x55f5a50832d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8005060 tx=0x7f7bb8004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5e4775c7d58c07 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:14.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.514+0000 7f7bbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:40:14.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.514+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2512137536 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8010c60 con 0x55f5a50782e0 2024-03-30T02:40:14.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.514+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2512137536 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f7bb8004030 con 0x55f5a50782e0 2024-03-30T02:40:14.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.514+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2512137536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5a4f89380 con 0x55f5a50782e0 2024-03-30T02:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.517+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2512137536 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb80050c0 con 0x55f5a50782e0 2024-03-30T02:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.517+0000 7f7be95dcb80 10 client.16416 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:14.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.517+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2512137536 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f5a4df6b60 con 0x55f5a50782e0 2024-03-30T02:40:14.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.519+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2512137536 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5a4f89380 con 0x55f5a50782e0 2024-03-30T02:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.521+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2512137536 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7bb8003c20 con 0x55f5a50782e0 2024-03-30T02:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.521+0000 7f7be95dcb80 10 client.16416 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.521+0000 7f7be95dcb80 10 client.16416 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.521+0000 7f7be95dcb80 20 client.16416 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:14.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.521+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55f5a5095d70 0x55f5a5098190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.521+0000 7f7be95dcb80 4 client.16416 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:40:14.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.521+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2512137536 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5a4fc1700 con 0x55f5a5095d70 2024-03-30T02:40:14.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.524+0000 7f7bcea00700 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55f5a5095d70 0x55f5a5098190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.527+0000 7f7bbcc00700 10 client.16416 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:40:14.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.527+0000 7f7bcea00700 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55f5a5095d70 0x55f5a5098190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=92a1abf09dba26f8 server_cookie=29ed70b29f4e9967 in_seq=0 out_seq=0 2024-03-30T02:40:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.527+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2512137536 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f5a4fc1700 con 0x55f5a5095d70 2024-03-30T02:40:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.527+0000 7f7bbcc00700 10 client.16416 handle_command_reply: tid=0 2024-03-30T02:40:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.528+0000 7f7be95dcb80 1 client.16416 shutdown 2024-03-30T02:40:14.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.528+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2512137536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 msgr2=0x55f5a50832d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.528+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 0x55f5a50832d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8005060 tx=0x7f7bb8004b10 comp rx=0 tx=0).stop 2024-03-30T02:40:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.529+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2512137536 shutdown_connections 2024-03-30T02:40:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.529+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55f5a5095d70 0x55f5a5098190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.530+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5085eb0 0x55f5a5083810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.530+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 0x55f5a5082d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.530+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2512137536 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 0x55f5a50832d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.530+0000 7f7bbcc00700 0 client.16416 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:40:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.530+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2512137536 >> 172.21.15.67:0/2512137536 conn(0x55f5a4fd66a0 msgr2=0x55f5a50786e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.531+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2512137536 shutdown_connections 2024-03-30T02:40:14.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.531+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2512137536 wait complete. 2024-03-30T02:40:14.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.531+0000 7f7be95dcb80 20 client.16416 trim_cache size 0 max 16384 2024-03-30T02:40:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.569+0000 7f7be95dcb80 1 Processor -- start 2024-03-30T02:40:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.570+0000 7f7be95dcb80 1 -- start start 2024-03-30T02:40:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.570+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 0x55f5a5091f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.570+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 0x55f5a508ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.570+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5085eb0 0x55f5a5083630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.570+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a50842f0 con 0x55f5a50782e0 2024-03-30T02:40:14.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.570+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5074320 con 0x55f5a5076b10 2024-03-30T02:40:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.570+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5088ad0 con 0x55f5a5085eb0 2024-03-30T02:40:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.570+0000 7f7bcea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 0x55f5a5091f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.571+0000 7f7bcea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 0x55f5a5091f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57506/0 (socket says 172.21.15.67:57506) 2024-03-30T02:40:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.571+0000 7f7bcea00700 1 -- 172.21.15.67:0/1841532473 learned_addr learned my addr 172.21.15.67:0/1841532473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:14.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.571+0000 7f7bcea00700 1 -- 172.21.15.67:0/1841532473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5085eb0 msgr2=0x55f5a5083630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:14.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.571+0000 7f7bcea00700 1 --2- 172.21.15.67:0/1841532473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5085eb0 0x55f5a5083630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.571+0000 7f7bcea00700 1 -- 172.21.15.67:0/1841532473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 msgr2=0x55f5a508ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:14.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.571+0000 7f7bcea00700 1 --2- 172.21.15.67:0/1841532473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 0x55f5a508ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.571+0000 7f7bcea00700 1 -- 172.21.15.67:0/1841532473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a4f8eea0 con 0x55f5a5076b10 2024-03-30T02:40:14.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.574+0000 7f7bcea00700 1 --2- 172.21.15.67:0/1841532473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 0x55f5a5091f70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000b9f0 tx=0x7f7bc0012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1843304fe52541aa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:14.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.575+0000 7f7bc4c00700 1 -- 172.21.15.67:0/1841532473 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0017040 con 0x55f5a5076b10 2024-03-30T02:40:14.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.575+0000 7f7bc4c00700 1 -- 172.21.15.67:0/1841532473 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc000fe60 con 0x55f5a5076b10 2024-03-30T02:40:14.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.575+0000 7f7bc4c00700 1 -- 172.21.15.67:0/1841532473 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc002da70 con 0x55f5a5076b10 2024-03-30T02:40:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.576+0000 7f7be95dcb80 1 -- 172.21.15.67:0/1841532473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 msgr2=0x55f5a5091f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.576+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/1841532473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 0x55f5a5091f70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000b9f0 tx=0x7f7bc0012000 comp rx=0 tx=0).stop 2024-03-30T02:40:14.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.576+0000 7f7be95dcb80 1 -- 172.21.15.67:0/1841532473 shutdown_connections 2024-03-30T02:40:14.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.576+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/1841532473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5085eb0 0x55f5a5083630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.576+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/1841532473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a5076b10 0x55f5a5091f70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.576+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/1841532473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a50782e0 0x55f5a508ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.576+0000 7f7be95dcb80 1 -- 172.21.15.67:0/1841532473 >> 172.21.15.67:0/1841532473 conn(0x55f5a4fd66a0 msgr2=0x55f5a5076ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:14.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.577+0000 7f7be95dcb80 1 -- 172.21.15.67:0/1841532473 shutdown_connections 2024-03-30T02:40:14.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.577+0000 7f7be95dcb80 1 -- 172.21.15.67:0/1841532473 wait complete. 2024-03-30T02:40:14.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.578+0000 7f7be95dcb80 1 -- start start 2024-03-30T02:40:14.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.578+0000 7f7be95dcb80 1 Processor -- start 2024-03-30T02:40:14.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.580+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 0x55f5a508bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.581+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a50782e0 0x55f5a508c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.581+0000 7f7be95dcb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a507fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.581+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a508c7d0 con 0x55f5a5085eb0 2024-03-30T02:40:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.581+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a508c940 con 0x55f5a50782e0 2024-03-30T02:40:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.581+0000 7f7be95dcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a507ffc0 con 0x55f5a5076b10 2024-03-30T02:40:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.581+0000 7f7be95dcb80 10 client.0 _ll_register_callbacks cb 0x55f5a4f6e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.581+0000 7f7bcf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a507fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.582+0000 7f7bcf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a507fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44904/0 (socket says 172.21.15.67:44904) 2024-03-30T02:40:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.582+0000 7f7bcf400700 1 -- 172.21.15.67:0/2228504151 learned_addr learned my addr 172.21.15.67:0/2228504151 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:14.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.582+0000 7f7bc6000700 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a50782e0 0x55f5a508c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.582+0000 7f7bcf400700 1 -- 172.21.15.67:0/2228504151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 msgr2=0x55f5a508bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.582+0000 7f7bcf400700 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 0x55f5a508bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.582+0000 7f7bcf400700 1 -- 172.21.15.67:0/2228504151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a50782e0 msgr2=0x55f5a508c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.582+0000 7f7bcf400700 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a50782e0 0x55f5a508c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.582+0000 7f7bcf400700 1 -- 172.21.15.67:0/2228504151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc00041e0 con 0x55f5a5085eb0 2024-03-30T02:40:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.583+0000 7f7bcf400700 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a507fa80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80cb570 tx=0x7f7bc809ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d77557b158f9a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.583+0000 7f7bbcc00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:40:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.584+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2228504151 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc801d920 con 0x55f5a5085eb0 2024-03-30T02:40:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.584+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2228504151 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f7bc809b660 con 0x55f5a5085eb0 2024-03-30T02:40:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.584+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2228504151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bb8010dc0 con 0x55f5a5085eb0 2024-03-30T02:40:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.585+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2228504151 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bc80a1070 con 0x55f5a5085eb0 2024-03-30T02:40:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.585+0000 7f7be95dcb80 10 client.16419 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:14.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.585+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2228504151 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f5a4e4ecd0 con 0x55f5a5085eb0 2024-03-30T02:40:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.586+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2228504151 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb8010dc0 con 0x55f5a5085eb0 2024-03-30T02:40:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.586+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2228504151 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7bc809b1c0 con 0x55f5a5085eb0 2024-03-30T02:40:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.586+0000 7f7be95dcb80 10 client.16419 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.586+0000 7f7be95dcb80 10 client.16419 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:40:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.586+0000 7f7be95dcb80 20 client.16419 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.586+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55f5a50eee70 0x55f5a50f1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.586+0000 7f7be95dcb80 4 client.16419 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:40:14.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.586+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2228504151 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5a4e17cf0 con 0x55f5a50eee70 2024-03-30T02:40:14.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.588+0000 7f7bcea00700 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55f5a50eee70 0x55f5a50f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:14.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.590+0000 7f7bcea00700 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55f5a50eee70 0x55f5a50f1290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=47fdb28aa1b3cbfa server_cookie=dcd5788092624b4d in_seq=0 out_seq=0 2024-03-30T02:40:14.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.590+0000 7f7bbcc00700 10 client.16419 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:40:14.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.593+0000 7f7bbcc00700 1 -- 172.21.15.67:0/2228504151 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5a4e17cf0 con 0x55f5a50eee70 2024-03-30T02:40:14.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.593+0000 7f7bbcc00700 10 client.16419 handle_command_reply: tid=0 2024-03-30T02:40:14.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.593+0000 7f7be95dcb80 1 client.16419 shutdown 2024-03-30T02:40:14.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.594+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2228504151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 msgr2=0x55f5a507fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.594+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a507fa80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80cb570 tx=0x7f7bc809ab20 comp rx=0 tx=0).stop 2024-03-30T02:40:14.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.596+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2228504151 shutdown_connections 2024-03-30T02:40:14.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.596+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f5a5085eb0 0x55f5a507fa80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.596+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55f5a50eee70 0x55f5a50f1290 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.596+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f5a5076b10 0x55f5a508bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.596+0000 7f7be95dcb80 1 --2- 172.21.15.67:0/2228504151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f5a50782e0 0x55f5a508c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.596+0000 7f7bbcc00700 0 client.16419 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:40:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.596+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2228504151 >> 172.21.15.67:0/2228504151 conn(0x55f5a4fd66a0 msgr2=0x55f5a5086a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.597+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2228504151 shutdown_connections 2024-03-30T02:40:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.597+0000 7f7be95dcb80 1 -- 172.21.15.67:0/2228504151 wait complete. 2024-03-30T02:40:14.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.599+0000 7f7be95dcb80 20 client.16419 trim_cache size 0 max 16384 2024-03-30T02:40:14.599 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.600+0000 7f7bd5a00700 1 -- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7bb40a1420 msgr2=0x7f7bb40a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.600+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7bb40a1420 0x7f7bb40a38e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55f5a5077730 tx=0x7f7bc000b2c0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.601+0000 7f7bd5a00700 1 -- 172.21.15.67:0/3419632185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 msgr2=0x7f7bd019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.601+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 0x7f7bd019e710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800b6c0 tx=0x7f7bc80075b0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.601+0000 7f7bd5a00700 1 -- 172.21.15.67:0/3419632185 shutdown_connections 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.601+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7bb40a1420 0x7f7bb40a38e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.601+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7bd0109750 0x7f7bd01a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.601+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bd0112d00 0x7f7bd019e710 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.601+0000 7f7bd5a00700 1 --2- 172.21.15.67:0/3419632185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7bd010a070 0x7f7bd01a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:14.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.601+0000 7f7bd5a00700 1 -- 172.21.15.67:0/3419632185 >> 172.21.15.67:0/3419632185 conn(0x7f7bd006c240 msgr2=0x7f7bd006fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:14.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.602+0000 7f7bd5a00700 1 -- 172.21.15.67:0/3419632185 shutdown_connections 2024-03-30T02:40:14.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:14.602+0000 7f7bd5a00700 1 -- 172.21.15.67:0/3419632185 wait complete. 2024-03-30T02:40:14.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:40:14.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:14 smithi170 ceph-mon[78798]: pgmap v98: 129 pgs: 129 active+clean; 480 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 316 op/s 2024-03-30T02:40:14.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:14 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2134832178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:40:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:14 smithi082 ceph-mon[79176]: pgmap v98: 129 pgs: 129 active+clean; 480 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.5 MiB/s wr, 316 op/s 2024-03-30T02:40:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:14 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2134832178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:40:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.043+0000 7f9e0ba00700 1 -- 172.21.15.67:0/3691710141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e04108020 msgr2=0x7f9e04108400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.043+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/3691710141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e04108020 0x7f9e04108400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9df8007570 tx=0x7f9df8007880 comp rx=0 tx=0).stop 2024-03-30T02:40:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.043+0000 7f9e0ba00700 1 -- 172.21.15.67:0/3691710141 shutdown_connections 2024-03-30T02:40:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.043+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/3691710141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e04108020 0x7f9e04108400 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.044+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/3691710141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e0410c6a0 0x7f9e040790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.044+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/3691710141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e04108940 0x7f9e0410bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.044+0000 7f9e0ba00700 1 -- 172.21.15.67:0/3691710141 >> 172.21.15.67:0/3691710141 conn(0x7f9e0406bdf0 msgr2=0x7f9e0406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.044+0000 7f9e0ba00700 1 -- 172.21.15.67:0/3691710141 shutdown_connections 2024-03-30T02:40:15.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.044+0000 7f9e0ba00700 1 -- 172.21.15.67:0/3691710141 wait complete. 2024-03-30T02:40:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.045+0000 7f9e0ba00700 1 Processor -- start 2024-03-30T02:40:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.046+0000 7f9e0ba00700 1 -- start start 2024-03-30T02:40:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.047+0000 7f9e0ba00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e04108020 0x7f9e04081c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.047+0000 7f9e0ba00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e04108940 0x7f9e040821d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.047+0000 7f9e0ba00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e0410c6a0 0x7f9e04082710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.047+0000 7f9e0ba00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e041bef30 con 0x7f9e04108940 2024-03-30T02:40:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.047+0000 7f9e0ba00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e041bf070 con 0x7f9e0410c6a0 2024-03-30T02:40:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.047+0000 7f9e0ba00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e041bf230 con 0x7f9e04108020 2024-03-30T02:40:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.047+0000 7f9e09000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e04108020 0x7f9e04081c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.047+0000 7f9e09000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e04108020 0x7f9e04081c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60162/0 (socket says 172.21.15.67:60162) 2024-03-30T02:40:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.047+0000 7f9e09000700 1 -- 172.21.15.67:0/1065439596 learned_addr learned my addr 172.21.15.67:0/1065439596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.048+0000 7f9e09000700 1 -- 172.21.15.67:0/1065439596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e0410c6a0 msgr2=0x7f9e04082710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.048+0000 7f9e09000700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e0410c6a0 0x7f9e04082710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.048+0000 7f9e09000700 1 -- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e04108940 msgr2=0x7f9e040821d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.048+0000 7f9e03e00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e04108940 0x7f9e040821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.049+0000 7f9e09000700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e04108940 0x7f9e040821d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.049+0000 7f9e09000700 1 -- 172.21.15.67:0/1065439596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8007220 con 0x7f9e04108020 2024-03-30T02:40:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.049+0000 7f9e03e00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e04108940 0x7f9e040821d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.049+0000 7f9e09000700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e04108020 0x7f9e04081c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9df8004980 tx=0x7f9df800a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.051+0000 7f9e01600700 1 -- 172.21.15.67:0/1065439596 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800f070 con 0x7f9e04108020 2024-03-30T02:40:15.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.051+0000 7f9e01600700 1 -- 172.21.15.67:0/1065439596 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9df8007ba0 con 0x7f9e04108020 2024-03-30T02:40:15.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.051+0000 7f9e01600700 1 -- 172.21.15.67:0/1065439596 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800c860 con 0x7f9e04108020 2024-03-30T02:40:15.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.051+0000 7f9e0ba00700 1 -- 172.21.15.67:0/1065439596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e041bf4b0 con 0x7f9e04108020 2024-03-30T02:40:15.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.052+0000 7f9e0ba00700 1 -- 172.21.15.67:0/1065439596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e041bf910 con 0x7f9e04108020 2024-03-30T02:40:15.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.061+0000 7f9e01600700 1 -- 172.21.15.67:0/1065439596 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f9df800c9c0 con 0x7f9e04108020 2024-03-30T02:40:15.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.065+0000 7f9e1f603b80 1 Processor -- start 2024-03-30T02:40:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.065+0000 7f9e01600700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9dec0a1310 0x7f9dec0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.065+0000 7f9e01600700 1 -- 172.21.15.67:0/1065439596 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9df80cd950 con 0x7f9e04108020 2024-03-30T02:40:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.066+0000 7f9e1f603b80 1 -- start start 2024-03-30T02:40:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.066+0000 7f9e09a00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9dec0a1310 0x7f9dec0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.067+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a4e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.067+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4eef0 0x5649a9a620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.067+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a62600 0x5649a9a649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.067+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a99a7990 con 0x5649a9a62600 2024-03-30T02:40:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.067+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a99a5350 con 0x5649a9a4eef0 2024-03-30T02:40:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.067+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a99a0c40 con 0x5649a9a4e5d0 2024-03-30T02:40:15.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.068+0000 7f9e09a00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9dec0a1310 0x7f9dec0a37d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00aba0 tx=0x7f9dfc00af60 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.068+0000 7f9e09000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a4e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.068+0000 7f9e09a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a62600 0x5649a9a649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.068+0000 7f9e09000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a4e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60176/0 (socket says 172.21.15.67:60176) 2024-03-30T02:40:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.068+0000 7f9e09000700 1 -- 172.21.15.67:0/3180573985 learned_addr learned my addr 172.21.15.67:0/3180573985 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.069+0000 7f9e09000700 1 -- 172.21.15.67:0/3180573985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4eef0 msgr2=0x5649a9a620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.069+0000 7f9e09000700 1 --2- 172.21.15.67:0/3180573985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4eef0 0x5649a9a620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.069+0000 7f9e09000700 1 -- 172.21.15.67:0/3180573985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a62600 msgr2=0x5649a9a649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.069+0000 7f9e09000700 1 --2- 172.21.15.67:0/3180573985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a62600 0x5649a9a649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.069+0000 7f9e09000700 1 -- 172.21.15.67:0/3180573985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649a98223b0 con 0x5649a9a4e5d0 2024-03-30T02:40:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.069+0000 7f9e09a00700 1 --2- 172.21.15.67:0/3180573985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a62600 0x5649a9a649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:15.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.070+0000 7f9e09000700 1 --2- 172.21.15.67:0/3180573985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a4e9b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9df809b570 tx=0x7f9df80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dfc8d3be746e7c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.070+0000 7f9df6000700 1 -- 172.21.15.67:0/3180573985 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8007d10 con 0x5649a9a4e5d0 2024-03-30T02:40:15.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.070+0000 7f9df6000700 1 -- 172.21.15.67:0/3180573985 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9df809bce0 con 0x5649a9a4e5d0 2024-03-30T02:40:15.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.070+0000 7f9df6000700 1 -- 172.21.15.67:0/3180573985 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8012390 con 0x5649a9a4e5d0 2024-03-30T02:40:15.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.072+0000 7f9e1f603b80 1 -- 172.21.15.67:0/3180573985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 msgr2=0x5649a9a4e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.072+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/3180573985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a4e9b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9df809b570 tx=0x7f9df80057f0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.072+0000 7f9e1f603b80 1 -- 172.21.15.67:0/3180573985 shutdown_connections 2024-03-30T02:40:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.072+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/3180573985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a62600 0x5649a9a649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.072+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/3180573985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a4e9b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.072+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/3180573985 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4eef0 0x5649a9a620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.072+0000 7f9e1f603b80 1 -- 172.21.15.67:0/3180573985 >> 172.21.15.67:0/3180573985 conn(0x5649a99ae6a0 msgr2=0x5649a9a4aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.076+0000 7f9e1f603b80 1 -- 172.21.15.67:0/3180573985 shutdown_connections 2024-03-30T02:40:15.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.076+0000 7f9e1f603b80 1 -- 172.21.15.67:0/3180573985 wait complete. 2024-03-30T02:40:15.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.077+0000 7f9e1f603b80 1 -- start start 2024-03-30T02:40:15.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.077+0000 7f9e1f603b80 1 Processor -- start 2024-03-30T02:40:15.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.079+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.079+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a5f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.079+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a5faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.079+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a993cd80 con 0x5649a9a4eef0 2024-03-30T02:40:15.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.079+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a993ca20 con 0x5649a9a4e5d0 2024-03-30T02:40:15.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.079+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a98246a0 con 0x5649a9a62600 2024-03-30T02:40:15.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.079+0000 7f9e1f603b80 10 client.0 _ll_register_callbacks cb 0x5649a9946470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:15.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.080+0000 7f9e09000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.080+0000 7f9e09000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a620c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57558/0 (socket says 172.21.15.67:57558) 2024-03-30T02:40:15.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.080+0000 7f9e09000700 1 -- 172.21.15.67:0/2668023368 learned_addr learned my addr 172.21.15.67:0/2668023368 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.080+0000 7f9e03e00700 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a5f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.080+0000 7f9e09a00700 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a5faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.081+0000 7f9e09000700 1 -- 172.21.15.67:0/2668023368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 msgr2=0x5649a9a5faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.081+0000 7f9e09000700 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a5faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.081+0000 7f9e09000700 1 -- 172.21.15.67:0/2668023368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 msgr2=0x5649a9a5f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.081+0000 7f9e09000700 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a5f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.081+0000 7f9e09000700 1 -- 172.21.15.67:0/2668023368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc00caf0 con 0x5649a9a4e5d0 2024-03-30T02:40:15.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.082+0000 7f9e09a00700 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a5faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:40:15.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.083+0000 7f9e09000700 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a620c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9df800e010 tx=0x7f9df8004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18fb6395dcde30a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.083+0000 7f9deaa00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:40:15.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.084+0000 7f9deaa00700 1 -- 172.21.15.67:0/2668023368 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df809f410 con 0x5649a9a4e5d0 2024-03-30T02:40:15.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.084+0000 7f9deaa00700 1 -- 172.21.15.67:0/2668023368 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9df80a39d0 con 0x5649a9a4e5d0 2024-03-30T02:40:15.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.084+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2668023368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649a9961380 con 0x5649a9a4e5d0 2024-03-30T02:40:15.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.086+0000 7f9deaa00700 1 -- 172.21.15.67:0/2668023368 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9df8012390 con 0x5649a9a4e5d0 2024-03-30T02:40:15.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.087+0000 7f9e1f603b80 10 client.26489 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:15.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.087+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2668023368 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f9e041bf910 con 0x5649a9a4e5d0 2024-03-30T02:40:15.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.089+0000 7f9deaa00700 1 -- 172.21.15.67:0/2668023368 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649a9961380 con 0x5649a9a4e5d0 2024-03-30T02:40:15.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.089+0000 7f9deaa00700 1 -- 172.21.15.67:0/2668023368 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9df809aad0 con 0x5649a9a4e5d0 2024-03-30T02:40:15.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.090+0000 7f9e1f603b80 10 client.26489 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:15.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.090+0000 7f9e1f603b80 10 client.26489 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:40:15.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.090+0000 7f9e1f603b80 20 client.26489 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:15.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.090+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5649a9a71bc0 0x5649a9a73fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.090+0000 7f9e1f603b80 4 client.26489 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:40:15.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.090+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2668023368 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649a9999700 con 0x5649a9a71bc0 2024-03-30T02:40:15.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.090+0000 7f9e03e00700 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5649a9a71bc0 0x5649a9a73fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.092+0000 7f9e03e00700 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5649a9a71bc0 0x5649a9a73fb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ff854d4d132ed9c server_cookie=511e34c7e7bef39e in_seq=0 out_seq=0 2024-03-30T02:40:15.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.092+0000 7f9deaa00700 10 client.26489 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:40:15.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.097+0000 7f9deaa00700 1 -- 172.21.15.67:0/2668023368 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5649a9999700 con 0x5649a9a71bc0 2024-03-30T02:40:15.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.097+0000 7f9deaa00700 10 client.26489 handle_command_reply: tid=0 2024-03-30T02:40:15.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.097+0000 7f9e1f603b80 1 client.26489 shutdown 2024-03-30T02:40:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.098+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2668023368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 msgr2=0x5649a9a620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.098+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a620c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9df800e010 tx=0x7f9df8004f70 comp rx=0 tx=0).stop 2024-03-30T02:40:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.098+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2668023368 shutdown_connections 2024-03-30T02:40:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.098+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5649a9a71bc0 0x5649a9a73fb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.098+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a5faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.098+0000 7f9deaa00700 0 client.26489 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:40:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.098+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a620c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.098+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2668023368 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a5f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.099+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2668023368 >> 172.21.15.67:0/2668023368 conn(0x5649a99ae6a0 msgr2=0x5649a9a506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.099+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2668023368 shutdown_connections 2024-03-30T02:40:15.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.099+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2668023368 wait complete. 2024-03-30T02:40:15.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.100+0000 7f9e1f603b80 20 client.26489 trim_cache size 0 max 16384 2024-03-30T02:40:15.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.131+0000 7f9e1f603b80 1 Processor -- start 2024-03-30T02:40:15.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.131+0000 7f9e1f603b80 1 -- start start 2024-03-30T02:40:15.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.132+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a46480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.132+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a49f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.132+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a69090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.132+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a9961380 con 0x5649a9a4eef0 2024-03-30T02:40:15.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.132+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a9a4b030 con 0x5649a9a4e5d0 2024-03-30T02:40:15.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.132+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a9a6e560 con 0x5649a9a62600 2024-03-30T02:40:15.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.133+0000 7f9e03e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a49f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.133+0000 7f9e09000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a46480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.133+0000 7f9e09a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a69090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.133+0000 7f9e09a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a69090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60202/0 (socket says 172.21.15.67:60202) 2024-03-30T02:40:15.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.133+0000 7f9e09a00700 1 -- 172.21.15.67:0/2341416595 learned_addr learned my addr 172.21.15.67:0/2341416595 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.133+0000 7f9e09000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a46480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57564/0 (socket says 172.21.15.67:57564) 2024-03-30T02:40:15.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.134+0000 7f9e09a00700 1 -- 172.21.15.67:0/2341416595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 msgr2=0x5649a9a46480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.134+0000 7f9e09a00700 1 --2- 172.21.15.67:0/2341416595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a46480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.134+0000 7f9e09a00700 1 -- 172.21.15.67:0/2341416595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 msgr2=0x5649a9a49f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.134+0000 7f9e09a00700 1 --2- 172.21.15.67:0/2341416595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a49f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.134+0000 7f9e09a00700 1 -- 172.21.15.67:0/2341416595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649a97ceb60 con 0x5649a9a62600 2024-03-30T02:40:15.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.135+0000 7f9e09a00700 1 --2- 172.21.15.67:0/2341416595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a69090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00d5b0 tx=0x7f9dfc015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=448da18dca9d6851 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.136+0000 7f9df6a00700 1 -- 172.21.15.67:0/2341416595 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc019600 con 0x5649a9a62600 2024-03-30T02:40:15.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.136+0000 7f9df6a00700 1 -- 172.21.15.67:0/2341416595 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dfc01f040 con 0x5649a9a62600 2024-03-30T02:40:15.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.136+0000 7f9df6a00700 1 -- 172.21.15.67:0/2341416595 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc023af0 con 0x5649a9a62600 2024-03-30T02:40:15.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.139+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2341416595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 msgr2=0x5649a9a69090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.139+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2341416595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a69090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00d5b0 tx=0x7f9dfc015000 comp rx=0 tx=0).stop 2024-03-30T02:40:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.140+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2341416595 shutdown_connections 2024-03-30T02:40:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.140+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2341416595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a62600 0x5649a9a69090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.140+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2341416595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a4e5d0 0x5649a9a46480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.140+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/2341416595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a49f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.140+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2341416595 >> 172.21.15.67:0/2341416595 conn(0x5649a99ae6a0 msgr2=0x5649a9a435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.140+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2341416595 shutdown_connections 2024-03-30T02:40:15.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.140+0000 7f9e1f603b80 1 -- 172.21.15.67:0/2341416595 wait complete. 2024-03-30T02:40:15.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.141+0000 7f9e1f603b80 1 -- start start 2024-03-30T02:40:15.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.141+0000 7f9e1f603b80 1 Processor -- start 2024-03-30T02:40:15.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.143+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a68fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.143+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a67670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.143+0000 7f9e1f603b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a62600 0x5649a9a67bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.144+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a9a681d0 con 0x5649a9a4eef0 2024-03-30T02:40:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.144+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a9a68340 con 0x5649a9a62600 2024-03-30T02:40:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.144+0000 7f9e1f603b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a9a68500 con 0x5649a9a4e5d0 2024-03-30T02:40:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.144+0000 7f9e1f603b80 10 client.0 _ll_register_callbacks cb 0x5649a9946470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.144+0000 7f9e03e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a67670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.144+0000 7f9e09000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a68fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.144+0000 7f9e03e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a67670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44942/0 (socket says 172.21.15.67:44942) 2024-03-30T02:40:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.144+0000 7f9e03e00700 1 -- 172.21.15.67:0/695124983 learned_addr learned my addr 172.21.15.67:0/695124983 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.144+0000 7f9e09a00700 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a62600 0x5649a9a67bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.145+0000 7f9e03e00700 1 -- 172.21.15.67:0/695124983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 msgr2=0x5649a9a68fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.145+0000 7f9e03e00700 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a68fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.145+0000 7f9e03e00700 1 -- 172.21.15.67:0/695124983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a62600 msgr2=0x5649a9a67bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.145+0000 7f9e03e00700 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a62600 0x5649a9a67bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.145+0000 7f9e03e00700 1 -- 172.21.15.67:0/695124983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc00ce90 con 0x5649a9a4eef0 2024-03-30T02:40:15.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.147+0000 7f9e03e00700 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a67670 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5649a9a608f0 tx=0x7f9df000ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484ed431bd5ffb93 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.147+0000 7f9deaa00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:40:15.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.149+0000 7f9deaa00700 1 -- 172.21.15.67:0/695124983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00194e0 con 0x5649a9a4eef0 2024-03-30T02:40:15.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.149+0000 7f9deaa00700 1 -- 172.21.15.67:0/695124983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9df0022440 con 0x5649a9a4eef0 2024-03-30T02:40:15.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.149+0000 7f9e1f603b80 1 -- 172.21.15.67:0/695124983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9df809bce0 con 0x5649a9a4eef0 2024-03-30T02:40:15.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.150+0000 7f9deaa00700 1 -- 172.21.15.67:0/695124983 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9df0022600 con 0x5649a9a4eef0 2024-03-30T02:40:15.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.150+0000 7f9e1f603b80 10 client.16431 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:15.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.150+0000 7f9e1f603b80 1 -- 172.21.15.67:0/695124983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5649a9966ea0 con 0x5649a9a4eef0 2024-03-30T02:40:15.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.150+0000 7f9deaa00700 1 -- 172.21.15.67:0/695124983 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9df809bce0 con 0x5649a9a4eef0 2024-03-30T02:40:15.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.151+0000 7f9deaa00700 1 -- 172.21.15.67:0/695124983 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9df0017880 con 0x5649a9a4eef0 2024-03-30T02:40:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.151+0000 7f9e1f603b80 10 client.16431 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.151+0000 7f9e1f603b80 10 client.16431 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:40:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.151+0000 7f9e1f603b80 20 client.16431 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.151+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5649a9a71bc0 0x5649a9ac8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.151+0000 7f9e1f603b80 4 client.16431 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:40:15.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.151+0000 7f9e1f603b80 1 -- 172.21.15.67:0/695124983 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5649a97efcf0 con 0x5649a9a71bc0 2024-03-30T02:40:15.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.152+0000 7f9e09000700 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5649a9a71bc0 0x5649a9ac8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.154+0000 7f9e09000700 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5649a9a71bc0 0x5649a9ac8fa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=37d666b35a8846f6 server_cookie=7c3858b4bdb04cb in_seq=0 out_seq=0 2024-03-30T02:40:15.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.154+0000 7f9deaa00700 10 client.16431 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:40:15.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.154+0000 7f9deaa00700 1 -- 172.21.15.67:0/695124983 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5649a97efcf0 con 0x5649a9a71bc0 2024-03-30T02:40:15.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.154+0000 7f9deaa00700 10 client.16431 handle_command_reply: tid=0 2024-03-30T02:40:15.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.155+0000 7f9e1f603b80 1 client.16431 shutdown 2024-03-30T02:40:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.155+0000 7f9e1f603b80 1 -- 172.21.15.67:0/695124983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 msgr2=0x5649a9a67670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.155+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a67670 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5649a9a608f0 tx=0x7f9df000ae80 comp rx=0 tx=0).stop 2024-03-30T02:40:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.156+0000 7f9e1f603b80 1 -- 172.21.15.67:0/695124983 shutdown_connections 2024-03-30T02:40:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.156+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5649a9a71bc0 0x5649a9ac8fa0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.156+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649a9a62600 0x5649a9a67bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.156+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649a9a4e5d0 0x5649a9a68fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.156+0000 7f9e1f603b80 1 --2- 172.21.15.67:0/695124983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649a9a4eef0 0x5649a9a67670 secure :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5649a9a608f0 tx=0x7f9df000ae80 comp rx=0 tx=0).stop 2024-03-30T02:40:15.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.156+0000 7f9deaa00700 0 client.16431 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:40:15.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.156+0000 7f9e1f603b80 1 -- 172.21.15.67:0/695124983 >> 172.21.15.67:0/695124983 conn(0x5649a99ae6a0 msgr2=0x5649a9a61d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.156+0000 7f9e1f603b80 1 -- 172.21.15.67:0/695124983 shutdown_connections 2024-03-30T02:40:15.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.156+0000 7f9e1f603b80 1 -- 172.21.15.67:0/695124983 wait complete. 2024-03-30T02:40:15.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.158+0000 7f9e1f603b80 20 client.16431 trim_cache size 0 max 16384 2024-03-30T02:40:15.158 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:40:15.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.159+0000 7f9e0ba00700 1 -- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9dec0a1310 msgr2=0x7f9dec0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.159+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9dec0a1310 0x7f9dec0a37d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00aba0 tx=0x7f9dfc00af60 comp rx=0 tx=0).stop 2024-03-30T02:40:15.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.159+0000 7f9e0ba00700 1 -- 172.21.15.67:0/1065439596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e04108020 msgr2=0x7f9e04081c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.159+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e04108020 0x7f9e04081c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9df8004980 tx=0x7f9df800a210 comp rx=0 tx=0).stop 2024-03-30T02:40:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.161+0000 7f9e0ba00700 1 -- 172.21.15.67:0/1065439596 shutdown_connections 2024-03-30T02:40:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.161+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f9dec0a1310 0x7f9dec0a37d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.161+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e0410c6a0 0x7f9e04082710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.161+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e04108020 0x7f9e04081c90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.161+0000 7f9e0ba00700 1 --2- 172.21.15.67:0/1065439596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9e04108940 0x7f9e040821d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.161+0000 7f9e0ba00700 1 -- 172.21.15.67:0/1065439596 >> 172.21.15.67:0/1065439596 conn(0x7f9e0406bdf0 msgr2=0x7f9e0410b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.162+0000 7f9e0ba00700 1 -- 172.21.15.67:0/1065439596 shutdown_connections 2024-03-30T02:40:15.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.162+0000 7f9e0ba00700 1 -- 172.21.15.67:0/1065439596 wait complete. 2024-03-30T02:40:15.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:40:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.595+0000 7f7ca8800700 1 -- 172.21.15.67:0/2871024203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0105ca0 msgr2=0x7f7ca0067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.595+0000 7f7ca8800700 1 --2- 172.21.15.67:0/2871024203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0105ca0 0x7f7ca0067e30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009990 tx=0x7f7c9c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.596+0000 7f7ca8800700 1 -- 172.21.15.67:0/2871024203 shutdown_connections 2024-03-30T02:40:15.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.596+0000 7f7ca8800700 1 --2- 172.21.15.67:0/2871024203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca0103740 0x7f7ca0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.596+0000 7f7ca8800700 1 --2- 172.21.15.67:0/2871024203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0105ca0 0x7f7ca0067e30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.596+0000 7f7ca8800700 1 --2- 172.21.15.67:0/2871024203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ca0068370 0x7f7ca00687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.596+0000 7f7ca8800700 1 -- 172.21.15.67:0/2871024203 >> 172.21.15.67:0/2871024203 conn(0x7f7ca0076d40 msgr2=0x7f7ca0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.598+0000 7f7ca8800700 1 -- 172.21.15.67:0/2871024203 shutdown_connections 2024-03-30T02:40:15.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.598+0000 7f7ca8800700 1 -- 172.21.15.67:0/2871024203 wait complete. 2024-03-30T02:40:15.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.600+0000 7f7ca8800700 1 Processor -- start 2024-03-30T02:40:15.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.601+0000 7f7ca8800700 1 -- start start 2024-03-30T02:40:15.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.601+0000 7f7ca8800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0068370 0x7f7ca01a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.601+0000 7f7ca8800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca0103740 0x7f7ca01a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.601+0000 7f7ca8800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ca0105ca0 0x7f7ca019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.602+0000 7f7ca5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0068370 0x7f7ca01a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.602+0000 7f7ca5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0068370 0x7f7ca01a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60226/0 (socket says 172.21.15.67:60226) 2024-03-30T02:40:15.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.602+0000 7f7ca5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca0103740 0x7f7ca01a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.602+0000 7f7ca5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca0103740 0x7f7ca01a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57584/0 (socket says 172.21.15.67:57584) 2024-03-30T02:40:15.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.602+0000 7f7ca8800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca019dda0 con 0x7f7ca0105ca0 2024-03-30T02:40:15.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.602+0000 7f7ca8800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca019df10 con 0x7f7ca0103740 2024-03-30T02:40:15.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.602+0000 7f7ca8800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca019e0d0 con 0x7f7ca0068370 2024-03-30T02:40:15.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.603+0000 7f7ca5e00700 1 -- 172.21.15.67:0/3491582792 learned_addr learned my addr 172.21.15.67:0/3491582792 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.603+0000 7f7ca6800700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ca0105ca0 0x7f7ca019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.604+0000 7f7ca5400700 1 -- 172.21.15.67:0/3491582792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0068370 msgr2=0x7f7ca01a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.604+0000 7f7ca5400700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0068370 0x7f7ca01a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.604+0000 7f7ca5400700 1 -- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ca0105ca0 msgr2=0x7f7ca019d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.604+0000 7f7ca5400700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ca0105ca0 0x7f7ca019d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.604+0000 7f7ca5400700 1 -- 172.21.15.67:0/3491582792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c0095f0 con 0x7f7ca0103740 2024-03-30T02:40:15.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.604+0000 7f7ca5e00700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0068370 0x7f7ca01a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:15.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.604+0000 7f7ca6800700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ca0105ca0 0x7f7ca019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:15.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.605+0000 7f7ca5400700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca0103740 0x7f7ca01a3b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000b3c0 tx=0x7f7c9000b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.606+0000 7f7c9aa00700 1 -- 172.21.15.67:0/3491582792 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90011620 con 0x7f7ca0103740 2024-03-30T02:40:15.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.606+0000 7f7c9aa00700 1 -- 172.21.15.67:0/3491582792 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c90011c40 con 0x7f7ca0103740 2024-03-30T02:40:15.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.606+0000 7f7ca8800700 1 -- 172.21.15.67:0/3491582792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca019e3b0 con 0x7f7ca0103740 2024-03-30T02:40:15.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.606+0000 7f7c9aa00700 1 -- 172.21.15.67:0/3491582792 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90010c50 con 0x7f7ca0103740 2024-03-30T02:40:15.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.606+0000 7f7ca8800700 1 -- 172.21.15.67:0/3491582792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca01aa5f0 con 0x7f7ca0103740 2024-03-30T02:40:15.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.617+0000 7f7c9aa00700 1 -- 172.21.15.67:0/3491582792 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7c90011780 con 0x7f7ca0103740 2024-03-30T02:40:15.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.620+0000 7f7c9aa00700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7c880a1290 0x7f7c880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.621+0000 7f7ca5e00700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7c880a1290 0x7f7c880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.622+0000 7f7cbc229b80 1 Processor -- start 2024-03-30T02:40:15.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.620+0000 7f7c9aa00700 1 -- 172.21.15.67:0/3491582792 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f7c900cc7e0 con 0x7f7ca0103740 2024-03-30T02:40:15.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.622+0000 7f7cbc229b80 1 -- start start 2024-03-30T02:40:15.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.623+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e416455d0 0x558e416459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.623+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 0x558e416508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.623+0000 7f7ca5e00700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7c880a1290 0x7f7c880a3750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c003800 tx=0x7f7c9c00b540 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.623+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e416531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.623+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e4159e990 con 0x558e416455d0 2024-03-30T02:40:15.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.623+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e4159c350 con 0x558e41645ef0 2024-03-30T02:40:15.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.623+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e41597c40 con 0x558e41650df0 2024-03-30T02:40:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e416455d0 0x558e416459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca6800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e416531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca6800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e416531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60240/0 (socket says 172.21.15.67:60240) 2024-03-30T02:40:15.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 0x558e416508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca6800700 1 -- 172.21.15.67:0/2315285894 learned_addr learned my addr 172.21.15.67:0/2315285894 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca6800700 1 -- 172.21.15.67:0/2315285894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 msgr2=0x558e416508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca6800700 1 --2- 172.21.15.67:0/2315285894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 0x558e416508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca6800700 1 -- 172.21.15.67:0/2315285894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e416455d0 msgr2=0x558e416459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca6800700 1 --2- 172.21.15.67:0/2315285894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e416455d0 0x558e416459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.624+0000 7f7ca6800700 1 -- 172.21.15.67:0/2315285894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e414193b0 con 0x558e41650df0 2024-03-30T02:40:15.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.625+0000 7f7ca6800700 1 --2- 172.21.15.67:0/2315285894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e416531e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7c940076f0 tx=0x7f7c94007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f61352b9082af2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.625+0000 7f7c8f000700 1 -- 172.21.15.67:0/2315285894 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94003e10 con 0x558e41650df0 2024-03-30T02:40:15.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.625+0000 7f7c8f000700 1 -- 172.21.15.67:0/2315285894 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c9400f040 con 0x558e41650df0 2024-03-30T02:40:15.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.626+0000 7f7c8f000700 1 -- 172.21.15.67:0/2315285894 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94015960 con 0x558e41650df0 2024-03-30T02:40:15.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.628+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2315285894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 msgr2=0x558e416531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.628+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/2315285894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e416531e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7c940076f0 tx=0x7f7c94007a00 comp rx=0 tx=0).stop 2024-03-30T02:40:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.629+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2315285894 shutdown_connections 2024-03-30T02:40:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.629+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/2315285894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e416455d0 0x558e416459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.629+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/2315285894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e416531e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.629+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/2315285894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 0x558e416508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.629+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2315285894 >> 172.21.15.67:0/2315285894 conn(0x558e415a56a0 msgr2=0x558e41641a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.630+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2315285894 shutdown_connections 2024-03-30T02:40:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.630+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2315285894 wait complete. 2024-03-30T02:40:15.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.631+0000 7f7cbc229b80 1 -- start start 2024-03-30T02:40:15.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.634+0000 7f7cbc229b80 1 Processor -- start 2024-03-30T02:40:15.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.635+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e416507c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.635+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41645ef0 0x558e4164dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.635+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e4164e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.635+0000 7f7ca5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e416507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.636+0000 7f7ca5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e416507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57604/0 (socket says 172.21.15.67:57604) 2024-03-30T02:40:15.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.636+0000 7f7ca5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41645ef0 0x558e4164dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.636+0000 7f7ca5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41645ef0 0x558e4164dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60246/0 (socket says 172.21.15.67:60246) 2024-03-30T02:40:15.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.637+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e41533d80 con 0x558e41650df0 2024-03-30T02:40:15.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.637+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e41533a20 con 0x558e416455d0 2024-03-30T02:40:15.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.637+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e4141b6a0 con 0x558e41645ef0 2024-03-30T02:40:15.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.637+0000 7f7cbc229b80 10 client.0 _ll_register_callbacks cb 0x558e4153d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:15.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.637+0000 7f7ca5e00700 1 -- 172.21.15.67:0/3479456728 learned_addr learned my addr 172.21.15.67:0/3479456728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.638+0000 7f7ca6800700 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e4164e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.639+0000 7f7ca5e00700 1 -- 172.21.15.67:0/3479456728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41645ef0 msgr2=0x558e4164dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.639+0000 7f7ca5e00700 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41645ef0 0x558e4164dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.639+0000 7f7ca5e00700 1 -- 172.21.15.67:0/3479456728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 msgr2=0x558e4164e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.639+0000 7f7ca5e00700 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e4164e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.639+0000 7f7ca5e00700 1 -- 172.21.15.67:0/3479456728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c94009e20 con 0x558e416455d0 2024-03-30T02:40:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.639+0000 7f7ca6800700 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e4164e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.640+0000 7f7ca5e00700 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e416507c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005110 tx=0x7f7c9c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff8ffd608e6d515 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.640+0000 7f7c83400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:40:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.640+0000 7f7c83400700 1 -- 172.21.15.67:0/3479456728 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c024a40 con 0x558e416455d0 2024-03-30T02:40:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.640+0000 7f7c83400700 1 -- 172.21.15.67:0/3479456728 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f7c9c024ba0 con 0x558e416455d0 2024-03-30T02:40:15.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.640+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3479456728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e41558380 con 0x558e416455d0 2024-03-30T02:40:15.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.641+0000 7f7c83400700 1 -- 172.21.15.67:0/3479456728 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c9c024d60 con 0x558e416455d0 2024-03-30T02:40:15.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.641+0000 7f7cbc229b80 10 client.26507 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:15.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.641+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3479456728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558e413c5b60 con 0x558e416455d0 2024-03-30T02:40:15.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.641+0000 7f7c83400700 1 -- 172.21.15.67:0/3479456728 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e41558380 con 0x558e416455d0 2024-03-30T02:40:15.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.643+0000 7f7c83400700 1 -- 172.21.15.67:0/3479456728 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7c9c010470 con 0x558e416455d0 2024-03-30T02:40:15.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.643+0000 7f7cbc229b80 10 client.26507 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:15.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.643+0000 7f7cbc229b80 10 client.26507 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:40:15.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.643+0000 7f7cbc229b80 20 client.26507 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:15.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.643+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x558e41660980 0x558e41662da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.643+0000 7f7cbc229b80 4 client.26507 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:40:15.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.643+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3479456728 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e41590700 con 0x558e41660980 2024-03-30T02:40:15.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.644+0000 7f7ca5400700 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x558e41660980 0x558e41662da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.645+0000 7f7c83400700 10 client.26507 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:40:15.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.645+0000 7f7ca5400700 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x558e41660980 0x558e41662da0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f8af57179c4cbc9 server_cookie=177c1ab54b8af2b5 in_seq=0 out_seq=0 2024-03-30T02:40:15.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.645+0000 7f7c83400700 1 -- 172.21.15.67:0/3479456728 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e41590700 con 0x558e41660980 2024-03-30T02:40:15.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.645+0000 7f7c83400700 10 client.26507 handle_command_reply: tid=0 2024-03-30T02:40:15.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.646+0000 7f7cbc229b80 1 client.26507 shutdown 2024-03-30T02:40:15.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.647+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3479456728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 msgr2=0x558e416507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.647+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e416507c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005110 tx=0x7f7c9c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.647+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3479456728 shutdown_connections 2024-03-30T02:40:15.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.647+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x558e41660980 0x558e41662da0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.647+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e4164e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.647+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e416507c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.647+0000 7f7c83400700 0 client.26507 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:40:15.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.647+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3479456728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41645ef0 0x558e4164dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.647+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3479456728 >> 172.21.15.67:0/3479456728 conn(0x558e415a56a0 msgr2=0x558e416476c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.648+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3479456728 shutdown_connections 2024-03-30T02:40:15.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.648+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3479456728 wait complete. 2024-03-30T02:40:15.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.649+0000 7f7cbc229b80 20 client.26507 trim_cache size 0 max 16384 2024-03-30T02:40:15.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.685+0000 7f7cbc229b80 1 Processor -- start 2024-03-30T02:40:15.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.685+0000 7f7cbc229b80 1 -- start start 2024-03-30T02:40:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.686+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e416455d0 0x558e41648510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.686+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 0x558e4164e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.687+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e41657aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.687+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e41657fe0 con 0x558e41650df0 2024-03-30T02:40:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.687+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e41641f20 con 0x558e41645ef0 2024-03-30T02:40:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.687+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e41660710 con 0x558e416455d0 2024-03-30T02:40:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.687+0000 7f7ca5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e416455d0 0x558e41648510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.687+0000 7f7ca5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 0x558e4164e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.687+0000 7f7ca5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e416455d0 0x558e41648510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:60248/0 (socket says 172.21.15.67:60248) 2024-03-30T02:40:15.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.687+0000 7f7ca5e00700 1 -- 172.21.15.67:0/2006675155 learned_addr learned my addr 172.21.15.67:0/2006675155 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.687+0000 7f7ca6800700 1 --2- 172.21.15.67:0/2006675155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e41657aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7ca5400700 1 -- 172.21.15.67:0/2006675155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e416455d0 msgr2=0x558e41648510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7ca5400700 1 --2- 172.21.15.67:0/2006675155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e416455d0 0x558e41648510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7ca5400700 1 -- 172.21.15.67:0/2006675155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 msgr2=0x558e41657aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7ca5400700 1 --2- 172.21.15.67:0/2006675155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e41657aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7ca5400700 1 -- 172.21.15.67:0/2006675155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e4155dea0 con 0x558e41645ef0 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7ca6800700 1 --2- 172.21.15.67:0/2006675155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e41657aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7ca5400700 1 --2- 172.21.15.67:0/2006675155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 0x558e4164e090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c900c9cf0 tx=0x7f7c900b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2c0f40248c85e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7c8fa00700 1 -- 172.21.15.67:0/2006675155 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9009d4e0 con 0x558e41645ef0 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7c8fa00700 1 -- 172.21.15.67:0/2006675155 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c900a1b30 con 0x558e41645ef0 2024-03-30T02:40:15.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.688+0000 7f7c8fa00700 1 -- 172.21.15.67:0/2006675155 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90010dd0 con 0x558e41645ef0 2024-03-30T02:40:15.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.690+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2006675155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 msgr2=0x558e4164e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.690+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/2006675155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 0x558e4164e090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c900c9cf0 tx=0x7f7c900b9040 comp rx=0 tx=0).stop 2024-03-30T02:40:15.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.691+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2006675155 shutdown_connections 2024-03-30T02:40:15.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.691+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/2006675155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41650df0 0x558e41657aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.691+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/2006675155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e416455d0 0x558e41648510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.691+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/2006675155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e41645ef0 0x558e4164e090 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.691+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2006675155 >> 172.21.15.67:0/2006675155 conn(0x558e415a56a0 msgr2=0x558e4163a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.691+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2006675155 shutdown_connections 2024-03-30T02:40:15.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.691+0000 7f7cbc229b80 1 -- 172.21.15.67:0/2006675155 wait complete. 2024-03-30T02:40:15.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.692+0000 7f7cbc229b80 1 -- start start 2024-03-30T02:40:15.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.693+0000 7f7cbc229b80 1 Processor -- start 2024-03-30T02:40:15.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.694+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e41657290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41645ef0 0x558e416577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7cbc229b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e41655990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e41655f20 con 0x558e41645ef0 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e41656090 con 0x558e416455d0 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7cbc229b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e41656250 con 0x558e41650df0 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7cbc229b80 10 client.0 _ll_register_callbacks cb 0x558e4153d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7ca5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e41657290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7ca5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e41657290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57620/0 (socket says 172.21.15.67:57620) 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7ca5e00700 1 -- 172.21.15.67:0/3271331853 learned_addr learned my addr 172.21.15.67:0/3271331853 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:15.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.695+0000 7f7ca6800700 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e41655990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.696+0000 7f7ca5400700 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41645ef0 0x558e416577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.696+0000 7f7ca5e00700 1 -- 172.21.15.67:0/3271331853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 msgr2=0x558e41655990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.697+0000 7f7ca5e00700 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e41655990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.697+0000 7f7ca5e00700 1 -- 172.21.15.67:0/3271331853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41645ef0 msgr2=0x558e416577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.697+0000 7f7ca5e00700 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41645ef0 0x558e416577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.697+0000 7f7ca5e00700 1 -- 172.21.15.67:0/3271331853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca01aa5f0 con 0x558e416455d0 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.697+0000 7f7ca5400700 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41645ef0 0x558e416577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.697+0000 7f7ca5e00700 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e41657290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005110 tx=0x7f7c9c0106a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1592f777711e06d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.697+0000 7f7c83400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.697+0000 7f7c83400700 1 -- 172.21.15.67:0/3271331853 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c024c50 con 0x558e416455d0 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.697+0000 7f7c83400700 1 -- 172.21.15.67:0/3271331853 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f7c9c024db0 con 0x558e416455d0 2024-03-30T02:40:15.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.698+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3271331853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c9400f040 con 0x558e416455d0 2024-03-30T02:40:15.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.698+0000 7f7c83400700 1 -- 172.21.15.67:0/3271331853 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c9c009290 con 0x558e416455d0 2024-03-30T02:40:15.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.698+0000 7f7cbc229b80 10 client.26513 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:15.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.698+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3271331853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558e4141dcd0 con 0x558e416455d0 2024-03-30T02:40:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.698+0000 7f7c83400700 1 -- 172.21.15.67:0/3271331853 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c9400f040 con 0x558e416455d0 2024-03-30T02:40:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.699+0000 7f7c83400700 1 -- 172.21.15.67:0/3271331853 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7c9c0173f0 con 0x558e416455d0 2024-03-30T02:40:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.699+0000 7f7cbc229b80 10 client.26513 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.699+0000 7f7cbc229b80 10 client.26513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:40:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.699+0000 7f7cbc229b80 20 client.26513 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.699+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x558e416b5560 0x558e416b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.699+0000 7f7cbc229b80 4 client.26513 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:40:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.699+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3271331853 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e413e6cf0 con 0x558e416b5560 2024-03-30T02:40:15.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.700+0000 7f7ca5400700 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x558e416b5560 0x558e416b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:15.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.701+0000 7f7c83400700 10 client.26513 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:40:15.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.701+0000 7f7ca5400700 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x558e416b5560 0x558e416b7980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8247d8a0eca11112 server_cookie=f3fdb36bda29eda1 in_seq=0 out_seq=0 2024-03-30T02:40:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.702+0000 7f7c83400700 1 -- 172.21.15.67:0/3271331853 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e413e6cf0 con 0x558e416b5560 2024-03-30T02:40:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.702+0000 7f7c83400700 10 client.26513 handle_command_reply: tid=0 2024-03-30T02:40:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.702+0000 7f7cbc229b80 1 client.26513 shutdown 2024-03-30T02:40:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.702+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3271331853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 msgr2=0x558e41657290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.702+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e41657290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005110 tx=0x7f7c9c0106a0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.703+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3271331853 shutdown_connections 2024-03-30T02:40:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.703+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x558e416b5560 0x558e416b7980 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.703+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558e41650df0 0x558e41655990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.703+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558e416455d0 0x558e41657290 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.703+0000 7f7cbc229b80 1 --2- 172.21.15.67:0/3271331853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558e41645ef0 0x558e416577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.703+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3271331853 >> 172.21.15.67:0/3271331853 conn(0x558e415a56a0 msgr2=0x558e4164c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.703+0000 7f7c83400700 0 client.26513 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:40:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.703+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3271331853 shutdown_connections 2024-03-30T02:40:15.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.703+0000 7f7cbc229b80 1 -- 172.21.15.67:0/3271331853 wait complete. 2024-03-30T02:40:15.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.704+0000 7f7cbc229b80 20 client.26513 trim_cache size 0 max 16384 2024-03-30T02:40:15.704 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:40:15.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.705+0000 7f7ca8800700 1 -- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7c880a1290 msgr2=0x7f7c880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.705+0000 7f7ca8800700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7c880a1290 0x7f7c880a3750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c003800 tx=0x7f7c9c00b540 comp rx=0 tx=0).stop 2024-03-30T02:40:15.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.705+0000 7f7ca8800700 1 -- 172.21.15.67:0/3491582792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca0103740 msgr2=0x7f7ca01a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:15.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.705+0000 7f7ca8800700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca0103740 0x7f7ca01a3b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000b3c0 tx=0x7f7c9000b6d0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.706+0000 7f7ca8800700 1 -- 172.21.15.67:0/3491582792 shutdown_connections 2024-03-30T02:40:15.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.706+0000 7f7ca8800700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7c880a1290 0x7f7c880a3750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.706+0000 7f7ca8800700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7ca0105ca0 0x7f7ca019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.706+0000 7f7ca8800700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ca0068370 0x7f7ca01a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.706+0000 7f7ca8800700 1 --2- 172.21.15.67:0/3491582792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca0103740 0x7f7ca01a3b40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:15.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.706+0000 7f7ca8800700 1 -- 172.21.15.67:0/3491582792 >> 172.21.15.67:0/3491582792 conn(0x7f7ca0076d40 msgr2=0x7f7ca01025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.707+0000 7f7ca8800700 1 -- 172.21.15.67:0/3491582792 shutdown_connections 2024-03-30T02:40:15.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:15.707+0000 7f7ca8800700 1 -- 172.21.15.67:0/3491582792 wait complete. 2024-03-30T02:40:15.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:40:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.154+0000 7fbecc400700 1 -- 172.21.15.67:0/2723063485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec41082f0 msgr2=0x7fbec4108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.154+0000 7fbecc400700 1 --2- 172.21.15.67:0/2723063485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec41082f0 0x7fbec4108770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00098f0 tx=0x7fbeb0009c00 comp rx=0 tx=0).stop 2024-03-30T02:40:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.154+0000 7fbecc400700 1 -- 172.21.15.67:0/2723063485 shutdown_connections 2024-03-30T02:40:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.154+0000 7fbecc400700 1 --2- 172.21.15.67:0/2723063485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbec411a590 0x7fbec411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.154+0000 7fbecc400700 1 --2- 172.21.15.67:0/2723063485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbec410f540 0x7fbec4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.154+0000 7fbecc400700 1 --2- 172.21.15.67:0/2723063485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec41082f0 0x7fbec4108770 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.154+0000 7fbecc400700 1 -- 172.21.15.67:0/2723063485 >> 172.21.15.67:0/2723063485 conn(0x7fbec406bdf0 msgr2=0x7fbec406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.155+0000 7fbecc400700 1 -- 172.21.15.67:0/2723063485 shutdown_connections 2024-03-30T02:40:16.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.155+0000 7fbecc400700 1 -- 172.21.15.67:0/2723063485 wait complete. 2024-03-30T02:40:16.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.156+0000 7fbecc400700 1 Processor -- start 2024-03-30T02:40:16.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.157+0000 7fbecc400700 1 -- start start 2024-03-30T02:40:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.158+0000 7fbecc400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbec410f540 0x7fbec41a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.158+0000 7fbecc400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec411a590 0x7fbec41a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.158+0000 7fbecc400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbec41aaad0 0x7fbec41aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.158+0000 7fbecc400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec41ab550 con 0x7fbec41aaad0 2024-03-30T02:40:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.158+0000 7fbecc400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec41ab6c0 con 0x7fbec411a590 2024-03-30T02:40:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.158+0000 7fbecc400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec41ab880 con 0x7fbec410f540 2024-03-30T02:40:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.158+0000 7fbec9000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec411a590 0x7fbec41a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.158+0000 7fbec9000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec411a590 0x7fbec41a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55342/0 (socket says 172.21.15.67:55342) 2024-03-30T02:40:16.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.158+0000 7fbec9000700 1 -- 172.21.15.67:0/1399553276 learned_addr learned my addr 172.21.15.67:0/1399553276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.159+0000 7fbec9a00700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbec410f540 0x7fbec41a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.159+0000 7fbec9000700 1 -- 172.21.15.67:0/1399553276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbec410f540 msgr2=0x7fbec41a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.159+0000 7fbec9000700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbec410f540 0x7fbec41a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.159+0000 7fbeca400700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbec41aaad0 0x7fbec41aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.159+0000 7fbec9000700 1 -- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbec41aaad0 msgr2=0x7fbec41aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.159+0000 7fbec9000700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbec41aaad0 0x7fbec41aaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.159+0000 7fbec9000700 1 -- 172.21.15.67:0/1399553276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb0009520 con 0x7fbec411a590 2024-03-30T02:40:16.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.160+0000 7fbec9000700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec411a590 0x7fbec41a6730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000c00 tx=0x7fbeb000bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.160+0000 7fbec2000700 1 -- 172.21.15.67:0/1399553276 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb001c070 con 0x7fbec411a590 2024-03-30T02:40:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.160+0000 7fbec2000700 1 -- 172.21.15.67:0/1399553276 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb000bbd0 con 0x7fbec411a590 2024-03-30T02:40:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.160+0000 7fbecc400700 1 -- 172.21.15.67:0/1399553276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec41b2d80 con 0x7fbec411a590 2024-03-30T02:40:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.160+0000 7fbec2000700 1 -- 172.21.15.67:0/1399553276 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb00038e0 con 0x7fbec411a590 2024-03-30T02:40:16.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.160+0000 7fbecc400700 1 -- 172.21.15.67:0/1399553276 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec41b3150 con 0x7fbec411a590 2024-03-30T02:40:16.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.167+0000 7fbec2000700 1 -- 172.21.15.67:0/1399553276 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbeb0020030 con 0x7fbec411a590 2024-03-30T02:40:16.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.169+0000 7fbee0009b80 1 Processor -- start 2024-03-30T02:40:16.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.169+0000 7fbee0009b80 1 -- start start 2024-03-30T02:40:16.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.170+0000 7fbec2000700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbeac0a13d0 0x7fbeac0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.170+0000 7fbec2000700 1 -- 172.21.15.67:0/1399553276 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fbeb00cdd50 con 0x7fbec411a590 2024-03-30T02:40:16.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.170+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d0f5d0 0x55e269d0f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.171+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d1a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.171+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d1adf0 0x55e269d1d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.171+0000 7fbee0009b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269c68990 con 0x55e269d1adf0 2024-03-30T02:40:16.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.171+0000 7fbee0009b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269c66350 con 0x55e269d0f5d0 2024-03-30T02:40:16.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.171+0000 7fbee0009b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269c61c40 con 0x55e269d0fef0 2024-03-30T02:40:16.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.171+0000 7fbeca400700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbeac0a13d0 0x7fbeac0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.171+0000 7fbec9000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d1a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.171+0000 7fbec9000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d1a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44038/0 (socket says 172.21.15.67:44038) 2024-03-30T02:40:16.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.171+0000 7fbec9000700 1 -- 172.21.15.67:0/3517089735 learned_addr learned my addr 172.21.15.67:0/3517089735 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.172+0000 7fbec9000700 1 -- 172.21.15.67:0/3517089735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d0f5d0 msgr2=0x55e269d0f9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:40:16.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.172+0000 7fbec9000700 1 --2- 172.21.15.67:0/3517089735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d0f5d0 0x55e269d0f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.172+0000 7fbec9000700 1 -- 172.21.15.67:0/3517089735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d1adf0 msgr2=0x55e269d1d1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:40:16.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.172+0000 7fbec9000700 1 --2- 172.21.15.67:0/3517089735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d1adf0 0x55e269d1d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.172+0000 7fbec9000700 1 -- 172.21.15.67:0/3517089735 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e269ae33b0 con 0x55e269d0fef0 2024-03-30T02:40:16.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.173+0000 7fbeca400700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbeac0a13d0 0x7fbeac0a3890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00d1a0 tx=0x7fbebc00d560 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.173+0000 7fbec9000700 1 --2- 172.21.15.67:0/3517089735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d1a8b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0097040 tx=0x7fbeb009a910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c277582748a6fd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.174+0000 7fbeb6a00700 1 -- 172.21.15.67:0/3517089735 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb009ab40 con 0x55e269d0fef0 2024-03-30T02:40:16.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.174+0000 7fbeb6a00700 1 -- 172.21.15.67:0/3517089735 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb009e970 con 0x55e269d0fef0 2024-03-30T02:40:16.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.174+0000 7fbeb6a00700 1 -- 172.21.15.67:0/3517089735 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb00cb3b0 con 0x55e269d0fef0 2024-03-30T02:40:16.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.175+0000 7fbee0009b80 1 -- 172.21.15.67:0/3517089735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 msgr2=0x55e269d1a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.175+0000 7fbee0009b80 1 --2- 172.21.15.67:0/3517089735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d1a8b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0097040 tx=0x7fbeb009a910 comp rx=0 tx=0).stop 2024-03-30T02:40:16.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.175+0000 7fbee0009b80 1 -- 172.21.15.67:0/3517089735 shutdown_connections 2024-03-30T02:40:16.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.175+0000 7fbee0009b80 1 --2- 172.21.15.67:0/3517089735 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d1adf0 0x55e269d1d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.175+0000 7fbee0009b80 1 --2- 172.21.15.67:0/3517089735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d0f5d0 0x55e269d0f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.175+0000 7fbee0009b80 1 --2- 172.21.15.67:0/3517089735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d1a8b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.175+0000 7fbee0009b80 1 -- 172.21.15.67:0/3517089735 >> 172.21.15.67:0/3517089735 conn(0x55e269c6f6a0 msgr2=0x55e269d0ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.176+0000 7fbee0009b80 1 -- 172.21.15.67:0/3517089735 shutdown_connections 2024-03-30T02:40:16.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.176+0000 7fbee0009b80 1 -- 172.21.15.67:0/3517089735 wait complete. 2024-03-30T02:40:16.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.176+0000 7fbee0009b80 1 -- start start 2024-03-30T02:40:16.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.177+0000 7fbee0009b80 1 Processor -- start 2024-03-30T02:40:16.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.178+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0f5d0 0x55e269d1a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.178+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d17ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.178+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d18120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.178+0000 7fbee0009b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269bfdd80 con 0x55e269d0f5d0 2024-03-30T02:40:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.178+0000 7fbee0009b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269bfda20 con 0x55e269d1adf0 2024-03-30T02:40:16.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.178+0000 7fbee0009b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269ae56a0 con 0x55e269d0fef0 2024-03-30T02:40:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.178+0000 7fbee0009b80 10 client.0 _ll_register_callbacks cb 0x55e269c07470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.178+0000 7fbeca400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d18120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.179+0000 7fbec9000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d17ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.178+0000 7fbeca400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d18120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55358/0 (socket says 172.21.15.67:55358) 2024-03-30T02:40:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.179+0000 7fbeca400700 1 -- 172.21.15.67:0/149119548 learned_addr learned my addr 172.21.15.67:0/149119548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.179+0000 7fbeca400700 1 -- 172.21.15.67:0/149119548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 msgr2=0x55e269d17ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.179+0000 7fbeca400700 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d17ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.179+0000 7fbeca400700 1 -- 172.21.15.67:0/149119548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0f5d0 msgr2=0x55e269d1a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.179+0000 7fbeca400700 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0f5d0 0x55e269d1a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.179+0000 7fbeca400700 1 -- 172.21.15.67:0/149119548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec41b3150 con 0x55e269d1adf0 2024-03-30T02:40:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.180+0000 7fbeca400700 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d18120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbebc019ba0 tx=0x7fbebc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c8e36d4476925a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.180+0000 7fbeab400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:40:16.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.180+0000 7fbeab400700 1 -- 172.21.15.67:0/149119548 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc014880 con 0x55e269d1adf0 2024-03-30T02:40:16.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.180+0000 7fbeab400700 1 -- 172.21.15.67:0/149119548 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fbebc0149e0 con 0x55e269d1adf0 2024-03-30T02:40:16.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.181+0000 7fbee0009b80 1 -- 172.21.15.67:0/149119548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e269c22380 con 0x55e269d1adf0 2024-03-30T02:40:16.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.181+0000 7fbeab400700 1 -- 172.21.15.67:0/149119548 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbebc014ba0 con 0x55e269d1adf0 2024-03-30T02:40:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.182+0000 7fbee0009b80 10 client.26522 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.182+0000 7fbee0009b80 1 -- 172.21.15.67:0/149119548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e269a8fb60 con 0x55e269d1adf0 2024-03-30T02:40:16.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.183+0000 7fbeab400700 1 -- 172.21.15.67:0/149119548 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e269c22380 con 0x55e269d1adf0 2024-03-30T02:40:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.184+0000 7fbeab400700 1 -- 172.21.15.67:0/149119548 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbebc029070 con 0x55e269d1adf0 2024-03-30T02:40:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.184+0000 7fbee0009b80 10 client.26522 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:16.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.184+0000 7fbee0009b80 10 client.26522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:40:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.184+0000 7fbee0009b80 20 client.26522 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.184+0000 7fbee0009b80 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e269d2a5f0 0x55e269d2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.184+0000 7fbee0009b80 4 client.26522 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:40:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.184+0000 7fbee0009b80 1 -- 172.21.15.67:0/149119548 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e269c5a700 con 0x55e269d2a5f0 2024-03-30T02:40:16.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.185+0000 7fbec9a00700 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e269d2a5f0 0x55e269d2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.186+0000 7fbec9a00700 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e269d2a5f0 0x55e269d2ca10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f261db6c23957bfa server_cookie=139e4e3dadcc6c0d in_seq=0 out_seq=0 2024-03-30T02:40:16.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.187+0000 7fbeab400700 10 client.26522 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:40:16.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.188+0000 7fbeab400700 1 -- 172.21.15.67:0/149119548 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e269c5a700 con 0x55e269d2a5f0 2024-03-30T02:40:16.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.188+0000 7fbeab400700 10 client.26522 handle_command_reply: tid=0 2024-03-30T02:40:16.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.188+0000 7fbee0009b80 1 client.26522 shutdown 2024-03-30T02:40:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.191+0000 7fbee0009b80 1 -- 172.21.15.67:0/149119548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 msgr2=0x55e269d18120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.191+0000 7fbee0009b80 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d18120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbebc019ba0 tx=0x7fbebc004ad0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.192+0000 7fbee0009b80 1 -- 172.21.15.67:0/149119548 shutdown_connections 2024-03-30T02:40:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.192+0000 7fbee0009b80 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e269d2a5f0 0x55e269d2ca10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.192+0000 7fbee0009b80 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0f5d0 0x55e269d1a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.192+0000 7fbee0009b80 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d18120 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.192+0000 7fbeab400700 0 client.26522 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:40:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.192+0000 7fbee0009b80 1 --2- 172.21.15.67:0/149119548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0fef0 0x55e269d17ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.192+0000 7fbee0009b80 1 -- 172.21.15.67:0/149119548 >> 172.21.15.67:0/149119548 conn(0x55e269c6f6a0 msgr2=0x55e269d116c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.192+0000 7fbee0009b80 1 -- 172.21.15.67:0/149119548 shutdown_connections 2024-03-30T02:40:16.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.192+0000 7fbee0009b80 1 -- 172.21.15.67:0/149119548 wait complete. 2024-03-30T02:40:16.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.193+0000 7fbee0009b80 20 client.26522 trim_cache size 0 max 16384 2024-03-30T02:40:16.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.225+0000 7fbee0009b80 1 Processor -- start 2024-03-30T02:40:16.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.225+0000 7fbee0009b80 1 -- start start 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0f5d0 0x55e269d18050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0fef0 0x55e269d217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbee0009b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269d2a410 con 0x55e269d0fef0 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbee0009b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269d114d0 con 0x55e269d1adf0 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbee0009b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269d18a60 con 0x55e269d0f5d0 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbec9a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0f5d0 0x55e269d18050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbeca400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbeca400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55374/0 (socket says 172.21.15.67:55374) 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.226+0000 7fbeca400700 1 -- 172.21.15.67:0/2864938861 learned_addr learned my addr 172.21.15.67:0/2864938861 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.227+0000 7fbeca400700 1 -- 172.21.15.67:0/2864938861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0f5d0 msgr2=0x55e269d18050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.227+0000 7fbeca400700 1 --2- 172.21.15.67:0/2864938861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0f5d0 0x55e269d18050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.227+0000 7fbeca400700 1 -- 172.21.15.67:0/2864938861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0fef0 msgr2=0x55e269d217e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.227+0000 7fbeca400700 1 --2- 172.21.15.67:0/2864938861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0fef0 0x55e269d217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.227+0000 7fbeca400700 1 -- 172.21.15.67:0/2864938861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e269c27ea0 con 0x55e269d1adf0 2024-03-30T02:40:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.227+0000 7fbeca400700 1 --2- 172.21.15.67:0/2864938861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d116c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbebc021e70 tx=0x7fbebc007e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85835a9dcf591e85 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.228+0000 7fbeb7400700 1 -- 172.21.15.67:0/2864938861 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc0167e0 con 0x55e269d1adf0 2024-03-30T02:40:16.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.228+0000 7fbeb7400700 1 -- 172.21.15.67:0/2864938861 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbebc016e00 con 0x55e269d1adf0 2024-03-30T02:40:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.229+0000 7fbeb7400700 1 -- 172.21.15.67:0/2864938861 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc0167e0 con 0x55e269d1adf0 2024-03-30T02:40:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.230+0000 7fbee0009b80 1 -- 172.21.15.67:0/2864938861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 msgr2=0x55e269d116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.230+0000 7fbee0009b80 1 --2- 172.21.15.67:0/2864938861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d116c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbebc021e70 tx=0x7fbebc007e50 comp rx=0 tx=0).stop 2024-03-30T02:40:16.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.230+0000 7fbee0009b80 1 -- 172.21.15.67:0/2864938861 shutdown_connections 2024-03-30T02:40:16.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.230+0000 7fbee0009b80 1 --2- 172.21.15.67:0/2864938861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d116c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.230+0000 7fbee0009b80 1 --2- 172.21.15.67:0/2864938861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0f5d0 0x55e269d18050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.230+0000 7fbee0009b80 1 --2- 172.21.15.67:0/2864938861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0fef0 0x55e269d217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.230+0000 7fbee0009b80 1 -- 172.21.15.67:0/2864938861 >> 172.21.15.67:0/2864938861 conn(0x55e269c6f6a0 msgr2=0x55e269d163d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.230+0000 7fbee0009b80 1 -- 172.21.15.67:0/2864938861 shutdown_connections 2024-03-30T02:40:16.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.231+0000 7fbee0009b80 1 -- 172.21.15.67:0/2864938861 wait complete. 2024-03-30T02:40:16.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.231+0000 7fbee0009b80 1 -- start start 2024-03-30T02:40:16.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.231+0000 7fbee0009b80 1 Processor -- start 2024-03-30T02:40:16.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.232+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0f5d0 0x55e269d1fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.232+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0fef0 0x55e269d20320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.232+0000 7fbee0009b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d20860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.232+0000 7fbee0009b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269d21bd0 con 0x55e269d0fef0 2024-03-30T02:40:16.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.232+0000 7fbee0009b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269d7e430 con 0x55e269d1adf0 2024-03-30T02:40:16.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.232+0000 7fbee0009b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e269d7e570 con 0x55e269d0f5d0 2024-03-30T02:40:16.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.232+0000 7fbee0009b80 10 client.0 _ll_register_callbacks cb 0x55e269c07470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:16.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.233+0000 7fbeca400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d20860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.233+0000 7fbeca400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d20860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55384/0 (socket says 172.21.15.67:55384) 2024-03-30T02:40:16.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.233+0000 7fbeca400700 1 -- 172.21.15.67:0/1719912532 learned_addr learned my addr 172.21.15.67:0/1719912532 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbeca400700 1 -- 172.21.15.67:0/1719912532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0f5d0 msgr2=0x55e269d1fde0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:40:16.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbeca400700 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0f5d0 0x55e269d1fde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbeca400700 1 -- 172.21.15.67:0/1719912532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0fef0 msgr2=0x55e269d20320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbeca400700 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0fef0 0x55e269d20320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbeca400700 1 -- 172.21.15.67:0/1719912532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e269a8fb60 con 0x55e269d1adf0 2024-03-30T02:40:16.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbeca400700 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d20860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbebc022fe0 tx=0x7fbebc014c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42cdc9f59dd64c96 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbeab400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:40:16.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbeab400700 1 -- 172.21.15.67:0/1719912532 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc004e80 con 0x55e269d1adf0 2024-03-30T02:40:16.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbeab400700 1 -- 172.21.15.67:0/1719912532 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fbebc0167e0 con 0x55e269d1adf0 2024-03-30T02:40:16.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.234+0000 7fbee0009b80 1 -- 172.21.15.67:0/1719912532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbeb009e970 con 0x55e269d1adf0 2024-03-30T02:40:16.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.235+0000 7fbeab400700 1 -- 172.21.15.67:0/1719912532 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbebc021ea0 con 0x55e269d1adf0 2024-03-30T02:40:16.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.235+0000 7fbee0009b80 10 client.26528 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:16.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.235+0000 7fbee0009b80 1 -- 172.21.15.67:0/1719912532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e269ae7cd0 con 0x55e269d1adf0 2024-03-30T02:40:16.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.235+0000 7fbeab400700 1 -- 172.21.15.67:0/1719912532 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb009e970 con 0x55e269d1adf0 2024-03-30T02:40:16.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.235+0000 7fbeab400700 1 -- 172.21.15.67:0/1719912532 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbebc03c030 con 0x55e269d1adf0 2024-03-30T02:40:16.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.236+0000 7fbee0009b80 10 client.26528 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:16.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.236+0000 7fbee0009b80 10 client.26528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:40:16.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.236+0000 7fbee0009b80 20 client.26528 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:16.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.236+0000 7fbee0009b80 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e269d7f950 0x55e269d81d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.236+0000 7fbee0009b80 4 client.26528 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:40:16.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.236+0000 7fbee0009b80 1 -- 172.21.15.67:0/1719912532 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e269ab0cf0 con 0x55e269d7f950 2024-03-30T02:40:16.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.236+0000 7fbec9a00700 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e269d7f950 0x55e269d81d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.237+0000 7fbec9a00700 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e269d7f950 0x55e269d81d40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f50d7a853b9b8897 server_cookie=bae6dc08a9570282 in_seq=0 out_seq=0 2024-03-30T02:40:16.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.238+0000 7fbeab400700 10 client.26528 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:40:16.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.239+0000 7fbeab400700 1 -- 172.21.15.67:0/1719912532 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e269ab0cf0 con 0x55e269d7f950 2024-03-30T02:40:16.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.239+0000 7fbeab400700 10 client.26528 handle_command_reply: tid=0 2024-03-30T02:40:16.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.239+0000 7fbee0009b80 1 client.26528 shutdown 2024-03-30T02:40:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.240+0000 7fbee0009b80 1 -- 172.21.15.67:0/1719912532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 msgr2=0x55e269d20860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.240+0000 7fbee0009b80 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d20860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbebc022fe0 tx=0x7fbebc014c00 comp rx=0 tx=0).stop 2024-03-30T02:40:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.240+0000 7fbee0009b80 1 -- 172.21.15.67:0/1719912532 shutdown_connections 2024-03-30T02:40:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.240+0000 7fbee0009b80 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e269d1adf0 0x55e269d20860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.240+0000 7fbee0009b80 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e269d0f5d0 0x55e269d1fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.240+0000 7fbee0009b80 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e269d7f950 0x55e269d81d40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.240+0000 7fbee0009b80 1 --2- 172.21.15.67:0/1719912532 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e269d0fef0 0x55e269d20320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.240+0000 7fbeab400700 0 client.26528 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:40:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.240+0000 7fbee0009b80 1 -- 172.21.15.67:0/1719912532 >> 172.21.15.67:0/1719912532 conn(0x55e269c6f6a0 msgr2=0x55e269d15fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.241+0000 7fbee0009b80 1 -- 172.21.15.67:0/1719912532 shutdown_connections 2024-03-30T02:40:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.241+0000 7fbee0009b80 1 -- 172.21.15.67:0/1719912532 wait complete. 2024-03-30T02:40:16.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.241+0000 7fbee0009b80 20 client.26528 trim_cache size 0 max 16384 2024-03-30T02:40:16.241 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:40:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 -- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbeac0a13d0 msgr2=0x7fbeac0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbeac0a13d0 0x7fbeac0a3890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00d1a0 tx=0x7fbebc00d560 comp rx=0 tx=0).stop 2024-03-30T02:40:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 -- 172.21.15.67:0/1399553276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec411a590 msgr2=0x7fbec41a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec411a590 0x7fbec41a6730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0000c00 tx=0x7fbeb000bfa0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 -- 172.21.15.67:0/1399553276 shutdown_connections 2024-03-30T02:40:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbeac0a13d0 0x7fbeac0a3890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbec41aaad0 0x7fbec41aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbec410f540 0x7fbec41a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 --2- 172.21.15.67:0/1399553276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbec411a590 0x7fbec41a6730 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 -- 172.21.15.67:0/1399553276 >> 172.21.15.67:0/1399553276 conn(0x7fbec406bdf0 msgr2=0x7fbec4118580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 -- 172.21.15.67:0/1399553276 shutdown_connections 2024-03-30T02:40:16.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.243+0000 7fbecc400700 1 -- 172.21.15.67:0/1399553276 wait complete. 2024-03-30T02:40:16.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:40:16.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.643+0000 7f314f400700 1 -- 172.21.15.67:0/3461225009 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314800fe90 con 0x7f315010d5a0 2024-03-30T02:40:16.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.643+0000 7f3157000700 1 -- 172.21.15.67:0/3461225009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f315010d5a0 msgr2=0x7f315010f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.643+0000 7f3157000700 1 --2- 172.21.15.67:0/3461225009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f315010d5a0 0x7f315010f9d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f31480098f0 tx=0x7f3148009c00 comp rx=0 tx=0).stop 2024-03-30T02:40:16.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.643+0000 7f3157000700 1 -- 172.21.15.67:0/3461225009 shutdown_connections 2024-03-30T02:40:16.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.643+0000 7f3157000700 1 --2- 172.21.15.67:0/3461225009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3150073180 0x7f3150073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.643+0000 7f3157000700 1 --2- 172.21.15.67:0/3461225009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f315010d5a0 0x7f315010f9d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.643+0000 7f3157000700 1 --2- 172.21.15.67:0/3461225009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3150073aa0 0x7f315010d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.643+0000 7f3157000700 1 -- 172.21.15.67:0/3461225009 >> 172.21.15.67:0/3461225009 conn(0x7f31500fc7f0 msgr2=0x7f31500fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.644+0000 7f3157000700 1 -- 172.21.15.67:0/3461225009 shutdown_connections 2024-03-30T02:40:16.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.644+0000 7f3157000700 1 -- 172.21.15.67:0/3461225009 wait complete. 2024-03-30T02:40:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.646+0000 7f3157000700 1 Processor -- start 2024-03-30T02:40:16.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.646+0000 7f3157000700 1 -- start start 2024-03-30T02:40:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.646+0000 7f3157000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3150073180 0x7f31501991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.646+0000 7f3157000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3150073aa0 0x7f3150199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.646+0000 7f3157000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f315010d5a0 0x7f315019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.646+0000 7f3157000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315019e100 con 0x7f315010d5a0 2024-03-30T02:40:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.646+0000 7f3157000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315019e270 con 0x7f3150073aa0 2024-03-30T02:40:16.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.646+0000 7f3157000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315019e430 con 0x7f3150073180 2024-03-30T02:40:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.647+0000 7f3147400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3150073aa0 0x7f3150199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.647+0000 7f3147400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3150073aa0 0x7f3150199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55404/0 (socket says 172.21.15.67:55404) 2024-03-30T02:40:16.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.647+0000 7f3147400700 1 -- 172.21.15.67:0/3824393402 learned_addr learned my addr 172.21.15.67:0/3824393402 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.647+0000 7f314fe00700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3150073180 0x7f31501991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.647+0000 7f3147400700 1 -- 172.21.15.67:0/3824393402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3150073180 msgr2=0x7f31501991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.647+0000 7f3147400700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3150073180 0x7f31501991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.647+0000 7f3147400700 1 -- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f315010d5a0 msgr2=0x7f315019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.647+0000 7f3155000700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f315010d5a0 0x7f315019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.648+0000 7f3147400700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f315010d5a0 0x7f315019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.648+0000 7f3147400700 1 -- 172.21.15.67:0/3824393402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31380095f0 con 0x7f3150073aa0 2024-03-30T02:40:16.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.648+0000 7f3155000700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f315010d5a0 0x7f315019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:16.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.648+0000 7f3147400700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3150073aa0 0x7f3150199720 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f313800ec10 tx=0x7f313800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.649+0000 7f314d600700 1 -- 172.21.15.67:0/3824393402 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800cde0 con 0x7f3150073aa0 2024-03-30T02:40:16.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.649+0000 7f314d600700 1 -- 172.21.15.67:0/3824393402 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31380044e0 con 0x7f3150073aa0 2024-03-30T02:40:16.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.649+0000 7f3157000700 1 -- 172.21.15.67:0/3824393402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3148009520 con 0x7f3150073aa0 2024-03-30T02:40:16.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.649+0000 7f314d600700 1 -- 172.21.15.67:0/3824393402 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138010490 con 0x7f3150073aa0 2024-03-30T02:40:16.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.649+0000 7f3157000700 1 -- 172.21.15.67:0/3824393402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315019e900 con 0x7f3150073aa0 2024-03-30T02:40:16.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.654+0000 7f314d600700 1 -- 172.21.15.67:0/3824393402 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3138003780 con 0x7f3150073aa0 2024-03-30T02:40:16.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.656+0000 7f314d600700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f31340a1310 0x7f31340a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.657+0000 7f316ab29b80 1 Processor -- start 2024-03-30T02:40:16.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.657+0000 7f314fe00700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f31340a1310 0x7f31340a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.657+0000 7f314d600700 1 -- 172.21.15.67:0/3824393402 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f3138014070 con 0x7f3150073aa0 2024-03-30T02:40:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.657+0000 7f316ab29b80 1 -- start start 2024-03-30T02:40:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.658+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cde9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.658+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cdeef0 0x564eb4cf20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.658+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cf2600 0x564eb4cf49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.658+0000 7f316ab29b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4c37990 con 0x564eb4cde5d0 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.658+0000 7f316ab29b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4c35350 con 0x564eb4cdeef0 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.658+0000 7f316ab29b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4c30c40 con 0x564eb4cf2600 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.658+0000 7f3155000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cf2600 0x564eb4cf49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.658+0000 7f3155000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cf2600 0x564eb4cf49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44090/0 (socket says 172.21.15.67:44090) 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.658+0000 7f3155000700 1 -- 172.21.15.67:0/3003827970 learned_addr learned my addr 172.21.15.67:0/3003827970 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.659+0000 7f3147400700 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cdeef0 0x564eb4cf20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.659+0000 7f314fe00700 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cde9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.659+0000 7f3155000700 1 -- 172.21.15.67:0/3003827970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cdeef0 msgr2=0x564eb4cf20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.659+0000 7f3155000700 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cdeef0 0x564eb4cf20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.659+0000 7f3155000700 1 -- 172.21.15.67:0/3003827970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 msgr2=0x564eb4cde9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.659+0000 7f3155000700 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cde9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.659+0000 7f3155000700 1 -- 172.21.15.67:0/3003827970 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eb4ab23b0 con 0x564eb4cf2600 2024-03-30T02:40:16.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.659+0000 7f314fe00700 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cde9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:16.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.660+0000 7f314fe00700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f31340a1310 0x7f31340a37d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3140009d60 tx=0x7f31400077e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.660+0000 7f3155000700 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cf2600 0x564eb4cf49f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3148009fd0 tx=0x7f314800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d5f817dfce0a0db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.660+0000 7f3145600700 1 -- 172.21.15.67:0/3003827970 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314801d070 con 0x564eb4cf2600 2024-03-30T02:40:16.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.660+0000 7f3145600700 1 -- 172.21.15.67:0/3003827970 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f314800f920 con 0x564eb4cf2600 2024-03-30T02:40:16.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.660+0000 7f3145600700 1 -- 172.21.15.67:0/3003827970 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31480052a0 con 0x564eb4cf2600 2024-03-30T02:40:16.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.661+0000 7f316ab29b80 1 -- 172.21.15.67:0/3003827970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cf2600 msgr2=0x564eb4cf49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.661+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cf2600 0x564eb4cf49f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3148009fd0 tx=0x7f314800bf10 comp rx=0 tx=0).stop 2024-03-30T02:40:16.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.661+0000 7f316ab29b80 1 -- 172.21.15.67:0/3003827970 shutdown_connections 2024-03-30T02:40:16.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.661+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cde9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.661+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cf2600 0x564eb4cf49f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.661+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3003827970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cdeef0 0x564eb4cf20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.661+0000 7f316ab29b80 1 -- 172.21.15.67:0/3003827970 >> 172.21.15.67:0/3003827970 conn(0x564eb4c3e6a0 msgr2=0x564eb4cdaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.662+0000 7f316ab29b80 1 -- 172.21.15.67:0/3003827970 shutdown_connections 2024-03-30T02:40:16.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.662+0000 7f316ab29b80 1 -- 172.21.15.67:0/3003827970 wait complete. 2024-03-30T02:40:16.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.662+0000 7f316ab29b80 1 -- start start 2024-03-30T02:40:16.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.663+0000 7f316ab29b80 1 Processor -- start 2024-03-30T02:40:16.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.663+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cde5d0 0x564eb4cf1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cdeef0 0x564eb4cef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f316ab29b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4bccd80 con 0x564eb4cdeef0 2024-03-30T02:40:16.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f316ab29b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4bcca20 con 0x564eb4cf2600 2024-03-30T02:40:16.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f316ab29b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4ab46a0 con 0x564eb4cde5d0 2024-03-30T02:40:16.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f316ab29b80 10 client.0 _ll_register_callbacks cb 0x564eb4bd6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:16.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:16 smithi067 ceph-mon[37808]: pgmap v99: 129 pgs: 129 active+clean; 517 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 505 op/s 2024-03-30T02:40:16.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f3155000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f3155000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cef880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55416/0 (socket says 172.21.15.67:55416) 2024-03-30T02:40:16.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f3155000700 1 -- 172.21.15.67:0/3294775468 learned_addr learned my addr 172.21.15.67:0/3294775468 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.664+0000 7f3147400700 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cdeef0 0x564eb4cef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.665+0000 7f3155000700 1 -- 172.21.15.67:0/3294775468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cde5d0 msgr2=0x564eb4cf1ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:40:16.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.665+0000 7f3155000700 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cde5d0 0x564eb4cf1ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.665+0000 7f3155000700 1 -- 172.21.15.67:0/3294775468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cdeef0 msgr2=0x564eb4cef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.665+0000 7f3155000700 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cdeef0 0x564eb4cef270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.665+0000 7f3155000700 1 -- 172.21.15.67:0/3294775468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31501106e0 con 0x564eb4cf2600 2024-03-30T02:40:16.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.665+0000 7f3155000700 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cef880 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f314800b810 tx=0x7f3148005b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d826265e00c5c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.665+0000 7f313e000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:40:16.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.665+0000 7f313e000700 1 -- 172.21.15.67:0/3294775468 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314801d040 con 0x564eb4cf2600 2024-03-30T02:40:16.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.666+0000 7f313e000700 1 -- 172.21.15.67:0/3294775468 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f3148005240 con 0x564eb4cf2600 2024-03-30T02:40:16.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.666+0000 7f316ab29b80 1 -- 172.21.15.67:0/3294775468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564eb4bf1380 con 0x564eb4cf2600 2024-03-30T02:40:16.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.666+0000 7f313e000700 1 -- 172.21.15.67:0/3294775468 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3148005400 con 0x564eb4cf2600 2024-03-30T02:40:16.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.666+0000 7f316ab29b80 10 client.26540 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:16.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.666+0000 7f316ab29b80 1 -- 172.21.15.67:0/3294775468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564eb4a5eb60 con 0x564eb4cf2600 2024-03-30T02:40:16.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.667+0000 7f313e000700 1 -- 172.21.15.67:0/3294775468 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564eb4bf1380 con 0x564eb4cf2600 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.667+0000 7f313e000700 1 -- 172.21.15.67:0/3294775468 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3148015020 con 0x564eb4cf2600 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.667+0000 7f316ab29b80 10 client.26540 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.667+0000 7f316ab29b80 10 client.26540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.667+0000 7f316ab29b80 20 client.26540 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.667+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564eb4d12da0 0x564eb4d151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.667+0000 7f316ab29b80 4 client.26540 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.667+0000 7f316ab29b80 1 -- 172.21.15.67:0/3294775468 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564eb4c29700 con 0x564eb4d12da0 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.667+0000 7f314fe00700 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564eb4d12da0 0x564eb4d151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.668+0000 7f314fe00700 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564eb4d12da0 0x564eb4d151c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7638ee6f4eb53ab server_cookie=c8d11f348f6e4127 in_seq=0 out_seq=0 2024-03-30T02:40:16.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.668+0000 7f313e000700 10 client.26540 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.670+0000 7f313e000700 1 -- 172.21.15.67:0/3294775468 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564eb4c29700 con 0x564eb4d12da0 2024-03-30T02:40:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.670+0000 7f313e000700 10 client.26540 handle_command_reply: tid=0 2024-03-30T02:40:16.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.670+0000 7f316ab29b80 1 client.26540 shutdown 2024-03-30T02:40:16.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.670+0000 7f316ab29b80 1 -- 172.21.15.67:0/3294775468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 msgr2=0x564eb4cef880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.670+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cef880 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f314800b810 tx=0x7f3148005b80 comp rx=0 tx=0).stop 2024-03-30T02:40:16.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.671+0000 7f316ab29b80 1 -- 172.21.15.67:0/3294775468 shutdown_connections 2024-03-30T02:40:16.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.671+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cef880 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.671+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cde5d0 0x564eb4cf1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.671+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564eb4d12da0 0x564eb4d151c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.671+0000 7f316ab29b80 1 --2- 172.21.15.67:0/3294775468 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cdeef0 0x564eb4cef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.671+0000 7f313e000700 0 client.26540 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.671+0000 7f316ab29b80 1 -- 172.21.15.67:0/3294775468 >> 172.21.15.67:0/3294775468 conn(0x564eb4c3e6a0 msgr2=0x564eb4ce05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.672+0000 7f316ab29b80 1 -- 172.21.15.67:0/3294775468 shutdown_connections 2024-03-30T02:40:16.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.672+0000 7f316ab29b80 1 -- 172.21.15.67:0/3294775468 wait complete. 2024-03-30T02:40:16.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.672+0000 7f316ab29b80 20 client.26540 trim_cache size 0 max 16384 2024-03-30T02:40:16.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:16 smithi170 ceph-mon[78798]: pgmap v99: 129 pgs: 129 active+clean; 517 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 505 op/s 2024-03-30T02:40:16.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.694+0000 7f316ab29b80 1 Processor -- start 2024-03-30T02:40:16.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.694+0000 7f316ab29b80 1 -- start start 2024-03-30T02:40:16.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cf9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cea560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4ceaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f316ab29b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4cea060 con 0x564eb4cde5d0 2024-03-30T02:40:16.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f316ab29b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4cf0160 con 0x564eb4cf2600 2024-03-30T02:40:16.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f316ab29b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4ced7c0 con 0x564eb4cdeef0 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f3155000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4ceaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f314fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cf9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f3147400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f3155000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4ceaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55422/0 (socket says 172.21.15.67:55422) 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.695+0000 7f3155000700 1 -- 172.21.15.67:0/2128516047 learned_addr learned my addr 172.21.15.67:0/2128516047 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3147400700 1 -- 172.21.15.67:0/2128516047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 msgr2=0x564eb4ceaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3147400700 1 --2- 172.21.15.67:0/2128516047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4ceaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3147400700 1 -- 172.21.15.67:0/2128516047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 msgr2=0x564eb4cf9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3147400700 1 --2- 172.21.15.67:0/2128516047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cf9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3147400700 1 -- 172.21.15.67:0/2128516047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eb4bf6ea0 con 0x564eb4cdeef0 2024-03-30T02:40:16.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3155000700 1 --2- 172.21.15.67:0/2128516047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4ceaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:16.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f314fe00700 1 --2- 172.21.15.67:0/2128516047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cf9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:16.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3147400700 1 --2- 172.21.15.67:0/2128516047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cea560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31380ca980 tx=0x7f3138099e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb03fd2af9b7f2a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3146000700 1 -- 172.21.15.67:0/2128516047 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313809a340 con 0x564eb4cdeef0 2024-03-30T02:40:16.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3146000700 1 -- 172.21.15.67:0/2128516047 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31380cd040 con 0x564eb4cdeef0 2024-03-30T02:40:16.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.696+0000 7f3146000700 1 -- 172.21.15.67:0/2128516047 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138096a50 con 0x564eb4cdeef0 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 -- 172.21.15.67:0/2128516047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 msgr2=0x564eb4cea560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2128516047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cea560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31380ca980 tx=0x7f3138099e80 comp rx=0 tx=0).stop 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 -- 172.21.15.67:0/2128516047 shutdown_connections 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2128516047 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cf9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2128516047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4ceaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2128516047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cea560 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 -- 172.21.15.67:0/2128516047 >> 172.21.15.67:0/2128516047 conn(0x564eb4c3e6a0 msgr2=0x564eb4cee7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 -- 172.21.15.67:0/2128516047 shutdown_connections 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 -- 172.21.15.67:0/2128516047 wait complete. 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.697+0000 7f316ab29b80 1 -- start start 2024-03-30T02:40:16.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f316ab29b80 1 Processor -- start 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cf8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cf7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f316ab29b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cf7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f316ab29b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4cf7de0 con 0x564eb4cde5d0 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f316ab29b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4cf7f20 con 0x564eb4cf2600 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f316ab29b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eb4cf80e0 con 0x564eb4cdeef0 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f316ab29b80 10 client.0 _ll_register_callbacks cb 0x564eb4bd6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f3147400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cf7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f3155000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cf7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f314fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cf8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f3147400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cf7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44116/0 (socket says 172.21.15.67:44116) 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.698+0000 7f3147400700 1 -- 172.21.15.67:0/2103509291 learned_addr learned my addr 172.21.15.67:0/2103509291 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f3147400700 1 -- 172.21.15.67:0/2103509291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 msgr2=0x564eb4cf7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f3147400700 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cf7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f3147400700 1 -- 172.21.15.67:0/2103509291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 msgr2=0x564eb4cf8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f3147400700 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cf8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f3147400700 1 -- 172.21.15.67:0/2103509291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314001d040 con 0x564eb4cdeef0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f3147400700 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cf7320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f31380a2290 tx=0x7f3138005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c9a0331745327a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f313e000700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f313e000700 1 -- 172.21.15.67:0/2103509291 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31380a27c0 con 0x564eb4cdeef0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f316ab29b80 1 -- 172.21.15.67:0/2103509291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f314800f920 con 0x564eb4cdeef0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f313e000700 1 -- 172.21.15.67:0/2103509291 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f313809a340 con 0x564eb4cdeef0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f313e000700 1 -- 172.21.15.67:0/2103509291 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31380cd1b0 con 0x564eb4cdeef0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f316ab29b80 10 client.26503 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f316ab29b80 1 -- 172.21.15.67:0/2103509291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564eb4a5eb60 con 0x564eb4cdeef0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.699+0000 7f313e000700 1 -- 172.21.15.67:0/2103509291 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f314800f920 con 0x564eb4cdeef0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f313e000700 1 -- 172.21.15.67:0/2103509291 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f31380a2330 con 0x564eb4cdeef0 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f316ab29b80 10 client.26503 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:16.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f316ab29b80 10 client.26503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f316ab29b80 20 client.26503 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564eb4d67ef0 0x564eb4d6a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f316ab29b80 4 client.26503 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:40:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f316ab29b80 1 -- 172.21.15.67:0/2103509291 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564eb4a7fcf0 con 0x564eb4d67ef0 2024-03-30T02:40:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f314fe00700 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564eb4d67ef0 0x564eb4d6a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f314fe00700 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564eb4d67ef0 0x564eb4d6a310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abac1126ac8665e9 server_cookie=ee5643398959a1bb in_seq=0 out_seq=0 2024-03-30T02:40:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.700+0000 7f313e000700 10 client.26503 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:16.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.702+0000 7f313e000700 1 -- 172.21.15.67:0/2103509291 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564eb4a7fcf0 con 0x564eb4d67ef0 2024-03-30T02:40:16.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.702+0000 7f313e000700 10 client.26503 handle_command_reply: tid=0 2024-03-30T02:40:16.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.702+0000 7f316ab29b80 1 client.26503 shutdown 2024-03-30T02:40:16.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 -- 172.21.15.67:0/2103509291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 msgr2=0x564eb4cf7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cf7320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f31380a2290 tx=0x7f3138005a90 comp rx=0 tx=0).stop 2024-03-30T02:40:16.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 -- 172.21.15.67:0/2103509291 shutdown_connections 2024-03-30T02:40:16.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x564eb4d67ef0 0x564eb4d6a310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564eb4cde5d0 0x564eb4cf8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564eb4cf2600 0x564eb4cf7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f313e000700 0 client.26503 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:16.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 --2- 172.21.15.67:0/2103509291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564eb4cdeef0 0x564eb4cf7320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 -- 172.21.15.67:0/2103509291 >> 172.21.15.67:0/2103509291 conn(0x564eb4c3e6a0 msgr2=0x564eb4ced950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 -- 172.21.15.67:0/2103509291 shutdown_connections 2024-03-30T02:40:16.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 1 -- 172.21.15.67:0/2103509291 wait complete. 2024-03-30T02:40:16.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.703+0000 7f316ab29b80 20 client.26503 trim_cache size 0 max 16384 2024-03-30T02:40:16.705 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:40:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 -- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f31340a1310 msgr2=0x7f31340a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f31340a1310 0x7f31340a37d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3140009d60 tx=0x7f31400077e0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 -- 172.21.15.67:0/3824393402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3150073aa0 msgr2=0x7f3150199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:16.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3150073aa0 0x7f3150199720 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f313800ec10 tx=0x7f313800c2d0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 -- 172.21.15.67:0/3824393402 shutdown_connections 2024-03-30T02:40:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f31340a1310 0x7f31340a37d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f315010d5a0 0x7f315019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3150073180 0x7f31501991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 --2- 172.21.15.67:0/3824393402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3150073aa0 0x7f3150199720 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 -- 172.21.15.67:0/3824393402 >> 172.21.15.67:0/3824393402 conn(0x7f31500fc7f0 msgr2=0x7f315010ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 -- 172.21.15.67:0/3824393402 shutdown_connections 2024-03-30T02:40:16.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:16.704+0000 7f3157000700 1 -- 172.21.15.67:0/3824393402 wait complete. 2024-03-30T02:40:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:16 smithi082 ceph-mon[79176]: pgmap v99: 129 pgs: 129 active+clean; 517 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 505 op/s 2024-03-30T02:40:17.722 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:40:17.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:40:17.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.933+0000 7f71b6200700 1 -- 172.21.15.67:0/1076718245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 msgr2=0x7f71b010dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:17.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.933+0000 7f71b6200700 1 --2- 172.21.15.67:0/1076718245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 0x7f71b010dbf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71a800b0a0 tx=0x7f71a800b3b0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.933+0000 7f71b6200700 1 -- 172.21.15.67:0/1076718245 shutdown_connections 2024-03-30T02:40:17.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.933+0000 7f71b6200700 1 --2- 172.21.15.67:0/1076718245 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b0100f50 0x7f71b0101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.933+0000 7f71b6200700 1 --2- 172.21.15.67:0/1076718245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 0x7f71b010dbf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.933+0000 7f71b6200700 1 --2- 172.21.15.67:0/1076718245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0101900 0x7f71b0105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.933+0000 7f71b6200700 1 -- 172.21.15.67:0/1076718245 >> 172.21.15.67:0/1076718245 conn(0x7f71b00fc7f0 msgr2=0x7f71b00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:17.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.934+0000 7f71b6200700 1 -- 172.21.15.67:0/1076718245 shutdown_connections 2024-03-30T02:40:17.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.934+0000 7f71b6200700 1 -- 172.21.15.67:0/1076718245 wait complete. 2024-03-30T02:40:17.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71b6200700 1 Processor -- start 2024-03-30T02:40:17.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71b6200700 1 -- start start 2024-03-30T02:40:17.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71b6200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0100f50 0x7f71b0199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71b6200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 0x7f71b0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71b6200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b019dc00 0x7f71b019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71b6200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b019e680 con 0x7f71b019dc00 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71b6200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b019e7f0 con 0x7f71b0106070 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71b6200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b019e9b0 con 0x7f71b0100f50 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71af400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0100f50 0x7f71b0199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 0x7f71b0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 0x7f71b0199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55444/0 (socket says 172.21.15.67:55444) 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71af400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0100f50 0x7f71b0199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44136/0 (socket says 172.21.15.67:44136) 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71af400700 1 -- 172.21.15.67:0/3968595351 learned_addr learned my addr 172.21.15.67:0/3968595351 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.935+0000 7f71afe00700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b019dc00 0x7f71b019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71aea00700 1 -- 172.21.15.67:0/3968595351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0100f50 msgr2=0x7f71b0199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71aea00700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0100f50 0x7f71b0199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71aea00700 1 -- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b019dc00 msgr2=0x7f71b019e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71aea00700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b019dc00 0x7f71b019e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71aea00700 1 -- 172.21.15.67:0/3968595351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0009520 con 0x7f71b0106070 2024-03-30T02:40:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71afe00700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b019dc00 0x7f71b019e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71aea00700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 0x7f71b0199860 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f719c00e9c0 tx=0x7f719c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71a7e00700 1 -- 172.21.15.67:0/3968595351 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00c9c0 con 0x7f71b0106070 2024-03-30T02:40:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71b6200700 1 -- 172.21.15.67:0/3968595351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a8009d00 con 0x7f71b0106070 2024-03-30T02:40:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71a7e00700 1 -- 172.21.15.67:0/3968595351 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f719c010430 con 0x7f71b0106070 2024-03-30T02:40:17.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.936+0000 7f71a7e00700 1 -- 172.21.15.67:0/3968595351 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c018680 con 0x7f71b0106070 2024-03-30T02:40:17.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.937+0000 7f71b6200700 1 -- 172.21.15.67:0/3968595351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b01a61f0 con 0x7f71b0106070 2024-03-30T02:40:17.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.940+0000 7f71a7e00700 1 -- 172.21.15.67:0/3968595351 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f719c00cb20 con 0x7f71b0106070 2024-03-30T02:40:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.941+0000 7f71c9dc7b80 1 Processor -- start 2024-03-30T02:40:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.941+0000 7f71c9dc7b80 1 -- start start 2024-03-30T02:40:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.941+0000 7f71a7e00700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.941+0000 7f71a7e00700 1 -- 172.21.15.67:0/3968595351 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f719c014070 con 0x7f71b0106070 2024-03-30T02:40:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.941+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926ba5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71afe00700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926bb3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926afea10 con 0x562926bb0e70 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926afc3d0 con 0x562926ba5f70 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926af7cc0 con 0x562926ba5650 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71af400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926ba5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71af400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926ba5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44148/0 (socket says 172.21.15.67:44148) 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71af400700 1 -- 172.21.15.67:0/1032738152 learned_addr learned my addr 172.21.15.67:0/1032738152 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71afe00700 1 --2- 172.21.15.67:0/1032738152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926bb3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71aea00700 1 -- 172.21.15.67:0/1032738152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 msgr2=0x562926ba5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71aea00700 1 --2- 172.21.15.67:0/1032738152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926ba5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71aea00700 1 -- 172.21.15.67:0/1032738152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 msgr2=0x562926bb3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71aea00700 1 --2- 172.21.15.67:0/1032738152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926bb3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71aea00700 1 -- 172.21.15.67:0/1032738152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562926979430 con 0x562926ba5f70 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71aea00700 1 --2- 172.21.15.67:0/1032738152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb0930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f719c0ca4d0 tx=0x7f719c099a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61c8ae33f565ed9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:17.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.942+0000 7f71afe00700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009040 tx=0x7f71a800be60 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f719be00700 1 -- 172.21.15.67:0/1032738152 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c09aea0 con 0x562926ba5f70 2024-03-30T02:40:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f719be00700 1 -- 172.21.15.67:0/1032738152 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f719c099e90 con 0x562926ba5f70 2024-03-30T02:40:17.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f719be00700 1 -- 172.21.15.67:0/1032738152 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c010860 con 0x562926ba5f70 2024-03-30T02:40:17.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/1032738152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 msgr2=0x562926bb0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:17.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/1032738152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb0930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f719c0ca4d0 tx=0x7f719c099a50 comp rx=0 tx=0).stop 2024-03-30T02:40:17.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/1032738152 shutdown_connections 2024-03-30T02:40:17.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/1032738152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926bb3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/1032738152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926ba5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/1032738152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb0930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/1032738152 >> 172.21.15.67:0/1032738152 conn(0x562926b05720 msgr2=0x562926ba1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:17.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/1032738152 shutdown_connections 2024-03-30T02:40:17.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.943+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/1032738152 wait complete. 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.944+0000 7f71c9dc7b80 1 -- start start 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.944+0000 7f71c9dc7b80 1 Processor -- start 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926bb08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926badc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926bae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926a93e00 con 0x562926bb0e70 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926a93aa0 con 0x562926ba5f70 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56292697b720 con 0x562926ba5650 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71c9dc7b80 10 client.0 _ll_register_callbacks cb 0x562926a9d4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926badc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71afe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926bae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926badc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55466/0 (socket says 172.21.15.67:55466) 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71aea00700 1 -- 172.21.15.67:0/312790508 learned_addr learned my addr 172.21.15.67:0/312790508 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71af400700 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926bb08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71aea00700 1 -- 172.21.15.67:0/312790508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 msgr2=0x562926bb08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71aea00700 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926bb08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71aea00700 1 -- 172.21.15.67:0/312790508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 msgr2=0x562926bae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71aea00700 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926bae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71aea00700 1 -- 172.21.15.67:0/312790508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b01a61f0 con 0x562926ba5f70 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.945+0000 7f71afe00700 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926bae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71aea00700 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926badc00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f719c099b40 tx=0x7f719c010ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0f520b496d1b6d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f7198c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f7198c00700 1 -- 172.21.15.67:0/312790508 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c09aea0 con 0x562926ba5f70 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f7198c00700 1 -- 172.21.15.67:0/312790508 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f719c0105a0 con 0x562926ba5f70 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/312790508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562926ab8400 con 0x562926ba5f70 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f7198c00700 1 -- 172.21.15.67:0/312790508 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f719c010860 con 0x562926ba5f70 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71c9dc7b80 10 client.26558 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/312790508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562926925be0 con 0x562926ba5f70 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f7198c00700 1 -- 172.21.15.67:0/312790508 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562926ab8400 con 0x562926ba5f70 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f7198c00700 1 -- 172.21.15.67:0/312790508 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f719c0a1e80 con 0x562926ba5f70 2024-03-30T02:40:17.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71c9dc7b80 10 client.26558 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71c9dc7b80 10 client.26558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71c9dc7b80 20 client.26558 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562926bc0740 0x562926bc2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71c9dc7b80 4 client.26558 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:40:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.946+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/312790508 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562926af0780 con 0x562926bc0740 2024-03-30T02:40:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.947+0000 7f71af400700 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562926bc0740 0x562926bc2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.947+0000 7f71af400700 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562926bc0740 0x562926bc2b60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301f466fea166109 server_cookie=e26d07fcbe9e3ee1 in_seq=0 out_seq=0 2024-03-30T02:40:17.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.947+0000 7f7198c00700 10 client.26558 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:17.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f7198c00700 1 -- 172.21.15.67:0/312790508 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562926af0780 con 0x562926bc0740 2024-03-30T02:40:17.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f7198c00700 10 client.26558 handle_command_reply: tid=0 2024-03-30T02:40:17.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 client.26558 shutdown 2024-03-30T02:40:17.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/312790508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 msgr2=0x562926badc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:17.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926badc00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f719c099b40 tx=0x7f719c010ba0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/312790508 shutdown_connections 2024-03-30T02:40:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562926bc0740 0x562926bc2b60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926bae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926bb08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/312790508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926badc00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f7198c00700 0 client.26558 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/312790508 >> 172.21.15.67:0/312790508 conn(0x562926b05720 msgr2=0x562926ba7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/312790508 shutdown_connections 2024-03-30T02:40:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.949+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/312790508 wait complete. 2024-03-30T02:40:17.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.950+0000 7f71c9dc7b80 20 client.26558 trim_cache size 0 max 16384 2024-03-30T02:40:17.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.968+0000 7f71c9dc7b80 1 Processor -- start 2024-03-30T02:40:17.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.968+0000 7f71c9dc7b80 1 -- start start 2024-03-30T02:40:17.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.968+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926ba5650 0x562926bad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.968+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926bb0e70 0x562926ba8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926baf0d0 con 0x562926ba5650 2024-03-30T02:40:17.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926ba2710 con 0x562926ba5f70 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926bae940 con 0x562926bb0e70 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71af400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926ba5650 0x562926bad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55468/0 (socket says 172.21.15.67:55468) 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71aea00700 1 -- 172.21.15.67:0/3863025685 learned_addr learned my addr 172.21.15.67:0/3863025685 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71afe00700 1 --2- 172.21.15.67:0/3863025685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926bb0e70 0x562926ba8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71af400700 1 -- 172.21.15.67:0/3863025685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926bb0e70 msgr2=0x562926ba8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71af400700 1 --2- 172.21.15.67:0/3863025685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926bb0e70 0x562926ba8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71af400700 1 -- 172.21.15.67:0/3863025685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 msgr2=0x562926bb7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71af400700 1 --2- 172.21.15.67:0/3863025685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71af400700 1 -- 172.21.15.67:0/3863025685 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562926bc7750 con 0x562926ba5650 2024-03-30T02:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.969+0000 7f71aea00700 1 --2- 172.21.15.67:0/3863025685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.970+0000 7f71af400700 1 --2- 172.21.15.67:0/3863025685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926ba5650 0x562926bad4e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x562926bc3400 tx=0x7f71a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879aad645972a18b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.970+0000 7f71a5200700 1 -- 172.21.15.67:0/3863025685 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0017040 con 0x562926ba5650 2024-03-30T02:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.970+0000 7f71a5200700 1 -- 172.21.15.67:0/3863025685 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a0018040 con 0x562926ba5650 2024-03-30T02:40:17.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.970+0000 7f71a5200700 1 -- 172.21.15.67:0/3863025685 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0004a90 con 0x562926ba5650 2024-03-30T02:40:17.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/3863025685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926ba5650 msgr2=0x562926bad4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:17.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/3863025685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926ba5650 0x562926bad4e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x562926bc3400 tx=0x7f71a0009260 comp rx=0 tx=0).stop 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/3863025685 shutdown_connections 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/3863025685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926ba5650 0x562926bad4e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/3863025685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926bb0e70 0x562926ba8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/3863025685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/3863025685 >> 172.21.15.67:0/3863025685 conn(0x562926b05720 msgr2=0x562926bb04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/3863025685 shutdown_connections 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/3863025685 wait complete. 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 -- start start 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.971+0000 7f71c9dc7b80 1 Processor -- start 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926bb6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71c9dc7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926c14540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926bb7400 con 0x562926bb0e70 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926c14a80 con 0x562926ba5f70 2024-03-30T02:40:17.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71c9dc7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562926c14c40 con 0x562926ba5650 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71c9dc7b80 10 client.0 _ll_register_callbacks cb 0x562926a9d4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71aea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55480/0 (socket says 172.21.15.67:55480) 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71aea00700 1 -- 172.21.15.67:0/60202787 learned_addr learned my addr 172.21.15.67:0/60202787 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71afe00700 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926c14540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.972+0000 7f71af400700 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926bb6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f71af400700 1 -- 172.21.15.67:0/60202787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 msgr2=0x562926bb6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f71af400700 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f71af400700 1 -- 172.21.15.67:0/60202787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 msgr2=0x562926c14540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f71af400700 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926c14540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f71af400700 1 -- 172.21.15.67:0/60202787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562926925be0 con 0x562926ba5650 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f71af400700 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926bb6770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009970 tx=0x7f71a0021660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89fb4ccf7b1e46a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f7198c00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:40:17.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f7198c00700 1 -- 172.21.15.67:0/60202787 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00122d0 con 0x562926ba5650 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f7198c00700 1 -- 172.21.15.67:0/60202787 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f71a00129d0 con 0x562926ba5650 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/60202787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f719c099e90 con 0x562926ba5650 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.973+0000 7f7198c00700 1 -- 172.21.15.67:0/60202787 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71a0012b90 con 0x562926ba5650 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f71c9dc7b80 10 client.26521 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/60202787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f71a0009e50 con 0x562926ba5650 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f7198c00700 1 -- 172.21.15.67:0/60202787 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f719c099e90 con 0x562926ba5650 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f7198c00700 1 -- 172.21.15.67:0/60202787 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f71a0030020 con 0x562926ba5650 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f71c9dc7b80 10 client.26521 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f71c9dc7b80 10 client.26521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f71c9dc7b80 20 client.26521 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562926c164b0 0x562926c188d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f71c9dc7b80 4 client.26521 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/60202787 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562926946d70 con 0x562926c164b0 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.974+0000 7f71aea00700 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562926c164b0 0x562926c188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:17.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.975+0000 7f71aea00700 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562926c164b0 0x562926c188d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6569e2d2ffaf3a43 server_cookie=1451d157bdc6e1ef in_seq=0 out_seq=0 2024-03-30T02:40:17.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:17.975+0000 7f7198c00700 10 client.26521 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:18 smithi067 ceph-mon[37808]: pgmap v100: 129 pgs: 129 active+clean; 517 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 328 op/s 2024-03-30T02:40:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:18 smithi170 ceph-mon[78798]: pgmap v100: 129 pgs: 129 active+clean; 517 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 328 op/s 2024-03-30T02:40:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:18 smithi082 ceph-mon[79176]: pgmap v100: 129 pgs: 129 active+clean; 517 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 328 op/s 2024-03-30T02:40:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.196+0000 7f7198c00700 1 -- 172.21.15.67:0/60202787 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562926946d70 con 0x562926c164b0 2024-03-30T02:40:19.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.196+0000 7f7198c00700 10 client.26521 handle_command_reply: tid=0 2024-03-30T02:40:19.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.197+0000 7f71c9dc7b80 1 client.26521 shutdown 2024-03-30T02:40:19.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.198+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/60202787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 msgr2=0x562926bb6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.198+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926bb6770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009970 tx=0x7f71a0021660 comp rx=0 tx=0).stop 2024-03-30T02:40:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.199+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/60202787 shutdown_connections 2024-03-30T02:40:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.199+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562926c164b0 0x562926c188d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.199+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562926bb0e70 0x562926c14540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.199+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562926ba5650 0x562926bb6770 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.199+0000 7f71c9dc7b80 1 --2- 172.21.15.67:0/60202787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562926ba5f70 0x562926bb6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.199+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/60202787 >> 172.21.15.67:0/60202787 conn(0x562926b05720 msgr2=0x562926ba0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.200+0000 7f7198c00700 0 client.26521 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.200+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/60202787 shutdown_connections 2024-03-30T02:40:19.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.200+0000 7f71c9dc7b80 1 -- 172.21.15.67:0/60202787 wait complete. 2024-03-30T02:40:19.201 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:40:19.202 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:40:19.202 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "5f5c22d4-f0cb-42f3-b02c-98f21450b6a1", 2024-03-30T02:40:19.202 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:40:19.202 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:40:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.201+0000 7f71c9dc7b80 20 client.26521 trim_cache size 0 max 16384 2024-03-30T02:40:19.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.203+0000 7f71b6200700 1 -- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 msgr2=0x7f71940a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.203+0000 7f71b6200700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009040 tx=0x7f71a800be60 comp rx=0 tx=0).stop 2024-03-30T02:40:19.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.204+0000 7f71b6200700 1 -- 172.21.15.67:0/3968595351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 msgr2=0x7f71b0199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.204+0000 7f71b6200700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 0x7f71b0199860 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f719c00e9c0 tx=0x7f719c00ecd0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.204+0000 7f71b6200700 1 -- 172.21.15.67:0/3968595351 shutdown_connections 2024-03-30T02:40:19.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.204+0000 7f71b6200700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71940a1290 0x7f71940a3750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.205+0000 7f71b6200700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71b019dc00 0x7f71b019e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.205+0000 7f71b6200700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0100f50 0x7f71b0199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.205+0000 7f71b6200700 1 --2- 172.21.15.67:0/3968595351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b0106070 0x7f71b0199860 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.205+0000 7f71b6200700 1 -- 172.21.15.67:0/3968595351 >> 172.21.15.67:0/3968595351 conn(0x7f71b00fc7f0 msgr2=0x7f71b00feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.205+0000 7f71b6200700 1 -- 172.21.15.67:0/3968595351 shutdown_connections 2024-03-30T02:40:19.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.205+0000 7f71b6200700 1 -- 172.21.15.67:0/3968595351 wait complete. 2024-03-30T02:40:19.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:40:19.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.554+0000 7f3a31a00700 1 -- 172.21.15.67:0/3001690749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 msgr2=0x7f3a2c073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.554+0000 7f3a31a00700 1 --2- 172.21.15.67:0/3001690749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 0x7f3a2c073530 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c01a9a0 tx=0x7f3a1c01acb0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.555+0000 7f3a31a00700 1 -- 172.21.15.67:0/3001690749 shutdown_connections 2024-03-30T02:40:19.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.555+0000 7f3a31a00700 1 --2- 172.21.15.67:0/3001690749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2c111d00 0x7f3a2c1140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.555+0000 7f3a31a00700 1 --2- 172.21.15.67:0/3001690749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 0x7f3a2c073530 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.555+0000 7f3a31a00700 1 --2- 172.21.15.67:0/3001690749 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a2c073a70 0x7f3a2c1117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.555+0000 7f3a31a00700 1 -- 172.21.15.67:0/3001690749 >> 172.21.15.67:0/3001690749 conn(0x7f3a2c0fc7c0 msgr2=0x7f3a2c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:19.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.555+0000 7f3a31a00700 1 -- 172.21.15.67:0/3001690749 shutdown_connections 2024-03-30T02:40:19.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.555+0000 7f3a31a00700 1 -- 172.21.15.67:0/3001690749 wait complete. 2024-03-30T02:40:19.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.557+0000 7f3a31a00700 1 Processor -- start 2024-03-30T02:40:19.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.557+0000 7f3a31a00700 1 -- start start 2024-03-30T02:40:19.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.558+0000 7f3a31a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 0x7f3a2c110f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.558+0000 7f3a31a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2c073a70 0x7f3a2c1114d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.558+0000 7f3a31a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a2c111d00 0x7f3a2c10d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.558+0000 7f3a31a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c10d890 con 0x7f3a2c111d00 2024-03-30T02:40:19.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.558+0000 7f3a31a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c10da00 con 0x7f3a2c073a70 2024-03-30T02:40:19.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.558+0000 7f3a31a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c10dbc0 con 0x7f3a2c073150 2024-03-30T02:40:19.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.558+0000 7f3a2aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 0x7f3a2c110f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.559+0000 7f3a2aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 0x7f3a2c110f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44200/0 (socket says 172.21.15.67:44200) 2024-03-30T02:40:19.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.559+0000 7f3a2aa00700 1 -- 172.21.15.67:0/464580324 learned_addr learned my addr 172.21.15.67:0/464580324 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:19.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.559+0000 7f3a2aa00700 1 -- 172.21.15.67:0/464580324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2c073a70 msgr2=0x7f3a2c1114d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T02:40:19.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.559+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2c073a70 0x7f3a2c1114d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.559+0000 7f3a2aa00700 1 -- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a2c111d00 msgr2=0x7f3a2c10d100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:40:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.559+0000 7f3a2b400700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a2c111d00 0x7f3a2c10d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.559+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a2c111d00 0x7f3a2c10d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.560+0000 7f3a2aa00700 1 -- 172.21.15.67:0/464580324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a18009520 con 0x7f3a2c073150 2024-03-30T02:40:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.560+0000 7f3a2b400700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a2c111d00 0x7f3a2c10d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.560+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 0x7f3a2c110f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005660 tx=0x7f3a1c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.560+0000 7f3a23400700 1 -- 172.21.15.67:0/464580324 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c003930 con 0x7f3a2c073150 2024-03-30T02:40:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.560+0000 7f3a23400700 1 -- 172.21.15.67:0/464580324 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1c0283f0 con 0x7f3a2c073150 2024-03-30T02:40:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.560+0000 7f3a23400700 1 -- 172.21.15.67:0/464580324 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c028ea0 con 0x7f3a2c073150 2024-03-30T02:40:19.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.561+0000 7f3a31a00700 1 -- 172.21.15.67:0/464580324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c01a600 con 0x7f3a2c073150 2024-03-30T02:40:19.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.561+0000 7f3a31a00700 1 -- 172.21.15.67:0/464580324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c06a9b0 con 0x7f3a2c073150 2024-03-30T02:40:19.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.568+0000 7f3a23400700 1 -- 172.21.15.67:0/464580324 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3a1c003a90 con 0x7f3a2c073150 2024-03-30T02:40:19.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.570+0000 7f3a4551db80 1 Processor -- start 2024-03-30T02:40:19.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.570+0000 7f3a4551db80 1 -- start start 2024-03-30T02:40:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d4395d0 0x56167d4399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d439ef0 0x56167d4448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d4471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a4551db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d392990 con 0x56167d444df0 2024-03-30T02:40:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a4551db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d390350 con 0x56167d439ef0 2024-03-30T02:40:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a4551db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d38bc40 con 0x56167d4395d0 2024-03-30T02:40:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a2b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d4395d0 0x56167d4399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a2b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d4395d0 0x56167d4399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44216/0 (socket says 172.21.15.67:44216) 2024-03-30T02:40:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a2b400700 1 -- 172.21.15.67:0/3185798322 learned_addr learned my addr 172.21.15.67:0/3185798322 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.571+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d439ef0 0x56167d4448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.572+0000 7f3a2a000700 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d4471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.572+0000 7f3a2aa00700 1 -- 172.21.15.67:0/3185798322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d4395d0 msgr2=0x56167d4399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.572+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d4395d0 0x56167d4399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.572+0000 7f3a2aa00700 1 -- 172.21.15.67:0/3185798322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 msgr2=0x56167d4471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:19.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.572+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d4471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.572+0000 7f3a2aa00700 1 -- 172.21.15.67:0/3185798322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56167d20d3b0 con 0x56167d439ef0 2024-03-30T02:40:19.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.572+0000 7f3a2b400700 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d4395d0 0x56167d4399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:19.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.573+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d439ef0 0x56167d4448b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0dc450 tx=0x7f3a1c0e73c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470890f7eaaa1f26 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:19.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.573+0000 7f3a17e00700 1 -- 172.21.15.67:0/3185798322 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0e7a90 con 0x56167d439ef0 2024-03-30T02:40:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.573+0000 7f3a17e00700 1 -- 172.21.15.67:0/3185798322 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1c0e7bf0 con 0x56167d439ef0 2024-03-30T02:40:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.573+0000 7f3a17e00700 1 -- 172.21.15.67:0/3185798322 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0ed660 con 0x56167d439ef0 2024-03-30T02:40:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.573+0000 7f3a23400700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a100a1310 0x7f3a100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.574+0000 7f3a23400700 1 -- 172.21.15.67:0/464580324 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f3a1c0df050 con 0x7f3a2c073150 2024-03-30T02:40:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.574+0000 7f3a2b400700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a100a1310 0x7f3a100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.574+0000 7f3a4551db80 1 -- 172.21.15.67:0/3185798322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d439ef0 msgr2=0x56167d4448b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.574+0000 7f3a4551db80 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d439ef0 0x56167d4448b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0dc450 tx=0x7f3a1c0e73c0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.575+0000 7f3a4551db80 1 -- 172.21.15.67:0/3185798322 shutdown_connections 2024-03-30T02:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.575+0000 7f3a4551db80 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d4471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.575+0000 7f3a4551db80 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d4395d0 0x56167d4399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.575+0000 7f3a4551db80 1 --2- 172.21.15.67:0/3185798322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d439ef0 0x56167d4448b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.575+0000 7f3a4551db80 1 -- 172.21.15.67:0/3185798322 >> 172.21.15.67:0/3185798322 conn(0x56167d3996a0 msgr2=0x56167d435a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:19.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.575+0000 7f3a2b400700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a100a1310 0x7f3a100a37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400e6a0 tx=0x7f3a24009040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:19.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.575+0000 7f3a4551db80 1 -- 172.21.15.67:0/3185798322 shutdown_connections 2024-03-30T02:40:19.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.575+0000 7f3a4551db80 1 -- 172.21.15.67:0/3185798322 wait complete. 2024-03-30T02:40:19.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.576+0000 7f3a4551db80 1 -- start start 2024-03-30T02:40:19.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.576+0000 7f3a4551db80 1 Processor -- start 2024-03-30T02:40:19.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.578+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 0x56167d4447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.578+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d439ef0 0x56167d441b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.578+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d442130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.578+0000 7f3a4551db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d327d80 con 0x56167d444df0 2024-03-30T02:40:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.578+0000 7f3a4551db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d327a20 con 0x56167d4395d0 2024-03-30T02:40:19.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.578+0000 7f3a4551db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d20f6a0 con 0x56167d439ef0 2024-03-30T02:40:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.578+0000 7f3a4551db80 10 client.0 _ll_register_callbacks cb 0x56167d331470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.578+0000 7f3a2b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d442130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.579+0000 7f3a2b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d442130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40022/0 (socket says 172.21.15.67:40022) 2024-03-30T02:40:19.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.579+0000 7f3a2b400700 1 -- 172.21.15.67:0/4219229206 learned_addr learned my addr 172.21.15.67:0/4219229206 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:19.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.579+0000 7f3a2b400700 1 -- 172.21.15.67:0/4219229206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d439ef0 msgr2=0x56167d441b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:40:19.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.579+0000 7f3a2b400700 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d439ef0 0x56167d441b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.579+0000 7f3a2b400700 1 -- 172.21.15.67:0/4219229206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 msgr2=0x56167d4447e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:40:19.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.579+0000 7f3a2b400700 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 0x56167d4447e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.579+0000 7f3a2b400700 1 -- 172.21.15.67:0/4219229206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a24007d10 con 0x56167d444df0 2024-03-30T02:40:19.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.580+0000 7f3a2b400700 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d442130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3a24016ee0 tx=0x7f3a24004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a8a14f434dba1b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.580+0000 7f3a14c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:40:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.581+0000 7f3a14c00700 1 -- 172.21.15.67:0/4219229206 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24014d70 con 0x56167d444df0 2024-03-30T02:40:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.581+0000 7f3a4551db80 1 -- 172.21.15.67:0/4219229206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56167d34c380 con 0x56167d444df0 2024-03-30T02:40:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.581+0000 7f3a14c00700 1 -- 172.21.15.67:0/4219229206 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f3a24021440 con 0x56167d444df0 2024-03-30T02:40:19.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.582+0000 7f3a14c00700 1 -- 172.21.15.67:0/4219229206 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a24014d70 con 0x56167d444df0 2024-03-30T02:40:19.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.584+0000 7f3a4551db80 10 client.16476 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:19.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.584+0000 7f3a4551db80 1 -- 172.21.15.67:0/4219229206 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f3a2c06a9b0 con 0x56167d444df0 2024-03-30T02:40:19.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.585+0000 7f3a14c00700 1 -- 172.21.15.67:0/4219229206 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56167d34c380 con 0x56167d444df0 2024-03-30T02:40:19.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.586+0000 7f3a14c00700 1 -- 172.21.15.67:0/4219229206 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3a24013550 con 0x56167d444df0 2024-03-30T02:40:19.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.586+0000 7f3a4551db80 10 client.16476 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:19.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.586+0000 7f3a4551db80 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:19.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.586+0000 7f3a4551db80 20 client.16476 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:19.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.586+0000 7f3a4551db80 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56167d4547d0 0x56167d456bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.586+0000 7f3a4551db80 4 client.16476 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:40:19.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.586+0000 7f3a4551db80 1 -- 172.21.15.67:0/4219229206 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56167d384700 con 0x56167d4547d0 2024-03-30T02:40:19.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.586+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56167d4547d0 0x56167d456bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.588+0000 7f3a14c00700 10 client.16476 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:19.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.588+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56167d4547d0 0x56167d456bf0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d00d36ca66dedbef server_cookie=2cf66e1a99941494 in_seq=0 out_seq=0 2024-03-30T02:40:19.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.592+0000 7f3a14c00700 1 -- 172.21.15.67:0/4219229206 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56167d384700 con 0x56167d4547d0 2024-03-30T02:40:19.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.592+0000 7f3a14c00700 10 client.16476 handle_command_reply: tid=0 2024-03-30T02:40:19.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a4551db80 1 client.16476 shutdown 2024-03-30T02:40:19.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a4551db80 1 -- 172.21.15.67:0/4219229206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 msgr2=0x56167d442130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a4551db80 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d442130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3a24016ee0 tx=0x7f3a24004b40 comp rx=0 tx=0).stop 2024-03-30T02:40:19.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a4551db80 1 -- 172.21.15.67:0/4219229206 shutdown_connections 2024-03-30T02:40:19.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a4551db80 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56167d4547d0 0x56167d456bf0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a4551db80 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d444df0 0x56167d442130 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a4551db80 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 0x56167d4447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a14c00700 0 client.16476 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:19.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a4551db80 1 --2- 172.21.15.67:0/4219229206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d439ef0 0x56167d441b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.593+0000 7f3a4551db80 1 -- 172.21.15.67:0/4219229206 >> 172.21.15.67:0/4219229206 conn(0x56167d3996a0 msgr2=0x56167d43b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:19.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.595+0000 7f3a4551db80 1 -- 172.21.15.67:0/4219229206 shutdown_connections 2024-03-30T02:40:19.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.596+0000 7f3a4551db80 1 -- 172.21.15.67:0/4219229206 wait complete. 2024-03-30T02:40:19.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.597+0000 7f3a4551db80 20 client.16476 trim_cache size 0 max 16384 2024-03-30T02:40:19.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.621+0000 7f3a4551db80 1 Processor -- start 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.621+0000 7f3a4551db80 1 -- start start 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d4395d0 0x56167d44b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d439ef0 0x56167d4403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d444df0 0x56167d440930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a4551db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d440e70 con 0x56167d4395d0 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a4551db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d436660 con 0x56167d444df0 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a4551db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d44bd20 con 0x56167d439ef0 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a2aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d4395d0 0x56167d44b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a2b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d444df0 0x56167d440930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a2aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d4395d0 0x56167d44b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40034/0 (socket says 172.21.15.67:40034) 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a2aa00700 1 -- 172.21.15.67:0/698118955 learned_addr learned my addr 172.21.15.67:0/698118955 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:19.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a2a000700 1 --2- 172.21.15.67:0/698118955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d439ef0 0x56167d4403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.622+0000 7f3a2b400700 1 -- 172.21.15.67:0/698118955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d439ef0 msgr2=0x56167d4403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:19.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.623+0000 7f3a2b400700 1 --2- 172.21.15.67:0/698118955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d439ef0 0x56167d4403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.623+0000 7f3a2b400700 1 -- 172.21.15.67:0/698118955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d4395d0 msgr2=0x56167d44b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:19.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.623+0000 7f3a2b400700 1 --2- 172.21.15.67:0/698118955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d4395d0 0x56167d44b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.623+0000 7f3a2b400700 1 -- 172.21.15.67:0/698118955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56167d1b9b60 con 0x56167d444df0 2024-03-30T02:40:19.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.623+0000 7f3a2b400700 1 --2- 172.21.15.67:0/698118955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d444df0 0x56167d440930 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3a24013980 tx=0x7f3a240148f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ab45ae802c2cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:19.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.623+0000 7f3a20c00700 1 -- 172.21.15.67:0/698118955 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400edf0 con 0x56167d444df0 2024-03-30T02:40:19.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.623+0000 7f3a20c00700 1 -- 172.21.15.67:0/698118955 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a2401a070 con 0x56167d444df0 2024-03-30T02:40:19.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.623+0000 7f3a20c00700 1 -- 172.21.15.67:0/698118955 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24004e10 con 0x56167d444df0 2024-03-30T02:40:19.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.624+0000 7f3a4551db80 1 -- 172.21.15.67:0/698118955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d444df0 msgr2=0x56167d440930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.624+0000 7f3a4551db80 1 --2- 172.21.15.67:0/698118955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d444df0 0x56167d440930 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3a24013980 tx=0x7f3a240148f0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.624+0000 7f3a4551db80 1 -- 172.21.15.67:0/698118955 shutdown_connections 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.624+0000 7f3a4551db80 1 --2- 172.21.15.67:0/698118955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d4395d0 0x56167d44b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.624+0000 7f3a4551db80 1 --2- 172.21.15.67:0/698118955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d444df0 0x56167d440930 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.624+0000 7f3a4551db80 1 --2- 172.21.15.67:0/698118955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d439ef0 0x56167d4403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.624+0000 7f3a4551db80 1 -- 172.21.15.67:0/698118955 >> 172.21.15.67:0/698118955 conn(0x56167d3996a0 msgr2=0x56167d441ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.624+0000 7f3a4551db80 1 -- 172.21.15.67:0/698118955 shutdown_connections 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.624+0000 7f3a4551db80 1 -- 172.21.15.67:0/698118955 wait complete. 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.625+0000 7f3a4551db80 1 -- start start 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.625+0000 7f3a4551db80 1 Processor -- start 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.625+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 0x56167d44b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.625+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d439ef0 0x56167d449c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.625+0000 7f3a4551db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d444df0 0x56167d44a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.625+0000 7f3a4551db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d44a9b0 con 0x56167d439ef0 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a4551db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d44ab20 con 0x56167d4395d0 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a4551db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167d4a86d0 con 0x56167d444df0 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a4551db80 10 client.0 _ll_register_callbacks cb 0x56167d331470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d444df0 0x56167d44a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 0x56167d44b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d444df0 0x56167d44a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44226/0 (socket says 172.21.15.67:44226) 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2b400700 1 -- 172.21.15.67:0/2887850962 learned_addr learned my addr 172.21.15.67:0/2887850962 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2a000700 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d439ef0 0x56167d449c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2b400700 1 -- 172.21.15.67:0/2887850962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 msgr2=0x56167d44b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2b400700 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 0x56167d44b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2b400700 1 -- 172.21.15.67:0/2887850962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d439ef0 msgr2=0x56167d449c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2b400700 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d439ef0 0x56167d449c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2b400700 1 -- 172.21.15.67:0/2887850962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c06a9b0 con 0x56167d444df0 2024-03-30T02:40:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.626+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 0x56167d44b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:19.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.627+0000 7f3a2b400700 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d444df0 0x56167d44a2a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a24004bd0 tx=0x7f3a24014b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7e1d6b4982a2daa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:19.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.627+0000 7f3a14c00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:40:19.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.627+0000 7f3a14c00700 1 -- 172.21.15.67:0/2887850962 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24013be0 con 0x56167d444df0 2024-03-30T02:40:19.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.627+0000 7f3a4551db80 1 -- 172.21.15.67:0/2887850962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a1c0e7bf0 con 0x56167d444df0 2024-03-30T02:40:19.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.627+0000 7f3a14c00700 1 -- 172.21.15.67:0/2887850962 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f3a24013070 con 0x56167d444df0 2024-03-30T02:40:19.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.627+0000 7f3a14c00700 1 -- 172.21.15.67:0/2887850962 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a24013230 con 0x56167d444df0 2024-03-30T02:40:19.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.627+0000 7f3a4551db80 10 client.26536 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:19.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.627+0000 7f3a4551db80 1 -- 172.21.15.67:0/2887850962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56167d351ea0 con 0x56167d444df0 2024-03-30T02:40:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.628+0000 7f3a14c00700 1 -- 172.21.15.67:0/2887850962 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1c0e7bf0 con 0x56167d444df0 2024-03-30T02:40:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.628+0000 7f3a14c00700 1 -- 172.21.15.67:0/2887850962 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3a24021440 con 0x56167d444df0 2024-03-30T02:40:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.628+0000 7f3a4551db80 10 client.26536 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.628+0000 7f3a4551db80 10 client.26536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.628+0000 7f3a4551db80 20 client.26536 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.628+0000 7f3a4551db80 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56167d4a9b50 0x56167d4abf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.628+0000 7f3a4551db80 4 client.26536 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:40:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.628+0000 7f3a4551db80 1 -- 172.21.15.67:0/2887850962 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56167d1dacf0 con 0x56167d4a9b50 2024-03-30T02:40:19.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.628+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56167d4a9b50 0x56167d4abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:19.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.629+0000 7f3a2aa00700 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56167d4a9b50 0x56167d4abf70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d82887753930627 server_cookie=d67fa6e653519ea4 in_seq=0 out_seq=0 2024-03-30T02:40:19.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.629+0000 7f3a14c00700 10 client.26536 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:19.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a14c00700 1 -- 172.21.15.67:0/2887850962 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56167d1dacf0 con 0x56167d4a9b50 2024-03-30T02:40:19.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a14c00700 10 client.26536 handle_command_reply: tid=0 2024-03-30T02:40:19.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 client.26536 shutdown 2024-03-30T02:40:19.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 -- 172.21.15.67:0/2887850962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d444df0 msgr2=0x56167d44a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d444df0 0x56167d44a2a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a24004bd0 tx=0x7f3a24014b70 comp rx=0 tx=0).stop 2024-03-30T02:40:19.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 -- 172.21.15.67:0/2887850962 shutdown_connections 2024-03-30T02:40:19.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56167d444df0 0x56167d44a2a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56167d4395d0 0x56167d44b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56167d4a9b50 0x56167d4abf70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 --2- 172.21.15.67:0/2887850962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56167d439ef0 0x56167d449c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 -- 172.21.15.67:0/2887850962 >> 172.21.15.67:0/2887850962 conn(0x56167d3996a0 msgr2=0x56167d441a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:19.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a14c00700 0 client.26536 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:19.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 -- 172.21.15.67:0/2887850962 shutdown_connections 2024-03-30T02:40:19.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.631+0000 7f3a4551db80 1 -- 172.21.15.67:0/2887850962 wait complete. 2024-03-30T02:40:19.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a4551db80 20 client.26536 trim_cache size 0 max 16384 2024-03-30T02:40:19.633 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:40:19.633 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (3 inodes in the stack)", 2024-03-30T02:40:19.634 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:40:19.634 INFO:teuthology.orchestra.run.smithi067.stdout: "5f5c22d4-f0cb-42f3-b02c-98f21450b6a1": { 2024-03-30T02:40:19.634 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:40:19.634 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "5f5c22d4-f0cb-42f3-b02c-98f21450b6a1", 2024-03-30T02:40:19.634 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:40:19.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:40:19.634 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:40:19.634 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a31a00700 1 -- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a100a1310 msgr2=0x7f3a100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a31a00700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a100a1310 0x7f3a100a37d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400e6a0 tx=0x7f3a24009040 comp rx=0 tx=0).stop 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a31a00700 1 -- 172.21.15.67:0/464580324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 msgr2=0x7f3a2c110f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a31a00700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 0x7f3a2c110f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005660 tx=0x7f3a1c005740 comp rx=0 tx=0).stop 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a31a00700 1 -- 172.21.15.67:0/464580324 shutdown_connections 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a31a00700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a100a1310 0x7f3a100a37d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a31a00700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a2c111d00 0x7f3a2c10d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a31a00700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a2c073150 0x7f3a2c110f70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.632+0000 7f3a31a00700 1 --2- 172.21.15.67:0/464580324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2c073a70 0x7f3a2c1114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.633+0000 7f3a31a00700 1 -- 172.21.15.67:0/464580324 >> 172.21.15.67:0/464580324 conn(0x7f3a2c0fc7c0 msgr2=0x7f3a2c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:19.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.633+0000 7f3a31a00700 1 -- 172.21.15.67:0/464580324 shutdown_connections 2024-03-30T02:40:19.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:19.633+0000 7f3a31a00700 1 -- 172.21.15.67:0/464580324 wait complete. 2024-03-30T02:40:19.654 INFO:tasks.cephfs.filesystem:scrub status for tag:5f5c22d4-f0cb-42f3-b02c-98f21450b6a1 - {'path': '/', 'tag': '5f5c22d4-f0cb-42f3-b02c-98f21450b6a1', 'options': 'recursive,force'} 2024-03-30T02:40:20.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:20 smithi067 ceph-mon[37808]: pgmap v101: 129 pgs: 129 active+clean; 528 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.3 MiB/s wr, 346 op/s 2024-03-30T02:40:20.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:20 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:40:20.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:20.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:20 smithi170 ceph-mon[78798]: pgmap v101: 129 pgs: 129 active+clean; 528 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.3 MiB/s wr, 346 op/s 2024-03-30T02:40:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:20 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:40:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:20 smithi082 ceph-mon[79176]: pgmap v101: 129 pgs: 129 active+clean; 528 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.3 MiB/s wr, 346 op/s 2024-03-30T02:40:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:20 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:40:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:22.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:22 smithi067 ceph-mon[37808]: pgmap v102: 129 pgs: 129 active+clean; 667 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 16 MiB/s wr, 783 op/s 2024-03-30T02:40:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:22 smithi170 ceph-mon[78798]: pgmap v102: 129 pgs: 129 active+clean; 667 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 16 MiB/s wr, 783 op/s 2024-03-30T02:40:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:22 smithi082 ceph-mon[79176]: pgmap v102: 129 pgs: 129 active+clean; 667 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 16 MiB/s wr, 783 op/s 2024-03-30T02:40:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:24 smithi067 ceph-mon[37808]: pgmap v103: 129 pgs: 129 active+clean; 667 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 16 MiB/s wr, 769 op/s 2024-03-30T02:40:24.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:24 smithi170 ceph-mon[78798]: pgmap v103: 129 pgs: 129 active+clean; 667 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 16 MiB/s wr, 769 op/s 2024-03-30T02:40:24.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:24 smithi082 ceph-mon[79176]: pgmap v103: 129 pgs: 129 active+clean; 667 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 16 MiB/s wr, 769 op/s 2024-03-30T02:40:25.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:25.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:25.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:25.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:25.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:25.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:26 smithi067 ceph-mon[37808]: pgmap v104: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 57 MiB/s wr, 1.45k op/s 2024-03-30T02:40:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:26.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:26.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:26 smithi170 ceph-mon[78798]: pgmap v104: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 57 MiB/s wr, 1.45k op/s 2024-03-30T02:40:26.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:26.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:26.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:26.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:26.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:26.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:26 smithi082 ceph-mon[79176]: pgmap v104: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 57 MiB/s wr, 1.45k op/s 2024-03-30T02:40:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:27.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:27 smithi082 ceph-mon[79176]: pgmap v105: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 54 MiB/s wr, 1.27k op/s 2024-03-30T02:40:27.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:27 smithi067 ceph-mon[37808]: pgmap v105: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 54 MiB/s wr, 1.27k op/s 2024-03-30T02:40:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:27 smithi170 ceph-mon[78798]: pgmap v105: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 54 MiB/s wr, 1.27k op/s 2024-03-30T02:40:28.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:29.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:29 smithi067 ceph-mon[37808]: pgmap v106: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 54 MiB/s wr, 1.34k op/s 2024-03-30T02:40:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:29 smithi170 ceph-mon[78798]: pgmap v106: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 54 MiB/s wr, 1.34k op/s 2024-03-30T02:40:30.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:29 smithi082 ceph-mon[79176]: pgmap v106: 129 pgs: 129 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 54 MiB/s wr, 1.34k op/s 2024-03-30T02:40:32.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:32 smithi067 ceph-mon[37808]: pgmap v107: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 55 MiB/s wr, 2.77k op/s 2024-03-30T02:40:32.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:32 smithi170 ceph-mon[78798]: pgmap v107: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 55 MiB/s wr, 2.77k op/s 2024-03-30T02:40:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:32 smithi082 ceph-mon[79176]: pgmap v107: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 55 MiB/s wr, 2.77k op/s 2024-03-30T02:40:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:34.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:34 smithi067 ceph-mon[37808]: pgmap v108: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 43 MiB/s wr, 2.21k op/s 2024-03-30T02:40:34.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:34 smithi170 ceph-mon[78798]: pgmap v108: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 43 MiB/s wr, 2.21k op/s 2024-03-30T02:40:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:34 smithi082 ceph-mon[79176]: pgmap v108: 129 pgs: 129 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 43 MiB/s wr, 2.21k op/s 2024-03-30T02:40:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:36 smithi067 ceph-mon[37808]: pgmap v109: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 44 MiB/s wr, 4.47k op/s 2024-03-30T02:40:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:36.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:36 smithi170 ceph-mon[78798]: pgmap v109: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 44 MiB/s wr, 4.47k op/s 2024-03-30T02:40:36.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:36.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:36 smithi082 ceph-mon[79176]: pgmap v109: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 44 MiB/s wr, 4.47k op/s 2024-03-30T02:40:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:38 smithi067 ceph-mon[37808]: pgmap v110: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 3.79k op/s 2024-03-30T02:40:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:38 smithi170 ceph-mon[78798]: pgmap v110: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 3.79k op/s 2024-03-30T02:40:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:38 smithi082 ceph-mon[79176]: pgmap v110: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 3.79k op/s 2024-03-30T02:40:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.801 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.802 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:39.803 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: pgmap v111: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 3.81k op/s 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:40.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: pgmap v111: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 3.81k op/s 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:40.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: pgmap v111: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 3.81k op/s 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:42 smithi067 ceph-mon[37808]: pgmap v112: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 4.49k op/s 2024-03-30T02:40:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:42 smithi170 ceph-mon[78798]: pgmap v112: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 4.49k op/s 2024-03-30T02:40:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:42 smithi082 ceph-mon[79176]: pgmap v112: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 4.49k op/s 2024-03-30T02:40:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: pgmap v113: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 3.04k op/s 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: pgmap v113: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 3.04k op/s 2024-03-30T02:40:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: pgmap v113: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 3.04k op/s 2024-03-30T02:40:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:45.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:45.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: pgmap v114: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 3.75k op/s 2024-03-30T02:40:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: pgmap v114: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 3.75k op/s 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: pgmap v114: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.1 MiB/s wr, 3.75k op/s 2024-03-30T02:40:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:48 smithi067 ceph-mon[37808]: pgmap v115: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 1.49k op/s 2024-03-30T02:40:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:48 smithi170 ceph-mon[78798]: pgmap v115: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 1.49k op/s 2024-03-30T02:40:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:48 smithi082 ceph-mon[79176]: pgmap v115: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 1.49k op/s 2024-03-30T02:40:49.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:40:49.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:49.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.869+0000 7f214ca00700 1 -- 172.21.15.67:0/1826372134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148101510 msgr2=0x7f214810d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:49.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.869+0000 7f214ca00700 1 --2- 172.21.15.67:0/1826372134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148101510 0x7f214810d9d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f214000b0d0 tx=0x7f214000b3e0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.869+0000 7f214ca00700 1 -- 172.21.15.67:0/1826372134 shutdown_connections 2024-03-30T02:40:49.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.869+0000 7f214ca00700 1 --2- 172.21.15.67:0/1826372134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2148106c10 0x7f2148106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.869+0000 7f214ca00700 1 --2- 172.21.15.67:0/1826372134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148101510 0x7f214810d9d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.869+0000 7f214ca00700 1 --2- 172.21.15.67:0/1826372134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2148100b50 0x7f2148100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.869+0000 7f214ca00700 1 -- 172.21.15.67:0/1826372134 >> 172.21.15.67:0/1826372134 conn(0x7f21480fc620 msgr2=0x7f21480fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:49.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.869+0000 7f214ca00700 1 -- 172.21.15.67:0/1826372134 shutdown_connections 2024-03-30T02:40:49.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.869+0000 7f214ca00700 1 -- 172.21.15.67:0/1826372134 wait complete. 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.870+0000 7f214ca00700 1 Processor -- start 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f214ca00700 1 -- start start 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f214ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148100b50 0x7f2148199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f214ca00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2148101510 0x7f2148199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f214ca00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2148106c10 0x7f214819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f214ca00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214819e250 con 0x7f2148106c10 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f214ca00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214819e3c0 con 0x7f2148100b50 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f214ca00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214819e580 con 0x7f2148101510 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f2146800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148100b50 0x7f2148199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f2146800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148100b50 0x7f2148199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40430/0 (socket says 172.21.15.67:40430) 2024-03-30T02:40:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f2146800700 1 -- 172.21.15.67:0/3156507072 learned_addr learned my addr 172.21.15.67:0/3156507072 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.871+0000 7f2147200700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2148106c10 0x7f214819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f2146800700 1 -- 172.21.15.67:0/3156507072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2148101510 msgr2=0x7f2148199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f2146800700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2148101510 0x7f2148199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f2146800700 1 -- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2148106c10 msgr2=0x7f214819dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f2146800700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2148106c10 0x7f214819dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f2146800700 1 -- 172.21.15.67:0/3156507072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2140009d00 con 0x7f2148100b50 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f2146800700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148100b50 0x7f2148199330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f213800e9a0 tx=0x7f213800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f213f400700 1 -- 172.21.15.67:0/3156507072 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2138009790 con 0x7f2148100b50 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f214ca00700 1 -- 172.21.15.67:0/3156507072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214819e860 con 0x7f2148100b50 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f213f400700 1 -- 172.21.15.67:0/3156507072 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f213800cbd0 con 0x7f2148100b50 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f213f400700 1 -- 172.21.15.67:0/3156507072 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2138018680 con 0x7f2148100b50 2024-03-30T02:40:49.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.872+0000 7f214ca00700 1 -- 172.21.15.67:0/3156507072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21481a60a0 con 0x7f2148100b50 2024-03-30T02:40:49.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.877+0000 7f213f400700 1 -- 172.21.15.67:0/3156507072 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f213800cd40 con 0x7f2148100b50 2024-03-30T02:40:49.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f2160e89b80 1 Processor -- start 2024-03-30T02:40:49.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f2160e89b80 1 -- start start 2024-03-30T02:40:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f213f400700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f212c0a1290 0x7f212c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb66eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb679a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f213f400700 1 -- 172.21.15.67:0/3156507072 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f2138014070 con 0x7f2148100b50 2024-03-30T02:40:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb679fc0 0x563bfb67c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f2160e89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb5c7990 con 0x563bfb679fc0 2024-03-30T02:40:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f2160e89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb5c5350 con 0x563bfb66f0c0 2024-03-30T02:40:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f2160e89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb5c0c40 con 0x563bfb66e7a0 2024-03-30T02:40:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.878+0000 7f2147200700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f212c0a1290 0x7f212c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2145e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb679a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2145e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb679a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40432/0 (socket says 172.21.15.67:40432) 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2145e00700 1 -- 172.21.15.67:0/1000191213 learned_addr learned my addr 172.21.15.67:0/1000191213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2147200700 1 --2- 172.21.15.67:0/1000191213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb679fc0 0x563bfb67c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2145e00700 1 -- 172.21.15.67:0/1000191213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 msgr2=0x563bfb66eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2145e00700 1 --2- 172.21.15.67:0/1000191213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb66eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2145e00700 1 -- 172.21.15.67:0/1000191213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb679fc0 msgr2=0x563bfb67c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2145e00700 1 --2- 172.21.15.67:0/1000191213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb679fc0 0x563bfb67c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2145e00700 1 -- 172.21.15.67:0/1000191213 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bfb4423b0 con 0x563bfb66f0c0 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2147200700 1 --2- 172.21.15.67:0/1000191213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb679fc0 0x563bfb67c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2145e00700 1 --2- 172.21.15.67:0/1000191213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb679a80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2134009c70 tx=0x7f2134009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11de54fc6fc21a78 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2147200700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f212c0a1290 0x7f212c0a3750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f21400060b0 tx=0x7f2140006000 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.879+0000 7f2133400700 1 -- 172.21.15.67:0/1000191213 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21340097c0 con 0x563bfb66f0c0 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2133400700 1 -- 172.21.15.67:0/1000191213 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2134004cf0 con 0x563bfb66f0c0 2024-03-30T02:40:49.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2133400700 1 -- 172.21.15.67:0/1000191213 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134010430 con 0x563bfb66f0c0 2024-03-30T02:40:49.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2160e89b80 1 -- 172.21.15.67:0/1000191213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 msgr2=0x563bfb679a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:49.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2160e89b80 1 --2- 172.21.15.67:0/1000191213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb679a80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2134009c70 tx=0x7f2134009f80 comp rx=0 tx=0).stop 2024-03-30T02:40:49.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2160e89b80 1 -- 172.21.15.67:0/1000191213 shutdown_connections 2024-03-30T02:40:49.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2160e89b80 1 --2- 172.21.15.67:0/1000191213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb679fc0 0x563bfb67c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2160e89b80 1 --2- 172.21.15.67:0/1000191213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb66eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2160e89b80 1 --2- 172.21.15.67:0/1000191213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb679a80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2160e89b80 1 -- 172.21.15.67:0/1000191213 >> 172.21.15.67:0/1000191213 conn(0x563bfb5ce6a0 msgr2=0x563bfb66ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2160e89b80 1 -- 172.21.15.67:0/1000191213 shutdown_connections 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.880+0000 7f2160e89b80 1 -- 172.21.15.67:0/1000191213 wait complete. 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.881+0000 7f2160e89b80 1 -- start start 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.881+0000 7f2160e89b80 1 Processor -- start 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 0x563bfb676cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb6772e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2160e89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb55cd80 con 0x563bfb66f0c0 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2160e89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb55ca20 con 0x563bfb679fc0 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2160e89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb4446a0 con 0x563bfb66e7a0 2024-03-30T02:40:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2160e89b80 10 client.0 _ll_register_callbacks cb 0x563bfb566470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2147200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb6772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2146800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2147200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb6772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40444/0 (socket says 172.21.15.67:40444) 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2147200700 1 -- 172.21.15.67:0/380172457 learned_addr learned my addr 172.21.15.67:0/380172457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2145e00700 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 0x563bfb676cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2146800700 1 -- 172.21.15.67:0/380172457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 msgr2=0x563bfb6772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2146800700 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb6772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2146800700 1 -- 172.21.15.67:0/380172457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 msgr2=0x563bfb676cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2146800700 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 0x563bfb676cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2146800700 1 -- 172.21.15.67:0/380172457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2134009920 con 0x563bfb66e7a0 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.882+0000 7f2147200700 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb6772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:49.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.883+0000 7f2146800700 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6799e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f21380ca5f0 tx=0x7f2138099990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f29cd84b6408f352 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.883+0000 7f2127e00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:40:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.883+0000 7f2127e00700 1 -- 172.21.15.67:0/380172457 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213809dde0 con 0x563bfb66e7a0 2024-03-30T02:40:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.883+0000 7f2160e89b80 1 -- 172.21.15.67:0/380172457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bfb581380 con 0x563bfb66e7a0 2024-03-30T02:40:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.883+0000 7f2127e00700 1 -- 172.21.15.67:0/380172457 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f21380a2430 con 0x563bfb66e7a0 2024-03-30T02:40:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.883+0000 7f2127e00700 1 -- 172.21.15.67:0/380172457 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21380a1fa0 con 0x563bfb66e7a0 2024-03-30T02:40:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.883+0000 7f2160e89b80 10 client.26542 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.883+0000 7f2160e89b80 1 -- 172.21.15.67:0/380172457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563bfb3eeb60 con 0x563bfb66e7a0 2024-03-30T02:40:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2127e00700 1 -- 172.21.15.67:0/380172457 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563bfb581380 con 0x563bfb66e7a0 2024-03-30T02:40:49.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2127e00700 1 -- 172.21.15.67:0/380172457 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2138098f10 con 0x563bfb66e7a0 2024-03-30T02:40:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2160e89b80 10 client.26542 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2160e89b80 10 client.26542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2160e89b80 20 client.26542 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2160e89b80 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563bfb689bc0 0x563bfb68bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2160e89b80 4 client.26542 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:40:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2160e89b80 1 -- 172.21.15.67:0/380172457 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bfb5b9700 con 0x563bfb689bc0 2024-03-30T02:40:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2145e00700 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563bfb689bc0 0x563bfb68bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2145e00700 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563bfb689bc0 0x563bfb68bfe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e96f349d62bf53 server_cookie=e07fbd2042b7c3aa in_seq=0 out_seq=0 2024-03-30T02:40:49.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.884+0000 7f2127e00700 10 client.26542 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:49.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2127e00700 1 -- 172.21.15.67:0/380172457 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563bfb5b9700 con 0x563bfb689bc0 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2127e00700 10 client.26542 handle_command_reply: tid=0 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 client.26542 shutdown 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 -- 172.21.15.67:0/380172457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 msgr2=0x563bfb6799e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6799e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f21380ca5f0 tx=0x7f2138099990 comp rx=0 tx=0).stop 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 -- 172.21.15.67:0/380172457 shutdown_connections 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb6772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6799e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563bfb689bc0 0x563bfb68bfe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 --2- 172.21.15.67:0/380172457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 0x563bfb676cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2127e00700 0 client.26542 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 -- 172.21.15.67:0/380172457 >> 172.21.15.67:0/380172457 conn(0x563bfb5ce6a0 msgr2=0x563bfb670890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 -- 172.21.15.67:0/380172457 shutdown_connections 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 1 -- 172.21.15.67:0/380172457 wait complete. 2024-03-30T02:40:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.887+0000 7f2160e89b80 20 client.26542 trim_cache size 0 max 16384 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.905+0000 7f2160e89b80 1 Processor -- start 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.905+0000 7f2160e89b80 1 -- start start 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.905+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66e7a0 0x563bfb66ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.905+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb680c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.905+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb679fc0 0x563bfb679510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.905+0000 7f2160e89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb6778e0 con 0x563bfb66e7a0 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.905+0000 7f2160e89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb675220 con 0x563bfb66f0c0 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.905+0000 7f2160e89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb67c3b0 con 0x563bfb679fc0 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2146800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66e7a0 0x563bfb66ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2146800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66e7a0 0x563bfb66ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37456/0 (socket says 172.21.15.67:37456) 2024-03-30T02:40:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2146800700 1 -- 172.21.15.67:0/226682137 learned_addr learned my addr 172.21.15.67:0/226682137 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2145e00700 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb680c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2147200700 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb679fc0 0x563bfb679510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2146800700 1 -- 172.21.15.67:0/226682137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb679fc0 msgr2=0x563bfb679510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2146800700 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb679fc0 0x563bfb679510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2146800700 1 -- 172.21.15.67:0/226682137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 msgr2=0x563bfb680c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2146800700 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb680c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2146800700 1 -- 172.21.15.67:0/226682137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bfb586ea0 con 0x563bfb66e7a0 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2147200700 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb679fc0 0x563bfb679510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2145e00700 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb680c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2146800700 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66e7a0 0x563bfb66ae10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f21380a1f00 tx=0x7f2138099380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9661fa3c9b61b0c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2133e00700 1 -- 172.21.15.67:0/226682137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21380a2430 con 0x563bfb66e7a0 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2133e00700 1 -- 172.21.15.67:0/226682137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f213809dde0 con 0x563bfb66e7a0 2024-03-30T02:40:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.906+0000 7f2133e00700 1 -- 172.21.15.67:0/226682137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21380a2200 con 0x563bfb66e7a0 2024-03-30T02:40:49.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 -- 172.21.15.67:0/226682137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66e7a0 msgr2=0x563bfb66ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66e7a0 0x563bfb66ae10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f21380a1f00 tx=0x7f2138099380 comp rx=0 tx=0).stop 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 -- 172.21.15.67:0/226682137 shutdown_connections 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66e7a0 0x563bfb66ae10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb679fc0 0x563bfb679510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 --2- 172.21.15.67:0/226682137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb66f0c0 0x563bfb680c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 -- 172.21.15.67:0/226682137 >> 172.21.15.67:0/226682137 conn(0x563bfb5ce6a0 msgr2=0x563bfb66a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 -- 172.21.15.67:0/226682137 shutdown_connections 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 -- 172.21.15.67:0/226682137 wait complete. 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.907+0000 7f2160e89b80 1 -- start start 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2160e89b80 1 Processor -- start 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 0x563bfb680d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2160e89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb67edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2160e89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb67f500 con 0x563bfb66f0c0 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2160e89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb67f670 con 0x563bfb679fc0 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2160e89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bfb67f830 con 0x563bfb66e7a0 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2160e89b80 10 client.0 _ll_register_callbacks cb 0x563bfb566470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2146800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2146800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6807d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37488/0 (socket says 172.21.15.67:37488) 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2147200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb67edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2146800700 1 -- 172.21.15.67:0/3658396129 learned_addr learned my addr 172.21.15.67:0/3658396129 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.908+0000 7f2145e00700 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 0x563bfb680d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2146800700 1 -- 172.21.15.67:0/3658396129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 msgr2=0x563bfb67edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2146800700 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb67edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2146800700 1 -- 172.21.15.67:0/3658396129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 msgr2=0x563bfb680d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2146800700 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 0x563bfb680d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2146800700 1 -- 172.21.15.67:0/3658396129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213401c040 con 0x563bfb66e7a0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2145e00700 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 0x563bfb680d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2146800700 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6807d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f213800c5b0 tx=0x7f21380cab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa137e5aa34d914e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2127e00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2127e00700 1 -- 172.21.15.67:0/3658396129 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213807b020 con 0x563bfb66e7a0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2127e00700 1 -- 172.21.15.67:0/3658396129 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f21380a2430 con 0x563bfb66e7a0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2160e89b80 1 -- 172.21.15.67:0/3658396129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2134004cf0 con 0x563bfb66e7a0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2127e00700 1 -- 172.21.15.67:0/3658396129 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2138099910 con 0x563bfb66e7a0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2160e89b80 10 client.26545 fetch_fsmap learned FSMap version 30 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.909+0000 7f2160e89b80 1 -- 172.21.15.67:0/3658396129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563bfb3eeb60 con 0x563bfb66e7a0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.910+0000 7f2127e00700 1 -- 172.21.15.67:0/3658396129 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2134004cf0 con 0x563bfb66e7a0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.910+0000 7f2127e00700 1 -- 172.21.15.67:0/3658396129 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f21380b2b60 con 0x563bfb66e7a0 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.910+0000 7f2160e89b80 10 client.26545 fetch_fsmap finished waiting for FSMap version 30 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.910+0000 7f2160e89b80 10 client.26545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:40:49.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.910+0000 7f2160e89b80 20 client.26545 populate_metadata read hostname 'smithi067' 2024-03-30T02:40:49.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.910+0000 7f2160e89b80 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563bfb6de8f0 0x563bfb6e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:40:49.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.910+0000 7f2160e89b80 4 client.26545 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:40:49.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.910+0000 7f2160e89b80 1 -- 172.21.15.67:0/3658396129 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563bfb40fcf0 con 0x563bfb6de8f0 2024-03-30T02:40:49.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.910+0000 7f2145e00700 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563bfb6de8f0 0x563bfb6e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:40:49.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.911+0000 7f2145e00700 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563bfb6de8f0 0x563bfb6e0d10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8917f9c4e5ec340 server_cookie=8eccfa6b2b6781f3 in_seq=0 out_seq=0 2024-03-30T02:40:49.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.911+0000 7f2127e00700 10 client.26545 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.913+0000 7f2127e00700 1 -- 172.21.15.67:0/3658396129 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+269 (crc 0 0 0) 0x563bfb40fcf0 con 0x563bfb6de8f0 2024-03-30T02:40:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.913+0000 7f2127e00700 10 client.26545 handle_command_reply: tid=0 2024-03-30T02:40:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.913+0000 7f2160e89b80 1 client.26545 shutdown 2024-03-30T02:40:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.913+0000 7f2160e89b80 1 -- 172.21.15.67:0/3658396129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 msgr2=0x563bfb6807d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:49.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.913+0000 7f2160e89b80 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6807d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f213800c5b0 tx=0x7f21380cab80 comp rx=0 tx=0).stop 2024-03-30T02:40:49.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2160e89b80 1 -- 172.21.15.67:0/3658396129 shutdown_connections 2024-03-30T02:40:49.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2160e89b80 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bfb679fc0 0x563bfb67edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2160e89b80 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563bfb66e7a0 0x563bfb6807d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2160e89b80 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563bfb6de8f0 0x563bfb6e0d10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2160e89b80 1 --2- 172.21.15.67:0/3658396129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563bfb66f0c0 0x563bfb680d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2127e00700 0 client.26545 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:40:49.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2160e89b80 1 -- 172.21.15.67:0/3658396129 >> 172.21.15.67:0/3658396129 conn(0x563bfb5ce6a0 msgr2=0x563bfb676460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:49.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2160e89b80 1 -- 172.21.15.67:0/3658396129 shutdown_connections 2024-03-30T02:40:49.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2160e89b80 1 -- 172.21.15.67:0/3658396129 wait complete. 2024-03-30T02:40:49.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f2160e89b80 20 client.26545 trim_cache size 0 max 16384 2024-03-30T02:40:49.918 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:40:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (10738 inodes in the stack)", 2024-03-30T02:40:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:40:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "5f5c22d4-f0cb-42f3-b02c-98f21450b6a1": { 2024-03-30T02:40:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:40:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "5f5c22d4-f0cb-42f3-b02c-98f21450b6a1", 2024-03-30T02:40:49.918 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:40:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:40:49.919 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:40:49.919 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:40:49.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f214ca00700 1 -- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f212c0a1290 msgr2=0x7f212c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:49.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f214ca00700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f212c0a1290 0x7f212c0a3750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f21400060b0 tx=0x7f2140006000 comp rx=0 tx=0).stop 2024-03-30T02:40:49.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f214ca00700 1 -- 172.21.15.67:0/3156507072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148100b50 msgr2=0x7f2148199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:40:49.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.914+0000 7f214ca00700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148100b50 0x7f2148199330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f213800e9a0 tx=0x7f213800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.915+0000 7f214ca00700 1 -- 172.21.15.67:0/3156507072 shutdown_connections 2024-03-30T02:40:49.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.915+0000 7f214ca00700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f212c0a1290 0x7f212c0a3750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.915+0000 7f214ca00700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2148106c10 0x7f214819dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.915+0000 7f214ca00700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2148100b50 0x7f2148199330 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.915+0000 7f214ca00700 1 --2- 172.21.15.67:0/3156507072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2148101510 0x7f2148199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:40:49.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.915+0000 7f214ca00700 1 -- 172.21.15.67:0/3156507072 >> 172.21.15.67:0/3156507072 conn(0x7f21480fc620 msgr2=0x7f21480fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:40:49.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.915+0000 7f214ca00700 1 -- 172.21.15.67:0/3156507072 shutdown_connections 2024-03-30T02:40:49.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:40:49.915+0000 7f214ca00700 1 -- 172.21.15.67:0/3156507072 wait complete. 2024-03-30T02:40:49.932 INFO:tasks.cephfs.filesystem:scrub status for tag:5f5c22d4-f0cb-42f3-b02c-98f21450b6a1 - {'path': '/', 'tag': '5f5c22d4-f0cb-42f3-b02c-98f21450b6a1', 'options': 'recursive,force'} 2024-03-30T02:40:50.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:50 smithi067 ceph-mon[37808]: pgmap v116: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 1.49k op/s 2024-03-30T02:40:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:50 smithi170 ceph-mon[78798]: pgmap v116: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 1.49k op/s 2024-03-30T02:40:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:50 smithi082 ceph-mon[79176]: pgmap v116: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 1.49k op/s 2024-03-30T02:40:51.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:51.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:51.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:51.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: pgmap v117: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.6 MiB/s wr, 2.19k op/s 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:52.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:52.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: pgmap v117: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.6 MiB/s wr, 2.19k op/s 2024-03-30T02:40:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:52.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: pgmap v117: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.6 MiB/s wr, 2.19k op/s 2024-03-30T02:40:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: pgmap v118: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.5 MiB/s wr, 1.45k op/s 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: pgmap v118: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.5 MiB/s wr, 1.45k op/s 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:40:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: pgmap v118: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.5 MiB/s wr, 1.45k op/s 2024-03-30T02:40:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:40:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:40:56.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:56 smithi170 ceph-mon[78798]: pgmap v119: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 2.71k op/s 2024-03-30T02:40:56.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:56 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:40:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:56 smithi082 ceph-mon[79176]: pgmap v119: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 2.71k op/s 2024-03-30T02:40:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:56 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:40:56.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:56 smithi067 ceph-mon[37808]: pgmap v119: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 12 MiB/s wr, 2.71k op/s 2024-03-30T02:40:56.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:56 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:40:58.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:40:58 smithi170 ceph-mon[78798]: pgmap v120: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 1.99k op/s 2024-03-30T02:40:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:40:58 smithi082 ceph-mon[79176]: pgmap v120: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 1.99k op/s 2024-03-30T02:40:58.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:40:58 smithi067 ceph-mon[37808]: pgmap v120: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 1.99k op/s 2024-03-30T02:41:00.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:00 smithi170 ceph-mon[78798]: pgmap v121: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 1.99k op/s 2024-03-30T02:41:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:00 smithi082 ceph-mon[79176]: pgmap v121: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 1.99k op/s 2024-03-30T02:41:00.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:00 smithi067 ceph-mon[37808]: pgmap v121: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 11 MiB/s wr, 1.99k op/s 2024-03-30T02:41:02.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:02 smithi067 ceph-mon[37808]: pgmap v122: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 2.01k op/s 2024-03-30T02:41:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:02 smithi170 ceph-mon[78798]: pgmap v122: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 2.01k op/s 2024-03-30T02:41:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:02 smithi082 ceph-mon[79176]: pgmap v122: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 12 MiB/s wr, 2.01k op/s 2024-03-30T02:41:04.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:04 smithi067 ceph-mon[37808]: pgmap v123: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 1.28k op/s 2024-03-30T02:41:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:04 smithi170 ceph-mon[78798]: pgmap v123: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 1.28k op/s 2024-03-30T02:41:04.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:04 smithi082 ceph-mon[79176]: pgmap v123: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.7 MiB/s wr, 1.28k op/s 2024-03-30T02:41:06.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:06 smithi067 ceph-mon[37808]: pgmap v124: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.7 MiB/s wr, 1.40k op/s 2024-03-30T02:41:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:06 smithi170 ceph-mon[78798]: pgmap v124: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.7 MiB/s wr, 1.40k op/s 2024-03-30T02:41:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:06 smithi082 ceph-mon[79176]: pgmap v124: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.7 MiB/s wr, 1.40k op/s 2024-03-30T02:41:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:07 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:41:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:07 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:41:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:07 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:41:07.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:07 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:41:07.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:07 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:41:07.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:07 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:41:07.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:07 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:41:07.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:07 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:41:07.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:07 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:41:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:08 smithi067 ceph-mon[37808]: pgmap v125: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 133 op/s 2024-03-30T02:41:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:08 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:41:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:08 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:41:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:08 smithi170 ceph-mon[78798]: pgmap v125: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 133 op/s 2024-03-30T02:41:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:08 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:41:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:08 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:41:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:08 smithi082 ceph-mon[79176]: pgmap v125: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 133 op/s 2024-03-30T02:41:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:08 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:41:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:08 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:41:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:10 smithi067 ceph-mon[37808]: pgmap v126: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 154 op/s 2024-03-30T02:41:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:10 smithi170 ceph-mon[78798]: pgmap v126: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 154 op/s 2024-03-30T02:41:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:10 smithi082 ceph-mon[79176]: pgmap v126: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 154 op/s 2024-03-30T02:41:12.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:12 smithi067 ceph-mon[37808]: pgmap v127: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.0 MiB/s wr, 633 op/s 2024-03-30T02:41:12.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:12 smithi170 ceph-mon[78798]: pgmap v127: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.0 MiB/s wr, 633 op/s 2024-03-30T02:41:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:12 smithi082 ceph-mon[79176]: pgmap v127: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 6.0 MiB/s wr, 633 op/s 2024-03-30T02:41:13.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:13 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:41:13.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:13 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:41:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:13 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:41:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:14 smithi067 ceph-mon[37808]: pgmap v128: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 614 op/s 2024-03-30T02:41:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:14 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:41:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:14 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:41:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:14 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:41:14.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:14 smithi170 ceph-mon[78798]: pgmap v128: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 614 op/s 2024-03-30T02:41:14.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:14 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:41:14.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:14 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:41:14.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:14 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:41:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:14 smithi082 ceph-mon[79176]: pgmap v128: 129 pgs: 129 active+clean; 1.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 614 op/s 2024-03-30T02:41:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:14 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:41:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:14 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:41:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:14 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:41:16.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:16 smithi067 ceph-mon[37808]: pgmap v129: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 1.20k op/s 2024-03-30T02:41:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:16 smithi170 ceph-mon[78798]: pgmap v129: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 1.20k op/s 2024-03-30T02:41:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:16 smithi082 ceph-mon[79176]: pgmap v129: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 12 MiB/s wr, 1.20k op/s 2024-03-30T02:41:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:18 smithi067 ceph-mon[37808]: pgmap v130: 129 pgs: 129 active+clean; 1.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-03-30T02:41:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:18 smithi170 ceph-mon[78798]: pgmap v130: 129 pgs: 129 active+clean; 1.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-03-30T02:41:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:18 smithi082 ceph-mon[79176]: pgmap v130: 129 pgs: 129 active+clean; 1.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-03-30T02:41:19.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:41:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.165+0000 7f4c21e00700 1 -- 172.21.15.67:0/190236455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c067e30 msgr2=0x7f4c1c0682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.165+0000 7f4c21e00700 1 --2- 172.21.15.67:0/190236455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c067e30 0x7f4c1c0682b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4c080099c0 tx=0x7f4c08009cd0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.165+0000 7f4c21e00700 1 -- 172.21.15.67:0/190236455 shutdown_connections 2024-03-30T02:41:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.165+0000 7f4c21e00700 1 --2- 172.21.15.67:0/190236455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c0687f0 0x7f4c1c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.165+0000 7f4c21e00700 1 --2- 172.21.15.67:0/190236455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c105ce0 0x7f4c1c1060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.165+0000 7f4c21e00700 1 --2- 172.21.15.67:0/190236455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c067e30 0x7f4c1c0682b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.165+0000 7f4c21e00700 1 -- 172.21.15.67:0/190236455 >> 172.21.15.67:0/190236455 conn(0x7f4c1c076d40 msgr2=0x7f4c1c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.165+0000 7f4c21e00700 1 -- 172.21.15.67:0/190236455 shutdown_connections 2024-03-30T02:41:20.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.165+0000 7f4c21e00700 1 -- 172.21.15.67:0/190236455 wait complete. 2024-03-30T02:41:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.166+0000 7f4c21e00700 1 Processor -- start 2024-03-30T02:41:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.166+0000 7f4c21e00700 1 -- start start 2024-03-30T02:41:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c21e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c067e30 0x7f4c1c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c21e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c0687f0 0x7f4c1c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c21e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c105ce0 0x7f4c1c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c21e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c19e360 con 0x7f4c1c067e30 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c21e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c19e4d0 con 0x7f4c1c105ce0 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c21e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c19e690 con 0x7f4c1c0687f0 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c1aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c067e30 0x7f4c1c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c1a000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c0687f0 0x7f4c1c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c1aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c067e30 0x7f4c1c199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52832/0 (socket says 172.21.15.67:52832) 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c1aa00700 1 -- 172.21.15.67:0/633596657 learned_addr learned my addr 172.21.15.67:0/633596657 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.167+0000 7f4c1b400700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c105ce0 0x7f4c1c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c1a000700 1 -- 172.21.15.67:0/633596657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c105ce0 msgr2=0x7f4c1c19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c1a000700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c105ce0 0x7f4c1c19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c1a000700 1 -- 172.21.15.67:0/633596657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c067e30 msgr2=0x7f4c1c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c1a000700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c067e30 0x7f4c1c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c1a000700 1 -- 172.21.15.67:0/633596657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080095f0 con 0x7f4c1c0687f0 2024-03-30T02:41:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c1a000700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c0687f0 0x7f4c1c199980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4c080098e0 tx=0x7f4c0800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c20a00700 1 -- 172.21.15.67:0/633596657 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0801c070 con 0x7f4c1c0687f0 2024-03-30T02:41:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c20a00700 1 -- 172.21.15.67:0/633596657 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c0800fd00 con 0x7f4c1c0687f0 2024-03-30T02:41:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c21e00700 1 -- 172.21.15.67:0/633596657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c19e910 con 0x7f4c1c0687f0 2024-03-30T02:41:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c20a00700 1 -- 172.21.15.67:0/633596657 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08022c60 con 0x7f4c1c0687f0 2024-03-30T02:41:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.168+0000 7f4c21e00700 1 -- 172.21.15.67:0/633596657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c1a61b0 con 0x7f4c1c0687f0 2024-03-30T02:41:20.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.171+0000 7f4c20a00700 1 -- 172.21.15.67:0/633596657 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4c08020070 con 0x7f4c1c0687f0 2024-03-30T02:41:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.173+0000 7f4c20a00700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c000a1310 0x7f4c000a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.174+0000 7f4c20a00700 1 -- 172.21.15.67:0/633596657 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f4c080cdbc0 con 0x7f4c1c0687f0 2024-03-30T02:41:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.174+0000 7f4c1aa00700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c000a1310 0x7f4c000a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.174+0000 7f4c358a2b80 1 Processor -- start 2024-03-30T02:41:20.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.174+0000 7f4c358a2b80 1 -- start start 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.174+0000 7f4c1aa00700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c000a1310 0x7f4c000a37d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009d20 tx=0x7f4c0c009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 0x561a835739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 0x561a8357e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a835811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a834cc990 con 0x561a835735d0 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a834ca350 con 0x561a8357edf0 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a834c5c40 con 0x561a83573ef0 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 0x561a835739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a835811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 0x561a835739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52838/0 (socket says 172.21.15.67:52838) 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1aa00700 1 -- 172.21.15.67:0/3758056848 learned_addr learned my addr 172.21.15.67:0/3758056848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1a000700 1 --2- 172.21.15.67:0/3758056848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 0x561a8357e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1b400700 1 -- 172.21.15.67:0/3758056848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 msgr2=0x561a8357e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1b400700 1 --2- 172.21.15.67:0/3758056848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 0x561a8357e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1b400700 1 -- 172.21.15.67:0/3758056848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 msgr2=0x561a835739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1b400700 1 --2- 172.21.15.67:0/3758056848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 0x561a835739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1b400700 1 -- 172.21.15.67:0/3758056848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a833473b0 con 0x561a8357edf0 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.175+0000 7f4c1a000700 1 --2- 172.21.15.67:0/3758056848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 0x561a8357e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.176+0000 7f4c1b400700 1 --2- 172.21.15.67:0/3758056848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a835811e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4c14011580 tx=0x7f4c14009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644376b801422847 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.176+0000 7f4c07e00700 1 -- 172.21.15.67:0/3758056848 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400e040 con 0x561a8357edf0 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.176+0000 7f4c07e00700 1 -- 172.21.15.67:0/3758056848 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c1400f040 con 0x561a8357edf0 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.176+0000 7f4c07e00700 1 -- 172.21.15.67:0/3758056848 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c140044f0 con 0x561a8357edf0 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.176+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3758056848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 msgr2=0x561a835811e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.176+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3758056848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a835811e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4c14011580 tx=0x7f4c14009920 comp rx=0 tx=0).stop 2024-03-30T02:41:20.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.177+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3758056848 shutdown_connections 2024-03-30T02:41:20.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.177+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3758056848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 0x561a835739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.177+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3758056848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a835811e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.177+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3758056848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 0x561a8357e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.177+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3758056848 >> 172.21.15.67:0/3758056848 conn(0x561a834d36a0 msgr2=0x561a8356fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:20.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.177+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3758056848 shutdown_connections 2024-03-30T02:41:20.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.177+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3758056848 wait complete. 2024-03-30T02:41:20.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.177+0000 7f4c358a2b80 1 -- start start 2024-03-30T02:41:20.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.177+0000 7f4c358a2b80 1 Processor -- start 2024-03-30T02:41:20.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 0x561a8357e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a83573ef0 0x561a8357bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a8357edf0 0x561a8357c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a83461d80 con 0x561a835735d0 2024-03-30T02:41:20.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a83461a20 con 0x561a83573ef0 2024-03-30T02:41:20.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a833496a0 con 0x561a8357edf0 2024-03-30T02:41:20.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c358a2b80 10 client.0 _ll_register_callbacks cb 0x561a8346b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:20.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c1a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a83573ef0 0x561a8357bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c1a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a83573ef0 0x561a8357bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60676/0 (socket says 172.21.15.67:60676) 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c1a000700 1 -- 172.21.15.67:0/3087801722 learned_addr learned my addr 172.21.15.67:0/3087801722 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c1aa00700 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 0x561a8357e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.178+0000 7f4c1b400700 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a8357edf0 0x561a8357c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c1a000700 1 -- 172.21.15.67:0/3087801722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a8357edf0 msgr2=0x561a8357c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c1a000700 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a8357edf0 0x561a8357c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c1a000700 1 -- 172.21.15.67:0/3087801722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 msgr2=0x561a8357e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c1a000700 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 0x561a8357e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c1a000700 1 -- 172.21.15.67:0/3087801722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c14011230 con 0x561a83573ef0 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c1a000700 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a83573ef0 0x561a8357bb40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800b580 tx=0x7f4c0809a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32f5ac4d72302e79 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c04c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c04c00700 1 -- 172.21.15.67:0/3087801722 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0809ae10 con 0x561a83573ef0 2024-03-30T02:41:20.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c04c00700 1 -- 172.21.15.67:0/3087801722 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f4c0809a9b0 con 0x561a83573ef0 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3087801722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a83486380 con 0x561a83573ef0 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c04c00700 1 -- 172.21.15.67:0/3087801722 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c0800fa50 con 0x561a83573ef0 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c358a2b80 10 client.26603 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.179+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3087801722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561a832f3b60 con 0x561a83573ef0 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.180+0000 7f4c04c00700 1 -- 172.21.15.67:0/3087801722 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a83486380 con 0x561a83573ef0 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.180+0000 7f4c04c00700 1 -- 172.21.15.67:0/3087801722 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f4c080a2910 con 0x561a83573ef0 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.180+0000 7f4c358a2b80 10 client.26603 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.180+0000 7f4c358a2b80 10 client.26603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.180+0000 7f4c358a2b80 20 client.26603 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.180+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561a8358e700 0x561a83590b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.180+0000 7f4c358a2b80 4 client.26603 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.180+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3087801722 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a834be700 con 0x561a8358e700 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.180+0000 7f4c1aa00700 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561a8358e700 0x561a83590b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.181+0000 7f4c1aa00700 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561a8358e700 0x561a83590b20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=497d987a0355d33f server_cookie=e4e6ab5ab4609497 in_seq=0 out_seq=0 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.181+0000 7f4c04c00700 10 client.26603 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c04c00700 1 -- 172.21.15.67:0/3087801722 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561a834be700 con 0x561a8358e700 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c04c00700 10 client.26603 handle_command_reply: tid=0 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 client.26603 shutdown 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3087801722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a83573ef0 msgr2=0x561a8357bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a83573ef0 0x561a8357bb40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800b580 tx=0x7f4c0809a380 comp rx=0 tx=0).stop 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3087801722 shutdown_connections 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561a8358e700 0x561a83590b20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a835735d0 0x561a8357e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a8357edf0 0x561a8357c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/3087801722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a83573ef0 0x561a8357bb40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3087801722 >> 172.21.15.67:0/3087801722 conn(0x561a834d36a0 msgr2=0x561a835756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c04c00700 0 client.26603 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3087801722 shutdown_connections 2024-03-30T02:41:20.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 1 -- 172.21.15.67:0/3087801722 wait complete. 2024-03-30T02:41:20.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.182+0000 7f4c358a2b80 20 client.26603 trim_cache size 0 max 16384 2024-03-30T02:41:20.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.201+0000 7f4c358a2b80 1 Processor -- start 2024-03-30T02:41:20.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.201+0000 7f4c358a2b80 1 -- start start 2024-03-30T02:41:20.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a835735d0 0x561a83585a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a83573ef0 0x561a8357b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a8357b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a8357d0a0 con 0x561a83573ef0 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a83571370 con 0x561a8357edf0 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a83574850 con 0x561a835735d0 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a83573ef0 0x561a8357b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a8357b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1aa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a835735d0 0x561a83585a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a83573ef0 0x561a8357b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52854/0 (socket says 172.21.15.67:52854) 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a8357b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60682/0 (socket says 172.21.15.67:60682) 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1a000700 1 -- 172.21.15.67:0/1582590583 learned_addr learned my addr 172.21.15.67:0/1582590583 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1a000700 1 -- 172.21.15.67:0/1582590583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a835735d0 msgr2=0x561a83585a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:20.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1a000700 1 --2- 172.21.15.67:0/1582590583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a835735d0 0x561a83585a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1a000700 1 -- 172.21.15.67:0/1582590583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 msgr2=0x561a8357b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:20.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1a000700 1 --2- 172.21.15.67:0/1582590583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a8357b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.202+0000 7f4c1a000700 1 -- 172.21.15.67:0/1582590583 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a8348bea0 con 0x561a83573ef0 2024-03-30T02:41:20.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.203+0000 7f4c1a000700 1 --2- 172.21.15.67:0/1582590583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a83573ef0 0x561a8357b3d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800c010 tx=0x7f4c080047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa8ee9c38f2950f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:20.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.203+0000 7f4c11200700 1 -- 172.21.15.67:0/1582590583 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0800fa50 con 0x561a83573ef0 2024-03-30T02:41:20.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.203+0000 7f4c11200700 1 -- 172.21.15.67:0/1582590583 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c080b3240 con 0x561a83573ef0 2024-03-30T02:41:20.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.203+0000 7f4c11200700 1 -- 172.21.15.67:0/1582590583 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c080a2da0 con 0x561a83573ef0 2024-03-30T02:41:20.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.203+0000 7f4c358a2b80 1 -- 172.21.15.67:0/1582590583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a83573ef0 msgr2=0x561a8357b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.203+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/1582590583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a83573ef0 0x561a8357b3d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800c010 tx=0x7f4c080047a0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.204+0000 7f4c358a2b80 1 -- 172.21.15.67:0/1582590583 shutdown_connections 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.204+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/1582590583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a8357edf0 0x561a8357b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.204+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/1582590583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a835735d0 0x561a83585a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.204+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/1582590583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a83573ef0 0x561a8357b3d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.204+0000 7f4c358a2b80 1 -- 172.21.15.67:0/1582590583 >> 172.21.15.67:0/1582590583 conn(0x561a834d36a0 msgr2=0x561a835685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.204+0000 7f4c358a2b80 1 -- 172.21.15.67:0/1582590583 shutdown_connections 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.204+0000 7f4c358a2b80 1 -- 172.21.15.67:0/1582590583 wait complete. 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.204+0000 7f4c358a2b80 1 -- start start 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.204+0000 7f4c358a2b80 1 Processor -- start 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a835735d0 0x561a835857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 0x561a83583c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c358a2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a8357edf0 0x561a83584170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a835846b0 con 0x561a8357edf0 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a83584820 con 0x561a835735d0 2024-03-30T02:41:20.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c358a2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a835849e0 con 0x561a83573ef0 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c358a2b80 10 client.0 _ll_register_callbacks cb 0x561a8346b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a835735d0 0x561a835857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a835735d0 0x561a835857c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60694/0 (socket says 172.21.15.67:60694) 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1aa00700 1 -- 172.21.15.67:0/2273931640 learned_addr learned my addr 172.21.15.67:0/2273931640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1b400700 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a8357edf0 0x561a83584170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1a000700 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 0x561a83583c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1aa00700 1 -- 172.21.15.67:0/2273931640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 msgr2=0x561a83583c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1aa00700 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 0x561a83583c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1aa00700 1 -- 172.21.15.67:0/2273931640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a8357edf0 msgr2=0x561a83584170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1aa00700 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a8357edf0 0x561a83584170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1aa00700 1 -- 172.21.15.67:0/2273931640 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a832f3b60 con 0x561a835735d0 2024-03-30T02:41:20.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.205+0000 7f4c1b400700 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a8357edf0 0x561a83584170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.206+0000 7f4c1aa00700 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a835735d0 0x561a835857c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x561a83591390 tx=0x7f4c0c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627dae76c89cbf06 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.206+0000 7f4c04c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.206+0000 7f4c04c00700 1 -- 172.21.15.67:0/2273931640 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c022040 con 0x561a835735d0 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.206+0000 7f4c358a2b80 1 -- 172.21.15.67:0/2273931640 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c1400f040 con 0x561a835735d0 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.206+0000 7f4c04c00700 1 -- 172.21.15.67:0/2273931640 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f4c0c031070 con 0x561a835735d0 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.206+0000 7f4c04c00700 1 -- 172.21.15.67:0/2273931640 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c0c00ee90 con 0x561a835735d0 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.206+0000 7f4c358a2b80 10 client.26609 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.206+0000 7f4c358a2b80 1 -- 172.21.15.67:0/2273931640 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561a8334bcd0 con 0x561a835735d0 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.206+0000 7f4c04c00700 1 -- 172.21.15.67:0/2273931640 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c1400f040 con 0x561a835735d0 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c04c00700 1 -- 172.21.15.67:0/2273931640 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f4c0c01d410 con 0x561a835735d0 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c358a2b80 10 client.26609 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c358a2b80 10 client.26609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c358a2b80 20 client.26609 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561a835e3a30 0x561a835e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c358a2b80 4 client.26609 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c358a2b80 1 -- 172.21.15.67:0/2273931640 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561a83314cf0 con 0x561a835e3a30 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c1a000700 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561a835e3a30 0x561a835e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c1a000700 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561a835e3a30 0x561a835e5e50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410fe3046fc3f47d server_cookie=b1413bc71e74df88 in_seq=0 out_seq=0 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.207+0000 7f4c04c00700 10 client.26609 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c04c00700 1 -- 172.21.15.67:0/2273931640 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561a83314cf0 con 0x561a835e3a30 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c04c00700 10 client.26609 handle_command_reply: tid=0 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 client.26609 shutdown 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 -- 172.21.15.67:0/2273931640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a835735d0 msgr2=0x561a835857c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a835735d0 0x561a835857c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x561a83591390 tx=0x7f4c0c0047c0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 -- 172.21.15.67:0/2273931640 shutdown_connections 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561a835e3a30 0x561a835e5e50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561a8357edf0 0x561a83584170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c04c00700 0 client.26609 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561a835735d0 0x561a835857c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 --2- 172.21.15.67:0/2273931640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a83573ef0 0x561a83583c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 -- 172.21.15.67:0/2273931640 >> 172.21.15.67:0/2273931640 conn(0x561a834d36a0 msgr2=0x561a8358a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 -- 172.21.15.67:0/2273931640 shutdown_connections 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.208+0000 7f4c358a2b80 1 -- 172.21.15.67:0/2273931640 wait complete. 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c358a2b80 20 client.26609 trim_cache size 0 max 16384 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:41:20.215 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:41:20.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 -- 172.21.15.67:0/633596657 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c000a1310 msgr2=0x7f4c000a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c000a1310 0x7f4c000a37d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009d20 tx=0x7f4c0c009260 comp rx=0 tx=0).stop 2024-03-30T02:41:20.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 -- 172.21.15.67:0/633596657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c0687f0 msgr2=0x7f4c1c199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c0687f0 0x7f4c1c199980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4c080098e0 tx=0x7f4c0800f690 comp rx=0 tx=0).stop 2024-03-30T02:41:20.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 -- 172.21.15.67:0/633596657 shutdown_connections 2024-03-30T02:41:20.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c000a1310 0x7f4c000a37d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c1c067e30 0x7f4c1c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c1c105ce0 0x7f4c1c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 --2- 172.21.15.67:0/633596657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c1c0687f0 0x7f4c1c199980 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 -- 172.21.15.67:0/633596657 >> 172.21.15.67:0/633596657 conn(0x7f4c1c076d40 msgr2=0x7f4c1c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:20.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 -- 172.21.15.67:0/633596657 shutdown_connections 2024-03-30T02:41:20.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.209+0000 7f4c21e00700 1 -- 172.21.15.67:0/633596657 wait complete. 2024-03-30T02:41:20.228 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:41:20.229 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:41:20.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.461+0000 7f63eba00700 1 -- 172.21.15.67:0/2959096996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e4101810 msgr2=0x7f63e4105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.461+0000 7f63eba00700 1 --2- 172.21.15.67:0/2959096996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e4101810 0x7f63e4105850 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009990 tx=0x7f63d0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.461+0000 7f63eba00700 1 -- 172.21.15.67:0/2959096996 shutdown_connections 2024-03-30T02:41:20.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.461+0000 7f63eba00700 1 --2- 172.21.15.67:0/2959096996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e4105f80 0x7f63e410db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.461+0000 7f63eba00700 1 --2- 172.21.15.67:0/2959096996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63e4100e60 0x7f63e4101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.461+0000 7f63eba00700 1 --2- 172.21.15.67:0/2959096996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e4101810 0x7f63e4105850 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.461+0000 7f63eba00700 1 -- 172.21.15.67:0/2959096996 >> 172.21.15.67:0/2959096996 conn(0x7f63e40fc700 msgr2=0x7f63e40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:20.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.461+0000 7f63eba00700 1 -- 172.21.15.67:0/2959096996 shutdown_connections 2024-03-30T02:41:20.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.461+0000 7f63eba00700 1 -- 172.21.15.67:0/2959096996 wait complete. 2024-03-30T02:41:20.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.462+0000 7f63eba00700 1 Processor -- start 2024-03-30T02:41:20.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.462+0000 7f63eba00700 1 -- start start 2024-03-30T02:41:20.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63eba00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e4100e60 0x7f63e41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63eba00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e4105f80 0x7f63e4199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63eba00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63e419dcb0 0x7f63e419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63eba00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e419e730 con 0x7f63e4100e60 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63eba00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e419e8a0 con 0x7f63e419dcb0 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63eba00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e419ea60 con 0x7f63e4105f80 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e9a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63e419dcb0 0x7f63e419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e9a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63e419dcb0 0x7f63e419e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60702/0 (socket says 172.21.15.67:60702) 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e9a00700 1 -- 172.21.15.67:0/3725572995 learned_addr learned my addr 172.21.15.67:0/3725572995 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e9000700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e4100e60 0x7f63e41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e3e00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e4105f80 0x7f63e4199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e9a00700 1 -- 172.21.15.67:0/3725572995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e4105f80 msgr2=0x7f63e4199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e9a00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e4105f80 0x7f63e4199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e9a00700 1 -- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e4100e60 msgr2=0x7f63e41993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e9a00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e4100e60 0x7f63e41993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e9a00700 1 -- 172.21.15.67:0/3725572995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d00095f0 con 0x7f63e419dcb0 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.463+0000 7f63e3e00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e4105f80 0x7f63e4199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.464+0000 7f63e9a00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63e419dcb0 0x7f63e419e130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00ef30 tx=0x7f63dc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:20.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.464+0000 7f63e1600700 1 -- 172.21.15.67:0/3725572995 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc018070 con 0x7f63e419dcb0 2024-03-30T02:41:20.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.464+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e41a5f60 con 0x7f63e419dcb0 2024-03-30T02:41:20.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.464+0000 7f63e1600700 1 -- 172.21.15.67:0/3725572995 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63dc00f040 con 0x7f63e419dcb0 2024-03-30T02:41:20.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.464+0000 7f63e1600700 1 -- 172.21.15.67:0/3725572995 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00cdb0 con 0x7f63e419dcb0 2024-03-30T02:41:20.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.464+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e41a6440 con 0x7f63e419dcb0 2024-03-30T02:41:20.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.465+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e4102330 con 0x7f63e419dcb0 2024-03-30T02:41:20.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.467+0000 7f63e1600700 1 -- 172.21.15.67:0/3725572995 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f63dc003a00 con 0x7f63e419dcb0 2024-03-30T02:41:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.469+0000 7f63e1600700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63cc0a1290 0x7f63cc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.469+0000 7f63e1600700 1 -- 172.21.15.67:0/3725572995 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f63dc0ccd40 con 0x7f63e419dcb0 2024-03-30T02:41:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.469+0000 7f63e9000700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63cc0a1290 0x7f63cc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.469+0000 7f63e9000700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63cc0a1290 0x7f63cc0a3750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009c60 tx=0x7f63d8009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:20.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.473+0000 7f63e1600700 1 -- 172.21.15.67:0/3725572995 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f63dc095e70 con 0x7f63e419dcb0 2024-03-30T02:41:20.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:20 smithi067 ceph-mon[37808]: pgmap v131: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.18k op/s 2024-03-30T02:41:20.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:20 smithi170 ceph-mon[78798]: pgmap v131: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.18k op/s 2024-03-30T02:41:20.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:20 smithi082 ceph-mon[79176]: pgmap v131: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.18k op/s 2024-03-30T02:41:20.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.755+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63e4067a10 con 0x7f63e419dcb0 2024-03-30T02:41:20.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.756+0000 7f63e1600700 1 -- 172.21.15.67:0/3725572995 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10923 (secure 0 0 0) 0x7f63dc099490 con 0x7f63e419dcb0 2024-03-30T02:41:20.757 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:41:20.757 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:40:55.410287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63cc0a1290 msgr2=0x7f63cc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63cc0a1290 0x7f63cc0a3750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009c60 tx=0x7f63d8009260 comp rx=0 tx=0).stop 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63e419dcb0 msgr2=0x7f63e419e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63e419dcb0 0x7f63e419e130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00ef30 tx=0x7f63dc00c3a0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 shutdown_connections 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63cc0a1290 0x7f63cc0a3750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f63e4100e60 0x7f63e41993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63e419dcb0 0x7f63e419e130 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 --2- 172.21.15.67:0/3725572995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63e4105f80 0x7f63e4199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 >> 172.21.15.67:0/3725572995 conn(0x7f63e40fc700 msgr2=0x7f63e40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 shutdown_connections 2024-03-30T02:41:20.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.759+0000 7f63eba00700 1 -- 172.21.15.67:0/3725572995 wait complete. 2024-03-30T02:41:20.761 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 31 2024-03-30T02:41:20.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:41:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.993+0000 7fa933600700 1 -- 172.21.15.67:0/1029195087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c100f60 msgr2=0x7fa92c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.993+0000 7fa933600700 1 --2- 172.21.15.67:0/1029195087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c100f60 0x7fa92c101340 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009990 tx=0x7fa91c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.993+0000 7fa933600700 1 -- 172.21.15.67:0/1029195087 shutdown_connections 2024-03-30T02:41:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.993+0000 7fa933600700 1 --2- 172.21.15.67:0/1029195087 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa92c106080 0x7fa92c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.993+0000 7fa933600700 1 --2- 172.21.15.67:0/1029195087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c100f60 0x7fa92c101340 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.993+0000 7fa933600700 1 --2- 172.21.15.67:0/1029195087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c101910 0x7fa92c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.993+0000 7fa933600700 1 -- 172.21.15.67:0/1029195087 >> 172.21.15.67:0/1029195087 conn(0x7fa92c0fc7c0 msgr2=0x7fa92c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.993+0000 7fa933600700 1 -- 172.21.15.67:0/1029195087 shutdown_connections 2024-03-30T02:41:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.993+0000 7fa933600700 1 -- 172.21.15.67:0/1029195087 wait complete. 2024-03-30T02:41:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.994+0000 7fa933600700 1 Processor -- start 2024-03-30T02:41:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.994+0000 7fa933600700 1 -- start start 2024-03-30T02:41:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa933600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa92c100f60 0x7fa92c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa933600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c106080 0x7fa92c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa933600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c19ba40 0x7fa92c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa933600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19c4c0 con 0x7fa92c100f60 2024-03-30T02:41:20.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa933600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19c630 con 0x7fa92c106080 2024-03-30T02:41:20.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa933600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19c7f0 con 0x7fa92c19ba40 2024-03-30T02:41:20.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa92be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c106080 0x7fa92c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:20.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa92be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c106080 0x7fa92c1976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60712/0 (socket says 172.21.15.67:60712) 2024-03-30T02:41:20.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa92be00700 1 -- 172.21.15.67:0/3871847762 learned_addr learned my addr 172.21.15.67:0/3871847762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa930c00700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa92c100f60 0x7fa92c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa931600700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c19ba40 0x7fa92c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa92be00700 1 -- 172.21.15.67:0/3871847762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c19ba40 msgr2=0x7fa92c19bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa92be00700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c19ba40 0x7fa92c19bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa92be00700 1 -- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa92c100f60 msgr2=0x7fa92c1971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa92be00700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa92c100f60 0x7fa92c1971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.995+0000 7fa92be00700 1 -- 172.21.15.67:0/3871847762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c0095f0 con 0x7fa92c106080 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.996+0000 7fa92be00700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c106080 0x7fa92c1976f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa92000d980 tx=0x7fa92000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.996+0000 7fa929600700 1 -- 172.21.15.67:0/3871847762 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920009930 con 0x7fa92c106080 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.996+0000 7fa929600700 1 -- 172.21.15.67:0/3871847762 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa920010460 con 0x7fa92c106080 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.996+0000 7fa933600700 1 -- 172.21.15.67:0/3871847762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c1a3d50 con 0x7fa92c106080 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.996+0000 7fa929600700 1 -- 172.21.15.67:0/3871847762 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000f620 con 0x7fa92c106080 2024-03-30T02:41:21.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.996+0000 7fa933600700 1 -- 172.21.15.67:0/3871847762 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c1a41d0 con 0x7fa92c106080 2024-03-30T02:41:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:20.999+0000 7fa929600700 1 -- 172.21.15.67:0/3871847762 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa9200108d0 con 0x7fa92c106080 2024-03-30T02:41:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.000+0000 7fa929600700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9140a1290 0x7fa9140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.000+0000 7fa929600700 1 -- 172.21.15.67:0/3871847762 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa9200ccb10 con 0x7fa92c106080 2024-03-30T02:41:21.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.000+0000 7fa930c00700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9140a1290 0x7fa9140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.001+0000 7fa930c00700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9140a1290 0x7fa9140a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa91c006010 tx=0x7fa91c005960 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.002+0000 7fa9470a8b80 1 Processor -- start 2024-03-30T02:41:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.002+0000 7fa9470a8b80 1 -- start start 2024-03-30T02:41:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e2159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 0x55d39e224cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e2275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e16e990 con 0x55d39e2155d0 2024-03-30T02:41:21.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e16c350 con 0x55d39e225200 2024-03-30T02:41:21.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e167c40 con 0x55d39e215ef0 2024-03-30T02:41:21.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa930c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e2159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa930c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e2159b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52908/0 (socket says 172.21.15.67:52908) 2024-03-30T02:41:21.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa930c00700 1 -- 172.21.15.67:0/639493651 learned_addr learned my addr 172.21.15.67:0/639493651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa92be00700 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 0x55d39e224cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa931600700 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e2275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa931600700 1 -- 172.21.15.67:0/639493651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 msgr2=0x55d39e224cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa931600700 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 0x55d39e224cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa931600700 1 -- 172.21.15.67:0/639493651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 msgr2=0x55d39e2159b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa931600700 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e2159b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa931600700 1 -- 172.21.15.67:0/639493651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d39dfe93b0 con 0x55d39e225200 2024-03-30T02:41:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa92be00700 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 0x55d39e224cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:21.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.003+0000 7fa930c00700 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e2159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.004+0000 7fa931600700 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e2275f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa924009310 tx=0x7fa924009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a283da4a72f272 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.004+0000 7fa919c00700 1 -- 172.21.15.67:0/639493651 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400e070 con 0x55d39e225200 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.004+0000 7fa919c00700 1 -- 172.21.15.67:0/639493651 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa92400f040 con 0x55d39e225200 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.004+0000 7fa919c00700 1 -- 172.21.15.67:0/639493651 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924014710 con 0x55d39e225200 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.004+0000 7fa9470a8b80 1 -- 172.21.15.67:0/639493651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 msgr2=0x55d39e2275f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.004+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e2275f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa924009310 tx=0x7fa924009920 comp rx=0 tx=0).stop 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.005+0000 7fa9470a8b80 1 -- 172.21.15.67:0/639493651 shutdown_connections 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.005+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e2159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.005+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e2275f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.005+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/639493651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 0x55d39e224cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.005+0000 7fa9470a8b80 1 -- 172.21.15.67:0/639493651 >> 172.21.15.67:0/639493651 conn(0x55d39e1756a0 msgr2=0x55d39e211a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.005+0000 7fa9470a8b80 1 -- 172.21.15.67:0/639493651 shutdown_connections 2024-03-30T02:41:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.005+0000 7fa9470a8b80 1 -- 172.21.15.67:0/639493651 wait complete. 2024-03-30T02:41:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.005+0000 7fa9470a8b80 1 -- start start 2024-03-30T02:41:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.006+0000 7fa9470a8b80 1 Processor -- start 2024-03-30T02:41:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.006+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e2155d0 0x55d39e224b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.006+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e215ef0 0x55d39e221eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.006+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e225200 0x55d39e222450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.006+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e103d80 con 0x55d39e225200 2024-03-30T02:41:21.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.006+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e103a20 con 0x55d39e215ef0 2024-03-30T02:41:21.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.006+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39dfeb6a0 con 0x55d39e2155d0 2024-03-30T02:41:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa9470a8b80 10 client.0 _ll_register_callbacks cb 0x55d39e10d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa92be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e215ef0 0x55d39e221eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa931600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e225200 0x55d39e222450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa92be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e215ef0 0x55d39e221eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60734/0 (socket says 172.21.15.67:60734) 2024-03-30T02:41:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa92be00700 1 -- 172.21.15.67:0/1491046203 learned_addr learned my addr 172.21.15.67:0/1491046203 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa930c00700 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e2155d0 0x55d39e224b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa92be00700 1 -- 172.21.15.67:0/1491046203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e2155d0 msgr2=0x55d39e224b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa92be00700 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e2155d0 0x55d39e224b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa92be00700 1 -- 172.21.15.67:0/1491046203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e225200 msgr2=0x55d39e222450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa92be00700 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e225200 0x55d39e222450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa92be00700 1 -- 172.21.15.67:0/1491046203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c1a41d0 con 0x55d39e215ef0 2024-03-30T02:41:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa92be00700 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e215ef0 0x55d39e221eb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa920009520 tx=0x7fa9200998e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32a1c7c56bfe86d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.007+0000 7fa90e000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa90e000700 1 -- 172.21.15.67:0/1491046203 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200b92b0 con 0x55d39e215ef0 2024-03-30T02:41:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa90e000700 1 -- 172.21.15.67:0/1491046203 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa920099ad0 con 0x55d39e215ef0 2024-03-30T02:41:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa9470a8b80 1 -- 172.21.15.67:0/1491046203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d39e128380 con 0x55d39e215ef0 2024-03-30T02:41:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa90e000700 1 -- 172.21.15.67:0/1491046203 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9200b2240 con 0x55d39e215ef0 2024-03-30T02:41:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa9470a8b80 10 client.26627 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa9470a8b80 1 -- 172.21.15.67:0/1491046203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fa924009050 con 0x55d39e215ef0 2024-03-30T02:41:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa90e000700 1 -- 172.21.15.67:0/1491046203 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d39e128380 con 0x55d39e215ef0 2024-03-30T02:41:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa90e000700 1 -- 172.21.15.67:0/1491046203 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fa9200a1860 con 0x55d39e215ef0 2024-03-30T02:41:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa9470a8b80 10 client.26627 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa9470a8b80 10 client.26627 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:41:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa9470a8b80 20 client.26627 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55d39e234dd0 0x55d39e2371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa9470a8b80 4 client.26627 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:41:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.008+0000 7fa9470a8b80 1 -- 172.21.15.67:0/1491046203 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d39e160700 con 0x55d39e234dd0 2024-03-30T02:41:21.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.009+0000 7fa930c00700 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55d39e234dd0 0x55d39e2371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.009+0000 7fa930c00700 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55d39e234dd0 0x55d39e2371f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=73623e3fa0423b74 server_cookie=3e3420d827e10b30 in_seq=0 out_seq=0 2024-03-30T02:41:21.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.009+0000 7fa90e000700 10 client.26627 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:41:21.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa90e000700 1 -- 172.21.15.67:0/1491046203 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d39e160700 con 0x55d39e234dd0 2024-03-30T02:41:21.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa90e000700 10 client.26627 handle_command_reply: tid=0 2024-03-30T02:41:21.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 client.26627 shutdown 2024-03-30T02:41:21.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 -- 172.21.15.67:0/1491046203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e215ef0 msgr2=0x55d39e221eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e215ef0 0x55d39e221eb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa920009520 tx=0x7fa9200998e0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 -- 172.21.15.67:0/1491046203 shutdown_connections 2024-03-30T02:41:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e225200 0x55d39e222450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55d39e234dd0 0x55d39e2371f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e2155d0 0x55d39e224b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/1491046203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e215ef0 0x55d39e221eb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 -- 172.21.15.67:0/1491046203 >> 172.21.15.67:0/1491046203 conn(0x55d39e1756a0 msgr2=0x55d39e217530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa90e000700 0 client.26627 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:41:21.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.010+0000 7fa9470a8b80 1 -- 172.21.15.67:0/1491046203 shutdown_connections 2024-03-30T02:41:21.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.011+0000 7fa9470a8b80 1 -- 172.21.15.67:0/1491046203 wait complete. 2024-03-30T02:41:21.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.011+0000 7fa9470a8b80 20 client.26627 trim_cache size 0 max 16384 2024-03-30T02:41:21.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.026+0000 7fa9470a8b80 1 Processor -- start 2024-03-30T02:41:21.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.026+0000 7fa9470a8b80 1 -- start start 2024-03-30T02:41:21.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.026+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e2155d0 0x55d39e222420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.026+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e215ef0 0x55d39e22be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.026+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e225200 0x55d39e2207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.026+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e212710 con 0x55d39e215ef0 2024-03-30T02:41:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.026+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e22c3c0 con 0x55d39e2155d0 2024-03-30T02:41:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.026+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e222d60 con 0x55d39e225200 2024-03-30T02:41:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa930c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e2155d0 0x55d39e222420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa92be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e215ef0 0x55d39e22be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa930c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e2155d0 0x55d39e222420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60748/0 (socket says 172.21.15.67:60748) 2024-03-30T02:41:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa930c00700 1 -- 172.21.15.67:0/100977710 learned_addr learned my addr 172.21.15.67:0/100977710 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa931600700 1 --2- 172.21.15.67:0/100977710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e225200 0x55d39e2207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa930c00700 1 -- 172.21.15.67:0/100977710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e225200 msgr2=0x55d39e2207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa930c00700 1 --2- 172.21.15.67:0/100977710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e225200 0x55d39e2207a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa930c00700 1 -- 172.21.15.67:0/100977710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e215ef0 msgr2=0x55d39e22be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa930c00700 1 --2- 172.21.15.67:0/100977710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e215ef0 0x55d39e22be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa930c00700 1 -- 172.21.15.67:0/100977710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d39df95b60 con 0x55d39e2155d0 2024-03-30T02:41:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa930c00700 1 --2- 172.21.15.67:0/100977710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e2155d0 0x55d39e222420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0059d0 tx=0x7fa91c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1494830071f77668 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.027+0000 7fa91a600700 1 -- 172.21.15.67:0/100977710 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00beb0 con 0x55d39e2155d0 2024-03-30T02:41:21.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa91a600700 1 -- 172.21.15.67:0/100977710 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91c01e040 con 0x55d39e2155d0 2024-03-30T02:41:21.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa91a600700 1 -- 172.21.15.67:0/100977710 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c004c80 con 0x55d39e2155d0 2024-03-30T02:41:21.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa9470a8b80 1 -- 172.21.15.67:0/100977710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e2155d0 msgr2=0x55d39e222420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/100977710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e2155d0 0x55d39e222420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0059d0 tx=0x7fa91c020000 comp rx=0 tx=0).stop 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa9470a8b80 1 -- 172.21.15.67:0/100977710 shutdown_connections 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/100977710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e225200 0x55d39e2207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/100977710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e2155d0 0x55d39e222420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/100977710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e215ef0 0x55d39e22be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa9470a8b80 1 -- 172.21.15.67:0/100977710 >> 172.21.15.67:0/100977710 conn(0x55d39e1756a0 msgr2=0x55d39e21d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa9470a8b80 1 -- 172.21.15.67:0/100977710 shutdown_connections 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.028+0000 7fa9470a8b80 1 -- 172.21.15.67:0/100977710 wait complete. 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.029+0000 7fa9470a8b80 1 -- start start 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.029+0000 7fa9470a8b80 1 Processor -- start 2024-03-30T02:41:21.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e22b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 0x55d39e22bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa9470a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e229e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e22a5a0 con 0x55d39e2155d0 2024-03-30T02:41:21.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e22a710 con 0x55d39e225200 2024-03-30T02:41:21.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa9470a8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39e22a8d0 con 0x55d39e215ef0 2024-03-30T02:41:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa9470a8b80 10 client.0 _ll_register_callbacks cb 0x55d39e10d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa930c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e22b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa931600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e229e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa930c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e22b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52930/0 (socket says 172.21.15.67:52930) 2024-03-30T02:41:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa930c00700 1 -- 172.21.15.67:0/689428189 learned_addr learned my addr 172.21.15.67:0/689428189 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa92be00700 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 0x55d39e22bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa931600700 1 -- 172.21.15.67:0/689428189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 msgr2=0x55d39e22bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa931600700 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 0x55d39e22bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa931600700 1 -- 172.21.15.67:0/689428189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 msgr2=0x55d39e22b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa931600700 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e22b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa931600700 1 -- 172.21.15.67:0/689428189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c00bc70 con 0x55d39e225200 2024-03-30T02:41:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.030+0000 7fa930c00700 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e22b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa931600700 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e229e60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa9240092e0 tx=0x7fa924015670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f904da097aaff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa90e000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:21.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa90e000700 1 -- 172.21.15.67:0/689428189 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400e040 con 0x55d39e225200 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa9470a8b80 1 -- 172.21.15.67:0/689428189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa92400f040 con 0x55d39e225200 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa90e000700 1 -- 172.21.15.67:0/689428189 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa924015ca0 con 0x55d39e225200 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa90e000700 1 -- 172.21.15.67:0/689428189 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9240093a0 con 0x55d39e225200 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa9470a8b80 10 client.26633 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa9470a8b80 1 -- 172.21.15.67:0/689428189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d39e12dea0 con 0x55d39e225200 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa90e000700 1 -- 172.21.15.67:0/689428189 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa92400f040 con 0x55d39e225200 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa90e000700 1 -- 172.21.15.67:0/689428189 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fa9240044a0 con 0x55d39e225200 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa9470a8b80 10 client.26633 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa9470a8b80 10 client.26633 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:41:21.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa9470a8b80 20 client.26633 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.031+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55d39e289e00 0x55d39e28c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.032+0000 7fa9470a8b80 4 client.26633 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:41:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.032+0000 7fa9470a8b80 1 -- 172.21.15.67:0/689428189 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d39dfb6cf0 con 0x55d39e289e00 2024-03-30T02:41:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.032+0000 7fa930c00700 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55d39e289e00 0x55d39e28c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.032+0000 7fa930c00700 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55d39e289e00 0x55d39e28c220 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77d29f4cea683669 server_cookie=ab9838887ef2b9c in_seq=0 out_seq=0 2024-03-30T02:41:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.032+0000 7fa90e000700 10 client.26633 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:41:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa90e000700 1 -- 172.21.15.67:0/689428189 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d39dfb6cf0 con 0x55d39e289e00 2024-03-30T02:41:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa90e000700 10 client.26633 handle_command_reply: tid=0 2024-03-30T02:41:21.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 client.26633 shutdown 2024-03-30T02:41:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 -- 172.21.15.67:0/689428189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 msgr2=0x55d39e229e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e229e60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa9240092e0 tx=0x7fa924015670 comp rx=0 tx=0).stop 2024-03-30T02:41:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 -- 172.21.15.67:0/689428189 shutdown_connections 2024-03-30T02:41:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d39e2155d0 0x55d39e22b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55d39e289e00 0x55d39e28c220 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d39e225200 0x55d39e229e60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 --2- 172.21.15.67:0/689428189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d39e215ef0 0x55d39e22bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa90e000700 0 client.26633 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:41:21.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 -- 172.21.15.67:0/689428189 >> 172.21.15.67:0/689428189 conn(0x55d39e1756a0 msgr2=0x55d39e220720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 -- 172.21.15.67:0/689428189 shutdown_connections 2024-03-30T02:41:21.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 1 -- 172.21.15.67:0/689428189 wait complete. 2024-03-30T02:41:21.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.033+0000 7fa9470a8b80 20 client.26633 trim_cache size 0 max 16384 2024-03-30T02:41:21.046 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:41:21.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 -- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9140a1290 msgr2=0x7fa9140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9140a1290 0x7fa9140a3750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa91c006010 tx=0x7fa91c005960 comp rx=0 tx=0).stop 2024-03-30T02:41:21.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 -- 172.21.15.67:0/3871847762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c106080 msgr2=0x7fa92c1976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c106080 0x7fa92c1976f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa92000d980 tx=0x7fa92000dc90 comp rx=0 tx=0).stop 2024-03-30T02:41:21.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 -- 172.21.15.67:0/3871847762 shutdown_connections 2024-03-30T02:41:21.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9140a1290 0x7fa9140a3750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa92c100f60 0x7fa92c1971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa92c19ba40 0x7fa92c19bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 --2- 172.21.15.67:0/3871847762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa92c106080 0x7fa92c1976f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 -- 172.21.15.67:0/3871847762 >> 172.21.15.67:0/3871847762 conn(0x7fa92c0fc7c0 msgr2=0x7fa92c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 -- 172.21.15.67:0/3871847762 shutdown_connections 2024-03-30T02:41:21.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.034+0000 7fa933600700 1 -- 172.21.15.67:0/3871847762 wait complete. 2024-03-30T02:41:21.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:41:21.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.277+0000 7f9882800700 1 -- 172.21.15.67:0/3333562209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c10d380 msgr2=0x7f987c10f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.277+0000 7f9882800700 1 --2- 172.21.15.67:0/3333562209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c10d380 0x7f987c10f7f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f987400b0a0 tx=0x7f987400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.278+0000 7f9882800700 1 -- 172.21.15.67:0/3333562209 shutdown_connections 2024-03-30T02:41:21.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.278+0000 7f9882800700 1 --2- 172.21.15.67:0/3333562209 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f987c0731e0 0x7f987c0735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.278+0000 7f9882800700 1 --2- 172.21.15.67:0/3333562209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c10d380 0x7f987c10f7f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.278+0000 7f9882800700 1 --2- 172.21.15.67:0/3333562209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f987c073b00 0x7f987c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.278+0000 7f9882800700 1 -- 172.21.15.67:0/3333562209 >> 172.21.15.67:0/3333562209 conn(0x7f987c0fc670 msgr2=0x7f987c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.278+0000 7f9882800700 1 -- 172.21.15.67:0/3333562209 shutdown_connections 2024-03-30T02:41:21.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.278+0000 7f9882800700 1 -- 172.21.15.67:0/3333562209 wait complete. 2024-03-30T02:41:21.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.279+0000 7f9882800700 1 Processor -- start 2024-03-30T02:41:21.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.279+0000 7f9882800700 1 -- start start 2024-03-30T02:41:21.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.279+0000 7f9882800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c0731e0 0x7f987c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.279+0000 7f9882800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f987c073b00 0x7f987c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.279+0000 7f9882800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f987c10d380 0x7f987c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.279+0000 7f9882800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c19e250 con 0x7f987c10d380 2024-03-30T02:41:21.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.279+0000 7f9882800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c19e3c0 con 0x7f987c0731e0 2024-03-30T02:41:21.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.279+0000 7f9882800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c19e580 con 0x7f987c073b00 2024-03-30T02:41:21.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c0731e0 0x7f987c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c0731e0 0x7f987c199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60784/0 (socket says 172.21.15.67:60784) 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881400700 1 -- 172.21.15.67:0/1550087477 learned_addr learned my addr 172.21.15.67:0/1550087477 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9880a00700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f987c073b00 0x7f987c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881e00700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f987c10d380 0x7f987c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881400700 1 -- 172.21.15.67:0/1550087477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f987c073b00 msgr2=0x7f987c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881400700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f987c073b00 0x7f987c1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881400700 1 -- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f987c10d380 msgr2=0x7f987c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881400700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f987c10d380 0x7f987c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881400700 1 -- 172.21.15.67:0/1550087477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9874009d00 con 0x7f987c0731e0 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f9881400700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c0731e0 0x7f987c199380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f987000e9a0 tx=0x7f987000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.280+0000 7f987a000700 1 -- 172.21.15.67:0/1550087477 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870009790 con 0x7f987c0731e0 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.281+0000 7f987a000700 1 -- 172.21.15.67:0/1550087477 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9870010430 con 0x7f987c0731e0 2024-03-30T02:41:21.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.281+0000 7f987a000700 1 -- 172.21.15.67:0/1550087477 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870018680 con 0x7f987c0731e0 2024-03-30T02:41:21.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.281+0000 7f9882800700 1 -- 172.21.15.67:0/1550087477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987c19e860 con 0x7f987c0731e0 2024-03-30T02:41:21.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.281+0000 7f9882800700 1 -- 172.21.15.67:0/1550087477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c1a60a0 con 0x7f987c0731e0 2024-03-30T02:41:21.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.284+0000 7f987a000700 1 -- 172.21.15.67:0/1550087477 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f98700105a0 con 0x7f987c0731e0 2024-03-30T02:41:21.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.286+0000 7f9897954b80 1 Processor -- start 2024-03-30T02:41:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.286+0000 7f987a000700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f98640a1310 0x7f98640a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.286+0000 7f9897954b80 1 -- start start 2024-03-30T02:41:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.286+0000 7f987a000700 1 -- 172.21.15.67:0/1550087477 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9870014070 con 0x7f987c0731e0 2024-03-30T02:41:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.286+0000 7f9881e00700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f98640a1310 0x7f98640a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9897954b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b633f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9897954b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f60c0 0x564b63400a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9897954b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 0x564b634033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9897954b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b6334e990 con 0x564b63400fc0 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9897954b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b6334c350 con 0x564b633f57a0 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9897954b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b63347c40 con 0x564b633f60c0 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9881400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b633f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9881e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 0x564b634033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9881400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b633f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60790/0 (socket says 172.21.15.67:60790) 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9881400700 1 -- 172.21.15.67:0/1513184321 learned_addr learned my addr 172.21.15.67:0/1513184321 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9880a00700 1 --2- 172.21.15.67:0/1513184321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f60c0 0x564b63400a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9881400700 1 -- 172.21.15.67:0/1513184321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f60c0 msgr2=0x564b63400a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9881400700 1 --2- 172.21.15.67:0/1513184321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f60c0 0x564b63400a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9881400700 1 -- 172.21.15.67:0/1513184321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 msgr2=0x564b634033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9881400700 1 --2- 172.21.15.67:0/1513184321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 0x564b634033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.287+0000 7f9881400700 1 -- 172.21.15.67:0/1513184321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b631c93b0 con 0x564b633f57a0 2024-03-30T02:41:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.288+0000 7f9881e00700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f98640a1310 0x7f98640a37d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f987400b780 tx=0x7f9874007d90 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.288+0000 7f9881400700 1 --2- 172.21.15.67:0/1513184321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b633f5b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f98700969a0 tx=0x7f9870099ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ca17f0e2abf716 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.288+0000 7f986e600700 1 -- 172.21.15.67:0/1513184321 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987009dd60 con 0x564b633f57a0 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.288+0000 7f986e600700 1 -- 172.21.15.67:0/1513184321 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98700a23b0 con 0x564b633f57a0 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.288+0000 7f986e600700 1 -- 172.21.15.67:0/1513184321 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870099420 con 0x564b633f57a0 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.289+0000 7f9897954b80 1 -- 172.21.15.67:0/1513184321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 msgr2=0x564b633f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.289+0000 7f9897954b80 1 --2- 172.21.15.67:0/1513184321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b633f5b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f98700969a0 tx=0x7f9870099ba0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.289+0000 7f9897954b80 1 -- 172.21.15.67:0/1513184321 shutdown_connections 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.289+0000 7f9897954b80 1 --2- 172.21.15.67:0/1513184321 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 0x564b634033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.289+0000 7f9897954b80 1 --2- 172.21.15.67:0/1513184321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b633f5b80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.289+0000 7f9897954b80 1 --2- 172.21.15.67:0/1513184321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f60c0 0x564b63400a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.289+0000 7f9897954b80 1 -- 172.21.15.67:0/1513184321 >> 172.21.15.67:0/1513184321 conn(0x564b633556a0 msgr2=0x564b633f1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.289+0000 7f9897954b80 1 -- 172.21.15.67:0/1513184321 shutdown_connections 2024-03-30T02:41:21.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.290+0000 7f9897954b80 1 -- 172.21.15.67:0/1513184321 wait complete. 2024-03-30T02:41:21.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.290+0000 7f9897954b80 1 -- start start 2024-03-30T02:41:21.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.291+0000 7f9897954b80 1 Processor -- start 2024-03-30T02:41:21.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9897954b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b63400a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9897954b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f60c0 0x564b633fdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9897954b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 0x564b633fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9897954b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b632e3d80 con 0x564b633f60c0 2024-03-30T02:41:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9897954b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b632e3a20 con 0x564b633f57a0 2024-03-30T02:41:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9897954b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b631cb6a0 con 0x564b63400fc0 2024-03-30T02:41:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9897954b80 10 client.0 _ll_register_callbacks cb 0x564b632ed470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9881400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b63400a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9881400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b63400a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60802/0 (socket says 172.21.15.67:60802) 2024-03-30T02:41:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9881400700 1 -- 172.21.15.67:0/1709992638 learned_addr learned my addr 172.21.15.67:0/1709992638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.292+0000 7f9880a00700 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f60c0 0x564b633fdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9881e00700 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 0x564b633fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9881400700 1 -- 172.21.15.67:0/1709992638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 msgr2=0x564b633fe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9881400700 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 0x564b633fe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9881400700 1 -- 172.21.15.67:0/1709992638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f60c0 msgr2=0x564b633fdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9881400700 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f60c0 0x564b633fdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9881400700 1 -- 172.21.15.67:0/1709992638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c1a60a0 con 0x564b633f57a0 2024-03-30T02:41:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9880a00700 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f60c0 0x564b633fdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:41:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9881400700 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b63400a60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f98700a1e80 tx=0x7f98700ca510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7fdb7c5c464df47 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9862a00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9862a00700 1 -- 172.21.15.67:0/1709992638 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987009a190 con 0x564b633f57a0 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9862a00700 1 -- 172.21.15.67:0/1709992638 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f987009a2f0 con 0x564b633f57a0 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.293+0000 7f9897954b80 1 -- 172.21.15.67:0/1709992638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b63308380 con 0x564b633f57a0 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9862a00700 1 -- 172.21.15.67:0/1709992638 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98700b80d0 con 0x564b633f57a0 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9897954b80 10 client.26645 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9897954b80 1 -- 172.21.15.67:0/1709992638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564b63175b60 con 0x564b633f57a0 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9862a00700 1 -- 172.21.15.67:0/1709992638 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b63308380 con 0x564b633f57a0 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9862a00700 1 -- 172.21.15.67:0/1709992638 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f98700b2b50 con 0x564b633f57a0 2024-03-30T02:41:21.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9897954b80 10 client.26645 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9897954b80 10 client.26645 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9897954b80 20 client.26645 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9897954b80 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x564b63410720 0x564b63412b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9897954b80 4 client.26645 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.294+0000 7f9897954b80 1 -- 172.21.15.67:0/1709992638 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b63340700 con 0x564b63410720 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.295+0000 7f9880a00700 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x564b63410720 0x564b63412b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.295+0000 7f9880a00700 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x564b63410720 0x564b63412b10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=adc20cb10698fd96 server_cookie=d3612dc2d4856cae in_seq=0 out_seq=0 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.295+0000 7f9862a00700 10 client.26645 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9862a00700 1 -- 172.21.15.67:0/1709992638 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564b63340700 con 0x564b63410720 2024-03-30T02:41:21.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9862a00700 10 client.26645 handle_command_reply: tid=0 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 client.26645 shutdown 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 -- 172.21.15.67:0/1709992638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 msgr2=0x564b63400a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b63400a60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f98700a1e80 tx=0x7f98700ca510 comp rx=0 tx=0).stop 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 -- 172.21.15.67:0/1709992638 shutdown_connections 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x564b63410720 0x564b63412b10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 0x564b633fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f57a0 0x564b63400a60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 --2- 172.21.15.67:0/1709992638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f60c0 0x564b633fdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9862a00700 0 client.26645 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 -- 172.21.15.67:0/1709992638 >> 172.21.15.67:0/1709992638 conn(0x564b633556a0 msgr2=0x564b633f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 -- 172.21.15.67:0/1709992638 shutdown_connections 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 1 -- 172.21.15.67:0/1709992638 wait complete. 2024-03-30T02:41:21.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.296+0000 7f9897954b80 20 client.26645 trim_cache size 0 max 16384 2024-03-30T02:41:21.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.311+0000 7f9897954b80 1 Processor -- start 2024-03-30T02:41:21.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.311+0000 7f9897954b80 1 -- start start 2024-03-30T02:41:21.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.311+0000 7f9897954b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f57a0 0x564b633fdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.311+0000 7f9897954b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9897954b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 0x564b633ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9897954b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b633f64d0 con 0x564b63400fc0 2024-03-30T02:41:21.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9897954b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b633f34e0 con 0x564b633f60c0 2024-03-30T02:41:21.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9897954b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b633ff350 con 0x564b633f57a0 2024-03-30T02:41:21.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9881400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f57a0 0x564b633fdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9880a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9881400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f57a0 0x564b633fdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36448/0 (socket says 172.21.15.67:36448) 2024-03-30T02:41:21.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9880a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60816/0 (socket says 172.21.15.67:60816) 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9881e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 0x564b633ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9881400700 1 -- 172.21.15.67:0/1558390407 learned_addr learned my addr 172.21.15.67:0/1558390407 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9881400700 1 -- 172.21.15.67:0/1558390407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 msgr2=0x564b63407c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9881400700 1 --2- 172.21.15.67:0/1558390407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9881400700 1 -- 172.21.15.67:0/1558390407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 msgr2=0x564b633ea7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9881400700 1 --2- 172.21.15.67:0/1558390407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 0x564b633ea7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9881400700 1 -- 172.21.15.67:0/1558390407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b6330dea0 con 0x564b633f57a0 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.312+0000 7f9880a00700 1 --2- 172.21.15.67:0/1558390407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.313+0000 7f9881400700 1 --2- 172.21.15.67:0/1558390407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f57a0 0x564b633fdd10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9870098280 tx=0x7f9870010a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec785d93f2dce41 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.313+0000 7f986f000700 1 -- 172.21.15.67:0/1558390407 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987009a6b0 con 0x564b633f57a0 2024-03-30T02:41:21.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.313+0000 7f986f000700 1 -- 172.21.15.67:0/1558390407 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f987009dd60 con 0x564b633f57a0 2024-03-30T02:41:21.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.313+0000 7f986f000700 1 -- 172.21.15.67:0/1558390407 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98700b8330 con 0x564b633f57a0 2024-03-30T02:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 -- 172.21.15.67:0/1558390407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f57a0 msgr2=0x564b633fdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 --2- 172.21.15.67:0/1558390407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f57a0 0x564b633fdd10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9870098280 tx=0x7f9870010a90 comp rx=0 tx=0).stop 2024-03-30T02:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 -- 172.21.15.67:0/1558390407 shutdown_connections 2024-03-30T02:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 --2- 172.21.15.67:0/1558390407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b63400fc0 0x564b633ea7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 --2- 172.21.15.67:0/1558390407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b633f57a0 0x564b633fdd10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 --2- 172.21.15.67:0/1558390407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 -- 172.21.15.67:0/1558390407 >> 172.21.15.67:0/1558390407 conn(0x564b633556a0 msgr2=0x564b633fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 -- 172.21.15.67:0/1558390407 shutdown_connections 2024-03-30T02:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 -- 172.21.15.67:0/1558390407 wait complete. 2024-03-30T02:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 -- start start 2024-03-30T02:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.314+0000 7f9897954b80 1 Processor -- start 2024-03-30T02:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9897954b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f57a0 0x564b63407430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9897954b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9897954b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 0x564b63405c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9897954b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b63406300 con 0x564b633f57a0 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9897954b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b63406470 con 0x564b633f60c0 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9897954b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b63406630 con 0x564b63400fc0 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9897954b80 10 client.0 _ll_register_callbacks cb 0x564b632ed470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9880a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9880a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60830/0 (socket says 172.21.15.67:60830) 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9880a00700 1 -- 172.21.15.67:0/1640709247 learned_addr learned my addr 172.21.15.67:0/1640709247 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9881400700 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f57a0 0x564b63407430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.315+0000 7f9881e00700 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 0x564b63405c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9880a00700 1 -- 172.21.15.67:0/1640709247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 msgr2=0x564b63405c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9880a00700 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 0x564b63405c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9880a00700 1 -- 172.21.15.67:0/1640709247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f57a0 msgr2=0x564b63407430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9880a00700 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f57a0 0x564b63407430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9880a00700 1 -- 172.21.15.67:0/1640709247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b63175b60 con 0x564b633f60c0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9880a00700 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407970 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x564b633fd5b0 tx=0x7f986801cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adf60d70b20333a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9862a00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9862a00700 1 -- 172.21.15.67:0/1640709247 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868017040 con 0x564b633f60c0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9862a00700 1 -- 172.21.15.67:0/1640709247 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9868009e10 con 0x564b633f60c0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.316+0000 7f9897954b80 1 -- 172.21.15.67:0/1640709247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98700a23b0 con 0x564b633f60c0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9862a00700 1 -- 172.21.15.67:0/1640709247 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9868003680 con 0x564b633f60c0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9897954b80 10 client.26651 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9897954b80 1 -- 172.21.15.67:0/1640709247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564b631cdcd0 con 0x564b633f60c0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9862a00700 1 -- 172.21.15.67:0/1640709247 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98700a23b0 con 0x564b633f60c0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9862a00700 1 -- 172.21.15.67:0/1640709247 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f9868025a10 con 0x564b633f60c0 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9897954b80 10 client.26651 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9897954b80 10 client.26651 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:41:21.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9897954b80 20 client.26651 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9897954b80 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x564b634656a0 0x564b63467a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9897954b80 4 client.26651 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.317+0000 7f9897954b80 1 -- 172.21.15.67:0/1640709247 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564b63196cf0 con 0x564b634656a0 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.318+0000 7f9881400700 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x564b634656a0 0x564b63467a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.318+0000 7f9881400700 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x564b634656a0 0x564b63467a90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bad610bf8c75a2b1 server_cookie=91d2633acf9a412d in_seq=0 out_seq=0 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.318+0000 7f9862a00700 10 client.26651 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.318+0000 7f9862a00700 1 -- 172.21.15.67:0/1640709247 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564b63196cf0 con 0x564b634656a0 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.318+0000 7f9862a00700 10 client.26651 handle_command_reply: tid=0 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.318+0000 7f9897954b80 1 client.26651 shutdown 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 -- 172.21.15.67:0/1640709247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 msgr2=0x564b63407970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407970 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x564b633fd5b0 tx=0x7f986801cd70 comp rx=0 tx=0).stop 2024-03-30T02:41:21.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 -- 172.21.15.67:0/1640709247 shutdown_connections 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x564b634656a0 0x564b63467a90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564b633f57a0 0x564b63407430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564b63400fc0 0x564b63405c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 --2- 172.21.15.67:0/1640709247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b633f60c0 0x564b63407970 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9862a00700 0 client.26651 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 -- 172.21.15.67:0/1640709247 >> 172.21.15.67:0/1640709247 conn(0x564b633556a0 msgr2=0x564b633eb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 -- 172.21.15.67:0/1640709247 shutdown_connections 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 1 -- 172.21.15.67:0/1640709247 wait complete. 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.319+0000 7f9897954b80 20 client.26651 trim_cache size 0 max 16384 2024-03-30T02:41:21.325 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:41:21.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 -- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f98640a1310 msgr2=0x7f98640a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f98640a1310 0x7f98640a37d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f987400b780 tx=0x7f9874007d90 comp rx=0 tx=0).stop 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 -- 172.21.15.67:0/1550087477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c0731e0 msgr2=0x7f987c199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c0731e0 0x7f987c199380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f987000e9a0 tx=0x7f987000ecb0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 -- 172.21.15.67:0/1550087477 shutdown_connections 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f98640a1310 0x7f98640a37d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f987c10d380 0x7f987c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f987c0731e0 0x7f987c199380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 --2- 172.21.15.67:0/1550087477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f987c073b00 0x7f987c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 -- 172.21.15.67:0/1550087477 >> 172.21.15.67:0/1550087477 conn(0x7f987c0fc670 msgr2=0x7f987c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 -- 172.21.15.67:0/1550087477 shutdown_connections 2024-03-30T02:41:21.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.320+0000 7f9882800700 1 -- 172.21.15.67:0/1550087477 wait complete. 2024-03-30T02:41:21.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:41:21.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.581+0000 7f28f8a00700 1 -- 172.21.15.67:0/1073219581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f4111d00 msgr2=0x7f28f41140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.582+0000 7f28f8a00700 1 --2- 172.21.15.67:0/1073219581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f4111d00 0x7f28f41140f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009f90 tx=0x7f28ec00b330 comp rx=0 tx=0).stop 2024-03-30T02:41:21.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.582+0000 7f28f8a00700 1 -- 172.21.15.67:0/1073219581 shutdown_connections 2024-03-30T02:41:21.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.582+0000 7f28f8a00700 1 --2- 172.21.15.67:0/1073219581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f4111d00 0x7f28f41140f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.582+0000 7f28f8a00700 1 --2- 172.21.15.67:0/1073219581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4073150 0x7f28f4073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.582+0000 7f28f8a00700 1 --2- 172.21.15.67:0/1073219581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28f4073a70 0x7f28f41117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.582+0000 7f28f8a00700 1 -- 172.21.15.67:0/1073219581 >> 172.21.15.67:0/1073219581 conn(0x7f28f40fc7c0 msgr2=0x7f28f40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.583+0000 7f28f8a00700 1 -- 172.21.15.67:0/1073219581 shutdown_connections 2024-03-30T02:41:21.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.583+0000 7f28f8a00700 1 -- 172.21.15.67:0/1073219581 wait complete. 2024-03-30T02:41:21.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.584+0000 7f28f8a00700 1 Processor -- start 2024-03-30T02:41:21.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.585+0000 7f28f8a00700 1 -- start start 2024-03-30T02:41:21.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.586+0000 7f28f8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f4073150 0x7f28f419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.586+0000 7f28f8a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4073a70 0x7f28f419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.586+0000 7f28f8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28f4111d00 0x7f28f41a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.586+0000 7f28f8a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41a15f0 con 0x7f28f4073150 2024-03-30T02:41:21.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.586+0000 7f28f8a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41a1760 con 0x7f28f4111d00 2024-03-30T02:41:21.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.586+0000 7f28f8a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41a1920 con 0x7f28f4073a70 2024-03-30T02:41:21.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.586+0000 7f28f1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4073a70 0x7f28f419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.587+0000 7f28f1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4073a70 0x7f28f419dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36492/0 (socket says 172.21.15.67:36492) 2024-03-30T02:41:21.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.587+0000 7f28f1400700 1 -- 172.21.15.67:0/2085198979 learned_addr learned my addr 172.21.15.67:0/2085198979 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.587+0000 7f28f1e00700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f4073150 0x7f28f419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.588+0000 7f28f1400700 1 -- 172.21.15.67:0/2085198979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28f4111d00 msgr2=0x7f28f41a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.588+0000 7f28f1400700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28f4111d00 0x7f28f41a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.588+0000 7f28f1400700 1 -- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f4073150 msgr2=0x7f28f419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.588+0000 7f28f1400700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f4073150 0x7f28f419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.588+0000 7f28f1400700 1 -- 172.21.15.67:0/2085198979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec009c40 con 0x7f28f4073a70 2024-03-30T02:41:21.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.588+0000 7f28f1e00700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f4073150 0x7f28f419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:41:21.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.589+0000 7f28f1400700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4073a70 0x7f28f419dd00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28e000d950 tx=0x7f28e000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.589+0000 7f28eaa00700 1 -- 172.21.15.67:0/2085198979 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0009940 con 0x7f28f4073a70 2024-03-30T02:41:21.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.589+0000 7f28eaa00700 1 -- 172.21.15.67:0/2085198979 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28e0010460 con 0x7f28f4073a70 2024-03-30T02:41:21.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.589+0000 7f28f8a00700 1 -- 172.21.15.67:0/2085198979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f41a1c00 con 0x7f28f4073a70 2024-03-30T02:41:21.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.589+0000 7f28f8a00700 1 -- 172.21.15.67:0/2085198979 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f41aa560 con 0x7f28f4073a70 2024-03-30T02:41:21.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.590+0000 7f28eaa00700 1 -- 172.21.15.67:0/2085198979 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e000f620 con 0x7f28f4073a70 2024-03-30T02:41:21.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.600+0000 7f28eaa00700 1 -- 172.21.15.67:0/2085198979 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f28e000f780 con 0x7f28f4073a70 2024-03-30T02:41:21.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.603+0000 7f290c57ab80 1 Processor -- start 2024-03-30T02:41:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.603+0000 7f28eaa00700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f28d80b23a0 0x7f28d80b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.603+0000 7f290c57ab80 1 -- start start 2024-03-30T02:41:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.603+0000 7f28eaa00700 1 -- 172.21.15.67:0/2085198979 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f28e00ce120 con 0x7f28f4073a70 2024-03-30T02:41:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.604+0000 7f28f2800700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f28d80b23a0 0x7f28d80b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.604+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.604+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 0x5630b4c5c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.604+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.604+0000 7f290c57ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4baa990 con 0x5630b4c5cdf0 2024-03-30T02:41:21.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.604+0000 7f290c57ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4ba8350 con 0x5630b4c515d0 2024-03-30T02:41:21.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.604+0000 7f290c57ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4ba3c40 con 0x5630b4c51ef0 2024-03-30T02:41:21.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.605+0000 7f28f1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.605+0000 7f28f1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 0x5630b4c5c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.605+0000 7f28f1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 0x5630b4c5c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36502/0 (socket says 172.21.15.67:36502) 2024-03-30T02:41:21.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.605+0000 7f28f1400700 1 -- 172.21.15.67:0/3763255528 learned_addr learned my addr 172.21.15.67:0/3763255528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.606+0000 7f28f2800700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f28d80b23a0 0x7f28d80b4860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28f406a380 tx=0x7f28ec009730 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.606+0000 7f28f1e00700 1 -- 172.21.15.67:0/3763255528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 msgr2=0x5630b4c5c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.606+0000 7f28f1e00700 1 --2- 172.21.15.67:0/3763255528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 0x5630b4c5c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.606+0000 7f28f1e00700 1 -- 172.21.15.67:0/3763255528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 msgr2=0x5630b4c5f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.606+0000 7f28f1e00700 1 --2- 172.21.15.67:0/3763255528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.606+0000 7f28f1e00700 1 -- 172.21.15.67:0/3763255528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630b4a253b0 con 0x5630b4c515d0 2024-03-30T02:41:21.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.612+0000 7f28f1e00700 1 --2- 172.21.15.67:0/3763255528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c519b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f28e4003830 tx=0x7f28e4003b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa83c3bfb7444ff3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.616+0000 7f28df000700 1 -- 172.21.15.67:0/3763255528 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400b9d0 con 0x5630b4c515d0 2024-03-30T02:41:21.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.616+0000 7f28df000700 1 -- 172.21.15.67:0/3763255528 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28e4011440 con 0x5630b4c515d0 2024-03-30T02:41:21.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.616+0000 7f28df000700 1 -- 172.21.15.67:0/3763255528 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4019660 con 0x5630b4c515d0 2024-03-30T02:41:21.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.617+0000 7f290c57ab80 1 -- 172.21.15.67:0/3763255528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 msgr2=0x5630b4c519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.617+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3763255528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c519b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f28e4003830 tx=0x7f28e4003b40 comp rx=0 tx=0).stop 2024-03-30T02:41:21.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.618+0000 7f290c57ab80 1 -- 172.21.15.67:0/3763255528 shutdown_connections 2024-03-30T02:41:21.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.618+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3763255528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.618+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3763255528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c519b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.618+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3763255528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 0x5630b4c5c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.618+0000 7f290c57ab80 1 -- 172.21.15.67:0/3763255528 >> 172.21.15.67:0/3763255528 conn(0x5630b4bb16a0 msgr2=0x5630b4c4da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.618+0000 7f290c57ab80 1 -- 172.21.15.67:0/3763255528 shutdown_connections 2024-03-30T02:41:21.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.621+0000 7f290c57ab80 1 -- 172.21.15.67:0/3763255528 wait complete. 2024-03-30T02:41:21.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.621+0000 7f290c57ab80 1 -- start start 2024-03-30T02:41:21.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.622+0000 7f290c57ab80 1 Processor -- start 2024-03-30T02:41:21.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.625+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c5c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.625+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c59b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.625+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.625+0000 7f290c57ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4b3fd80 con 0x5630b4c51ef0 2024-03-30T02:41:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.625+0000 7f290c57ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4b3fa20 con 0x5630b4c515d0 2024-03-30T02:41:21.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.625+0000 7f290c57ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4a276a0 con 0x5630b4c5cdf0 2024-03-30T02:41:21.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.626+0000 7f290c57ab80 10 client.0 _ll_register_callbacks cb 0x5630b4b49470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:21.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.626+0000 7f28f1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c59b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.626+0000 7f28f1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c59b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53016/0 (socket says 172.21.15.67:53016) 2024-03-30T02:41:21.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.626+0000 7f28f1400700 1 -- 172.21.15.67:0/3585043610 learned_addr learned my addr 172.21.15.67:0/3585043610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.626+0000 7f28f1400700 1 -- 172.21.15.67:0/3585043610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c5cdf0 msgr2=0x5630b4c5a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.626+0000 7f28f1400700 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.627+0000 7f28f1400700 1 -- 172.21.15.67:0/3585043610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 msgr2=0x5630b4c5c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.627+0000 7f28f1400700 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c5c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.627+0000 7f28f1400700 1 -- 172.21.15.67:0/3585043610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e40044b0 con 0x5630b4c51ef0 2024-03-30T02:41:21.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.627+0000 7f28f1400700 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c59b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f28e009ae10 tx=0x7f28e0099430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb3ffcbe7eebaa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.627+0000 7f28d3400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:41:21.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.628+0000 7f28d3400700 1 -- 172.21.15.67:0/3585043610 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e009b2e0 con 0x5630b4c51ef0 2024-03-30T02:41:21.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.628+0000 7f28d3400700 1 -- 172.21.15.67:0/3585043610 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f28e009a370 con 0x5630b4c51ef0 2024-03-30T02:41:21.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.628+0000 7f290c57ab80 1 -- 172.21.15.67:0/3585043610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630b4b64380 con 0x5630b4c51ef0 2024-03-30T02:41:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.630+0000 7f28d3400700 1 -- 172.21.15.67:0/3585043610 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e00cb950 con 0x5630b4c51ef0 2024-03-30T02:41:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.630+0000 7f290c57ab80 10 client.16563 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.630+0000 7f290c57ab80 1 -- 172.21.15.67:0/3585043610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5630b49d1b60 con 0x5630b4c51ef0 2024-03-30T02:41:21.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.631+0000 7f28d3400700 1 -- 172.21.15.67:0/3585043610 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5630b4b64380 con 0x5630b4c51ef0 2024-03-30T02:41:21.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.631+0000 7f28d3400700 1 -- 172.21.15.67:0/3585043610 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f28e0099de0 con 0x5630b4c51ef0 2024-03-30T02:41:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.631+0000 7f290c57ab80 10 client.16563 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.631+0000 7f290c57ab80 10 client.16563 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:41:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.631+0000 7f290c57ab80 20 client.16563 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.631+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5630b4c6ca70 0x5630b4c6ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.631+0000 7f290c57ab80 4 client.16563 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:41:21.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.631+0000 7f290c57ab80 1 -- 172.21.15.67:0/3585043610 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630b4b9c700 con 0x5630b4c6ca70 2024-03-30T02:41:21.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.637+0000 7f28f1e00700 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5630b4c6ca70 0x5630b4c6ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.638+0000 7f28d3400700 10 client.16563 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:41:21.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.638+0000 7f28f1e00700 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5630b4c6ca70 0x5630b4c6ee90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52d04c2c873225e8 server_cookie=198d42f2ac180f1b in_seq=0 out_seq=0 2024-03-30T02:41:21.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.653+0000 7f28d3400700 1 -- 172.21.15.67:0/3585043610 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5630b4b9c700 con 0x5630b4c6ca70 2024-03-30T02:41:21.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.653+0000 7f28d3400700 10 client.16563 handle_command_reply: tid=0 2024-03-30T02:41:21.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f290c57ab80 1 client.16563 shutdown 2024-03-30T02:41:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f290c57ab80 1 -- 172.21.15.67:0/3585043610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 msgr2=0x5630b4c59b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c59b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f28e009ae10 tx=0x7f28e0099430 comp rx=0 tx=0).stop 2024-03-30T02:41:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f290c57ab80 1 -- 172.21.15.67:0/3585043610 shutdown_connections 2024-03-30T02:41:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5630b4c6ca70 0x5630b4c6ee90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f28d3400700 0 client.16563 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:41:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c5c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3585043610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c59b00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.656+0000 7f290c57ab80 1 -- 172.21.15.67:0/3585043610 >> 172.21.15.67:0/3585043610 conn(0x5630b4bb16a0 msgr2=0x5630b4c536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.657+0000 7f290c57ab80 1 -- 172.21.15.67:0/3585043610 shutdown_connections 2024-03-30T02:41:21.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.657+0000 7f290c57ab80 1 -- 172.21.15.67:0/3585043610 wait complete. 2024-03-30T02:41:21.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.657+0000 7f290c57ab80 20 client.16563 trim_cache size 0 max 16384 2024-03-30T02:41:21.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:21 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3725572995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:41:21.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.686+0000 7f290c57ab80 1 Processor -- start 2024-03-30T02:41:21.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.687+0000 7f290c57ab80 1 -- start start 2024-03-30T02:41:21.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.687+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c6ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.687+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 0x5630b4c63b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.687+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.688+0000 7f290c57ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4c52880 con 0x5630b4c5cdf0 2024-03-30T02:41:21.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.688+0000 7f290c57ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4c6a0e0 con 0x5630b4c515d0 2024-03-30T02:41:21.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.688+0000 7f290c57ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4c545a0 con 0x5630b4c51ef0 2024-03-30T02:41:21.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.688+0000 7f28f1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 0x5630b4c63b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.688+0000 7f28f2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.688+0000 7f28f2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53024/0 (socket says 172.21.15.67:53024) 2024-03-30T02:41:21.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.688+0000 7f28f2800700 1 -- 172.21.15.67:0/3710975848 learned_addr learned my addr 172.21.15.67:0/3710975848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.689+0000 7f28f2800700 1 -- 172.21.15.67:0/3710975848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 msgr2=0x5630b4c63b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.689+0000 7f28f2800700 1 --2- 172.21.15.67:0/3710975848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 0x5630b4c63b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.689+0000 7f28f2800700 1 -- 172.21.15.67:0/3710975848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 msgr2=0x5630b4c6ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.689+0000 7f28f2800700 1 --2- 172.21.15.67:0/3710975848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c6ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.689+0000 7f28f2800700 1 -- 172.21.15.67:0/3710975848 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630b4b69ea0 con 0x5630b4c5cdf0 2024-03-30T02:41:21.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.689+0000 7f28f2800700 1 --2- 172.21.15.67:0/3710975848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5c2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f28ec007f70 tx=0x7f28ec00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5510586373f5d51d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:21 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3725572995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:41:21.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.694+0000 7f28dfa00700 1 -- 172.21.15.67:0/3710975848 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec00e040 con 0x5630b4c5cdf0 2024-03-30T02:41:21.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.694+0000 7f28dfa00700 1 -- 172.21.15.67:0/3710975848 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28ec01b430 con 0x5630b4c5cdf0 2024-03-30T02:41:21.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.694+0000 7f28dfa00700 1 -- 172.21.15.67:0/3710975848 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec028680 con 0x5630b4c5cdf0 2024-03-30T02:41:21.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.695+0000 7f290c57ab80 1 -- 172.21.15.67:0/3710975848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 msgr2=0x5630b4c5c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.695+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3710975848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5c2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f28ec007f70 tx=0x7f28ec00b730 comp rx=0 tx=0).stop 2024-03-30T02:41:21.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.696+0000 7f290c57ab80 1 -- 172.21.15.67:0/3710975848 shutdown_connections 2024-03-30T02:41:21.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.696+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3710975848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c5cdf0 0x5630b4c5c2f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.696+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3710975848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c6ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.696+0000 7f290c57ab80 1 --2- 172.21.15.67:0/3710975848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c51ef0 0x5630b4c63b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.696+0000 7f290c57ab80 1 -- 172.21.15.67:0/3710975848 >> 172.21.15.67:0/3710975848 conn(0x5630b4bb16a0 msgr2=0x5630b4c465a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.698+0000 7f290c57ab80 1 -- 172.21.15.67:0/3710975848 shutdown_connections 2024-03-30T02:41:21.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.698+0000 7f290c57ab80 1 -- 172.21.15.67:0/3710975848 wait complete. 2024-03-30T02:41:21.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.698+0000 7f290c57ab80 1 -- start start 2024-03-30T02:41:21.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.699+0000 7f290c57ab80 1 Processor -- start 2024-03-30T02:41:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c63300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c63840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f290c57ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c5cdf0 0x5630b4c61960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f28f1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c63300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f290c57ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4c62040 con 0x5630b4c51ef0 2024-03-30T02:41:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f290c57ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4c621b0 con 0x5630b4c515d0 2024-03-30T02:41:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f28f1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c63300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60876/0 (socket says 172.21.15.67:60876) 2024-03-30T02:41:21.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f28f1e00700 1 -- 172.21.15.67:0/2713571868 learned_addr learned my addr 172.21.15.67:0/2713571868 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f290c57ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630b4c62370 con 0x5630b4c5cdf0 2024-03-30T02:41:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f290c57ab80 10 client.0 _ll_register_callbacks cb 0x5630b4b49470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.700+0000 7f28f1400700 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c63840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.701+0000 7f28f1400700 1 -- 172.21.15.67:0/2713571868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c5cdf0 msgr2=0x5630b4c61960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.701+0000 7f28f1400700 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c5cdf0 0x5630b4c61960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.701+0000 7f28f1400700 1 -- 172.21.15.67:0/2713571868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 msgr2=0x5630b4c63300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:21.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.701+0000 7f28f1400700 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c63300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.701+0000 7f28f1400700 1 -- 172.21.15.67:0/2713571868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f4114aa0 con 0x5630b4c51ef0 2024-03-30T02:41:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.702+0000 7f28f1400700 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c63840 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5630b4c695e0 tx=0x7f28e0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f959bc40ff6b17b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.702+0000 7f28d3400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:41:21.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.702+0000 7f28d3400700 1 -- 172.21.15.67:0/2713571868 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0014050 con 0x5630b4c51ef0 2024-03-30T02:41:21.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.702+0000 7f28d3400700 1 -- 172.21.15.67:0/2713571868 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f28e0009aa0 con 0x5630b4c51ef0 2024-03-30T02:41:21.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.703+0000 7f290c57ab80 1 -- 172.21.15.67:0/2713571868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28e4011440 con 0x5630b4c51ef0 2024-03-30T02:41:21.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.703+0000 7f28d3400700 1 -- 172.21.15.67:0/2713571868 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28e00cb950 con 0x5630b4c51ef0 2024-03-30T02:41:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.703+0000 7f290c57ab80 10 client.16569 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.703+0000 7f290c57ab80 1 -- 172.21.15.67:0/2713571868 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5630b4a29cd0 con 0x5630b4c51ef0 2024-03-30T02:41:21.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.705+0000 7f28d3400700 1 -- 172.21.15.67:0/2713571868 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28e4011440 con 0x5630b4c51ef0 2024-03-30T02:41:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.705+0000 7f28d3400700 1 -- 172.21.15.67:0/2713571868 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f28e009b0e0 con 0x5630b4c51ef0 2024-03-30T02:41:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.705+0000 7f290c57ab80 10 client.16569 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.705+0000 7f290c57ab80 10 client.16569 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:41:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.705+0000 7f290c57ab80 20 client.16569 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:21.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.705+0000 7f290c57ab80 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5630b4cc1800 0x5630b4cc3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:21.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.705+0000 7f290c57ab80 4 client.16569 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:41:21.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.705+0000 7f290c57ab80 1 -- 172.21.15.67:0/2713571868 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5630b49f2cf0 con 0x5630b4cc1800 2024-03-30T02:41:21.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.706+0000 7f28f1e00700 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5630b4cc1800 0x5630b4cc3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:21.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.707+0000 7f28f1e00700 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5630b4cc1800 0x5630b4cc3c20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=990602ea4f3e467 server_cookie=1118771476023c9d in_seq=0 out_seq=0 2024-03-30T02:41:21.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.707+0000 7f28d3400700 10 client.16569 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:41:21.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.709+0000 7f28d3400700 1 -- 172.21.15.67:0/2713571868 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5630b49f2cf0 con 0x5630b4cc1800 2024-03-30T02:41:21.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.709+0000 7f28d3400700 10 client.16569 handle_command_reply: tid=0 2024-03-30T02:41:21.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.709+0000 7f290c57ab80 1 client.16569 shutdown 2024-03-30T02:41:21.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.709+0000 7f290c57ab80 1 -- 172.21.15.67:0/2713571868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 msgr2=0x5630b4c63840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.709+0000 7f290c57ab80 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c63840 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x5630b4c695e0 tx=0x7f28e0000bc0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.710+0000 7f290c57ab80 1 -- 172.21.15.67:0/2713571868 shutdown_connections 2024-03-30T02:41:21.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.710+0000 7f290c57ab80 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5630b4cc1800 0x5630b4cc3c20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.710+0000 7f290c57ab80 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5630b4c5cdf0 0x5630b4c61960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.710+0000 7f290c57ab80 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630b4c515d0 0x5630b4c63300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.710+0000 7f290c57ab80 1 --2- 172.21.15.67:0/2713571868 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5630b4c51ef0 0x5630b4c63840 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.710+0000 7f28d3400700 0 client.16569 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:41:21.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.710+0000 7f290c57ab80 1 -- 172.21.15.67:0/2713571868 >> 172.21.15.67:0/2713571868 conn(0x5630b4bb16a0 msgr2=0x5630b4c58260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.711+0000 7f290c57ab80 1 -- 172.21.15.67:0/2713571868 shutdown_connections 2024-03-30T02:41:21.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.711+0000 7f290c57ab80 1 -- 172.21.15.67:0/2713571868 wait complete. 2024-03-30T02:41:21.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.711+0000 7f290c57ab80 20 client.16569 trim_cache size 0 max 16384 2024-03-30T02:41:21.714 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:41:21.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.714+0000 7f28f8a00700 1 -- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f28d80b23a0 msgr2=0x7f28d80b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.714+0000 7f28f8a00700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f28d80b23a0 0x7f28d80b4860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28f406a380 tx=0x7f28ec009730 comp rx=0 tx=0).stop 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.714+0000 7f28f8a00700 1 -- 172.21.15.67:0/2085198979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4073a70 msgr2=0x7f28f419dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.714+0000 7f28f8a00700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4073a70 0x7f28f419dd00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28e000d950 tx=0x7f28e000dc60 comp rx=0 tx=0).stop 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.715+0000 7f28f8a00700 1 -- 172.21.15.67:0/2085198979 shutdown_connections 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.715+0000 7f28f8a00700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f28d80b23a0 0x7f28d80b4860 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.715+0000 7f28f8a00700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28f4073150 0x7f28f419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.715+0000 7f28f8a00700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28f4111d00 0x7f28f41a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.715+0000 7f28f8a00700 1 --2- 172.21.15.67:0/2085198979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28f4073a70 0x7f28f419dd00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.715+0000 7f28f8a00700 1 -- 172.21.15.67:0/2085198979 >> 172.21.15.67:0/2085198979 conn(0x7f28f40fc7c0 msgr2=0x7f28f40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:21.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.715+0000 7f28f8a00700 1 -- 172.21.15.67:0/2085198979 shutdown_connections 2024-03-30T02:41:21.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:21.715+0000 7f28f8a00700 1 -- 172.21.15.67:0/2085198979 wait complete. 2024-03-30T02:41:21.751 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:41:21.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:21 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3725572995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:41:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.139+0000 7f159f000700 1 -- 172.21.15.67:0/1001112230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598108280 msgr2=0x7f159810fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.139+0000 7f159f000700 1 --2- 172.21.15.67:0/1001112230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598108280 0x7f159810fe00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1590014a30 tx=0x7f1590014d40 comp rx=0 tx=0).stop 2024-03-30T02:41:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.140+0000 7f159f000700 1 -- 172.21.15.67:0/1001112230 shutdown_connections 2024-03-30T02:41:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.140+0000 7f159f000700 1 --2- 172.21.15.67:0/1001112230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598108280 0x7f159810fe00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.140+0000 7f159f000700 1 --2- 172.21.15.67:0/1001112230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1598103160 0x7f1598103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.140+0000 7f159f000700 1 --2- 172.21.15.67:0/1001112230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1598103b10 0x7f1598107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.140+0000 7f159f000700 1 -- 172.21.15.67:0/1001112230 >> 172.21.15.67:0/1001112230 conn(0x7f15980fe9c0 msgr2=0x7f1598100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.140+0000 7f159f000700 1 -- 172.21.15.67:0/1001112230 shutdown_connections 2024-03-30T02:41:22.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.141+0000 7f159f000700 1 -- 172.21.15.67:0/1001112230 wait complete. 2024-03-30T02:41:22.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.142+0000 7f159f000700 1 Processor -- start 2024-03-30T02:41:22.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.143+0000 7f159f000700 1 -- start start 2024-03-30T02:41:22.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.143+0000 7f159f000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598103160 0x7f159806a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.143+0000 7f159f000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1598108280 0x7f159806aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.143+0000 7f159f000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159806fdd0 0x7f1598070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.144+0000 7f159f000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1598070790 con 0x7f1598103160 2024-03-30T02:41:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.144+0000 7f159f000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1598070900 con 0x7f159806fdd0 2024-03-30T02:41:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.144+0000 7f159f000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1598070ac0 con 0x7f1598108280 2024-03-30T02:41:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.144+0000 7f1597e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598103160 0x7f159806a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.144+0000 7f1597e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598103160 0x7f159806a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53036/0 (socket says 172.21.15.67:53036) 2024-03-30T02:41:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.144+0000 7f1597e00700 1 -- 172.21.15.67:0/1221716937 learned_addr learned my addr 172.21.15.67:0/1221716937 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.144+0000 7f1597400700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1598108280 0x7f159806aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.145+0000 7f159d000700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159806fdd0 0x7f1598070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.145+0000 7f1597e00700 1 -- 172.21.15.67:0/1221716937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1598108280 msgr2=0x7f159806aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.145+0000 7f1597e00700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1598108280 0x7f159806aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.145+0000 7f1597e00700 1 -- 172.21.15.67:0/1221716937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159806fdd0 msgr2=0x7f1598070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.145+0000 7f1597e00700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159806fdd0 0x7f1598070250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.145+0000 7f1597e00700 1 -- 172.21.15.67:0/1221716937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1590014660 con 0x7f1598103160 2024-03-30T02:41:22.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.145+0000 7f159d000700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159806fdd0 0x7f1598070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:41:22.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.150+0000 7f1597e00700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598103160 0x7f159806a9b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f158800b530 tx=0x7f158800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.152+0000 7f1594c00700 1 -- 172.21.15.67:0/1221716937 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588011830 con 0x7f1598103160 2024-03-30T02:41:22.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.152+0000 7f1594c00700 1 -- 172.21.15.67:0/1221716937 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1588011e50 con 0x7f1598103160 2024-03-30T02:41:22.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.152+0000 7f1594c00700 1 -- 172.21.15.67:0/1221716937 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800f2e0 con 0x7f1598103160 2024-03-30T02:41:22.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.152+0000 7f159f000700 1 -- 172.21.15.67:0/1221716937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15981aa600 con 0x7f1598103160 2024-03-30T02:41:22.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.153+0000 7f159f000700 1 -- 172.21.15.67:0/1221716937 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15981aa9d0 con 0x7f1598103160 2024-03-30T02:41:22.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.162+0000 7f1594c00700 1 -- 172.21.15.67:0/1221716937 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1588010580 con 0x7f1598103160 2024-03-30T02:41:22.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.164+0000 7f1594c00700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f157c0a1390 0x7f157c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.164+0000 7f1594c00700 1 -- 172.21.15.67:0/1221716937 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f15880cca00 con 0x7f1598103160 2024-03-30T02:41:22.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.164+0000 7f1597400700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f157c0a1390 0x7f157c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.166+0000 7f1597400700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f157c0a1390 0x7f157c0a3850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f159806ba30 tx=0x7f1584009310 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.167+0000 7f15b2afab80 1 Processor -- start 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.168+0000 7f15b2afab80 1 -- start start 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.168+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f397f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.168+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f398038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.168+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f398061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.168+0000 7f15b2afab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f39751990 con 0x557f397f8ef0 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.168+0000 7f15b2afab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f3974f350 con 0x557f39803df0 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.169+0000 7f15b2afab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f3974ac40 con 0x557f397f85d0 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.169+0000 7f159d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f398061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.169+0000 7f159d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f398061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60910/0 (socket says 172.21.15.67:60910) 2024-03-30T02:41:22.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.169+0000 7f159d000700 1 -- 172.21.15.67:0/126414837 learned_addr learned my addr 172.21.15.67:0/126414837 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.170+0000 7f1597e00700 1 --2- 172.21.15.67:0/126414837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f397f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.170+0000 7f159d000700 1 -- 172.21.15.67:0/126414837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 msgr2=0x557f397f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.170+0000 7f159d000700 1 --2- 172.21.15.67:0/126414837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f397f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.170+0000 7f159d000700 1 -- 172.21.15.67:0/126414837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 msgr2=0x557f398038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.170+0000 7f159d000700 1 --2- 172.21.15.67:0/126414837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f398038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.170+0000 7f159d000700 1 -- 172.21.15.67:0/126414837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f395cc3b0 con 0x557f39803df0 2024-03-30T02:41:22.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.171+0000 7f159d000700 1 --2- 172.21.15.67:0/126414837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f398061e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f15900136d0 tx=0x7f159001c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4d652d9d279319 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.172+0000 7f158d600700 1 -- 172.21.15.67:0/126414837 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590014010 con 0x557f39803df0 2024-03-30T02:41:22.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.172+0000 7f158d600700 1 -- 172.21.15.67:0/126414837 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1590014170 con 0x557f39803df0 2024-03-30T02:41:22.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.173+0000 7f158d600700 1 -- 172.21.15.67:0/126414837 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590013ae0 con 0x557f39803df0 2024-03-30T02:41:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.174+0000 7f15b2afab80 1 -- 172.21.15.67:0/126414837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 msgr2=0x557f398061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.174+0000 7f15b2afab80 1 --2- 172.21.15.67:0/126414837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f398061e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f15900136d0 tx=0x7f159001c820 comp rx=0 tx=0).stop 2024-03-30T02:41:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.174+0000 7f15b2afab80 1 -- 172.21.15.67:0/126414837 shutdown_connections 2024-03-30T02:41:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.174+0000 7f15b2afab80 1 --2- 172.21.15.67:0/126414837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f398061e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.174+0000 7f15b2afab80 1 --2- 172.21.15.67:0/126414837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f397f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.174+0000 7f15b2afab80 1 --2- 172.21.15.67:0/126414837 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f398038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.174+0000 7f15b2afab80 1 -- 172.21.15.67:0/126414837 >> 172.21.15.67:0/126414837 conn(0x557f397586a0 msgr2=0x557f397f4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.175+0000 7f15b2afab80 1 -- 172.21.15.67:0/126414837 shutdown_connections 2024-03-30T02:41:22.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.175+0000 7f15b2afab80 1 -- 172.21.15.67:0/126414837 wait complete. 2024-03-30T02:41:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.176+0000 7f15b2afab80 1 -- start start 2024-03-30T02:41:22.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.176+0000 7f15b2afab80 1 Processor -- start 2024-03-30T02:41:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39800bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f39801110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f1597e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39800bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f1597e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39800bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36560/0 (socket says 172.21.15.67:36560) 2024-03-30T02:41:22.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f1597e00700 1 -- 172.21.15.67:0/3905732496 learned_addr learned my addr 172.21.15.67:0/3905732496 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f396e6d80 con 0x557f397f8ef0 2024-03-30T02:41:22.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f396e6a20 con 0x557f39803df0 2024-03-30T02:41:22.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f395ce6a0 con 0x557f397f85d0 2024-03-30T02:41:22.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.178+0000 7f15b2afab80 10 client.0 _ll_register_callbacks cb 0x557f396f0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:22.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.179+0000 7f1597400700 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f39801110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.179+0000 7f159d000700 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.180+0000 7f1597e00700 1 -- 172.21.15.67:0/3905732496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 msgr2=0x557f39801650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.180+0000 7f1597e00700 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.180+0000 7f1597e00700 1 -- 172.21.15.67:0/3905732496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 msgr2=0x557f39801110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.180+0000 7f1597e00700 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f39801110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.180+0000 7f1597e00700 1 -- 172.21.15.67:0/3905732496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1598110670 con 0x557f397f85d0 2024-03-30T02:41:22.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.187+0000 7f1597e00700 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39800bd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f158800b500 tx=0x7f1588010400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cfcba53948a078 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.187+0000 7f1582000700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:41:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.193+0000 7f1582000700 1 -- 172.21.15.67:0/3905732496 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15880b2200 con 0x557f397f85d0 2024-03-30T02:41:22.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.193+0000 7f1582000700 1 -- 172.21.15.67:0/3905732496 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1588099870 con 0x557f397f85d0 2024-03-30T02:41:22.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.193+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f3970b380 con 0x557f397f85d0 2024-03-30T02:41:22.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.195+0000 7f1582000700 1 -- 172.21.15.67:0/3905732496 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f158809d6a0 con 0x557f397f85d0 2024-03-30T02:41:22.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.196+0000 7f15b2afab80 10 client.26623 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:22.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.196+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557f39578b60 con 0x557f397f85d0 2024-03-30T02:41:22.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.204+0000 7f1582000700 1 -- 172.21.15.67:0/3905732496 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f158807c020 con 0x557f397f85d0 2024-03-30T02:41:22.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.207+0000 7f1582000700 1 -- 172.21.15.67:0/3905732496 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f15880a1860 con 0x557f397f85d0 2024-03-30T02:41:22.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.208+0000 7f15b2afab80 10 client.26623 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.208+0000 7f15b2afab80 10 client.26623 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:41:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.208+0000 7f15b2afab80 20 client.26623 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.208+0000 7f15b2afab80 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557f398249e0 0x557f39826e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.208+0000 7f15b2afab80 4 client.26623 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:41:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.208+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f39743700 con 0x557f398249e0 2024-03-30T02:41:22.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.209+0000 7f1597400700 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557f398249e0 0x557f39826e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.213+0000 7f1582000700 10 client.26623 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:41:22.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.213+0000 7f1597400700 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557f398249e0 0x557f39826e00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc6e9a26d0ddf6df server_cookie=c608568102b05fe9 in_seq=0 out_seq=0 2024-03-30T02:41:22.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.220+0000 7f1582000700 1 -- 172.21.15.67:0/3905732496 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557f39743700 con 0x557f398249e0 2024-03-30T02:41:22.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.220+0000 7f1582000700 10 client.26623 handle_command_reply: tid=0 2024-03-30T02:41:22.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.220+0000 7f15b2afab80 1 client.26623 shutdown 2024-03-30T02:41:22.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.221+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 msgr2=0x557f39800bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.221+0000 7f15b2afab80 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39800bd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f158800b500 tx=0x7f1588010400 comp rx=0 tx=0).stop 2024-03-30T02:41:22.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.222+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 shutdown_connections 2024-03-30T02:41:22.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.222+0000 7f15b2afab80 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.222+0000 7f15b2afab80 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39800bd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.222+0000 7f15b2afab80 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557f398249e0 0x557f39826e00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.222+0000 7f15b2afab80 1 --2- 172.21.15.67:0/3905732496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f39801110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.222+0000 7f1582000700 0 client.26623 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:41:22.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.222+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 >> 172.21.15.67:0/3905732496 conn(0x557f397586a0 msgr2=0x557f397fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.223+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 shutdown_connections 2024-03-30T02:41:22.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.224+0000 7f15b2afab80 1 -- 172.21.15.67:0/3905732496 wait complete. 2024-03-30T02:41:22.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.224+0000 7f15b2afab80 20 client.26623 trim_cache size 0 max 16384 2024-03-30T02:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.258+0000 7f15b2afab80 1 Processor -- start 2024-03-30T02:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.258+0000 7f15b2afab80 1 -- start start 2024-03-30T02:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.258+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f85d0 0x557f397fb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.258+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f8ef0 0x557f3981ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.258+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.258+0000 7f15b2afab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f3981bf60 con 0x557f397f85d0 2024-03-30T02:41:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.258+0000 7f15b2afab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f397ff120 con 0x557f39803df0 2024-03-30T02:41:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.258+0000 7f15b2afab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f397f83c0 con 0x557f397f8ef0 2024-03-30T02:41:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.259+0000 7f159d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.259+0000 7f1597e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f85d0 0x557f397fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.259+0000 7f159d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60916/0 (socket says 172.21.15.67:60916) 2024-03-30T02:41:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.259+0000 7f159d000700 1 -- 172.21.15.67:0/6117644 learned_addr learned my addr 172.21.15.67:0/6117644 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.259+0000 7f1597400700 1 --2- 172.21.15.67:0/6117644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f8ef0 0x557f3981ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.260+0000 7f1597e00700 1 -- 172.21.15.67:0/6117644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f8ef0 msgr2=0x557f3981ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.260+0000 7f1597e00700 1 --2- 172.21.15.67:0/6117644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f8ef0 0x557f3981ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.260+0000 7f1597e00700 1 -- 172.21.15.67:0/6117644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 msgr2=0x557f39801470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.260+0000 7f1597e00700 1 --2- 172.21.15.67:0/6117644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.260+0000 7f1597e00700 1 -- 172.21.15.67:0/6117644 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f39710ea0 con 0x557f397f85d0 2024-03-30T02:41:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.260+0000 7f159d000700 1 --2- 172.21.15.67:0/6117644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.260+0000 7f1597e00700 1 --2- 172.21.15.67:0/6117644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f85d0 0x557f397fb610 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f158800bcf0 tx=0x7f1588010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3738c0a400fc5759 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.261+0000 7f158e000700 1 -- 172.21.15.67:0/6117644 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588099020 con 0x557f397f85d0 2024-03-30T02:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.261+0000 7f158e000700 1 -- 172.21.15.67:0/6117644 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f158807c070 con 0x557f397f85d0 2024-03-30T02:41:22.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.261+0000 7f158e000700 1 -- 172.21.15.67:0/6117644 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15880a1860 con 0x557f397f85d0 2024-03-30T02:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.263+0000 7f15b2afab80 1 -- 172.21.15.67:0/6117644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f85d0 msgr2=0x557f397fb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.263+0000 7f15b2afab80 1 --2- 172.21.15.67:0/6117644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f85d0 0x557f397fb610 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f158800bcf0 tx=0x7f1588010980 comp rx=0 tx=0).stop 2024-03-30T02:41:22.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.263+0000 7f15b2afab80 1 -- 172.21.15.67:0/6117644 shutdown_connections 2024-03-30T02:41:22.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.264+0000 7f15b2afab80 1 --2- 172.21.15.67:0/6117644 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f85d0 0x557f397fb610 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.265+0000 7f15b2afab80 1 --2- 172.21.15.67:0/6117644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39801470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.265+0000 7f15b2afab80 1 --2- 172.21.15.67:0/6117644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f8ef0 0x557f3981ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.265+0000 7f15b2afab80 1 -- 172.21.15.67:0/6117644 >> 172.21.15.67:0/6117644 conn(0x557f397586a0 msgr2=0x557f397ed5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.265+0000 7f15b2afab80 1 -- 172.21.15.67:0/6117644 shutdown_connections 2024-03-30T02:41:22.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.265+0000 7f15b2afab80 1 -- 172.21.15.67:0/6117644 wait complete. 2024-03-30T02:41:22.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.266+0000 7f15b2afab80 1 -- start start 2024-03-30T02:41:22.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.266+0000 7f15b2afab80 1 Processor -- start 2024-03-30T02:41:22.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.267+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39819bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.267+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f3981a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.269+0000 7f15b2afab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39871800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.269+0000 7f15b2afab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f39871d40 con 0x557f397f8ef0 2024-03-30T02:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.269+0000 7f15b2afab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f39871e80 con 0x557f39803df0 2024-03-30T02:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.269+0000 7f15b2afab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f39872040 con 0x557f397f85d0 2024-03-30T02:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.269+0000 7f15b2afab80 10 client.0 _ll_register_callbacks cb 0x557f396f0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.269+0000 7f159d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39871800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.269+0000 7f159d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39871800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60930/0 (socket says 172.21.15.67:60930) 2024-03-30T02:41:22.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.269+0000 7f159d000700 1 -- 172.21.15.67:0/1351873057 learned_addr learned my addr 172.21.15.67:0/1351873057 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.270+0000 7f1597e00700 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39819bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.270+0000 7f159d000700 1 -- 172.21.15.67:0/1351873057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 msgr2=0x557f39819bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.271+0000 7f159d000700 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39819bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.271+0000 7f159d000700 1 -- 172.21.15.67:0/1351873057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 msgr2=0x557f3981a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.271+0000 7f159d000700 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f3981a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.271+0000 7f159d000700 1 -- 172.21.15.67:0/1351873057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f395cc3b0 con 0x557f39803df0 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.271+0000 7f159d000700 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39871800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15900121f0 tx=0x7f159001ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d23cd673e75b5f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.271+0000 7f1582000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.271+0000 7f1582000700 1 -- 172.21.15.67:0/1351873057 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590003e10 con 0x557f39803df0 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.271+0000 7f1582000700 1 -- 172.21.15.67:0/1351873057 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1590013780 con 0x557f39803df0 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.272+0000 7f15b2afab80 1 -- 172.21.15.67:0/1351873057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1590014170 con 0x557f39803df0 2024-03-30T02:41:22.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.272+0000 7f1597e00700 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39819bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:41:22.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.279+0000 7f1582000700 1 -- 172.21.15.67:0/1351873057 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1590013940 con 0x557f39803df0 2024-03-30T02:41:22.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.279+0000 7f15b2afab80 10 client.26675 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:22.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.279+0000 7f15b2afab80 1 -- 172.21.15.67:0/1351873057 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557f39578b60 con 0x557f39803df0 2024-03-30T02:41:22.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.279+0000 7f1582000700 1 -- 172.21.15.67:0/1351873057 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1590014170 con 0x557f39803df0 2024-03-30T02:41:22.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.280+0000 7f1582000700 1 -- 172.21.15.67:0/1351873057 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f1590023020 con 0x557f39803df0 2024-03-30T02:41:22.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.280+0000 7f15b2afab80 10 client.26675 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:22.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.280+0000 7f15b2afab80 10 client.26675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:41:22.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.280+0000 7f15b2afab80 20 client.26675 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:22.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.280+0000 7f15b2afab80 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557f39879540 0x557f3987b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.280+0000 7f15b2afab80 4 client.26675 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:41:22.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.280+0000 7f15b2afab80 1 -- 172.21.15.67:0/1351873057 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f39599cf0 con 0x557f39879540 2024-03-30T02:41:22.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.281+0000 7f1597e00700 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557f39879540 0x557f3987b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.284+0000 7f1582000700 10 client.26675 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:41:22.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.284+0000 7f1597e00700 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557f39879540 0x557f3987b960 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed885fff728f4424 server_cookie=a69fcbccef7d885d in_seq=0 out_seq=0 2024-03-30T02:41:22.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.287+0000 7f1582000700 1 -- 172.21.15.67:0/1351873057 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f39599cf0 con 0x557f39879540 2024-03-30T02:41:22.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.287+0000 7f1582000700 10 client.26675 handle_command_reply: tid=0 2024-03-30T02:41:22.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.287+0000 7f15b2afab80 1 client.26675 shutdown 2024-03-30T02:41:22.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.287+0000 7f15b2afab80 1 -- 172.21.15.67:0/1351873057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 msgr2=0x557f39871800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.287+0000 7f15b2afab80 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39871800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f15900121f0 tx=0x7f159001ce40 comp rx=0 tx=0).stop 2024-03-30T02:41:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.290+0000 7f15b2afab80 1 -- 172.21.15.67:0/1351873057 shutdown_connections 2024-03-30T02:41:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.290+0000 7f15b2afab80 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f39803df0 0x557f39871800 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.290+0000 7f15b2afab80 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f397f85d0 0x557f39819bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.290+0000 7f15b2afab80 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x557f39879540 0x557f3987b960 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.290+0000 7f15b2afab80 1 --2- 172.21.15.67:0/1351873057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f397f8ef0 0x557f3981a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.290+0000 7f1582000700 0 client.26675 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:41:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.290+0000 7f15b2afab80 1 -- 172.21.15.67:0/1351873057 >> 172.21.15.67:0/1351873057 conn(0x557f397586a0 msgr2=0x557f397fbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.291+0000 7f15b2afab80 1 -- 172.21.15.67:0/1351873057 shutdown_connections 2024-03-30T02:41:22.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.293+0000 7f15b2afab80 1 -- 172.21.15.67:0/1351873057 wait complete. 2024-03-30T02:41:22.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.293+0000 7f15b2afab80 20 client.26675 trim_cache size 0 max 16384 2024-03-30T02:41:22.294 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 -- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f157c0a1390 msgr2=0x7f157c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f157c0a1390 0x7f157c0a3850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f159806ba30 tx=0x7f1584009310 comp rx=0 tx=0).stop 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 -- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598103160 msgr2=0x7f159806a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598103160 0x7f159806a9b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f158800b530 tx=0x7f158800b8f0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 -- 172.21.15.67:0/1221716937 shutdown_connections 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f157c0a1390 0x7f157c0a3850 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1598103160 0x7f159806a9b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159806fdd0 0x7f1598070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 --2- 172.21.15.67:0/1221716937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1598108280 0x7f159806aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 -- 172.21.15.67:0/1221716937 >> 172.21.15.67:0/1221716937 conn(0x7f15980fe9c0 msgr2=0x7f1598106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 -- 172.21.15.67:0/1221716937 shutdown_connections 2024-03-30T02:41:22.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.297+0000 7f159f000700 1 -- 172.21.15.67:0/1221716937 wait complete. 2024-03-30T02:41:22.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:41:22.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:22 smithi067 ceph-mon[37808]: pgmap v132: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 22 MiB/s wr, 2.21k op/s 2024-03-30T02:41:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:22 smithi170 ceph-mon[78798]: pgmap v132: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 22 MiB/s wr, 2.21k op/s 2024-03-30T02:41:22.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.748+0000 7f603ec00700 1 -- 172.21.15.67:0/1487747506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6038067e30 msgr2=0x7f60380682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.748+0000 7f603ec00700 1 --2- 172.21.15.67:0/1487747506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6038067e30 0x7f60380682b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f60280099c0 tx=0x7f6028009cd0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.748+0000 7f603ec00700 1 -- 172.21.15.67:0/1487747506 shutdown_connections 2024-03-30T02:41:22.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.748+0000 7f603ec00700 1 --2- 172.21.15.67:0/1487747506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6038105ce0 0x7f60381060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.748+0000 7f603ec00700 1 --2- 172.21.15.67:0/1487747506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60380687f0 0x7f603810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:22 smithi082 ceph-mon[79176]: pgmap v132: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 22 MiB/s wr, 2.21k op/s 2024-03-30T02:41:22.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.748+0000 7f603ec00700 1 --2- 172.21.15.67:0/1487747506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6038067e30 0x7f60380682b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.748+0000 7f603ec00700 1 -- 172.21.15.67:0/1487747506 >> 172.21.15.67:0/1487747506 conn(0x7f6038076d40 msgr2=0x7f6038077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.750+0000 7f603ec00700 1 -- 172.21.15.67:0/1487747506 shutdown_connections 2024-03-30T02:41:22.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.750+0000 7f603ec00700 1 -- 172.21.15.67:0/1487747506 wait complete. 2024-03-30T02:41:22.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.752+0000 7f603ec00700 1 Processor -- start 2024-03-30T02:41:22.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.752+0000 7f603ec00700 1 -- start start 2024-03-30T02:41:22.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.753+0000 7f603ec00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6038067e30 0x7f60381a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.753+0000 7f603ec00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60380687f0 0x7f60381a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.753+0000 7f603ec00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6038105ce0 0x7f603819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.753+0000 7f603ec00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819de40 con 0x7f60380687f0 2024-03-30T02:41:22.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.753+0000 7f603ec00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819dfb0 con 0x7f6038105ce0 2024-03-30T02:41:22.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.753+0000 7f603ec00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603819e170 con 0x7f6038067e30 2024-03-30T02:41:22.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.754+0000 7f6037e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6038067e30 0x7f60381a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.754+0000 7f6037e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6038067e30 0x7f60381a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36582/0 (socket says 172.21.15.67:36582) 2024-03-30T02:41:22.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.754+0000 7f6037e00700 1 -- 172.21.15.67:0/2752874959 learned_addr learned my addr 172.21.15.67:0/2752874959 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.754+0000 7f603cc00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6038105ce0 0x7f603819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.754+0000 7f6037e00700 1 -- 172.21.15.67:0/2752874959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6038105ce0 msgr2=0x7f603819d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.754+0000 7f6037e00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6038105ce0 0x7f603819d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.754+0000 7f6037e00700 1 -- 172.21.15.67:0/2752874959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60380687f0 msgr2=0x7f60381a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.754+0000 7f6037e00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60380687f0 0x7f60381a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.754+0000 7f6037e00700 1 -- 172.21.15.67:0/2752874959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280095f0 con 0x7f6038067e30 2024-03-30T02:41:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.755+0000 7f6037e00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6038067e30 0x7f60381a36a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f602401c540 tx=0x7f602401c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.755+0000 7f6034c00700 1 -- 172.21.15.67:0/2752874959 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024022640 con 0x7f6038067e30 2024-03-30T02:41:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.755+0000 7f6034c00700 1 -- 172.21.15.67:0/2752874959 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6024022c60 con 0x7f6038067e30 2024-03-30T02:41:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.755+0000 7f6034c00700 1 -- 172.21.15.67:0/2752874959 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024021cf0 con 0x7f6038067e30 2024-03-30T02:41:22.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.756+0000 7f603ec00700 1 -- 172.21.15.67:0/2752874959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603819e450 con 0x7f6038067e30 2024-03-30T02:41:22.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.757+0000 7f603ec00700 1 -- 172.21.15.67:0/2752874959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603806a9b0 con 0x7f6038067e30 2024-03-30T02:41:22.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.766+0000 7f6034c00700 1 -- 172.21.15.67:0/2752874959 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6024021390 con 0x7f6038067e30 2024-03-30T02:41:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.769+0000 7f6034c00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f601c0a1340 0x7f601c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.769+0000 7f6034c00700 1 -- 172.21.15.67:0/2752874959 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f60240dd100 con 0x7f6038067e30 2024-03-30T02:41:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.771+0000 7f6037400700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f601c0a1340 0x7f601c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.773+0000 7f6037400700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f601c0a1340 0x7f601c0a3800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f60280098e0 tx=0x7f6028000bc0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.773+0000 7f6052659b80 1 Processor -- start 2024-03-30T02:41:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.774+0000 7f6052659b80 1 -- start start 2024-03-30T02:41:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f6052659b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 0x55b1165ec9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f6052659b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ecef0 0x55b1165f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f6052659b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f6052659b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b116545990 con 0x55b1165ec5d0 2024-03-30T02:41:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f6052659b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b116543350 con 0x55b1165ecef0 2024-03-30T02:41:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f6052659b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11653ec40 con 0x55b1165f7df0 2024-03-30T02:41:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f603cc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f603cc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36586/0 (socket says 172.21.15.67:36586) 2024-03-30T02:41:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f603cc00700 1 -- 172.21.15.67:0/3103576507 learned_addr learned my addr 172.21.15.67:0/3103576507 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f603cc00700 1 -- 172.21.15.67:0/3103576507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ecef0 msgr2=0x55b1165f78b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:41:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f603cc00700 1 --2- 172.21.15.67:0/3103576507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ecef0 0x55b1165f78b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f603cc00700 1 -- 172.21.15.67:0/3103576507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 msgr2=0x55b1165ec9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:41:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f603cc00700 1 --2- 172.21.15.67:0/3103576507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 0x55b1165ec9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.777+0000 7f603cc00700 1 -- 172.21.15.67:0/3103576507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1163c03b0 con 0x55b1165f7df0 2024-03-30T02:41:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.778+0000 7f603cc00700 1 --2- 172.21.15.67:0/3103576507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fa1e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6030009f10 tx=0x7f603000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a8cd980204ab5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.778+0000 7f602d200700 1 -- 172.21.15.67:0/3103576507 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030011070 con 0x55b1165f7df0 2024-03-30T02:41:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.779+0000 7f602d200700 1 -- 172.21.15.67:0/3103576507 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60300158d0 con 0x55b1165f7df0 2024-03-30T02:41:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.779+0000 7f602d200700 1 -- 172.21.15.67:0/3103576507 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000a950 con 0x55b1165f7df0 2024-03-30T02:41:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.780+0000 7f6052659b80 1 -- 172.21.15.67:0/3103576507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 msgr2=0x55b1165fa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.780+0000 7f6052659b80 1 --2- 172.21.15.67:0/3103576507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fa1e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6030009f10 tx=0x7f603000c6e0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.780+0000 7f6052659b80 1 -- 172.21.15.67:0/3103576507 shutdown_connections 2024-03-30T02:41:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.780+0000 7f6052659b80 1 --2- 172.21.15.67:0/3103576507 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 0x55b1165ec9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.780+0000 7f6052659b80 1 --2- 172.21.15.67:0/3103576507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fa1e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.780+0000 7f6052659b80 1 --2- 172.21.15.67:0/3103576507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ecef0 0x55b1165f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.780+0000 7f6052659b80 1 -- 172.21.15.67:0/3103576507 >> 172.21.15.67:0/3103576507 conn(0x55b11654c6a0 msgr2=0x55b1165e8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.781+0000 7f6052659b80 1 -- 172.21.15.67:0/3103576507 shutdown_connections 2024-03-30T02:41:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.781+0000 7f6052659b80 1 -- 172.21.15.67:0/3103576507 wait complete. 2024-03-30T02:41:22.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.783+0000 7f6052659b80 1 -- start start 2024-03-30T02:41:22.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.783+0000 7f6052659b80 1 Processor -- start 2024-03-30T02:41:22.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6052659b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 0x55b1165f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6052659b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ecef0 0x55b1165f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6052659b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6052659b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1164dad80 con 0x55b1165ec5d0 2024-03-30T02:41:22.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6052659b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1164daa20 con 0x55b1165f7df0 2024-03-30T02:41:22.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6052659b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1163c26a0 con 0x55b1165ecef0 2024-03-30T02:41:22.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6052659b80 10 client.0 _ll_register_callbacks cb 0x55b1164e4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:22.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6037400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ecef0 0x55b1165f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6037400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ecef0 0x55b1165f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36596/0 (socket says 172.21.15.67:36596) 2024-03-30T02:41:22.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f603cc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.785+0000 7f6037400700 1 -- 172.21.15.67:0/2040530986 learned_addr learned my addr 172.21.15.67:0/2040530986 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.786+0000 7f6037e00700 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 0x55b1165f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.786+0000 7f6037400700 1 -- 172.21.15.67:0/2040530986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 msgr2=0x55b1165f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.786+0000 7f6037400700 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.786+0000 7f6037400700 1 -- 172.21.15.67:0/2040530986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 msgr2=0x55b1165f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.786+0000 7f6037400700 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 0x55b1165f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.786+0000 7f6037400700 1 -- 172.21.15.67:0/2040530986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6030009bc0 con 0x55b1165ecef0 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.786+0000 7f6037e00700 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 0x55b1165f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.786+0000 7f603cc00700 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.786+0000 7f6037400700 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ecef0 0x55b1165f4b80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6028003890 tx=0x7f6028009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7064d5d2969241e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.787+0000 7f6021600700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:41:22.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.788+0000 7f6021600700 1 -- 172.21.15.67:0/2040530986 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602801d070 con 0x55b1165ecef0 2024-03-30T02:41:22.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.788+0000 7f6021600700 1 -- 172.21.15.67:0/2040530986 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f60280056c0 con 0x55b1165ecef0 2024-03-30T02:41:22.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.788+0000 7f6052659b80 1 -- 172.21.15.67:0/2040530986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1164ff380 con 0x55b1165ecef0 2024-03-30T02:41:22.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.789+0000 7f6021600700 1 -- 172.21.15.67:0/2040530986 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6028003ee0 con 0x55b1165ecef0 2024-03-30T02:41:22.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.789+0000 7f6052659b80 10 client.26635 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:22.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.789+0000 7f6052659b80 1 -- 172.21.15.67:0/2040530986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b11636cb60 con 0x55b1165ecef0 2024-03-30T02:41:22.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.789+0000 7f6021600700 1 -- 172.21.15.67:0/2040530986 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b1164ff380 con 0x55b1165ecef0 2024-03-30T02:41:22.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.789+0000 7f6021600700 1 -- 172.21.15.67:0/2040530986 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f602802f440 con 0x55b1165ecef0 2024-03-30T02:41:22.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.789+0000 7f6052659b80 10 client.26635 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:22.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.789+0000 7f6052659b80 10 client.26635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:22.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.789+0000 7f6052659b80 20 client.26635 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:22.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.790+0000 7f6052659b80 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55b116607b40 0x55b116609f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.790+0000 7f6052659b80 4 client.26635 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:41:22.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.790+0000 7f6052659b80 1 -- 172.21.15.67:0/2040530986 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b116537700 con 0x55b116607b40 2024-03-30T02:41:22.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.791+0000 7f6037e00700 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55b116607b40 0x55b116609f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.792+0000 7f6021600700 10 client.26635 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:22.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.793+0000 7f6037e00700 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55b116607b40 0x55b116609f60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a3bd36f727ef25b server_cookie=f8845a849097c8da in_seq=0 out_seq=0 2024-03-30T02:41:22.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.794+0000 7f6021600700 1 -- 172.21.15.67:0/2040530986 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b116537700 con 0x55b116607b40 2024-03-30T02:41:22.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.794+0000 7f6021600700 10 client.26635 handle_command_reply: tid=0 2024-03-30T02:41:22.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.794+0000 7f6052659b80 1 client.26635 shutdown 2024-03-30T02:41:22.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.799+0000 7f6052659b80 1 -- 172.21.15.67:0/2040530986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ecef0 msgr2=0x55b1165f4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.799+0000 7f6052659b80 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ecef0 0x55b1165f4b80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6028003890 tx=0x7f6028009310 comp rx=0 tx=0).stop 2024-03-30T02:41:22.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.799+0000 7f6052659b80 1 -- 172.21.15.67:0/2040530986 shutdown_connections 2024-03-30T02:41:22.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.799+0000 7f6052659b80 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55b116607b40 0x55b116609f60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.799+0000 7f6052659b80 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ec5d0 0x55b1165f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.799+0000 7f6052659b80 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.799+0000 7f6052659b80 1 --2- 172.21.15.67:0/2040530986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ecef0 0x55b1165f4b80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.799+0000 7f6052659b80 1 -- 172.21.15.67:0/2040530986 >> 172.21.15.67:0/2040530986 conn(0x55b11654c6a0 msgr2=0x55b1165ee6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.799+0000 7f6021600700 0 client.26635 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:22.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.801+0000 7f6052659b80 1 -- 172.21.15.67:0/2040530986 shutdown_connections 2024-03-30T02:41:22.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.801+0000 7f6052659b80 1 -- 172.21.15.67:0/2040530986 wait complete. 2024-03-30T02:41:22.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.802+0000 7f6052659b80 20 client.26635 trim_cache size 0 max 16384 2024-03-30T02:41:22.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.843+0000 7f6052659b80 1 Processor -- start 2024-03-30T02:41:22.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.845+0000 7f6052659b80 1 -- start start 2024-03-30T02:41:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.847+0000 7f6052659b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ec5d0 0x55b1165f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.847+0000 7f6052659b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 0x55b1165fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.847+0000 7f6052659b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.847+0000 7f6052659b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b116605170 con 0x55b1165ecef0 2024-03-30T02:41:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.847+0000 7f6052659b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1165ee140 con 0x55b1165f7df0 2024-03-30T02:41:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.847+0000 7f6052659b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1165ed850 con 0x55b1165ec5d0 2024-03-30T02:41:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.848+0000 7f6037e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ec5d0 0x55b1165f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.848+0000 7f6037e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ec5d0 0x55b1165f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36606/0 (socket says 172.21.15.67:36606) 2024-03-30T02:41:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.848+0000 7f6037e00700 1 -- 172.21.15.67:0/1319130910 learned_addr learned my addr 172.21.15.67:0/1319130910 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.848+0000 7f603cc00700 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.848+0000 7f603cc00700 1 -- 172.21.15.67:0/1319130910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ec5d0 msgr2=0x55b1165f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.848+0000 7f603cc00700 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ec5d0 0x55b1165f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.848+0000 7f6037400700 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 0x55b1165fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.849+0000 7f603cc00700 1 -- 172.21.15.67:0/1319130910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 msgr2=0x55b1165fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.849+0000 7f603cc00700 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 0x55b1165fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.849+0000 7f603cc00700 1 -- 172.21.15.67:0/1319130910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b116504ea0 con 0x55b1165f7df0 2024-03-30T02:41:22.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.849+0000 7f6037400700 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 0x55b1165fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:41:22.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.850+0000 7f603cc00700 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165e1f70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f603000cae0 tx=0x7f6030015840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c115bf6df504a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.851+0000 7f602dc00700 1 -- 172.21.15.67:0/1319130910 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030011040 con 0x55b1165f7df0 2024-03-30T02:41:22.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.851+0000 7f602dc00700 1 -- 172.21.15.67:0/1319130910 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6030012040 con 0x55b1165f7df0 2024-03-30T02:41:22.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.851+0000 7f602dc00700 1 -- 172.21.15.67:0/1319130910 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000ab40 con 0x55b1165f7df0 2024-03-30T02:41:22.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.853+0000 7f6052659b80 1 -- 172.21.15.67:0/1319130910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 msgr2=0x55b1165e1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.853+0000 7f6052659b80 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165e1f70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f603000cae0 tx=0x7f6030015840 comp rx=0 tx=0).stop 2024-03-30T02:41:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.854+0000 7f6052659b80 1 -- 172.21.15.67:0/1319130910 shutdown_connections 2024-03-30T02:41:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.854+0000 7f6052659b80 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165f7df0 0x55b1165e1f70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.854+0000 7f6052659b80 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165ec5d0 0x55b1165f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.854+0000 7f6052659b80 1 --2- 172.21.15.67:0/1319130910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 0x55b1165fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.854+0000 7f6052659b80 1 -- 172.21.15.67:0/1319130910 >> 172.21.15.67:0/1319130910 conn(0x55b11654c6a0 msgr2=0x55b1165e15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.854+0000 7f6052659b80 1 -- 172.21.15.67:0/1319130910 shutdown_connections 2024-03-30T02:41:22.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.854+0000 7f6052659b80 1 -- 172.21.15.67:0/1319130910 wait complete. 2024-03-30T02:41:22.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.855+0000 7f6052659b80 1 -- start start 2024-03-30T02:41:22.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.855+0000 7f6052659b80 1 Processor -- start 2024-03-30T02:41:22.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.857+0000 7f6052659b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ec5d0 0x55b1165fe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.857+0000 7f6052659b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 0x55b1165fe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.857+0000 7f6052659b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.857+0000 7f6037e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ec5d0 0x55b1165fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.857+0000 7f6037e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ec5d0 0x55b1165fe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:60984/0 (socket says 172.21.15.67:60984) 2024-03-30T02:41:22.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.858+0000 7f603cc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.858+0000 7f603cc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36612/0 (socket says 172.21.15.67:36612) 2024-03-30T02:41:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.858+0000 7f6052659b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1165fd070 con 0x55b1165ecef0 2024-03-30T02:41:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.858+0000 7f6052659b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1165fd1e0 con 0x55b1165ec5d0 2024-03-30T02:41:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.858+0000 7f6052659b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1165fd3a0 con 0x55b1165f7df0 2024-03-30T02:41:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.858+0000 7f6052659b80 10 client.0 _ll_register_callbacks cb 0x55b1164e4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.858+0000 7f6037e00700 1 -- 172.21.15.67:0/2005529717 learned_addr learned my addr 172.21.15.67:0/2005529717 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.859+0000 7f603cc00700 1 -- 172.21.15.67:0/2005529717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ec5d0 msgr2=0x55b1165fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.859+0000 7f603cc00700 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ec5d0 0x55b1165fe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.859+0000 7f603cc00700 1 -- 172.21.15.67:0/2005529717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 msgr2=0x55b1165fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.859+0000 7f603cc00700 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 0x55b1165fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.859+0000 7f603cc00700 1 -- 172.21.15.67:0/2005529717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1163c03b0 con 0x55b1165f7df0 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.859+0000 7f6037e00700 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ec5d0 0x55b1165fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.859+0000 7f603cc00700 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fca50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6030000f80 tx=0x7f6030015d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3db945d7a6814eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.860+0000 7f6021600700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:41:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.860+0000 7f6021600700 1 -- 172.21.15.67:0/2005529717 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603002f070 con 0x55b1165f7df0 2024-03-30T02:41:22.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.861+0000 7f6021600700 1 -- 172.21.15.67:0/2005529717 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f603000a4c0 con 0x55b1165f7df0 2024-03-30T02:41:22.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.861+0000 7f6052659b80 1 -- 172.21.15.67:0/2005529717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60300158d0 con 0x55b1165f7df0 2024-03-30T02:41:22.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.861+0000 7f6021600700 1 -- 172.21.15.67:0/2005529717 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6030011040 con 0x55b1165f7df0 2024-03-30T02:41:22.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.861+0000 7f6052659b80 10 client.26641 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:22.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.862+0000 7f6052659b80 1 -- 172.21.15.67:0/2005529717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b1163c4cd0 con 0x55b1165f7df0 2024-03-30T02:41:22.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.862+0000 7f6021600700 1 -- 172.21.15.67:0/2005529717 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60300158d0 con 0x55b1165f7df0 2024-03-30T02:41:22.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.865+0000 7f6021600700 1 -- 172.21.15.67:0/2005529717 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f6030032030 con 0x55b1165f7df0 2024-03-30T02:41:22.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.866+0000 7f6052659b80 10 client.26641 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:22.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.866+0000 7f6052659b80 10 client.26641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:22.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.866+0000 7f6052659b80 20 client.26641 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:22.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.866+0000 7f6052659b80 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55b11665c4d0 0x55b11665e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:22.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.866+0000 7f6052659b80 4 client.26641 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:41:22.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.866+0000 7f6052659b80 1 -- 172.21.15.67:0/2005529717 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b11638dcf0 con 0x55b11665c4d0 2024-03-30T02:41:22.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.869+0000 7f6037e00700 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55b11665c4d0 0x55b11665e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.870+0000 7f6037e00700 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55b11665c4d0 0x55b11665e8c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=783be708688be588 server_cookie=897357af41383e1e in_seq=0 out_seq=0 2024-03-30T02:41:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.870+0000 7f6021600700 10 client.26641 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.870+0000 7f6021600700 1 -- 172.21.15.67:0/2005529717 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b11638dcf0 con 0x55b11665c4d0 2024-03-30T02:41:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.870+0000 7f6021600700 10 client.26641 handle_command_reply: tid=0 2024-03-30T02:41:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.870+0000 7f6052659b80 1 client.26641 shutdown 2024-03-30T02:41:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.871+0000 7f6052659b80 1 -- 172.21.15.67:0/2005529717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 msgr2=0x55b1165fca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.871+0000 7f6052659b80 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fca50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6030000f80 tx=0x7f6030015d20 comp rx=0 tx=0).stop 2024-03-30T02:41:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.872+0000 7f6052659b80 1 -- 172.21.15.67:0/2005529717 shutdown_connections 2024-03-30T02:41:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.872+0000 7f6052659b80 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b1165f7df0 0x55b1165fca50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.872+0000 7f6052659b80 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b1165ec5d0 0x55b1165fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.872+0000 7f6052659b80 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55b11665c4d0 0x55b11665e8c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.872+0000 7f6052659b80 1 --2- 172.21.15.67:0/2005529717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b1165ecef0 0x55b1165fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.872+0000 7f6052659b80 1 -- 172.21.15.67:0/2005529717 >> 172.21.15.67:0/2005529717 conn(0x55b11654c6a0 msgr2=0x55b1165e1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.872+0000 7f6021600700 0 client.26641 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:22.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.872+0000 7f6052659b80 1 -- 172.21.15.67:0/2005529717 shutdown_connections 2024-03-30T02:41:22.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.872+0000 7f6052659b80 1 -- 172.21.15.67:0/2005529717 wait complete. 2024-03-30T02:41:22.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.873+0000 7f6052659b80 20 client.26641 trim_cache size 0 max 16384 2024-03-30T02:41:22.873 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:41:22.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.876+0000 7f603ec00700 1 -- 172.21.15.67:0/2752874959 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f601c0a1340 msgr2=0x7f601c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.876+0000 7f603ec00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f601c0a1340 0x7f601c0a3800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f60280098e0 tx=0x7f6028000bc0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.876+0000 7f603ec00700 1 -- 172.21.15.67:0/2752874959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6038067e30 msgr2=0x7f60381a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.876+0000 7f603ec00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6038067e30 0x7f60381a36a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f602401c540 tx=0x7f602401c900 comp rx=0 tx=0).stop 2024-03-30T02:41:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.876+0000 7f603ec00700 1 -- 172.21.15.67:0/2752874959 shutdown_connections 2024-03-30T02:41:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.876+0000 7f603ec00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f601c0a1340 0x7f601c0a3800 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.877+0000 7f603ec00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6038105ce0 0x7f603819d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.877+0000 7f603ec00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6038067e30 0x7f60381a36a0 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f602401c540 tx=0x7f602401c900 comp rx=0 tx=0).stop 2024-03-30T02:41:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.877+0000 7f603ec00700 1 --2- 172.21.15.67:0/2752874959 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60380687f0 0x7f60381a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.877+0000 7f603ec00700 1 -- 172.21.15.67:0/2752874959 >> 172.21.15.67:0/2752874959 conn(0x7f6038076d40 msgr2=0x7f60380fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.877+0000 7f603ec00700 1 -- 172.21.15.67:0/2752874959 shutdown_connections 2024-03-30T02:41:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:22.877+0000 7f603ec00700 1 -- 172.21.15.67:0/2752874959 wait complete. 2024-03-30T02:41:23.921 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:41:23.921 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:41:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.134+0000 7fecf6600700 1 -- 172.21.15.67:0/1044541805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0100f60 msgr2=0x7fecf0101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.134+0000 7fecf6600700 1 --2- 172.21.15.67:0/1044541805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0100f60 0x7fecf0101340 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fece0009990 tx=0x7fece0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.134+0000 7fecf6600700 1 -- 172.21.15.67:0/1044541805 shutdown_connections 2024-03-30T02:41:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.134+0000 7fecf6600700 1 --2- 172.21.15.67:0/1044541805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf0106080 0x7fecf010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.134+0000 7fecf6600700 1 --2- 172.21.15.67:0/1044541805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0100f60 0x7fecf0101340 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.134+0000 7fecf6600700 1 --2- 172.21.15.67:0/1044541805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0101910 0x7fecf0105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.134+0000 7fecf6600700 1 -- 172.21.15.67:0/1044541805 >> 172.21.15.67:0/1044541805 conn(0x7fecf00fc7c0 msgr2=0x7fecf00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.134+0000 7fecf6600700 1 -- 172.21.15.67:0/1044541805 shutdown_connections 2024-03-30T02:41:24.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.134+0000 7fecf6600700 1 -- 172.21.15.67:0/1044541805 wait complete. 2024-03-30T02:41:24.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.135+0000 7fecf6600700 1 Processor -- start 2024-03-30T02:41:24.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.135+0000 7fecf6600700 1 -- start start 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.135+0000 7fecf6600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0100f60 0x7fecf0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecf6600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0106080 0x7fecf0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecf6600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf019dd10 0x7fecf019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecf6600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf019e790 con 0x7fecf019dd10 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecf6600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf019e900 con 0x7fecf0100f60 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecf6600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf019eac0 con 0x7fecf0106080 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecefe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf019dd10 0x7fecf019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7feceea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0106080 0x7fecf0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecefe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf019dd10 0x7fecf019e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53110/0 (socket says 172.21.15.67:53110) 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7feceea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0106080 0x7fecf0199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36628/0 (socket says 172.21.15.67:36628) 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecefe00700 1 -- 172.21.15.67:0/1769460640 learned_addr learned my addr 172.21.15.67:0/1769460640 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecef400700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0100f60 0x7fecf0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecefe00700 1 -- 172.21.15.67:0/1769460640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0106080 msgr2=0x7fecf0199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecefe00700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0106080 0x7fecf0199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecefe00700 1 -- 172.21.15.67:0/1769460640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0100f60 msgr2=0x7fecf0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecefe00700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0100f60 0x7fecf0199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecefe00700 1 -- 172.21.15.67:0/1769460640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece00095f0 con 0x7fecf019dd10 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.136+0000 7fecef400700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0100f60 0x7fecf0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.137+0000 7fecefe00700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf019dd10 0x7fecf019e190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fece800ef30 tx=0x7fece800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.137+0000 7fece7e00700 1 -- 172.21.15.67:0/1769460640 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8018070 con 0x7fecf019dd10 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.137+0000 7fece7e00700 1 -- 172.21.15.67:0/1769460640 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece800f040 con 0x7fecf019dd10 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.137+0000 7fecf6600700 1 -- 172.21.15.67:0/1769460640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf01a5fc0 con 0x7fecf019dd10 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.137+0000 7fece7e00700 1 -- 172.21.15.67:0/1769460640 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece800cd00 con 0x7fecf019dd10 2024-03-30T02:41:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.137+0000 7fecf6600700 1 -- 172.21.15.67:0/1769460640 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf01a64f0 con 0x7fecf019dd10 2024-03-30T02:41:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.141+0000 7fece7e00700 1 -- 172.21.15.67:0/1769460640 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fece8003a00 con 0x7fecf019dd10 2024-03-30T02:41:24.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.142+0000 7fed0a091b80 1 Processor -- start 2024-03-30T02:41:24.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fed0a091b80 1 -- start start 2024-03-30T02:41:24.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fece7e00700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fecd40a1310 0x7fecd40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fece7e00700 1 -- 172.21.15.67:0/1769460640 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fece80ccac0 con 0x7fecf019dd10 2024-03-30T02:41:24.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fed0a091b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a591ea10 con 0x5589a59d0e70 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fed0a091b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a591c3d0 con 0x5589a59c5f70 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fed0a091b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a5917cc0 con 0x5589a59c5650 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7feceea00700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fecd40a1310 0x7fecd40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7feceea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fecef400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fecef400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36630/0 (socket says 172.21.15.67:36630) 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fecef400700 1 -- 172.21.15.67:0/1093878504 learned_addr learned my addr 172.21.15.67:0/1093878504 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.143+0000 7fecefe00700 1 --2- 172.21.15.67:0/1093878504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7fecef400700 1 -- 172.21.15.67:0/1093878504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 msgr2=0x5589a59d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7fecef400700 1 --2- 172.21.15.67:0/1093878504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7fecef400700 1 -- 172.21.15.67:0/1093878504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 msgr2=0x5589a59d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7fecef400700 1 --2- 172.21.15.67:0/1093878504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7fecef400700 1 -- 172.21.15.67:0/1093878504 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589a5799430 con 0x5589a59c5650 2024-03-30T02:41:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7fecef400700 1 --2- 172.21.15.67:0/1093878504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59c5a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fece000b6c0 tx=0x7fece0005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3cf5f9857f1c72 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7feceea00700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fecd40a1310 0x7fecd40a37d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00b6c0 tx=0x7fecdc00bbd0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7fece4c00700 1 -- 172.21.15.67:0/1093878504 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece001d070 con 0x5589a59c5650 2024-03-30T02:41:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7fece4c00700 1 -- 172.21.15.67:0/1093878504 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece001e040 con 0x5589a59c5650 2024-03-30T02:41:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.144+0000 7fece4c00700 1 -- 172.21.15.67:0/1093878504 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0004be0 con 0x5589a59c5650 2024-03-30T02:41:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.145+0000 7fed0a091b80 1 -- 172.21.15.67:0/1093878504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 msgr2=0x5589a59c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.145+0000 7fed0a091b80 1 --2- 172.21.15.67:0/1093878504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59c5a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fece000b6c0 tx=0x7fece0005f00 comp rx=0 tx=0).stop 2024-03-30T02:41:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.145+0000 7fed0a091b80 1 -- 172.21.15.67:0/1093878504 shutdown_connections 2024-03-30T02:41:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.145+0000 7fed0a091b80 1 --2- 172.21.15.67:0/1093878504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.145+0000 7fed0a091b80 1 --2- 172.21.15.67:0/1093878504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59c5a30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.145+0000 7fed0a091b80 1 --2- 172.21.15.67:0/1093878504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.145+0000 7fed0a091b80 1 -- 172.21.15.67:0/1093878504 >> 172.21.15.67:0/1093878504 conn(0x5589a5925720 msgr2=0x5589a59c1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.145+0000 7fed0a091b80 1 -- 172.21.15.67:0/1093878504 shutdown_connections 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.145+0000 7fed0a091b80 1 -- 172.21.15.67:0/1093878504 wait complete. 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.146+0000 7fed0a091b80 1 -- start start 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.146+0000 7fed0a091b80 1 Processor -- start 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.146+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.146+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.146+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.146+0000 7fed0a091b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a58b3e00 con 0x5589a59d0e70 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.146+0000 7fed0a091b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a58b3aa0 con 0x5589a59c5f70 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.146+0000 7fed0a091b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a579b720 con 0x5589a59c5650 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.146+0000 7fed0a091b80 10 client.0 _ll_register_callbacks cb 0x5589a58bd4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7feceea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7fecef400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7feceea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:32796/0 (socket says 172.21.15.67:32796) 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7feceea00700 1 -- 172.21.15.67:0/265619107 learned_addr learned my addr 172.21.15.67:0/265619107 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7fecefe00700 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7feceea00700 1 -- 172.21.15.67:0/265619107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 msgr2=0x5589a59d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7feceea00700 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7feceea00700 1 -- 172.21.15.67:0/265619107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 msgr2=0x5589a59ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7feceea00700 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7feceea00700 1 -- 172.21.15.67:0/265619107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf010e470 con 0x5589a59c5f70 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7fecef400700 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7feceea00700 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cdb20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0042f0 tx=0x7fecdc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1411b8c3deac0ae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.147+0000 7fecd9600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fecd9600700 1 -- 172.21.15.67:0/265619107 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc010e50 con 0x5589a59c5f70 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fed0a091b80 1 -- 172.21.15.67:0/265619107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589a58d8400 con 0x5589a59c5f70 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fecd9600700 1 -- 172.21.15.67:0/265619107 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fecdc003680 con 0x5589a59c5f70 2024-03-30T02:41:24.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fecd9600700 1 -- 172.21.15.67:0/265619107 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecdc003840 con 0x5589a59c5f70 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fed0a091b80 10 client.26699 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fed0a091b80 1 -- 172.21.15.67:0/265619107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5589a5745be0 con 0x5589a59c5f70 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fecd9600700 1 -- 172.21.15.67:0/265619107 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589a58d8400 con 0x5589a59c5f70 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fecd9600700 1 -- 172.21.15.67:0/265619107 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fecdc02aa20 con 0x5589a59c5f70 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fed0a091b80 10 client.26699 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.148+0000 7fed0a091b80 10 client.26699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.149+0000 7fed0a091b80 20 client.26699 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.149+0000 7fed0a091b80 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5589a59e90d0 0x5589a59eb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.149+0000 7fed0a091b80 4 client.26699 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:41:24.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.149+0000 7fed0a091b80 1 -- 172.21.15.67:0/265619107 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589a5910780 con 0x5589a59e90d0 2024-03-30T02:41:24.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.149+0000 7fecef400700 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5589a59e90d0 0x5589a59eb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.149+0000 7fecef400700 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5589a59e90d0 0x5589a59eb4f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e6d028853e45f8 server_cookie=4a26829623f5d404 in_seq=0 out_seq=0 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.149+0000 7fecd9600700 10 client.26699 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fecd9600700 1 -- 172.21.15.67:0/265619107 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5589a5910780 con 0x5589a59e90d0 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fecd9600700 10 client.26699 handle_command_reply: tid=0 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fed0a091b80 1 client.26699 shutdown 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fed0a091b80 1 -- 172.21.15.67:0/265619107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 msgr2=0x5589a59cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fed0a091b80 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cdb20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0042f0 tx=0x7fecdc0047c0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fed0a091b80 1 -- 172.21.15.67:0/265619107 shutdown_connections 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fed0a091b80 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5589a59e90d0 0x5589a59eb4f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fed0a091b80 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fecd9600700 0 client.26699 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fed0a091b80 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fed0a091b80 1 --2- 172.21.15.67:0/265619107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cdb20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.150+0000 7fed0a091b80 1 -- 172.21.15.67:0/265619107 >> 172.21.15.67:0/265619107 conn(0x5589a5925720 msgr2=0x5589a59c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.151+0000 7fed0a091b80 1 -- 172.21.15.67:0/265619107 shutdown_connections 2024-03-30T02:41:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.151+0000 7fed0a091b80 1 -- 172.21.15.67:0/265619107 wait complete. 2024-03-30T02:41:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.151+0000 7fed0a091b80 20 client.26699 trim_cache size 0 max 16384 2024-03-30T02:41:24.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.169+0000 7fed0a091b80 1 Processor -- start 2024-03-30T02:41:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.169+0000 7fed0a091b80 1 -- start start 2024-03-30T02:41:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.169+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.169+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.169+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.169+0000 7fed0a091b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a58d8400 con 0x5589a59d0e70 2024-03-30T02:41:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.169+0000 7fed0a091b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a59c8640 con 0x5589a59c5f70 2024-03-30T02:41:24.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.169+0000 7fed0a091b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a59e5c50 con 0x5589a59c5650 2024-03-30T02:41:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7feceea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7fecef400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7fecefe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7feceea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:32802/0 (socket says 172.21.15.67:32802) 2024-03-30T02:41:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7feceea00700 1 -- 172.21.15.67:0/221239828 learned_addr learned my addr 172.21.15.67:0/221239828 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7feceea00700 1 -- 172.21.15.67:0/221239828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 msgr2=0x5589a59e02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7feceea00700 1 --2- 172.21.15.67:0/221239828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59e02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7feceea00700 1 -- 172.21.15.67:0/221239828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 msgr2=0x5589a59cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7feceea00700 1 --2- 172.21.15.67:0/221239828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7feceea00700 1 -- 172.21.15.67:0/221239828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589a58ddf20 con 0x5589a59c5f70 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7feceea00700 1 --2- 172.21.15.67:0/221239828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cc410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fecdc004790 tx=0x7fecdc012750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=264cd62be3ced5a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7fece5600700 1 -- 172.21.15.67:0/221239828 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc03b070 con 0x5589a59c5f70 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7fece5600700 1 -- 172.21.15.67:0/221239828 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecdc00bea0 con 0x5589a59c5f70 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.170+0000 7fece5600700 1 -- 172.21.15.67:0/221239828 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0206d0 con 0x5589a59c5f70 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.171+0000 7fed0a091b80 1 -- 172.21.15.67:0/221239828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 msgr2=0x5589a59cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.171+0000 7fed0a091b80 1 --2- 172.21.15.67:0/221239828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cc410 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fecdc004790 tx=0x7fecdc012750 comp rx=0 tx=0).stop 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.171+0000 7fed0a091b80 1 -- 172.21.15.67:0/221239828 shutdown_connections 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.171+0000 7fed0a091b80 1 --2- 172.21.15.67:0/221239828 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59d0e70 0x5589a59cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.171+0000 7fed0a091b80 1 --2- 172.21.15.67:0/221239828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59e02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.171+0000 7fed0a091b80 1 --2- 172.21.15.67:0/221239828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59c5f70 0x5589a59cc410 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.171+0000 7fed0a091b80 1 -- 172.21.15.67:0/221239828 >> 172.21.15.67:0/221239828 conn(0x5589a5925720 msgr2=0x5589a59c8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.171+0000 7fed0a091b80 1 -- 172.21.15.67:0/221239828 shutdown_connections 2024-03-30T02:41:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.171+0000 7fed0a091b80 1 -- 172.21.15.67:0/221239828 wait complete. 2024-03-30T02:41:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.172+0000 7fed0a091b80 1 -- start start 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.172+0000 7fed0a091b80 1 Processor -- start 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.172+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.172+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59c5f70 0x5589a59df970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.172+0000 7fed0a091b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59d0e70 0x5589a5a36f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.172+0000 7fed0a091b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a5a37440 con 0x5589a59c5f70 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.172+0000 7fed0a091b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a5a375b0 con 0x5589a59d0e70 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.172+0000 7fed0a091b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589a5a37770 con 0x5589a59c5650 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.172+0000 7fed0a091b80 10 client.0 _ll_register_callbacks cb 0x5589a58bd4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecef400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecef400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59df430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36648/0 (socket says 172.21.15.67:36648) 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecef400700 1 -- 172.21.15.67:0/92619684 learned_addr learned my addr 172.21.15.67:0/92619684 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7feceea00700 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59c5f70 0x5589a59df970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecefe00700 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59d0e70 0x5589a5a36f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecef400700 1 -- 172.21.15.67:0/92619684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59d0e70 msgr2=0x5589a5a36f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecef400700 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59d0e70 0x5589a5a36f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecef400700 1 -- 172.21.15.67:0/92619684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59c5f70 msgr2=0x5589a59df970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecef400700 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59c5f70 0x5589a59df970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecef400700 1 -- 172.21.15.67:0/92619684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589a5745be0 con 0x5589a59c5650 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7feceea00700 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59c5f70 0x5589a59df970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecef400700 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59df430 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fece0005a80 tx=0x7fece0004500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7caddd1d02698984 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecd9600700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.173+0000 7fecd9600700 1 -- 172.21.15.67:0/92619684 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0032040 con 0x5589a59c5650 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fecd9600700 1 -- 172.21.15.67:0/92619684 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fece0034810 con 0x5589a59c5650 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fed0a091b80 1 -- 172.21.15.67:0/92619684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fece001e040 con 0x5589a59c5650 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fecd9600700 1 -- 172.21.15.67:0/92619684 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fece001d040 con 0x5589a59c5650 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fed0a091b80 10 client.26659 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fed0a091b80 1 -- 172.21.15.67:0/92619684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5589a579dd50 con 0x5589a59c5650 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fecd9600700 1 -- 172.21.15.67:0/92619684 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece001e040 con 0x5589a59c5650 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fecd9600700 1 -- 172.21.15.67:0/92619684 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fece0022bd0 con 0x5589a59c5650 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fed0a091b80 10 client.26659 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fed0a091b80 10 client.26659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fed0a091b80 20 client.26659 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fed0a091b80 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5589a5a3edd0 0x5589a5a411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fed0a091b80 4 client.26659 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.174+0000 7fed0a091b80 1 -- 172.21.15.67:0/92619684 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5589a5766d70 con 0x5589a5a3edd0 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.175+0000 7feceea00700 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5589a5a3edd0 0x5589a5a411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.175+0000 7feceea00700 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5589a5a3edd0 0x5589a5a411f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc787e8f63b4d9d server_cookie=b5823b981c26c94f in_seq=0 out_seq=0 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.175+0000 7fecd9600700 10 client.26659 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fecd9600700 1 -- 172.21.15.67:0/92619684 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5589a5766d70 con 0x5589a5a3edd0 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fecd9600700 10 client.26659 handle_command_reply: tid=0 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fed0a091b80 1 client.26659 shutdown 2024-03-30T02:41:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fed0a091b80 1 -- 172.21.15.67:0/92619684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 msgr2=0x5589a59df430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fed0a091b80 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59df430 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fece0005a80 tx=0x7fece0004500 comp rx=0 tx=0).stop 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fed0a091b80 1 -- 172.21.15.67:0/92619684 shutdown_connections 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fed0a091b80 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5589a59d0e70 0x5589a5a36f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fed0a091b80 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589a59c5650 0x5589a59df430 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fed0a091b80 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5589a5a3edd0 0x5589a5a411f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fed0a091b80 1 --2- 172.21.15.67:0/92619684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5589a59c5f70 0x5589a59df970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fed0a091b80 1 -- 172.21.15.67:0/92619684 >> 172.21.15.67:0/92619684 conn(0x5589a5925720 msgr2=0x5589a59e4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.176+0000 7fecd9600700 0 client.26659 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.177+0000 7fed0a091b80 1 -- 172.21.15.67:0/92619684 shutdown_connections 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.177+0000 7fed0a091b80 1 -- 172.21.15.67:0/92619684 wait complete. 2024-03-30T02:41:24.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.177+0000 7fed0a091b80 20 client.26659 trim_cache size 0 max 16384 2024-03-30T02:41:24.183 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:41:24.183 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:41:24.183 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "7a376898-7302-4db7-9b68-928808ede83e", 2024-03-30T02:41:24.183 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:41:24.183 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:41:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.177+0000 7fecf6600700 1 -- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fecd40a1310 msgr2=0x7fecd40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.177+0000 7fecf6600700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fecd40a1310 0x7fecd40a37d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00b6c0 tx=0x7fecdc00bbd0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.177+0000 7fecf6600700 1 -- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf019dd10 msgr2=0x7fecf019e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.177+0000 7fecf6600700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf019dd10 0x7fecf019e190 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fece800ef30 tx=0x7fece800c3a0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.178+0000 7fecf6600700 1 -- 172.21.15.67:0/1769460640 shutdown_connections 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.178+0000 7fecf6600700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fecd40a1310 0x7fecd40a37d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.178+0000 7fecf6600700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fecf019dd10 0x7fecf019e190 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.178+0000 7fecf6600700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf0100f60 0x7fecf0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.178+0000 7fecf6600700 1 --2- 172.21.15.67:0/1769460640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fecf0106080 0x7fecf0199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.178+0000 7fecf6600700 1 -- 172.21.15.67:0/1769460640 >> 172.21.15.67:0/1769460640 conn(0x7fecf00fc7c0 msgr2=0x7fecf00fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.178+0000 7fecf6600700 1 -- 172.21.15.67:0/1769460640 shutdown_connections 2024-03-30T02:41:24.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.178+0000 7fecf6600700 1 -- 172.21.15.67:0/1769460640 wait complete. 2024-03-30T02:41:24.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:41:24.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.432+0000 7fa83a400700 1 -- 172.21.15.67:0/793526318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa83410b110 msgr2=0x7fa83410d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.432+0000 7fa83a400700 1 --2- 172.21.15.67:0/793526318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa83410b110 0x7fa83410d5d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00b0a0 tx=0x7fa82c00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.432+0000 7fa83a400700 1 -- 172.21.15.67:0/793526318 shutdown_connections 2024-03-30T02:41:24.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.432+0000 7fa83a400700 1 --2- 172.21.15.67:0/793526318 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa834101cf0 0x7fa8341020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.432+0000 7fa83a400700 1 --2- 172.21.15.67:0/793526318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa83410b110 0x7fa83410d5d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.432+0000 7fa83a400700 1 --2- 172.21.15.67:0/793526318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa834102610 0x7fa83410aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.432+0000 7fa83a400700 1 -- 172.21.15.67:0/793526318 >> 172.21.15.67:0/793526318 conn(0x7fa8340fb510 msgr2=0x7fa8340fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.432+0000 7fa83a400700 1 -- 172.21.15.67:0/793526318 shutdown_connections 2024-03-30T02:41:24.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.432+0000 7fa83a400700 1 -- 172.21.15.67:0/793526318 wait complete. 2024-03-30T02:41:24.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.433+0000 7fa83a400700 1 Processor -- start 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa83a400700 1 -- start start 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa83a400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa834101cf0 0x7fa8340ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa83a400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa834102610 0x7fa8340ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa83a400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83410b110 0x7fa8341003c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa83a400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8341a5500 con 0x7fa834102610 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa83a400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8341a5670 con 0x7fa834101cf0 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa83a400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8341a5830 con 0x7fa83410b110 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa839000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa834101cf0 0x7fa8340ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa839000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa834101cf0 0x7fa8340ff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:32830/0 (socket says 172.21.15.67:32830) 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa839000700 1 -- 172.21.15.67:0/2920060661 learned_addr learned my addr 172.21.15.67:0/2920060661 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa839a00700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83410b110 0x7fa8341003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.434+0000 7fa833e00700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa834102610 0x7fa8340ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa839000700 1 -- 172.21.15.67:0/2920060661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83410b110 msgr2=0x7fa8341003c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa839000700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83410b110 0x7fa8341003c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa839000700 1 -- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa834102610 msgr2=0x7fa8340ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa839000700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa834102610 0x7fa8340ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa839000700 1 -- 172.21.15.67:0/2920060661 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c009d00 con 0x7fa834101cf0 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa833e00700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa834102610 0x7fa8340ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa839000700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa834101cf0 0x7fa8340ff940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa82800e9a0 tx=0x7fa82800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa831600700 1 -- 172.21.15.67:0/2920060661 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828009790 con 0x7fa834101cf0 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa831600700 1 -- 172.21.15.67:0/2920060661 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa82800cbd0 con 0x7fa834101cf0 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa83a400700 1 -- 172.21.15.67:0/2920060661 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8341a5b10 con 0x7fa834101cf0 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa831600700 1 -- 172.21.15.67:0/2920060661 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828018680 con 0x7fa834101cf0 2024-03-30T02:41:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.435+0000 7fa83a400700 1 -- 172.21.15.67:0/2920060661 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8341a6070 con 0x7fa834101cf0 2024-03-30T02:41:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.438+0000 7fa831600700 1 -- 172.21.15.67:0/2920060661 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa82800cd40 con 0x7fa834101cf0 2024-03-30T02:41:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa831600700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa81c0a1310 0x7fa81c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa831600700 1 -- 172.21.15.67:0/2920060661 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa828014070 con 0x7fa834101cf0 2024-03-30T02:41:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa84f57db80 1 Processor -- start 2024-03-30T02:41:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa833e00700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa81c0a1310 0x7fa81c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa84f57db80 1 -- start start 2024-03-30T02:41:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 0x56421c4afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4baa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4bd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa84f57db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c408990 con 0x56421c4af7a0 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa84f57db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c406350 con 0x56421c4bafc0 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.440+0000 7fa84f57db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c401c40 con 0x56421c4b00c0 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 0x56421c4afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 0x56421c4afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53142/0 (socket says 172.21.15.67:53142) 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839000700 1 -- 172.21.15.67:0/3695614467 learned_addr learned my addr 172.21.15.67:0/3695614467 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839a00700 1 --2- 172.21.15.67:0/3695614467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4bd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa833e00700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa81c0a1310 0x7fa81c0a37d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa834101020 tx=0x7fa820009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa833e00700 1 --2- 172.21.15.67:0/3695614467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4baa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839a00700 1 -- 172.21.15.67:0/3695614467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 msgr2=0x56421c4baa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839a00700 1 --2- 172.21.15.67:0/3695614467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4baa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839a00700 1 -- 172.21.15.67:0/3695614467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 msgr2=0x56421c4afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839a00700 1 --2- 172.21.15.67:0/3695614467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 0x56421c4afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839a00700 1 -- 172.21.15.67:0/3695614467 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56421c2833b0 con 0x56421c4bafc0 2024-03-30T02:41:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.441+0000 7fa839a00700 1 --2- 172.21.15.67:0/3695614467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4bd3b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009fd0 tx=0x7fa82c009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b02e9062c17c7de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.442+0000 7fa826000700 1 -- 172.21.15.67:0/3695614467 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00e040 con 0x56421c4bafc0 2024-03-30T02:41:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.442+0000 7fa826000700 1 -- 172.21.15.67:0/3695614467 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa82c007d30 con 0x56421c4bafc0 2024-03-30T02:41:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.442+0000 7fa826000700 1 -- 172.21.15.67:0/3695614467 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c007740 con 0x56421c4bafc0 2024-03-30T02:41:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.442+0000 7fa84f57db80 1 -- 172.21.15.67:0/3695614467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 msgr2=0x56421c4bd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.442+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3695614467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4bd3b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009fd0 tx=0x7fa82c009c40 comp rx=0 tx=0).stop 2024-03-30T02:41:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.443+0000 7fa84f57db80 1 -- 172.21.15.67:0/3695614467 shutdown_connections 2024-03-30T02:41:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.443+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3695614467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 0x56421c4afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.443+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3695614467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4bd3b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.443+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3695614467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4baa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.443+0000 7fa84f57db80 1 -- 172.21.15.67:0/3695614467 >> 172.21.15.67:0/3695614467 conn(0x56421c40f6a0 msgr2=0x56421c4abc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.443+0000 7fa84f57db80 1 -- 172.21.15.67:0/3695614467 shutdown_connections 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.443+0000 7fa84f57db80 1 -- 172.21.15.67:0/3695614467 wait complete. 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.443+0000 7fa84f57db80 1 -- start start 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa84f57db80 1 Processor -- start 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 0x56421c4ba990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa84f57db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c39dd80 con 0x56421c4af7a0 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa84f57db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c39da20 con 0x56421c4bafc0 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa84f57db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c2856a0 con 0x56421c4b00c0 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa84f57db80 10 client.0 _ll_register_callbacks cb 0x56421c3a7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:24.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa839a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa833e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa839000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 0x56421c4ba990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.444+0000 7fa833e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4b7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36676/0 (socket says 172.21.15.67:36676) 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa833e00700 1 -- 172.21.15.67:0/2150314931 learned_addr learned my addr 172.21.15.67:0/2150314931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa839a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4b8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:32856/0 (socket says 172.21.15.67:32856) 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa839a00700 1 -- 172.21.15.67:0/2150314931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 msgr2=0x56421c4b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa839a00700 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa839a00700 1 -- 172.21.15.67:0/2150314931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 msgr2=0x56421c4ba990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa839a00700 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 0x56421c4ba990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa839a00700 1 -- 172.21.15.67:0/2150314931 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83410e290 con 0x56421c4bafc0 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa833e00700 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa839a00700 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4b8220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0062a0 tx=0x7fa82c00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20912e2db4d068d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa81aa00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa81aa00700 1 -- 172.21.15.67:0/2150314931 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00e040 con 0x56421c4bafc0 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa84f57db80 1 -- 172.21.15.67:0/2150314931 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56421c3c2380 con 0x56421c4bafc0 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.445+0000 7fa81aa00700 1 -- 172.21.15.67:0/2150314931 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa82c0076b0 con 0x56421c4bafc0 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa81aa00700 1 -- 172.21.15.67:0/2150314931 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa82c003ea0 con 0x56421c4bafc0 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa84f57db80 10 client.26717 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa84f57db80 1 -- 172.21.15.67:0/2150314931 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56421c22fb60 con 0x56421c4bafc0 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa81aa00700 1 -- 172.21.15.67:0/2150314931 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56421c3c2380 con 0x56421c4bafc0 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa81aa00700 1 -- 172.21.15.67:0/2150314931 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fa82c019030 con 0x56421c4bafc0 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa84f57db80 10 client.26717 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa84f57db80 10 client.26717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa84f57db80 20 client.26717 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa84f57db80 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56421c4ca8c0 0x56421c4ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa84f57db80 4 client.26717 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa84f57db80 1 -- 172.21.15.67:0/2150314931 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56421c3fa700 con 0x56421c4ca8c0 2024-03-30T02:41:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.446+0000 7fa839000700 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56421c4ca8c0 0x56421c4ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.447+0000 7fa839000700 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56421c4ca8c0 0x56421c4ccce0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3bf6b37693ee8c server_cookie=411b793be0fbc714 in_seq=0 out_seq=0 2024-03-30T02:41:24.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.447+0000 7fa81aa00700 10 client.26717 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:24.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa81aa00700 1 -- 172.21.15.67:0/2150314931 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56421c3fa700 con 0x56421c4ca8c0 2024-03-30T02:41:24.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa81aa00700 10 client.26717 handle_command_reply: tid=0 2024-03-30T02:41:24.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 client.26717 shutdown 2024-03-30T02:41:24.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 -- 172.21.15.67:0/2150314931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 msgr2=0x56421c4b8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4b8220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0062a0 tx=0x7fa82c00bce0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 -- 172.21.15.67:0/2150314931 shutdown_connections 2024-03-30T02:41:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56421c4ca8c0 0x56421c4ccce0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4af7a0 0x56421c4ba990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4bafc0 0x56421c4b8220 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 --2- 172.21.15.67:0/2150314931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4b00c0 0x56421c4b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa81aa00700 0 client.26717 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 -- 172.21.15.67:0/2150314931 >> 172.21.15.67:0/2150314931 conn(0x56421c40f6a0 msgr2=0x56421c4b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 -- 172.21.15.67:0/2150314931 shutdown_connections 2024-03-30T02:41:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.451+0000 7fa84f57db80 1 -- 172.21.15.67:0/2150314931 wait complete. 2024-03-30T02:41:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.452+0000 7fa84f57db80 20 client.26717 trim_cache size 0 max 16384 2024-03-30T02:41:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa84f57db80 1 Processor -- start 2024-03-30T02:41:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa84f57db80 1 -- start start 2024-03-30T02:41:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4ba4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 0x56421c4b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa84f57db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c4b6160 con 0x56421c4b00c0 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa84f57db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c4b8ad0 con 0x56421c4af7a0 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa84f57db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c4b8800 con 0x56421c4bafc0 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa833e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4ba4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.474+0000 7fa839000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa833e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4ba4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53152/0 (socket says 172.21.15.67:53152) 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa833e00700 1 -- 172.21.15.67:0/4133832599 learned_addr learned my addr 172.21.15.67:0/4133832599 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa839a00700 1 --2- 172.21.15.67:0/4133832599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 0x56421c4b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa833e00700 1 -- 172.21.15.67:0/4133832599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 msgr2=0x56421c4b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa833e00700 1 --2- 172.21.15.67:0/4133832599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 0x56421c4b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa833e00700 1 -- 172.21.15.67:0/4133832599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 msgr2=0x56421c4c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa833e00700 1 --2- 172.21.15.67:0/4133832599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa833e00700 1 -- 172.21.15.67:0/4133832599 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56421c3c7ea0 con 0x56421c4b00c0 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa839000700 1 --2- 172.21.15.67:0/4133832599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.475+0000 7fa833e00700 1 --2- 172.21.15.67:0/4133832599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4ba4c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa820010a40 tx=0x7fa820013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d061e04454f622 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.476+0000 7fa826a00700 1 -- 172.21.15.67:0/4133832599 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820024070 con 0x56421c4b00c0 2024-03-30T02:41:24.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.476+0000 7fa826a00700 1 -- 172.21.15.67:0/4133832599 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa820015070 con 0x56421c4b00c0 2024-03-30T02:41:24.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.476+0000 7fa826a00700 1 -- 172.21.15.67:0/4133832599 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820028620 con 0x56421c4b00c0 2024-03-30T02:41:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.476+0000 7fa84f57db80 1 -- 172.21.15.67:0/4133832599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 msgr2=0x56421c4ba4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.476+0000 7fa84f57db80 1 --2- 172.21.15.67:0/4133832599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4ba4c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa820010a40 tx=0x7fa820013000 comp rx=0 tx=0).stop 2024-03-30T02:41:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.477+0000 7fa84f57db80 1 -- 172.21.15.67:0/4133832599 shutdown_connections 2024-03-30T02:41:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.477+0000 7fa84f57db80 1 --2- 172.21.15.67:0/4133832599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 0x56421c4b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.477+0000 7fa84f57db80 1 --2- 172.21.15.67:0/4133832599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.477+0000 7fa84f57db80 1 --2- 172.21.15.67:0/4133832599 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4ba4c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.477+0000 7fa84f57db80 1 -- 172.21.15.67:0/4133832599 >> 172.21.15.67:0/4133832599 conn(0x56421c40f6a0 msgr2=0x56421c4c6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.477+0000 7fa84f57db80 1 -- 172.21.15.67:0/4133832599 shutdown_connections 2024-03-30T02:41:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.477+0000 7fa84f57db80 1 -- 172.21.15.67:0/4133832599 wait complete. 2024-03-30T02:41:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.477+0000 7fa84f57db80 1 -- start start 2024-03-30T02:41:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.477+0000 7fa84f57db80 1 Processor -- start 2024-03-30T02:41:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa84f57db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 0x56421c4c02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa84f57db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c4c21d0 con 0x56421c4b00c0 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa84f57db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c4c0810 con 0x56421c4af7a0 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa84f57db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421c4c09d0 con 0x56421c4bafc0 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa84f57db80 10 client.0 _ll_register_callbacks cb 0x56421c3a7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa839000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa839000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:32874/0 (socket says 172.21.15.67:32874) 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa839000700 1 -- 172.21.15.67:0/3496238785 learned_addr learned my addr 172.21.15.67:0/3496238785 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa839a00700 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 0x56421c4c02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa833e00700 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa839000700 1 -- 172.21.15.67:0/3496238785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 msgr2=0x56421c4c02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa839000700 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 0x56421c4c02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.478+0000 7fa839000700 1 -- 172.21.15.67:0/3496238785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 msgr2=0x56421c4bfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa839000700 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4bfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa839000700 1 -- 172.21.15.67:0/3496238785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8341a6070 con 0x56421c4af7a0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa833e00700 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4bfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa839000700 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c18f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8280964f0 tx=0x7fa8280cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a46a1ac1d271281 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa81aa00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa81aa00700 1 -- 172.21.15.67:0/3496238785 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82809de20 con 0x56421c4af7a0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa84f57db80 1 -- 172.21.15.67:0/3496238785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa82c007d30 con 0x56421c4af7a0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa81aa00700 1 -- 172.21.15.67:0/3496238785 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa8280a2470 con 0x56421c4af7a0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa81aa00700 1 -- 172.21.15.67:0/3496238785 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa82809a250 con 0x56421c4af7a0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa84f57db80 10 client.26723 fetch_fsmap learned FSMap version 31 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.479+0000 7fa84f57db80 1 -- 172.21.15.67:0/3496238785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fa820011440 con 0x56421c4af7a0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa81aa00700 1 -- 172.21.15.67:0/3496238785 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa82c007d30 con 0x56421c4af7a0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa81aa00700 1 -- 172.21.15.67:0/3496238785 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fa8280a1fe0 con 0x56421c4af7a0 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa84f57db80 10 client.26723 fetch_fsmap finished waiting for FSMap version 31 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa84f57db80 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa84f57db80 20 client.26723 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56421c51fa10 0x56421c521e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa84f57db80 4 client.26723 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:41:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa84f57db80 1 -- 172.21.15.67:0/3496238785 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56421c250cf0 con 0x56421c51fa10 2024-03-30T02:41:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa833e00700 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56421c51fa10 0x56421c521e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa833e00700 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56421c51fa10 0x56421c521e30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9454d8c24b759a1 server_cookie=beb2064838afbb41 in_seq=0 out_seq=0 2024-03-30T02:41:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.480+0000 7fa81aa00700 10 client.26723 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:24.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.483+0000 7fa81aa00700 1 -- 172.21.15.67:0/3496238785 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x56421c250cf0 con 0x56421c51fa10 2024-03-30T02:41:24.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.483+0000 7fa81aa00700 10 client.26723 handle_command_reply: tid=0 2024-03-30T02:41:24.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.483+0000 7fa84f57db80 1 client.26723 shutdown 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 -- 172.21.15.67:0/3496238785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 msgr2=0x56421c4c18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c18f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa8280964f0 tx=0x7fa8280cc040 comp rx=0 tx=0).stop 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 -- 172.21.15.67:0/3496238785 shutdown_connections 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56421c4bafc0 0x56421c4c02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56421c4af7a0 0x56421c4c18f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56421c51fa10 0x56421c521e30 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 --2- 172.21.15.67:0/3496238785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56421c4b00c0 0x56421c4bfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa81aa00700 0 client.26723 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 -- 172.21.15.67:0/3496238785 >> 172.21.15.67:0/3496238785 conn(0x56421c40f6a0 msgr2=0x56421c4b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 -- 172.21.15.67:0/3496238785 shutdown_connections 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 1 -- 172.21.15.67:0/3496238785 wait complete. 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.484+0000 7fa84f57db80 20 client.26723 trim_cache size 0 max 16384 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (2796 inodes in the stack)", 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "7a376898-7302-4db7-9b68-928808ede83e": { 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "7a376898-7302-4db7-9b68-928808ede83e", 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:41:24.488 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:41:24.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 -- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa81c0a1310 msgr2=0x7fa81c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa81c0a1310 0x7fa81c0a37d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa834101020 tx=0x7fa820009260 comp rx=0 tx=0).stop 2024-03-30T02:41:24.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 -- 172.21.15.67:0/2920060661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa834101cf0 msgr2=0x7fa8340ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa834101cf0 0x7fa8340ff940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa82800e9a0 tx=0x7fa82800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 -- 172.21.15.67:0/2920060661 shutdown_connections 2024-03-30T02:41:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa81c0a1310 0x7fa81c0a37d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa83410b110 0x7fa8341003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa834101cf0 0x7fa8340ff940 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 --2- 172.21.15.67:0/2920060661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa834102610 0x7fa8340ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 -- 172.21.15.67:0/2920060661 >> 172.21.15.67:0/2920060661 conn(0x7fa8340fb510 msgr2=0x7fa8340fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 -- 172.21.15.67:0/2920060661 shutdown_connections 2024-03-30T02:41:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:24.485+0000 7fa83a400700 1 -- 172.21.15.67:0/2920060661 wait complete. 2024-03-30T02:41:24.505 INFO:tasks.cephfs.filesystem:scrub status for tag:7a376898-7302-4db7-9b68-928808ede83e - {'path': '/', 'tag': '7a376898-7302-4db7-9b68-928808ede83e', 'options': 'recursive,force'} 2024-03-30T02:41:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:24 smithi067 ceph-mon[37808]: pgmap v133: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-03-30T02:41:24.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:24 smithi170 ceph-mon[78798]: pgmap v133: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-03-30T02:41:24.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:24 smithi082 ceph-mon[79176]: pgmap v133: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:25.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:41:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:25.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:41:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: pgmap v134: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 20 MiB/s wr, 2.45k op/s 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:26.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:26.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: pgmap v134: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 20 MiB/s wr, 2.45k op/s 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: pgmap v134: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 20 MiB/s wr, 2.45k op/s 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:26.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:27.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:27.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:27.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:27.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: pgmap v135: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 1.97k op/s 2024-03-30T02:41:28.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:28.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: pgmap v135: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 1.97k op/s 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:28.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: pgmap v135: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 14 MiB/s wr, 1.97k op/s 2024-03-30T02:41:28.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:29.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:29.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:29 smithi082 ceph-mon[79176]: pgmap v136: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 1.91k op/s 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:29 smithi067 ceph-mon[37808]: pgmap v136: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 1.91k op/s 2024-03-30T02:41:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:29.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:29.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:29 smithi170 ceph-mon[78798]: pgmap v136: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 13 MiB/s wr, 1.91k op/s 2024-03-30T02:41:31.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:31 smithi067 ceph-mon[37808]: pgmap v137: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 2.67k op/s 2024-03-30T02:41:31.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:31.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:31.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:31 smithi170 ceph-mon[78798]: pgmap v137: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 2.67k op/s 2024-03-30T02:41:31.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:31.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:31.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:31 smithi082 ceph-mon[79176]: pgmap v137: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 17 MiB/s wr, 2.67k op/s 2024-03-30T02:41:31.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:32.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: pgmap v138: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 1.62k op/s 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: pgmap v138: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 1.62k op/s 2024-03-30T02:41:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: pgmap v138: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 1.62k op/s 2024-03-30T02:41:34.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:34.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:34.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:34.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:34.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:34.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:34.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.006 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.007 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.008 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.009 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.009 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.009 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.009 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.009 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.009 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.009 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.009 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:35.010 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:35.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:35 smithi067 ceph-mon[37808]: pgmap v139: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.8 MiB/s wr, 1.97k op/s 2024-03-30T02:41:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:35 smithi170 ceph-mon[78798]: pgmap v139: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.8 MiB/s wr, 1.97k op/s 2024-03-30T02:41:35.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:35 smithi082 ceph-mon[79176]: pgmap v139: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.8 MiB/s wr, 1.97k op/s 2024-03-30T02:41:37.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:37.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:37.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:37.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:37.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:37.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:38.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:38 smithi067 ceph-mon[37808]: pgmap v140: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.81k op/s 2024-03-30T02:41:38.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:38 smithi170 ceph-mon[78798]: pgmap v140: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.81k op/s 2024-03-30T02:41:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:38 smithi082 ceph-mon[79176]: pgmap v140: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.81k op/s 2024-03-30T02:41:39.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: pgmap v141: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.73k op/s 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: pgmap v141: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.73k op/s 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: pgmap v141: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.73k op/s 2024-03-30T02:41:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:41.731 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:41.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: pgmap v142: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 13 MiB/s wr, 2.31k op/s 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: pgmap v142: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 13 MiB/s wr, 2.31k op/s 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: pgmap v142: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 13 MiB/s wr, 2.31k op/s 2024-03-30T02:41:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:43.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: pgmap v143: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.3 MiB/s wr, 1.53k op/s 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:44.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: pgmap v143: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.3 MiB/s wr, 1.53k op/s 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:44.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: pgmap v143: 129 pgs: 129 active+clean; 1.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.3 MiB/s wr, 1.53k op/s 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:45.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: pgmap v144: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 1.92k op/s 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: pgmap v144: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 1.92k op/s 2024-03-30T02:41:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: pgmap v144: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 1.92k op/s 2024-03-30T02:41:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:47.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:48.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:48 smithi067 ceph-mon[37808]: pgmap v145: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.3 MiB/s wr, 2.27k op/s 2024-03-30T02:41:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:48 smithi170 ceph-mon[78798]: pgmap v145: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.3 MiB/s wr, 2.27k op/s 2024-03-30T02:41:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:48 smithi082 ceph-mon[79176]: pgmap v145: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.3 MiB/s wr, 2.27k op/s 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: pgmap v146: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 1.73k op/s 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: pgmap v146: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 1.73k op/s 2024-03-30T02:41:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: pgmap v146: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 1.73k op/s 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:50.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:51.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:51.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:51.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:51.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:51.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:52 smithi170 ceph-mon[78798]: pgmap v147: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 3.00k op/s 2024-03-30T02:41:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:52 smithi082 ceph-mon[79176]: pgmap v147: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 3.00k op/s 2024-03-30T02:41:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:52 smithi067 ceph-mon[37808]: pgmap v147: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 3.00k op/s 2024-03-30T02:41:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:41:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:53.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:53.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:53.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:41:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: pgmap v148: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 2.40k op/s 2024-03-30T02:41:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.724+0000 7ff995600700 1 -- 172.21.15.67:0/514540296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff990073180 msgr2=0x7ff990073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:54.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.724+0000 7ff995600700 1 --2- 172.21.15.67:0/514540296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff990073180 0x7ff990073560 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff980009990 tx=0x7ff980009ca0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.724+0000 7ff995600700 1 -- 172.21.15.67:0/514540296 shutdown_connections 2024-03-30T02:41:54.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.724+0000 7ff995600700 1 --2- 172.21.15.67:0/514540296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff990073180 0x7ff990073560 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.724+0000 7ff995600700 1 --2- 172.21.15.67:0/514540296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff99010d3e0 0x7ff99010f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.724+0000 7ff995600700 1 --2- 172.21.15.67:0/514540296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff990073aa0 0x7ff99010cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.724+0000 7ff995600700 1 -- 172.21.15.67:0/514540296 >> 172.21.15.67:0/514540296 conn(0x7ff9900fc670 msgr2=0x7ff9900fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:54.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.725+0000 7ff995600700 1 -- 172.21.15.67:0/514540296 shutdown_connections 2024-03-30T02:41:54.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.725+0000 7ff995600700 1 -- 172.21.15.67:0/514540296 wait complete. 2024-03-30T02:41:54.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.726+0000 7ff995600700 1 Processor -- start 2024-03-30T02:41:54.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.726+0000 7ff995600700 1 -- start start 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff995600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff990073180 0x7ff990199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff995600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff990073aa0 0x7ff9901997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff995600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff99010d3e0 0x7ff99019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff995600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99019e1a0 con 0x7ff990073180 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff995600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99019e310 con 0x7ff990073aa0 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff995600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99019e4d0 con 0x7ff99010d3e0 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff98fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff990073180 0x7ff990199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff994c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff99010d3e0 0x7ff99019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff98fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff990073180 0x7ff990199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40384/0 (socket says 172.21.15.67:40384) 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff994c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff99010d3e0 0x7ff99019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55996/0 (socket says 172.21.15.67:55996) 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.727+0000 7ff98fe00700 1 -- 172.21.15.67:0/3554360582 learned_addr learned my addr 172.21.15.67:0/3554360582 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:54.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.728+0000 7ff98f400700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff990073aa0 0x7ff9901997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.728+0000 7ff994c00700 1 -- 172.21.15.67:0/3554360582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff990073aa0 msgr2=0x7ff9901997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.728+0000 7ff994c00700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff990073aa0 0x7ff9901997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.728+0000 7ff994c00700 1 -- 172.21.15.67:0/3554360582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff990073180 msgr2=0x7ff990199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.728+0000 7ff994c00700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff990073180 0x7ff990199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.728+0000 7ff994c00700 1 -- 172.21.15.67:0/3554360582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800095f0 con 0x7ff99010d3e0 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.728+0000 7ff994c00700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff99010d3e0 0x7ff99019db60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff98800ef90 tx=0x7ff98800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.729+0000 7ff98cc00700 1 -- 172.21.15.67:0/3554360582 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988018070 con 0x7ff99010d3e0 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.729+0000 7ff98cc00700 1 -- 172.21.15.67:0/3554360582 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98800f040 con 0x7ff99010d3e0 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.729+0000 7ff98cc00700 1 -- 172.21.15.67:0/3554360582 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800cce0 con 0x7ff99010d3e0 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.729+0000 7ff995600700 1 -- 172.21.15.67:0/3554360582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99019e750 con 0x7ff99010d3e0 2024-03-30T02:41:54.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.729+0000 7ff995600700 1 -- 172.21.15.67:0/3554360582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9901a5ff0 con 0x7ff99010d3e0 2024-03-30T02:41:54.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.732+0000 7ff98cc00700 1 -- 172.21.15.67:0/3554360582 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff988003a00 con 0x7ff99010d3e0 2024-03-30T02:41:54.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.735+0000 7ff9aa6a9b80 1 Processor -- start 2024-03-30T02:41:54.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.735+0000 7ff98cc00700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9740a1290 0x7ff9740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.735+0000 7ff98cc00700 1 -- 172.21.15.67:0/3554360582 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff9880ccbd0 con 0x7ff99010d3e0 2024-03-30T02:41:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.735+0000 7ff9aa6a9b80 1 -- start start 2024-03-30T02:41:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.735+0000 7ff98f400700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9740a1290 0x7ff9740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.735+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef1347b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef135b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef135dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef12a0990 con 0x55aef13477a0 2024-03-30T02:41:54.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef129e350 con 0x55aef135b7d0 2024-03-30T02:41:54.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef1299c40 con 0x55aef13480c0 2024-03-30T02:41:54.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff98f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef135b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff98fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef1347b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff98f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef135b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56002/0 (socket says 172.21.15.67:56002) 2024-03-30T02:41:54.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff98f400700 1 -- 172.21.15.67:0/2098516307 learned_addr learned my addr 172.21.15.67:0/2098516307 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:54.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.736+0000 7ff994c00700 1 --2- 172.21.15.67:0/2098516307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef135dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff98f400700 1 -- 172.21.15.67:0/2098516307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 msgr2=0x55aef135dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:54.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff98f400700 1 --2- 172.21.15.67:0/2098516307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef135dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff98f400700 1 -- 172.21.15.67:0/2098516307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 msgr2=0x55aef1347b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:54.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff98f400700 1 --2- 172.21.15.67:0/2098516307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef1347b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff98f400700 1 -- 172.21.15.67:0/2098516307 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef111b3b0 con 0x55aef13480c0 2024-03-30T02:41:54.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff98fe00700 1 --2- 172.21.15.67:0/2098516307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef1347b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:41:54.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff98f400700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9740a1290 0x7ff9740a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0159f0 tx=0x7ff97c0155b0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:54.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff98f400700 1 --2- 172.21.15.67:0/2098516307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef135b290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff97c011a90 tx=0x7ff97c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0bf57a8012e17a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:54.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff985200700 1 -- 172.21.15.67:0/2098516307 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0156a0 con 0x55aef13480c0 2024-03-30T02:41:54.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff985200700 1 -- 172.21.15.67:0/2098516307 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff97c009d40 con 0x55aef13480c0 2024-03-30T02:41:54.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.737+0000 7ff985200700 1 -- 172.21.15.67:0/2098516307 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c00fc60 con 0x55aef13480c0 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.738+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/2098516307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 msgr2=0x55aef135b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.738+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/2098516307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef135b290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff97c011a90 tx=0x7ff97c01d000 comp rx=0 tx=0).stop 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.738+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/2098516307 shutdown_connections 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.738+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/2098516307 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef1347b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.738+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/2098516307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef135dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.738+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/2098516307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef135b290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.738+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/2098516307 >> 172.21.15.67:0/2098516307 conn(0x55aef12a76a0 msgr2=0x55aef1343c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.738+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/2098516307 shutdown_connections 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.739+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/2098516307 wait complete. 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.739+0000 7ff9aa6a9b80 1 -- start start 2024-03-30T02:41:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.739+0000 7ff9aa6a9b80 1 Processor -- start 2024-03-30T02:41:54.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.740+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef135b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.740+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef13584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.740+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1358be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.740+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef1235d80 con 0x55aef13477a0 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.740+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef1235a20 con 0x55aef135b7d0 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.740+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef111d6a0 con 0x55aef13480c0 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.740+0000 7ff9aa6a9b80 10 client.0 _ll_register_callbacks cb 0x55aef123f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.740+0000 7ff98fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef135b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff98fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef135b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40406/0 (socket says 172.21.15.67:40406) 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff98fe00700 1 -- 172.21.15.67:0/1029166758 learned_addr learned my addr 172.21.15.67:0/1029166758 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff994c00700 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1358be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff98f400700 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef13584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff994c00700 1 -- 172.21.15.67:0/1029166758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 msgr2=0x55aef13584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff994c00700 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef13584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff994c00700 1 -- 172.21.15.67:0/1029166758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 msgr2=0x55aef135b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff994c00700 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef135b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff994c00700 1 -- 172.21.15.67:0/1029166758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97c011850 con 0x55aef135b7d0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff98fe00700 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef135b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff994c00700 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1358be0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff9880c9f70 tx=0x7ff988099580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab1c7bb509c2fed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff979600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.741+0000 7ff979600700 1 -- 172.21.15.67:0/1029166758 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a9350 con 0x55aef135b7d0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff979600700 1 -- 172.21.15.67:0/1029166758 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff98809a0c0 con 0x55aef135b7d0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1029166758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aef1292700 con 0x55aef135b7d0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff979600700 1 -- 172.21.15.67:0/1029166758 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9880999c0 con 0x55aef135b7d0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff9aa6a9b80 10 client.26735 fetch_fsmap learned FSMap version 32 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1029166758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55aef10c7b60 con 0x55aef135b7d0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff979600700 1 -- 172.21.15.67:0/1029166758 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9880b1070 con 0x55aef135b7d0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff979600700 1 -- 172.21.15.67:0/1029166758 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff988099d70 con 0x55aef135b7d0 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff9aa6a9b80 10 client.26735 fetch_fsmap finished waiting for FSMap version 32 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff9aa6a9b80 10 client.26735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff9aa6a9b80 20 client.26735 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55aef136b3c0 0x55aef136d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff9aa6a9b80 4 client.26735 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:41:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.742+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1029166758 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aef10e8cf0 con 0x55aef136b3c0 2024-03-30T02:41:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.743+0000 7ff98fe00700 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55aef136b3c0 0x55aef136d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.743+0000 7ff98fe00700 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55aef136b3c0 0x55aef136d7e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33da5b01b9b10d11 server_cookie=1e96599fc757021 in_seq=0 out_seq=0 2024-03-30T02:41:54.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.743+0000 7ff979600700 10 client.26735 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.744+0000 7ff979600700 1 -- 172.21.15.67:0/1029166758 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aef10e8cf0 con 0x55aef136b3c0 2024-03-30T02:41:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.744+0000 7ff979600700 10 client.26735 handle_command_reply: tid=0 2024-03-30T02:41:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 client.26735 shutdown 2024-03-30T02:41:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1029166758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 msgr2=0x55aef1358be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1358be0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff9880c9f70 tx=0x7ff988099580 comp rx=0 tx=0).stop 2024-03-30T02:41:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1029166758 shutdown_connections 2024-03-30T02:41:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55aef136b3c0 0x55aef136d7e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef135b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1358be0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1029166758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef13584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1029166758 >> 172.21.15.67:0/1029166758 conn(0x55aef12a76a0 msgr2=0x55aef1349700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:54.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff979600700 0 client.26735 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:54.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1029166758 shutdown_connections 2024-03-30T02:41:54.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1029166758 wait complete. 2024-03-30T02:41:54.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.745+0000 7ff9aa6a9b80 20 client.26735 trim_cache size 0 max 16384 2024-03-30T02:41:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: pgmap v148: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 2.40k op/s 2024-03-30T02:41:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.764+0000 7ff9aa6a9b80 1 Processor -- start 2024-03-30T02:41:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff9aa6a9b80 1 -- start start 2024-03-30T02:41:54.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef13477a0 0x55aef1357d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13480c0 0x55aef13623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef135b7d0 0x55aef133c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef13582c0 con 0x55aef13480c0 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef13593a0 con 0x55aef13477a0 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef1362930 con 0x55aef135b7d0 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff98fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef13477a0 0x55aef1357d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff98fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef13477a0 0x55aef1357d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:47724/0 (socket says 172.21.15.67:47724) 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff98fe00700 1 -- 172.21.15.67:0/509760773 learned_addr learned my addr 172.21.15.67:0/509760773 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff98f400700 1 --2- 172.21.15.67:0/509760773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13480c0 0x55aef13623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff994c00700 1 --2- 172.21.15.67:0/509760773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef135b7d0 0x55aef133c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff98fe00700 1 -- 172.21.15.67:0/509760773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef135b7d0 msgr2=0x55aef133c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.765+0000 7ff98fe00700 1 --2- 172.21.15.67:0/509760773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef135b7d0 0x55aef133c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.766+0000 7ff98fe00700 1 -- 172.21.15.67:0/509760773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13480c0 msgr2=0x55aef13623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.766+0000 7ff98fe00700 1 --2- 172.21.15.67:0/509760773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13480c0 0x55aef13623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.766+0000 7ff98fe00700 1 -- 172.21.15.67:0/509760773 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef125fea0 con 0x55aef13477a0 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.766+0000 7ff98fe00700 1 --2- 172.21.15.67:0/509760773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef13477a0 0x55aef1357d30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55aef136e080 tx=0x7ff98000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57cddffbbb5a7e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.766+0000 7ff985c00700 1 -- 172.21.15.67:0/509760773 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98001d070 con 0x55aef13477a0 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.766+0000 7ff985c00700 1 -- 172.21.15.67:0/509760773 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98001e040 con 0x55aef13477a0 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.766+0000 7ff985c00700 1 -- 172.21.15.67:0/509760773 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9800049c0 con 0x55aef13477a0 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/509760773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef13477a0 msgr2=0x55aef1357d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/509760773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef13477a0 0x55aef1357d30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55aef136e080 tx=0x7ff98000b260 comp rx=0 tx=0).stop 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/509760773 shutdown_connections 2024-03-30T02:41:54.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/509760773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef135b7d0 0x55aef133c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/509760773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef13477a0 0x55aef1357d30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/509760773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13480c0 0x55aef13623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/509760773 >> 172.21.15.67:0/509760773 conn(0x55aef12a76a0 msgr2=0x55aef1372100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:54.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/509760773 shutdown_connections 2024-03-30T02:41:54.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/509760773 wait complete. 2024-03-30T02:41:54.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.767+0000 7ff9aa6a9b80 1 -- start start 2024-03-30T02:41:54.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff9aa6a9b80 1 Processor -- start 2024-03-30T02:41:54.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef1362270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef1360750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff9aa6a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1360c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef13613a0 con 0x55aef13477a0 2024-03-30T02:41:54.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef1361510 con 0x55aef135b7d0 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff9aa6a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef13bf1c0 con 0x55aef13480c0 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff9aa6a9b80 10 client.0 _ll_register_callbacks cb 0x55aef123f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff98f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef1360750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff98fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef1362270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff98f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef1360750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56028/0 (socket says 172.21.15.67:56028) 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff994c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1360c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.768+0000 7ff98f400700 1 -- 172.21.15.67:0/1318218989 learned_addr learned my addr 172.21.15.67:0/1318218989 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.769+0000 7ff98f400700 1 -- 172.21.15.67:0/1318218989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 msgr2=0x55aef1360c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.769+0000 7ff98f400700 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1360c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.769+0000 7ff98f400700 1 -- 172.21.15.67:0/1318218989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 msgr2=0x55aef1362270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.769+0000 7ff98f400700 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef1362270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.769+0000 7ff98f400700 1 -- 172.21.15.67:0/1318218989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff990110520 con 0x55aef13480c0 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.769+0000 7ff994c00700 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1360c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.769+0000 7ff98f400700 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef1360750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0116c0 tx=0x7ff97c00fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9673ff08d93ec19 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:41:54.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.769+0000 7ff979600700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.769+0000 7ff979600700 1 -- 172.21.15.67:0/1318218989 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c015620 con 0x55aef13480c0 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.770+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1318218989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aef125a380 con 0x55aef13480c0 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.770+0000 7ff979600700 1 -- 172.21.15.67:0/1318218989 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff97c011da0 con 0x55aef13480c0 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.770+0000 7ff979600700 1 -- 172.21.15.67:0/1318218989 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff97c004020 con 0x55aef13480c0 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.770+0000 7ff9aa6a9b80 10 client.26695 fetch_fsmap learned FSMap version 32 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.770+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1318218989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55aef111fcd0 con 0x55aef13480c0 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.770+0000 7ff979600700 1 -- 172.21.15.67:0/1318218989 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aef125a380 con 0x55aef13480c0 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.770+0000 7ff979600700 1 -- 172.21.15.67:0/1318218989 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff97c033070 con 0x55aef13480c0 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.770+0000 7ff9aa6a9b80 10 client.26695 fetch_fsmap finished waiting for FSMap version 32 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.771+0000 7ff9aa6a9b80 10 client.26695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.771+0000 7ff9aa6a9b80 20 client.26695 populate_metadata read hostname 'smithi067' 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.771+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55aef13c0850 0x55aef13c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.771+0000 7ff9aa6a9b80 4 client.26695 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:41:54.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.771+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1318218989 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aef111dce0 con 0x55aef13c0850 2024-03-30T02:41:54.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.771+0000 7ff98fe00700 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55aef13c0850 0x55aef13c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.771+0000 7ff98fe00700 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55aef13c0850 0x55aef13c2c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50fb509082e094ef server_cookie=1fe386e2a5cdd7e in_seq=0 out_seq=0 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.771+0000 7ff979600700 10 client.26695 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff979600700 1 -- 172.21.15.67:0/1318218989 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aef111dce0 con 0x55aef13c0850 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff979600700 10 client.26695 handle_command_reply: tid=0 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 client.26695 shutdown 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1318218989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 msgr2=0x55aef1360750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef1360750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0116c0 tx=0x7ff97c00fdc0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1318218989 shutdown_connections 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55aef13c0850 0x55aef13c2c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55aef13477a0 0x55aef1362270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55aef135b7d0 0x55aef1360c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 --2- 172.21.15.67:0/1318218989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aef13480c0 0x55aef1360750 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff979600700 0 client.26695 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1318218989 >> 172.21.15.67:0/1318218989 conn(0x55aef12a76a0 msgr2=0x55aef133c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1318218989 shutdown_connections 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 1 -- 172.21.15.67:0/1318218989 wait complete. 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.774+0000 7ff9aa6a9b80 20 client.26695 trim_cache size 0 max 16384 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stdout: "7a376898-7302-4db7-9b68-928808ede83e": { 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "7a376898-7302-4db7-9b68-928808ede83e", 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:41:54.776 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:41:54.777 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:41:54.777 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:41:54.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 -- 172.21.15.67:0/3554360582 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9740a1290 msgr2=0x7ff9740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:54.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9740a1290 0x7ff9740a3750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0159f0 tx=0x7ff97c0155b0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 -- 172.21.15.67:0/3554360582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff99010d3e0 msgr2=0x7ff99019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:41:54.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff99010d3e0 0x7ff99019db60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff98800ef90 tx=0x7ff98800c3a0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 -- 172.21.15.67:0/3554360582 shutdown_connections 2024-03-30T02:41:54.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9740a1290 0x7ff9740a3750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff990073180 0x7ff990199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff99010d3e0 0x7ff99019db60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 --2- 172.21.15.67:0/3554360582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff990073aa0 0x7ff9901997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:41:54.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 -- 172.21.15.67:0/3554360582 >> 172.21.15.67:0/3554360582 conn(0x7ff9900fc670 msgr2=0x7ff9900fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:41:54.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 -- 172.21.15.67:0/3554360582 shutdown_connections 2024-03-30T02:41:54.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:41:54.775+0000 7ff995600700 1 -- 172.21.15.67:0/3554360582 wait complete. 2024-03-30T02:41:54.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: pgmap v148: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 2.40k op/s 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.793 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:54.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:54.796 INFO:tasks.cephfs.filesystem:scrub status for tag:7a376898-7302-4db7-9b68-928808ede83e - {'path': '/', 'tag': '7a376898-7302-4db7-9b68-928808ede83e', 'options': 'recursive,force'} 2024-03-30T02:41:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:41:55.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:55 smithi082 ceph-mon[79176]: pgmap v149: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 2.88k op/s 2024-03-30T02:41:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:41:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:55 smithi067 ceph-mon[37808]: pgmap v149: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 2.88k op/s 2024-03-30T02:41:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:55 smithi170 ceph-mon[78798]: pgmap v149: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 2.88k op/s 2024-03-30T02:41:56.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:56.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:57.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:41:58.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:58 smithi067 ceph-mon[37808]: pgmap v150: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.7 MiB/s wr, 3.27k op/s 2024-03-30T02:41:58.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:41:58 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:41:58.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:58 smithi170 ceph-mon[78798]: pgmap v150: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.7 MiB/s wr, 3.27k op/s 2024-03-30T02:41:58.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:41:58 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:41:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:58 smithi082 ceph-mon[79176]: pgmap v150: 129 pgs: 129 active+clean; 1.6 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.7 MiB/s wr, 3.27k op/s 2024-03-30T02:41:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:41:58 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:42:00.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:00 smithi067 ceph-mon[37808]: pgmap v151: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 2.58k op/s 2024-03-30T02:42:00.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:00 smithi170 ceph-mon[78798]: pgmap v151: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 2.58k op/s 2024-03-30T02:42:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:00 smithi082 ceph-mon[79176]: pgmap v151: 129 pgs: 129 active+clean; 1.6 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 2.58k op/s 2024-03-30T02:42:02.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:02 smithi067 ceph-mon[37808]: pgmap v152: 129 pgs: 129 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.8 MiB/s wr, 2.98k op/s 2024-03-30T02:42:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:02 smithi170 ceph-mon[78798]: pgmap v152: 129 pgs: 129 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.8 MiB/s wr, 2.98k op/s 2024-03-30T02:42:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:02 smithi082 ceph-mon[79176]: pgmap v152: 129 pgs: 129 active+clean; 1.6 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.8 MiB/s wr, 2.98k op/s 2024-03-30T02:42:04.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:04 smithi067 ceph-mon[37808]: pgmap v153: 129 pgs: 129 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.7 MiB/s wr, 1.92k op/s 2024-03-30T02:42:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:04 smithi170 ceph-mon[78798]: pgmap v153: 129 pgs: 129 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.7 MiB/s wr, 1.92k op/s 2024-03-30T02:42:04.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:04 smithi082 ceph-mon[79176]: pgmap v153: 129 pgs: 129 active+clean; 1.7 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.7 MiB/s wr, 1.92k op/s 2024-03-30T02:42:06.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:06 smithi067 ceph-mon[37808]: pgmap v154: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 2.03k op/s 2024-03-30T02:42:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:06 smithi170 ceph-mon[78798]: pgmap v154: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 2.03k op/s 2024-03-30T02:42:06.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:06 smithi082 ceph-mon[79176]: pgmap v154: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 12 MiB/s wr, 2.03k op/s 2024-03-30T02:42:07.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:07 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:42:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:07 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:42:07.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:07 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:42:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:07 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:42:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:07 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:42:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:07 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:42:07.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:07 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:42:07.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:07 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:42:07.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:07 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:42:08.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:08 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:42:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:08 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:42:08.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:08 smithi067 ceph-mon[37808]: pgmap v155: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 1.68k op/s 2024-03-30T02:42:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:08 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:42:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:08 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:42:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:08 smithi170 ceph-mon[78798]: pgmap v155: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 1.68k op/s 2024-03-30T02:42:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:08 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:42:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:08 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:42:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:08 smithi082 ceph-mon[79176]: pgmap v155: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 1.68k op/s 2024-03-30T02:42:10.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:10 smithi067 ceph-mon[37808]: pgmap v156: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 920 op/s 2024-03-30T02:42:10.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:10 smithi170 ceph-mon[78798]: pgmap v156: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 920 op/s 2024-03-30T02:42:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:10 smithi082 ceph-mon[79176]: pgmap v156: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 15 MiB/s wr, 920 op/s 2024-03-30T02:42:12.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:12 smithi067 ceph-mon[37808]: pgmap v157: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 16 MiB/s wr, 931 op/s 2024-03-30T02:42:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:12 smithi170 ceph-mon[78798]: pgmap v157: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 16 MiB/s wr, 931 op/s 2024-03-30T02:42:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:12 smithi082 ceph-mon[79176]: pgmap v157: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 16 MiB/s wr, 931 op/s 2024-03-30T02:42:14.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:14 smithi067 ceph-mon[37808]: pgmap v158: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 508 op/s 2024-03-30T02:42:14.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:14 smithi170 ceph-mon[78798]: pgmap v158: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 508 op/s 2024-03-30T02:42:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:14 smithi082 ceph-mon[79176]: pgmap v158: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 11 MiB/s wr, 508 op/s 2024-03-30T02:42:15.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:15 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:42:15.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:15 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:42:15.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:15 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:42:15.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:15 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:42:15.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:15 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:42:15.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:15 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:42:15.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:15 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:42:15.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:15 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:42:15.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:15 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:42:15.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:15 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:42:15.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:15 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:42:15.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:15 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:42:16.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:16 smithi067 ceph-mon[37808]: pgmap v159: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.9 MiB/s wr, 327 op/s 2024-03-30T02:42:16.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:16 smithi170 ceph-mon[78798]: pgmap v159: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.9 MiB/s wr, 327 op/s 2024-03-30T02:42:16.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:16 smithi082 ceph-mon[79176]: pgmap v159: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.9 MiB/s wr, 327 op/s 2024-03-30T02:42:17.588 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/basic/fixdep 2024-03-30T02:42:17.796 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/conf.o 2024-03-30T02:42:18.034 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/confdata.o 2024-03-30T02:42:18.307 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/expr.o 2024-03-30T02:42:18.540 INFO:tasks.workunit.client.0.smithi067.stdout: LEX scripts/kconfig/lexer.lex.c 2024-03-30T02:42:18.565 INFO:tasks.workunit.client.0.smithi067.stdout: YACC scripts/kconfig/parser.tab.[ch] 2024-03-30T02:42:18.623 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/lexer.lex.o 2024-03-30T02:42:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:18 smithi067 ceph-mon[37808]: pgmap v160: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.3 MiB/s wr, 231 op/s 2024-03-30T02:42:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:18 smithi170 ceph-mon[78798]: pgmap v160: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.3 MiB/s wr, 231 op/s 2024-03-30T02:42:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:18 smithi082 ceph-mon[79176]: pgmap v160: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.3 MiB/s wr, 231 op/s 2024-03-30T02:42:18.931 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/menu.o 2024-03-30T02:42:19.134 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/parser.tab.o 2024-03-30T02:42:19.360 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/preprocess.o 2024-03-30T02:42:19.508 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/symbol.o 2024-03-30T02:42:19.818 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kconfig/util.o 2024-03-30T02:42:19.896 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTLD scripts/kconfig/conf 2024-03-30T02:42:19.933 INFO:tasks.workunit.client.0.smithi067.stdout:*** Default configuration is based on 'x86_64_defconfig' 2024-03-30T02:42:20.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:20 smithi067 ceph-mon[37808]: pgmap v161: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 110 op/s 2024-03-30T02:42:20.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:20 smithi170 ceph-mon[78798]: pgmap v161: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 110 op/s 2024-03-30T02:42:20.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:20 smithi082 ceph-mon[79176]: pgmap v161: 129 pgs: 129 active+clean; 1.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 110 op/s 2024-03-30T02:42:22.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:22 smithi067 ceph-mon[37808]: pgmap v162: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 211 op/s 2024-03-30T02:42:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:22 smithi170 ceph-mon[78798]: pgmap v162: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 211 op/s 2024-03-30T02:42:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:22 smithi082 ceph-mon[79176]: pgmap v162: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 211 op/s 2024-03-30T02:42:24.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:24 smithi067 ceph-mon[37808]: pgmap v163: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-03-30T02:42:24.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:24 smithi170 ceph-mon[78798]: pgmap v163: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-03-30T02:42:24.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:24 smithi082 ceph-mon[79176]: pgmap v163: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 231 op/s 2024-03-30T02:42:24.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:42:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.038+0000 7fa7dfa00700 1 -- 172.21.15.67:0/807946265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d8100f50 msgr2=0x7fa7d8101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.038+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/807946265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d8100f50 0x7fa7d8101330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009990 tx=0x7fa7cc009ca0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.038+0000 7fa7dfa00700 1 -- 172.21.15.67:0/807946265 shutdown_connections 2024-03-30T02:42:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.038+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/807946265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7d8106070 0x7fa7d810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.038+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/807946265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d8100f50 0x7fa7d8101330 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.038+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/807946265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d8101900 0x7fa7d8105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.038+0000 7fa7dfa00700 1 -- 172.21.15.67:0/807946265 >> 172.21.15.67:0/807946265 conn(0x7fa7d80fc7f0 msgr2=0x7fa7d80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.038+0000 7fa7dfa00700 1 -- 172.21.15.67:0/807946265 shutdown_connections 2024-03-30T02:42:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.038+0000 7fa7dfa00700 1 -- 172.21.15.67:0/807946265 wait complete. 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.039+0000 7fa7dfa00700 1 Processor -- start 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.039+0000 7fa7dfa00700 1 -- start start 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.039+0000 7fa7dfa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7d8100f50 0x7fa7d8199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dfa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d8106070 0x7fa7d8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dfa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d819dd00 0x7fa7d819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dfa00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d819e780 con 0x7fa7d8100f50 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dfa00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d819e8f0 con 0x7fa7d819dd00 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dfa00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d819eab0 con 0x7fa7d8106070 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dd000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7d8100f50 0x7fa7d8199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dda00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d819dd00 0x7fa7d819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dd000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7d8100f50 0x7fa7d8199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35810/0 (socket says 172.21.15.67:35810) 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dd000700 1 -- 172.21.15.67:0/270568718 learned_addr learned my addr 172.21.15.67:0/270568718 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d8106070 0x7fa7d8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dd000700 1 -- 172.21.15.67:0/270568718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d8106070 msgr2=0x7fa7d8199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dd000700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d8106070 0x7fa7d8199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dd000700 1 -- 172.21.15.67:0/270568718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d819dd00 msgr2=0x7fa7d819e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dd000700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d819dd00 0x7fa7d819e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dd000700 1 -- 172.21.15.67:0/270568718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc0095f0 con 0x7fa7d8100f50 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.040+0000 7fa7dda00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d819dd00 0x7fa7d819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.041+0000 7fa7dd000700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7d8100f50 0x7fa7d8199420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.041+0000 7fa7d5600700 1 -- 172.21.15.67:0/270568718 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc01d070 con 0x7fa7d8100f50 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.041+0000 7fa7d5600700 1 -- 172.21.15.67:0/270568718 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7cc022470 con 0x7fa7d8100f50 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.041+0000 7fa7dfa00700 1 -- 172.21.15.67:0/270568718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d81a5fb0 con 0x7fa7d8100f50 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.041+0000 7fa7d5600700 1 -- 172.21.15.67:0/270568718 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc017650 con 0x7fa7d8100f50 2024-03-30T02:42:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.041+0000 7fa7dfa00700 1 -- 172.21.15.67:0/270568718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d81a63e0 con 0x7fa7d8100f50 2024-03-30T02:42:25.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.046+0000 7fa7d5600700 1 -- 172.21.15.67:0/270568718 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa7cc0225e0 con 0x7fa7d8100f50 2024-03-30T02:42:25.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.047+0000 7fa7d5600700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7c00a1310 0x7fa7c00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.047+0000 7fa7d5600700 1 -- 172.21.15.67:0/270568718 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa7cc0ce260 con 0x7fa7d8100f50 2024-03-30T02:42:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.047+0000 7fa7f3429b80 1 Processor -- start 2024-03-30T02:42:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.047+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7c00a1310 0x7fa7c00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.047+0000 7fa7f3429b80 1 -- start start 2024-03-30T02:42:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a8fef0 0x5609e8a9a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a9adf0 0x5609e8a9d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e89e8990 con 0x5609e8a8fef0 2024-03-30T02:42:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e89e6350 con 0x5609e8a8f5d0 2024-03-30T02:42:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e89e1c40 con 0x5609e8a9adf0 2024-03-30T02:42:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7dd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7dd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:36772/0 (socket says 172.21.15.67:36772) 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7dd000700 1 -- 172.21.15.67:0/842476265 learned_addr learned my addr 172.21.15.67:0/842476265 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/842476265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a8fef0 0x5609e8a9a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7dda00700 1 --2- 172.21.15.67:0/842476265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a9adf0 0x5609e8a9d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7dd000700 1 -- 172.21.15.67:0/842476265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a9adf0 msgr2=0x5609e8a9d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7dd000700 1 --2- 172.21.15.67:0/842476265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a9adf0 0x5609e8a9d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7dd000700 1 -- 172.21.15.67:0/842476265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a8fef0 msgr2=0x5609e8a9a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7dd000700 1 --2- 172.21.15.67:0/842476265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a8fef0 0x5609e8a9a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7c00a1310 0x7fa7c00a37d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7d819a970 tx=0x7fa7c400b410 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.048+0000 7fa7dd000700 1 -- 172.21.15.67:0/842476265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e88633b0 con 0x5609e8a8f5d0 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.049+0000 7fa7dd000700 1 --2- 172.21.15.67:0/842476265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc09adb0 tx=0x7fa7cc09af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4171292dd2f5d69 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.049+0000 7fa7ca000700 1 -- 172.21.15.67:0/842476265 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc09b7b0 con 0x5609e8a8f5d0 2024-03-30T02:42:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.049+0000 7fa7ca000700 1 -- 172.21.15.67:0/842476265 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7cc09b0b0 con 0x5609e8a8f5d0 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.049+0000 7fa7ca000700 1 -- 172.21.15.67:0/842476265 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0a1070 con 0x5609e8a8f5d0 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.049+0000 7fa7f3429b80 1 -- 172.21.15.67:0/842476265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 msgr2=0x5609e8a8f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.049+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/842476265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc09adb0 tx=0x7fa7cc09af00 comp rx=0 tx=0).stop 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.050+0000 7fa7f3429b80 1 -- 172.21.15.67:0/842476265 shutdown_connections 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.050+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/842476265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a9adf0 0x5609e8a9d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.050+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/842476265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.050+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/842476265 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a8fef0 0x5609e8a9a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.050+0000 7fa7f3429b80 1 -- 172.21.15.67:0/842476265 >> 172.21.15.67:0/842476265 conn(0x5609e89ef6a0 msgr2=0x5609e8a8ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.050+0000 7fa7f3429b80 1 -- 172.21.15.67:0/842476265 shutdown_connections 2024-03-30T02:42:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.050+0000 7fa7f3429b80 1 -- 172.21.15.67:0/842476265 wait complete. 2024-03-30T02:42:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.050+0000 7fa7f3429b80 1 -- start start 2024-03-30T02:42:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.050+0000 7fa7f3429b80 1 Processor -- start 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a9b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 0x5609e8a9b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8aa6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e897dd80 con 0x5609e8a9adf0 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e897da20 con 0x5609e8a8f5d0 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e88656a0 con 0x5609e8a8fef0 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7f3429b80 10 client.0 _ll_register_callbacks cb 0x5609e8987470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7dd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a9b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7dda00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8aa6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7dd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a9b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:36788/0 (socket says 172.21.15.67:36788) 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7dd000700 1 -- 172.21.15.67:0/3938751317 learned_addr learned my addr 172.21.15.67:0/3938751317 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.051+0000 7fa7d7e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 0x5609e8a9b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7dd000700 1 -- 172.21.15.67:0/3938751317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 msgr2=0x5609e8a9b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7dd000700 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 0x5609e8a9b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7dd000700 1 -- 172.21.15.67:0/3938751317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 msgr2=0x5609e8aa6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7dd000700 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8aa6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7dd000700 1 -- 172.21.15.67:0/3938751317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d81a63e0 con 0x5609e8a8f5d0 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7dd000700 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a9b1d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0975b0 tx=0x7fa7cc09b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e080a020ba7812 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7bea00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7bea00700 1 -- 172.21.15.67:0/3938751317 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0a34a0 con 0x5609e8a8f5d0 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7bea00700 1 -- 172.21.15.67:0/3938751317 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa7cc0a10c0 con 0x5609e8a8f5d0 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7f3429b80 1 -- 172.21.15.67:0/3938751317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609e89a2380 con 0x5609e8a8f5d0 2024-03-30T02:42:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7bea00700 1 -- 172.21.15.67:0/3938751317 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7cc0b9250 con 0x5609e8a8f5d0 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7f3429b80 10 client.26753 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7f3429b80 1 -- 172.21.15.67:0/3938751317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5609e880fb60 con 0x5609e8a8f5d0 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.052+0000 7fa7bea00700 1 -- 172.21.15.67:0/3938751317 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609e89a2380 con 0x5609e8a8f5d0 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7bea00700 1 -- 172.21.15.67:0/3938751317 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fa7cc07b020 con 0x5609e8a8f5d0 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7f3429b80 10 client.26753 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7f3429b80 10 client.26753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7f3429b80 20 client.26753 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609e8aaa200 0x5609e8aac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7f3429b80 4 client.26753 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7f3429b80 1 -- 172.21.15.67:0/3938751317 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609e89da700 con 0x5609e8aaa200 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609e8aaa200 0x5609e8aac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609e8aaa200 0x5609e8aac5f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74db9de62949d50 server_cookie=46e71a20c12d7e38 in_seq=0 out_seq=0 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.053+0000 7fa7bea00700 10 client.26753 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.054+0000 7fa7bea00700 1 -- 172.21.15.67:0/3938751317 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5609e89da700 con 0x5609e8aaa200 2024-03-30T02:42:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.054+0000 7fa7bea00700 10 client.26753 handle_command_reply: tid=0 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.054+0000 7fa7f3429b80 1 client.26753 shutdown 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.054+0000 7fa7f3429b80 1 -- 172.21.15.67:0/3938751317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 msgr2=0x5609e8a9b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.054+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a9b1d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0975b0 tx=0x7fa7cc09b6d0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7f3429b80 1 -- 172.21.15.67:0/3938751317 shutdown_connections 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609e8aaa200 0x5609e8aac5f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8aa6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a9b1d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/3938751317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 0x5609e8a9b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7bea00700 0 client.26753 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7f3429b80 1 -- 172.21.15.67:0/3938751317 >> 172.21.15.67:0/3938751317 conn(0x5609e89ef6a0 msgr2=0x5609e8a916c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7f3429b80 1 -- 172.21.15.67:0/3938751317 shutdown_connections 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7f3429b80 1 -- 172.21.15.67:0/3938751317 wait complete. 2024-03-30T02:42:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.055+0000 7fa7f3429b80 20 client.26753 trim_cache size 0 max 16384 2024-03-30T02:42:25.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.073+0000 7fa7f3429b80 1 Processor -- start 2024-03-30T02:42:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.073+0000 7fa7f3429b80 1 -- start start 2024-03-30T02:42:25.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 0x5609e8aa01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8a95440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e8a9a6b0 con 0x5609e8a9adf0 2024-03-30T02:42:25.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e8a8d160 con 0x5609e8a8f5d0 2024-03-30T02:42:25.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e8a90850 con 0x5609e8a8fef0 2024-03-30T02:42:25.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7dd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7dda00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8a95440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7dd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:36798/0 (socket says 172.21.15.67:36798) 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7dd000700 1 -- 172.21.15.67:0/1371510196 learned_addr learned my addr 172.21.15.67:0/1371510196 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/1371510196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 0x5609e8aa01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7dd000700 1 -- 172.21.15.67:0/1371510196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 msgr2=0x5609e8aa01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7dd000700 1 --2- 172.21.15.67:0/1371510196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 0x5609e8aa01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7dd000700 1 -- 172.21.15.67:0/1371510196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 msgr2=0x5609e8a95440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7dd000700 1 --2- 172.21.15.67:0/1371510196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8a95440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.074+0000 7fa7dd000700 1 -- 172.21.15.67:0/1371510196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e89a7ea0 con 0x5609e8a8f5d0 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7dd000700 1 --2- 172.21.15.67:0/1371510196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc09adb0 tx=0x7fa7cc022b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f477bfe1f1d74ce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7caa00700 1 -- 172.21.15.67:0/1371510196 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc09abf0 con 0x5609e8a8f5d0 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7caa00700 1 -- 172.21.15.67:0/1371510196 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7cc0a34a0 con 0x5609e8a8f5d0 2024-03-30T02:42:25.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7caa00700 1 -- 172.21.15.67:0/1371510196 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0b9320 con 0x5609e8a8f5d0 2024-03-30T02:42:25.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7f3429b80 1 -- 172.21.15.67:0/1371510196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 msgr2=0x5609e8a8f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/1371510196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc09adb0 tx=0x7fa7cc022b90 comp rx=0 tx=0).stop 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7f3429b80 1 -- 172.21.15.67:0/1371510196 shutdown_connections 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/1371510196 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8a95440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/1371510196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8f5d0 0x5609e8a8f9b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/1371510196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8fef0 0x5609e8aa01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7f3429b80 1 -- 172.21.15.67:0/1371510196 >> 172.21.15.67:0/1371510196 conn(0x5609e89ef6a0 msgr2=0x5609e8a845a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.075+0000 7fa7f3429b80 1 -- 172.21.15.67:0/1371510196 shutdown_connections 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.076+0000 7fa7f3429b80 1 -- 172.21.15.67:0/1371510196 wait complete. 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.076+0000 7fa7f3429b80 1 -- start start 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.076+0000 7fa7f3429b80 1 Processor -- start 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.076+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8f5d0 0x5609e8af1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8fef0 0x5609e8af2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7f3429b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8afcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e8af2c30 con 0x5609e8a9adf0 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e8af2da0 con 0x5609e8a8fef0 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7f3429b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e8afd510 con 0x5609e8a8f5d0 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7f3429b80 10 client.0 _ll_register_callbacks cb 0x5609e8987470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7d7e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8fef0 0x5609e8af2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7d7e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8fef0 0x5609e8af2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:36800/0 (socket says 172.21.15.67:36800) 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7d7e00700 1 -- 172.21.15.67:0/4265546675 learned_addr learned my addr 172.21.15.67:0/4265546675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7dda00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8afcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7dd000700 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8f5d0 0x5609e8af1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7d7e00700 1 -- 172.21.15.67:0/4265546675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8f5d0 msgr2=0x5609e8af1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8f5d0 0x5609e8af1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7d7e00700 1 -- 172.21.15.67:0/4265546675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 msgr2=0x5609e8afcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8afcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7d7e00700 1 -- 172.21.15.67:0/4265546675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609e880fb60 con 0x5609e8a8fef0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7d7e00700 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8fef0 0x5609e8af2490 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5609e8a99950 tx=0x7fa7c4013070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceee834d5e2601a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.077+0000 7fa7bea00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7bea00700 1 -- 172.21.15.67:0/4265546675 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c402f910 con 0x5609e8a8fef0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7bea00700 1 -- 172.21.15.67:0/4265546675 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa7c4017070 con 0x5609e8a8fef0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7f3429b80 1 -- 172.21.15.67:0/4265546675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7cc09b0b0 con 0x5609e8a8fef0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7bea00700 1 -- 172.21.15.67:0/4265546675 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7c402fa70 con 0x5609e8a8fef0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7f3429b80 10 client.26759 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7f3429b80 1 -- 172.21.15.67:0/4265546675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5609e8867cd0 con 0x5609e8a8fef0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7bea00700 1 -- 172.21.15.67:0/4265546675 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7cc09b0b0 con 0x5609e8a8fef0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7bea00700 1 -- 172.21.15.67:0/4265546675 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fa7c4011c60 con 0x5609e8a8fef0 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7f3429b80 10 client.26759 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7f3429b80 10 client.26759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7f3429b80 20 client.26759 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.078+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609e8afee40 0x5609e8b01260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7f3429b80 4 client.26759 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7f3429b80 1 -- 172.21.15.67:0/4265546675 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5609e8830cf0 con 0x5609e8afee40 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7dd000700 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609e8afee40 0x5609e8b01260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7dd000700 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609e8afee40 0x5609e8b01260 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28bc6a5a310e0ac server_cookie=b3b0e6f977573621 in_seq=0 out_seq=0 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7bea00700 10 client.26759 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7bea00700 1 -- 172.21.15.67:0/4265546675 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5609e8830cf0 con 0x5609e8afee40 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7bea00700 10 client.26759 handle_command_reply: tid=0 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7f3429b80 1 client.26759 shutdown 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7f3429b80 1 -- 172.21.15.67:0/4265546675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8fef0 msgr2=0x5609e8af2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8fef0 0x5609e8af2490 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x5609e8a99950 tx=0x7fa7c4013070 comp rx=0 tx=0).stop 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7f3429b80 1 -- 172.21.15.67:0/4265546675 shutdown_connections 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.079+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5609e8afee40 0x5609e8b01260 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5609e8a9adf0 0x5609e8afcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609e8a8f5d0 0x5609e8af1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7f3429b80 1 --2- 172.21.15.67:0/4265546675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609e8a8fef0 0x5609e8af2490 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7bea00700 0 client.26759 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:25.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7f3429b80 1 -- 172.21.15.67:0/4265546675 >> 172.21.15.67:0/4265546675 conn(0x5609e89ef6a0 msgr2=0x5609e8a95440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7f3429b80 1 -- 172.21.15.67:0/4265546675 shutdown_connections 2024-03-30T02:42:25.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7f3429b80 1 -- 172.21.15.67:0/4265546675 wait complete. 2024-03-30T02:42:25.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7f3429b80 20 client.26759 trim_cache size 0 max 16384 2024-03-30T02:42:25.084 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:42:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:42:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:42:25.084 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7dfa00700 1 -- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7c00a1310 msgr2=0x7fa7c00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7c00a1310 0x7fa7c00a37d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa7d819a970 tx=0x7fa7c400b410 comp rx=0 tx=0).stop 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7dfa00700 1 -- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7d8100f50 msgr2=0x7fa7d8199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.080+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7d8100f50 0x7fa7d8199420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc005740 comp rx=0 tx=0).stop 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.081+0000 7fa7dfa00700 1 -- 172.21.15.67:0/270568718 shutdown_connections 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.081+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7c00a1310 0x7fa7c00a37d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.081+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa7d8100f50 0x7fa7d8199420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.081+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d819dd00 0x7fa7d819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.081+0000 7fa7dfa00700 1 --2- 172.21.15.67:0/270568718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa7d8106070 0x7fa7d8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.081+0000 7fa7dfa00700 1 -- 172.21.15.67:0/270568718 >> 172.21.15.67:0/270568718 conn(0x7fa7d80fc7f0 msgr2=0x7fa7d80fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.081+0000 7fa7dfa00700 1 -- 172.21.15.67:0/270568718 shutdown_connections 2024-03-30T02:42:25.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.081+0000 7fa7dfa00700 1 -- 172.21.15.67:0/270568718 wait complete. 2024-03-30T02:42:25.098 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:42:25.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:42:25.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.302+0000 7f8a35a00700 1 -- 172.21.15.67:0/2076147996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 msgr2=0x7f8a3010cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.302+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2076147996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 0x7f8a3010cf70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0098f0 tx=0x7f8a1c009c00 comp rx=0 tx=0).stop 2024-03-30T02:42:25.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.303+0000 7f8a35a00700 1 -- 172.21.15.67:0/2076147996 shutdown_connections 2024-03-30T02:42:25.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.303+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2076147996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a3010d4b0 0x7f8a3010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.303+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2076147996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a30073180 0x7f8a30073560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.303+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2076147996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 0x7f8a3010cf70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.303+0000 7f8a35a00700 1 -- 172.21.15.67:0/2076147996 >> 172.21.15.67:0/2076147996 conn(0x7f8a300fc720 msgr2=0x7f8a300feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.303+0000 7f8a35a00700 1 -- 172.21.15.67:0/2076147996 shutdown_connections 2024-03-30T02:42:25.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.303+0000 7f8a35a00700 1 -- 172.21.15.67:0/2076147996 wait complete. 2024-03-30T02:42:25.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.305+0000 7f8a35a00700 1 Processor -- start 2024-03-30T02:42:25.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.305+0000 7f8a35a00700 1 -- start start 2024-03-30T02:42:25.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.305+0000 7f8a35a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a30073180 0x7f8a30199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.305+0000 7f8a35a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 0x7f8a301999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a35a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a3010d4b0 0x7f8a3019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a35a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3019e350 con 0x7f8a30073aa0 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a35a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3019e4c0 con 0x7f8a3010d4b0 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a35a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3019e680 con 0x7f8a30073180 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 0x7f8a301999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 0x7f8a301999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35842/0 (socket says 172.21.15.67:35842) 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a3010d4b0 0x7f8a3019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2e000700 1 -- 172.21.15.67:0/2084401831 learned_addr learned my addr 172.21.15.67:0/2084401831 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2ea00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a30073180 0x7f8a30199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2f400700 1 -- 172.21.15.67:0/2084401831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a30073180 msgr2=0x7f8a30199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2f400700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a30073180 0x7f8a30199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2f400700 1 -- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 msgr2=0x7f8a301999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2f400700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 0x7f8a301999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2f400700 1 -- 172.21.15.67:0/2084401831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c009520 con 0x7f8a3010d4b0 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2ea00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a30073180 0x7f8a30199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.306+0000 7f8a2e000700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 0x7f8a301999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:25.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.307+0000 7f8a2f400700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a3010d4b0 0x7f8a3019dd10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8a28007ee0 tx=0x7f8a2800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.307+0000 7f8a27400700 1 -- 172.21.15.67:0/2084401831 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2800ddd0 con 0x7f8a3010d4b0 2024-03-30T02:42:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.307+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3019e960 con 0x7f8a3010d4b0 2024-03-30T02:42:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.307+0000 7f8a27400700 1 -- 172.21.15.67:0/2084401831 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a2800f040 con 0x7f8a3010d4b0 2024-03-30T02:42:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.307+0000 7f8a27400700 1 -- 172.21.15.67:0/2084401831 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28005a00 con 0x7f8a3010d4b0 2024-03-30T02:42:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.307+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a301a61a0 con 0x7f8a3010d4b0 2024-03-30T02:42:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.308+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a301086c0 con 0x7f8a3010d4b0 2024-03-30T02:42:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.310+0000 7f8a27400700 1 -- 172.21.15.67:0/2084401831 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8a28004ad0 con 0x7f8a3010d4b0 2024-03-30T02:42:25.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.312+0000 7f8a27400700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8a140a1290 0x7f8a140a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.312+0000 7f8a27400700 1 -- 172.21.15.67:0/2084401831 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f8a280cc9d0 con 0x7f8a3010d4b0 2024-03-30T02:42:25.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.312+0000 7f8a2ea00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8a140a1290 0x7f8a140a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.312+0000 7f8a2ea00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8a140a1290 0x7f8a140a3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005fd0 tx=0x7f8a20009310 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.314+0000 7f8a27400700 1 -- 172.21.15.67:0/2084401831 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8a28095a70 con 0x7f8a3010d4b0 2024-03-30T02:42:25.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.613+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a3019a7d0 con 0x7f8a3010d4b0 2024-03-30T02:42:25.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.614+0000 7f8a27400700 1 -- 172.21.15.67:0/2084401831 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10926 (secure 0 0 0) 0x7f8a28099090 con 0x7f8a3010d4b0 2024-03-30T02:42:25.613 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:42:25.613 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:41:57.505554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[0,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:42:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8a140a1290 msgr2=0x7f8a140a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8a140a1290 0x7f8a140a3750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005fd0 tx=0x7f8a20009310 comp rx=0 tx=0).stop 2024-03-30T02:42:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a3010d4b0 msgr2=0x7f8a3019dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a3010d4b0 0x7f8a3019dd10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8a28007ee0 tx=0x7f8a2800d3b0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 shutdown_connections 2024-03-30T02:42:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8a140a1290 0x7f8a140a3750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a3010d4b0 0x7f8a3019dd10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a30073180 0x7f8a30199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 --2- 172.21.15.67:0/2084401831 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8a30073aa0 0x7f8a301999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 >> 172.21.15.67:0/2084401831 conn(0x7f8a300fc720 msgr2=0x7f8a300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.616+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 shutdown_connections 2024-03-30T02:42:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.617+0000 7f8a35a00700 1 -- 172.21.15.67:0/2084401831 wait complete. 2024-03-30T02:42:25.617 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 33 2024-03-30T02:42:25.636 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:42:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.850+0000 7f1ad9200700 1 -- 172.21.15.67:0/758374907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4101910 msgr2=0x7f1ad4105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.850+0000 7f1ad9200700 1 --2- 172.21.15.67:0/758374907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4101910 0x7f1ad4105950 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009990 tx=0x7f1ac0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.850+0000 7f1ad9200700 1 -- 172.21.15.67:0/758374907 shutdown_connections 2024-03-30T02:42:25.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.850+0000 7f1ad9200700 1 --2- 172.21.15.67:0/758374907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4100f60 0x7f1ad4101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.850+0000 7f1ad9200700 1 --2- 172.21.15.67:0/758374907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ad4106080 0x7f1ad410dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.850+0000 7f1ad9200700 1 --2- 172.21.15.67:0/758374907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4101910 0x7f1ad4105950 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.850+0000 7f1ad9200700 1 -- 172.21.15.67:0/758374907 >> 172.21.15.67:0/758374907 conn(0x7f1ad40fc7c0 msgr2=0x7f1ad40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.850+0000 7f1ad9200700 1 -- 172.21.15.67:0/758374907 shutdown_connections 2024-03-30T02:42:25.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.850+0000 7f1ad9200700 1 -- 172.21.15.67:0/758374907 wait complete. 2024-03-30T02:42:25.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.851+0000 7f1ad9200700 1 Processor -- start 2024-03-30T02:42:25.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad9200700 1 -- start start 2024-03-30T02:42:25.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad9200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4100f60 0x7f1ad4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad9200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4106080 0x7f1ad4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad9200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ad419dce0 0x7f1ad419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad9200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419e760 con 0x7f1ad4100f60 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad9200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419e8d0 con 0x7f1ad4106080 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad9200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419ea90 con 0x7f1ad419dce0 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4106080 0x7f1ad4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4100f60 0x7f1ad4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4106080 0x7f1ad4199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39350/0 (socket says 172.21.15.67:39350) 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad1400700 1 -- 172.21.15.67:0/2941813908 learned_addr learned my addr 172.21.15.67:0/2941813908 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.852+0000 7f1ad2800700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ad419dce0 0x7f1ad419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1ad1400700 1 -- 172.21.15.67:0/2941813908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ad419dce0 msgr2=0x7f1ad419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1ad1400700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ad419dce0 0x7f1ad419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1ad1400700 1 -- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4100f60 msgr2=0x7f1ad4199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1ad1400700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4100f60 0x7f1ad4199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1ad1400700 1 -- 172.21.15.67:0/2941813908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00095f0 con 0x7f1ad4106080 2024-03-30T02:42:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1ad1e00700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4100f60 0x7f1ad4199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1ad1400700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4106080 0x7f1ad4199940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00059d0 tx=0x7f1ac0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1acaa00700 1 -- 172.21.15.67:0/2941813908 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac001d070 con 0x7f1ad4106080 2024-03-30T02:42:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1acaa00700 1 -- 172.21.15.67:0/2941813908 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac0022470 con 0x7f1ad4106080 2024-03-30T02:42:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1acaa00700 1 -- 172.21.15.67:0/2941813908 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0017650 con 0x7f1ad4106080 2024-03-30T02:42:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.853+0000 7f1ad9200700 1 -- 172.21.15.67:0/2941813908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad41a5f90 con 0x7f1ad4106080 2024-03-30T02:42:25.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.854+0000 7f1ad9200700 1 -- 172.21.15.67:0/2941813908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad41a63b0 con 0x7f1ad4106080 2024-03-30T02:42:25.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.857+0000 7f1acaa00700 1 -- 172.21.15.67:0/2941813908 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1ac00225e0 con 0x7f1ad4106080 2024-03-30T02:42:25.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.858+0000 7f1aece0fb80 1 Processor -- start 2024-03-30T02:42:25.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.858+0000 7f1acaa00700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1ab80a1310 0x7f1ab80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.858+0000 7f1aece0fb80 1 -- start start 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.858+0000 7f1acaa00700 1 -- 172.21.15.67:0/2941813908 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1ac00ce390 con 0x7f1ad4106080 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad2800700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1ab80a1310 0x7f1ab80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9c5d0 0x561768f9c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9cef0 0x561768fa78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768faa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768ef5990 con 0x561768f9cef0 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768ef3350 con 0x561768f9c5d0 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768eeec40 con 0x561768fa7df0 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9c5d0 0x561768f9c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9c5d0 0x561768f9c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39362/0 (socket says 172.21.15.67:39362) 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad1e00700 1 -- 172.21.15.67:0/877198999 learned_addr learned my addr 172.21.15.67:0/877198999 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad1400700 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9cef0 0x561768fa78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad2800700 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768faa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad1e00700 1 -- 172.21.15.67:0/877198999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 msgr2=0x561768faa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad1e00700 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768faa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad1e00700 1 -- 172.21.15.67:0/877198999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9cef0 msgr2=0x561768fa78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad1e00700 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9cef0 0x561768fa78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.859+0000 7f1ad1e00700 1 -- 172.21.15.67:0/877198999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561768d703b0 con 0x561768f9c5d0 2024-03-30T02:42:25.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.860+0000 7f1ad2800700 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768faa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:42:25.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.860+0000 7f1ad2800700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1ab80a1310 0x7f1ab80a37d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009550 tx=0x7f1acc007500 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.860+0000 7f1ad1e00700 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9c5d0 0x561768f9c9b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40115f0 tx=0x7f1ac4011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994e02db3c058ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.860+0000 7f1abf400700 1 -- 172.21.15.67:0/877198999 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400f680 con 0x561768f9c5d0 2024-03-30T02:42:25.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.860+0000 7f1abf400700 1 -- 172.21.15.67:0/877198999 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac400fca0 con 0x561768f9c5d0 2024-03-30T02:42:25.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.860+0000 7f1abf400700 1 -- 172.21.15.67:0/877198999 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4019d10 con 0x561768f9c5d0 2024-03-30T02:42:25.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.861+0000 7f1aece0fb80 1 -- 172.21.15.67:0/877198999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9c5d0 msgr2=0x561768f9c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.861+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9c5d0 0x561768f9c9b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40115f0 tx=0x7f1ac4011900 comp rx=0 tx=0).stop 2024-03-30T02:42:25.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.861+0000 7f1aece0fb80 1 -- 172.21.15.67:0/877198999 shutdown_connections 2024-03-30T02:42:25.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.861+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768faa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.861+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9c5d0 0x561768f9c9b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.861+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/877198999 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9cef0 0x561768fa78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.861+0000 7f1aece0fb80 1 -- 172.21.15.67:0/877198999 >> 172.21.15.67:0/877198999 conn(0x561768efc6a0 msgr2=0x561768f98a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.861+0000 7f1aece0fb80 1 -- 172.21.15.67:0/877198999 shutdown_connections 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.861+0000 7f1aece0fb80 1 -- 172.21.15.67:0/877198999 wait complete. 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.862+0000 7f1aece0fb80 1 -- start start 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.862+0000 7f1aece0fb80 1 Processor -- start 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.862+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768fa77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.862+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 0x561768fa4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.862+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561768fa50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.862+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768e8ad80 con 0x561768f9c5d0 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.862+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768e8aa20 con 0x561768fa7df0 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.862+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768d726a0 con 0x561768f9cef0 2024-03-30T02:42:25.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.862+0000 7f1aece0fb80 10 client.0 _ll_register_callbacks cb 0x561768e94470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561768fa50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768fa77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561768fa50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39370/0 (socket says 172.21.15.67:39370) 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad2800700 1 -- 172.21.15.67:0/1481937932 learned_addr learned my addr 172.21.15.67:0/1481937932 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad1400700 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 0x561768fa4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad2800700 1 -- 172.21.15.67:0/1481937932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 msgr2=0x561768fa4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad2800700 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 0x561768fa4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad2800700 1 -- 172.21.15.67:0/1481937932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 msgr2=0x561768fa77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad2800700 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768fa77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad2800700 1 -- 172.21.15.67:0/1481937932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac4009dd0 con 0x561768fa7df0 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ad2800700 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561768fa50f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1acc01bc20 tx=0x7f1acc004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb38e1cf5a01d56d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.863+0000 7f1ab3e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1ab3e00700 1 -- 172.21.15.67:0/1481937932 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc016890 con 0x561768fa7df0 2024-03-30T02:42:25.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1ab3e00700 1 -- 172.21.15.67:0/1481937932 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1acc0169f0 con 0x561768fa7df0 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1aece0fb80 1 -- 172.21.15.67:0/1481937932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561768eaf380 con 0x561768fa7df0 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1ab3e00700 1 -- 172.21.15.67:0/1481937932 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1acc016bb0 con 0x561768fa7df0 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1aece0fb80 10 client.26777 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1aece0fb80 1 -- 172.21.15.67:0/1481937932 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561768d1cb60 con 0x561768fa7df0 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1ab3e00700 1 -- 172.21.15.67:0/1481937932 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561768eaf380 con 0x561768fa7df0 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1ab3e00700 1 -- 172.21.15.67:0/1481937932 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f1acc018400 con 0x561768fa7df0 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1aece0fb80 10 client.26777 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.864+0000 7f1aece0fb80 10 client.26777 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.865+0000 7f1aece0fb80 20 client.26777 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.865+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x561768fb7980 0x561768fb9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.865+0000 7f1aece0fb80 4 client.26777 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.865+0000 7f1aece0fb80 1 -- 172.21.15.67:0/1481937932 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561768ee7700 con 0x561768fb7980 2024-03-30T02:42:25.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.865+0000 7f1ad1e00700 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x561768fb7980 0x561768fb9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.866+0000 7f1ad1e00700 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x561768fb7980 0x561768fb9da0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd0e6bb2074b0d29 server_cookie=aaaa1177dc1c30f0 in_seq=0 out_seq=0 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.866+0000 7f1ab3e00700 10 client.26777 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.866+0000 7f1ab3e00700 1 -- 172.21.15.67:0/1481937932 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561768ee7700 con 0x561768fb7980 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.866+0000 7f1ab3e00700 10 client.26777 handle_command_reply: tid=0 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 client.26777 shutdown 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 -- 172.21.15.67:0/1481937932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 msgr2=0x561768fa50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561768fa50f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1acc01bc20 tx=0x7f1acc004b40 comp rx=0 tx=0).stop 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 -- 172.21.15.67:0/1481937932 shutdown_connections 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768fa77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x561768fb7980 0x561768fb9da0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561768fa50f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/1481937932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 0x561768fa4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 -- 172.21.15.67:0/1481937932 >> 172.21.15.67:0/1481937932 conn(0x561768efc6a0 msgr2=0x561768f9e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1ab3e00700 0 client.26777 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:42:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 -- 172.21.15.67:0/1481937932 shutdown_connections 2024-03-30T02:42:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 1 -- 172.21.15.67:0/1481937932 wait complete. 2024-03-30T02:42:25.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.867+0000 7f1aece0fb80 20 client.26777 trim_cache size 0 max 16384 2024-03-30T02:42:25.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.883+0000 7f1aece0fb80 1 Processor -- start 2024-03-30T02:42:25.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.883+0000 7f1aece0fb80 1 -- start start 2024-03-30T02:42:25.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.883+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768faeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.883+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9cef0 0x561768fa33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.883+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768fa3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.883+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768fa3e80 con 0x561768f9c5d0 2024-03-30T02:42:25.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.883+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768fa6090 con 0x561768f9cef0 2024-03-30T02:42:25.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.883+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561768faefe0 con 0x561768fa7df0 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9cef0 0x561768fa33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768faeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9cef0 0x561768fa33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39372/0 (socket says 172.21.15.67:39372) 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1400700 1 -- 172.21.15.67:0/7358858 learned_addr learned my addr 172.21.15.67:0/7358858 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad2800700 1 --2- 172.21.15.67:0/7358858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768fa3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1400700 1 -- 172.21.15.67:0/7358858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 msgr2=0x561768fa3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1400700 1 --2- 172.21.15.67:0/7358858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768fa3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1400700 1 -- 172.21.15.67:0/7358858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 msgr2=0x561768faeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1400700 1 --2- 172.21.15.67:0/7358858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768faeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1400700 1 -- 172.21.15.67:0/7358858 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561768eb4ea0 con 0x561768f9cef0 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad2800700 1 --2- 172.21.15.67:0/7358858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768fa3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.884+0000 7f1ad1400700 1 --2- 172.21.15.67:0/7358858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9cef0 0x561768fa33d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1ac009ae80 tx=0x7f1ac0022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b8e25c0b46a114d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1abfe00700 1 -- 172.21.15.67:0/7358858 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac009b880 con 0x561768f9cef0 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1abfe00700 1 -- 172.21.15.67:0/7358858 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac009b180 con 0x561768f9cef0 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1abfe00700 1 -- 172.21.15.67:0/7358858 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a1070 con 0x561768f9cef0 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1aece0fb80 1 -- 172.21.15.67:0/7358858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9cef0 msgr2=0x561768fa33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/7358858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9cef0 0x561768fa33d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1ac009ae80 tx=0x7f1ac0022f90 comp rx=0 tx=0).stop 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1aece0fb80 1 -- 172.21.15.67:0/7358858 shutdown_connections 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/7358858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768faeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/7358858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768fa7df0 0x561768fa3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/7358858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768f9cef0 0x561768fa33d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1aece0fb80 1 -- 172.21.15.67:0/7358858 >> 172.21.15.67:0/7358858 conn(0x561768efc6a0 msgr2=0x561768f9fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1aece0fb80 1 -- 172.21.15.67:0/7358858 shutdown_connections 2024-03-30T02:42:25.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.885+0000 7f1aece0fb80 1 -- 172.21.15.67:0/7358858 wait complete. 2024-03-30T02:42:25.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.886+0000 7f1aece0fb80 1 -- start start 2024-03-30T02:42:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.886+0000 7f1aece0fb80 1 Processor -- start 2024-03-30T02:42:25.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768facc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 0x561768fad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1aece0fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561769004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561769004cc0 con 0x561768f9c5d0 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561769004e30 con 0x561768fa7df0 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1aece0fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561769004ff0 con 0x561768f9cef0 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1aece0fb80 10 client.0 _ll_register_callbacks cb 0x561768e94470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561769004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768facc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561769004780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39378/0 (socket says 172.21.15.67:39378) 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad2800700 1 -- 172.21.15.67:0/256615851 learned_addr learned my addr 172.21.15.67:0/256615851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad1e00700 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768facc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53352/0 (socket says 172.21.15.67:53352) 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad1400700 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 0x561768fad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad2800700 1 -- 172.21.15.67:0/256615851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 msgr2=0x561768fad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad2800700 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 0x561768fad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad2800700 1 -- 172.21.15.67:0/256615851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 msgr2=0x561768facc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad2800700 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768facc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.887+0000 7f1ad2800700 1 -- 172.21.15.67:0/256615851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad41a63b0 con 0x561768fa7df0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.888+0000 7f1ad2800700 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561769004780 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1acc018760 tx=0x7f1acc0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26ce4ad8c824afc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.888+0000 7f1ab3e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.888+0000 7f1ab3e00700 1 -- 172.21.15.67:0/256615851 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc018a50 con 0x561768fa7df0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.888+0000 7f1aece0fb80 1 -- 172.21.15.67:0/256615851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ac400fca0 con 0x561768fa7df0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.888+0000 7f1ab3e00700 1 -- 172.21.15.67:0/256615851 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f1acc0168f0 con 0x561768fa7df0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.888+0000 7f1ab3e00700 1 -- 172.21.15.67:0/256615851 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1acc016ab0 con 0x561768fa7df0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.888+0000 7f1aece0fb80 10 client.26783 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.888+0000 7f1aece0fb80 1 -- 172.21.15.67:0/256615851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561768d74cd0 con 0x561768fa7df0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.889+0000 7f1ab3e00700 1 -- 172.21.15.67:0/256615851 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac400fca0 con 0x561768fa7df0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.889+0000 7f1ab3e00700 1 -- 172.21.15.67:0/256615851 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f1acc02b040 con 0x561768fa7df0 2024-03-30T02:42:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.889+0000 7f1aece0fb80 10 client.26783 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.889+0000 7f1aece0fb80 10 client.26783 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.889+0000 7f1aece0fb80 20 client.26783 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.889+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56176900c890 0x56176900ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.889+0000 7f1aece0fb80 4 client.26783 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.889+0000 7f1aece0fb80 1 -- 172.21.15.67:0/256615851 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561768d3dcf0 con 0x56176900c890 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.889+0000 7f1ad1e00700 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56176900c890 0x56176900ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1ad1e00700 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56176900c890 0x56176900ecb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=966741afd07872b2 server_cookie=ae8094cfbd53f2aa in_seq=0 out_seq=0 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1ab3e00700 10 client.26783 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1ab3e00700 1 -- 172.21.15.67:0/256615851 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561768d3dcf0 con 0x56176900c890 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1ab3e00700 10 client.26783 handle_command_reply: tid=0 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 client.26783 shutdown 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 -- 172.21.15.67:0/256615851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 msgr2=0x561769004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561769004780 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1acc018760 tx=0x7f1acc0095f0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 -- 172.21.15.67:0/256615851 shutdown_connections 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561768f9c5d0 0x561768facc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56176900c890 0x56176900ecb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561768fa7df0 0x561769004780 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1ab3e00700 0 client.26783 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 --2- 172.21.15.67:0/256615851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561768f9cef0 0x561768fad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 -- 172.21.15.67:0/256615851 >> 172.21.15.67:0/256615851 conn(0x561768efc6a0 msgr2=0x561768fa2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.890+0000 7f1aece0fb80 1 -- 172.21.15.67:0/256615851 shutdown_connections 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1aece0fb80 1 -- 172.21.15.67:0/256615851 wait complete. 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1aece0fb80 20 client.26783 trim_cache size 0 max 16384 2024-03-30T02:42:25.894 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:42:25.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 -- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1ab80a1310 msgr2=0x7f1ab80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1ab80a1310 0x7f1ab80a37d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009550 tx=0x7f1acc007500 comp rx=0 tx=0).stop 2024-03-30T02:42:25.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 -- 172.21.15.67:0/2941813908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4106080 msgr2=0x7f1ad4199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:25.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4106080 0x7f1ad4199940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00059d0 tx=0x7f1ac0004730 comp rx=0 tx=0).stop 2024-03-30T02:42:25.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 -- 172.21.15.67:0/2941813908 shutdown_connections 2024-03-30T02:42:25.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f1ab80a1310 0x7f1ab80a37d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1ad4100f60 0x7f1ad4199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ad419dce0 0x7f1ad419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 --2- 172.21.15.67:0/2941813908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ad4106080 0x7f1ad4199940 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.891+0000 7f1ad9200700 1 -- 172.21.15.67:0/2941813908 >> 172.21.15.67:0/2941813908 conn(0x7f1ad40fc7c0 msgr2=0x7f1ad40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.892+0000 7f1ad9200700 1 -- 172.21.15.67:0/2941813908 shutdown_connections 2024-03-30T02:42:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:25.892+0000 7f1ad9200700 1 -- 172.21.15.67:0/2941813908 wait complete. 2024-03-30T02:42:25.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:42:26.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.131+0000 7f8923600700 1 -- 172.21.15.67:0/1105269446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f891c100f70 msgr2=0x7f891c101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.131+0000 7f8923600700 1 --2- 172.21.15.67:0/1105269446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f891c100f70 0x7f891c101350 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8910009990 tx=0x7f8910009ca0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.131+0000 7f8923600700 1 -- 172.21.15.67:0/1105269446 shutdown_connections 2024-03-30T02:42:26.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.131+0000 7f8923600700 1 --2- 172.21.15.67:0/1105269446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f891c106090 0x7f891c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.131+0000 7f8923600700 1 --2- 172.21.15.67:0/1105269446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f891c100f70 0x7f891c101350 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.131+0000 7f8923600700 1 --2- 172.21.15.67:0/1105269446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891c101920 0x7f891c105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.131+0000 7f8923600700 1 -- 172.21.15.67:0/1105269446 >> 172.21.15.67:0/1105269446 conn(0x7f891c0fc7f0 msgr2=0x7f891c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.131+0000 7f8923600700 1 -- 172.21.15.67:0/1105269446 shutdown_connections 2024-03-30T02:42:26.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.131+0000 7f8923600700 1 -- 172.21.15.67:0/1105269446 wait complete. 2024-03-30T02:42:26.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.133+0000 7f8923600700 1 Processor -- start 2024-03-30T02:42:26.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.133+0000 7f8923600700 1 -- start start 2024-03-30T02:42:26.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.133+0000 7f8923600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f891c100f70 0x7f891c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.133+0000 7f8923600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f891c106090 0x7f891c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.133+0000 7f8923600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891c074d70 0x7f891c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.133+0000 7f8923600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c1a9e10 con 0x7f891c074d70 2024-03-30T02:42:26.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.133+0000 7f8923600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c1a9f80 con 0x7f891c106090 2024-03-30T02:42:26.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.133+0000 7f8923600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c1aa140 con 0x7f891c100f70 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8921600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891c074d70 0x7f891c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8920c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f891c100f70 0x7f891c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8920c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f891c100f70 0x7f891c0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52330/0 (socket says 172.21.15.67:52330) 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8920c00700 1 -- 172.21.15.67:0/3270780908 learned_addr learned my addr 172.21.15.67:0/3270780908 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f891be00700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f891c106090 0x7f891c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8920c00700 1 -- 172.21.15.67:0/3270780908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f891c106090 msgr2=0x7f891c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8920c00700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f891c106090 0x7f891c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8920c00700 1 -- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891c074d70 msgr2=0x7f891c073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8920c00700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891c074d70 0x7f891c073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8920c00700 1 -- 172.21.15.67:0/3270780908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100095f0 con 0x7f891c100f70 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8921600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891c074d70 0x7f891c073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.134+0000 7f8920c00700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f891c100f70 0x7f891c0730b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8910000c00 tx=0x7f8910005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.135+0000 7f8919600700 1 -- 172.21.15.67:0/3270780908 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891001d070 con 0x7f891c100f70 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.135+0000 7f8919600700 1 -- 172.21.15.67:0/3270780908 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8910022470 con 0x7f891c100f70 2024-03-30T02:42:26.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.135+0000 7f8923600700 1 -- 172.21.15.67:0/3270780908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c1aa3c0 con 0x7f891c100f70 2024-03-30T02:42:26.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.135+0000 7f8919600700 1 -- 172.21.15.67:0/3270780908 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910017650 con 0x7f891c100f70 2024-03-30T02:42:26.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.135+0000 7f8923600700 1 -- 172.21.15.67:0/3270780908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c1aa890 con 0x7f891c100f70 2024-03-30T02:42:26.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.138+0000 7f8919600700 1 -- 172.21.15.67:0/3270780908 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f89100225e0 con 0x7f891c100f70 2024-03-30T02:42:26.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.139+0000 7f8937219b80 1 Processor -- start 2024-03-30T02:42:26.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.139+0000 7f8937219b80 1 -- start start 2024-03-30T02:42:26.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.139+0000 7f8919600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89040a1310 0x7f89040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.139+0000 7f8919600700 1 -- 172.21.15.67:0/3270780908 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f89100ce1b0 con 0x7f891c100f70 2024-03-30T02:42:26.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f8937219b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 0x563d38eb79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f8937219b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb7ef0 0x563d38ec28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f8921600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89040a1310 0x7f89040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f8937219b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38ec2df0 0x563d38ec51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f8937219b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38e10990 con 0x563d38eb75d0 2024-03-30T02:42:26.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f8937219b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38e0e350 con 0x563d38eb7ef0 2024-03-30T02:42:26.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f8937219b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38e09c40 con 0x563d38ec2df0 2024-03-30T02:42:26.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f891be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb7ef0 0x563d38ec28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f8920c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 0x563d38eb79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f891be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb7ef0 0x563d38ec28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39404/0 (socket says 172.21.15.67:39404) 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f891be00700 1 -- 172.21.15.67:0/133941313 learned_addr learned my addr 172.21.15.67:0/133941313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f8921600700 1 --2- 172.21.15.67:0/133941313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38ec2df0 0x563d38ec51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f891be00700 1 -- 172.21.15.67:0/133941313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38ec2df0 msgr2=0x563d38ec51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f891be00700 1 --2- 172.21.15.67:0/133941313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38ec2df0 0x563d38ec51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f891be00700 1 -- 172.21.15.67:0/133941313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 msgr2=0x563d38eb79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f891be00700 1 --2- 172.21.15.67:0/133941313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 0x563d38eb79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.140+0000 7f891be00700 1 -- 172.21.15.67:0/133941313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d38c8b3b0 con 0x563d38eb7ef0 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.141+0000 7f8921600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89040a1310 0x7f89040a37d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8914003fd0 tx=0x7f89140041f0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.141+0000 7f891be00700 1 --2- 172.21.15.67:0/133941313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb7ef0 0x563d38ec28b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8908009cb0 tx=0x7f8908011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bed98a9e7cddfe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.141+0000 7f890dc00700 1 -- 172.21.15.67:0/133941313 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908011d80 con 0x563d38eb7ef0 2024-03-30T02:42:26.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.141+0000 7f890dc00700 1 -- 172.21.15.67:0/133941313 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8908004510 con 0x563d38eb7ef0 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.141+0000 7f890dc00700 1 -- 172.21.15.67:0/133941313 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080193e0 con 0x563d38eb7ef0 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 -- 172.21.15.67:0/133941313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb7ef0 msgr2=0x563d38ec28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 --2- 172.21.15.67:0/133941313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb7ef0 0x563d38ec28b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8908009cb0 tx=0x7f8908011440 comp rx=0 tx=0).stop 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 -- 172.21.15.67:0/133941313 shutdown_connections 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 --2- 172.21.15.67:0/133941313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 0x563d38eb79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 --2- 172.21.15.67:0/133941313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38ec2df0 0x563d38ec51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 --2- 172.21.15.67:0/133941313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb7ef0 0x563d38ec28b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 -- 172.21.15.67:0/133941313 >> 172.21.15.67:0/133941313 conn(0x563d38e176a0 msgr2=0x563d38eb3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 -- 172.21.15.67:0/133941313 shutdown_connections 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 -- 172.21.15.67:0/133941313 wait complete. 2024-03-30T02:42:26.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.142+0000 7f8937219b80 1 -- start start 2024-03-30T02:42:26.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.143+0000 7f8937219b80 1 Processor -- start 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.143+0000 7f8937219b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb75d0 0x563d38ec2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.144+0000 7f8937219b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb7ef0 0x563d38ebfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.144+0000 7f8937219b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.144+0000 7f8937219b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38da5d80 con 0x563d38eb7ef0 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.144+0000 7f8937219b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38da5a20 con 0x563d38ec2df0 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.144+0000 7f8937219b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38c8d6a0 con 0x563d38eb75d0 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.144+0000 7f8937219b80 10 client.0 _ll_register_callbacks cb 0x563d38daf470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.144+0000 7f8920c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb75d0 0x563d38ec2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f8920c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb75d0 0x563d38ec2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52342/0 (socket says 172.21.15.67:52342) 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f8920c00700 1 -- 172.21.15.67:0/3024811401 learned_addr learned my addr 172.21.15.67:0/3024811401 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f891be00700 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb7ef0 0x563d38ebfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f8920c00700 1 -- 172.21.15.67:0/3024811401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 msgr2=0x563d38ec02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f8921600700 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f8920c00700 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f8920c00700 1 -- 172.21.15.67:0/3024811401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb7ef0 msgr2=0x563d38ebfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f8920c00700 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb7ef0 0x563d38ebfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f8920c00700 1 -- 172.21.15.67:0/3024811401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89080098e0 con 0x563d38eb75d0 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.145+0000 7f8921600700 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.146+0000 7f8920c00700 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb75d0 0x563d38ec2840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89100cb610 tx=0x7f891009aef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a5df77414e830b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.146+0000 7f8902000700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.146+0000 7f8902000700 1 -- 172.21.15.67:0/3024811401 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89100b2070 con 0x563d38eb75d0 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.146+0000 7f8902000700 1 -- 172.21.15.67:0/3024811401 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f891009b820 con 0x563d38eb75d0 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.146+0000 7f8937219b80 1 -- 172.21.15.67:0/3024811401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d38dca380 con 0x563d38eb75d0 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.146+0000 7f8902000700 1 -- 172.21.15.67:0/3024811401 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f891009d070 con 0x563d38eb75d0 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.146+0000 7f8937219b80 10 client.26746 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.146+0000 7f8937219b80 1 -- 172.21.15.67:0/3024811401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563d38c37b60 con 0x563d38eb75d0 2024-03-30T02:42:26.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.147+0000 7f8902000700 1 -- 172.21.15.67:0/3024811401 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d38dca380 con 0x563d38eb75d0 2024-03-30T02:42:26.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.147+0000 7f8902000700 1 -- 172.21.15.67:0/3024811401 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f891009b380 con 0x563d38eb75d0 2024-03-30T02:42:26.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.147+0000 7f8937219b80 10 client.26746 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:26.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.147+0000 7f8937219b80 10 client.26746 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:42:26.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.147+0000 7f8937219b80 20 client.26746 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:26.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.147+0000 7f8937219b80 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563d38ed2600 0x563d38ed4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.147+0000 7f8937219b80 4 client.26746 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:42:26.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.147+0000 7f8937219b80 1 -- 172.21.15.67:0/3024811401 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d38e02700 con 0x563d38ed2600 2024-03-30T02:42:26.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.147+0000 7f891be00700 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563d38ed2600 0x563d38ed4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.148+0000 7f891be00700 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563d38ed2600 0x563d38ed4a20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ac2128d4b9884f9 server_cookie=3ac2af671c0969fe in_seq=0 out_seq=0 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.148+0000 7f8902000700 10 client.26746 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.148+0000 7f8902000700 1 -- 172.21.15.67:0/3024811401 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d38e02700 con 0x563d38ed2600 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.148+0000 7f8902000700 10 client.26746 handle_command_reply: tid=0 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.148+0000 7f8937219b80 1 client.26746 shutdown 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 -- 172.21.15.67:0/3024811401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb75d0 msgr2=0x563d38ec2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb75d0 0x563d38ec2840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89100cb610 tx=0x7f891009aef0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 -- 172.21.15.67:0/3024811401 shutdown_connections 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563d38ed2600 0x563d38ed4a20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8902000700 0 client.26746 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb75d0 0x563d38ec2840 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 --2- 172.21.15.67:0/3024811401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb7ef0 0x563d38ebfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 -- 172.21.15.67:0/3024811401 >> 172.21.15.67:0/3024811401 conn(0x563d38e176a0 msgr2=0x563d38eb96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 -- 172.21.15.67:0/3024811401 shutdown_connections 2024-03-30T02:42:26.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 1 -- 172.21.15.67:0/3024811401 wait complete. 2024-03-30T02:42:26.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.149+0000 7f8937219b80 20 client.26746 trim_cache size 0 max 16384 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8937219b80 1 Processor -- start 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8937219b80 1 -- start start 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8937219b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 0x563d38ed49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8937219b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 0x563d38ec9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8937219b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8937219b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38ec1130 con 0x563d38eb75d0 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8937219b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38ec08c0 con 0x563d38ec2df0 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8937219b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38eb9140 con 0x563d38eb7ef0 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8921600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8920c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 0x563d38ed49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8921600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39418/0 (socket says 172.21.15.67:39418) 2024-03-30T02:42:26.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.164+0000 7f8921600700 1 -- 172.21.15.67:0/1547485756 learned_addr learned my addr 172.21.15.67:0/1547485756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f891be00700 1 --2- 172.21.15.67:0/1547485756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 0x563d38ec9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f8921600700 1 -- 172.21.15.67:0/1547485756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 msgr2=0x563d38ec9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f8921600700 1 --2- 172.21.15.67:0/1547485756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 0x563d38ec9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f8921600700 1 -- 172.21.15.67:0/1547485756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 msgr2=0x563d38ed49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f8921600700 1 --2- 172.21.15.67:0/1547485756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 0x563d38ed49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f8921600700 1 -- 172.21.15.67:0/1547485756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d38dcfea0 con 0x563d38ec2df0 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f8920c00700 1 --2- 172.21.15.67:0/1547485756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 0x563d38ed49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f8921600700 1 --2- 172.21.15.67:0/1547485756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec2370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f891400ed00 tx=0x7f891401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef5aec74b78636b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f890e600700 1 -- 172.21.15.67:0/1547485756 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914004db0 con 0x563d38ec2df0 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.165+0000 7f890e600700 1 -- 172.21.15.67:0/1547485756 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891400f040 con 0x563d38ec2df0 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f890e600700 1 -- 172.21.15.67:0/1547485756 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914005760 con 0x563d38ec2df0 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f8937219b80 1 -- 172.21.15.67:0/1547485756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 msgr2=0x563d38ec2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f8937219b80 1 --2- 172.21.15.67:0/1547485756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec2370 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f891400ed00 tx=0x7f891401d000 comp rx=0 tx=0).stop 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f8937219b80 1 -- 172.21.15.67:0/1547485756 shutdown_connections 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f8937219b80 1 --2- 172.21.15.67:0/1547485756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38eb75d0 0x563d38ed49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f8937219b80 1 --2- 172.21.15.67:0/1547485756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38ec2df0 0x563d38ec2370 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f8937219b80 1 --2- 172.21.15.67:0/1547485756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 0x563d38ec9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f8937219b80 1 -- 172.21.15.67:0/1547485756 >> 172.21.15.67:0/1547485756 conn(0x563d38e176a0 msgr2=0x563d38eac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f8937219b80 1 -- 172.21.15.67:0/1547485756 shutdown_connections 2024-03-30T02:42:26.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.166+0000 7f8937219b80 1 -- 172.21.15.67:0/1547485756 wait complete. 2024-03-30T02:42:26.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.167+0000 7f8937219b80 1 -- start start 2024-03-30T02:42:26.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.167+0000 7f8937219b80 1 Processor -- start 2024-03-30T02:42:26.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.167+0000 7f8937219b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb75d0 0x563d38ec92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.167+0000 7f8937219b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 0x563d38ec97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8937219b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38ec2df0 0x563d38ec7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8937219b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38ec8130 con 0x563d38ec2df0 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8937219b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38ec82a0 con 0x563d38eb75d0 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8937219b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d38ec8460 con 0x563d38eb7ef0 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8937219b80 10 client.0 _ll_register_callbacks cb 0x563d38daf470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8920c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb75d0 0x563d38ec92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8920c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb75d0 0x563d38ec92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39420/0 (socket says 172.21.15.67:39420) 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8920c00700 1 -- 172.21.15.67:0/3695760608 learned_addr learned my addr 172.21.15.67:0/3695760608 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8921600700 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38ec2df0 0x563d38ec7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f891be00700 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 0x563d38ec97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8920c00700 1 -- 172.21.15.67:0/3695760608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 msgr2=0x563d38ec97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8920c00700 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 0x563d38ec97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8920c00700 1 -- 172.21.15.67:0/3695760608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38ec2df0 msgr2=0x563d38ec7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8920c00700 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38ec2df0 0x563d38ec7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8920c00700 1 -- 172.21.15.67:0/3695760608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d38c37b60 con 0x563d38eb75d0 2024-03-30T02:42:26.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8920c00700 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb75d0 0x563d38ec92a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8910097620 tx=0x7f891009a9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbedec4cd2b52757 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.168+0000 7f8902000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8902000700 1 -- 172.21.15.67:0/3695760608 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891009d040 con 0x563d38eb75d0 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8902000700 1 -- 172.21.15.67:0/3695760608 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f891009b820 con 0x563d38eb75d0 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8937219b80 1 -- 172.21.15.67:0/3695760608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8908004510 con 0x563d38eb75d0 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8902000700 1 -- 172.21.15.67:0/3695760608 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89100b2040 con 0x563d38eb75d0 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8937219b80 10 client.26798 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8937219b80 1 -- 172.21.15.67:0/3695760608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f891400ea40 con 0x563d38eb75d0 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8902000700 1 -- 172.21.15.67:0/3695760608 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8908004510 con 0x563d38eb75d0 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8902000700 1 -- 172.21.15.67:0/3695760608 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f89100a3080 con 0x563d38eb75d0 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8937219b80 10 client.26798 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8937219b80 10 client.26798 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8937219b80 20 client.26798 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8937219b80 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563d38f27100 0x563d38f294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8937219b80 4 client.26798 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f8937219b80 1 -- 172.21.15.67:0/3695760608 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d38c58cf0 con 0x563d38f27100 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.169+0000 7f891be00700 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563d38f27100 0x563d38f294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f891be00700 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563d38f27100 0x563d38f294f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=325335e820a403fb server_cookie=9cfabace2e1417c2 in_seq=0 out_seq=0 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8902000700 10 client.26798 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8902000700 1 -- 172.21.15.67:0/3695760608 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d38c58cf0 con 0x563d38f27100 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8902000700 10 client.26798 handle_command_reply: tid=0 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8937219b80 1 client.26798 shutdown 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8937219b80 1 -- 172.21.15.67:0/3695760608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb75d0 msgr2=0x563d38ec92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8937219b80 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb75d0 0x563d38ec92a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8910097620 tx=0x7f891009a9d0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8937219b80 1 -- 172.21.15.67:0/3695760608 shutdown_connections 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8937219b80 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x563d38f27100 0x563d38f294f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8937219b80 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d38ec2df0 0x563d38ec7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8937219b80 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563d38eb75d0 0x563d38ec92a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8937219b80 1 --2- 172.21.15.67:0/3695760608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d38eb7ef0 0x563d38ec97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8937219b80 1 -- 172.21.15.67:0/3695760608 >> 172.21.15.67:0/3695760608 conn(0x563d38e176a0 msgr2=0x563d38ebaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.170+0000 7f8902000700 0 client.26798 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:42:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.171+0000 7f8937219b80 1 -- 172.21.15.67:0/3695760608 shutdown_connections 2024-03-30T02:42:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.171+0000 7f8937219b80 1 -- 172.21.15.67:0/3695760608 wait complete. 2024-03-30T02:42:26.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.171+0000 7f8937219b80 20 client.26798 trim_cache size 0 max 16384 2024-03-30T02:42:26.175 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:42:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.171+0000 7f8923600700 1 -- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89040a1310 msgr2=0x7f89040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.171+0000 7f8923600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89040a1310 0x7f89040a37d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8914003fd0 tx=0x7f89140041f0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.171+0000 7f8923600700 1 -- 172.21.15.67:0/3270780908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f891c100f70 msgr2=0x7f891c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.171+0000 7f8923600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f891c100f70 0x7f891c0730b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8910000c00 tx=0x7f8910005740 comp rx=0 tx=0).stop 2024-03-30T02:42:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.172+0000 7f8923600700 1 -- 172.21.15.67:0/3270780908 shutdown_connections 2024-03-30T02:42:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.172+0000 7f8923600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f89040a1310 0x7f89040a37d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.172+0000 7f8923600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f891c074d70 0x7f891c073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.172+0000 7f8923600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f891c100f70 0x7f891c0730b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.172+0000 7f8923600700 1 --2- 172.21.15.67:0/3270780908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f891c106090 0x7f891c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.172+0000 7f8923600700 1 -- 172.21.15.67:0/3270780908 >> 172.21.15.67:0/3270780908 conn(0x7f891c0fc7f0 msgr2=0x7f891c0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.172+0000 7f8923600700 1 -- 172.21.15.67:0/3270780908 shutdown_connections 2024-03-30T02:42:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.172+0000 7f8923600700 1 -- 172.21.15.67:0/3270780908 wait complete. 2024-03-30T02:42:26.191 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:42:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.412+0000 7f8524e00700 1 -- 172.21.15.67:0/4094611338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073150 msgr2=0x7f8520073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.412+0000 7f8524e00700 1 --2- 172.21.15.67:0/4094611338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073150 0x7f8520073530 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8510009990 tx=0x7f8510009ca0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.412+0000 7f8524e00700 1 -- 172.21.15.67:0/4094611338 shutdown_connections 2024-03-30T02:42:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.412+0000 7f8524e00700 1 --2- 172.21.15.67:0/4094611338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520111c30 0x7f8520114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.412+0000 7f8524e00700 1 --2- 172.21.15.67:0/4094611338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073150 0x7f8520073530 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.412+0000 7f8524e00700 1 --2- 172.21.15.67:0/4094611338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8520073a70 0x7f85201116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.412+0000 7f8524e00700 1 -- 172.21.15.67:0/4094611338 >> 172.21.15.67:0/4094611338 conn(0x7f85200fc710 msgr2=0x7f85200feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.412+0000 7f8524e00700 1 -- 172.21.15.67:0/4094611338 shutdown_connections 2024-03-30T02:42:26.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.412+0000 7f8524e00700 1 -- 172.21.15.67:0/4094611338 wait complete. 2024-03-30T02:42:26.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.413+0000 7f8524e00700 1 Processor -- start 2024-03-30T02:42:26.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.413+0000 7f8524e00700 1 -- start start 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f8524e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8520073150 0x7f852019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f8524e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073a70 0x7f852019dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f8524e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520111c30 0x7f85201a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f8524e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85201a2640 con 0x7f8520111c30 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f8524e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85201a27b0 con 0x7f8520073a70 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f8524e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85201a2970 con 0x7f8520073150 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073a70 0x7f852019dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073a70 0x7f852019dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39448/0 (socket says 172.21.15.67:39448) 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851d400700 1 -- 172.21.15.67:0/2626889938 learned_addr learned my addr 172.21.15.67:0/2626889938 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851e800700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520111c30 0x7f85201a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851de00700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8520073150 0x7f852019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851d400700 1 -- 172.21.15.67:0/2626889938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8520073150 msgr2=0x7f852019d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851d400700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8520073150 0x7f852019d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851d400700 1 -- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520111c30 msgr2=0x7f85201a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851d400700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520111c30 0x7f85201a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.414+0000 7f851d400700 1 -- 172.21.15.67:0/2626889938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85100095f0 con 0x7f8520073a70 2024-03-30T02:42:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.415+0000 7f851d400700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073a70 0x7f852019dcb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e9a0 tx=0x7f850c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.415+0000 7f8516a00700 1 -- 172.21.15.67:0/2626889938 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c009790 con 0x7f8520073a70 2024-03-30T02:42:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.415+0000 7f8524e00700 1 -- 172.21.15.67:0/2626889938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85201a2c50 con 0x7f8520073a70 2024-03-30T02:42:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.415+0000 7f8516a00700 1 -- 172.21.15.67:0/2626889938 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850c00cbd0 con 0x7f8520073a70 2024-03-30T02:42:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.415+0000 7f8516a00700 1 -- 172.21.15.67:0/2626889938 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c018680 con 0x7f8520073a70 2024-03-30T02:42:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.415+0000 7f8524e00700 1 -- 172.21.15.67:0/2626889938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85201aa490 con 0x7f8520073a70 2024-03-30T02:42:26.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.418+0000 7f8516a00700 1 -- 172.21.15.67:0/2626889938 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f850c00cd40 con 0x7f8520073a70 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.420+0000 7f85389aab80 1 Processor -- start 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f85389aab80 1 -- start start 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f8516a00700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f85040a1280 0x7f85040a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f8516a00700 1 -- 172.21.15.67:0/2626889938 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f850c014070 con 0x7f8520073a70 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f85389aab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f85389aab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f2204528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f85389aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 0x55f2204551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f85389aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2203a0990 con 0x55f220452df0 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f85389aab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22039e350 con 0x55f220447ef0 2024-03-30T02:42:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f85389aab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f220399c40 con 0x55f2204475d0 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f851e800700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f85040a1280 0x7f85040a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f851de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f851de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204479b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52380/0 (socket says 172.21.15.67:52380) 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.421+0000 7f851de00700 1 -- 172.21.15.67:0/3677922537 learned_addr learned my addr 172.21.15.67:0/3677922537 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f851e800700 1 --2- 172.21.15.67:0/3677922537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 0x55f2204551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f851d400700 1 --2- 172.21.15.67:0/3677922537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f2204528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f851de00700 1 -- 172.21.15.67:0/3677922537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 msgr2=0x55f2204528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f851de00700 1 --2- 172.21.15.67:0/3677922537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f2204528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f851de00700 1 -- 172.21.15.67:0/3677922537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 msgr2=0x55f2204551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f851de00700 1 --2- 172.21.15.67:0/3677922537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 0x55f2204551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f851de00700 1 -- 172.21.15.67:0/3677922537 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f22021b3b0 con 0x55f2204475d0 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f851e800700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f85040a1280 0x7f85040a3740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f852006a2a0 tx=0x7f851800fc00 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f851de00700 1 --2- 172.21.15.67:0/3677922537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204479b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f851000b2e0 tx=0x7f8510005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bc9cf234d57761b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.422+0000 7f850b400700 1 -- 172.21.15.67:0/3677922537 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100056d0 con 0x55f2204475d0 2024-03-30T02:42:26.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f850b400700 1 -- 172.21.15.67:0/3677922537 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f851001e040 con 0x55f2204475d0 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f850b400700 1 -- 172.21.15.67:0/3677922537 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510004920 con 0x55f2204475d0 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f85389aab80 1 -- 172.21.15.67:0/3677922537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 msgr2=0x55f2204479b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f85389aab80 1 --2- 172.21.15.67:0/3677922537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204479b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f851000b2e0 tx=0x7f8510005ea0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f85389aab80 1 -- 172.21.15.67:0/3677922537 shutdown_connections 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f85389aab80 1 --2- 172.21.15.67:0/3677922537 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 0x55f2204551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f85389aab80 1 --2- 172.21.15.67:0/3677922537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204479b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f85389aab80 1 --2- 172.21.15.67:0/3677922537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f2204528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f85389aab80 1 -- 172.21.15.67:0/3677922537 >> 172.21.15.67:0/3677922537 conn(0x55f2203a76a0 msgr2=0x55f220443a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.423+0000 7f85389aab80 1 -- 172.21.15.67:0/3677922537 shutdown_connections 2024-03-30T02:42:26.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.424+0000 7f85389aab80 1 -- 172.21.15.67:0/3677922537 wait complete. 2024-03-30T02:42:26.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.424+0000 7f85389aab80 1 -- start start 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.424+0000 7f85389aab80 1 Processor -- start 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f85389aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2204475d0 0x55f2204527b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f85389aab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f22044faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f85389aab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f220452df0 0x55f2204500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f85389aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f220335d80 con 0x55f2204475d0 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f85389aab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f220335a20 con 0x55f220447ef0 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f85389aab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22021d6a0 con 0x55f220452df0 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f85389aab80 10 client.0 _ll_register_callbacks cb 0x55f22033f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f22044faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f22044faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39466/0 (socket says 172.21.15.67:39466) 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851d400700 1 -- 172.21.15.67:0/524370809 learned_addr learned my addr 172.21.15.67:0/524370809 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851e800700 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f220452df0 0x55f2204500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851de00700 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2204475d0 0x55f2204527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851d400700 1 -- 172.21.15.67:0/524370809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f220452df0 msgr2=0x55f2204500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851d400700 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f220452df0 0x55f2204500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851d400700 1 -- 172.21.15.67:0/524370809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2204475d0 msgr2=0x55f2204527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851d400700 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2204475d0 0x55f2204527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.425+0000 7f851d400700 1 -- 172.21.15.67:0/524370809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85201149d0 con 0x55f220447ef0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.426+0000 7f851d400700 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f22044faa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e970 tx=0x7f850c0cd0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c6cc340df5c1931 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.426+0000 7f84ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.426+0000 7f84ffe00700 1 -- 172.21.15.67:0/524370809 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c09a050 con 0x55f220447ef0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.426+0000 7f84ffe00700 1 -- 172.21.15.67:0/524370809 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f850c09de80 con 0x55f220447ef0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.426+0000 7f85389aab80 1 -- 172.21.15.67:0/524370809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f22035a380 con 0x55f220447ef0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.426+0000 7f84ffe00700 1 -- 172.21.15.67:0/524370809 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f850c0a24d0 con 0x55f220447ef0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.426+0000 7f85389aab80 10 client.26810 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.426+0000 7f85389aab80 1 -- 172.21.15.67:0/524370809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f2201c7b60 con 0x55f220447ef0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.426+0000 7f84ffe00700 1 -- 172.21.15.67:0/524370809 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f22035a380 con 0x55f220447ef0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.427+0000 7f84ffe00700 1 -- 172.21.15.67:0/524370809 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f850c098fb0 con 0x55f220447ef0 2024-03-30T02:42:26.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.427+0000 7f85389aab80 10 client.26810 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.427+0000 7f85389aab80 10 client.26810 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.427+0000 7f85389aab80 20 client.26810 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.427+0000 7f85389aab80 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f220462860 0x55f220464c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.427+0000 7f85389aab80 4 client.26810 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.427+0000 7f85389aab80 1 -- 172.21.15.67:0/524370809 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f220392700 con 0x55f220462860 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.427+0000 7f851de00700 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f220462860 0x55f220464c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.428+0000 7f84ffe00700 10 client.26810 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.428+0000 7f851de00700 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f220462860 0x55f220464c80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc29421180bedbc1 server_cookie=3aacee2b73ddda8f in_seq=0 out_seq=0 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.428+0000 7f84ffe00700 1 -- 172.21.15.67:0/524370809 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f220392700 con 0x55f220462860 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f84ffe00700 10 client.26810 handle_command_reply: tid=0 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 client.26810 shutdown 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 -- 172.21.15.67:0/524370809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 msgr2=0x55f22044faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f22044faa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e970 tx=0x7f850c0cd0c0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 -- 172.21.15.67:0/524370809 shutdown_connections 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f220462860 0x55f220464c80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f2204475d0 0x55f2204527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f220452df0 0x55f2204500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 --2- 172.21.15.67:0/524370809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f22044faa0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f84ffe00700 0 client.26810 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 -- 172.21.15.67:0/524370809 >> 172.21.15.67:0/524370809 conn(0x55f2203a76a0 msgr2=0x55f2204496c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 -- 172.21.15.67:0/524370809 shutdown_connections 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 1 -- 172.21.15.67:0/524370809 wait complete. 2024-03-30T02:42:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.429+0000 7f85389aab80 20 client.26810 trim_cache size 0 max 16384 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.443+0000 7f85389aab80 1 Processor -- start 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.443+0000 7f85389aab80 1 -- start start 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.443+0000 7f85389aab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f220450060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.443+0000 7f85389aab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f220459a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.443+0000 7f85389aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 0x55f22043cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.443+0000 7f85389aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f220447480 con 0x55f220452df0 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.443+0000 7f85389aab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f220448880 con 0x55f220447ef0 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.443+0000 7f85389aab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f220444660 con 0x55f2204475d0 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f220459a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851e800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 0x55f22043cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f220450060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f220459a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39478/0 (socket says 172.21.15.67:39478) 2024-03-30T02:42:26.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851d400700 1 -- 172.21.15.67:0/1706987457 learned_addr learned my addr 172.21.15.67:0/1706987457 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851d400700 1 -- 172.21.15.67:0/1706987457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 msgr2=0x55f220450060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851d400700 1 --2- 172.21.15.67:0/1706987457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f220450060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851d400700 1 -- 172.21.15.67:0/1706987457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 msgr2=0x55f22043cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851d400700 1 --2- 172.21.15.67:0/1706987457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 0x55f22043cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851d400700 1 -- 172.21.15.67:0/1706987457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f22035fea0 con 0x55f220447ef0 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851de00700 1 --2- 172.21.15.67:0/1706987457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f220450060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f851d400700 1 --2- 172.21.15.67:0/1706987457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f220459a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f850c096550 tx=0x7f850c09a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e5821313eada2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f850be00700 1 -- 172.21.15.67:0/1706987457 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c09de80 con 0x55f220447ef0 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f850be00700 1 -- 172.21.15.67:0/1706987457 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f850c09a050 con 0x55f220447ef0 2024-03-30T02:42:26.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.444+0000 7f850be00700 1 -- 172.21.15.67:0/1706987457 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c0b8200 con 0x55f220447ef0 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.445+0000 7f85389aab80 1 -- 172.21.15.67:0/1706987457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 msgr2=0x55f220459a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.445+0000 7f85389aab80 1 --2- 172.21.15.67:0/1706987457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f220459a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f850c096550 tx=0x7f850c09a590 comp rx=0 tx=0).stop 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.445+0000 7f85389aab80 1 -- 172.21.15.67:0/1706987457 shutdown_connections 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.445+0000 7f85389aab80 1 --2- 172.21.15.67:0/1706987457 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220452df0 0x55f22043cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.445+0000 7f85389aab80 1 --2- 172.21.15.67:0/1706987457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f220450060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.445+0000 7f85389aab80 1 --2- 172.21.15.67:0/1706987457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220447ef0 0x55f220459a10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.445+0000 7f85389aab80 1 -- 172.21.15.67:0/1706987457 >> 172.21.15.67:0/1706987457 conn(0x55f2203a76a0 msgr2=0x55f22043c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.445+0000 7f85389aab80 1 -- 172.21.15.67:0/1706987457 shutdown_connections 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.445+0000 7f85389aab80 1 -- 172.21.15.67:0/1706987457 wait complete. 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.446+0000 7f85389aab80 1 -- start start 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.446+0000 7f85389aab80 1 Processor -- start 2024-03-30T02:42:26.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.446+0000 7f85389aab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.446+0000 7f85389aab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220447ef0 0x55f2204afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.446+0000 7f85389aab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220452df0 0x55f2204579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.446+0000 7f85389aab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f220457fc0 con 0x55f220447ef0 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.446+0000 7f85389aab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f220458130 con 0x55f220452df0 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.446+0000 7f85389aab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2204582f0 con 0x55f2204475d0 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.446+0000 7f85389aab80 10 client.0 _ll_register_callbacks cb 0x55f22033f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220447ef0 0x55f2204afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851e800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220452df0 0x55f2204579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220447ef0 0x55f2204afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53462/0 (socket says 172.21.15.67:53462) 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851d400700 1 -- 172.21.15.67:0/300922966 learned_addr learned my addr 172.21.15.67:0/300922966 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851de00700 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851e800700 1 -- 172.21.15.67:0/300922966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 msgr2=0x55f2204af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851e800700 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851e800700 1 -- 172.21.15.67:0/300922966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220447ef0 msgr2=0x55f2204afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851e800700 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220447ef0 0x55f2204afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851e800700 1 -- 172.21.15.67:0/300922966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f22021b3b0 con 0x55f220452df0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851d400700 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220447ef0 0x55f2204afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f851e800700 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220452df0 0x55f2204579f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55f220462740 tx=0x7f8518004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebebe917ea7c6d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.447+0000 7f84ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f84ffe00700 1 -- 172.21.15.67:0/300922966 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851800e040 con 0x55f220452df0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f85389aab80 1 -- 172.21.15.67:0/300922966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f851001e040 con 0x55f220452df0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f84ffe00700 1 -- 172.21.15.67:0/300922966 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f8518003c00 con 0x55f220452df0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f84ffe00700 1 -- 172.21.15.67:0/300922966 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8518003dc0 con 0x55f220452df0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f85389aab80 10 client.26816 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f85389aab80 1 -- 172.21.15.67:0/300922966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f2201c7b60 con 0x55f220452df0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f84ffe00700 1 -- 172.21.15.67:0/300922966 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f851001e040 con 0x55f220452df0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f84ffe00700 1 -- 172.21.15.67:0/300922966 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f85180046d0 con 0x55f220452df0 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f85389aab80 10 client.26816 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f85389aab80 10 client.26816 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:42:26.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f85389aab80 20 client.26816 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f85389aab80 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2204b79c0 0x55f2204b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f85389aab80 4 client.26816 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.448+0000 7f85389aab80 1 -- 172.21.15.67:0/300922966 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f2201e8cf0 con 0x55f2204b79c0 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.449+0000 7f851de00700 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2204b79c0 0x55f2204b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.449+0000 7f84ffe00700 10 client.26816 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.449+0000 7f851de00700 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2204b79c0 0x55f2204b9de0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3e0377fbb3b5df server_cookie=4ac12ecff6cd4b67 in_seq=0 out_seq=0 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.449+0000 7f84ffe00700 1 -- 172.21.15.67:0/300922966 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f2201e8cf0 con 0x55f2204b79c0 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.449+0000 7f84ffe00700 10 client.26816 handle_command_reply: tid=0 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.449+0000 7f85389aab80 1 client.26816 shutdown 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 -- 172.21.15.67:0/300922966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220452df0 msgr2=0x55f2204579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220452df0 0x55f2204579f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55f220462740 tx=0x7f8518004b40 comp rx=0 tx=0).stop 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 -- 172.21.15.67:0/300922966 shutdown_connections 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55f2204b79c0 0x55f2204b9de0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f220452df0 0x55f2204579f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f2204475d0 0x55f2204af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 --2- 172.21.15.67:0/300922966 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f220447ef0 0x55f2204afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f84ffe00700 0 client.26816 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:42:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 -- 172.21.15.67:0/300922966 >> 172.21.15.67:0/300922966 conn(0x55f2203a76a0 msgr2=0x55f22044e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 -- 172.21.15.67:0/300922966 shutdown_connections 2024-03-30T02:42:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 1 -- 172.21.15.67:0/300922966 wait complete. 2024-03-30T02:42:26.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f85389aab80 20 client.26816 trim_cache size 0 max 16384 2024-03-30T02:42:26.454 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:42:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f8524e00700 1 -- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f85040a1280 msgr2=0x7f85040a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.450+0000 7f8524e00700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f85040a1280 0x7f85040a3740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f852006a2a0 tx=0x7f851800fc00 comp rx=0 tx=0).stop 2024-03-30T02:42:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 -- 172.21.15.67:0/2626889938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073a70 msgr2=0x7f852019dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073a70 0x7f852019dcb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e9a0 tx=0x7f850c00ecb0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 -- 172.21.15.67:0/2626889938 shutdown_connections 2024-03-30T02:42:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f85040a1280 0x7f85040a3740 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8520111c30 0x7f85201a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8520073150 0x7f852019d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 --2- 172.21.15.67:0/2626889938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8520073a70 0x7f852019dcb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 -- 172.21.15.67:0/2626889938 >> 172.21.15.67:0/2626889938 conn(0x7f85200fc710 msgr2=0x7f85200fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 -- 172.21.15.67:0/2626889938 shutdown_connections 2024-03-30T02:42:26.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.451+0000 7f8524e00700 1 -- 172.21.15.67:0/2626889938 wait complete. 2024-03-30T02:42:26.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:26 smithi067 ceph-mon[37808]: pgmap v164: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 260 op/s 2024-03-30T02:42:26.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:26 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2084401831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:42:26.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:42:26.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:26 smithi170 ceph-mon[78798]: pgmap v164: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 260 op/s 2024-03-30T02:42:26.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:26 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2084401831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:42:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:26 smithi082 ceph-mon[79176]: pgmap v164: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 260 op/s 2024-03-30T02:42:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:26 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2084401831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:42:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.761+0000 7fab1fe00700 1 -- 172.21.15.67:0/3705046463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab20105ed0 msgr2=0x7fab2010da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.761+0000 7fab1fe00700 1 --2- 172.21.15.67:0/3705046463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab20105ed0 0x7fab2010da50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fab18009f90 tx=0x7fab1800b330 comp rx=0 tx=0).stop 2024-03-30T02:42:26.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.762+0000 7fab1fe00700 1 -- 172.21.15.67:0/3705046463 shutdown_connections 2024-03-30T02:42:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.762+0000 7fab1fe00700 1 --2- 172.21.15.67:0/3705046463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab20105ed0 0x7fab2010da50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.762+0000 7fab1fe00700 1 --2- 172.21.15.67:0/3705046463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20100db0 0x7fab20101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.762+0000 7fab1fe00700 1 --2- 172.21.15.67:0/3705046463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab20101760 0x7fab201057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.762+0000 7fab1fe00700 1 -- 172.21.15.67:0/3705046463 >> 172.21.15.67:0/3705046463 conn(0x7fab200fc670 msgr2=0x7fab200fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.763+0000 7fab1fe00700 1 -- 172.21.15.67:0/3705046463 shutdown_connections 2024-03-30T02:42:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.763+0000 7fab1fe00700 1 -- 172.21.15.67:0/3705046463 wait complete. 2024-03-30T02:42:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.765+0000 7fab1fe00700 1 Processor -- start 2024-03-30T02:42:26.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.768+0000 7fab1fe00700 1 -- start start 2024-03-30T02:42:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.769+0000 7fab1fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab20100db0 0x7fab201992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.769+0000 7fab1fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20105ed0 0x7fab20199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.769+0000 7fab1fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab2019dba0 0x7fab2019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.769+0000 7fab1fe00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2019e620 con 0x7fab20100db0 2024-03-30T02:42:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.769+0000 7fab1fe00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2019e790 con 0x7fab2019dba0 2024-03-30T02:42:26.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.769+0000 7fab1fe00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2019e950 con 0x7fab20105ed0 2024-03-30T02:42:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.770+0000 7fab1e000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20105ed0 0x7fab20199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.770+0000 7fab1e000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20105ed0 0x7fab20199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52432/0 (socket says 172.21.15.67:52432) 2024-03-30T02:42:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.770+0000 7fab1e000700 1 -- 172.21.15.67:0/2337758057 learned_addr learned my addr 172.21.15.67:0/2337758057 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.770+0000 7fab1ea00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab20100db0 0x7fab201992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.771+0000 7fab1e000700 1 -- 172.21.15.67:0/2337758057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab2019dba0 msgr2=0x7fab2019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.771+0000 7fab1e000700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab2019dba0 0x7fab2019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.771+0000 7fab1e000700 1 -- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab20100db0 msgr2=0x7fab201992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.771+0000 7fab1e000700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab20100db0 0x7fab201992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.771+0000 7fab1e000700 1 -- 172.21.15.67:0/2337758057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18009c40 con 0x7fab20105ed0 2024-03-30T02:42:26.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.771+0000 7fab1ea00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab20100db0 0x7fab201992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:42:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.778+0000 7fab1e000700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20105ed0 0x7fab20199800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00e9f0 tx=0x7fab0c00edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.779+0000 7fab17400700 1 -- 172.21.15.67:0/2337758057 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c009690 con 0x7fab20105ed0 2024-03-30T02:42:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.779+0000 7fab17400700 1 -- 172.21.15.67:0/2337758057 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab0c00cb50 con 0x7fab20105ed0 2024-03-30T02:42:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.779+0000 7fab1fe00700 1 -- 172.21.15.67:0/2337758057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab201a5eb0 con 0x7fab20105ed0 2024-03-30T02:42:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.779+0000 7fab17400700 1 -- 172.21.15.67:0/2337758057 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c018680 con 0x7fab20105ed0 2024-03-30T02:42:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.780+0000 7fab1fe00700 1 -- 172.21.15.67:0/2337758057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab201a6380 con 0x7fab20105ed0 2024-03-30T02:42:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.788+0000 7fab17400700 1 -- 172.21.15.67:0/2337758057 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fab0c00ccc0 con 0x7fab20105ed0 2024-03-30T02:42:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.790+0000 7fab17400700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fab040a1350 0x7fab040a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.791+0000 7fab17400700 1 -- 172.21.15.67:0/2337758057 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fab0c014070 con 0x7fab20105ed0 2024-03-30T02:42:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.791+0000 7fab1ea00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fab040a1350 0x7fab040a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.791+0000 7fab39351b80 1 Processor -- start 2024-03-30T02:42:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.792+0000 7fab39351b80 1 -- start start 2024-03-30T02:42:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.792+0000 7fab39351b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224920a0 0x55e222492480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.792+0000 7fab39351b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 0x55e22248f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.792+0000 7fab39351b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224a6230 0x55e2224a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.792+0000 7fab39351b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2223e2990 con 0x55e2224920a0 2024-03-30T02:42:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.792+0000 7fab39351b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2223e0350 con 0x55e2224929c0 2024-03-30T02:42:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.792+0000 7fab39351b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2223dbc40 con 0x55e2224a6230 2024-03-30T02:42:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.793+0000 7fab1ea00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fab040a1350 0x7fab040a3810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab10007900 tx=0x7fab10008040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.793+0000 7fab1e000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 0x55e22248f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.793+0000 7fab1e000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 0x55e22248f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39520/0 (socket says 172.21.15.67:39520) 2024-03-30T02:42:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.793+0000 7fab1e000700 1 -- 172.21.15.67:0/1406502834 learned_addr learned my addr 172.21.15.67:0/1406502834 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.794+0000 7fab1ea00700 1 --2- 172.21.15.67:0/1406502834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224920a0 0x55e222492480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.795+0000 7fab1ea00700 1 -- 172.21.15.67:0/1406502834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224a6230 msgr2=0x55e2224a8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.795+0000 7fab1ea00700 1 --2- 172.21.15.67:0/1406502834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224a6230 0x55e2224a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.795+0000 7fab1ea00700 1 -- 172.21.15.67:0/1406502834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 msgr2=0x55e22248f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.795+0000 7fab1ea00700 1 --2- 172.21.15.67:0/1406502834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 0x55e22248f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.795+0000 7fab1ea00700 1 -- 172.21.15.67:0/1406502834 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e22225d3b0 con 0x55e2224920a0 2024-03-30T02:42:26.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.795+0000 7fab1ea00700 1 --2- 172.21.15.67:0/1406502834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224920a0 0x55e222492480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c430 tx=0x7fab1000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faedf9b24f9458a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.796+0000 7fab0be00700 1 -- 172.21.15.67:0/1406502834 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10013830 con 0x55e2224920a0 2024-03-30T02:42:26.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.797+0000 7fab0be00700 1 -- 172.21.15.67:0/1406502834 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab10007e70 con 0x55e2224920a0 2024-03-30T02:42:26.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.797+0000 7fab0be00700 1 -- 172.21.15.67:0/1406502834 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10013830 con 0x55e2224920a0 2024-03-30T02:42:26.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.798+0000 7fab39351b80 1 -- 172.21.15.67:0/1406502834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224920a0 msgr2=0x55e222492480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.798+0000 7fab39351b80 1 --2- 172.21.15.67:0/1406502834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224920a0 0x55e222492480 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c430 tx=0x7fab1000f000 comp rx=0 tx=0).stop 2024-03-30T02:42:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.799+0000 7fab39351b80 1 -- 172.21.15.67:0/1406502834 shutdown_connections 2024-03-30T02:42:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.799+0000 7fab39351b80 1 --2- 172.21.15.67:0/1406502834 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224920a0 0x55e222492480 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.799+0000 7fab39351b80 1 --2- 172.21.15.67:0/1406502834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224a6230 0x55e2224a8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.799+0000 7fab39351b80 1 --2- 172.21.15.67:0/1406502834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 0x55e22248f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.799+0000 7fab39351b80 1 -- 172.21.15.67:0/1406502834 >> 172.21.15.67:0/1406502834 conn(0x55e2223e96a0 msgr2=0x55e222485c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.800+0000 7fab39351b80 1 -- 172.21.15.67:0/1406502834 shutdown_connections 2024-03-30T02:42:26.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.801+0000 7fab39351b80 1 -- 172.21.15.67:0/1406502834 wait complete. 2024-03-30T02:42:26.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.801+0000 7fab39351b80 1 -- start start 2024-03-30T02:42:26.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.802+0000 7fab39351b80 1 Processor -- start 2024-03-30T02:42:26.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.804+0000 7fab39351b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224920a0 0x55e22248f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.804+0000 7fab39351b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e22248c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.805+0000 7fab39351b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224a6230 0x55e22248cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.805+0000 7fab39351b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e222377d80 con 0x55e2224929c0 2024-03-30T02:42:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.805+0000 7fab39351b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e222377a20 con 0x55e2224920a0 2024-03-30T02:42:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.805+0000 7fab39351b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e22225f6a0 con 0x55e2224a6230 2024-03-30T02:42:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.805+0000 7fab1ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224920a0 0x55e22248f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.805+0000 7fab1e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e22248c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.805+0000 7fab1e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e22248c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53484/0 (socket says 172.21.15.67:53484) 2024-03-30T02:42:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.805+0000 7fab1e000700 1 -- 172.21.15.67:0/3166084157 learned_addr learned my addr 172.21.15.67:0/3166084157 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.805+0000 7fab1f400700 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224a6230 0x55e22248cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.806+0000 7fab39351b80 10 client.0 _ll_register_callbacks cb 0x55e222381470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.807+0000 7fab1e000700 1 -- 172.21.15.67:0/3166084157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224a6230 msgr2=0x55e22248cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.807+0000 7fab1e000700 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224a6230 0x55e22248cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.807+0000 7fab1e000700 1 -- 172.21.15.67:0/3166084157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224920a0 msgr2=0x55e22248f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.807+0000 7fab1e000700 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224920a0 0x55e22248f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.807+0000 7fab1e000700 1 -- 172.21.15.67:0/3166084157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1000c680 con 0x55e2224929c0 2024-03-30T02:42:26.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.807+0000 7fab1e000700 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e22248c760 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0ca640 tx=0x7fab0c09de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91366da6d4055b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.808+0000 7fab08c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:42:26.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.808+0000 7fab08c00700 1 -- 172.21.15.67:0/3166084157 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c09a480 con 0x55e2224929c0 2024-03-30T02:42:26.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.808+0000 7fab08c00700 1 -- 172.21.15.67:0/3166084157 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fab0c099440 con 0x55e2224929c0 2024-03-30T02:42:26.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.808+0000 7fab39351b80 1 -- 172.21.15.67:0/3166084157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e22239c380 con 0x55e2224929c0 2024-03-30T02:42:26.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.809+0000 7fab08c00700 1 -- 172.21.15.67:0/3166084157 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab0c099600 con 0x55e2224929c0 2024-03-30T02:42:26.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.810+0000 7fab39351b80 10 client.16719 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:26.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.810+0000 7fab39351b80 1 -- 172.21.15.67:0/3166084157 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e222209b60 con 0x55e2224929c0 2024-03-30T02:42:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.811+0000 7fab08c00700 1 -- 172.21.15.67:0/3166084157 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e22239c380 con 0x55e2224929c0 2024-03-30T02:42:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.811+0000 7fab08c00700 1 -- 172.21.15.67:0/3166084157 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fab0c094b60 con 0x55e2224929c0 2024-03-30T02:42:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.812+0000 7fab39351b80 10 client.16719 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.812+0000 7fab39351b80 10 client.16719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:42:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.812+0000 7fab39351b80 20 client.16719 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.812+0000 7fab39351b80 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e2224b5950 0x55e2224b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.812+0000 7fab39351b80 4 client.16719 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:42:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.812+0000 7fab39351b80 1 -- 172.21.15.67:0/3166084157 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2223d4700 con 0x55e2224b5950 2024-03-30T02:42:26.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.812+0000 7fab1ea00700 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e2224b5950 0x55e2224b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.817+0000 7fab08c00700 10 client.16719 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:42:26.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.817+0000 7fab1ea00700 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e2224b5950 0x55e2224b7d70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17a2db56ab51460c server_cookie=c34e50db0806c921 in_seq=0 out_seq=0 2024-03-30T02:42:26.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.821+0000 7fab08c00700 1 -- 172.21.15.67:0/3166084157 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2223d4700 con 0x55e2224b5950 2024-03-30T02:42:26.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.821+0000 7fab08c00700 10 client.16719 handle_command_reply: tid=0 2024-03-30T02:42:26.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.821+0000 7fab39351b80 1 client.16719 shutdown 2024-03-30T02:42:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.821+0000 7fab39351b80 1 -- 172.21.15.67:0/3166084157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 msgr2=0x55e22248c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.821+0000 7fab39351b80 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e22248c760 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0ca640 tx=0x7fab0c09de30 comp rx=0 tx=0).stop 2024-03-30T02:42:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.821+0000 7fab39351b80 1 -- 172.21.15.67:0/3166084157 shutdown_connections 2024-03-30T02:42:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.821+0000 7fab39351b80 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224a6230 0x55e22248cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.822+0000 7fab39351b80 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224920a0 0x55e22248f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.822+0000 7fab39351b80 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e2224b5950 0x55e2224b7d70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.822+0000 7fab39351b80 1 --2- 172.21.15.67:0/3166084157 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e22248c760 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.822+0000 7fab39351b80 1 -- 172.21.15.67:0/3166084157 >> 172.21.15.67:0/3166084157 conn(0x55e2223e96a0 msgr2=0x55e222494070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.823+0000 7fab08c00700 0 client.16719 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:42:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.823+0000 7fab39351b80 1 -- 172.21.15.67:0/3166084157 shutdown_connections 2024-03-30T02:42:26.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.823+0000 7fab39351b80 1 -- 172.21.15.67:0/3166084157 wait complete. 2024-03-30T02:42:26.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.826+0000 7fab39351b80 20 client.16719 trim_cache size 0 max 16384 2024-03-30T02:42:26.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.865+0000 7fab39351b80 1 Processor -- start 2024-03-30T02:42:26.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.865+0000 7fab39351b80 1 -- start start 2024-03-30T02:42:26.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.865+0000 7fab39351b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e22248c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.865+0000 7fab39351b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 0x55e2224acc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.866+0000 7fab1ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e22248c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.867+0000 7fab1ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e22248c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52454/0 (socket says 172.21.15.67:52454) 2024-03-30T02:42:26.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.867+0000 7fab1ea00700 1 -- 172.21.15.67:0/75944739 learned_addr learned my addr 172.21.15.67:0/75944739 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.867+0000 7fab39351b80 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224a6230 0x55e22247f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.867+0000 7fab39351b80 1 -- 172.21.15.67:0/75944739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e222491ef0 con 0x55e2224a6230 2024-03-30T02:42:26.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.867+0000 7fab1e000700 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 0x55e2224acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.868+0000 7fab39351b80 1 -- 172.21.15.67:0/75944739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e222487600 con 0x55e2224929c0 2024-03-30T02:42:26.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.868+0000 7fab39351b80 1 -- 172.21.15.67:0/75944739 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e222486160 con 0x55e2224920a0 2024-03-30T02:42:26.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.868+0000 7fab1f400700 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224a6230 0x55e22247f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.869+0000 7fab1f400700 1 -- 172.21.15.67:0/75944739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 msgr2=0x55e22248c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.869+0000 7fab1f400700 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e22248c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.869+0000 7fab1f400700 1 -- 172.21.15.67:0/75944739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 msgr2=0x55e2224acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.869+0000 7fab1f400700 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 0x55e2224acc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.869+0000 7fab1f400700 1 -- 172.21.15.67:0/75944739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2223a1ea0 con 0x55e2224a6230 2024-03-30T02:42:26.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.870+0000 7fab1f400700 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224a6230 0x55e22247f010 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab18015000 tx=0x7fab18009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e9fb898f5dd34b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.871+0000 7fab14c00700 1 -- 172.21.15.67:0/75944739 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1800e070 con 0x55e2224a6230 2024-03-30T02:42:26.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.872+0000 7fab14c00700 1 -- 172.21.15.67:0/75944739 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1800bcc0 con 0x55e2224a6230 2024-03-30T02:42:26.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.872+0000 7fab14c00700 1 -- 172.21.15.67:0/75944739 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab180078e0 con 0x55e2224a6230 2024-03-30T02:42:26.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.873+0000 7fab39351b80 1 -- 172.21.15.67:0/75944739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224a6230 msgr2=0x55e22247f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.873+0000 7fab39351b80 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224a6230 0x55e22247f010 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab18015000 tx=0x7fab18009510 comp rx=0 tx=0).stop 2024-03-30T02:42:26.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.874+0000 7fab39351b80 1 -- 172.21.15.67:0/75944739 shutdown_connections 2024-03-30T02:42:26.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.874+0000 7fab39351b80 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224a6230 0x55e22247f010 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.874+0000 7fab39351b80 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e22248c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.874+0000 7fab39351b80 1 --2- 172.21.15.67:0/75944739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224929c0 0x55e2224acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.874+0000 7fab39351b80 1 -- 172.21.15.67:0/75944739 >> 172.21.15.67:0/75944739 conn(0x55e2223e96a0 msgr2=0x55e22247e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.875+0000 7fab39351b80 1 -- 172.21.15.67:0/75944739 shutdown_connections 2024-03-30T02:42:26.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.875+0000 7fab39351b80 1 -- 172.21.15.67:0/75944739 wait complete. 2024-03-30T02:42:26.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.876+0000 7fab39351b80 1 -- start start 2024-03-30T02:42:26.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.876+0000 7fab39351b80 1 Processor -- start 2024-03-30T02:42:26.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.877+0000 7fab39351b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e2224855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.877+0000 7fab39351b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e222485b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.877+0000 7fab39351b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224a6230 0x55e2224abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.877+0000 7fab39351b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2224ac3d0 con 0x55e2224929c0 2024-03-30T02:42:26.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.877+0000 7fab39351b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2224ac540 con 0x55e2224a6230 2024-03-30T02:42:26.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.877+0000 7fab39351b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2224ac700 con 0x55e2224920a0 2024-03-30T02:42:26.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.877+0000 7fab39351b80 10 client.0 _ll_register_callbacks cb 0x55e222381470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.879+0000 7fab1e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e222485b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.879+0000 7fab1ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e2224855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.879+0000 7fab1e000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e222485b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53506/0 (socket says 172.21.15.67:53506) 2024-03-30T02:42:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.879+0000 7fab1ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e2224855d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52464/0 (socket says 172.21.15.67:52464) 2024-03-30T02:42:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.879+0000 7fab1e000700 1 -- 172.21.15.67:0/3146806268 learned_addr learned my addr 172.21.15.67:0/3146806268 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.880+0000 7fab1e000700 1 -- 172.21.15.67:0/3146806268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 msgr2=0x55e2224855d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.880+0000 7fab1e000700 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e2224855d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.880+0000 7fab1e000700 1 -- 172.21.15.67:0/3146806268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224a6230 msgr2=0x55e2224abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.880+0000 7fab1e000700 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224a6230 0x55e2224abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.880+0000 7fab1e000700 1 -- 172.21.15.67:0/3146806268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2010e2c0 con 0x55e2224929c0 2024-03-30T02:42:26.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.880+0000 7fab1ea00700 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e2224855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.880+0000 7fab1e000700 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e222485b10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c5b0 tx=0x7fab0c00c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e2ee026b657f3a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.880+0000 7fab08c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:42:26.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.881+0000 7fab08c00700 1 -- 172.21.15.67:0/3146806268 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c094d70 con 0x55e2224929c0 2024-03-30T02:42:26.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.881+0000 7fab08c00700 1 -- 172.21.15.67:0/3146806268 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fab0c0a1f40 con 0x55e2224929c0 2024-03-30T02:42:26.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.881+0000 7fab39351b80 1 -- 172.21.15.67:0/3146806268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab10007e70 con 0x55e2224929c0 2024-03-30T02:42:26.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.882+0000 7fab08c00700 1 -- 172.21.15.67:0/3146806268 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab0c096290 con 0x55e2224929c0 2024-03-30T02:42:26.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.882+0000 7fab39351b80 10 client.16725 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:26.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.882+0000 7fab39351b80 1 -- 172.21.15.67:0/3146806268 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e222261cd0 con 0x55e2224929c0 2024-03-30T02:42:26.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.883+0000 7fab08c00700 1 -- 172.21.15.67:0/3146806268 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab10007e70 con 0x55e2224929c0 2024-03-30T02:42:26.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.883+0000 7fab08c00700 1 -- 172.21.15.67:0/3146806268 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fab0c098eb0 con 0x55e2224929c0 2024-03-30T02:42:26.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.884+0000 7fab39351b80 10 client.16725 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:26.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.884+0000 7fab39351b80 10 client.16725 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:42:26.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.884+0000 7fab39351b80 20 client.16725 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:26.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.884+0000 7fab39351b80 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e22250a550 0x55e22250c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:26.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.884+0000 7fab39351b80 4 client.16725 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:42:26.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.884+0000 7fab39351b80 1 -- 172.21.15.67:0/3146806268 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e22222acf0 con 0x55e22250a550 2024-03-30T02:42:26.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.885+0000 7fab1ea00700 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e22250a550 0x55e22250c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:26.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.887+0000 7fab1ea00700 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e22250a550 0x55e22250c970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e24400a4342504b server_cookie=4f6c0c3abf7ed079 in_seq=0 out_seq=0 2024-03-30T02:42:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.887+0000 7fab08c00700 10 client.16725 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:42:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.889+0000 7fab08c00700 1 -- 172.21.15.67:0/3146806268 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e22222acf0 con 0x55e22250a550 2024-03-30T02:42:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.889+0000 7fab08c00700 10 client.16725 handle_command_reply: tid=0 2024-03-30T02:42:26.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.889+0000 7fab39351b80 1 client.16725 shutdown 2024-03-30T02:42:26.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.889+0000 7fab39351b80 1 -- 172.21.15.67:0/3146806268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 msgr2=0x55e222485b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.889+0000 7fab39351b80 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e222485b10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00c5b0 tx=0x7fab0c00c6d0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.890+0000 7fab39351b80 1 -- 172.21.15.67:0/3146806268 shutdown_connections 2024-03-30T02:42:26.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.890+0000 7fab39351b80 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2224a6230 0x55e2224abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.890+0000 7fab39351b80 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2224920a0 0x55e2224855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.890+0000 7fab39351b80 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e22250a550 0x55e22250c970 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.890+0000 7fab39351b80 1 --2- 172.21.15.67:0/3146806268 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e2224929c0 0x55e222485b10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.890+0000 7fab08c00700 0 client.16725 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:42:26.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.890+0000 7fab39351b80 1 -- 172.21.15.67:0/3146806268 >> 172.21.15.67:0/3146806268 conn(0x55e2223e96a0 msgr2=0x55e22247f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.891+0000 7fab39351b80 1 -- 172.21.15.67:0/3146806268 shutdown_connections 2024-03-30T02:42:26.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.891+0000 7fab39351b80 1 -- 172.21.15.67:0/3146806268 wait complete. 2024-03-30T02:42:26.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.892+0000 7fab39351b80 20 client.16725 trim_cache size 0 max 16384 2024-03-30T02:42:26.891 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:42:26.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 -- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fab040a1350 msgr2=0x7fab040a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fab040a1350 0x7fab040a3810 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab10007900 tx=0x7fab10008040 comp rx=0 tx=0).stop 2024-03-30T02:42:26.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 -- 172.21.15.67:0/2337758057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20105ed0 msgr2=0x7fab20199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:26.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20105ed0 0x7fab20199800 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00e9f0 tx=0x7fab0c00edb0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 -- 172.21.15.67:0/2337758057 shutdown_connections 2024-03-30T02:42:26.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fab040a1350 0x7fab040a3810 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab20100db0 0x7fab201992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab2019dba0 0x7fab2019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 --2- 172.21.15.67:0/2337758057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab20105ed0 0x7fab20199800 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:26.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.893+0000 7fab1fe00700 1 -- 172.21.15.67:0/2337758057 >> 172.21.15.67:0/2337758057 conn(0x7fab200fc670 msgr2=0x7fab200fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:26.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.894+0000 7fab1fe00700 1 -- 172.21.15.67:0/2337758057 shutdown_connections 2024-03-30T02:42:26.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:26.894+0000 7fab1fe00700 1 -- 172.21.15.67:0/2337758057 wait complete. 2024-03-30T02:42:26.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:42:27.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.344+0000 7fd2c5a00700 1 -- 172.21.15.67:0/3482103129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2c0100f60 msgr2=0x7fd2c0101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:27.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.344+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/3482103129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2c0100f60 0x7fd2c0101340 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009990 tx=0x7fd2b0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.344+0000 7fd2c5a00700 1 -- 172.21.15.67:0/3482103129 shutdown_connections 2024-03-30T02:42:27.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.344+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/3482103129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2c0100f60 0x7fd2c0101340 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.344+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/3482103129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c0106080 0x7fd2c010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.344+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/3482103129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c0101910 0x7fd2c0105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.344+0000 7fd2c5a00700 1 -- 172.21.15.67:0/3482103129 >> 172.21.15.67:0/3482103129 conn(0x7fd2c00fc7c0 msgr2=0x7fd2c00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:27.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.345+0000 7fd2c5a00700 1 -- 172.21.15.67:0/3482103129 shutdown_connections 2024-03-30T02:42:27.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.345+0000 7fd2c5a00700 1 -- 172.21.15.67:0/3482103129 wait complete. 2024-03-30T02:42:27.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.346+0000 7fd2c5a00700 1 Processor -- start 2024-03-30T02:42:27.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.348+0000 7fd2c5a00700 1 -- start start 2024-03-30T02:42:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.348+0000 7fd2c5a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c0100f60 0x7fd2c01993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.348+0000 7fd2c5a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2c0106080 0x7fd2c0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.348+0000 7fd2c5a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c019dca0 0x7fd2c019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.349+0000 7fd2c5a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c019e720 con 0x7fd2c0106080 2024-03-30T02:42:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.349+0000 7fd2c5a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c019e890 con 0x7fd2c0100f60 2024-03-30T02:42:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.349+0000 7fd2c5a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c019ea50 con 0x7fd2c019dca0 2024-03-30T02:42:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.349+0000 7fd2bea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c0100f60 0x7fd2c01993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.349+0000 7fd2bea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c0100f60 0x7fd2c01993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39576/0 (socket says 172.21.15.67:39576) 2024-03-30T02:42:27.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.349+0000 7fd2bea00700 1 -- 172.21.15.67:0/366675138 learned_addr learned my addr 172.21.15.67:0/366675138 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:27.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.350+0000 7fd2be000700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2c0106080 0x7fd2c0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.350+0000 7fd2bf400700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c019dca0 0x7fd2c019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.351+0000 7fd2be000700 1 -- 172.21.15.67:0/366675138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c019dca0 msgr2=0x7fd2c019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:27.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.351+0000 7fd2be000700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c019dca0 0x7fd2c019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.351+0000 7fd2be000700 1 -- 172.21.15.67:0/366675138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c0100f60 msgr2=0x7fd2c01993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:27.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.351+0000 7fd2be000700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c0100f60 0x7fd2c01993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.351+0000 7fd2be000700 1 -- 172.21.15.67:0/366675138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00095f0 con 0x7fd2c0106080 2024-03-30T02:42:27.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.351+0000 7fd2be000700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2c0106080 0x7fd2c0199900 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd2c006a380 tx=0x7fd2ac00dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:27.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.353+0000 7fd2b7400700 1 -- 172.21.15.67:0/366675138 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac009940 con 0x7fd2c0106080 2024-03-30T02:42:27.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.353+0000 7fd2b7400700 1 -- 172.21.15.67:0/366675138 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ac010460 con 0x7fd2c0106080 2024-03-30T02:42:27.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.353+0000 7fd2c5a00700 1 -- 172.21.15.67:0/366675138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c01a5fb0 con 0x7fd2c0106080 2024-03-30T02:42:27.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.353+0000 7fd2c5a00700 1 -- 172.21.15.67:0/366675138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c01a6480 con 0x7fd2c0106080 2024-03-30T02:42:27.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.353+0000 7fd2b7400700 1 -- 172.21.15.67:0/366675138 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac00b5d0 con 0x7fd2c0106080 2024-03-30T02:42:27.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.362+0000 7fd2b7400700 1 -- 172.21.15.67:0/366675138 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd2ac00f620 con 0x7fd2c0106080 2024-03-30T02:42:27.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.362+0000 7fd2d95c2b80 1 Processor -- start 2024-03-30T02:42:27.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.362+0000 7fd2d95c2b80 1 -- start start 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.363+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.363+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7ecd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.363+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ed01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.363+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7e1b990 con 0x55dbd7ec25d0 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.363+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7e19350 con 0x55dbd7ec2ef0 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.363+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7e14c40 con 0x55dbd7ecddf0 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.364+0000 7fd2bf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.364+0000 7fd2b7400700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd2a40a1390 0x7fd2a40a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.365+0000 7fd2bf400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53536/0 (socket says 172.21.15.67:53536) 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.365+0000 7fd2b7400700 1 -- 172.21.15.67:0/366675138 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd2ac0ccb40 con 0x7fd2c0106080 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.365+0000 7fd2bf400700 1 -- 172.21.15.67:0/1871185470 learned_addr learned my addr 172.21.15.67:0/1871185470 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:27.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.366+0000 7fd2be000700 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ed01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.366+0000 7fd2bf400700 1 -- 172.21.15.67:0/1871185470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 msgr2=0x55dbd7ed01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.366+0000 7fd2bf400700 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ed01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.366+0000 7fd2bea00700 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7ecd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.366+0000 7fd2bf400700 1 -- 172.21.15.67:0/1871185470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 msgr2=0x55dbd7ecd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.367+0000 7fd2bf400700 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7ecd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.367+0000 7fd2bf400700 1 -- 172.21.15.67:0/1871185470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbd7c963b0 con 0x55dbd7ec25d0 2024-03-30T02:42:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.367+0000 7fd2bea00700 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7ecd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:42:27.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.369+0000 7fd2bf400700 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8010390 tx=0x7fd2b80106a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df48b01cbd2534e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:27.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.370+0000 7fd2bf400700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd2a40a1390 0x7fd2a40a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.370+0000 7fd2abe00700 1 -- 172.21.15.67:0/1871185470 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8019070 con 0x55dbd7ec25d0 2024-03-30T02:42:27.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.370+0000 7fd2abe00700 1 -- 172.21.15.67:0/1871185470 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b8004bb0 con 0x55dbd7ec25d0 2024-03-30T02:42:27.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.370+0000 7fd2abe00700 1 -- 172.21.15.67:0/1871185470 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8005390 con 0x55dbd7ec25d0 2024-03-30T02:42:27.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.371+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1871185470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 msgr2=0x55dbd7ec29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:27.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.371+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8010390 tx=0x7fd2b80106a0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.372+0000 7fd2bf400700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd2a40a1390 0x7fd2a40a3850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8010fd0 tx=0x7fd2b8005a90 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:27.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.373+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1871185470 shutdown_connections 2024-03-30T02:42:27.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.373+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.373+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ed01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.373+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1871185470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7ecd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.373+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1871185470 >> 172.21.15.67:0/1871185470 conn(0x55dbd7e226a0 msgr2=0x55dbd7ebea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:27.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.374+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1871185470 shutdown_connections 2024-03-30T02:42:27.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.374+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1871185470 wait complete. 2024-03-30T02:42:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.376+0000 7fd2d95c2b80 1 -- start start 2024-03-30T02:42:27.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.377+0000 7fd2d95c2b80 1 Processor -- start 2024-03-30T02:42:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.378+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ecd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.378+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7eca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.378+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ecae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.378+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7db0d80 con 0x55dbd7ec25d0 2024-03-30T02:42:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.378+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7db0a20 con 0x55dbd7ec2ef0 2024-03-30T02:42:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.378+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7c986a0 con 0x55dbd7ecddf0 2024-03-30T02:42:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.378+0000 7fd2d95c2b80 10 client.0 _ll_register_callbacks cb 0x55dbd7dba470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:27.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.379+0000 7fd2bea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ecd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.379+0000 7fd2bea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ecd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53548/0 (socket says 172.21.15.67:53548) 2024-03-30T02:42:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.379+0000 7fd2bea00700 1 -- 172.21.15.67:0/732480960 learned_addr learned my addr 172.21.15.67:0/732480960 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:27.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.380+0000 7fd2be000700 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7eca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.381+0000 7fd2bea00700 1 -- 172.21.15.67:0/732480960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 msgr2=0x55dbd7ecae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:27.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.382+0000 7fd2bea00700 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ecae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.382+0000 7fd2bea00700 1 -- 172.21.15.67:0/732480960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 msgr2=0x55dbd7eca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.382+0000 7fd2bea00700 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7eca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.382+0000 7fd2bea00700 1 -- 172.21.15.67:0/732480960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b8007e40 con 0x55dbd7ec25d0 2024-03-30T02:42:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.383+0000 7fd2bea00700 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ecd650 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000ba20 tx=0x7fd2b000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a336cb74385c74 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.383+0000 7fd2a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:42:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.384+0000 7fd2a8c00700 1 -- 172.21.15.67:0/732480960 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000bea0 con 0x55dbd7ec25d0 2024-03-30T02:42:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.384+0000 7fd2a8c00700 1 -- 172.21.15.67:0/732480960 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd2b0004500 con 0x55dbd7ec25d0 2024-03-30T02:42:27.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.385+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/732480960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbd7dd5380 con 0x55dbd7ec25d0 2024-03-30T02:42:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.386+0000 7fd2a8c00700 1 -- 172.21.15.67:0/732480960 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2b00046c0 con 0x55dbd7ec25d0 2024-03-30T02:42:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.387+0000 7fd2d95c2b80 10 client.16737 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:27.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.387+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/732480960 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55dbd7c42b60 con 0x55dbd7ec25d0 2024-03-30T02:42:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.391+0000 7fd2a8c00700 1 -- 172.21.15.67:0/732480960 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dbd7dd5380 con 0x55dbd7ec25d0 2024-03-30T02:42:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.391+0000 7fd2a8c00700 1 -- 172.21.15.67:0/732480960 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fd2b002b070 con 0x55dbd7ec25d0 2024-03-30T02:42:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.391+0000 7fd2d95c2b80 10 client.16737 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.391+0000 7fd2d95c2b80 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.391+0000 7fd2d95c2b80 20 client.16737 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:27.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.391+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55dbd7ee6060 0x55dbd7ee8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.391+0000 7fd2d95c2b80 4 client.16737 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:42:27.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.391+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/732480960 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbd7e0d700 con 0x55dbd7ee6060 2024-03-30T02:42:27.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.397+0000 7fd2be000700 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55dbd7ee6060 0x55dbd7ee8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.418+0000 7fd2be000700 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55dbd7ee6060 0x55dbd7ee8480 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1d9c9445b17e0 server_cookie=232325b024729c37 in_seq=0 out_seq=0 2024-03-30T02:42:27.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.419+0000 7fd2a8c00700 10 client.16737 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:27.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.433+0000 7fd2a8c00700 1 -- 172.21.15.67:0/732480960 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dbd7e0d700 con 0x55dbd7ee6060 2024-03-30T02:42:27.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.433+0000 7fd2a8c00700 10 client.16737 handle_command_reply: tid=0 2024-03-30T02:42:27.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.433+0000 7fd2d95c2b80 1 client.16737 shutdown 2024-03-30T02:42:27.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.434+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/732480960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 msgr2=0x55dbd7ecd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:27.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.434+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ecd650 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000ba20 tx=0x7fd2b000ba50 comp rx=0 tx=0).stop 2024-03-30T02:42:27.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.434+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/732480960 shutdown_connections 2024-03-30T02:42:27.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.435+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55dbd7ee6060 0x55dbd7ee8480 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.435+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ecd650 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.435+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ecae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.435+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/732480960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7eca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.435+0000 7fd2a8c00700 0 client.16737 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:27.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.436+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/732480960 >> 172.21.15.67:0/732480960 conn(0x55dbd7e226a0 msgr2=0x55dbd7ec46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:27.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.436+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/732480960 shutdown_connections 2024-03-30T02:42:27.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.436+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/732480960 wait complete. 2024-03-30T02:42:27.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.438+0000 7fd2d95c2b80 20 client.16737 trim_cache size 0 max 16384 2024-03-30T02:42:27.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.471+0000 7fd2d95c2b80 1 Processor -- start 2024-03-30T02:42:27.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.471+0000 7fd2d95c2b80 1 -- start start 2024-03-30T02:42:27.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.471+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.471+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7edd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.471+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ec9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.471+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7ecbe10 con 0x55dbd7ec2ef0 2024-03-30T02:42:27.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.471+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7ecbfe0 con 0x55dbd7ec25d0 2024-03-30T02:42:27.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.471+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7ebf7d0 con 0x55dbd7ecddf0 2024-03-30T02:42:27.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.473+0000 7fd2bf400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ec9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.473+0000 7fd2bf400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ec9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52518/0 (socket says 172.21.15.67:52518) 2024-03-30T02:42:27.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.473+0000 7fd2bf400700 1 -- 172.21.15.67:0/486349272 learned_addr learned my addr 172.21.15.67:0/486349272 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:27.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.473+0000 7fd2bea00700 1 --2- 172.21.15.67:0/486349272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.473+0000 7fd2be000700 1 --2- 172.21.15.67:0/486349272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7edd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.474+0000 7fd2be000700 1 -- 172.21.15.67:0/486349272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 msgr2=0x55dbd7ec9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:27.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.475+0000 7fd2be000700 1 --2- 172.21.15.67:0/486349272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ec9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.475+0000 7fd2be000700 1 -- 172.21.15.67:0/486349272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec25d0 msgr2=0x55dbd7ec29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:27.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.475+0000 7fd2be000700 1 --2- 172.21.15.67:0/486349272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.475+0000 7fd2be000700 1 -- 172.21.15.67:0/486349272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbd7ddaea0 con 0x55dbd7ec2ef0 2024-03-30T02:42:27.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.477+0000 7fd2be000700 1 --2- 172.21.15.67:0/486349272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7edd310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0c9ee0 tx=0x7fd2ac095c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4396628b7e1ed08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:27.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.478+0000 7fd2b4c00700 1 -- 172.21.15.67:0/486349272 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac014070 con 0x55dbd7ec2ef0 2024-03-30T02:42:27.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.478+0000 7fd2b4c00700 1 -- 172.21.15.67:0/486349272 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2ac0b7e90 con 0x55dbd7ec2ef0 2024-03-30T02:42:27.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.478+0000 7fd2b4c00700 1 -- 172.21.15.67:0/486349272 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac098ff0 con 0x55dbd7ec2ef0 2024-03-30T02:42:27.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.479+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/486349272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec2ef0 msgr2=0x55dbd7edd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:27.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.479+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/486349272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7edd310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0c9ee0 tx=0x7fd2ac095c60 comp rx=0 tx=0).stop 2024-03-30T02:42:27.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.479+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/486349272 shutdown_connections 2024-03-30T02:42:27.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.479+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/486349272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ecddf0 0x55dbd7ec9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.479+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/486349272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec25d0 0x55dbd7ec29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.479+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/486349272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ec2ef0 0x55dbd7edd310 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.479+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/486349272 >> 172.21.15.67:0/486349272 conn(0x55dbd7e226a0 msgr2=0x55dbd7eb7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:27.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.480+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/486349272 shutdown_connections 2024-03-30T02:42:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.481+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/486349272 wait complete. 2024-03-30T02:42:27.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.481+0000 7fd2d95c2b80 1 -- start start 2024-03-30T02:42:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.482+0000 7fd2d95c2b80 1 Processor -- start 2024-03-30T02:42:27.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.482+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec25d0 0x55dbd7edcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.482+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ecddf0 0x55dbd7edb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.482+0000 7fd2d95c2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ee6060 0x55dbd7edb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.483+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7edbdd0 con 0x55dbd7ecddf0 2024-03-30T02:42:27.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.483+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7edbf10 con 0x55dbd7ec25d0 2024-03-30T02:42:27.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.483+0000 7fd2d95c2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbd7edc0d0 con 0x55dbd7ee6060 2024-03-30T02:42:27.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.483+0000 7fd2d95c2b80 10 client.0 _ll_register_callbacks cb 0x55dbd7dba470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:27.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.484+0000 7fd2be000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ecddf0 0x55dbd7edb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.484+0000 7fd2be000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ecddf0 0x55dbd7edb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53560/0 (socket says 172.21.15.67:53560) 2024-03-30T02:42:27.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.485+0000 7fd2be000700 1 -- 172.21.15.67:0/1745531898 learned_addr learned my addr 172.21.15.67:0/1745531898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.485+0000 7fd2be000700 1 -- 172.21.15.67:0/1745531898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ee6060 msgr2=0x55dbd7edb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.485+0000 7fd2be000700 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ee6060 0x55dbd7edb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.485+0000 7fd2be000700 1 -- 172.21.15.67:0/1745531898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec25d0 msgr2=0x55dbd7edcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.485+0000 7fd2be000700 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec25d0 0x55dbd7edcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.485+0000 7fd2be000700 1 -- 172.21.15.67:0/1745531898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c01a6480 con 0x55dbd7ecddf0 2024-03-30T02:42:27.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.486+0000 7fd2be000700 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ecddf0 0x55dbd7edb350 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac000c00 tx=0x7fd2ac097b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966ebb7e25f967c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:27.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.486+0000 7fd2a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:42:27.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.487+0000 7fd2a8c00700 1 -- 172.21.15.67:0/1745531898 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac014030 con 0x55dbd7ecddf0 2024-03-30T02:42:27.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.489+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1745531898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2b8004bb0 con 0x55dbd7ecddf0 2024-03-30T02:42:27.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.490+0000 7fd2a8c00700 1 -- 172.21.15.67:0/1745531898 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd2ac015070 con 0x55dbd7ecddf0 2024-03-30T02:42:27.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.490+0000 7fd2a8c00700 1 -- 172.21.15.67:0/1745531898 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2ac099b00 con 0x55dbd7ecddf0 2024-03-30T02:42:27.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.491+0000 7fd2d95c2b80 10 client.16743 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:27.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.491+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1745531898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55dbd7c9acd0 con 0x55dbd7ecddf0 2024-03-30T02:42:27.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.492+0000 7fd2a8c00700 1 -- 172.21.15.67:0/1745531898 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b8004bb0 con 0x55dbd7ecddf0 2024-03-30T02:42:27.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.492+0000 7fd2a8c00700 1 -- 172.21.15.67:0/1745531898 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fd2ac095690 con 0x55dbd7ecddf0 2024-03-30T02:42:27.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.492+0000 7fd2d95c2b80 10 client.16743 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:27.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.492+0000 7fd2d95c2b80 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:27.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.492+0000 7fd2d95c2b80 20 client.16743 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:27.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.492+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55dbd7f3af90 0x55dbd7f3d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:27.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.493+0000 7fd2d95c2b80 4 client.16743 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:42:27.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.494+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1745531898 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dbd7c63cf0 con 0x55dbd7f3af90 2024-03-30T02:42:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.499+0000 7fd2bea00700 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55dbd7f3af90 0x55dbd7f3d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:27.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.509+0000 7fd2bea00700 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55dbd7f3af90 0x55dbd7f3d3b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a7c16f2e7a52a6 server_cookie=9231a6fc2acf9a58 in_seq=0 out_seq=0 2024-03-30T02:42:27.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.509+0000 7fd2a8c00700 10 client.16743 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.538+0000 7fd2a8c00700 1 -- 172.21.15.67:0/1745531898 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dbd7c63cf0 con 0x55dbd7f3af90 2024-03-30T02:42:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.538+0000 7fd2a8c00700 10 client.16743 handle_command_reply: tid=0 2024-03-30T02:42:27.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.538+0000 7fd2d95c2b80 1 client.16743 shutdown 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.538+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1745531898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ecddf0 msgr2=0x55dbd7edb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.538+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ecddf0 0x55dbd7edb350 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac000c00 tx=0x7fd2ac097b00 comp rx=0 tx=0).stop 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1745531898 shutdown_connections 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbd7ee6060 0x55dbd7edb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbd7ec25d0 0x55dbd7edcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55dbd7f3af90 0x55dbd7f3d3b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2d95c2b80 1 --2- 172.21.15.67:0/1745531898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55dbd7ecddf0 0x55dbd7edb350 secure :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac000c00 tx=0x7fd2ac097b00 comp rx=0 tx=0).stop 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1745531898 >> 172.21.15.67:0/1745531898 conn(0x55dbd7e226a0 msgr2=0x55dbd7ec9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2a8c00700 0 client.16743 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1745531898 shutdown_connections 2024-03-30T02:42:27.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2d95c2b80 1 -- 172.21.15.67:0/1745531898 wait complete. 2024-03-30T02:42:27.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.539+0000 7fd2d95c2b80 20 client.16743 trim_cache size 0 max 16384 2024-03-30T02:42:27.540 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.541+0000 7fd2c5a00700 1 -- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd2a40a1390 msgr2=0x7fd2a40a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.541+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd2a40a1390 0x7fd2a40a3850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8010fd0 tx=0x7fd2b8005a90 comp rx=0 tx=0).stop 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.541+0000 7fd2c5a00700 1 -- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2c0106080 msgr2=0x7fd2c0199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.541+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2c0106080 0x7fd2c0199900 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd2c006a380 tx=0x7fd2ac00dce0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.542+0000 7fd2c5a00700 1 -- 172.21.15.67:0/366675138 shutdown_connections 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.542+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd2a40a1390 0x7fd2a40a3850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.542+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c019dca0 0x7fd2c019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.542+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c0100f60 0x7fd2c01993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.542+0000 7fd2c5a00700 1 --2- 172.21.15.67:0/366675138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2c0106080 0x7fd2c0199900 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.542+0000 7fd2c5a00700 1 -- 172.21.15.67:0/366675138 >> 172.21.15.67:0/366675138 conn(0x7fd2c00fc7c0 msgr2=0x7fd2c00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:27.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.542+0000 7fd2c5a00700 1 -- 172.21.15.67:0/366675138 shutdown_connections 2024-03-30T02:42:27.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:27.543+0000 7fd2c5a00700 1 -- 172.21.15.67:0/366675138 wait complete. 2024-03-30T02:42:28.587 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:42:28.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:42:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:28 smithi170 ceph-mon[78798]: pgmap v165: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-03-30T02:42:28.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:28 smithi067 ceph-mon[37808]: pgmap v165: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-03-30T02:42:28.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:28 smithi082 ceph-mon[79176]: pgmap v165: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-03-30T02:42:29.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.007+0000 7fd429200700 1 -- 172.21.15.67:0/1848059163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 msgr2=0x7fd424073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.007+0000 7fd429200700 1 --2- 172.21.15.67:0/1848059163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 0x7fd424073560 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009990 tx=0x7fd41c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.007+0000 7fd429200700 1 -- 172.21.15.67:0/1848059163 shutdown_connections 2024-03-30T02:42:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.007+0000 7fd429200700 1 --2- 172.21.15.67:0/1848059163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42410d580 0x7fd42410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.007+0000 7fd429200700 1 --2- 172.21.15.67:0/1848059163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 0x7fd424073560 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.007+0000 7fd429200700 1 --2- 172.21.15.67:0/1848059163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd424073aa0 0x7fd42410d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.007+0000 7fd429200700 1 -- 172.21.15.67:0/1848059163 >> 172.21.15.67:0/1848059163 conn(0x7fd4240fc7f0 msgr2=0x7fd4240fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.007+0000 7fd429200700 1 -- 172.21.15.67:0/1848059163 shutdown_connections 2024-03-30T02:42:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.008+0000 7fd429200700 1 -- 172.21.15.67:0/1848059163 wait complete. 2024-03-30T02:42:29.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.009+0000 7fd429200700 1 Processor -- start 2024-03-30T02:42:29.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.010+0000 7fd429200700 1 -- start start 2024-03-30T02:42:29.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.011+0000 7fd429200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 0x7fd4241994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.012+0000 7fd429200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd424073aa0 0x7fd424199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.012+0000 7fd429200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42410d580 0x7fd42419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.012+0000 7fd421e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 0x7fd4241994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.013+0000 7fd421e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 0x7fd4241994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39628/0 (socket says 172.21.15.67:39628) 2024-03-30T02:42:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.013+0000 7fd421400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd424073aa0 0x7fd424199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.013+0000 7fd421400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd424073aa0 0x7fd424199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52526/0 (socket says 172.21.15.67:52526) 2024-03-30T02:42:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.013+0000 7fd422800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42410d580 0x7fd42419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.013+0000 7fd422800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42410d580 0x7fd42419dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53572/0 (socket says 172.21.15.67:53572) 2024-03-30T02:42:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.014+0000 7fd422800700 1 -- 172.21.15.67:0/1536790667 learned_addr learned my addr 172.21.15.67:0/1536790667 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.014+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42419e380 con 0x7fd42410d580 2024-03-30T02:42:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.014+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42419e4f0 con 0x7fd424073180 2024-03-30T02:42:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.014+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42419e6b0 con 0x7fd424073aa0 2024-03-30T02:42:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.015+0000 7fd421e00700 1 -- 172.21.15.67:0/1536790667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd424073aa0 msgr2=0x7fd424199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.015+0000 7fd421e00700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd424073aa0 0x7fd424199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.015+0000 7fd421e00700 1 -- 172.21.15.67:0/1536790667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42410d580 msgr2=0x7fd42419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.015+0000 7fd421e00700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42410d580 0x7fd42419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.015+0000 7fd421e00700 1 -- 172.21.15.67:0/1536790667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c0095f0 con 0x7fd424073180 2024-03-30T02:42:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.016+0000 7fd421e00700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 0x7fd4241994d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009990 tx=0x7fd41c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.017+0000 7fd41aa00700 1 -- 172.21.15.67:0/1536790667 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c01d070 con 0x7fd424073180 2024-03-30T02:42:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.017+0000 7fd41aa00700 1 -- 172.21.15.67:0/1536790667 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd41c022470 con 0x7fd424073180 2024-03-30T02:42:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.017+0000 7fd41aa00700 1 -- 172.21.15.67:0/1536790667 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c017650 con 0x7fd424073180 2024-03-30T02:42:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.017+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42419e930 con 0x7fd424073180 2024-03-30T02:42:29.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.017+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42419ebb0 con 0x7fd424073180 2024-03-30T02:42:29.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.028+0000 7fd41aa00700 1 -- 172.21.15.67:0/1536790667 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd41c022850 con 0x7fd424073180 2024-03-30T02:42:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.032+0000 7fd41aa00700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4080a1310 0x7fd4080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.032+0000 7fd41aa00700 1 -- 172.21.15.67:0/1536790667 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd41c0ce0a0 con 0x7fd424073180 2024-03-30T02:42:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.033+0000 7fd43cc49b80 1 Processor -- start 2024-03-30T02:42:29.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.033+0000 7fd43cc49b80 1 -- start start 2024-03-30T02:42:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.033+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd113a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.034+0000 7fd421400700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4080a1310 0x7fd4080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.035+0000 7fd421400700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4080a1310 0x7fd4080a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4240fef50 tx=0x7fd40c009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.035+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113f70 0x5575bd11e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.036+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd121260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.036+0000 7fd43cc49b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd06ca10 con 0x5575bd11ee70 2024-03-30T02:42:29.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.036+0000 7fd43cc49b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd06a3d0 con 0x5575bd113f70 2024-03-30T02:42:29.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.036+0000 7fd43cc49b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd065cc0 con 0x5575bd113650 2024-03-30T02:42:29.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.036+0000 7fd421400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113f70 0x5575bd11e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.036+0000 7fd421400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113f70 0x5575bd11e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39638/0 (socket says 172.21.15.67:39638) 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.036+0000 7fd421400700 1 -- 172.21.15.67:0/1019018803 learned_addr learned my addr 172.21.15.67:0/1019018803 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.036+0000 7fd421e00700 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.036+0000 7fd422800700 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd121260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.037+0000 7fd421e00700 1 -- 172.21.15.67:0/1019018803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113f70 msgr2=0x5575bd11e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.037+0000 7fd421e00700 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113f70 0x5575bd11e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.037+0000 7fd421e00700 1 -- 172.21.15.67:0/1019018803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 msgr2=0x5575bd121260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.037+0000 7fd421e00700 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd121260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.037+0000 7fd421e00700 1 -- 172.21.15.67:0/1019018803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575bcee7430 con 0x5575bd113650 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.037+0000 7fd422800700 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd121260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:42:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.037+0000 7fd421400700 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113f70 0x5575bd11e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:42:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.038+0000 7fd421e00700 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd113a30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0cb4a0 tx=0x7fd41c022bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ba4da2c794e106 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.038+0000 7fd413400700 1 -- 172.21.15.67:0/1019018803 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c09b010 con 0x5575bd113650 2024-03-30T02:42:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.038+0000 7fd413400700 1 -- 172.21.15.67:0/1019018803 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd41c09ee40 con 0x5575bd113650 2024-03-30T02:42:29.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.038+0000 7fd413400700 1 -- 172.21.15.67:0/1019018803 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c09b4d0 con 0x5575bd113650 2024-03-30T02:42:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.039+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1019018803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 msgr2=0x5575bd113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.039+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd113a30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0cb4a0 tx=0x7fd41c022bf0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.041+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1019018803 shutdown_connections 2024-03-30T02:42:29.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.041+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd121260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.041+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd113a30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.041+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1019018803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113f70 0x5575bd11e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.041+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1019018803 >> 172.21.15.67:0/1019018803 conn(0x5575bd073720 msgr2=0x5575bd10fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.042+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1019018803 shutdown_connections 2024-03-30T02:42:29.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.042+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1019018803 wait complete. 2024-03-30T02:42:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.042+0000 7fd43cc49b80 1 -- start start 2024-03-30T02:42:29.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.043+0000 7fd43cc49b80 1 Processor -- start 2024-03-30T02:42:29.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.045+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd11e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.045+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113f70 0x5575bd11bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.046+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 0x5575bd11c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.046+0000 7fd43cc49b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd001e00 con 0x5575bd113f70 2024-03-30T02:42:29.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.046+0000 7fd43cc49b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd001aa0 con 0x5575bd11ee70 2024-03-30T02:42:29.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.046+0000 7fd43cc49b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bcee9720 con 0x5575bd113650 2024-03-30T02:42:29.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.046+0000 7fd43cc49b80 10 client.0 _ll_register_callbacks cb 0x5575bd00b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:29.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.046+0000 7fd421e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd11e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.048+0000 7fd422800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 0x5575bd11c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.048+0000 7fd421e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd11e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52534/0 (socket says 172.21.15.67:52534) 2024-03-30T02:42:29.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.048+0000 7fd421e00700 1 -- 172.21.15.67:0/2974560308 learned_addr learned my addr 172.21.15.67:0/2974560308 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.048+0000 7fd421400700 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113f70 0x5575bd11bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.049+0000 7fd421e00700 1 -- 172.21.15.67:0/2974560308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 msgr2=0x5575bd11c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.049+0000 7fd421e00700 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 0x5575bd11c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.049+0000 7fd421e00700 1 -- 172.21.15.67:0/2974560308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113f70 msgr2=0x5575bd11bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.049+0000 7fd421e00700 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113f70 0x5575bd11bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.049+0000 7fd421e00700 1 -- 172.21.15.67:0/2974560308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42419e930 con 0x5575bd113650 2024-03-30T02:42:29.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.049+0000 7fd421400700 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113f70 0x5575bd11bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:42:29.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.049+0000 7fd421e00700 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd11e8b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0ce380 tx=0x7fd41c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4928e79b199af server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.049+0000 7fd407e00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:42:29.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.050+0000 7fd407e00700 1 -- 172.21.15.67:0/2974560308 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0b9370 con 0x5575bd113650 2024-03-30T02:42:29.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.050+0000 7fd407e00700 1 -- 172.21.15.67:0/2974560308 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd41c09b010 con 0x5575bd113650 2024-03-30T02:42:29.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.050+0000 7fd43cc49b80 1 -- 172.21.15.67:0/2974560308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575bd026400 con 0x5575bd113650 2024-03-30T02:42:29.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.051+0000 7fd407e00700 1 -- 172.21.15.67:0/2974560308 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd41c09b270 con 0x5575bd113650 2024-03-30T02:42:29.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.051+0000 7fd43cc49b80 10 client.26803 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:29.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.051+0000 7fd43cc49b80 1 -- 172.21.15.67:0/2974560308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5575bce93be0 con 0x5575bd113650 2024-03-30T02:42:29.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.052+0000 7fd407e00700 1 -- 172.21.15.67:0/2974560308 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575bd026400 con 0x5575bd113650 2024-03-30T02:42:29.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.052+0000 7fd407e00700 1 -- 172.21.15.67:0/2974560308 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fd41c096f10 con 0x5575bd113650 2024-03-30T02:42:29.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.052+0000 7fd43cc49b80 10 client.26803 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:29.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.052+0000 7fd43cc49b80 10 client.26803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:29.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.052+0000 7fd43cc49b80 20 client.26803 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:29.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.052+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5575bd1369e0 0x5575bd138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.052+0000 7fd43cc49b80 4 client.26803 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:42:29.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.052+0000 7fd43cc49b80 1 -- 172.21.15.67:0/2974560308 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575bd05e780 con 0x5575bd1369e0 2024-03-30T02:42:29.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.053+0000 7fd421400700 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5575bd1369e0 0x5575bd138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.054+0000 7fd407e00700 10 client.26803 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:29.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.054+0000 7fd421400700 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5575bd1369e0 0x5575bd138dd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b82d75d4844f2a server_cookie=2765be39f3c1567a in_seq=0 out_seq=0 2024-03-30T02:42:29.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.056+0000 7fd407e00700 1 -- 172.21.15.67:0/2974560308 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5575bd05e780 con 0x5575bd1369e0 2024-03-30T02:42:29.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.056+0000 7fd407e00700 10 client.26803 handle_command_reply: tid=0 2024-03-30T02:42:29.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.056+0000 7fd43cc49b80 1 client.26803 shutdown 2024-03-30T02:42:29.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.058+0000 7fd43cc49b80 1 -- 172.21.15.67:0/2974560308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 msgr2=0x5575bd11e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.058+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd11e8b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0ce380 tx=0x7fd41c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.058+0000 7fd43cc49b80 1 -- 172.21.15.67:0/2974560308 shutdown_connections 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.058+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 0x5575bd11c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.058+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113650 0x5575bd11e8b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.058+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5575bd1369e0 0x5575bd138dd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.058+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/2974560308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113f70 0x5575bd11bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.058+0000 7fd43cc49b80 1 -- 172.21.15.67:0/2974560308 >> 172.21.15.67:0/2974560308 conn(0x5575bd073720 msgr2=0x5575bd115740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.059+0000 7fd407e00700 0 client.26803 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.059+0000 7fd43cc49b80 1 -- 172.21.15.67:0/2974560308 shutdown_connections 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.059+0000 7fd43cc49b80 1 -- 172.21.15.67:0/2974560308 wait complete. 2024-03-30T02:42:29.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.059+0000 7fd43cc49b80 20 client.26803 trim_cache size 0 max 16384 2024-03-30T02:42:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.104+0000 7fd43cc49b80 1 Processor -- start 2024-03-30T02:42:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.104+0000 7fd43cc49b80 1 -- start start 2024-03-30T02:42:29.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113650 0x5575bd12e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd11b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 0x5575bd11b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd43cc49b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd1146c0 con 0x5575bd113650 2024-03-30T02:42:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd43cc49b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd11d1a0 con 0x5575bd11ee70 2024-03-30T02:42:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd43cc49b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd11c8c0 con 0x5575bd113f70 2024-03-30T02:42:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd421400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd11b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd421e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113650 0x5575bd12e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd421400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd11b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52546/0 (socket says 172.21.15.67:52546) 2024-03-30T02:42:29.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd421400700 1 -- 172.21.15.67:0/4064061230 learned_addr learned my addr 172.21.15.67:0/4064061230 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.105+0000 7fd422800700 1 --2- 172.21.15.67:0/4064061230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 0x5575bd11b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.106+0000 7fd421400700 1 -- 172.21.15.67:0/4064061230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 msgr2=0x5575bd11b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.106+0000 7fd421400700 1 --2- 172.21.15.67:0/4064061230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 0x5575bd11b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.106+0000 7fd421400700 1 -- 172.21.15.67:0/4064061230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113650 msgr2=0x5575bd12e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.106+0000 7fd421400700 1 --2- 172.21.15.67:0/4064061230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113650 0x5575bd12e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.106+0000 7fd421400700 1 -- 172.21.15.67:0/4064061230 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575bd02bf20 con 0x5575bd113f70 2024-03-30T02:42:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.107+0000 7fd421400700 1 --2- 172.21.15.67:0/4064061230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd11b390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0117d0 tx=0x7fd40c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603a72e239b08647 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.107+0000 7fd413e00700 1 -- 172.21.15.67:0/4064061230 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c024070 con 0x5575bd113f70 2024-03-30T02:42:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.107+0000 7fd413e00700 1 -- 172.21.15.67:0/4064061230 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c004510 con 0x5575bd113f70 2024-03-30T02:42:29.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.107+0000 7fd413e00700 1 -- 172.21.15.67:0/4064061230 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0265d0 con 0x5575bd113f70 2024-03-30T02:42:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.108+0000 7fd43cc49b80 1 -- 172.21.15.67:0/4064061230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 msgr2=0x5575bd11b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.108+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/4064061230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd11b390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0117d0 tx=0x7fd40c014000 comp rx=0 tx=0).stop 2024-03-30T02:42:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.109+0000 7fd43cc49b80 1 -- 172.21.15.67:0/4064061230 shutdown_connections 2024-03-30T02:42:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.109+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/4064061230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd113650 0x5575bd12e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.109+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/4064061230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd11ee70 0x5575bd11b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.109+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/4064061230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd11b390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.109+0000 7fd43cc49b80 1 -- 172.21.15.67:0/4064061230 >> 172.21.15.67:0/4064061230 conn(0x5575bd073720 msgr2=0x5575bd116ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.109+0000 7fd43cc49b80 1 -- 172.21.15.67:0/4064061230 shutdown_connections 2024-03-30T02:42:29.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.109+0000 7fd43cc49b80 1 -- 172.21.15.67:0/4064061230 wait complete. 2024-03-30T02:42:29.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.111+0000 7fd43cc49b80 1 -- start start 2024-03-30T02:42:29.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.112+0000 7fd43cc49b80 1 Processor -- start 2024-03-30T02:42:29.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.113+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113650 0x5575bd184b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.113+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd185060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.113+0000 7fd43cc49b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd12cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.114+0000 7fd421e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113650 0x5575bd184b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.114+0000 7fd421e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113650 0x5575bd184b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39670/0 (socket says 172.21.15.67:39670) 2024-03-30T02:42:29.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.114+0000 7fd421e00700 1 -- 172.21.15.67:0/1251340339 learned_addr learned my addr 172.21.15.67:0/1251340339 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.114+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd12d570 con 0x5575bd11ee70 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.114+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd12d6e0 con 0x5575bd113650 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.114+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575bd12d8a0 con 0x5575bd113f70 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.114+0000 7fd422800700 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd12cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.114+0000 7fd43cc49b80 10 client.0 _ll_register_callbacks cb 0x5575bd00b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.114+0000 7fd421400700 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd185060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.115+0000 7fd421e00700 1 -- 172.21.15.67:0/1251340339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 msgr2=0x5575bd185060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.115+0000 7fd421e00700 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd185060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.115+0000 7fd421e00700 1 -- 172.21.15.67:0/1251340339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 msgr2=0x5575bd12cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.115+0000 7fd421e00700 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd12cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.115+0000 7fd421e00700 1 -- 172.21.15.67:0/1251340339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c017070 con 0x5575bd113650 2024-03-30T02:42:29.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.115+0000 7fd421400700 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd185060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:42:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.115+0000 7fd422800700 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd12cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:42:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.116+0000 7fd407e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.116+0000 7fd421e00700 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113650 0x5575bd184b20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd41c099360 tx=0x7fd41c09aeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06e9d68fa89b20d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.116+0000 7fd407e00700 1 -- 172.21.15.67:0/1251340339 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00fbf0 con 0x5575bd113650 2024-03-30T02:42:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.116+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd41c09ee40 con 0x5575bd113650 2024-03-30T02:42:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.116+0000 7fd407e00700 1 -- 172.21.15.67:0/1251340339 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fd41c09b010 con 0x5575bd113650 2024-03-30T02:42:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.116+0000 7fd407e00700 1 -- 172.21.15.67:0/1251340339 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd41c00fbf0 con 0x5575bd113650 2024-03-30T02:42:29.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.116+0000 7fd43cc49b80 10 client.26852 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:29.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.116+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5575bceebd50 con 0x5575bd113650 2024-03-30T02:42:29.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.117+0000 7fd407e00700 1 -- 172.21.15.67:0/1251340339 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd41c09ee40 con 0x5575bd113650 2024-03-30T02:42:29.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.117+0000 7fd407e00700 1 -- 172.21.15.67:0/1251340339 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fd41c096f00 con 0x5575bd113650 2024-03-30T02:42:29.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.117+0000 7fd43cc49b80 10 client.26852 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:29.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.117+0000 7fd43cc49b80 10 client.26852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:29.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.117+0000 7fd43cc49b80 20 client.26852 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:29.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.117+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5575bd18c5b0 0x5575bd18e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.117+0000 7fd43cc49b80 4 client.26852 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:42:29.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.117+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5575bceb4d70 con 0x5575bd18c5b0 2024-03-30T02:42:29.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.117+0000 7fd421400700 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5575bd18c5b0 0x5575bd18e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.118+0000 7fd421400700 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5575bd18c5b0 0x5575bd18e9a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da79cf997c91767 server_cookie=baffbb536acaada4 in_seq=0 out_seq=0 2024-03-30T02:42:29.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.118+0000 7fd407e00700 10 client.26852 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:29.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.120+0000 7fd407e00700 1 -- 172.21.15.67:0/1251340339 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5575bceb4d70 con 0x5575bd18c5b0 2024-03-30T02:42:29.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.120+0000 7fd407e00700 10 client.26852 handle_command_reply: tid=0 2024-03-30T02:42:29.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.120+0000 7fd43cc49b80 1 client.26852 shutdown 2024-03-30T02:42:29.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.122+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113650 msgr2=0x5575bd184b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.122+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113650 0x5575bd184b20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd41c099360 tx=0x7fd41c09aeb0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.122+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 shutdown_connections 2024-03-30T02:42:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.122+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5575bd18c5b0 0x5575bd18e9a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.122+0000 7fd407e00700 0 client.26852 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.122+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5575bd11ee70 0x5575bd12cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.122+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575bd113650 0x5575bd184b20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.122+0000 7fd43cc49b80 1 --2- 172.21.15.67:0/1251340339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5575bd113f70 0x5575bd185060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.122+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 >> 172.21.15.67:0/1251340339 conn(0x5575bd073720 msgr2=0x5575bd11d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.125+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 shutdown_connections 2024-03-30T02:42:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.125+0000 7fd43cc49b80 1 -- 172.21.15.67:0/1251340339 wait complete. 2024-03-30T02:42:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.125+0000 7fd43cc49b80 20 client.26852 trim_cache size 0 max 16384 2024-03-30T02:42:29.125 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:42:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:42:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "e32ff20d-c7a4-48b3-b5dc-08cb42a587a6", 2024-03-30T02:42:29.125 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:42:29.125 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:42:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4080a1310 msgr2=0x7fd4080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4080a1310 0x7fd4080a37d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd4240fef50 tx=0x7fd40c009260 comp rx=0 tx=0).stop 2024-03-30T02:42:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 msgr2=0x7fd4241994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 0x7fd4241994d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009990 tx=0x7fd41c005740 comp rx=0 tx=0).stop 2024-03-30T02:42:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 shutdown_connections 2024-03-30T02:42:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4080a1310 0x7fd4080a37d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd42410d580 0x7fd42419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd424073180 0x7fd4241994d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 --2- 172.21.15.67:0/1536790667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd424073aa0 0x7fd424199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.126+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 >> 172.21.15.67:0/1536790667 conn(0x7fd4240fc7f0 msgr2=0x7fd4240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.127+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 shutdown_connections 2024-03-30T02:42:29.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.127+0000 7fd429200700 1 -- 172.21.15.67:0/1536790667 wait complete. 2024-03-30T02:42:29.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:42:29.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.557+0000 7fc90f200700 1 -- 172.21.15.67:0/1130408153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc908104f60 msgr2=0x7fc90810cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.557+0000 7fc90f200700 1 --2- 172.21.15.67:0/1130408153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc908104f60 0x7fc90810cae0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80098f0 tx=0x7fc8f8009c00 comp rx=0 tx=0).stop 2024-03-30T02:42:29.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.559+0000 7fc90f200700 1 -- 172.21.15.67:0/1130408153 shutdown_connections 2024-03-30T02:42:29.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.559+0000 7fc90f200700 1 --2- 172.21.15.67:0/1130408153 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc9080ffe50 0x7fc908100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.559+0000 7fc90f200700 1 --2- 172.21.15.67:0/1130408153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc908104f60 0x7fc90810cae0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.559+0000 7fc90f200700 1 --2- 172.21.15.67:0/1130408153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc908100800 0x7fc908104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.559+0000 7fc90f200700 1 -- 172.21.15.67:0/1130408153 >> 172.21.15.67:0/1130408153 conn(0x7fc908076d40 msgr2=0x7fc908077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.559+0000 7fc90f200700 1 -- 172.21.15.67:0/1130408153 shutdown_connections 2024-03-30T02:42:29.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.559+0000 7fc90f200700 1 -- 172.21.15.67:0/1130408153 wait complete. 2024-03-30T02:42:29.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.562+0000 7fc90f200700 1 Processor -- start 2024-03-30T02:42:29.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.562+0000 7fc90f200700 1 -- start start 2024-03-30T02:42:29.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.563+0000 7fc90f200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9080ffe50 0x7fc9081a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.563+0000 7fc907e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9080ffe50 0x7fc9081a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.564+0000 7fc907e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9080ffe50 0x7fc9081a3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39698/0 (socket says 172.21.15.67:39698) 2024-03-30T02:42:29.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.564+0000 7fc90f200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc908104f60 0x7fc9081a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.564+0000 7fc907e00700 1 -- 172.21.15.67:0/2465841179 learned_addr learned my addr 172.21.15.67:0/2465841179 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.564+0000 7fc90f200700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc90819d640 0x7fc90819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.564+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819e0c0 con 0x7fc90819d640 2024-03-30T02:42:29.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.564+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819e230 con 0x7fc9080ffe50 2024-03-30T02:42:29.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.564+0000 7fc907400700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc908104f60 0x7fc9081a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.564+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819e3f0 con 0x7fc908104f60 2024-03-30T02:42:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.564+0000 7fc90d200700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc90819d640 0x7fc90819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.565+0000 7fc907e00700 1 -- 172.21.15.67:0/2465841179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc908104f60 msgr2=0x7fc9081a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.565+0000 7fc907e00700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc908104f60 0x7fc9081a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.565+0000 7fc907e00700 1 -- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc90819d640 msgr2=0x7fc90819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.565+0000 7fc907e00700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc90819d640 0x7fc90819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.565+0000 7fc907e00700 1 -- 172.21.15.67:0/2465841179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f8009520 con 0x7fc9080ffe50 2024-03-30T02:42:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.565+0000 7fc907e00700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9080ffe50 0x7fc9081a3510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc9000060b0 tx=0x7fc900007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.566+0000 7fc907400700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc908104f60 0x7fc9081a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:29.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.566+0000 7fc90d200700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc90819d640 0x7fc90819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:42:29.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.566+0000 7fc904c00700 1 -- 172.21.15.67:0/2465841179 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90000dd60 con 0x7fc9080ffe50 2024-03-30T02:42:29.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.566+0000 7fc904c00700 1 -- 172.21.15.67:0/2465841179 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc90000f040 con 0x7fc9080ffe50 2024-03-30T02:42:29.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.566+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9081aa3e0 con 0x7fc9080ffe50 2024-03-30T02:42:29.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.566+0000 7fc904c00700 1 -- 172.21.15.67:0/2465841179 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900014690 con 0x7fc9080ffe50 2024-03-30T02:42:29.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.566+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9081aa8c0 con 0x7fc9080ffe50 2024-03-30T02:42:29.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.571+0000 7fc904c00700 1 -- 172.21.15.67:0/2465841179 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc900015950 con 0x7fc9080ffe50 2024-03-30T02:42:29.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.574+0000 7fc904c00700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc8ec0a13c0 0x7fc8ec0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.574+0000 7fc904c00700 1 -- 172.21.15.67:0/2465841179 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fc9000cc980 con 0x7fc9080ffe50 2024-03-30T02:42:29.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.575+0000 7fc907400700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc8ec0a13c0 0x7fc8ec0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.577+0000 7fc922d47b80 1 Processor -- start 2024-03-30T02:42:29.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.577+0000 7fc922d47b80 1 -- start start 2024-03-30T02:42:29.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.578+0000 7fc907400700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc8ec0a13c0 0x7fc8ec0a3880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc90819ed50 tx=0x7fc8f4009310 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.578+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 0x5594987ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.578+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987ceef0 0x5594987d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.578+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987d9df0 0x5594987dc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.578+0000 7fc922d47b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x559498727990 con 0x5594987ce5d0 2024-03-30T02:42:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.578+0000 7fc922d47b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559498725350 con 0x5594987ceef0 2024-03-30T02:42:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.578+0000 7fc922d47b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559498720c40 con 0x5594987d9df0 2024-03-30T02:42:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.578+0000 7fc907400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987ceef0 0x5594987d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.578+0000 7fc907400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987ceef0 0x5594987d98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39700/0 (socket says 172.21.15.67:39700) 2024-03-30T02:42:29.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.579+0000 7fc907400700 1 -- 172.21.15.67:0/3380852953 learned_addr learned my addr 172.21.15.67:0/3380852953 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.579+0000 7fc907e00700 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 0x5594987ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.579+0000 7fc90d200700 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987d9df0 0x5594987dc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.580+0000 7fc907400700 1 -- 172.21.15.67:0/3380852953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987d9df0 msgr2=0x5594987dc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.580+0000 7fc907400700 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987d9df0 0x5594987dc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.580+0000 7fc907400700 1 -- 172.21.15.67:0/3380852953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 msgr2=0x5594987ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.580+0000 7fc907400700 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 0x5594987ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.580+0000 7fc907400700 1 -- 172.21.15.67:0/3380852953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594985a23b0 con 0x5594987ceef0 2024-03-30T02:42:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.580+0000 7fc907e00700 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 0x5594987ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:42:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.580+0000 7fc90d200700 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987d9df0 0x5594987dc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:42:29.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.580+0000 7fc907400700 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987ceef0 0x5594987d98b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4004350 tx=0x7fc8f4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9a447579a0120fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.581+0000 7fc8fd600700 1 -- 172.21.15.67:0/3380852953 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4003d10 con 0x5594987ceef0 2024-03-30T02:42:29.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.581+0000 7fc8fd600700 1 -- 172.21.15.67:0/3380852953 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8f4003e70 con 0x5594987ceef0 2024-03-30T02:42:29.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.581+0000 7fc8fd600700 1 -- 172.21.15.67:0/3380852953 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f401e7b0 con 0x5594987ceef0 2024-03-30T02:42:29.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.582+0000 7fc922d47b80 1 -- 172.21.15.67:0/3380852953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987ceef0 msgr2=0x5594987d98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.582+0000 7fc922d47b80 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987ceef0 0x5594987d98b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4004350 tx=0x7fc8f4014000 comp rx=0 tx=0).stop 2024-03-30T02:42:29.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.583+0000 7fc922d47b80 1 -- 172.21.15.67:0/3380852953 shutdown_connections 2024-03-30T02:42:29.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.583+0000 7fc922d47b80 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 0x5594987ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.583+0000 7fc922d47b80 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987d9df0 0x5594987dc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.583+0000 7fc922d47b80 1 --2- 172.21.15.67:0/3380852953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987ceef0 0x5594987d98b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.583+0000 7fc922d47b80 1 -- 172.21.15.67:0/3380852953 >> 172.21.15.67:0/3380852953 conn(0x55949872e6a0 msgr2=0x5594987caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.583+0000 7fc922d47b80 1 -- 172.21.15.67:0/3380852953 shutdown_connections 2024-03-30T02:42:29.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.583+0000 7fc922d47b80 1 -- 172.21.15.67:0/3380852953 wait complete. 2024-03-30T02:42:29.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.584+0000 7fc922d47b80 1 -- start start 2024-03-30T02:42:29.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.584+0000 7fc922d47b80 1 Processor -- start 2024-03-30T02:42:29.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.586+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987d97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.586+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 0x5594987d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.586+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987d7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc922d47b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594986bcd80 con 0x5594987ceef0 2024-03-30T02:42:29.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.586+0000 7fc907e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987d97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc907e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987d97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52570/0 (socket says 172.21.15.67:52570) 2024-03-30T02:42:29.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc907e00700 1 -- 172.21.15.67:0/1623303032 learned_addr learned my addr 172.21.15.67:0/1623303032 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594986bca20 con 0x5594987d9df0 2024-03-30T02:42:29.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594985a46a0 con 0x5594987ce5d0 2024-03-30T02:42:29.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc922d47b80 10 client.0 _ll_register_callbacks cb 0x5594986c6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:29.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc90d200700 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987d7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc90d200700 1 -- 172.21.15.67:0/1623303032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 msgr2=0x5594987d97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc90d200700 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987d97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc90d200700 1 -- 172.21.15.67:0/1623303032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 msgr2=0x5594987d6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc90d200700 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 0x5594987d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.587+0000 7fc90d200700 1 -- 172.21.15.67:0/1623303032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f4017070 con 0x5594987d9df0 2024-03-30T02:42:29.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.588+0000 7fc907e00700 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987d97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:42:29.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.591+0000 7fc90d200700 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987d7110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009230 tx=0x7fc8f800f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb0a392eec756d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.592+0000 7fc8f2000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:29.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.592+0000 7fc8f2000700 1 -- 172.21.15.67:0/1623303032 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800b970 con 0x5594987d9df0 2024-03-30T02:42:29.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.593+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594986e1380 con 0x5594987d9df0 2024-03-30T02:42:29.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.593+0000 7fc8f2000700 1 -- 172.21.15.67:0/1623303032 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fc8f800bad0 con 0x5594987d9df0 2024-03-30T02:42:29.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.595+0000 7fc8f2000700 1 -- 172.21.15.67:0/1623303032 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8f800b970 con 0x5594987d9df0 2024-03-30T02:42:29.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.596+0000 7fc922d47b80 10 client.26864 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:29.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.596+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55949854eb60 con 0x5594987d9df0 2024-03-30T02:42:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.596+0000 7fc8f2000700 1 -- 172.21.15.67:0/1623303032 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5594986e1380 con 0x5594987d9df0 2024-03-30T02:42:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.596+0000 7fc8f2000700 1 -- 172.21.15.67:0/1623303032 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fc8f8020070 con 0x5594987d9df0 2024-03-30T02:42:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.596+0000 7fc922d47b80 10 client.26864 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.596+0000 7fc922d47b80 10 client.26864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.597+0000 7fc922d47b80 20 client.26864 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.597+0000 7fc922d47b80 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5594987e9910 0x5594987ebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.597+0000 7fc922d47b80 4 client.26864 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:42:29.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.597+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559498719700 con 0x5594987e9910 2024-03-30T02:42:29.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.597+0000 7fc907e00700 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5594987e9910 0x5594987ebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.598+0000 7fc907e00700 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5594987e9910 0x5594987ebd30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=755e8a1f4e03cc3e server_cookie=dd7d39c99ac88613 in_seq=0 out_seq=0 2024-03-30T02:42:29.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.598+0000 7fc8f2000700 10 client.26864 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:29.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.602+0000 7fc8f2000700 1 -- 172.21.15.67:0/1623303032 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559498719700 con 0x5594987e9910 2024-03-30T02:42:29.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.602+0000 7fc8f2000700 10 client.26864 handle_command_reply: tid=0 2024-03-30T02:42:29.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.602+0000 7fc922d47b80 1 client.26864 shutdown 2024-03-30T02:42:29.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.604+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 msgr2=0x5594987d7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.604+0000 7fc922d47b80 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987d7110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009230 tx=0x7fc8f800f740 comp rx=0 tx=0).stop 2024-03-30T02:42:29.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.604+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 shutdown_connections 2024-03-30T02:42:29.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.604+0000 7fc922d47b80 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987d7110 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.604+0000 7fc922d47b80 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987d97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.604+0000 7fc922d47b80 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5594987e9910 0x5594987ebd30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.604+0000 7fc922d47b80 1 --2- 172.21.15.67:0/1623303032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 0x5594987d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.604+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 >> 172.21.15.67:0/1623303032 conn(0x55949872e6a0 msgr2=0x5594987d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.604+0000 7fc8f2000700 0 client.26864 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.605+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 shutdown_connections 2024-03-30T02:42:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.605+0000 7fc922d47b80 1 -- 172.21.15.67:0/1623303032 wait complete. 2024-03-30T02:42:29.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.606+0000 7fc922d47b80 20 client.26864 trim_cache size 0 max 16384 2024-03-30T02:42:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.645+0000 7fc922d47b80 1 Processor -- start 2024-03-30T02:42:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.645+0000 7fc922d47b80 1 -- start start 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.645+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 0x5594987e0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.645+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ceef0 0x5594987d92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.645+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987e51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.645+0000 7fc922d47b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594987cf850 con 0x5594987ce5d0 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.645+0000 7fc922d47b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594987e64d0 con 0x5594987d9df0 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.645+0000 7fc922d47b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594987e0fe0 con 0x5594987ceef0 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc90d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987e51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc90d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987e51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39708/0 (socket says 172.21.15.67:39708) 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc907e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 0x5594987e0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc90d200700 1 -- 172.21.15.67:0/3972693503 learned_addr learned my addr 172.21.15.67:0/3972693503 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc907400700 1 --2- 172.21.15.67:0/3972693503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ceef0 0x5594987d92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc90d200700 1 -- 172.21.15.67:0/3972693503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ceef0 msgr2=0x5594987d92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc90d200700 1 --2- 172.21.15.67:0/3972693503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ceef0 0x5594987d92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc90d200700 1 -- 172.21.15.67:0/3972693503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 msgr2=0x5594987e0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc90d200700 1 --2- 172.21.15.67:0/3972693503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 0x5594987e0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.646+0000 7fc90d200700 1 -- 172.21.15.67:0/3972693503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594986e6ea0 con 0x5594987d9df0 2024-03-30T02:42:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.647+0000 7fc90d200700 1 --2- 172.21.15.67:0/3972693503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987e51f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009fd0 tx=0x7fc8f80038e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d036d81640716012 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.647+0000 7fc8fe000700 1 -- 172.21.15.67:0/3972693503 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800b970 con 0x5594987d9df0 2024-03-30T02:42:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.647+0000 7fc8fe000700 1 -- 172.21.15.67:0/3972693503 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8f801d040 con 0x5594987d9df0 2024-03-30T02:42:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.647+0000 7fc8fe000700 1 -- 172.21.15.67:0/3972693503 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8020320 con 0x5594987d9df0 2024-03-30T02:42:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.648+0000 7fc922d47b80 1 -- 172.21.15.67:0/3972693503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 msgr2=0x5594987e51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.648+0000 7fc922d47b80 1 --2- 172.21.15.67:0/3972693503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987e51f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8009fd0 tx=0x7fc8f80038e0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.648+0000 7fc922d47b80 1 -- 172.21.15.67:0/3972693503 shutdown_connections 2024-03-30T02:42:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.649+0000 7fc922d47b80 1 --2- 172.21.15.67:0/3972693503 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ce5d0 0x5594987e0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.649+0000 7fc922d47b80 1 --2- 172.21.15.67:0/3972693503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987e51f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.649+0000 7fc922d47b80 1 --2- 172.21.15.67:0/3972693503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ceef0 0x5594987d92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.649+0000 7fc922d47b80 1 -- 172.21.15.67:0/3972693503 >> 172.21.15.67:0/3972693503 conn(0x55949872e6a0 msgr2=0x5594987d53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.649+0000 7fc922d47b80 1 -- 172.21.15.67:0/3972693503 shutdown_connections 2024-03-30T02:42:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.649+0000 7fc922d47b80 1 -- 172.21.15.67:0/3972693503 wait complete. 2024-03-30T02:42:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.649+0000 7fc922d47b80 1 -- start start 2024-03-30T02:42:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.650+0000 7fc922d47b80 1 Processor -- start 2024-03-30T02:42:29.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.651+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987e0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.651+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 0x5594987deae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.651+0000 7fc922d47b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987df020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc922d47b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594987df560 con 0x5594987ceef0 2024-03-30T02:42:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc922d47b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594987df6d0 con 0x5594987d9df0 2024-03-30T02:42:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc90d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987df020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc90d200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987df020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39714/0 (socket says 172.21.15.67:39714) 2024-03-30T02:42:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc90d200700 1 -- 172.21.15.67:0/58396419 learned_addr learned my addr 172.21.15.67:0/58396419 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc922d47b80 1 -- 172.21.15.67:0/58396419 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594987df890 con 0x5594987ce5d0 2024-03-30T02:42:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc907400700 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 0x5594987deae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc922d47b80 10 client.0 _ll_register_callbacks cb 0x5594986c6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc907e00700 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987e0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc90d200700 1 -- 172.21.15.67:0/58396419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 msgr2=0x5594987e0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc90d200700 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987e0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc90d200700 1 -- 172.21.15.67:0/58396419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 msgr2=0x5594987deae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc90d200700 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 0x5594987deae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc90d200700 1 -- 172.21.15.67:0/58396419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55949854eb60 con 0x5594987d9df0 2024-03-30T02:42:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.652+0000 7fc907400700 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 0x5594987deae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:42:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.653+0000 7fc90d200700 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987df020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8037040 tx=0x7fc8f800f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f1c66360683d0d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.654+0000 7fc8f2000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.654+0000 7fc8f2000700 1 -- 172.21.15.67:0/58396419 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8020520 con 0x5594987d9df0 2024-03-30T02:42:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.655+0000 7fc8f2000700 1 -- 172.21.15.67:0/58396419 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fc8f8000e40 con 0x5594987d9df0 2024-03-30T02:42:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.655+0000 7fc922d47b80 1 -- 172.21.15.67:0/58396419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8f4003e70 con 0x5594987d9df0 2024-03-30T02:42:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.656+0000 7fc8f2000700 1 -- 172.21.15.67:0/58396419 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8f8003a30 con 0x5594987d9df0 2024-03-30T02:42:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.656+0000 7fc922d47b80 10 client.26870 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.656+0000 7fc922d47b80 1 -- 172.21.15.67:0/58396419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5594985a6cd0 con 0x5594987d9df0 2024-03-30T02:42:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.656+0000 7fc8f2000700 1 -- 172.21.15.67:0/58396419 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8f4003e70 con 0x5594987d9df0 2024-03-30T02:42:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.656+0000 7fc8f2000700 1 -- 172.21.15.67:0/58396419 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fc8f8009c70 con 0x5594987d9df0 2024-03-30T02:42:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.656+0000 7fc922d47b80 10 client.26870 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.656+0000 7fc922d47b80 10 client.26870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.657+0000 7fc922d47b80 20 client.26870 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.657+0000 7fc922d47b80 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55949883e7b0 0x559498840bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.657+0000 7fc922d47b80 4 client.26870 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:42:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.657+0000 7fc922d47b80 1 -- 172.21.15.67:0/58396419 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55949856fcf0 con 0x55949883e7b0 2024-03-30T02:42:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.657+0000 7fc907e00700 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55949883e7b0 0x559498840bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.658+0000 7fc907e00700 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55949883e7b0 0x559498840bd0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1ae6824dbd2162 server_cookie=3c51d3f24f1d1cc5 in_seq=0 out_seq=0 2024-03-30T02:42:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.658+0000 7fc8f2000700 10 client.26870 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:29.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.662+0000 7fc8f2000700 1 -- 172.21.15.67:0/58396419 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55949856fcf0 con 0x55949883e7b0 2024-03-30T02:42:29.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.662+0000 7fc8f2000700 10 client.26870 handle_command_reply: tid=0 2024-03-30T02:42:29.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.662+0000 7fc922d47b80 1 client.26870 shutdown 2024-03-30T02:42:29.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.662+0000 7fc922d47b80 1 -- 172.21.15.67:0/58396419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 msgr2=0x5594987df020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.662+0000 7fc922d47b80 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987df020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8037040 tx=0x7fc8f800f740 comp rx=0 tx=0).stop 2024-03-30T02:42:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.662+0000 7fc922d47b80 1 -- 172.21.15.67:0/58396419 shutdown_connections 2024-03-30T02:42:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.662+0000 7fc922d47b80 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5594987d9df0 0x5594987df020 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.663+0000 7fc922d47b80 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5594987ce5d0 0x5594987e0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.663+0000 7fc922d47b80 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55949883e7b0 0x559498840bd0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.663+0000 7fc922d47b80 1 --2- 172.21.15.67:0/58396419 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5594987ceef0 0x5594987deae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.663+0000 7fc8f2000700 0 client.26870 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:29.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.663+0000 7fc922d47b80 1 -- 172.21.15.67:0/58396419 >> 172.21.15.67:0/58396419 conn(0x55949872e6a0 msgr2=0x5594987e51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.663+0000 7fc922d47b80 1 -- 172.21.15.67:0/58396419 shutdown_connections 2024-03-30T02:42:29.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.663+0000 7fc922d47b80 1 -- 172.21.15.67:0/58396419 wait complete. 2024-03-30T02:42:29.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.664+0000 7fc922d47b80 20 client.26870 trim_cache size 0 max 16384 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (956 inodes in the stack)", 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout: "e32ff20d-c7a4-48b3-b5dc-08cb42a587a6": { 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "e32ff20d-c7a4-48b3-b5dc-08cb42a587a6", 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:42:29.664 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:42:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc8ec0a13c0 msgr2=0x7fc8ec0a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc8ec0a13c0 0x7fc8ec0a3880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc90819ed50 tx=0x7fc8f4009310 comp rx=0 tx=0).stop 2024-03-30T02:42:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9080ffe50 msgr2=0x7fc9081a3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9080ffe50 0x7fc9081a3510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc9000060b0 tx=0x7fc900007f90 comp rx=0 tx=0).stop 2024-03-30T02:42:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 shutdown_connections 2024-03-30T02:42:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc8ec0a13c0 0x7fc8ec0a3880 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc90819d640 0x7fc90819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9080ffe50 0x7fc9081a3510 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 --2- 172.21.15.67:0/2465841179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc908104f60 0x7fc9081a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.665+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 >> 172.21.15.67:0/2465841179 conn(0x7fc908076d40 msgr2=0x7fc9080fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.666+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 shutdown_connections 2024-03-30T02:42:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:29.666+0000 7fc90f200700 1 -- 172.21.15.67:0/2465841179 wait complete. 2024-03-30T02:42:29.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:29.695 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:29.697 INFO:tasks.cephfs.filesystem:scrub status for tag:e32ff20d-c7a4-48b3-b5dc-08cb42a587a6 - {'path': '/', 'tag': 'e32ff20d-c7a4-48b3-b5dc-08cb42a587a6', 'options': 'recursive,force'} 2024-03-30T02:42:29.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:30 smithi170 ceph-mon[78798]: pgmap v166: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2024-03-30T02:42:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:30 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:42:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:30 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:30 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:30 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:30 smithi082 ceph-mon[79176]: pgmap v166: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2024-03-30T02:42:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:30 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:42:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:30 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:30 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:30 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:30 smithi067 ceph-mon[37808]: pgmap v166: 129 pgs: 129 active+clean; 1.7 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 247 op/s 2024-03-30T02:42:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:30 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:42:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:30 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:30 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:30 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:30.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:31.147 INFO:tasks.workunit.client.0.smithi067.stdout:# 2024-03-30T02:42:31.147 INFO:tasks.workunit.client.0.smithi067.stdout:# configuration written to .config 2024-03-30T02:42:31.147 INFO:tasks.workunit.client.0.smithi067.stdout:# 2024-03-30T02:42:31.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:31.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:31.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:31 smithi082 ceph-mon[79176]: pgmap v167: 129 pgs: 129 active+clean; 1.7 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 750 op/s 2024-03-30T02:42:31.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:31.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:31.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:31.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:31.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:31.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:31.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:31 smithi067 ceph-mon[37808]: pgmap v167: 129 pgs: 129 active+clean; 1.7 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 750 op/s 2024-03-30T02:42:31.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:31.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:31.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:31.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:31.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:31.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:31.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:31 smithi170 ceph-mon[78798]: pgmap v167: 129 pgs: 129 active+clean; 1.7 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 750 op/s 2024-03-30T02:42:31.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:31.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:31.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:31.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:32.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:32.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:32.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:32.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:32.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:33.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:33 smithi082 ceph-mon[79176]: pgmap v168: 129 pgs: 129 active+clean; 1.7 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 860 op/s 2024-03-30T02:42:33.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:33 smithi067 ceph-mon[37808]: pgmap v168: 129 pgs: 129 active+clean; 1.7 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 860 op/s 2024-03-30T02:42:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:33 smithi170 ceph-mon[78798]: pgmap v168: 129 pgs: 129 active+clean; 1.7 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 860 op/s 2024-03-30T02:42:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:34.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:34.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:34.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:35.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:35 smithi067 ceph-mon[37808]: pgmap v169: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 1.06k op/s 2024-03-30T02:42:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:35 smithi170 ceph-mon[78798]: pgmap v169: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 1.06k op/s 2024-03-30T02:42:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:35.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:35 smithi082 ceph-mon[79176]: pgmap v169: 129 pgs: 129 active+clean; 1.8 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.7 MiB/s wr, 1.06k op/s 2024-03-30T02:42:35.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:35.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: pgmap v170: 129 pgs: 129 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.7 MiB/s wr, 1.37k op/s 2024-03-30T02:42:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: pgmap v170: 129 pgs: 129 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.7 MiB/s wr, 1.37k op/s 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: pgmap v170: 129 pgs: 129 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.7 MiB/s wr, 1.37k op/s 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.231 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/errno.h 2024-03-30T02:42:39.232 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/fcntl.h 2024-03-30T02:42:39.232 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/bpf_perf_event.h 2024-03-30T02:42:39.235 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctl.h 2024-03-30T02:42:39.235 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/ioctls.h 2024-03-30T02:42:39.238 INFO:tasks.workunit.client.0.smithi067.stdout: GEN arch/x86/include/generated/asm/orc_hash.h 2024-03-30T02:42:39.239 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/ipcbuf.h 2024-03-30T02:42:39.241 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/param.h 2024-03-30T02:42:39.241 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/poll.h 2024-03-30T02:42:39.244 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/resource.h 2024-03-30T02:42:39.245 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_32.h 2024-03-30T02:42:39.246 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/socket.h 2024-03-30T02:42:39.247 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_64.h 2024-03-30T02:42:39.252 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/uapi/asm/unistd_x32.h 2024-03-30T02:42:39.256 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/sockios.h 2024-03-30T02:42:39.268 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/termbits.h 2024-03-30T02:42:39.282 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/termios.h 2024-03-30T02:42:39.282 INFO:tasks.workunit.client.0.smithi067.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_32.h 2024-03-30T02:42:39.289 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC arch/x86/tools/relocs_32.o 2024-03-30T02:42:39.291 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/asm/unistd_32_ia32.h 2024-03-30T02:42:39.294 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/uapi/asm/types.h 2024-03-30T02:42:39.333 INFO:tasks.workunit.client.0.smithi067.stdout: SYSHDR arch/x86/include/generated/asm/unistd_64_x32.h 2024-03-30T02:42:39.346 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC arch/x86/tools/relocs_64.o 2024-03-30T02:42:39.375 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/config/kernel.release 2024-03-30T02:42:39.383 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/early_ioremap.h 2024-03-30T02:42:39.388 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/export.h 2024-03-30T02:42:39.394 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/mcs_spinlock.h 2024-03-30T02:42:39.394 INFO:tasks.workunit.client.0.smithi067.stdout: SYSTBL arch/x86/include/generated/asm/syscalls_64.h 2024-03-30T02:42:39.399 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/irq_regs.h 2024-03-30T02:42:39.399 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/kmap_size.h 2024-03-30T02:42:39.401 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/local64.h 2024-03-30T02:42:39.404 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/uapi/linux/version.h 2024-03-30T02:42:39.410 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/mmiowb.h 2024-03-30T02:42:39.410 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/module.lds.h 2024-03-30T02:42:39.418 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/rwonce.h 2024-03-30T02:42:39.418 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC arch/x86/tools/relocs_common.o 2024-03-30T02:42:39.426 INFO:tasks.workunit.client.0.smithi067.stdout: WRAP arch/x86/include/generated/asm/unaligned.h 2024-03-30T02:42:39.439 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/utsrelease.h 2024-03-30T02:42:39.480 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/compile.h 2024-03-30T02:42:39.500 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/kallsyms 2024-03-30T02:42:39.502 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/sorttable 2024-03-30T02:42:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: DESCEND objtool 2024-03-30T02:42:39.615 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/selinux/genheaders/genheaders 2024-03-30T02:42:39.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.670 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/selinux/mdp/mdp 2024-03-30T02:42:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.713 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/asn1_compiler 2024-03-30T02:42:39.729 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTLD arch/x86/tools/relocs 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:39.894 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep.o 2024-03-30T02:42:39.969 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTLD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep-in.o 2024-03-30T02:42:39.978 INFO:tasks.workunit.client.0.smithi067.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/fixdep 2024-03-30T02:42:40.058 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/help.h 2024-03-30T02:42:40.060 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/exec-cmd.h 2024-03-30T02:42:40.061 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/pager.h 2024-03-30T02:42:40.067 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/parse-options.h 2024-03-30T02:42:40.067 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/exec-cmd.o 2024-03-30T02:42:40.091 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/help.o 2024-03-30T02:42:40.091 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/include/subcmd/run-command.h 2024-03-30T02:42:40.093 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/pager.o 2024-03-30T02:42:40.102 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/parse-options.o 2024-03-30T02:42:40.106 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/run-command.o 2024-03-30T02:42:40.117 INFO:tasks.workunit.client.0.smithi067.stdout: INSTALL libsubcmd_headers 2024-03-30T02:42:40.128 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/sigchain.o 2024-03-30T02:42:40.180 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/subcmd-config.o 2024-03-30T02:42:40.602 INFO:tasks.workunit.client.0.smithi067.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd-in.o 2024-03-30T02:42:40.630 INFO:tasks.workunit.client.0.smithi067.stdout: AR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libsubcmd/libsubcmd.a 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: pgmap v171: 129 pgs: 129 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.7 MiB/s wr, 1.45k op/s 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: pgmap v171: 129 pgs: 129 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.7 MiB/s wr, 1.45k op/s 2024-03-30T02:42:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.726 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/weak.o 2024-03-30T02:42:40.728 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/check.o 2024-03-30T02:42:40.730 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/special.o 2024-03-30T02:42:40.732 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/builtin-check.o 2024-03-30T02:42:40.732 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/elf.o 2024-03-30T02:42:40.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool.o 2024-03-30T02:42:40.736 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/special.o 2024-03-30T02:42:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: pgmap v171: 129 pgs: 129 active+clean; 1.8 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.7 MiB/s wr, 1.45k op/s 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:40.938 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_gen.o 2024-03-30T02:42:40.965 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/orc_dump.o 2024-03-30T02:42:40.968 INFO:tasks.workunit.client.0.smithi067.stdout: MKDIR /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/ 2024-03-30T02:42:40.984 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libstring.o 2024-03-30T02:42:40.987 INFO:tasks.workunit.client.0.smithi067.stdout: GEN /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/lib/inat-tables.c 2024-03-30T02:42:41.066 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/decode.o 2024-03-30T02:42:41.067 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/libctype.o 2024-03-30T02:42:41.184 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/str_error_r.o 2024-03-30T02:42:41.208 INFO:tasks.workunit.client.0.smithi067.stdout: CC /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/librbtree.o 2024-03-30T02:42:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:41.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.001 INFO:tasks.workunit.client.0.smithi067.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/arch/x86/objtool-in.o 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: pgmap v172: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.9 MiB/s wr, 1.79k op/s 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.364 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.365 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.366 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.367 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.367 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.367 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.367 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.367 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.367 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.367 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.367 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: pgmap v172: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.9 MiB/s wr, 1.79k op/s 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.368 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.369 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.370 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: pgmap v172: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.9 MiB/s wr, 1.79k op/s 2024-03-30T02:42:42.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.102 INFO:tasks.workunit.client.0.smithi067.stdout: LD /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool-in.o 2024-03-30T02:42:43.130 INFO:tasks.workunit.client.0.smithi067.stdout: LINK /home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/tools/objtool/objtool 2024-03-30T02:42:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:43.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: pgmap v173: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 1.49k op/s 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:44.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: pgmap v173: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 1.49k op/s 2024-03-30T02:42:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:44.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: pgmap v173: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 1.49k op/s 2024-03-30T02:42:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:45.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:45.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: pgmap v174: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.1 MiB/s wr, 1.51k op/s 2024-03-30T02:42:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.671 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:46.674 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: pgmap v174: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.1 MiB/s wr, 1.51k op/s 2024-03-30T02:42:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.697 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.698 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:46.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: pgmap v174: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.1 MiB/s wr, 1.51k op/s 2024-03-30T02:42:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:46.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:46.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:47.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:47.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: pgmap v175: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 1.74k op/s 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:48.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: pgmap v175: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 1.74k op/s 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:48.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: pgmap v175: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 1.74k op/s 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:48.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:48.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:49.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:49.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:49.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:49.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: pgmap v176: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 1.56k op/s 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:50.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: pgmap v176: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 1.56k op/s 2024-03-30T02:42:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:50.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: pgmap v176: 129 pgs: 129 active+clean; 1.8 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 1.56k op/s 2024-03-30T02:42:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:50.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:51.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:51.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:51.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:51.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:51.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:51.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:51.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:51.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:52 smithi067 ceph-mon[37808]: pgmap v177: 129 pgs: 129 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 2.69k op/s 2024-03-30T02:42:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:52 smithi170 ceph-mon[78798]: pgmap v177: 129 pgs: 129 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 2.69k op/s 2024-03-30T02:42:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:52 smithi082 ceph-mon[79176]: pgmap v177: 129 pgs: 129 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.5 MiB/s wr, 2.69k op/s 2024-03-30T02:42:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:53.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:53.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:53 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:53 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:53.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:53 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: pgmap v178: 129 pgs: 129 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 2.87k op/s 2024-03-30T02:42:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: pgmap v178: 129 pgs: 129 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 2.87k op/s 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: pgmap v178: 129 pgs: 129 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 2.87k op/s 2024-03-30T02:42:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:55.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:55.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:42:55.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:55.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:42:56.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:56 smithi170 ceph-mon[78798]: pgmap v179: 129 pgs: 129 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 3.20k op/s 2024-03-30T02:42:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:56 smithi082 ceph-mon[79176]: pgmap v179: 129 pgs: 129 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 3.20k op/s 2024-03-30T02:42:56.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:56 smithi067 ceph-mon[37808]: pgmap v179: 129 pgs: 129 active+clean; 1.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 3.20k op/s 2024-03-30T02:42:58.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:42:58 smithi170 ceph-mon[78798]: pgmap v180: 129 pgs: 129 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.1 MiB/s wr, 3.65k op/s 2024-03-30T02:42:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:42:58 smithi082 ceph-mon[79176]: pgmap v180: 129 pgs: 129 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.1 MiB/s wr, 3.65k op/s 2024-03-30T02:42:58.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:42:58 smithi067 ceph-mon[37808]: pgmap v180: 129 pgs: 129 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.1 MiB/s wr, 3.65k op/s 2024-03-30T02:42:59.698 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:42:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.936+0000 7ff5d4800700 1 -- 172.21.15.67:0/3561247348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc101910 msgr2=0x7ff5cc105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.936+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3561247348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc101910 0x7ff5cc105950 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0098f0 tx=0x7ff5bc009c00 comp rx=0 tx=0).stop 2024-03-30T02:42:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.936+0000 7ff5d4800700 1 -- 172.21.15.67:0/3561247348 shutdown_connections 2024-03-30T02:42:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.936+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3561247348 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5cc106080 0x7ff5cc10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.936+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3561247348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5cc100f60 0x7ff5cc101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.936+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3561247348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc101910 0x7ff5cc105950 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.936+0000 7ff5d4800700 1 -- 172.21.15.67:0/3561247348 >> 172.21.15.67:0/3561247348 conn(0x7ff5cc0fc7c0 msgr2=0x7ff5cc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:59.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.936+0000 7ff5d4800700 1 -- 172.21.15.67:0/3561247348 shutdown_connections 2024-03-30T02:42:59.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.936+0000 7ff5d4800700 1 -- 172.21.15.67:0/3561247348 wait complete. 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.937+0000 7ff5d4800700 1 Processor -- start 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d4800700 1 -- start start 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d4800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5cc100f60 0x7ff5cc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d4800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc106080 0x7ff5cc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d4800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5cc073b30 0x7ff5cc1a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d4800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1a5ac0 con 0x7ff5cc100f60 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d4800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1a5c00 con 0x7ff5cc106080 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d4800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1a5dc0 con 0x7ff5cc073b30 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5cc100f60 0x7ff5cc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d2800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5cc073b30 0x7ff5cc1a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5cc100f60 0x7ff5cc0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41798/0 (socket says 172.21.15.67:41798) 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d1e00700 1 -- 172.21.15.67:0/3821074204 learned_addr learned my addr 172.21.15.67:0/3821074204 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.938+0000 7ff5d1400700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc106080 0x7ff5cc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5d2800700 1 -- 172.21.15.67:0/3821074204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc106080 msgr2=0x7ff5cc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5d2800700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc106080 0x7ff5cc0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5d2800700 1 -- 172.21.15.67:0/3821074204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5cc100f60 msgr2=0x7ff5cc0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5d2800700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5cc100f60 0x7ff5cc0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5d2800700 1 -- 172.21.15.67:0/3821074204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8009c40 con 0x7ff5cc073b30 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5d2800700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5cc073b30 0x7ff5cc1a54f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800efb0 tx=0x7ff5c800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5c6a00700 1 -- 172.21.15.67:0/3821074204 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8003a00 con 0x7ff5cc073b30 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5d4800700 1 -- 172.21.15.67:0/3821074204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc009520 con 0x7ff5cc073b30 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5c6a00700 1 -- 172.21.15.67:0/3821074204 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5c800f040 con 0x7ff5cc073b30 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.939+0000 7ff5c6a00700 1 -- 172.21.15.67:0/3821074204 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8004540 con 0x7ff5cc073b30 2024-03-30T02:42:59.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.940+0000 7ff5d4800700 1 -- 172.21.15.67:0/3821074204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc1a6440 con 0x7ff5cc073b30 2024-03-30T02:42:59.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.943+0000 7ff5c6a00700 1 -- 172.21.15.67:0/3821074204 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff5c8003bf0 con 0x7ff5cc073b30 2024-03-30T02:42:59.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5e823db80 1 Processor -- start 2024-03-30T02:42:59.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5c6a00700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff5b40a1310 0x7ff5b40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5e823db80 1 -- start start 2024-03-30T02:42:59.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5c6a00700 1 -- 172.21.15.67:0/3821074204 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff5c80ccb50 con 0x7ff5cc073b30 2024-03-30T02:42:59.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5d1400700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff5b40a1310 0x7ff5b40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ff5d0 0x559cbd1ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 0x559cbd20a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 0x559cbd20d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5e823db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd158990 con 0x559cbd1ff5d0 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5e823db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd156350 con 0x559cbd20adf0 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.945+0000 7ff5e823db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd151c40 con 0x559cbd1ffef0 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 0x559cbd20a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 0x559cbd20d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 0x559cbd20a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55298/0 (socket says 172.21.15.67:55298) 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d1400700 1 -- 172.21.15.67:0/4184460860 learned_addr learned my addr 172.21.15.67:0/4184460860 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/4184460860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ff5d0 0x559cbd1ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d2800700 1 -- 172.21.15.67:0/4184460860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 msgr2=0x559cbd20a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d2800700 1 --2- 172.21.15.67:0/4184460860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 0x559cbd20a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d2800700 1 -- 172.21.15.67:0/4184460860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ff5d0 msgr2=0x559cbd1ff9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d2800700 1 --2- 172.21.15.67:0/4184460860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ff5d0 0x559cbd1ff9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d2800700 1 -- 172.21.15.67:0/4184460860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cbcfd33b0 con 0x559cbd20adf0 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d1400700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff5b40a1310 0x7ff5b40a37d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc06a380 tx=0x7ff5bc00fe70 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:59.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.946+0000 7ff5d2800700 1 --2- 172.21.15.67:0/4184460860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 0x559cbd20d1e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c9ef0 tx=0x7ff5c80997e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdbdee6389e7eb52 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:59.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5bb000700 1 -- 172.21.15.67:0/4184460860 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80b7ea0 con 0x559cbd20adf0 2024-03-30T02:42:59.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5bb000700 1 -- 172.21.15.67:0/4184460860 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5c8084040 con 0x559cbd20adf0 2024-03-30T02:42:59.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5bb000700 1 -- 172.21.15.67:0/4184460860 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8097d70 con 0x559cbd20adf0 2024-03-30T02:42:59.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5e823db80 1 -- 172.21.15.67:0/4184460860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 msgr2=0x559cbd20d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:59.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5e823db80 1 --2- 172.21.15.67:0/4184460860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 0x559cbd20d1e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c9ef0 tx=0x7ff5c80997e0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5e823db80 1 -- 172.21.15.67:0/4184460860 shutdown_connections 2024-03-30T02:42:59.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5e823db80 1 --2- 172.21.15.67:0/4184460860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ff5d0 0x559cbd1ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5e823db80 1 --2- 172.21.15.67:0/4184460860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 0x559cbd20d1e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5e823db80 1 --2- 172.21.15.67:0/4184460860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 0x559cbd20a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.947+0000 7ff5e823db80 1 -- 172.21.15.67:0/4184460860 >> 172.21.15.67:0/4184460860 conn(0x559cbd15f6a0 msgr2=0x559cbd1fba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.948+0000 7ff5e823db80 1 -- 172.21.15.67:0/4184460860 shutdown_connections 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.948+0000 7ff5e823db80 1 -- 172.21.15.67:0/4184460860 wait complete. 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.948+0000 7ff5e823db80 1 -- start start 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.948+0000 7ff5e823db80 1 Processor -- start 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ff5d0 0x559cbd20a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 0x559cbd207b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 0x559cbd2082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5e823db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd0edd80 con 0x559cbd1ffef0 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5e823db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd0eda20 con 0x559cbd20adf0 2024-03-30T02:42:59.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5e823db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbcfd56a0 con 0x559cbd1ff5d0 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5e823db80 10 client.0 _ll_register_callbacks cb 0x559cbd0f7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5d1e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ff5d0 0x559cbd20a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5d1e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ff5d0 0x559cbd20a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:55310/0 (socket says 172.21.15.67:55310) 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5d1e00700 1 -- 172.21.15.67:0/666545731 learned_addr learned my addr 172.21.15.67:0/666545731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5d1400700 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 0x559cbd207b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.949+0000 7ff5d2800700 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 0x559cbd2082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5d1e00700 1 -- 172.21.15.67:0/666545731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 msgr2=0x559cbd2082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 0x559cbd2082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5d1e00700 1 -- 172.21.15.67:0/666545731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 msgr2=0x559cbd207b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 0x559cbd207b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5d1e00700 1 -- 172.21.15.67:0/666545731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc1a6440 con 0x559cbd1ff5d0 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5d1400700 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 0x559cbd207b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ff5d0 0x559cbd20a8a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0012980 tx=0x7ff5c0012c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63df08f78010ecbb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5af400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:42:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5af400700 1 -- 172.21.15.67:0/666545731 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0017070 con 0x559cbd1ff5d0 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5af400700 1 -- 172.21.15.67:0/666545731 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff5c0004020 con 0x559cbd1ff5d0 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.950+0000 7ff5e823db80 1 -- 172.21.15.67:0/666545731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cbd112380 con 0x559cbd1ff5d0 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5af400700 1 -- 172.21.15.67:0/666545731 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5c00122e0 con 0x559cbd1ff5d0 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5e823db80 10 client.26833 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5e823db80 1 -- 172.21.15.67:0/666545731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559cbcf7fb60 con 0x559cbd1ff5d0 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5af400700 1 -- 172.21.15.67:0/666545731 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559cbd112380 con 0x559cbd1ff5d0 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5af400700 1 -- 172.21.15.67:0/666545731 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7ff5c000f5e0 con 0x559cbd1ff5d0 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5e823db80 10 client.26833 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5e823db80 10 client.26833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5e823db80 20 client.26833 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5e823db80 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x559cbd21ab30 0x559cbd21cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5e823db80 4 client.26833 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5e823db80 1 -- 172.21.15.67:0/666545731 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cbd14a700 con 0x559cbd21ab30 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.951+0000 7ff5d1400700 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x559cbd21ab30 0x559cbd21cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.952+0000 7ff5d1400700 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x559cbd21ab30 0x559cbd21cf50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e140ac0f34c12286 server_cookie=1c68b0cec8bd67a1 in_seq=0 out_seq=0 2024-03-30T02:42:59.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.952+0000 7ff5af400700 10 client.26833 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.952+0000 7ff5af400700 1 -- 172.21.15.67:0/666545731 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559cbd14a700 con 0x559cbd21ab30 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.952+0000 7ff5af400700 10 client.26833 handle_command_reply: tid=0 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 client.26833 shutdown 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 -- 172.21.15.67:0/666545731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ff5d0 msgr2=0x559cbd20a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ff5d0 0x559cbd20a8a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0012980 tx=0x7ff5c0012c90 comp rx=0 tx=0).stop 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 -- 172.21.15.67:0/666545731 shutdown_connections 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd20adf0 0x559cbd2082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ff5d0 0x559cbd20a8a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x559cbd21ab30 0x559cbd21cf50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 --2- 172.21.15.67:0/666545731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 0x559cbd207b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 -- 172.21.15.67:0/666545731 >> 172.21.15.67:0/666545731 conn(0x559cbd15f6a0 msgr2=0x559cbd2016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5af400700 0 client.26833 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 -- 172.21.15.67:0/666545731 shutdown_connections 2024-03-30T02:42:59.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 1 -- 172.21.15.67:0/666545731 wait complete. 2024-03-30T02:42:59.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.953+0000 7ff5e823db80 20 client.26833 trim_cache size 0 max 16384 2024-03-30T02:42:59.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.974+0000 7ff5e823db80 1 Processor -- start 2024-03-30T02:42:59.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.974+0000 7ff5e823db80 1 -- start start 2024-03-30T02:42:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.974+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd21cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.974+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 0x559cbd211a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.974+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd20adf0 0x559cbd1f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.974+0000 7ff5e823db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd208f30 con 0x559cbd20adf0 2024-03-30T02:42:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.974+0000 7ff5e823db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd1f54b0 con 0x559cbd1ff5d0 2024-03-30T02:42:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.974+0000 7ff5e823db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd1fc660 con 0x559cbd1ffef0 2024-03-30T02:42:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd21cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd20adf0 0x559cbd1f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 0x559cbd211a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd21cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40154/0 (socket says 172.21.15.67:40154) 2024-03-30T02:42:59.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1e00700 1 -- 172.21.15.67:0/540101673 learned_addr learned my addr 172.21.15.67:0/540101673 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:59.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1e00700 1 -- 172.21.15.67:0/540101673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 msgr2=0x559cbd211a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:59.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/540101673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 0x559cbd211a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1e00700 1 -- 172.21.15.67:0/540101673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd20adf0 msgr2=0x559cbd1f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/540101673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd20adf0 0x559cbd1f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1e00700 1 -- 172.21.15.67:0/540101673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cbd117ea0 con 0x559cbd1ff5d0 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d2800700 1 --2- 172.21.15.67:0/540101673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd20adf0 0x559cbd1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.975+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/540101673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd21cf20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000c00 tx=0x7ff5c000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=360a40e4de2cabed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5bba00700 1 -- 172.21.15.67:0/540101673 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0017070 con 0x559cbd1ff5d0 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5bba00700 1 -- 172.21.15.67:0/540101673 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5c0018040 con 0x559cbd1ff5d0 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5bba00700 1 -- 172.21.15.67:0/540101673 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c00038e0 con 0x559cbd1ff5d0 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5e823db80 1 -- 172.21.15.67:0/540101673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 msgr2=0x559cbd21cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5e823db80 1 --2- 172.21.15.67:0/540101673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd21cf20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0000c00 tx=0x7ff5c000f3c0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5e823db80 1 -- 172.21.15.67:0/540101673 shutdown_connections 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5e823db80 1 --2- 172.21.15.67:0/540101673 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd20adf0 0x559cbd1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5e823db80 1 --2- 172.21.15.67:0/540101673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd21cf20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5e823db80 1 --2- 172.21.15.67:0/540101673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd1ffef0 0x559cbd211a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.976+0000 7ff5e823db80 1 -- 172.21.15.67:0/540101673 >> 172.21.15.67:0/540101673 conn(0x559cbd15f6a0 msgr2=0x559cbd1f45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:59.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.977+0000 7ff5e823db80 1 -- 172.21.15.67:0/540101673 shutdown_connections 2024-03-30T02:42:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.977+0000 7ff5e823db80 1 -- 172.21.15.67:0/540101673 wait complete. 2024-03-30T02:42:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.977+0000 7ff5e823db80 1 -- start start 2024-03-30T02:42:59.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.977+0000 7ff5e823db80 1 Processor -- start 2024-03-30T02:42:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd2111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 0x559cbd211720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5e823db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd20adf0 0x559cbd20fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5e823db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd210020 con 0x559cbd1ffef0 2024-03-30T02:42:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5e823db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd210190 con 0x559cbd1ff5d0 2024-03-30T02:42:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5e823db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cbd210350 con 0x559cbd20adf0 2024-03-30T02:42:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5e823db80 10 client.0 _ll_register_callbacks cb 0x559cbd0f7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:42:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd2111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d2800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd20adf0 0x559cbd20fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d1400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 0x559cbd211720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d1e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd2111e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40162/0 (socket says 172.21.15.67:40162) 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d1e00700 1 -- 172.21.15.67:0/3961664050 learned_addr learned my addr 172.21.15.67:0/3961664050 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d1e00700 1 -- 172.21.15.67:0/3961664050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd20adf0 msgr2=0x559cbd20fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd20adf0 0x559cbd20fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d1e00700 1 -- 172.21.15.67:0/3961664050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 msgr2=0x559cbd211720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 0x559cbd211720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.978+0000 7ff5d1e00700 1 -- 172.21.15.67:0/3961664050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cbcfd33b0 con 0x559cbd1ff5d0 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5d1e00700 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd2111e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009350 tx=0x7ff5c00123e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec7fb2b4355f88c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5af400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5af400700 1 -- 172.21.15.67:0/3961664050 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0017070 con 0x559cbd1ff5d0 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5af400700 1 -- 172.21.15.67:0/3961664050 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff5c00054c0 con 0x559cbd1ff5d0 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5e823db80 1 -- 172.21.15.67:0/3961664050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5c8084040 con 0x559cbd1ff5d0 2024-03-30T02:42:59.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5af400700 1 -- 172.21.15.67:0/3961664050 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5c0005680 con 0x559cbd1ff5d0 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5e823db80 10 client.26888 fetch_fsmap learned FSMap version 33 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5e823db80 1 -- 172.21.15.67:0/3961664050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559cbcf7fb60 con 0x559cbd1ff5d0 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5af400700 1 -- 172.21.15.67:0/3961664050 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5c0039080 con 0x559cbd1ff5d0 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.979+0000 7ff5af400700 1 -- 172.21.15.67:0/3961664050 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7510+0+0 (secure 0 0 0) 0x7ff5c0030070 con 0x559cbd1ff5d0 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.980+0000 7ff5e823db80 10 client.26888 fetch_fsmap finished waiting for FSMap version 33 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.980+0000 7ff5e823db80 10 client.26888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.980+0000 7ff5e823db80 20 client.26888 populate_metadata read hostname 'smithi067' 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.980+0000 7ff5e823db80 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x559cbd26f840 0x559cbd271c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.980+0000 7ff5e823db80 4 client.26888 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.980+0000 7ff5e823db80 1 -- 172.21.15.67:0/3961664050 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559cbcfa0cf0 con 0x559cbd26f840 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.980+0000 7ff5d1400700 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x559cbd26f840 0x559cbd271c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.980+0000 7ff5af400700 10 client.26888 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:59.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.980+0000 7ff5d1400700 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x559cbd26f840 0x559cbd271c60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37176457651c839a server_cookie=88684497493f701c in_seq=0 out_seq=0 2024-03-30T02:42:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5af400700 1 -- 172.21.15.67:0/3961664050 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559cbcfa0cf0 con 0x559cbd26f840 2024-03-30T02:42:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5af400700 10 client.26888 handle_command_reply: tid=0 2024-03-30T02:42:59.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 client.26888 shutdown 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 -- 172.21.15.67:0/3961664050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 msgr2=0x559cbd2111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd2111e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009350 tx=0x7ff5c00123e0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 -- 172.21.15.67:0/3961664050 shutdown_connections 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cbd20adf0 0x559cbd20fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559cbd1ff5d0 0x559cbd2111e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x559cbd26f840 0x559cbd271c60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 --2- 172.21.15.67:0/3961664050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559cbd1ffef0 0x559cbd211720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 -- 172.21.15.67:0/3961664050 >> 172.21.15.67:0/3961664050 conn(0x559cbd15f6a0 msgr2=0x559cbd202ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5af400700 0 client.26888 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 -- 172.21.15.67:0/3961664050 shutdown_connections 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.982+0000 7ff5e823db80 1 -- 172.21.15.67:0/3961664050 wait complete. 2024-03-30T02:42:59.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5e823db80 20 client.26888 trim_cache size 0 max 16384 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout: "e32ff20d-c7a4-48b3-b5dc-08cb42a587a6": { 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "e32ff20d-c7a4-48b3-b5dc-08cb42a587a6", 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:42:59.986 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:42:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 -- 172.21.15.67:0/3821074204 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff5b40a1310 msgr2=0x7ff5b40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff5b40a1310 0x7ff5b40a37d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc06a380 tx=0x7ff5bc00fe70 comp rx=0 tx=0).stop 2024-03-30T02:42:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 -- 172.21.15.67:0/3821074204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5cc073b30 msgr2=0x7ff5cc1a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:42:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5cc073b30 0x7ff5cc1a54f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800efb0 tx=0x7ff5c800c3a0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 -- 172.21.15.67:0/3821074204 shutdown_connections 2024-03-30T02:42:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff5b40a1310 0x7ff5b40a37d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5cc100f60 0x7ff5cc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5cc073b30 0x7ff5cc1a54f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 --2- 172.21.15.67:0/3821074204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc106080 0x7ff5cc0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:42:59.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 -- 172.21.15.67:0/3821074204 >> 172.21.15.67:0/3821074204 conn(0x7ff5cc0fc7c0 msgr2=0x7ff5cc0fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:42:59.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.983+0000 7ff5d4800700 1 -- 172.21.15.67:0/3821074204 shutdown_connections 2024-03-30T02:42:59.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:42:59.984+0000 7ff5d4800700 1 -- 172.21.15.67:0/3821074204 wait complete. 2024-03-30T02:43:00.002 INFO:tasks.cephfs.filesystem:scrub status for tag:e32ff20d-c7a4-48b3-b5dc-08cb42a587a6 - {'path': '/', 'tag': 'e32ff20d-c7a4-48b3-b5dc-08cb42a587a6', 'options': 'recursive,force'} 2024-03-30T02:43:00.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:00 smithi170 ceph-mon[78798]: pgmap v181: 129 pgs: 129 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 3.25k op/s 2024-03-30T02:43:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:00 smithi082 ceph-mon[79176]: pgmap v181: 129 pgs: 129 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 3.25k op/s 2024-03-30T02:43:00.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:00 smithi067 ceph-mon[37808]: pgmap v181: 129 pgs: 129 active+clean; 1.8 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 3.25k op/s 2024-03-30T02:43:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:02 smithi170 ceph-mon[78798]: pgmap v182: 129 pgs: 129 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 3.23k op/s 2024-03-30T02:43:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:43:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:43:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:43:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:43:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:43:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:02 smithi082 ceph-mon[79176]: pgmap v182: 129 pgs: 129 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 3.23k op/s 2024-03-30T02:43:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:43:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:43:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:43:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:43:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:43:02.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:02 smithi067 ceph-mon[37808]: pgmap v182: 129 pgs: 129 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.3 MiB/s wr, 3.23k op/s 2024-03-30T02:43:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:43:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:43:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:43:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:43:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:43:03.276 INFO:tasks.workunit.client.0.smithi067.stdout: CC scripts/mod/empty.o 2024-03-30T02:43:03.276 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/mod/mk_elfconfig 2024-03-30T02:43:03.277 INFO:tasks.workunit.client.0.smithi067.stdout: CC scripts/mod/devicetable-offsets.s 2024-03-30T02:43:03.357 INFO:tasks.workunit.client.0.smithi067.stdout: MKELF scripts/mod/elfconfig.h 2024-03-30T02:43:03.359 INFO:tasks.workunit.client.0.smithi067.stdout: UPD scripts/mod/devicetable-offsets.h 2024-03-30T02:43:03.364 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/mod/modpost.o 2024-03-30T02:43:03.365 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/mod/sumversion.o 2024-03-30T02:43:03.370 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC scripts/mod/file2alias.o 2024-03-30T02:43:03.826 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTLD scripts/mod/modpost 2024-03-30T02:43:03.868 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/bounds.s 2024-03-30T02:43:03.868 INFO:tasks.workunit.client.0.smithi067.stdout: CHKSHA1 include/linux/atomic/atomic-arch-fallback.h 2024-03-30T02:43:03.870 INFO:tasks.workunit.client.0.smithi067.stdout: CHKSHA1 include/linux/atomic/atomic-long.h 2024-03-30T02:43:03.870 INFO:tasks.workunit.client.0.smithi067.stdout: CHKSHA1 include/linux/atomic/atomic-instrumented.h 2024-03-30T02:43:03.901 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/timeconst.h 2024-03-30T02:43:03.949 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/bounds.h 2024-03-30T02:43:03.959 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/asm-offsets.s 2024-03-30T02:43:04.458 INFO:tasks.workunit.client.0.smithi067.stdout: UPD include/generated/asm-offsets.h 2024-03-30T02:43:04.467 INFO:tasks.workunit.client.0.smithi067.stdout: CALL scripts/checksyscalls.sh 2024-03-30T02:43:04.538 INFO:tasks.workunit.client.0.smithi067.stdout: LDS scripts/module.lds 2024-03-30T02:43:04.637 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/main.o 2024-03-30T02:43:04.638 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC usr/gen_init_cpio 2024-03-30T02:43:04.644 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/filemap.o 2024-03-30T02:43:04.644 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/compat.o 2024-03-30T02:43:04.646 INFO:tasks.workunit.client.0.smithi067.stdout: CC certs/system_keyring.o 2024-03-30T02:43:04.674 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/dnotify/dnotify.o 2024-03-30T02:43:04.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:04 smithi170 ceph-mon[78798]: pgmap v183: 129 pgs: 129 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 2.04k op/s 2024-03-30T02:43:04.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:04 smithi082 ceph-mon[79176]: pgmap v183: 129 pgs: 129 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 2.04k op/s 2024-03-30T02:43:04.763 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vma.o 2024-03-30T02:43:04.818 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sched/core.o 2024-03-30T02:43:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:04 smithi067 ceph-mon[37808]: pgmap v183: 129 pgs: 129 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 2.04k op/s 2024-03-30T02:43:04.984 INFO:tasks.workunit.client.0.smithi067.stdout: GEN usr/initramfs_data.cpio 2024-03-30T02:43:05.044 INFO:tasks.workunit.client.0.smithi067.stdout: COPY usr/initramfs_inc_data 2024-03-30T02:43:05.059 INFO:tasks.workunit.client.0.smithi067.stdout: AS usr/initramfs_data.o 2024-03-30T02:43:05.134 INFO:tasks.workunit.client.0.smithi067.stdout: AR usr/built-in.a 2024-03-30T02:43:05.141 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC certs/extract-cert 2024-03-30T02:43:05.210 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs_common/nfsacl.o 2024-03-30T02:43:05.416 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/util.o 2024-03-30T02:43:05.434 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/notify/dnotify/built-in.a 2024-03-30T02:43:05.487 INFO:tasks.workunit.client.0.smithi067.stdout: CERT certs/x509_certificate_list 2024-03-30T02:43:05.497 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/inotify/inotify_fsnotify.o 2024-03-30T02:43:05.554 INFO:tasks.workunit.client.0.smithi067.stdout: CERT certs/signing_key.x509 2024-03-30T02:43:05.569 INFO:tasks.workunit.client.0.smithi067.stdout: AS certs/system_certificates.o 2024-03-30T02:43:05.611 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/extable.o 2024-03-30T02:43:05.656 INFO:tasks.workunit.client.0.smithi067.stdout: AR certs/built-in.a 2024-03-30T02:43:05.714 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mempool.o 2024-03-30T02:43:05.931 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs_common/grace.o 2024-03-30T02:43:06.110 INFO:tasks.workunit.client.0.smithi067.stdout: UPD init/utsversion-tmp.h 2024-03-30T02:43:06.128 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/do_mounts.o 2024-03-30T02:43:06.139 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/inotify/inotify_user.o 2024-03-30T02:43:06.384 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso32-setup.o 2024-03-30T02:43:06.398 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/msgutil.o 2024-03-30T02:43:06.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:06 smithi170 ceph-mon[78798]: pgmap v184: 129 pgs: 129 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 1.66k op/s 2024-03-30T02:43:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:06 smithi082 ceph-mon[79176]: pgmap v184: 129 pgs: 129 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 1.66k op/s 2024-03-30T02:43:06.786 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/msg.o 2024-03-30T02:43:06.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:06 smithi067 ceph-mon[37808]: pgmap v184: 129 pgs: 129 active+clean; 1.8 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 1.66k op/s 2024-03-30T02:43:06.930 INFO:tasks.workunit.client.0.smithi067.stdout: LDS arch/x86/entry/vdso/vdso.lds 2024-03-30T02:43:07.003 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/nfs_common/built-in.a 2024-03-30T02:43:07.029 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vdso/vdso-note.o 2024-03-30T02:43:07.069 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vclock_gettime.o 2024-03-30T02:43:07.157 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sched/fair.o 2024-03-30T02:43:07.413 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/do_mounts_initrd.o 2024-03-30T02:43:07.675 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/notify/inotify/built-in.a 2024-03-30T02:43:07.743 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/notify/fanotify/built-in.a 2024-03-30T02:43:07.770 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/fsnotify.o 2024-03-30T02:43:07.785 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/oom_kill.o 2024-03-30T02:43:07.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:07 smithi067 ceph-mon[37808]: pgmap v185: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 1.28k op/s 2024-03-30T02:43:07.926 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vgetcpu.o 2024-03-30T02:43:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:07 smithi170 ceph-mon[78798]: pgmap v185: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 1.28k op/s 2024-03-30T02:43:07.959 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/gc.o 2024-03-30T02:43:07.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:07 smithi082 ceph-mon[79176]: pgmap v185: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 1.28k op/s 2024-03-30T02:43:08.048 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC arch/x86/entry/vdso/vdso2c 2024-03-30T02:43:08.369 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/initramfs.o 2024-03-30T02:43:08.388 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/sem.o 2024-03-30T02:43:08.582 INFO:tasks.workunit.client.0.smithi067.stdout: LDS arch/x86/entry/vdso/vdso32/vdso32.lds 2024-03-30T02:43:08.630 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/notification.o 2024-03-30T02:43:08.640 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vdso/vdso32/note.o 2024-03-30T02:43:08.758 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vdso/vdso32/system_call.o 2024-03-30T02:43:08.786 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/key.o 2024-03-30T02:43:08.925 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vdso/vdso32/sigreturn.o 2024-03-30T02:43:09.188 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso32/vclock_gettime.o 2024-03-30T02:43:09.337 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/group.o 2024-03-30T02:43:09.338 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/fadvise.o 2024-03-30T02:43:09.345 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/calibrate.o 2024-03-30T02:43:09.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:09 smithi067 ceph-mon[37808]: pgmap v186: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 615 op/s 2024-03-30T02:43:09.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:09 smithi170 ceph-mon[78798]: pgmap v186: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 615 op/s 2024-03-30T02:43:09.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:09 smithi082 ceph-mon[79176]: pgmap v186: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 615 op/s 2024-03-30T02:43:10.042 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/shm.o 2024-03-30T02:43:10.245 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/mark.o 2024-03-30T02:43:10.337 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso32/vgetcpu.o 2024-03-30T02:43:10.346 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/init_task.o 2024-03-30T02:43:10.346 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/maccess.o 2024-03-30T02:43:10.348 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/keyring.o 2024-03-30T02:43:10.381 INFO:tasks.workunit.client.0.smithi067.stdout: GEN security/selinux/flask.h security/selinux/av_permissions.h 2024-03-30T02:43:10.413 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/avc.o 2024-03-30T02:43:10.579 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sched/build_policy.o 2024-03-30T02:43:10.907 INFO:tasks.workunit.client.0.smithi067.stdout: VDSO arch/x86/entry/vdso/vdso64.so.dbg 2024-03-30T02:43:11.111 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page-writeback.o 2024-03-30T02:43:11.135 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/notify/fdinfo.o 2024-03-30T02:43:11.140 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/syscall.o 2024-03-30T02:43:11.292 INFO:tasks.workunit.client.0.smithi067.stdout: CC init/version.o 2024-03-30T02:43:11.664 INFO:tasks.workunit.client.0.smithi067.stdout: AR init/built-in.a 2024-03-30T02:43:11.674 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/keyctl.o 2024-03-30T02:43:11.699 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/permission.o 2024-03-30T02:43:11.905 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/notify/built-in.a 2024-03-30T02:43:11.916 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/ipc_sysctl.o 2024-03-30T02:43:11.923 INFO:tasks.workunit.client.0.smithi067.stdout: VDSO arch/x86/entry/vdso/vdso32.so.dbg 2024-03-30T02:43:11.929 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/hooks.o 2024-03-30T02:43:11.959 INFO:tasks.workunit.client.0.smithi067.stdout: OBJCOPY arch/x86/entry/vdso/vdso64.so 2024-03-30T02:43:11.967 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/trace.o 2024-03-30T02:43:11.973 INFO:tasks.workunit.client.0.smithi067.stdout: OBJCOPY arch/x86/entry/vdso/vdso32.so 2024-03-30T02:43:11.990 INFO:tasks.workunit.client.0.smithi067.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-64.c 2024-03-30T02:43:12.002 INFO:tasks.workunit.client.0.smithi067.stdout: VDSO2C arch/x86/entry/vdso/vdso-image-32.c 2024-03-30T02:43:12.019 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso-image-64.o 2024-03-30T02:43:12.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:12 smithi067 ceph-mon[37808]: pgmap v187: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 721 op/s 2024-03-30T02:43:12.680 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vdso/vdso-image-32.o 2024-03-30T02:43:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:12 smithi170 ceph-mon[78798]: pgmap v187: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 721 op/s 2024-03-30T02:43:12.693 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/mqueue.o 2024-03-30T02:43:12.695 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/namespace.o 2024-03-30T02:43:12.705 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/process_keys.o 2024-03-30T02:43:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:12 smithi082 ceph-mon[79176]: pgmap v187: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 721 op/s 2024-03-30T02:43:12.864 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/folio-compat.o 2024-03-30T02:43:12.915 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/iter.o 2024-03-30T02:43:12.986 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/entry/vdso/built-in.a 2024-03-30T02:43:13.069 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/vsyscall/vsyscall_64.o 2024-03-30T02:43:13.273 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sched/build_utility.o 2024-03-30T02:43:13.304 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/request_key.o 2024-03-30T02:43:13.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/buffered-io.o 2024-03-30T02:43:13.695 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/core.o 2024-03-30T02:43:13.705 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/readahead.o 2024-03-30T02:43:14.055 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/vsyscall/vsyscall_emu_64.o 2024-03-30T02:43:14.198 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/entry/vsyscall/built-in.a 2024-03-30T02:43:14.312 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/entry.o 2024-03-30T02:43:14.442 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/entry_64.o 2024-03-30T02:43:14.553 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/request_key_auth.o 2024-03-30T02:43:14.584 INFO:tasks.workunit.client.0.smithi067.stdout: CC ipc/mq_sysctl.o 2024-03-30T02:43:14.623 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/syscall_64.o 2024-03-30T02:43:14.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:14 smithi067 ceph-mon[37808]: pgmap v188: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.2 MiB/s wr, 615 op/s 2024-03-30T02:43:14.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:14 smithi170 ceph-mon[78798]: pgmap v188: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.2 MiB/s wr, 615 op/s 2024-03-30T02:43:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:14 smithi082 ceph-mon[79176]: pgmap v188: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.2 MiB/s wr, 615 op/s 2024-03-30T02:43:14.994 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/lbr.o 2024-03-30T02:43:15.066 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/swap.o 2024-03-30T02:43:15.133 INFO:tasks.workunit.client.0.smithi067.stdout: AR ipc/built-in.a 2024-03-30T02:43:15.187 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/truncate.o 2024-03-30T02:43:15.338 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/user_defined.o 2024-03-30T02:43:15.382 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/selinuxfs.o 2024-03-30T02:43:15.489 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/common.o 2024-03-30T02:43:15.653 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/direct-io.o 2024-03-30T02:43:15.890 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/ibs.o 2024-03-30T02:43:16.095 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/compat.o 2024-03-30T02:43:16.385 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/mutex.o 2024-03-30T02:43:16.415 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/thunk_64.o 2024-03-30T02:43:16.549 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/entry/entry_64_compat.o 2024-03-30T02:43:16.584 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/vmscan.o 2024-03-30T02:43:16.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:16 smithi067 ceph-mon[37808]: pgmap v189: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 585 op/s 2024-03-30T02:43:16.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:16 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:43:16.665 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/entry/syscall_32.o 2024-03-30T02:43:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:16 smithi170 ceph-mon[78798]: pgmap v189: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 585 op/s 2024-03-30T02:43:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:16 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:43:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:16 smithi082 ceph-mon[79176]: pgmap v189: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 585 op/s 2024-03-30T02:43:16.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:16 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:43:16.802 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/fiemap.o 2024-03-30T02:43:16.908 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/sched/built-in.a 2024-03-30T02:43:16.976 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/proc.o 2024-03-30T02:43:16.992 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/semaphore.o 2024-03-30T02:43:17.196 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/uncore.o 2024-03-30T02:43:17.252 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netlink.o 2024-03-30T02:43:17.577 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/entry/built-in.a 2024-03-30T02:43:17.642 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/amd/iommu.o 2024-03-30T02:43:17.667 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/seek.o 2024-03-30T02:43:17.717 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/iomap/swapfile.o 2024-03-30T02:43:17.831 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/sysctl.o 2024-03-30T02:43:17.896 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/rwsem.o 2024-03-30T02:43:18.159 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/nlmsgtab.o 2024-03-30T02:43:18.179 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/dquot.o 2024-03-30T02:43:18.486 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/keys/keyctl_pkey.o 2024-03-30T02:43:18.524 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/events/amd/built-in.a 2024-03-30T02:43:18.550 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/iomap/built-in.a 2024-03-30T02:43:18.603 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/core.o 2024-03-30T02:43:18.632 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/integrity/iint.o 2024-03-30T02:43:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:18 smithi067 ceph-mon[37808]: pgmap v190: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 817 KiB/s wr, 440 op/s 2024-03-30T02:43:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:18 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:43:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:18 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:43:18.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:18 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:43:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:18 smithi170 ceph-mon[78798]: pgmap v190: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 817 KiB/s wr, 440 op/s 2024-03-30T02:43:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:18 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:43:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:18 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:43:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:18 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:43:18.707 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/asymmetric_type.o 2024-03-30T02:43:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:18 smithi082 ceph-mon[79176]: pgmap v190: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 817 KiB/s wr, 440 op/s 2024-03-30T02:43:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:18 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:43:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:18 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:43:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:18 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:43:18.792 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/percpu-rwsem.o 2024-03-30T02:43:19.153 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netif.o 2024-03-30T02:43:19.181 INFO:tasks.workunit.client.0.smithi067.stdout: AR security/keys/built-in.a 2024-03-30T02:43:19.245 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/shmem.o 2024-03-30T02:43:19.273 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/quota_v2.o 2024-03-30T02:43:19.312 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/integrity/integrity_audit.o 2024-03-30T02:43:19.479 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/restrict.o 2024-03-30T02:43:19.490 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/spinlock.o 2024-03-30T02:43:19.837 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/bts.o 2024-03-30T02:43:19.962 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/osq_lock.o 2024-03-30T02:43:19.970 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/quota_tree.o 2024-03-30T02:43:19.984 INFO:tasks.workunit.client.0.smithi067.stdout: AR security/integrity/built-in.a 2024-03-30T02:43:20.013 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/signature.o 2024-03-30T02:43:20.051 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/init.o 2024-03-30T02:43:20.321 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netnode.o 2024-03-30T02:43:20.352 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/qspinlock.o 2024-03-30T02:43:20.616 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/public_key.o 2024-03-30T02:43:20.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:20 smithi067 ceph-mon[37808]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 879 KiB/s wr, 287 op/s 2024-03-30T02:43:20.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:20 smithi170 ceph-mon[78798]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 879 KiB/s wr, 287 op/s 2024-03-30T02:43:20.735 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/header.o 2024-03-30T02:43:20.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:20 smithi082 ceph-mon[79176]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 879 KiB/s wr, 287 op/s 2024-03-30T02:43:20.774 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/init.o 2024-03-30T02:43:20.810 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/trampoline_64.o 2024-03-30T02:43:20.812 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/quota.o 2024-03-30T02:43:20.884 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/stack.o 2024-03-30T02:43:20.893 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/ds.o 2024-03-30T02:43:20.948 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/reboot.o 2024-03-30T02:43:20.969 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/rtmutex_api.o 2024-03-30T02:43:21.017 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/wakeup_asm.o 2024-03-30T02:43:21.088 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/wakemain.o 2024-03-30T02:43:21.170 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/util.o 2024-03-30T02:43:21.227 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/video-mode.o 2024-03-30T02:43:21.239 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/bugs.o 2024-03-30T02:43:21.302 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/asymmetric_keys/x509.asn1.[ch] 2024-03-30T02:43:21.316 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/asymmetric_keys/x509_akid.asn1.[ch] 2024-03-30T02:43:21.331 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509_loader.o 2024-03-30T02:43:21.394 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/copy.o 2024-03-30T02:43:21.519 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netport.o 2024-03-30T02:43:21.604 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rm/bioscall.o 2024-03-30T02:43:21.617 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/core.o 2024-03-30T02:43:21.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:21 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:21.687 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509_public_key.o 2024-03-30T02:43:21.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:21 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:21.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:21 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:21.763 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/regs.o 2024-03-30T02:43:21.991 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/video-vga.o 2024-03-30T02:43:22.306 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/video-vesa.o 2024-03-30T02:43:22.389 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/locking/qrwlock.o 2024-03-30T02:43:22.509 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/realmode/rm/video-bios.o 2024-03-30T02:43:22.518 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/kqid.o 2024-03-30T02:43:22.660 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmzone.o 2024-03-30T02:43:22.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:22 smithi067 ceph-mon[37808]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.2 MiB/s wr, 267 op/s 2024-03-30T02:43:22.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:22 smithi170 ceph-mon[78798]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.2 MiB/s wr, 267 op/s 2024-03-30T02:43:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:22 smithi082 ceph-mon[79176]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 1.2 MiB/s wr, 267 op/s 2024-03-30T02:43:22.853 INFO:tasks.workunit.client.0.smithi067.stdout: PASYMS arch/x86/realmode/rm/pasyms.h 2024-03-30T02:43:22.858 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/asymmetric_keys/pkcs7.asn1.[ch] 2024-03-30T02:43:22.874 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/pkcs7_trust.o 2024-03-30T02:43:22.874 INFO:tasks.workunit.client.0.smithi067.stdout: LDS arch/x86/realmode/rm/realmode.lds 2024-03-30T02:43:22.920 INFO:tasks.workunit.client.0.smithi067.stdout: LD arch/x86/realmode/rm/realmode.elf 2024-03-30T02:43:22.942 INFO:tasks.workunit.client.0.smithi067.stdout: RELOCS arch/x86/realmode/rm/realmode.relocs 2024-03-30T02:43:22.955 INFO:tasks.workunit.client.0.smithi067.stdout: OBJCOPY arch/x86/realmode/rm/realmode.bin 2024-03-30T02:43:22.979 INFO:tasks.workunit.client.0.smithi067.stdout: AS arch/x86/realmode/rmpiggy.o 2024-03-30T02:43:23.050 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/realmode/built-in.a 2024-03-30T02:43:23.091 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/api.o 2024-03-30T02:43:23.130 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/knc.o 2024-03-30T02:43:23.201 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/status.o 2024-03-30T02:43:23.207 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/quota/netlink.o 2024-03-30T02:43:23.235 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/locking/built-in.a 2024-03-30T02:43:23.289 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/pkcs7_verify.o 2024-03-30T02:43:23.299 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/regset.o 2024-03-30T02:43:23.316 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/qos.o 2024-03-30T02:43:23.397 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/vmstat.o 2024-03-30T02:43:23.785 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509.asn1.o 2024-03-30T02:43:23.861 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509_akid.asn1.o 2024-03-30T02:43:23.941 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/x509_cert_parser.o 2024-03-30T02:43:23.974 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/lbr.o 2024-03-30T02:43:23.983 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pat/set_memory.o 2024-03-30T02:43:24.032 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/quota/built-in.a 2024-03-30T02:43:24.089 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/signal.o 2024-03-30T02:43:24.255 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/ebitmap.o 2024-03-30T02:43:24.314 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/main.o 2024-03-30T02:43:24.422 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/task_mmu.o 2024-03-30T02:43:24.537 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/pkcs7.asn1.o 2024-03-30T02:43:24.611 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/asymmetric_keys/pkcs7_parser.o 2024-03-30T02:43:24.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:24 smithi067 ceph-mon[37808]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-03-30T02:43:24.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:24 smithi170 ceph-mon[78798]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-03-30T02:43:24.749 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/backing-dev.o 2024-03-30T02:43:24.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:24 smithi082 ceph-mon[79176]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-03-30T02:43:24.996 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/fpu/xstate.o 2024-03-30T02:43:25.188 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/p4.o 2024-03-30T02:43:25.224 INFO:tasks.workunit.client.0.smithi067.stdout: AR crypto/asymmetric_keys/built-in.a 2024-03-30T02:43:25.275 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/cipher.o 2024-03-30T02:43:25.348 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/console.o 2024-03-30T02:43:25.453 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/hashtab.o 2024-03-30T02:43:25.465 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pat/memtype.o 2024-03-30T02:43:25.771 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mm_init.o 2024-03-30T02:43:25.840 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/inode.o 2024-03-30T02:43:25.964 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/compress.o 2024-03-30T02:43:25.995 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/symtab.o 2024-03-30T02:43:26.111 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/process.o 2024-03-30T02:43:26.146 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/p6.o 2024-03-30T02:43:26.200 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/fpu/built-in.a 2024-03-30T02:43:26.282 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/core.o 2024-03-30T02:43:26.356 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pat/memtype_interval.o 2024-03-30T02:43:26.386 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/sidtab.o 2024-03-30T02:43:26.628 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/algapi.o 2024-03-30T02:43:26.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:26 smithi170 ceph-mon[78798]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-03-30T02:43:26.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:26 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:26.702 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/root.o 2024-03-30T02:43:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:26 smithi082 ceph-mon[79176]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-03-30T02:43:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:26 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:26.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:26 smithi067 ceph-mon[37808]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-03-30T02:43:26.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:26 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:26.978 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/pt.o 2024-03-30T02:43:27.034 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/percpu.o 2024-03-30T02:43:27.067 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/mm/pat/built-in.a 2024-03-30T02:43:27.121 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/init.o 2024-03-30T02:43:27.160 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/suspend.o 2024-03-30T02:43:27.413 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/base.o 2024-03-30T02:43:27.639 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/avtab.o 2024-03-30T02:43:27.797 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/scatterwalk.o 2024-03-30T02:43:27.831 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/severity.o 2024-03-30T02:43:28.102 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/init_64.o 2024-03-30T02:43:28.219 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore.o 2024-03-30T02:43:28.246 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/hibernate.o 2024-03-30T02:43:28.508 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/proc.o 2024-03-30T02:43:28.660 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/genpool.o 2024-03-30T02:43:28.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:28 smithi170 ceph-mon[78798]: pgmap v195: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.0 MiB/s wr, 274 op/s 2024-03-30T02:43:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:28 smithi082 ceph-mon[79176]: pgmap v195: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.0 MiB/s wr, 274 op/s 2024-03-30T02:43:28.827 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/policydb.o 2024-03-30T02:43:28.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:28 smithi067 ceph-mon[37808]: pgmap v195: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.0 MiB/s wr, 274 op/s 2024-03-30T02:43:28.995 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/slab_common.o 2024-03-30T02:43:29.084 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/generic.o 2024-03-30T02:43:29.183 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/aead.o 2024-03-30T02:43:29.447 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/intel.o 2024-03-30T02:43:29.704 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/fault.o 2024-03-30T02:43:29.892 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/snapshot.o 2024-03-30T02:43:30.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:43:30.162 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore_nhmex.o 2024-03-30T02:43:30.282 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/array.o 2024-03-30T02:43:30.384 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/geniv.o 2024-03-30T02:43:30.403 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/amd.o 2024-03-30T02:43:30.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.500+0000 7f4c9ba00700 1 -- 172.21.15.67:0/69954895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c941082f0 msgr2=0x7f4c94108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:30.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.500+0000 7f4c93400700 1 -- 172.21.15.67:0/69954895 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80004230 con 0x7f4c941082f0 2024-03-30T02:43:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.503+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/69954895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c941082f0 0x7f4c94108770 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009990 tx=0x7f4c80009ca0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.503+0000 7f4c9ba00700 1 -- 172.21.15.67:0/69954895 shutdown_connections 2024-03-30T02:43:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.503+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/69954895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c9411a590 0x7f4c9411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.503+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/69954895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c9410f540 0x7f4c94107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.503+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/69954895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c941082f0 0x7f4c94108770 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.503+0000 7f4c9ba00700 1 -- 172.21.15.67:0/69954895 >> 172.21.15.67:0/69954895 conn(0x7f4c9406bdf0 msgr2=0x7f4c9406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:30.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.506+0000 7f4c9ba00700 1 -- 172.21.15.67:0/69954895 shutdown_connections 2024-03-30T02:43:30.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.506+0000 7f4c9ba00700 1 -- 172.21.15.67:0/69954895 wait complete. 2024-03-30T02:43:30.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.513+0000 7f4c9ba00700 1 Processor -- start 2024-03-30T02:43:30.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.513+0000 7f4c9ba00700 1 -- start start 2024-03-30T02:43:30.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.513+0000 7f4c9ba00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c941082f0 0x7f4c94118910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.517+0000 7f4c9ba00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c9410f540 0x7f4c94118e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.517+0000 7f4c9ba00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c9411a590 0x7f4c94111970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.517+0000 7f4c9ba00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94112140 con 0x7f4c941082f0 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.517+0000 7f4c9ba00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c941122b0 con 0x7f4c9411a590 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.517+0000 7f4c9ba00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94112470 con 0x7f4c9410f540 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.517+0000 7f4c99000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c941082f0 0x7f4c94118910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.517+0000 7f4c99000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c941082f0 0x7f4c94118910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60748/0 (socket says 172.21.15.67:60748) 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.517+0000 7f4c99000700 1 -- 172.21.15.67:0/3478582358 learned_addr learned my addr 172.21.15.67:0/3478582358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.520+0000 7f4c99000700 1 -- 172.21.15.67:0/3478582358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c9410f540 msgr2=0x7f4c94118e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.520+0000 7f4c99000700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c9410f540 0x7f4c94118e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.520+0000 7f4c99000700 1 -- 172.21.15.67:0/3478582358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c9411a590 msgr2=0x7f4c94111970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.520+0000 7f4c99000700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c9411a590 0x7f4c94111970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.520+0000 7f4c99000700 1 -- 172.21.15.67:0/3478582358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c800095f0 con 0x7f4c941082f0 2024-03-30T02:43:30.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.523+0000 7f4c99000700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c941082f0 0x7f4c94118910 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00ba50 tx=0x7f4c8c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:30.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.526+0000 7f4c91600700 1 -- 172.21.15.67:0/3478582358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00cab0 con 0x7f4c941082f0 2024-03-30T02:43:30.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.527+0000 7f4c91600700 1 -- 172.21.15.67:0/3478582358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c8c00cc10 con 0x7f4c941082f0 2024-03-30T02:43:30.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.527+0000 7f4c91600700 1 -- 172.21.15.67:0/3478582358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0075b0 con 0x7f4c941082f0 2024-03-30T02:43:30.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.528+0000 7f4c9ba00700 1 -- 172.21.15.67:0/3478582358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c94112750 con 0x7f4c941082f0 2024-03-30T02:43:30.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.528+0000 7f4c9ba00700 1 -- 172.21.15.67:0/3478582358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c941166d0 con 0x7f4c941082f0 2024-03-30T02:43:30.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.558+0000 7f4caf56ab80 1 Processor -- start 2024-03-30T02:43:30.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.559+0000 7f4c91600700 1 -- 172.21.15.67:0/3478582358 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4c8c014440 con 0x7f4c941082f0 2024-03-30T02:43:30.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.562+0000 7f4caf56ab80 1 -- start start 2024-03-30T02:43:30.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.562+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d75d0 0x55c1854d79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.562+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.563+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 0x55c1854e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.563+0000 7f4caf56ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c185430990 con 0x55c1854e2df0 2024-03-30T02:43:30.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.563+0000 7f4caf56ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c18542e350 con 0x55c1854d7ef0 2024-03-30T02:43:30.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.563+0000 7f4c99a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d75d0 0x55c1854d79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.563+0000 7f4caf56ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c185429c40 con 0x55c1854d75d0 2024-03-30T02:43:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.563+0000 7f4c99a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d75d0 0x55c1854d79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:42806/0 (socket says 172.21.15.67:42806) 2024-03-30T02:43:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.563+0000 7f4c99a00700 1 -- 172.21.15.67:0/2402424509 learned_addr learned my addr 172.21.15.67:0/2402424509 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.564+0000 7f4c99000700 1 --2- 172.21.15.67:0/2402424509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.565+0000 7f4c99000700 1 -- 172.21.15.67:0/2402424509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d75d0 msgr2=0x55c1854d79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.565+0000 7f4c99000700 1 --2- 172.21.15.67:0/2402424509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d75d0 0x55c1854d79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.565+0000 7f4c99000700 1 -- 172.21.15.67:0/2402424509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 msgr2=0x55c1854e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.565+0000 7f4c99000700 1 --2- 172.21.15.67:0/2402424509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 0x55c1854e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.565+0000 7f4c99000700 1 -- 172.21.15.67:0/2402424509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1852ab3b0 con 0x55c1854d7ef0 2024-03-30T02:43:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.567+0000 7f4c99000700 1 --2- 172.21.15.67:0/2402424509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854e28b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0c9af0 tx=0x7f4c8c0d28f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b04b12bd604d254d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.567+0000 7f4c91600700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c780a13d0 0x7f4c780a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.568+0000 7f4c91600700 1 -- 172.21.15.67:0/3478582358 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f4c8c0cc640 con 0x7f4c941082f0 2024-03-30T02:43:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.569+0000 7f4c8a000700 1 -- 172.21.15.67:0/2402424509 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c009930 con 0x55c1854d7ef0 2024-03-30T02:43:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.569+0000 7f4c8a000700 1 -- 172.21.15.67:0/2402424509 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c8c0957f0 con 0x55c1854d7ef0 2024-03-30T02:43:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.569+0000 7f4c8a000700 1 -- 172.21.15.67:0/2402424509 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0994b0 con 0x55c1854d7ef0 2024-03-30T02:43:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.570+0000 7f4caf56ab80 1 -- 172.21.15.67:0/2402424509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 msgr2=0x55c1854e28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:30.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.570+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/2402424509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854e28b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0c9af0 tx=0x7f4c8c0d28f0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.571+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c780a13d0 0x7f4c780a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.574+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c780a13d0 0x7f4c780a3890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c84005d90 tx=0x7f4c84007350 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.576+0000 7f4caf56ab80 1 -- 172.21.15.67:0/2402424509 shutdown_connections 2024-03-30T02:43:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.576+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/2402424509 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 0x55c1854e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.576+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/2402424509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d75d0 0x55c1854d79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.576+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/2402424509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854e28b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.576+0000 7f4caf56ab80 1 -- 172.21.15.67:0/2402424509 >> 172.21.15.67:0/2402424509 conn(0x55c1854376a0 msgr2=0x55c1854d3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.576+0000 7f4caf56ab80 1 -- 172.21.15.67:0/2402424509 shutdown_connections 2024-03-30T02:43:30.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.577+0000 7f4caf56ab80 1 -- 172.21.15.67:0/2402424509 wait complete. 2024-03-30T02:43:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.581+0000 7f4caf56ab80 1 -- start start 2024-03-30T02:43:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.581+0000 7f4caf56ab80 1 Processor -- start 2024-03-30T02:43:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.582+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d75d0 0x55c1854e2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.582+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.583+0000 7f4c99000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d75d0 0x55c1854e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.583+0000 7f4c99000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d75d0 0x55c1854e2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60768/0 (socket says 172.21.15.67:60768) 2024-03-30T02:43:30.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.583+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 0x55c1854dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.583+0000 7f4c93e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.583+0000 7f4c93e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854dfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52264/0 (socket says 172.21.15.67:52264) 2024-03-30T02:43:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.583+0000 7f4c93e00700 1 -- 172.21.15.67:0/3566924951 learned_addr learned my addr 172.21.15.67:0/3566924951 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.584+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 0x55c1854dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.584+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1853c5d80 con 0x55c1854d75d0 2024-03-30T02:43:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.584+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1853c5a20 con 0x55c1854d7ef0 2024-03-30T02:43:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.584+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1852ad6a0 con 0x55c1854e2df0 2024-03-30T02:43:30.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.584+0000 7f4c99a00700 1 -- 172.21.15.67:0/3566924951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 msgr2=0x55c1854dfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.584+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854dfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.584+0000 7f4c99a00700 1 -- 172.21.15.67:0/3566924951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d75d0 msgr2=0x55c1854e2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.584+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d75d0 0x55c1854e2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.584+0000 7f4c99a00700 1 -- 172.21.15.67:0/3566924951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c941166d0 con 0x55c1854e2df0 2024-03-30T02:43:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.585+0000 7f4caf56ab80 10 client.0 _ll_register_callbacks cb 0x55c1853cf470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.585+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1853ea380 con 0x55c1854e2df0 2024-03-30T02:43:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.585+0000 7f4c99000700 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d75d0 0x55c1854e2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-30T02:43:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.585+0000 7f4c7ea00700 10 client.26845 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:43:30.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.585+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 0x55c1854dffd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c84032e30 tx=0x7f4c840047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5384b06c8aa3e121 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:30.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.589+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3566924951 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84040620 con 0x55c1854e2df0 2024-03-30T02:43:30.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.589+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3566924951 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f4c84039070 con 0x55c1854e2df0 2024-03-30T02:43:30.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.589+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3566924951 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c84032e60 con 0x55c1854e2df0 2024-03-30T02:43:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.592+0000 7f4caf56ab80 10 client.26845 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.593+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=36}) v3 -- 0x55c185257b60 con 0x55c1854e2df0 2024-03-30T02:43:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.593+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3566924951 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f4c84040780 con 0x55c1854e2df0 2024-03-30T02:43:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.594+0000 7f4caf56ab80 10 client.26845 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.594+0000 7f4caf56ab80 10 client.26845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:43:30.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.594+0000 7f4caf56ab80 20 client.26845 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.594+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1854f2890 0x55c1854f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.594+0000 7f4caf56ab80 4 client.26845 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:43:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.594+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c185422700 con 0x55c1854f2890 2024-03-30T02:43:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.594+0000 7f4c99000700 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1854f2890 0x55c1854f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.595+0000 7f4c7ea00700 10 client.26845 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:30.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.595+0000 7f4c99000700 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1854f2890 0x55c1854f4cf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc4af7940a74c67 server_cookie=8ffd65abca48c6f6 in_seq=0 out_seq=0 2024-03-30T02:43:30.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.595+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3566924951 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c185422700 con 0x55c1854f2890 2024-03-30T02:43:30.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.596+0000 7f4c7ea00700 10 client.26845 handle_command_reply: tid=0 2024-03-30T02:43:30.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.598+0000 7f4caf56ab80 1 client.26845 shutdown 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.600+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 msgr2=0x55c1854dffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.600+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 0x55c1854dffd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4c84032e30 tx=0x7f4c840047c0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.600+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 shutdown_connections 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.600+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c1854f2890 0x55c1854f4cf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.600+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d75d0 0x55c1854e2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.600+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 0x55c1854dffd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.600+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3566924951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d7ef0 0x55c1854dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.600+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 >> 172.21.15.67:0/3566924951 conn(0x55c1854376a0 msgr2=0x55c1854d96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.600+0000 7f4c7ea00700 0 client.26845 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.601+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 shutdown_connections 2024-03-30T02:43:30.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.601+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3566924951 wait complete. 2024-03-30T02:43:30.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.601+0000 7f4caf56ab80 20 client.26845 trim_cache size 0 max 16384 2024-03-30T02:43:30.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.672+0000 7f4caf56ab80 1 Processor -- start 2024-03-30T02:43:30.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.672+0000 7f4caf56ab80 1 -- start start 2024-03-30T02:43:30.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.673+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 0x55c1854e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.673+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d7ef0 0x55c1854e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.673+0000 7f4c99000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 0x55c1854e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.674+0000 7f4c99000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 0x55c1854e9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52272/0 (socket says 172.21.15.67:52272) 2024-03-30T02:43:30.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.674+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 0x55c1854ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.674+0000 7f4c93e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d7ef0 0x55c1854e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.674+0000 7f4caf56ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1854ddf60 con 0x55c1854d7ef0 2024-03-30T02:43:30.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.674+0000 7f4caf56ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1854db600 con 0x55c1854d75d0 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.674+0000 7f4caf56ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1854d3f20 con 0x55c1854e2df0 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.674+0000 7f4c99000700 1 -- 172.21.15.67:0/3741057693 learned_addr learned my addr 172.21.15.67:0/3741057693 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.674+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3741057693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 0x55c1854ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.675+0000 7f4c99000700 1 -- 172.21.15.67:0/3741057693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 msgr2=0x55c1854ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.675+0000 7f4c99000700 1 --2- 172.21.15.67:0/3741057693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 0x55c1854ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.675+0000 7f4c99000700 1 -- 172.21.15.67:0/3741057693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d7ef0 msgr2=0x55c1854e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.675+0000 7f4c99000700 1 --2- 172.21.15.67:0/3741057693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d7ef0 0x55c1854e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.675+0000 7f4c99000700 1 -- 172.21.15.67:0/3741057693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1853efea0 con 0x55c1854d75d0 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.675+0000 7f4c99000700 1 --2- 172.21.15.67:0/3741057693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 0x55c1854e9b00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0c9b20 tx=0x7f4c8c0b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72854f5952257a16 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:30.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.675+0000 7f4c8aa00700 1 -- 172.21.15.67:0/3741057693 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0cb070 con 0x55c1854d75d0 2024-03-30T02:43:30.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.678+0000 7f4c8aa00700 1 -- 172.21.15.67:0/3741057693 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c8c009930 con 0x55c1854d75d0 2024-03-30T02:43:30.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.678+0000 7f4c8aa00700 1 -- 172.21.15.67:0/3741057693 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c099bb0 con 0x55c1854d75d0 2024-03-30T02:43:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.679+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3741057693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 msgr2=0x55c1854e9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.679+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3741057693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 0x55c1854e9b00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0c9b20 tx=0x7f4c8c0b9040 comp rx=0 tx=0).stop 2024-03-30T02:43:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.679+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3741057693 shutdown_connections 2024-03-30T02:43:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.679+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3741057693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854e2df0 0x55c1854ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.679+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3741057693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 0x55c1854e9b00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.679+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3741057693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854d7ef0 0x55c1854e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.679+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3741057693 >> 172.21.15.67:0/3741057693 conn(0x55c1854376a0 msgr2=0x55c1854de2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:30.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.679+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3741057693 shutdown_connections 2024-03-30T02:43:30.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.679+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3741057693 wait complete. 2024-03-30T02:43:30.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.680+0000 7f4caf56ab80 1 -- start start 2024-03-30T02:43:30.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.680+0000 7f4caf56ab80 1 Processor -- start 2024-03-30T02:43:30.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.688+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 0x55c1854e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.688+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d7ef0 0x55c1854e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.688+0000 7f4caf56ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 0x55c1854e8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.688+0000 7f4caf56ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1854e88a0 con 0x55c1854e2df0 2024-03-30T02:43:30.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.688+0000 7f4caf56ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1854e8a10 con 0x55c1854d75d0 2024-03-30T02:43:30.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.688+0000 7f4caf56ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1854e8bd0 con 0x55c1854d7ef0 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.688+0000 7f4caf56ab80 10 client.0 _ll_register_callbacks cb 0x55c1853cf470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.689+0000 7f4c99a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 0x55c1854e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.689+0000 7f4c99a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 0x55c1854e8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60794/0 (socket says 172.21.15.67:60794) 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.689+0000 7f4c99a00700 1 -- 172.21.15.67:0/3107519251 learned_addr learned my addr 172.21.15.67:0/3107519251 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.689+0000 7f4c93e00700 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d7ef0 0x55c1854e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.690+0000 7f4c99a00700 1 -- 172.21.15.67:0/3107519251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d7ef0 msgr2=0x55c1854e7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.690+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d7ef0 0x55c1854e7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.690+0000 7f4c99a00700 1 -- 172.21.15.67:0/3107519251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 msgr2=0x55c1854e98f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.690+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 0x55c1854e98f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.690+0000 7f4c99a00700 1 -- 172.21.15.67:0/3107519251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1852ab3b0 con 0x55c1854e2df0 2024-03-30T02:43:30.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.690+0000 7f4c99a00700 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 0x55c1854e8320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4c84040be0 tx=0x7f4c84039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bdc9ad25133168 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:30.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.690+0000 7f4c7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:43:30.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.694+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3107519251 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84005220 con 0x55c1854e2df0 2024-03-30T02:43:30.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.694+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3107519251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c185278cf0 con 0x55c1854e2df0 2024-03-30T02:43:30.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.694+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3107519251 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f4c84032e60 con 0x55c1854e2df0 2024-03-30T02:43:30.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.695+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3107519251 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c84005220 con 0x55c1854e2df0 2024-03-30T02:43:30.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.695+0000 7f4caf56ab80 10 client.16797 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:30.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.695+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3107519251 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55c1852afcd0 con 0x55c1854e2df0 2024-03-30T02:43:30.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.696+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3107519251 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c185278cf0 con 0x55c1854e2df0 2024-03-30T02:43:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.697+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3107519251 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f4c84005410 con 0x55c1854e2df0 2024-03-30T02:43:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.697+0000 7f4caf56ab80 10 client.16797 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.697+0000 7f4caf56ab80 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:43:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.697+0000 7f4caf56ab80 20 client.16797 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.697+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c185547ca0 0x55c18554a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:30.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.697+0000 7f4caf56ab80 4 client.16797 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:43:30.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.697+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3107519251 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1852adce0 con 0x55c185547ca0 2024-03-30T02:43:30.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.697+0000 7f4c99000700 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c185547ca0 0x55c18554a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:30.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.701+0000 7f4c99000700 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c185547ca0 0x55c18554a0c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f59e65b70dd9d9fa server_cookie=3b5e46be6774aec3 in_seq=0 out_seq=0 2024-03-30T02:43:30.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.701+0000 7f4c7ea00700 10 client.16797 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:30.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.701+0000 7f4c7ea00700 1 -- 172.21.15.67:0/3107519251 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c1852adce0 con 0x55c185547ca0 2024-03-30T02:43:30.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.701+0000 7f4c7ea00700 10 client.16797 handle_command_reply: tid=0 2024-03-30T02:43:30.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.701+0000 7f4caf56ab80 1 client.16797 shutdown 2024-03-30T02:43:30.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.702+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3107519251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 msgr2=0x55c1854e8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:30.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.702+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 0x55c1854e8320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4c84040be0 tx=0x7f4c84039000 comp rx=0 tx=0).stop 2024-03-30T02:43:30.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.703+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3107519251 shutdown_connections 2024-03-30T02:43:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.703+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c185547ca0 0x55c18554a0c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.703+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c1854e2df0 0x55c1854e8320 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.703+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1854d75d0 0x55c1854e98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.703+0000 7f4caf56ab80 1 --2- 172.21.15.67:0/3107519251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1854d7ef0 0x55c1854e7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.703+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3107519251 >> 172.21.15.67:0/3107519251 conn(0x55c1854376a0 msgr2=0x55c1854d96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.703+0000 7f4c7ea00700 0 client.16797 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.703+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3107519251 shutdown_connections 2024-03-30T02:43:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.703+0000 7f4caf56ab80 1 -- 172.21.15.67:0/3107519251 wait complete. 2024-03-30T02:43:30.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.704+0000 7f4caf56ab80 20 client.16797 trim_cache size 0 max 16384 2024-03-30T02:43:30.705 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:43:30.705 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:43:30.705 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:43:30.705 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:43:30.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.705+0000 7f4c9ba00700 1 -- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c780a13d0 msgr2=0x7f4c780a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:30.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.705+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c780a13d0 0x7f4c780a3890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c84005d90 tx=0x7f4c84007350 comp rx=0 tx=0).stop 2024-03-30T02:43:30.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.705+0000 7f4c9ba00700 1 -- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c941082f0 msgr2=0x7f4c94118910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:30.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.705+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c941082f0 0x7f4c94118910 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00ba50 tx=0x7f4c8c00be10 comp rx=0 tx=0).stop 2024-03-30T02:43:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.706+0000 7f4c9ba00700 1 -- 172.21.15.67:0/3478582358 shutdown_connections 2024-03-30T02:43:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.706+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4c780a13d0 0x7f4c780a3890 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.706+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4c941082f0 0x7f4c94118910 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.706+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c9411a590 0x7f4c94111970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.706+0000 7f4c9ba00700 1 --2- 172.21.15.67:0/3478582358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c9410f540 0x7f4c94118e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:30.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.706+0000 7f4c9ba00700 1 -- 172.21.15.67:0/3478582358 >> 172.21.15.67:0/3478582358 conn(0x7f4c9406bdf0 msgr2=0x7f4c9406f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:30.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.708+0000 7f4c9ba00700 1 -- 172.21.15.67:0/3478582358 shutdown_connections 2024-03-30T02:43:30.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:30.708+0000 7f4c9ba00700 1 -- 172.21.15.67:0/3478582358 wait complete. 2024-03-30T02:43:30.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:30 smithi082 ceph-mon[79176]: pgmap v196: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 749 KiB/s wr, 256 op/s 2024-03-30T02:43:30.785 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:30 smithi067 ceph-mon[37808]: pgmap v196: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 749 KiB/s wr, 256 op/s 2024-03-30T02:43:30.793 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:43:30.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:43:30.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:30 smithi170 ceph-mon[78798]: pgmap v196: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 749 KiB/s wr, 256 op/s 2024-03-30T02:43:31.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.377+0000 7fdee8a00700 1 -- 172.21.15.67:0/1794616760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee410e830 msgr2=0x7fdee4116330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:31.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.377+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1794616760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee410e830 0x7fdee4116330 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00b0d0 tx=0x7fdedc00b3e0 comp rx=0 tx=0).stop 2024-03-30T02:43:31.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.381+0000 7fdee8a00700 1 -- 172.21.15.67:0/1794616760 shutdown_connections 2024-03-30T02:43:31.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.381+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1794616760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee410e830 0x7fdee4116330 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:31.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.381+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1794616760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee4108020 0x7fdee4108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:31.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.381+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1794616760 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdee41089d0 0x7fdee410e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:31.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.381+0000 7fdee8a00700 1 -- 172.21.15.67:0/1794616760 >> 172.21.15.67:0/1794616760 conn(0x7fdee406c140 msgr2=0x7fdee406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:31.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.381+0000 7fdee8a00700 1 -- 172.21.15.67:0/1794616760 shutdown_connections 2024-03-30T02:43:31.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.381+0000 7fdee8a00700 1 -- 172.21.15.67:0/1794616760 wait complete. 2024-03-30T02:43:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.387+0000 7fdee8a00700 1 Processor -- start 2024-03-30T02:43:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.387+0000 7fdee8a00700 1 -- start start 2024-03-30T02:43:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.387+0000 7fdee8a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdee4108020 0x7fdee4118950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.387+0000 7fdee8a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee410e830 0x7fdee4118e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.387+0000 7fdee8a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee411d1b0 0x7fdee411d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.387+0000 7fdee8a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee411dba0 con 0x7fdee4108020 2024-03-30T02:43:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.387+0000 7fdee8a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee411dd10 con 0x7fdee411d1b0 2024-03-30T02:43:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.387+0000 7fdee8a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee411ded0 con 0x7fdee410e830 2024-03-30T02:43:31.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee411d1b0 0x7fdee411d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:31.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee411d1b0 0x7fdee411d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52292/0 (socket says 172.21.15.67:52292) 2024-03-30T02:43:31.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee2800700 1 -- 172.21.15.67:0/1289249114 learned_addr learned my addr 172.21.15.67:0/1289249114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:31.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee1e00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdee4108020 0x7fdee4118950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee2800700 1 -- 172.21.15.67:0/1289249114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee410e830 msgr2=0x7fdee4118e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee1400700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee410e830 0x7fdee4118e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee2800700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee410e830 0x7fdee4118e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee2800700 1 -- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdee4108020 msgr2=0x7fdee4118950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee2800700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdee4108020 0x7fdee4118950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee2800700 1 -- 172.21.15.67:0/1289249114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc009d00 con 0x7fdee411d1b0 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee1400700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee410e830 0x7fdee4118e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.388+0000 7fdee1e00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdee4108020 0x7fdee4118950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.389+0000 7fdee2800700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee411d1b0 0x7fdee411d660 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00b820 tx=0x7fdedc0078a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.389+0000 7fdedaa00700 1 -- 172.21.15.67:0/1289249114 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc00e040 con 0x7fdee411d1b0 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.389+0000 7fdedaa00700 1 -- 172.21.15.67:0/1289249114 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdedc005a80 con 0x7fdee411d1b0 2024-03-30T02:43:31.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.389+0000 7fdedaa00700 1 -- 172.21.15.67:0/1289249114 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc01bd50 con 0x7fdee411d1b0 2024-03-30T02:43:31.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.389+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee4123450 con 0x7fdee411d1b0 2024-03-30T02:43:31.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.390+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee4123910 con 0x7fdee411d1b0 2024-03-30T02:43:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.393+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee4072a30 con 0x7fdee411d1b0 2024-03-30T02:43:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.398+0000 7fdedaa00700 1 -- 172.21.15.67:0/1289249114 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fdedc01d890 con 0x7fdee411d1b0 2024-03-30T02:43:31.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.400+0000 7fdedaa00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdec80a1410 0x7fdec80a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:31.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.401+0000 7fdedaa00700 1 -- 172.21.15.67:0/1289249114 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fdedc0ce520 con 0x7fdee411d1b0 2024-03-30T02:43:31.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.401+0000 7fdee1e00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdec80a1410 0x7fdec80a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:31.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.404+0000 7fdee1e00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdec80a1410 0x7fdec80a38d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdee4119e90 tx=0x7fded400b690 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:31.407 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/compaction.o 2024-03-30T02:43:31.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:31.408+0000 7fdedaa00700 1 -- 172.21.15.67:0/1289249114 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fdedc097740 con 0x7fdee411d1b0 2024-03-30T02:43:31.489 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/ioremap.o 2024-03-30T02:43:31.625 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/services.o 2024-03-30T02:43:31.634 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/skcipher.o 2024-03-30T02:43:31.639 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore_snb.o 2024-03-30T02:43:31.697 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/fd.o 2024-03-30T02:43:31.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:31 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:31.871 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/swap.o 2024-03-30T02:43:31.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:31 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:31.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:31 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:32.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.028+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdee40679a0 con 0x7fdee411d1b0 2024-03-30T02:43:32.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.033+0000 7fdedaa00700 1 -- 172.21.15.67:0/1289249114 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10920 (secure 0 0 0) 0x7fdedc01beb0 con 0x7fdee411d1b0 2024-03-30T02:43:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:43:32.034 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:43:30.443130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[0,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:43:32.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdec80a1410 msgr2=0x7fdec80a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdec80a1410 0x7fdec80a38d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdee4119e90 tx=0x7fded400b690 comp rx=0 tx=0).stop 2024-03-30T02:43:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee411d1b0 msgr2=0x7fdee411d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee411d1b0 0x7fdee411d660 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00b820 tx=0x7fdedc0078a0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 shutdown_connections 2024-03-30T02:43:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fdec80a1410 0x7fdec80a38d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdee4108020 0x7fdee4118950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee411d1b0 0x7fdee411d660 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 --2- 172.21.15.67:0/1289249114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee410e830 0x7fdee4118e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.040+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 >> 172.21.15.67:0/1289249114 conn(0x7fdee406c140 msgr2=0x7fdee4070910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:32.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.043+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 shutdown_connections 2024-03-30T02:43:32.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.043+0000 7fdee8a00700 1 -- 172.21.15.67:0/1289249114 wait complete. 2024-03-30T02:43:32.044 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 36 2024-03-30T02:43:32.097 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:43:32.110 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mce/threshold.o 2024-03-30T02:43:32.576 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/extable.o 2024-03-30T02:43:32.599 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/proc_tty.o 2024-03-30T02:43:32.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.669+0000 7efd2b200700 1 -- 172.21.15.67:0/4035079445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd240981c0 msgr2=0x7efd240985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.669+0000 7efd2b200700 1 --2- 172.21.15.67:0/4035079445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd240981c0 0x7efd240985a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efd18013ef0 tx=0x7efd180097f0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.669+0000 7efd2b200700 1 -- 172.21.15.67:0/4035079445 shutdown_connections 2024-03-30T02:43:32.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.669+0000 7efd2b200700 1 --2- 172.21.15.67:0/4035079445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd240981c0 0x7efd240985a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.669+0000 7efd2b200700 1 --2- 172.21.15.67:0/4035079445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd2409d2e0 0x7efd240a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.669+0000 7efd2b200700 1 --2- 172.21.15.67:0/4035079445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd24098b70 0x7efd2409cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.669+0000 7efd2b200700 1 -- 172.21.15.67:0/4035079445 >> 172.21.15.67:0/4035079445 conn(0x7efd24093a60 msgr2=0x7efd24095e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:32.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.670+0000 7efd2b200700 1 -- 172.21.15.67:0/4035079445 shutdown_connections 2024-03-30T02:43:32.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.670+0000 7efd2b200700 1 -- 172.21.15.67:0/4035079445 wait complete. 2024-03-30T02:43:32.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.671+0000 7efd2b200700 1 Processor -- start 2024-03-30T02:43:32.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.672+0000 7efd2b200700 1 -- start start 2024-03-30T02:43:32.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.672+0000 7efd2b200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd240981c0 0x7efd2412a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.672+0000 7efd2b200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd2409d2e0 0x7efd2412ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.672+0000 7efd2b200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd2412e8d0 0x7efd2412ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.672+0000 7efd2b200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2412f350 con 0x7efd2412e8d0 2024-03-30T02:43:32.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.672+0000 7efd2b200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2412f4c0 con 0x7efd2409d2e0 2024-03-30T02:43:32.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.672+0000 7efd2b200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2412f680 con 0x7efd240981c0 2024-03-30T02:43:32.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.673+0000 7efd29400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd2409d2e0 0x7efd2412ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.673+0000 7efd29400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd2409d2e0 0x7efd2412ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52310/0 (socket says 172.21.15.67:52310) 2024-03-30T02:43:32.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.673+0000 7efd29400700 1 -- 172.21.15.67:0/213373059 learned_addr learned my addr 172.21.15.67:0/213373059 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:32.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.673+0000 7efd2a800700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd2412e8d0 0x7efd2412ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.674+0000 7efd29400700 1 -- 172.21.15.67:0/213373059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd240981c0 msgr2=0x7efd2412a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.674+0000 7efd29400700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd240981c0 0x7efd2412a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.674+0000 7efd29400700 1 -- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd2412e8d0 msgr2=0x7efd2412ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.674+0000 7efd29400700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd2412e8d0 0x7efd2412ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.674+0000 7efd29400700 1 -- 172.21.15.67:0/213373059 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18013b50 con 0x7efd2409d2e0 2024-03-30T02:43:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.674+0000 7efd2a800700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd2412e8d0 0x7efd2412ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.675+0000 7efd29400700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd2409d2e0 0x7efd2412ad50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efd2c04f8d0 tx=0x7efd2c079f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.675+0000 7efd22a00700 1 -- 172.21.15.67:0/213373059 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c06a040 con 0x7efd2409d2e0 2024-03-30T02:43:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.676+0000 7efd2b200700 1 -- 172.21.15.67:0/213373059 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd24136be0 con 0x7efd2409d2e0 2024-03-30T02:43:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.676+0000 7efd22a00700 1 -- 172.21.15.67:0/213373059 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd2c077e40 con 0x7efd2409d2e0 2024-03-30T02:43:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.676+0000 7efd22a00700 1 -- 172.21.15.67:0/213373059 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c083360 con 0x7efd2409d2e0 2024-03-30T02:43:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.676+0000 7efd2b200700 1 -- 172.21.15.67:0/213373059 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd24137070 con 0x7efd2409d2e0 2024-03-30T02:43:32.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.686+0000 7efd22a00700 1 -- 172.21.15.67:0/213373059 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efd2c07c640 con 0x7efd2409d2e0 2024-03-30T02:43:32.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.692+0000 7efd44d81b80 1 Processor -- start 2024-03-30T02:43:32.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.692+0000 7efd22a00700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efd100a1390 0x7efd100a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.693+0000 7efd22a00700 1 -- 172.21.15.67:0/213373059 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7efd2c069070 con 0x7efd2409d2e0 2024-03-30T02:43:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.692+0000 7efd44d81b80 1 -- start start 2024-03-30T02:43:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.693+0000 7efd2a800700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efd100a1390 0x7efd100a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.693+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac857a0 0x56031ac85b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.693+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac90a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.693+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.693+0000 7efd44d81b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031abde990 con 0x56031ac857a0 2024-03-30T02:43:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.693+0000 7efd44d81b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031abdc350 con 0x56031ac90fc0 2024-03-30T02:43:32.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.693+0000 7efd44d81b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031abd7c40 con 0x56031ac860c0 2024-03-30T02:43:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.694+0000 7efd29e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac857a0 0x56031ac85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.694+0000 7efd29e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac857a0 0x56031ac85b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60848/0 (socket says 172.21.15.67:60848) 2024-03-30T02:43:32.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.694+0000 7efd29e00700 1 -- 172.21.15.67:0/1447662812 learned_addr learned my addr 172.21.15.67:0/1447662812 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:32.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.694+0000 7efd2a800700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efd100a1390 0x7efd100a3850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd1c005fd0 tx=0x7efd1c005960 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:32.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.695+0000 7efd29400700 1 --2- 172.21.15.67:0/1447662812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac90a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.695+0000 7efd29e00700 1 -- 172.21.15.67:0/1447662812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 msgr2=0x56031ac90a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:32.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.695+0000 7efd29e00700 1 --2- 172.21.15.67:0/1447662812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac90a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.695+0000 7efd29e00700 1 -- 172.21.15.67:0/1447662812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 msgr2=0x56031ac933b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:43:32.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.695+0000 7efd29e00700 1 --2- 172.21.15.67:0/1447662812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac933b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.695+0000 7efd29e00700 1 -- 172.21.15.67:0/1447662812 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56031aa593b0 con 0x56031ac857a0 2024-03-30T02:43:32.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.695+0000 7efd29e00700 1 --2- 172.21.15.67:0/1447662812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac857a0 0x56031ac85b80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efd18009c70 tx=0x7efd18013090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b17fb219a7575b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:32.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.696+0000 7efd17400700 1 -- 172.21.15.67:0/1447662812 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18005e20 con 0x56031ac857a0 2024-03-30T02:43:32.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.696+0000 7efd17400700 1 -- 172.21.15.67:0/1447662812 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd18003880 con 0x56031ac857a0 2024-03-30T02:43:32.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.696+0000 7efd17400700 1 -- 172.21.15.67:0/1447662812 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd180139e0 con 0x56031ac857a0 2024-03-30T02:43:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.697+0000 7efd44d81b80 1 -- 172.21.15.67:0/1447662812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac857a0 msgr2=0x56031ac85b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.697+0000 7efd44d81b80 1 --2- 172.21.15.67:0/1447662812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac857a0 0x56031ac85b80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efd18009c70 tx=0x7efd18013090 comp rx=0 tx=0).stop 2024-03-30T02:43:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.698+0000 7efd44d81b80 1 -- 172.21.15.67:0/1447662812 shutdown_connections 2024-03-30T02:43:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.698+0000 7efd44d81b80 1 --2- 172.21.15.67:0/1447662812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac857a0 0x56031ac85b80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.698+0000 7efd44d81b80 1 --2- 172.21.15.67:0/1447662812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac933b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.698+0000 7efd44d81b80 1 --2- 172.21.15.67:0/1447662812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac90a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.698+0000 7efd44d81b80 1 -- 172.21.15.67:0/1447662812 >> 172.21.15.67:0/1447662812 conn(0x56031abe56a0 msgr2=0x56031ac81c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.698+0000 7efd44d81b80 1 -- 172.21.15.67:0/1447662812 shutdown_connections 2024-03-30T02:43:32.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.698+0000 7efd44d81b80 1 -- 172.21.15.67:0/1447662812 wait complete. 2024-03-30T02:43:32.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.699+0000 7efd44d81b80 1 -- start start 2024-03-30T02:43:32.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.699+0000 7efd44d81b80 1 Processor -- start 2024-03-30T02:43:32.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.700+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac8ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.700+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac8e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.700+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031acadd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.701+0000 7efd29e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac8ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.701+0000 7efd29e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac8ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:42892/0 (socket says 172.21.15.67:42892) 2024-03-30T02:43:32.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.701+0000 7efd29400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac8e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.701+0000 7efd44d81b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031ab73d80 con 0x56031ac8e870 2024-03-30T02:43:32.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.701+0000 7efd44d81b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031ab73a20 con 0x56031ac90fc0 2024-03-30T02:43:32.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.701+0000 7efd44d81b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031aa5b6a0 con 0x56031ac860c0 2024-03-30T02:43:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.701+0000 7efd29e00700 1 -- 172.21.15.67:0/2984436005 learned_addr learned my addr 172.21.15.67:0/2984436005 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.701+0000 7efd44d81b80 10 client.0 _ll_register_callbacks cb 0x56031ab7d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.702+0000 7efd2a800700 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031acadd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.702+0000 7efd29400700 1 -- 172.21.15.67:0/2984436005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 msgr2=0x56031ac8ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.702+0000 7efd29400700 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac8ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.702+0000 7efd29400700 1 -- 172.21.15.67:0/2984436005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 msgr2=0x56031acadd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.703+0000 7efd29400700 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031acadd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.703+0000 7efd29400700 1 -- 172.21.15.67:0/2984436005 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd240a56d0 con 0x56031ac90fc0 2024-03-30T02:43:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.703+0000 7efd29e00700 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac8ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:32.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.703+0000 7efd2a800700 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031acadd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.705+0000 7efd29400700 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac8e330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efd2c1351f0 tx=0x7efd2c104bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dbbe22a1d0c3479 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:32.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.705+0000 7efd0be00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:43:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.705+0000 7efd0be00700 1 -- 172.21.15.67:0/2984436005 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c135670 con 0x56031ac90fc0 2024-03-30T02:43:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.705+0000 7efd0be00700 1 -- 172.21.15.67:0/2984436005 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7efd2c104e10 con 0x56031ac90fc0 2024-03-30T02:43:32.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.706+0000 7efd44d81b80 1 -- 172.21.15.67:0/2984436005 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56031ab98380 con 0x56031ac90fc0 2024-03-30T02:43:32.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.709+0000 7efd0be00700 1 -- 172.21.15.67:0/2984436005 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd2c104fd0 con 0x56031ac90fc0 2024-03-30T02:43:32.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.709+0000 7efd44d81b80 10 client.26912 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:32.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.709+0000 7efd44d81b80 1 -- 172.21.15.67:0/2984436005 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56031aa05b60 con 0x56031ac90fc0 2024-03-30T02:43:32.710 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore_snbep.o 2024-03-30T02:43:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.709+0000 7efd0be00700 1 -- 172.21.15.67:0/2984436005 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd2c121070 con 0x56031ac90fc0 2024-03-30T02:43:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.710+0000 7efd0be00700 1 -- 172.21.15.67:0/2984436005 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7efd2c105bc0 con 0x56031ac90fc0 2024-03-30T02:43:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.711+0000 7efd44d81b80 10 client.26912 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.711+0000 7efd44d81b80 10 client.26912 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:43:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.711+0000 7efd44d81b80 20 client.26912 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.711+0000 7efd44d81b80 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56031acb20a0 0x56031acb44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.711+0000 7efd44d81b80 4 client.26912 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:43:32.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.711+0000 7efd44d81b80 1 -- 172.21.15.67:0/2984436005 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56031abd0700 con 0x56031acb20a0 2024-03-30T02:43:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.711+0000 7efd29e00700 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56031acb20a0 0x56031acb44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.712+0000 7efd29e00700 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56031acb20a0 0x56031acb44c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6e3eed21118a27de server_cookie=8607985c212b739e in_seq=0 out_seq=0 2024-03-30T02:43:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.712+0000 7efd0be00700 10 client.26912 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:43:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd0be00700 1 -- 172.21.15.67:0/2984436005 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56031abd0700 con 0x56031acb20a0 2024-03-30T02:43:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd0be00700 10 client.26912 handle_command_reply: tid=0 2024-03-30T02:43:32.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd44d81b80 1 client.26912 shutdown 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd44d81b80 1 -- 172.21.15.67:0/2984436005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 msgr2=0x56031ac8e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd44d81b80 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac8e330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efd2c1351f0 tx=0x7efd2c104bb0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd44d81b80 1 -- 172.21.15.67:0/2984436005 shutdown_connections 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd44d81b80 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031acadd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd44d81b80 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56031acb20a0 0x56031acb44c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd44d81b80 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031ac8ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd44d81b80 1 --2- 172.21.15.67:0/2984436005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac8e330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd0be00700 0 client.26912 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.713+0000 7efd44d81b80 1 -- 172.21.15.67:0/2984436005 >> 172.21.15.67:0/2984436005 conn(0x56031abe56a0 msgr2=0x56031ac87890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.714+0000 7efd44d81b80 1 -- 172.21.15.67:0/2984436005 shutdown_connections 2024-03-30T02:43:32.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.714+0000 7efd44d81b80 1 -- 172.21.15.67:0/2984436005 wait complete. 2024-03-30T02:43:32.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.714+0000 7efd44d81b80 20 client.26912 trim_cache size 0 max 16384 2024-03-30T02:43:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:32 smithi082 ceph-mon[79176]: pgmap v197: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.1 MiB/s wr, 276 op/s 2024-03-30T02:43:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:32 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1289249114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:43:32.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.749+0000 7efd44d81b80 1 Processor -- start 2024-03-30T02:43:32.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.749+0000 7efd44d81b80 1 -- start start 2024-03-30T02:43:32.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.750+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac860c0 0x56031aca8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.750+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac8e870 0x56031ac88780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.750+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac81700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.750+0000 7efd44d81b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031ac81520 con 0x56031ac860c0 2024-03-30T02:43:32.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.750+0000 7efd2a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac81700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.750+0000 7efd44d81b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031ac872e0 con 0x56031ac90fc0 2024-03-30T02:43:32.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.750+0000 7efd2a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac81700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52316/0 (socket says 172.21.15.67:52316) 2024-03-30T02:43:32.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.750+0000 7efd2a800700 1 -- 172.21.15.67:0/903354941 learned_addr learned my addr 172.21.15.67:0/903354941 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:32.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.750+0000 7efd44d81b80 1 -- 172.21.15.67:0/903354941 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031aca91c0 con 0x56031ac8e870 2024-03-30T02:43:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.752+0000 7efd29400700 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac8e870 0x56031ac88780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.755 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/seqiv.o 2024-03-30T02:43:32.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.753+0000 7efd29e00700 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac860c0 0x56031aca8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.753+0000 7efd29400700 1 -- 172.21.15.67:0/903354941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 msgr2=0x56031ac81700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:32.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.758+0000 7efd29400700 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac81700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.758+0000 7efd29400700 1 -- 172.21.15.67:0/903354941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac860c0 msgr2=0x56031aca8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:32.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.758+0000 7efd29400700 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac860c0 0x56031aca8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.758+0000 7efd29400700 1 -- 172.21.15.67:0/903354941 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56031ab9dea0 con 0x56031ac8e870 2024-03-30T02:43:32.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.758+0000 7efd29e00700 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac860c0 0x56031aca8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:43:32.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.759+0000 7efd29400700 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac8e870 0x56031ac88780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd2c064450 tx=0x7efd2c103160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e76f6d7e6b412199 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:32.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.759+0000 7efd17e00700 1 -- 172.21.15.67:0/903354941 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c100ef0 con 0x56031ac8e870 2024-03-30T02:43:32.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.759+0000 7efd17e00700 1 -- 172.21.15.67:0/903354941 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd2c121070 con 0x56031ac8e870 2024-03-30T02:43:32.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.759+0000 7efd17e00700 1 -- 172.21.15.67:0/903354941 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c1040a0 con 0x56031ac8e870 2024-03-30T02:43:32.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.760+0000 7efd44d81b80 1 -- 172.21.15.67:0/903354941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac8e870 msgr2=0x56031ac88780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.760+0000 7efd44d81b80 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac8e870 0x56031ac88780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd2c064450 tx=0x7efd2c103160 comp rx=0 tx=0).stop 2024-03-30T02:43:32.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.764+0000 7efd44d81b80 1 -- 172.21.15.67:0/903354941 shutdown_connections 2024-03-30T02:43:32.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.764+0000 7efd44d81b80 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac860c0 0x56031aca8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.764+0000 7efd44d81b80 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031ac81700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.764+0000 7efd44d81b80 1 --2- 172.21.15.67:0/903354941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac8e870 0x56031ac88780 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.764+0000 7efd44d81b80 1 -- 172.21.15.67:0/903354941 >> 172.21.15.67:0/903354941 conn(0x56031abe56a0 msgr2=0x56031ac7a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:32.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.765+0000 7efd44d81b80 1 -- 172.21.15.67:0/903354941 shutdown_connections 2024-03-30T02:43:32.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.765+0000 7efd44d81b80 1 -- 172.21.15.67:0/903354941 wait complete. 2024-03-30T02:43:32.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.765+0000 7efd44d81b80 1 -- start start 2024-03-30T02:43:32.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.766+0000 7efd44d81b80 1 Processor -- start 2024-03-30T02:43:32.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.766+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031aca6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.767+0000 7efd44d81b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031aca73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.767+0000 7efd29e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031aca6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.767+0000 7efd29e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031aca6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:42914/0 (socket says 172.21.15.67:42914) 2024-03-30T02:43:32.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.768+0000 7efd29400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031aca73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.768+0000 7efd29400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031aca73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60874/0 (socket says 172.21.15.67:60874) 2024-03-30T02:43:32.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.768+0000 7efd29e00700 1 -- 172.21.15.67:0/3554873420 learned_addr learned my addr 172.21.15.67:0/3554873420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:32.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.768+0000 7efd44d81b80 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031acfee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.768+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031acff3a0 con 0x56031ac8e870 2024-03-30T02:43:32.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.768+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031acff510 con 0x56031ac90fc0 2024-03-30T02:43:32.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.768+0000 7efd2a800700 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031acfee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.768+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56031acff6d0 con 0x56031ac860c0 2024-03-30T02:43:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.769+0000 7efd29e00700 1 -- 172.21.15.67:0/3554873420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 msgr2=0x56031acfee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.769+0000 7efd29e00700 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031acfee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.769+0000 7efd29e00700 1 -- 172.21.15.67:0/3554873420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 msgr2=0x56031aca73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.770+0000 7efd29e00700 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031aca73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.770+0000 7efd29e00700 1 -- 172.21.15.67:0/3554873420 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56031aa593b0 con 0x56031ac860c0 2024-03-30T02:43:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.770+0000 7efd29400700 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031aca73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:43:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.770+0000 7efd29e00700 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031aca6e70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd180094e0 tx=0x7efd180058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f897d0428b780bda server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:32.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.771+0000 7efd0be00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:43:32.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.771+0000 7efd0be00700 1 -- 172.21.15.67:0/3554873420 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1802b040 con 0x56031ac860c0 2024-03-30T02:43:32.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.771+0000 7efd0be00700 1 -- 172.21.15.67:0/3554873420 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7efd18013750 con 0x56031ac860c0 2024-03-30T02:43:32.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.771+0000 7efd0be00700 10 client.0 inactive, discarding osd_map(88..88 src has 1..88) v4 2024-03-30T02:43:32.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.771+0000 7efd44d81b80 10 client.0 _ll_register_callbacks cb 0x56031ab7d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:32.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.772+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd18003880 con 0x56031ac860c0 2024-03-30T02:43:32.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.772+0000 7efd0be00700 1 -- 172.21.15.67:0/3554873420 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd1802f730 con 0x56031ac860c0 2024-03-30T02:43:32.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.772+0000 7efd44d81b80 10 client.26866 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:32.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.772+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=36}) v3 -- 0x56031aa05b60 con 0x56031ac860c0 2024-03-30T02:43:32.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.773+0000 7efd0be00700 1 -- 172.21.15.67:0/3554873420 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7efd18037070 con 0x56031ac860c0 2024-03-30T02:43:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.773+0000 7efd44d81b80 10 client.26866 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.773+0000 7efd44d81b80 10 client.26866 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:43:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.773+0000 7efd44d81b80 20 client.26866 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.773+0000 7efd44d81b80 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56031ad066e0 0x56031ad08ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.774+0000 7efd44d81b80 4 client.26866 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:43:32.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.774+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7efd18003880 con 0x56031ad066e0 2024-03-30T02:43:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.774+0000 7efd29400700 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56031ad066e0 0x56031ad08ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.775+0000 7efd29400700 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56031ad066e0 0x56031ad08ba0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e614bbda9e22dacd server_cookie=585ad91287589813 in_seq=0 out_seq=0 2024-03-30T02:43:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.775+0000 7efd0be00700 10 client.26866 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:43:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.775+0000 7efd0be00700 1 -- 172.21.15.67:0/3554873420 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7efd18003880 con 0x56031ad066e0 2024-03-30T02:43:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.775+0000 7efd0be00700 10 client.26866 handle_command_reply: tid=0 2024-03-30T02:43:32.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.775+0000 7efd44d81b80 1 client.26866 shutdown 2024-03-30T02:43:32.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.781+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 msgr2=0x56031aca6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.781+0000 7efd44d81b80 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031aca6e70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd180094e0 tx=0x7efd180058e0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.785+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 shutdown_connections 2024-03-30T02:43:32.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.785+0000 7efd44d81b80 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x56031ad066e0 0x56031ad08ba0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.785+0000 7efd44d81b80 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56031ac90fc0 0x56031acfee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.785+0000 7efd44d81b80 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56031ac860c0 0x56031aca6e70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.785+0000 7efd44d81b80 1 --2- 172.21.15.67:0/3554873420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56031ac8e870 0x56031aca73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.785+0000 7efd0be00700 0 client.26866 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:43:32.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.785+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 >> 172.21.15.67:0/3554873420 conn(0x56031abe56a0 msgr2=0x56031ac8dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:32.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.785+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 shutdown_connections 2024-03-30T02:43:32.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.785+0000 7efd44d81b80 1 -- 172.21.15.67:0/3554873420 wait complete. 2024-03-30T02:43:32.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.786+0000 7efd44d81b80 20 client.26866 trim_cache size 0 max 16384 2024-03-30T02:43:32.789 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:43:32.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 -- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efd100a1390 msgr2=0x7efd100a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efd100a1390 0x7efd100a3850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd1c005fd0 tx=0x7efd1c005960 comp rx=0 tx=0).stop 2024-03-30T02:43:32.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 -- 172.21.15.67:0/213373059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd2409d2e0 msgr2=0x7efd2412ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:32.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd2409d2e0 0x7efd2412ad50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efd2c04f8d0 tx=0x7efd2c079f90 comp rx=0 tx=0).stop 2024-03-30T02:43:32.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 -- 172.21.15.67:0/213373059 shutdown_connections 2024-03-30T02:43:32.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efd100a1390 0x7efd100a3850 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efd2412e8d0 0x7efd2412ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd240981c0 0x7efd2412a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 --2- 172.21.15.67:0/213373059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd2409d2e0 0x7efd2412ad50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:32.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 -- 172.21.15.67:0/213373059 >> 172.21.15.67:0/213373059 conn(0x7efd24093a60 msgr2=0x7efd2409bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:32.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.796+0000 7efd2b200700 1 -- 172.21.15.67:0/213373059 shutdown_connections 2024-03-30T02:43:32.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:32.797+0000 7efd2b200700 1 -- 172.21.15.67:0/213373059 wait complete. 2024-03-30T02:43:32.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:32 smithi067 ceph-mon[37808]: pgmap v197: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.1 MiB/s wr, 276 op/s 2024-03-30T02:43:32.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:32 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1289249114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:43:32.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:43:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:32 smithi170 ceph-mon[78798]: pgmap v197: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.1 MiB/s wr, 276 op/s 2024-03-30T02:43:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:32 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1289249114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:43:33.111 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/cpu/mce/built-in.a 2024-03-30T02:43:33.199 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mtrr/mtrr.o 2024-03-30T02:43:33.210 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/user.o 2024-03-30T02:43:33.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.366+0000 7fc119800700 1 -- 172.21.15.67:0/2508311761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc10c0a59c0 msgr2=0x7fc10c0aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:33.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.366+0000 7fc119800700 1 --2- 172.21.15.67:0/2508311761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc10c0a59c0 0x7fc10c0aba20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc100009990 tx=0x7fc100009ca0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.369+0000 7fc119800700 1 -- 172.21.15.67:0/2508311761 shutdown_connections 2024-03-30T02:43:33.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.369+0000 7fc119800700 1 --2- 172.21.15.67:0/2508311761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc10c0a5010 0x7fc10c0a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.369+0000 7fc119800700 1 --2- 172.21.15.67:0/2508311761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10c0ac150 0x7fc10c0b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.369+0000 7fc119800700 1 --2- 172.21.15.67:0/2508311761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc10c0a59c0 0x7fc10c0aba20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.369+0000 7fc119800700 1 -- 172.21.15.67:0/2508311761 >> 172.21.15.67:0/2508311761 conn(0x7fc10c01a690 msgr2=0x7fc10c01aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:33.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.370+0000 7fc119800700 1 -- 172.21.15.67:0/2508311761 shutdown_connections 2024-03-30T02:43:33.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.370+0000 7fc119800700 1 -- 172.21.15.67:0/2508311761 wait complete. 2024-03-30T02:43:33.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.371+0000 7fc119800700 1 Processor -- start 2024-03-30T02:43:33.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.372+0000 7fc119800700 1 -- start start 2024-03-30T02:43:33.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.373+0000 7fc119800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc10c0a5010 0x7fc10c149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.373+0000 7fc119800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc10c0ac150 0x7fc10c149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.373+0000 7fc119800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10c0b2cf0 0x7fc10c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.374+0000 7fc119800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0b3770 con 0x7fc10c0ac150 2024-03-30T02:43:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.374+0000 7fc119800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0b38e0 con 0x7fc10c0b2cf0 2024-03-30T02:43:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.374+0000 7fc119800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10c0b3aa0 con 0x7fc10c0a5010 2024-03-30T02:43:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.374+0000 7fc113400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10c0b2cf0 0x7fc10c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.374+0000 7fc113400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10c0b2cf0 0x7fc10c0b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52344/0 (socket says 172.21.15.67:52344) 2024-03-30T02:43:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.374+0000 7fc113400700 1 -- 172.21.15.67:0/1674914761 learned_addr learned my addr 172.21.15.67:0/1674914761 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.374+0000 7fc112a00700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc10c0a5010 0x7fc10c149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.375+0000 7fc113400700 1 -- 172.21.15.67:0/1674914761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc10c0a5010 msgr2=0x7fc10c149550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:33.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.375+0000 7fc113400700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc10c0a5010 0x7fc10c149550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.375+0000 7fc113400700 1 -- 172.21.15.67:0/1674914761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc10c0ac150 msgr2=0x7fc10c149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:33.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.375+0000 7fc113400700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc10c0ac150 0x7fc10c149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.375+0000 7fc113400700 1 -- 172.21.15.67:0/1674914761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1000095f0 con 0x7fc10c0b2cf0 2024-03-30T02:43:33.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.375+0000 7fc112a00700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc10c0a5010 0x7fc10c149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:33.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.376+0000 7fc113400700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10c0b2cf0 0x7fc10c0b31a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc1140642a0 tx=0x7fc11406fac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:33.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.379+0000 7fc10b400700 1 -- 172.21.15.67:0/1674914761 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11406a040 con 0x7fc10c0b2cf0 2024-03-30T02:43:33.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.379+0000 7fc10b400700 1 -- 172.21.15.67:0/1674914761 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc11406de10 con 0x7fc10c0b2cf0 2024-03-30T02:43:33.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.379+0000 7fc10b400700 1 -- 172.21.15.67:0/1674914761 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114079e60 con 0x7fc10c0b2cf0 2024-03-30T02:43:33.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.379+0000 7fc119800700 1 -- 172.21.15.67:0/1674914761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c150420 con 0x7fc10c0b2cf0 2024-03-30T02:43:33.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.379+0000 7fc119800700 1 -- 172.21.15.67:0/1674914761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c1508b0 con 0x7fc10c0b2cf0 2024-03-30T02:43:33.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.386+0000 7fc12d337b80 1 Processor -- start 2024-03-30T02:43:33.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.387+0000 7fc10b400700 1 -- 172.21.15.67:0/1674914761 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc1140795c0 con 0x7fc10c0b2cf0 2024-03-30T02:43:33.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.391+0000 7fc10b400700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0f80a1310 0x7fc0f80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.391+0000 7fc12d337b80 1 -- start start 2024-03-30T02:43:33.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc10b400700 1 -- 172.21.15.67:0/1674914761 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fc11412f6c0 con 0x7fc10c0b2cf0 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703067a0 0x556a70306b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70311a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc112000700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0f80a1310 0x7fc0f80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a703143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc12d337b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7025f990 con 0x556a703070c0 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc12d337b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7025d350 con 0x556a703067a0 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc12d337b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a70258c40 con 0x556a70311fc0 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc113400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a703143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc113400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a703143b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:42938/0 (socket says 172.21.15.67:42938) 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.392+0000 7fc113400700 1 -- 172.21.15.67:0/430316356 learned_addr learned my addr 172.21.15.67:0/430316356 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.393+0000 7fc112000700 1 --2- 172.21.15.67:0/430316356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70311a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.393+0000 7fc113400700 1 -- 172.21.15.67:0/430316356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703067a0 msgr2=0x556a70306b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:43:33.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.393+0000 7fc113400700 1 --2- 172.21.15.67:0/430316356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703067a0 0x556a70306b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.393+0000 7fc113400700 1 -- 172.21.15.67:0/430316356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 msgr2=0x556a70311a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.393+0000 7fc113400700 1 --2- 172.21.15.67:0/430316356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70311a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.393+0000 7fc113400700 1 -- 172.21.15.67:0/430316356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a700da3b0 con 0x556a70311fc0 2024-03-30T02:43:33.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.393+0000 7fc112000700 1 --2- 172.21.15.67:0/430316356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70311a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:33.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.393+0000 7fc113400700 1 --2- 172.21.15.67:0/430316356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a703143b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc114104370 tx=0x7fc1140fc270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=332b3bbb6a10ee7c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:33.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.394+0000 7fc0ffe00700 1 -- 172.21.15.67:0/430316356 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140e70b0 con 0x556a70311fc0 2024-03-30T02:43:33.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.394+0000 7fc0ffe00700 1 -- 172.21.15.67:0/430316356 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1140fcb20 con 0x556a70311fc0 2024-03-30T02:43:33.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.394+0000 7fc0ffe00700 1 -- 172.21.15.67:0/430316356 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140fc680 con 0x556a70311fc0 2024-03-30T02:43:33.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.397+0000 7fc112000700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0f80a1310 0x7fc0f80a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc100005f50 tx=0x7fc100005fb0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:33.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.403+0000 7fc12d337b80 1 -- 172.21.15.67:0/430316356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 msgr2=0x556a703143b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:33.406 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/cmdline.o 2024-03-30T02:43:33.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.403+0000 7fc12d337b80 1 --2- 172.21.15.67:0/430316356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a703143b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc114104370 tx=0x7fc1140fc270 comp rx=0 tx=0).stop 2024-03-30T02:43:33.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.404+0000 7fc12d337b80 1 -- 172.21.15.67:0/430316356 shutdown_connections 2024-03-30T02:43:33.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.404+0000 7fc12d337b80 1 --2- 172.21.15.67:0/430316356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a703143b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.404+0000 7fc12d337b80 1 --2- 172.21.15.67:0/430316356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703067a0 0x556a70306b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.404+0000 7fc12d337b80 1 --2- 172.21.15.67:0/430316356 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70311a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.404+0000 7fc12d337b80 1 -- 172.21.15.67:0/430316356 >> 172.21.15.67:0/430316356 conn(0x556a702666a0 msgr2=0x556a70302c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:33.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.407+0000 7fc12d337b80 1 -- 172.21.15.67:0/430316356 shutdown_connections 2024-03-30T02:43:33.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.408+0000 7fc12d337b80 1 -- 172.21.15.67:0/430316356 wait complete. 2024-03-30T02:43:33.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.408+0000 7fc12d337b80 1 -- start start 2024-03-30T02:43:33.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.409+0000 7fc12d337b80 1 Processor -- start 2024-03-30T02:43:33.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.409+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a703119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.410+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a7030ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.411+0000 7fc112000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a7030ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.411+0000 7fc112000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a7030ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60894/0 (socket says 172.21.15.67:60894) 2024-03-30T02:43:33.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.411+0000 7fc112a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a703119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.411+0000 7fc112000700 1 -- 172.21.15.67:0/2764972573 learned_addr learned my addr 172.21.15.67:0/2764972573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:33.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.411+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 0x556a7030f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.414+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a701f4d80 con 0x556a703070c0 2024-03-30T02:43:33.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.414+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a701f4a20 con 0x556a70311fc0 2024-03-30T02:43:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.414+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a700dc6a0 con 0x556a703067a0 2024-03-30T02:43:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.414+0000 7fc12d337b80 10 client.0 _ll_register_callbacks cb 0x556a701fe470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.414+0000 7fc112000700 1 -- 172.21.15.67:0/2764972573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 msgr2=0x556a703119b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.414+0000 7fc112000700 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a703119b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.415+0000 7fc112000700 1 -- 172.21.15.67:0/2764972573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 msgr2=0x556a7030f240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:43:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.415+0000 7fc112000700 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 0x556a7030f240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.415+0000 7fc112000700 1 -- 172.21.15.67:0/2764972573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10c1508b0 con 0x556a703070c0 2024-03-30T02:43:33.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.415+0000 7fc112000700 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a7030ec30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc100015000 tx=0x7fc100009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d9543f3eefe15a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:33.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.416+0000 7fc0fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:43:33.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.417+0000 7fc0fcc00700 1 -- 172.21.15.67:0/2764972573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100042670 con 0x556a703070c0 2024-03-30T02:43:33.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.417+0000 7fc0fcc00700 1 -- 172.21.15.67:0/2764972573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fc1000056b0 con 0x556a703070c0 2024-03-30T02:43:33.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.417+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556a70219380 con 0x556a703070c0 2024-03-30T02:43:33.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.417+0000 7fc0fcc00700 1 -- 172.21.15.67:0/2764972573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1000041f0 con 0x556a703070c0 2024-03-30T02:43:33.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.417+0000 7fc12d337b80 10 client.16830 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:33.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.417+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556a70086b60 con 0x556a703070c0 2024-03-30T02:43:33.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.425+0000 7fc0fcc00700 1 -- 172.21.15.67:0/2764972573 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc10004a080 con 0x556a703070c0 2024-03-30T02:43:33.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.425+0000 7fc0fcc00700 1 -- 172.21.15.67:0/2764972573 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fc100039890 con 0x556a703070c0 2024-03-30T02:43:33.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.426+0000 7fc12d337b80 10 client.16830 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:33.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.426+0000 7fc12d337b80 10 client.16830 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:43:33.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.426+0000 7fc12d337b80 20 client.16830 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:33.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.426+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x556a70321bd0 0x556a70323ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.426+0000 7fc12d337b80 4 client.16830 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:43:33.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.426+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a70251700 con 0x556a70321bd0 2024-03-30T02:43:33.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.427+0000 7fc112a00700 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x556a70321bd0 0x556a70323ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.429+0000 7fc0fcc00700 10 client.16830 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:43:33.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.429+0000 7fc112a00700 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x556a70321bd0 0x556a70323ff0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=41476466e704c2af server_cookie=a0aaaee748d975c4 in_seq=0 out_seq=0 2024-03-30T02:43:33.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.429+0000 7fc0fcc00700 1 -- 172.21.15.67:0/2764972573 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556a70251700 con 0x556a70321bd0 2024-03-30T02:43:33.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.429+0000 7fc0fcc00700 10 client.16830 handle_command_reply: tid=0 2024-03-30T02:43:33.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.429+0000 7fc12d337b80 1 client.16830 shutdown 2024-03-30T02:43:33.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.430+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 msgr2=0x556a7030ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:33.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.430+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a7030ec30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc100015000 tx=0x7fc100009310 comp rx=0 tx=0).stop 2024-03-30T02:43:33.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.430+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 shutdown_connections 2024-03-30T02:43:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.431+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x556a70321bd0 0x556a70323ff0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.431+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 0x556a7030f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.431+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a703119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.431+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2764972573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a7030ec30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.431+0000 7fc0fcc00700 0 client.16830 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:43:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.431+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 >> 172.21.15.67:0/2764972573 conn(0x556a702666a0 msgr2=0x556a70308890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.431+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 shutdown_connections 2024-03-30T02:43:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.431+0000 7fc12d337b80 1 -- 172.21.15.67:0/2764972573 wait complete. 2024-03-30T02:43:33.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.432+0000 7fc12d337b80 20 client.16830 trim_cache size 0 max 16384 2024-03-30T02:43:33.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.460+0000 7fc12d337b80 1 Processor -- start 2024-03-30T02:43:33.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.460+0000 7fc12d337b80 1 -- start start 2024-03-30T02:43:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.461+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703067a0 0x556a70318c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.461+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703070c0 0x556a702fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.461+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a702fb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.462+0000 7fc112a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703067a0 0x556a70318c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.462+0000 7fc112a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703067a0 0x556a70318c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60898/0 (socket says 172.21.15.67:60898) 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.462+0000 7fc112000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703070c0 0x556a702fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.462+0000 7fc112000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703070c0 0x556a702fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52362/0 (socket says 172.21.15.67:52362) 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.462+0000 7fc112a00700 1 -- 172.21.15.67:0/2376316168 learned_addr learned my addr 172.21.15.67:0/2376316168 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.462+0000 7fc12d337b80 1 -- 172.21.15.67:0/2376316168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a70319180 con 0x556a703067a0 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.462+0000 7fc12d337b80 1 -- 172.21.15.67:0/2376316168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a7030d180 con 0x556a703070c0 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.462+0000 7fc12d337b80 1 -- 172.21.15.67:0/2376316168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a702fc550 con 0x556a70311fc0 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.464+0000 7fc113400700 1 --2- 172.21.15.67:0/2376316168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a702fb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.466+0000 7fc112a00700 1 -- 172.21.15.67:0/2376316168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 msgr2=0x556a702fb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:33.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.466+0000 7fc112a00700 1 --2- 172.21.15.67:0/2376316168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a702fb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.466+0000 7fc112a00700 1 -- 172.21.15.67:0/2376316168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703070c0 msgr2=0x556a702fc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.466+0000 7fc112a00700 1 --2- 172.21.15.67:0/2376316168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703070c0 0x556a702fc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.466+0000 7fc112a00700 1 -- 172.21.15.67:0/2376316168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a7021eea0 con 0x556a703067a0 2024-03-30T02:43:33.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.467+0000 7fc112a00700 1 --2- 172.21.15.67:0/2376316168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703067a0 0x556a70318c40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x556a703248d0 tx=0x7fc1040058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22058b862c008cee server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:33.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.476+0000 7fc108c00700 1 -- 172.21.15.67:0/2376316168 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104017040 con 0x556a703067a0 2024-03-30T02:43:33.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.476+0000 7fc108c00700 1 -- 172.21.15.67:0/2376316168 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1040044f0 con 0x556a703067a0 2024-03-30T02:43:33.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.476+0000 7fc108c00700 1 -- 172.21.15.67:0/2376316168 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040126b0 con 0x556a703067a0 2024-03-30T02:43:33.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.477+0000 7fc12d337b80 1 -- 172.21.15.67:0/2376316168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703067a0 msgr2=0x556a70318c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:33.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.477+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2376316168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703067a0 0x556a70318c40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x556a703248d0 tx=0x7fc1040058e0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.478+0000 7fc12d337b80 1 -- 172.21.15.67:0/2376316168 shutdown_connections 2024-03-30T02:43:33.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.478+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2376316168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703067a0 0x556a70318c40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.478+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2376316168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a70311fc0 0x556a702fb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.478+0000 7fc12d337b80 1 --2- 172.21.15.67:0/2376316168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a703070c0 0x556a702fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.478+0000 7fc12d337b80 1 -- 172.21.15.67:0/2376316168 >> 172.21.15.67:0/2376316168 conn(0x556a702666a0 msgr2=0x556a70328950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:33.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.483+0000 7fc12d337b80 1 -- 172.21.15.67:0/2376316168 shutdown_connections 2024-03-30T02:43:33.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.483+0000 7fc12d337b80 1 -- 172.21.15.67:0/2376316168 wait complete. 2024-03-30T02:43:33.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.484+0000 7fc12d337b80 1 -- start start 2024-03-30T02:43:33.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.484+0000 7fc12d337b80 1 Processor -- start 2024-03-30T02:43:33.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.485+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a70318750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.485+0000 7fc12d337b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70316b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.486+0000 7fc112a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a70318750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.486+0000 7fc112a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a70318750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:42976/0 (socket says 172.21.15.67:42976) 2024-03-30T02:43:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.487+0000 7fc112a00700 1 -- 172.21.15.67:0/3684136675 learned_addr learned my addr 172.21.15.67:0/3684136675 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.487+0000 7fc112000700 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70316b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.487+0000 7fc12d337b80 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 0x556a703170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.487+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a70319030 con 0x556a703070c0 2024-03-30T02:43:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.487+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a70317610 con 0x556a70311fc0 2024-03-30T02:43:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.487+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a703177d0 con 0x556a703067a0 2024-03-30T02:43:33.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.487+0000 7fc12d337b80 10 client.0 _ll_register_callbacks cb 0x556a701fe470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.488+0000 7fc113400700 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 0x556a703170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.488+0000 7fc112000700 1 -- 172.21.15.67:0/3684136675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 msgr2=0x556a70318750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.488+0000 7fc112000700 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a70318750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.488+0000 7fc112000700 1 -- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 msgr2=0x556a703170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.488+0000 7fc112000700 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 0x556a703170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.488+0000 7fc112000700 1 -- 172.21.15.67:0/3684136675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10400bcf0 con 0x556a703070c0 2024-03-30T02:43:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.488+0000 7fc113400700 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 0x556a703170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.489+0000 7fc112000700 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70316b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x556a70310170 tx=0x7fc100039ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677a3deff70969c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.489+0000 7fc0fcc00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:43:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.489+0000 7fc112a00700 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a70318750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.489+0000 7fc0fcc00700 1 -- 172.21.15.67:0/3684136675 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1000278d0 con 0x556a703070c0 2024-03-30T02:43:33.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.489+0000 7fc0fcc00700 1 -- 172.21.15.67:0/3684136675 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fc100027a30 con 0x556a703070c0 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.490+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1140fcb20 con 0x556a703070c0 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.490+0000 7fc0fcc00700 1 -- 172.21.15.67:0/3684136675 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1000041f0 con 0x556a703070c0 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.490+0000 7fc12d337b80 10 client.16836 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.490+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556a700decd0 con 0x556a703070c0 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.491+0000 7fc0fcc00700 1 -- 172.21.15.67:0/3684136675 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1140fcb20 con 0x556a703070c0 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.491+0000 7fc0fcc00700 1 -- 172.21.15.67:0/3684136675 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fc100042b30 con 0x556a703070c0 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.491+0000 7fc12d337b80 10 client.16836 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.491+0000 7fc12d337b80 10 client.16836 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.491+0000 7fc12d337b80 20 client.16836 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.491+0000 7fc12d337b80 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x556a70376980 0x556a70378d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:33.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.491+0000 7fc12d337b80 4 client.16836 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:43:33.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.491+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556a700a7cf0 con 0x556a70376980 2024-03-30T02:43:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.493+0000 7fc112a00700 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x556a70376980 0x556a70378d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.494+0000 7fc112a00700 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x556a70376980 0x556a70378d70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=87a7f3eea861e522 server_cookie=c197b95999b7b770 in_seq=0 out_seq=0 2024-03-30T02:43:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.494+0000 7fc0fcc00700 10 client.16836 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:43:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.494+0000 7fc0fcc00700 1 -- 172.21.15.67:0/3684136675 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556a700a7cf0 con 0x556a70376980 2024-03-30T02:43:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.494+0000 7fc0fcc00700 10 client.16836 handle_command_reply: tid=0 2024-03-30T02:43:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.494+0000 7fc12d337b80 1 client.16836 shutdown 2024-03-30T02:43:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.494+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 msgr2=0x556a70316b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.494+0000 7fc12d337b80 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70316b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x556a70310170 tx=0x7fc100039ad0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.495+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 shutdown_connections 2024-03-30T02:43:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.495+0000 7fc12d337b80 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x556a70376980 0x556a70378d70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.495+0000 7fc12d337b80 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556a70311fc0 0x556a703170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.495+0000 7fc12d337b80 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a703067a0 0x556a70318750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.495+0000 7fc12d337b80 1 --2- 172.21.15.67:0/3684136675 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556a703070c0 0x556a70316b90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.495+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 >> 172.21.15.67:0/3684136675 conn(0x556a702666a0 msgr2=0x556a7031d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.495+0000 7fc0fcc00700 0 client.16836 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:43:33.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.496+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 shutdown_connections 2024-03-30T02:43:33.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.496+0000 7fc12d337b80 1 -- 172.21.15.67:0/3684136675 wait complete. 2024-03-30T02:43:33.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.496+0000 7fc12d337b80 20 client.16836 trim_cache size 0 max 16384 2024-03-30T02:43:33.498 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.499+0000 7fc119800700 1 -- 172.21.15.67:0/1674914761 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0f80a1310 msgr2=0x7fc0f80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.499+0000 7fc119800700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0f80a1310 0x7fc0f80a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc100005f50 tx=0x7fc100005fb0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.499+0000 7fc119800700 1 -- 172.21.15.67:0/1674914761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10c0b2cf0 msgr2=0x7fc10c0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.499+0000 7fc119800700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10c0b2cf0 0x7fc10c0b31a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc1140642a0 tx=0x7fc11406fac0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.500+0000 7fc119800700 1 -- 172.21.15.67:0/1674914761 shutdown_connections 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.500+0000 7fc119800700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc0f80a1310 0x7fc0f80a37d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.500+0000 7fc119800700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc10c0b2cf0 0x7fc10c0b31a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.500+0000 7fc119800700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc10c0a5010 0x7fc10c149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.500+0000 7fc119800700 1 --2- 172.21.15.67:0/1674914761 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc10c0ac150 0x7fc10c149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:33.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.500+0000 7fc119800700 1 -- 172.21.15.67:0/1674914761 >> 172.21.15.67:0/1674914761 conn(0x7fc10c01a690 msgr2=0x7fc10c0a2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:33.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.505+0000 7fc119800700 1 -- 172.21.15.67:0/1674914761 shutdown_connections 2024-03-30T02:43:33.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:33.505+0000 7fc119800700 1 -- 172.21.15.67:0/1674914761 wait complete. 2024-03-30T02:43:33.527 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/echainiv.o 2024-03-30T02:43:33.530 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/mmap.o 2024-03-30T02:43:33.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:43:33.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:33 smithi082 ceph-mon[79176]: pgmap v198: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-03-30T02:43:33.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:33 smithi067 ceph-mon[37808]: pgmap v198: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-03-30T02:43:33.794 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/show_mem.o 2024-03-30T02:43:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:33 smithi170 ceph-mon[78798]: pgmap v198: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.0 MiB/s wr, 273 op/s 2024-03-30T02:43:34.078 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/consoles.o 2024-03-30T02:43:34.114 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/power/poweroff.o 2024-03-30T02:43:34.124 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/conditional.o 2024-03-30T02:43:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.148+0000 7f0a93c00700 1 -- 172.21.15.67:0/200453042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a8c108770 msgr2=0x7f0a8c114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.148+0000 7f0a93c00700 1 --2- 172.21.15.67:0/200453042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a8c108770 0x7f0a8c114430 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00bb20 tx=0x7f0a7c00be30 comp rx=0 tx=0).stop 2024-03-30T02:43:34.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.149+0000 7f0a93c00700 1 -- 172.21.15.67:0/200453042 shutdown_connections 2024-03-30T02:43:34.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.149+0000 7f0a93c00700 1 --2- 172.21.15.67:0/200453042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a8c108770 0x7f0a8c114430 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.149+0000 7f0a93c00700 1 --2- 172.21.15.67:0/200453042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a8c10d670 0x7f0a8c10da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.149+0000 7f0a93c00700 1 --2- 172.21.15.67:0/200453042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8c107db0 0x7f0a8c108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.150+0000 7f0a93c00700 1 -- 172.21.15.67:0/200453042 >> 172.21.15.67:0/200453042 conn(0x7f0a8c06bc80 msgr2=0x7f0a8c06c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.150+0000 7f0a93c00700 1 -- 172.21.15.67:0/200453042 shutdown_connections 2024-03-30T02:43:34.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.150+0000 7f0a93c00700 1 -- 172.21.15.67:0/200453042 wait complete. 2024-03-30T02:43:34.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.153+0000 7f0a93c00700 1 Processor -- start 2024-03-30T02:43:34.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.154+0000 7f0a93c00700 1 -- start start 2024-03-30T02:43:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a93c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a8c107db0 0x7f0a8c1a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a93c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a8c108770 0x7f0a8c1a7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a93c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8c10d670 0x7f0a8c1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a93c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1a1f00 con 0x7f0a8c10d670 2024-03-30T02:43:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a93c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1a2070 con 0x7f0a8c107db0 2024-03-30T02:43:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a93c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c1a2230 con 0x7f0a8c108770 2024-03-30T02:43:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a8be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a8c108770 0x7f0a8c1a7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a8be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a8c108770 0x7f0a8c1a7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:42988/0 (socket says 172.21.15.67:42988) 2024-03-30T02:43:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a8be00700 1 -- 172.21.15.67:0/4024757778 learned_addr learned my addr 172.21.15.67:0/4024757778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.156+0000 7f0a91c00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8c10d670 0x7f0a8c1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.157+0000 7f0a91200700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a8c107db0 0x7f0a8c1a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.157+0000 7f0a8be00700 1 -- 172.21.15.67:0/4024757778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a8c107db0 msgr2=0x7f0a8c1a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.157+0000 7f0a8be00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a8c107db0 0x7f0a8c1a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.157+0000 7f0a8be00700 1 -- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8c10d670 msgr2=0x7f0a8c1a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.157+0000 7f0a8be00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8c10d670 0x7f0a8c1a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.157+0000 7f0a8be00700 1 -- 172.21.15.67:0/4024757778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a780095f0 con 0x7f0a8c108770 2024-03-30T02:43:34.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.157+0000 7f0a91200700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a8c107db0 0x7f0a8c1a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:34.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.157+0000 7f0a91c00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8c10d670 0x7f0a8c1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:43:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.158+0000 7f0a8be00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a8c108770 0x7f0a8c1a7c50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800ec10 tx=0x7f0a7800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.158+0000 7f0a89600700 1 -- 172.21.15.67:0/4024757778 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7800cd40 con 0x7f0a8c108770 2024-03-30T02:43:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.158+0000 7f0a89600700 1 -- 172.21.15.67:0/4024757778 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a780044e0 con 0x7f0a8c108770 2024-03-30T02:43:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.158+0000 7f0a89600700 1 -- 172.21.15.67:0/4024757778 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78010690 con 0x7f0a8c108770 2024-03-30T02:43:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.159+0000 7f0a93c00700 1 -- 172.21.15.67:0/4024757778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c009520 con 0x7f0a8c108770 2024-03-30T02:43:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.160+0000 7f0a93c00700 1 -- 172.21.15.67:0/4024757778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c1ae520 con 0x7f0a8c108770 2024-03-30T02:43:34.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.170+0000 7f0a89600700 1 -- 172.21.15.67:0/4024757778 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0a78003780 con 0x7f0a8c108770 2024-03-30T02:43:34.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.172+0000 7f0a89600700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0a700a13d0 0x7f0a700a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.173+0000 7f0a89600700 1 -- 172.21.15.67:0/4024757778 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f0a78014070 con 0x7f0a8c108770 2024-03-30T02:43:34.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.173+0000 7f0a91200700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0a700a13d0 0x7f0a700a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.174+0000 7f0a91200700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0a700a13d0 0x7f0a700a3890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007e90 tx=0x7f0a840073a0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.176+0000 7f0aa7795b80 1 Processor -- start 2024-03-30T02:43:34.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.177+0000 7f0aa7795b80 1 -- start start 2024-03-30T02:43:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.177+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d57a0 0x55811c4d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.177+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d60c0 0x55811c4e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.178+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 0x55811c4e33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.178+0000 7f0aa7795b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c42e990 con 0x55811c4d60c0 2024-03-30T02:43:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.178+0000 7f0aa7795b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c42c350 con 0x55811c4d57a0 2024-03-30T02:43:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.178+0000 7f0aa7795b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c427c40 con 0x55811c4e0fc0 2024-03-30T02:43:34.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.178+0000 7f0a91200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d57a0 0x55811c4d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.178+0000 7f0a91200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d57a0 0x55811c4d5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52414/0 (socket says 172.21.15.67:52414) 2024-03-30T02:43:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.178+0000 7f0a91200700 1 -- 172.21.15.67:0/3868581053 learned_addr learned my addr 172.21.15.67:0/3868581053 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.179+0000 7f0a91200700 1 -- 172.21.15.67:0/3868581053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 msgr2=0x55811c4e33b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:43:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.179+0000 7f0a91200700 1 --2- 172.21.15.67:0/3868581053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 0x55811c4e33b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.179+0000 7f0a91200700 1 -- 172.21.15.67:0/3868581053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d60c0 msgr2=0x55811c4e0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.180+0000 7f0a8be00700 1 --2- 172.21.15.67:0/3868581053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d60c0 0x55811c4e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.180+0000 7f0a91200700 1 --2- 172.21.15.67:0/3868581053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d60c0 0x55811c4e0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.180+0000 7f0a91200700 1 -- 172.21.15.67:0/3868581053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55811c2a93b0 con 0x55811c4d57a0 2024-03-30T02:43:34.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.181+0000 7f0a8be00700 1 --2- 172.21.15.67:0/3868581053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d60c0 0x55811c4e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:34.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.183+0000 7f0a91200700 1 --2- 172.21.15.67:0/3868581053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d57a0 0x55811c4d5b80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400f420 tx=0x7f0a8400f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fe5ec3c405a55e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.183+0000 7f0a82000700 1 -- 172.21.15.67:0/3868581053 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84020070 con 0x55811c4d57a0 2024-03-30T02:43:34.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.183+0000 7f0a82000700 1 -- 172.21.15.67:0/3868581053 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a8400fbf0 con 0x55811c4d57a0 2024-03-30T02:43:34.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.183+0000 7f0a82000700 1 -- 172.21.15.67:0/3868581053 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84004810 con 0x55811c4d57a0 2024-03-30T02:43:34.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.187+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3868581053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d57a0 msgr2=0x55811c4d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.187+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3868581053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d57a0 0x55811c4d5b80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400f420 tx=0x7f0a8400f000 comp rx=0 tx=0).stop 2024-03-30T02:43:34.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.188+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3868581053 shutdown_connections 2024-03-30T02:43:34.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.188+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3868581053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 0x55811c4e33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.188+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3868581053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d57a0 0x55811c4d5b80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.188+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3868581053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d60c0 0x55811c4e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.188+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3868581053 >> 172.21.15.67:0/3868581053 conn(0x55811c4356a0 msgr2=0x55811c4d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.188+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3868581053 shutdown_connections 2024-03-30T02:43:34.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.188+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3868581053 wait complete. 2024-03-30T02:43:34.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.189+0000 7f0aa7795b80 1 -- start start 2024-03-30T02:43:34.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.190+0000 7f0aa7795b80 1 Processor -- start 2024-03-30T02:43:34.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.191+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.192+0000 7f0a91200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.192+0000 7f0a91200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60948/0 (socket says 172.21.15.67:60948) 2024-03-30T02:43:34.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.192+0000 7f0a91200700 1 -- 172.21.15.67:0/3651642751 learned_addr learned my addr 172.21.15.67:0/3651642751 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.192+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4ddc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.192+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 0x55811c4de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.193+0000 7f0a8be00700 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4ddc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.193+0000 7f0a91c00700 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 0x55811c4de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.193+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c3c3d80 con 0x55811c4d57a0 2024-03-30T02:43:34.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.193+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c3c3a20 con 0x55811c4e0fc0 2024-03-30T02:43:34.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.194+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c2ab6a0 con 0x55811c4d60c0 2024-03-30T02:43:34.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.194+0000 7f0aa7795b80 10 client.0 _ll_register_callbacks cb 0x55811c3cd470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:34.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.195+0000 7f0a8be00700 1 -- 172.21.15.67:0/3651642751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 msgr2=0x55811c4de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.195+0000 7f0a8be00700 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 0x55811c4de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.195+0000 7f0a8be00700 1 -- 172.21.15.67:0/3651642751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 msgr2=0x55811c4e0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.195+0000 7f0a8be00700 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.195+0000 7f0a8be00700 1 -- 172.21.15.67:0/3651642751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8400ccd0 con 0x55811c4d60c0 2024-03-30T02:43:34.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.195+0000 7f0a91200700 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:34.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.196+0000 7f0a8be00700 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4ddc60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a780cac80 tx=0x7f0a780cb1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f601f96e9f3973eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.196+0000 7f0a76a00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:43:34.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.197+0000 7f0a76a00700 1 -- 172.21.15.67:0/3651642751 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78085300 con 0x55811c4d60c0 2024-03-30T02:43:34.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.197+0000 7f0a76a00700 1 -- 172.21.15.67:0/3651642751 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f0a78083e40 con 0x55811c4d60c0 2024-03-30T02:43:34.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.197+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55811c3e8380 con 0x55811c4d60c0 2024-03-30T02:43:34.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.201+0000 7f0a76a00700 1 -- 172.21.15.67:0/3651642751 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a7801f2f0 con 0x55811c4d60c0 2024-03-30T02:43:34.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.201+0000 7f0aa7795b80 10 client.26887 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:34.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.201+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55811c255b60 con 0x55811c4d60c0 2024-03-30T02:43:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.201+0000 7f0a76a00700 1 -- 172.21.15.67:0/3651642751 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55811c3e8380 con 0x55811c4d60c0 2024-03-30T02:43:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.202+0000 7f0a76a00700 1 -- 172.21.15.67:0/3651642751 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f0a78099b30 con 0x55811c4d60c0 2024-03-30T02:43:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.202+0000 7f0aa7795b80 10 client.26887 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.202+0000 7f0aa7795b80 10 client.26887 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:43:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.202+0000 7f0aa7795b80 20 client.26887 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.202+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55811c4f0ac0 0x55811c4f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.202+0000 7f0aa7795b80 4 client.26887 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:43:34.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.202+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55811c420700 con 0x55811c4f0ac0 2024-03-30T02:43:34.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.204+0000 7f0a91200700 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55811c4f0ac0 0x55811c4f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.205+0000 7f0a91200700 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55811c4f0ac0 0x55811c4f2ee0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=891177890f3a00be server_cookie=d9cff03c5e0dd6e6 in_seq=0 out_seq=0 2024-03-30T02:43:34.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.205+0000 7f0a76a00700 10 client.26887 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:43:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.207+0000 7f0a76a00700 1 -- 172.21.15.67:0/3651642751 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55811c420700 con 0x55811c4f0ac0 2024-03-30T02:43:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.207+0000 7f0a76a00700 10 client.26887 handle_command_reply: tid=0 2024-03-30T02:43:34.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.207+0000 7f0aa7795b80 1 client.26887 shutdown 2024-03-30T02:43:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.211+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 msgr2=0x55811c4ddc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.211+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4ddc60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0a780cac80 tx=0x7f0a780cb1e0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.212+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 shutdown_connections 2024-03-30T02:43:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.212+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55811c4f0ac0 0x55811c4f2ee0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.212+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.212+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 0x55811c4de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.212+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/3651642751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4ddc60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.212+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 >> 172.21.15.67:0/3651642751 conn(0x55811c4356a0 msgr2=0x55811c4d76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.212+0000 7f0a76a00700 0 client.26887 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:43:34.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.212+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 shutdown_connections 2024-03-30T02:43:34.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.215+0000 7f0aa7795b80 1 -- 172.21.15.67:0/3651642751 wait complete. 2024-03-30T02:43:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mtrr/if.o 2024-03-30T02:43:34.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.217+0000 7f0aa7795b80 20 client.26887 trim_cache size 0 max 16384 2024-03-30T02:43:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.247+0000 7f0aa7795b80 1 Processor -- start 2024-03-30T02:43:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.247+0000 7f0aa7795b80 1 -- start start 2024-03-30T02:43:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.248+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.248+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.248+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 0x55811c4dc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.248+0000 7f0aa7795b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c4ee190 con 0x55811c4d57a0 2024-03-30T02:43:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.248+0000 7f0aa7795b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c4ed640 con 0x55811c4e0fc0 2024-03-30T02:43:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.248+0000 7f0aa7795b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c4d7300 con 0x55811c4d60c0 2024-03-30T02:43:34.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.249+0000 7f0a8be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.249+0000 7f0a8be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43010/0 (socket says 172.21.15.67:43010) 2024-03-30T02:43:34.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.249+0000 7f0a8be00700 1 -- 172.21.15.67:0/164137888 learned_addr learned my addr 172.21.15.67:0/164137888 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.249+0000 7f0a91c00700 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 0x55811c4dc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.249+0000 7f0a8be00700 1 -- 172.21.15.67:0/164137888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 msgr2=0x55811c4dc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.249+0000 7f0a8be00700 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 0x55811c4dc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.249+0000 7f0a8be00700 1 -- 172.21.15.67:0/164137888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 msgr2=0x55811c4e7c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:43:34.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.250+0000 7f0a91200700 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.250+0000 7f0a8be00700 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.250+0000 7f0a8be00700 1 -- 172.21.15.67:0/164137888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55811c3edea0 con 0x55811c4d60c0 2024-03-30T02:43:34.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.250+0000 7f0a91200700 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:34.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.251+0000 7f0a8be00700 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4cb010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a780970b0 tx=0x7f0a78000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de8d45be1f8a1957 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.251+0000 7f0a82a00700 1 -- 172.21.15.67:0/164137888 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a780a2ac0 con 0x55811c4d60c0 2024-03-30T02:43:34.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.252+0000 7f0a82a00700 1 -- 172.21.15.67:0/164137888 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a78085300 con 0x55811c4d60c0 2024-03-30T02:43:34.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.252+0000 7f0a82a00700 1 -- 172.21.15.67:0/164137888 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7801f2f0 con 0x55811c4d60c0 2024-03-30T02:43:34.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.253+0000 7f0aa7795b80 1 -- 172.21.15.67:0/164137888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 msgr2=0x55811c4cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.253+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4cb010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0a780970b0 tx=0x7f0a78000bc0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.253+0000 7f0aa7795b80 1 -- 172.21.15.67:0/164137888 shutdown_connections 2024-03-30T02:43:34.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.253+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.253+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4e0fc0 0x55811c4dc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.253+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/164137888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4d60c0 0x55811c4cb010 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.253+0000 7f0aa7795b80 1 -- 172.21.15.67:0/164137888 >> 172.21.15.67:0/164137888 conn(0x55811c4356a0 msgr2=0x55811c4ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.254+0000 7f0aa7795b80 1 -- 172.21.15.67:0/164137888 shutdown_connections 2024-03-30T02:43:34.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.254+0000 7f0aa7795b80 1 -- 172.21.15.67:0/164137888 wait complete. 2024-03-30T02:43:34.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.255+0000 7f0aa7795b80 1 -- start start 2024-03-30T02:43:34.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.255+0000 7f0aa7795b80 1 Processor -- start 2024-03-30T02:43:34.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.256+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.257+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d60c0 0x55811c5428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.257+0000 7f0aa7795b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 0x55811c4e7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.257+0000 7f0aa7795b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c4e58a0 con 0x55811c4d57a0 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.257+0000 7f0aa7795b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c4e5a10 con 0x55811c4d60c0 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.257+0000 7f0aa7795b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55811c4e5bd0 con 0x55811c4e0fc0 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.258+0000 7f0a91c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 0x55811c4e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.258+0000 7f0a91c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 0x55811c4e7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43018/0 (socket says 172.21.15.67:43018) 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.258+0000 7f0a91c00700 1 -- 172.21.15.67:0/505607063 learned_addr learned my addr 172.21.15.67:0/505607063 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.258+0000 7f0aa7795b80 10 client.0 _ll_register_callbacks cb 0x55811c3cd470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.258+0000 7f0a8be00700 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d60c0 0x55811c5428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.258+0000 7f0a91c00700 1 -- 172.21.15.67:0/505607063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d60c0 msgr2=0x55811c5428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.258+0000 7f0a91c00700 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d60c0 0x55811c5428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.258+0000 7f0a91c00700 1 -- 172.21.15.67:0/505607063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 msgr2=0x55811c4e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.258+0000 7f0a91200700 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.259+0000 7f0a91c00700 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.259+0000 7f0a91c00700 1 -- 172.21.15.67:0/505607063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55811c255b60 con 0x55811c4e0fc0 2024-03-30T02:43:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.259+0000 7f0a91200700 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:34.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.261+0000 7f0a91c00700 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 0x55811c4e7880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00bb20 tx=0x7f0a7c00e8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a727f23ec284ba37 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.262+0000 7f0a76a00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:43:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.262+0000 7f0a76a00700 1 -- 172.21.15.67:0/505607063 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c005580 con 0x55811c4e0fc0 2024-03-30T02:43:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.263+0000 7f0a76a00700 1 -- 172.21.15.67:0/505607063 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f0a7c00b2d0 con 0x55811c4e0fc0 2024-03-30T02:43:34.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.263+0000 7f0aa7795b80 1 -- 172.21.15.67:0/505607063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a8400fbf0 con 0x55811c4e0fc0 2024-03-30T02:43:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.263+0000 7f0a76a00700 1 -- 172.21.15.67:0/505607063 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a7c0056e0 con 0x55811c4e0fc0 2024-03-30T02:43:34.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.264+0000 7f0aa7795b80 10 client.26893 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:34.266 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/ahash.o 2024-03-30T02:43:34.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.267+0000 7f0aa7795b80 1 -- 172.21.15.67:0/505607063 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55811c2adcd0 con 0x55811c4e0fc0 2024-03-30T02:43:34.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.267+0000 7f0a76a00700 1 -- 172.21.15.67:0/505607063 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a8400fbf0 con 0x55811c4e0fc0 2024-03-30T02:43:34.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.267+0000 7f0a76a00700 1 -- 172.21.15.67:0/505607063 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f0a7c029020 con 0x55811c4e0fc0 2024-03-30T02:43:34.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.268+0000 7f0aa7795b80 10 client.26893 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.268+0000 7f0aa7795b80 10 client.26893 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:43:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.268+0000 7f0aa7795b80 20 client.26893 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.268+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55811c5568b0 0x55811c558cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.268+0000 7f0aa7795b80 4 client.26893 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:43:34.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.268+0000 7f0aa7795b80 1 -- 172.21.15.67:0/505607063 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55811c276cf0 con 0x55811c5568b0 2024-03-30T02:43:34.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.271+0000 7f0a91200700 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55811c5568b0 0x55811c558cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.272+0000 7f0a91200700 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55811c5568b0 0x55811c558cd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=392205d1ecd0e29b server_cookie=56eb654294ddac7a in_seq=0 out_seq=0 2024-03-30T02:43:34.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.272+0000 7f0a76a00700 10 client.26893 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:43:34.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.272+0000 7f0a76a00700 1 -- 172.21.15.67:0/505607063 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55811c276cf0 con 0x55811c5568b0 2024-03-30T02:43:34.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.272+0000 7f0a76a00700 10 client.26893 handle_command_reply: tid=0 2024-03-30T02:43:34.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.272+0000 7f0aa7795b80 1 client.26893 shutdown 2024-03-30T02:43:34.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.273+0000 7f0aa7795b80 1 -- 172.21.15.67:0/505607063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 msgr2=0x55811c4e7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.273+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 0x55811c4e7880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00bb20 tx=0x7f0a7c00e8e0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.274+0000 7f0aa7795b80 1 -- 172.21.15.67:0/505607063 shutdown_connections 2024-03-30T02:43:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.274+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55811c5568b0 0x55811c558cd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.274+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55811c4d57a0 0x55811c4e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.274+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55811c4e0fc0 0x55811c4e7880 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.274+0000 7f0a76a00700 0 client.26893 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:43:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.274+0000 7f0aa7795b80 1 --2- 172.21.15.67:0/505607063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55811c4d60c0 0x55811c5428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.274+0000 7f0aa7795b80 1 -- 172.21.15.67:0/505607063 >> 172.21.15.67:0/505607063 conn(0x55811c4356a0 msgr2=0x55811c4cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.275+0000 7f0aa7795b80 1 -- 172.21.15.67:0/505607063 shutdown_connections 2024-03-30T02:43:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.275+0000 7f0aa7795b80 1 -- 172.21.15.67:0/505607063 wait complete. 2024-03-30T02:43:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.275+0000 7f0aa7795b80 20 client.26893 trim_cache size 0 max 16384 2024-03-30T02:43:34.278 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:43:34.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.280+0000 7f0a93c00700 1 -- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0a700a13d0 msgr2=0x7f0a700a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.280+0000 7f0a93c00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0a700a13d0 0x7f0a700a3890 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007e90 tx=0x7f0a840073a0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.280+0000 7f0a93c00700 1 -- 172.21.15.67:0/4024757778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a8c108770 msgr2=0x7f0a8c1a7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.280+0000 7f0a93c00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a8c108770 0x7f0a8c1a7c50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800ec10 tx=0x7f0a7800ef20 comp rx=0 tx=0).stop 2024-03-30T02:43:34.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.281+0000 7f0a93c00700 1 -- 172.21.15.67:0/4024757778 shutdown_connections 2024-03-30T02:43:34.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.281+0000 7f0a93c00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0a700a13d0 0x7f0a700a3890 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.281+0000 7f0a93c00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0a8c10d670 0x7f0a8c1a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.281+0000 7f0a93c00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a8c107db0 0x7f0a8c1a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.281+0000 7f0a93c00700 1 --2- 172.21.15.67:0/4024757778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a8c108770 0x7f0a8c1a7c50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.281+0000 7f0a93c00700 1 -- 172.21.15.67:0/4024757778 >> 172.21.15.67:0/4024757778 conn(0x7f0a8c06bc80 msgr2=0x7f0a8c071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.282+0000 7f0a93c00700 1 -- 172.21.15.67:0/4024757778 shutdown_connections 2024-03-30T02:43:34.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.282+0000 7f0a93c00700 1 -- 172.21.15.67:0/4024757778 wait complete. 2024-03-30T02:43:34.287 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pgtable.o 2024-03-30T02:43:34.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:43:34.504 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/power/built-in.a 2024-03-30T02:43:34.597 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/printk/printk.o 2024-03-30T02:43:34.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.823+0000 7fba86200700 1 -- 172.21.15.67:0/2914460908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8010f540 msgr2=0x7fba80107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.824+0000 7fba86200700 1 --2- 172.21.15.67:0/2914460908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8010f540 0x7fba80107db0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009920 tx=0x7fba6c009c30 comp rx=0 tx=0).stop 2024-03-30T02:43:34.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.824+0000 7fba86200700 1 -- 172.21.15.67:0/2914460908 shutdown_connections 2024-03-30T02:43:34.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.824+0000 7fba86200700 1 --2- 172.21.15.67:0/2914460908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8010f540 0x7fba80107db0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.825 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/cpuinfo.o 2024-03-30T02:43:34.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.824+0000 7fba86200700 1 --2- 172.21.15.67:0/2914460908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba8011a590 0x7fba8011c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.824+0000 7fba86200700 1 --2- 172.21.15.67:0/2914460908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba801082f0 0x7fba80108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.824+0000 7fba86200700 1 -- 172.21.15.67:0/2914460908 >> 172.21.15.67:0/2914460908 conn(0x7fba8006bdf0 msgr2=0x7fba8006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.824+0000 7fba86200700 1 -- 172.21.15.67:0/2914460908 shutdown_connections 2024-03-30T02:43:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.826+0000 7fba86200700 1 -- 172.21.15.67:0/2914460908 wait complete. 2024-03-30T02:43:34.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.827+0000 7fba86200700 1 Processor -- start 2024-03-30T02:43:34.828 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/interval_tree.o 2024-03-30T02:43:34.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.829+0000 7fba86200700 1 -- start start 2024-03-30T02:43:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.832+0000 7fba86200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba801082f0 0x7fba801a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.832+0000 7fba86200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba8010f540 0x7fba801a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.832+0000 7fba86200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8011a590 0x7fba801aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.832+0000 7fba86200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801ab010 con 0x7fba8011a590 2024-03-30T02:43:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.832+0000 7fba86200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801ab180 con 0x7fba801082f0 2024-03-30T02:43:34.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.832+0000 7fba86200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801ab340 con 0x7fba8010f540 2024-03-30T02:43:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba8010f540 0x7fba801a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba8010f540 0x7fba801a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43046/0 (socket says 172.21.15.67:43046) 2024-03-30T02:43:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7ea00700 1 -- 172.21.15.67:0/1939088504 learned_addr learned my addr 172.21.15.67:0/1939088504 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7fe00700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8011a590 0x7fba801aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7ea00700 1 -- 172.21.15.67:0/1939088504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba801082f0 msgr2=0x7fba801a60f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T02:43:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7ea00700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba801082f0 0x7fba801a60f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7ea00700 1 -- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8011a590 msgr2=0x7fba801aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7ea00700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8011a590 0x7fba801aa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7ea00700 1 -- 172.21.15.67:0/1939088504 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c009520 con 0x7fba8010f540 2024-03-30T02:43:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.833+0000 7fba7fe00700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8011a590 0x7fba801aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.834+0000 7fba7ea00700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba8010f540 0x7fba801a6630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba7000e9c0 tx=0x7fba7000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.835+0000 7fba77e00700 1 -- 172.21.15.67:0/1939088504 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7000cbd0 con 0x7fba8010f540 2024-03-30T02:43:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.835+0000 7fba77e00700 1 -- 172.21.15.67:0/1939088504 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba7000cd30 con 0x7fba8010f540 2024-03-30T02:43:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.835+0000 7fba77e00700 1 -- 172.21.15.67:0/1939088504 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70005580 con 0x7fba8010f540 2024-03-30T02:43:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.835+0000 7fba86200700 1 -- 172.21.15.67:0/1939088504 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba801ab620 con 0x7fba8010f540 2024-03-30T02:43:34.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.835+0000 7fba86200700 1 -- 172.21.15.67:0/1939088504 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba801b2e60 con 0x7fba8010f540 2024-03-30T02:43:34.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.846+0000 7fba77e00700 1 -- 172.21.15.67:0/1939088504 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fba70021440 con 0x7fba8010f540 2024-03-30T02:43:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.849+0000 7fba77e00700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fba640a1420 0x7fba640a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.849+0000 7fba77e00700 1 -- 172.21.15.67:0/1939088504 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fba70014070 con 0x7fba8010f540 2024-03-30T02:43:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.849+0000 7fba7f400700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fba640a1420 0x7fba640a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.850+0000 7fba99c7db80 1 Processor -- start 2024-03-30T02:43:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.850+0000 7fba99c7db80 1 -- start start 2024-03-30T02:43:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.851+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 0x561b01f1b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.851+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.851+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.851+0000 7fba99c7db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01e74990 con 0x561b01f1b5d0 2024-03-30T02:43:34.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.851+0000 7fba99c7db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01e72350 con 0x561b01f1bef0 2024-03-30T02:43:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.851+0000 7fba99c7db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01e6dc40 con 0x561b01f26df0 2024-03-30T02:43:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.851+0000 7fba7fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.851+0000 7fba7fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f291e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43050/0 (socket says 172.21.15.67:43050) 2024-03-30T02:43:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.851+0000 7fba7fe00700 1 -- 172.21.15.67:0/2172729445 learned_addr learned my addr 172.21.15.67:0/2172729445 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.852+0000 7fba7fe00700 1 -- 172.21.15.67:0/2172729445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 msgr2=0x561b01f268b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:43:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.852+0000 7fba7fe00700 1 --2- 172.21.15.67:0/2172729445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f268b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.852+0000 7fba7fe00700 1 -- 172.21.15.67:0/2172729445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 msgr2=0x561b01f1b9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:43:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.852+0000 7fba7fe00700 1 --2- 172.21.15.67:0/2172729445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 0x561b01f1b9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.852+0000 7fba7fe00700 1 -- 172.21.15.67:0/2172729445 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b01cef3b0 con 0x561b01f26df0 2024-03-30T02:43:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.852+0000 7fba7fe00700 1 --2- 172.21.15.67:0/2172729445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f291e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fba7800c450 tx=0x7fba7800c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fa7e91db2d42f35 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.852+0000 7fba6be00700 1 -- 172.21.15.67:0/2172729445 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78011070 con 0x561b01f26df0 2024-03-30T02:43:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.853+0000 7fba6be00700 1 -- 172.21.15.67:0/2172729445 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba7800c0d0 con 0x561b01f26df0 2024-03-30T02:43:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.853+0000 7fba6be00700 1 -- 172.21.15.67:0/2172729445 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78015da0 con 0x561b01f26df0 2024-03-30T02:43:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.854+0000 7fba99c7db80 1 -- 172.21.15.67:0/2172729445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 msgr2=0x561b01f291e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.854+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2172729445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f291e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fba7800c450 tx=0x7fba7800c480 comp rx=0 tx=0).stop 2024-03-30T02:43:34.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.854+0000 7fba7f400700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fba640a1420 0x7fba640a38e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c010 tx=0x7fba6c010d90 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.857+0000 7fba99c7db80 1 -- 172.21.15.67:0/2172729445 shutdown_connections 2024-03-30T02:43:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.857+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2172729445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 0x561b01f1b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.857+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2172729445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f291e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.857+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2172729445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f268b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.857+0000 7fba99c7db80 1 -- 172.21.15.67:0/2172729445 >> 172.21.15.67:0/2172729445 conn(0x561b01e7b6a0 msgr2=0x561b01f17a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.857+0000 7fba99c7db80 1 -- 172.21.15.67:0/2172729445 shutdown_connections 2024-03-30T02:43:34.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.857+0000 7fba99c7db80 1 -- 172.21.15.67:0/2172729445 wait complete. 2024-03-30T02:43:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.858+0000 7fba99c7db80 1 -- start start 2024-03-30T02:43:34.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.858+0000 7fba99c7db80 1 Processor -- start 2024-03-30T02:43:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.859+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 0x561b01f26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.859+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f23aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.859+0000 7fba7f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 0x561b01f26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.859+0000 7fba7f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 0x561b01f26760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60986/0 (socket says 172.21.15.67:60986) 2024-03-30T02:43:34.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.861+0000 7fba7f400700 1 -- 172.21.15.67:0/91533207 learned_addr learned my addr 172.21.15.67:0/91533207 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.861+0000 7fba99c7db80 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f24020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.861+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01e09d80 con 0x561b01f1b5d0 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.861+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01e09a20 con 0x561b01f1bef0 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.861+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01cf16a0 con 0x561b01f26df0 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.861+0000 7fba99c7db80 10 client.0 _ll_register_callbacks cb 0x561b01e13470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.862+0000 7fba7ea00700 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f23aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.862+0000 7fba7fe00700 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f24020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.862+0000 7fba7f400700 1 -- 172.21.15.67:0/91533207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 msgr2=0x561b01f24020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.862+0000 7fba7f400700 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f24020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.862+0000 7fba7f400700 1 -- 172.21.15.67:0/91533207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 msgr2=0x561b01f23aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.862+0000 7fba7f400700 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f23aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.862+0000 7fba7f400700 1 -- 172.21.15.67:0/91533207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba78009ca0 con 0x561b01f1b5d0 2024-03-30T02:43:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.862+0000 7fba7ea00700 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f23aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:43:34.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.863+0000 7fba7f400700 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 0x561b01f26760 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00f740 tx=0x7fba6c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc1341d4b84edb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.863+0000 7fba68c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:43:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.863+0000 7fba68c00700 1 -- 172.21.15.67:0/91533207 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c027070 con 0x561b01f1b5d0 2024-03-30T02:43:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.863+0000 7fba68c00700 1 -- 172.21.15.67:0/91533207 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fba6c00b7f0 con 0x561b01f1b5d0 2024-03-30T02:43:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.863+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b01e2e380 con 0x561b01f1b5d0 2024-03-30T02:43:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.865+0000 7fba68c00700 1 -- 172.21.15.67:0/91533207 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6c01c070 con 0x561b01f1b5d0 2024-03-30T02:43:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.865+0000 7fba99c7db80 10 client.16854 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:34.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.865+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561b01c9bb60 con 0x561b01f1b5d0 2024-03-30T02:43:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.865+0000 7fba68c00700 1 -- 172.21.15.67:0/91533207 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b01e2e380 con 0x561b01f1b5d0 2024-03-30T02:43:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.866+0000 7fba68c00700 1 -- 172.21.15.67:0/91533207 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fba6c0037a0 con 0x561b01f1b5d0 2024-03-30T02:43:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.866+0000 7fba99c7db80 10 client.16854 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.866+0000 7fba99c7db80 10 client.16854 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:43:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.866+0000 7fba99c7db80 20 client.16854 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.866+0000 7fba99c7db80 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x561b01f47a70 0x561b01f49e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.866+0000 7fba99c7db80 4 client.16854 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:43:34.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.866+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b01e66700 con 0x561b01f47a70 2024-03-30T02:43:34.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.870+0000 7fba7ea00700 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x561b01f47a70 0x561b01f49e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.872 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/uncore_discovery.o 2024-03-30T02:43:34.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.873+0000 7fba7ea00700 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x561b01f47a70 0x561b01f49e90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9076fa5aa4830d63 server_cookie=1611862ed8bb2ee0 in_seq=0 out_seq=0 2024-03-30T02:43:34.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.876+0000 7fba68c00700 10 client.16854 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:43:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.877+0000 7fba68c00700 1 -- 172.21.15.67:0/91533207 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561b01e66700 con 0x561b01f47a70 2024-03-30T02:43:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.877+0000 7fba68c00700 10 client.16854 handle_command_reply: tid=0 2024-03-30T02:43:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.878+0000 7fba99c7db80 1 client.16854 shutdown 2024-03-30T02:43:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.878+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 msgr2=0x561b01f26760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.878+0000 7fba99c7db80 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 0x561b01f26760 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00f740 tx=0x7fba6c009c30 comp rx=0 tx=0).stop 2024-03-30T02:43:34.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.879+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 shutdown_connections 2024-03-30T02:43:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.879+0000 7fba99c7db80 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x561b01f47a70 0x561b01f49e90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.879+0000 7fba99c7db80 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f1b5d0 0x561b01f26760 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.879+0000 7fba99c7db80 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f26df0 0x561b01f24020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.879+0000 7fba68c00700 0 client.16854 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:43:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.879+0000 7fba99c7db80 1 --2- 172.21.15.67:0/91533207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f23aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.879+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 >> 172.21.15.67:0/91533207 conn(0x561b01e7b6a0 msgr2=0x561b01f1d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.879+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 shutdown_connections 2024-03-30T02:43:34.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.879+0000 7fba99c7db80 1 -- 172.21.15.67:0/91533207 wait complete. 2024-03-30T02:43:34.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.880+0000 7fba99c7db80 20 client.16854 trim_cache size 0 max 16384 2024-03-30T02:43:34.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.908+0000 7fba99c7db80 1 Processor -- start 2024-03-30T02:43:34.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.909+0000 7fba99c7db80 1 -- start start 2024-03-30T02:43:34.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.909+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f1bef0 0x561b01f43950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.909+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f26df0 0x561b01f26290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.910+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f4e7d0 0x561b01f1d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.910+0000 7fba7ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f26df0 0x561b01f26290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.910+0000 7fba7ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f26df0 0x561b01f26290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52472/0 (socket says 172.21.15.67:52472) 2024-03-30T02:43:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.910+0000 7fba7fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f4e7d0 0x561b01f1d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.910+0000 7fba7fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f4e7d0 0x561b01f1d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60994/0 (socket says 172.21.15.67:60994) 2024-03-30T02:43:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.911+0000 7fba7ea00700 1 -- 172.21.15.67:0/3984804148 learned_addr learned my addr 172.21.15.67:0/3984804148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.911+0000 7fba7f400700 1 --2- 172.21.15.67:0/3984804148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f1bef0 0x561b01f43950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.911+0000 7fba99c7db80 1 -- 172.21.15.67:0/3984804148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01f1d0f0 con 0x561b01f4e7d0 2024-03-30T02:43:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.911+0000 7fba99c7db80 1 -- 172.21.15.67:0/3984804148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01f1c850 con 0x561b01f26df0 2024-03-30T02:43:34.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.911+0000 7fba99c7db80 1 -- 172.21.15.67:0/3984804148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01f4a220 con 0x561b01f1bef0 2024-03-30T02:43:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.912+0000 7fba7ea00700 1 -- 172.21.15.67:0/3984804148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f1bef0 msgr2=0x561b01f43950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.912+0000 7fba7ea00700 1 --2- 172.21.15.67:0/3984804148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f1bef0 0x561b01f43950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.912+0000 7fba7ea00700 1 -- 172.21.15.67:0/3984804148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f4e7d0 msgr2=0x561b01f1d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.912+0000 7fba7ea00700 1 --2- 172.21.15.67:0/3984804148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f4e7d0 0x561b01f1d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.912+0000 7fba7ea00700 1 -- 172.21.15.67:0/3984804148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b01e33ea0 con 0x561b01f26df0 2024-03-30T02:43:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.912+0000 7fba7ea00700 1 --2- 172.21.15.67:0/3984804148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f26df0 0x561b01f26290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba70099df0 tx=0x7fba70005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3fd68af33d4d70c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.914+0000 7fba75200700 1 -- 172.21.15.67:0/3984804148 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba700b8200 con 0x561b01f26df0 2024-03-30T02:43:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.914+0000 7fba75200700 1 -- 172.21.15.67:0/3984804148 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba70005700 con 0x561b01f26df0 2024-03-30T02:43:34.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.914+0000 7fba75200700 1 -- 172.21.15.67:0/3984804148 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70016590 con 0x561b01f26df0 2024-03-30T02:43:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.915+0000 7fba99c7db80 1 -- 172.21.15.67:0/3984804148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f26df0 msgr2=0x561b01f26290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.915+0000 7fba99c7db80 1 --2- 172.21.15.67:0/3984804148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f26df0 0x561b01f26290 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba70099df0 tx=0x7fba70005910 comp rx=0 tx=0).stop 2024-03-30T02:43:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.915+0000 7fba99c7db80 1 -- 172.21.15.67:0/3984804148 shutdown_connections 2024-03-30T02:43:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.916+0000 7fba99c7db80 1 --2- 172.21.15.67:0/3984804148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f4e7d0 0x561b01f1d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.916+0000 7fba99c7db80 1 --2- 172.21.15.67:0/3984804148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f1bef0 0x561b01f43950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.916+0000 7fba99c7db80 1 --2- 172.21.15.67:0/3984804148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f26df0 0x561b01f26290 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.916+0000 7fba99c7db80 1 -- 172.21.15.67:0/3984804148 >> 172.21.15.67:0/3984804148 conn(0x561b01e7b6a0 msgr2=0x561b01f43240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.918+0000 7fba99c7db80 1 -- 172.21.15.67:0/3984804148 shutdown_connections 2024-03-30T02:43:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.920+0000 7fba99c7db80 1 -- 172.21.15.67:0/3984804148 wait complete. 2024-03-30T02:43:34.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.920+0000 7fba99c7db80 1 -- start start 2024-03-30T02:43:34.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.923+0000 7fba99c7db80 1 Processor -- start 2024-03-30T02:43:34.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.923+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f3e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.923+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f26df0 0x561b01f3cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.923+0000 7fba99c7db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f4e7d0 0x561b01f3d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.924+0000 7fba99c7db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01f3d9c0 con 0x561b01f26df0 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.924+0000 7fba99c7db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01f3db30 con 0x561b01f1bef0 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.924+0000 7fba99c7db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b01f3dcf0 con 0x561b01f4e7d0 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.924+0000 7fba99c7db80 10 client.0 _ll_register_callbacks cb 0x561b01e13470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.924+0000 7fba7f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f3e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.924+0000 7fba7f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f3e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52484/0 (socket says 172.21.15.67:52484) 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.924+0000 7fba7f400700 1 -- 172.21.15.67:0/2338185129 learned_addr learned my addr 172.21.15.67:0/2338185129 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.924+0000 7fba7ea00700 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f26df0 0x561b01f3cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.924+0000 7fba7fe00700 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f4e7d0 0x561b01f3d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.925+0000 7fba7f400700 1 -- 172.21.15.67:0/2338185129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f4e7d0 msgr2=0x561b01f3d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.925+0000 7fba7f400700 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f4e7d0 0x561b01f3d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.925+0000 7fba7f400700 1 -- 172.21.15.67:0/2338185129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f26df0 msgr2=0x561b01f3cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.925+0000 7fba7f400700 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f26df0 0x561b01f3cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.925+0000 7fba7f400700 1 -- 172.21.15.67:0/2338185129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba801b2e60 con 0x561b01f1bef0 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.925+0000 7fba7ea00700 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f26df0 0x561b01f3cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.925+0000 7fba7f400700 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f3e9d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fba6c020fd0 tx=0x7fba6c003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b64acd14f8a3b58 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.925+0000 7fba68c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.926+0000 7fba68c00700 1 -- 172.21.15.67:0/2338185129 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c03b020 con 0x561b01f1bef0 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.926+0000 7fba99c7db80 1 -- 172.21.15.67:0/2338185129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba7800c0d0 con 0x561b01f1bef0 2024-03-30T02:43:34.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.926+0000 7fba68c00700 1 -- 172.21.15.67:0/2338185129 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fba6c00ba00 con 0x561b01f1bef0 2024-03-30T02:43:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.926+0000 7fba68c00700 1 -- 172.21.15.67:0/2338185129 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6c027070 con 0x561b01f1bef0 2024-03-30T02:43:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.928+0000 7fba99c7db80 10 client.26951 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:34.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.928+0000 7fba99c7db80 1 -- 172.21.15.67:0/2338185129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561b01cf3cd0 con 0x561b01f1bef0 2024-03-30T02:43:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.930+0000 7fba68c00700 1 -- 172.21.15.67:0/2338185129 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba7800c0d0 con 0x561b01f1bef0 2024-03-30T02:43:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.930+0000 7fba68c00700 1 -- 172.21.15.67:0/2338185129 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7fba6c036770 con 0x561b01f1bef0 2024-03-30T02:43:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.930+0000 7fba99c7db80 10 client.26951 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.930+0000 7fba99c7db80 10 client.26951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:43:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.930+0000 7fba99c7db80 20 client.26951 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.930+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x561b01f9ca10 0x561b01f9ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:34.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.930+0000 7fba99c7db80 4 client.26951 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:43:34.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.930+0000 7fba99c7db80 1 -- 172.21.15.67:0/2338185129 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b01cbccf0 con 0x561b01f9ca10 2024-03-30T02:43:34.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.933+0000 7fba7ea00700 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x561b01f9ca10 0x561b01f9ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.934+0000 7fba7ea00700 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x561b01f9ca10 0x561b01f9ee30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6c0b9a452b74da4 server_cookie=87a3f7bd9d2debf4 in_seq=0 out_seq=0 2024-03-30T02:43:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.934+0000 7fba68c00700 10 client.26951 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:43:34.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.934+0000 7fba68c00700 1 -- 172.21.15.67:0/2338185129 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b01cbccf0 con 0x561b01f9ca10 2024-03-30T02:43:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.934+0000 7fba68c00700 10 client.26951 handle_command_reply: tid=0 2024-03-30T02:43:34.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.934+0000 7fba99c7db80 1 client.26951 shutdown 2024-03-30T02:43:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba99c7db80 1 -- 172.21.15.67:0/2338185129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 msgr2=0x561b01f3e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f3e9d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fba6c020fd0 tx=0x7fba6c003d40 comp rx=0 tx=0).stop 2024-03-30T02:43:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba99c7db80 1 -- 172.21.15.67:0/2338185129 shutdown_connections 2024-03-30T02:43:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561b01f4e7d0 0x561b01f3d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b01f1bef0 0x561b01f3e9d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x561b01f9ca10 0x561b01f9ee30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.938 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:43:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba99c7db80 1 --2- 172.21.15.67:0/2338185129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561b01f26df0 0x561b01f3cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba68c00700 0 client.26951 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:43:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba99c7db80 1 -- 172.21.15.67:0/2338185129 >> 172.21.15.67:0/2338185129 conn(0x561b01e7b6a0 msgr2=0x561b01f1d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.935+0000 7fba99c7db80 1 -- 172.21.15.67:0/2338185129 shutdown_connections 2024-03-30T02:43:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.936+0000 7fba99c7db80 1 -- 172.21.15.67:0/2338185129 wait complete. 2024-03-30T02:43:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.936+0000 7fba99c7db80 20 client.26951 trim_cache size 0 max 16384 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.937+0000 7fba86200700 1 -- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fba640a1420 msgr2=0x7fba640a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.937+0000 7fba86200700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fba640a1420 0x7fba640a38e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c010 tx=0x7fba6c010d90 comp rx=0 tx=0).stop 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.937+0000 7fba86200700 1 -- 172.21.15.67:0/1939088504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba8010f540 msgr2=0x7fba801a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.937+0000 7fba86200700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba8010f540 0x7fba801a6630 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba7000e9c0 tx=0x7fba7000ed80 comp rx=0 tx=0).stop 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.938+0000 7fba86200700 1 -- 172.21.15.67:0/1939088504 shutdown_connections 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.938+0000 7fba86200700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fba640a1420 0x7fba640a38e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.938+0000 7fba86200700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba8011a590 0x7fba801aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.938+0000 7fba86200700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba801082f0 0x7fba801a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.938+0000 7fba86200700 1 --2- 172.21.15.67:0/1939088504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba8010f540 0x7fba801a6630 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.938+0000 7fba86200700 1 -- 172.21.15.67:0/1939088504 >> 172.21.15.67:0/1939088504 conn(0x7fba8006bdf0 msgr2=0x7fba80118580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:34.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.938+0000 7fba86200700 1 -- 172.21.15.67:0/1939088504 shutdown_connections 2024-03-30T02:43:34.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:34.938+0000 7fba86200700 1 -- 172.21.15.67:0/1939088504 wait complete. 2024-03-30T02:43:34.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:43:35.019 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mtrr/generic.o 2024-03-30T02:43:35.337 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/physaddr.o 2024-03-30T02:43:35.392 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/shash.o 2024-03-30T02:43:35.434 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/mls.o 2024-03-30T02:43:35.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.457+0000 7f71b1000700 1 -- 172.21.15.67:0/963139269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac11a590 msgr2=0x7f71ac11c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:35.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.457+0000 7f71b1000700 1 --2- 172.21.15.67:0/963139269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac11a590 0x7f71ac11c980 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7194009920 tx=0x7f7194009c30 comp rx=0 tx=0).stop 2024-03-30T02:43:35.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.458+0000 7f71b1000700 1 -- 172.21.15.67:0/963139269 shutdown_connections 2024-03-30T02:43:35.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.458+0000 7f71b1000700 1 --2- 172.21.15.67:0/963139269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac11a590 0x7f71ac11c980 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.458+0000 7f71b1000700 1 --2- 172.21.15.67:0/963139269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac10f540 0x7f71ac107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.458+0000 7f71b1000700 1 --2- 172.21.15.67:0/963139269 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac1082f0 0x7f71ac108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.458+0000 7f71b1000700 1 -- 172.21.15.67:0/963139269 >> 172.21.15.67:0/963139269 conn(0x7f71ac06bdf0 msgr2=0x7f71ac06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:35.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.459+0000 7f71b1000700 1 -- 172.21.15.67:0/963139269 shutdown_connections 2024-03-30T02:43:35.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.459+0000 7f71b1000700 1 -- 172.21.15.67:0/963139269 wait complete. 2024-03-30T02:43:35.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.461+0000 7f71b1000700 1 Processor -- start 2024-03-30T02:43:35.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.463+0000 7f71b1000700 1 -- start start 2024-03-30T02:43:35.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.464+0000 7f71b1000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac1082f0 0x7f71ac115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.465+0000 7f71b1000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac10f540 0x7f71ac116020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.465+0000 7f71b1000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac11a590 0x7f71ac1a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.465+0000 7f71b1000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac1a3830 con 0x7f71ac10f540 2024-03-30T02:43:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.465+0000 7f71b1000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac1a39a0 con 0x7f71ac1082f0 2024-03-30T02:43:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.465+0000 7f71b1000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac1a3b60 con 0x7f71ac11a590 2024-03-30T02:43:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.467+0000 7f71a9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac1082f0 0x7f71ac115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.467+0000 7f71a9400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac10f540 0x7f71ac116020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.467+0000 7f71a9400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac10f540 0x7f71ac116020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32786/0 (socket says 172.21.15.67:32786) 2024-03-30T02:43:35.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.467+0000 7f71a9400700 1 -- 172.21.15.67:0/820760784 learned_addr learned my addr 172.21.15.67:0/820760784 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.467+0000 7f71aa800700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac11a590 0x7f71ac1a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.468+0000 7f71aa800700 1 -- 172.21.15.67:0/820760784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac1082f0 msgr2=0x7f71ac115ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.468+0000 7f71aa800700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac1082f0 0x7f71ac115ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.468+0000 7f71aa800700 1 -- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac10f540 msgr2=0x7f71ac116020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:35.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.468+0000 7f71aa800700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac10f540 0x7f71ac116020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.468+0000 7f71aa800700 1 -- 172.21.15.67:0/820760784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7194009520 con 0x7f71ac11a590 2024-03-30T02:43:35.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.468+0000 7f71a9400700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac10f540 0x7f71ac116020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:43:35.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.470+0000 7f71aa800700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac11a590 0x7f71ac1a31f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7194009230 tx=0x7f719400f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:35.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.470+0000 7f71a2a00700 1 -- 172.21.15.67:0/820760784 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719401c070 con 0x7f71ac11a590 2024-03-30T02:43:35.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.470+0000 7f71a2a00700 1 -- 172.21.15.67:0/820760784 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f719400baa0 con 0x7f71ac11a590 2024-03-30T02:43:35.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.470+0000 7f71a2a00700 1 -- 172.21.15.67:0/820760784 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194004710 con 0x7f71ac11a590 2024-03-30T02:43:35.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.471+0000 7f71b1000700 1 -- 172.21.15.67:0/820760784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac1a3de0 con 0x7f71ac11a590 2024-03-30T02:43:35.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.472+0000 7f71b1000700 1 -- 172.21.15.67:0/820760784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac1a4030 con 0x7f71ac11a590 2024-03-30T02:43:35.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.479+0000 7f71a2a00700 1 -- 172.21.15.67:0/820760784 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7194020070 con 0x7f71ac11a590 2024-03-30T02:43:35.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.484+0000 7f71c4bf8b80 1 Processor -- start 2024-03-30T02:43:35.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.485+0000 7f71a2a00700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71900a1310 0x7f71900a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.485+0000 7f71a2a00700 1 -- 172.21.15.67:0/820760784 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f71940cd870 con 0x7f71ac11a590 2024-03-30T02:43:35.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.488+0000 7f71a9400700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71900a1310 0x7f71900a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.489+0000 7f71c4bf8b80 1 -- start start 2024-03-30T02:43:35.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.489+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b47869b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.490+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4786ef0 0x5649b47918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.490+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4791df0 0x5649b47941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.490+0000 7f71a9400700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71900a1310 0x7f71900a37d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007fd0 tx=0x7f71a4009040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:35.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.490+0000 7f71c4bf8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b46df990 con 0x5649b4791df0 2024-03-30T02:43:35.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.490+0000 7f71aa800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4791df0 0x5649b47941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71a9400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4786ef0 0x5649b47918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71aa800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4791df0 0x5649b47941e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32798/0 (socket says 172.21.15.67:32798) 2024-03-30T02:43:35.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71aa800700 1 -- 172.21.15.67:0/3542412586 learned_addr learned my addr 172.21.15.67:0/3542412586 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:35.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/3542412586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b46dd350 con 0x5649b47865d0 2024-03-30T02:43:35.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/3542412586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b46d8c40 con 0x5649b4786ef0 2024-03-30T02:43:35.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71a9e00700 1 --2- 172.21.15.67:0/3542412586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b47869b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71a9e00700 1 -- 172.21.15.67:0/3542412586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4786ef0 msgr2=0x5649b47918b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:35.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71a9e00700 1 --2- 172.21.15.67:0/3542412586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4786ef0 0x5649b47918b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71a9e00700 1 -- 172.21.15.67:0/3542412586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4791df0 msgr2=0x5649b47941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:35.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.491+0000 7f71a9e00700 1 --2- 172.21.15.67:0/3542412586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4791df0 0x5649b47941e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.492+0000 7f71a9e00700 1 -- 172.21.15.67:0/3542412586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649b455a3b0 con 0x5649b47865d0 2024-03-30T02:43:35.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.492+0000 7f71a9e00700 1 --2- 172.21.15.67:0/3542412586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b47869b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f719c01aba0 tx=0x7f719c01aeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa755d01d48762f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:35.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.492+0000 7f719b400700 1 -- 172.21.15.67:0/3542412586 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c022bb0 con 0x5649b47865d0 2024-03-30T02:43:35.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.493+0000 7f71aa800700 1 --2- 172.21.15.67:0/3542412586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4791df0 0x5649b47941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:35.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.493+0000 7f719b400700 1 -- 172.21.15.67:0/3542412586 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f719c004510 con 0x5649b47865d0 2024-03-30T02:43:35.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.493+0000 7f719b400700 1 -- 172.21.15.67:0/3542412586 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c02b440 con 0x5649b47865d0 2024-03-30T02:43:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.494+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/3542412586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 msgr2=0x5649b47869b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.494+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/3542412586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b47869b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f719c01aba0 tx=0x7f719c01aeb0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.494+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/3542412586 shutdown_connections 2024-03-30T02:43:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.494+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/3542412586 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4791df0 0x5649b47941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.494+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/3542412586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b47869b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.494+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/3542412586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4786ef0 0x5649b47918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.494+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/3542412586 >> 172.21.15.67:0/3542412586 conn(0x5649b46e66a0 msgr2=0x5649b4782a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.494+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/3542412586 shutdown_connections 2024-03-30T02:43:35.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.495+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/3542412586 wait complete. 2024-03-30T02:43:35.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.495+0000 7f71c4bf8b80 1 -- start start 2024-03-30T02:43:35.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.496+0000 7f71c4bf8b80 1 Processor -- start 2024-03-30T02:43:35.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.499+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b47865d0 0x5649b47917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.499+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b4786ef0 0x5649b478eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.499+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.499+0000 7f71c4bf8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b4674d80 con 0x5649b47865d0 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.499+0000 7f71c4bf8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b4674a20 con 0x5649b4786ef0 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.499+0000 7f71c4bf8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b455c6a0 con 0x5649b4791df0 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.499+0000 7f71c4bf8b80 10 client.0 _ll_register_callbacks cb 0x5649b467e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.499+0000 7f71aa800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.500+0000 7f71aa800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43104/0 (socket says 172.21.15.67:43104) 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.500+0000 7f71aa800700 1 -- 172.21.15.67:0/649797839 learned_addr learned my addr 172.21.15.67:0/649797839 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.500+0000 7f71a9400700 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b4786ef0 0x5649b478eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.500+0000 7f71a9e00700 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b47865d0 0x5649b47917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.500+0000 7f71aa800700 1 -- 172.21.15.67:0/649797839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b4786ef0 msgr2=0x5649b478eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:35.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.500+0000 7f71aa800700 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b4786ef0 0x5649b478eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.500+0000 7f71aa800700 1 -- 172.21.15.67:0/649797839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b47865d0 msgr2=0x5649b47917e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:35.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.500+0000 7f71aa800700 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b47865d0 0x5649b47917e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.500+0000 7f71aa800700 1 -- 172.21.15.67:0/649797839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c01a800 con 0x5649b4791df0 2024-03-30T02:43:35.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.501+0000 7f71a9e00700 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b47865d0 0x5649b47917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:35.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.501+0000 7f71aa800700 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478f110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7194096bc0 tx=0x7f7194005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7058a09dd3d0c19 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:35.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.501+0000 7f718fe00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:43:35.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.501+0000 7f718fe00700 1 -- 172.21.15.67:0/649797839 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719409ab20 con 0x5649b4791df0 2024-03-30T02:43:35.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.501+0000 7f718fe00700 1 -- 172.21.15.67:0/649797839 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f719409adc0 con 0x5649b4791df0 2024-03-30T02:43:35.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.506+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/649797839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649b4699380 con 0x5649b4791df0 2024-03-30T02:43:35.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.506+0000 7f718fe00700 1 -- 172.21.15.67:0/649797839 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7194005250 con 0x5649b4791df0 2024-03-30T02:43:35.507 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/devices.o 2024-03-30T02:43:35.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.506+0000 7f71c4bf8b80 10 client.26923 fetch_fsmap learned FSMap version 36 2024-03-30T02:43:35.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.506+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/649797839 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5649b4506b60 con 0x5649b4791df0 2024-03-30T02:43:35.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.507+0000 7f718fe00700 1 -- 172.21.15.67:0/649797839 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71940b2020 con 0x5649b4791df0 2024-03-30T02:43:35.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.508+0000 7f718fe00700 1 -- 172.21.15.67:0/649797839 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 36) v1 ==== 7498+0+0 (secure 0 0 0) 0x7f719409e190 con 0x5649b4791df0 2024-03-30T02:43:35.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.508+0000 7f71c4bf8b80 10 client.26923 fetch_fsmap finished waiting for FSMap version 36 2024-03-30T02:43:35.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.508+0000 7f71c4bf8b80 10 client.26923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:43:35.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.508+0000 7f71c4bf8b80 20 client.26923 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:35.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.508+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5649b47a17b0 0x5649b47a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.508+0000 7f71c4bf8b80 4 client.26923 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:43:35.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.508+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/649797839 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649b46d1700 con 0x5649b47a17b0 2024-03-30T02:43:35.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.508+0000 7f71a9e00700 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5649b47a17b0 0x5649b47a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.511+0000 7f71a9e00700 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5649b47a17b0 0x5649b47a3bd0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b6ab66f483eccf server_cookie=ba33085270056b88 in_seq=0 out_seq=0 2024-03-30T02:43:35.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.511+0000 7f718fe00700 10 client.26923 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:35.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.511+0000 7f718fe00700 1 -- 172.21.15.67:0/649797839 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5649b46d1700 con 0x5649b47a17b0 2024-03-30T02:43:35.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.512+0000 7f718fe00700 10 client.26923 handle_command_reply: tid=0 2024-03-30T02:43:35.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.512+0000 7f71c4bf8b80 1 client.26923 shutdown 2024-03-30T02:43:35.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.512+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/649797839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 msgr2=0x5649b478f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:35.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.512+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478f110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7194096bc0 tx=0x7f7194005a00 comp rx=0 tx=0).stop 2024-03-30T02:43:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.514+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/649797839 shutdown_connections 2024-03-30T02:43:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.514+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5649b47a17b0 0x5649b47a3bd0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.514+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b47865d0 0x5649b47917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.514+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478f110 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.514+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/649797839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b4786ef0 0x5649b478eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.514+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/649797839 >> 172.21.15.67:0/649797839 conn(0x5649b46e66a0 msgr2=0x5649b47886c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.514+0000 7f718fe00700 0 client.26923 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.514+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/649797839 shutdown_connections 2024-03-30T02:43:35.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.514+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/649797839 wait complete. 2024-03-30T02:43:35.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.518+0000 7f71c4bf8b80 20 client.26923 trim_cache size 0 max 16384 2024-03-30T02:43:35.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.579+0000 7f71c4bf8b80 1 Processor -- start 2024-03-30T02:43:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.580+0000 7f71c4bf8b80 1 -- start start 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.580+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4798aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.580+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b4789db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.580+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.580+0000 7f71c4bf8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b4799060 con 0x5649b4786ef0 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.580+0000 7f71c4bf8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b478d050 con 0x5649b47865d0 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.580+0000 7f71c4bf8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b478f710 con 0x5649b4791df0 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4798aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4798aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52524/0 (socket says 172.21.15.67:52524) 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9e00700 1 -- 172.21.15.67:0/2073608920 learned_addr learned my addr 172.21.15.67:0/2073608920 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71aa800700 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9e00700 1 -- 172.21.15.67:0/2073608920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 msgr2=0x5649b478a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9e00700 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9e00700 1 -- 172.21.15.67:0/2073608920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 msgr2=0x5649b4789db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9400700 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b4789db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9e00700 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b4789db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9e00700 1 -- 172.21.15.67:0/2073608920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649b469eea0 con 0x5649b47865d0 2024-03-30T02:43:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.581+0000 7f71a9400700 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b4789db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.582+0000 7f71aa800700 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.583+0000 7f71a9e00700 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4798aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f719c01aba0 tx=0x7f719c005920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce8f2fb8faf5bc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.584+0000 7f719be00700 1 -- 172.21.15.67:0/2073608920 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c030040 con 0x5649b47865d0 2024-03-30T02:43:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.584+0000 7f719be00700 1 -- 172.21.15.67:0/2073608920 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f719c035dc0 con 0x5649b47865d0 2024-03-30T02:43:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.584+0000 7f719be00700 1 -- 172.21.15.67:0/2073608920 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c005690 con 0x5649b47865d0 2024-03-30T02:43:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.585+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/2073608920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 msgr2=0x5649b4798aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.585+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4798aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f719c01aba0 tx=0x7f719c005920 comp rx=0 tx=0).stop 2024-03-30T02:43:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.589+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/2073608920 shutdown_connections 2024-03-30T02:43:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.589+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b478a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.589+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4798aa0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.589+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/2073608920 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b4789db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.589+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/2073608920 >> 172.21.15.67:0/2073608920 conn(0x5649b46e66a0 msgr2=0x5649b478e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.589+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/2073608920 shutdown_connections 2024-03-30T02:43:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.589+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/2073608920 wait complete. 2024-03-30T02:43:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.590+0000 7f71c4bf8b80 1 -- start start 2024-03-30T02:43:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.590+0000 7f71c4bf8b80 1 Processor -- start 2024-03-30T02:43:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.591+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4796b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.591+0000 7f71c4bf8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b47970b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.591+0000 7f71a9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4796b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.591+0000 7f71a9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4796b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:52534/0 (socket says 172.21.15.67:52534) 2024-03-30T02:43:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71a9e00700 1 -- 172.21.15.67:0/1699596590 learned_addr learned my addr 172.21.15.67:0/1699596590 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71a9400700 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b47970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b47ed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b47edaa0 con 0x5649b4786ef0 2024-03-30T02:43:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b47edc10 con 0x5649b47865d0 2024-03-30T02:43:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649b47eddd0 con 0x5649b4791df0 2024-03-30T02:43:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71c4bf8b80 10 client.0 _ll_register_callbacks cb 0x5649b467e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71a9e00700 1 -- 172.21.15.67:0/1699596590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 msgr2=0x5649b47ed560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:43:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71a9e00700 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b47ed560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.592+0000 7f71a9e00700 1 -- 172.21.15.67:0/1699596590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 msgr2=0x5649b47970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.593+0000 7f71a9e00700 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b47970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.593+0000 7f71a9e00700 1 -- 172.21.15.67:0/1699596590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649b455a3b0 con 0x5649b47865d0 2024-03-30T02:43:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.593+0000 7f71a9400700 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b47970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:43:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.593+0000 7f71a9e00700 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4796b70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f719c035600 tx=0x7f719c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=524c5844db3eb15 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.593+0000 7f718fe00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:43:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.593+0000 7f718fe00700 1 -- 172.21.15.67:0/1699596590 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c030070 con 0x5649b47865d0 2024-03-30T02:43:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.594+0000 7f718fe00700 1 -- 172.21.15.67:0/1699596590 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f719c005050 con 0x5649b47865d0 2024-03-30T02:43:35.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.598+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f719c004510 con 0x5649b47865d0 2024-03-30T02:43:35.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.598+0000 7f718fe00700 1 -- 172.21.15.67:0/1699596590 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f719c005210 con 0x5649b47865d0 2024-03-30T02:43:35.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.599+0000 7f71c4bf8b80 10 client.26969 fetch_fsmap learned FSMap version 37 2024-03-30T02:43:35.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.599+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5649b455ecd0 con 0x5649b47865d0 2024-03-30T02:43:35.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.599+0000 7f718fe00700 1 -- 172.21.15.67:0/1699596590 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f719c004510 con 0x5649b47865d0 2024-03-30T02:43:35.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.600+0000 7f718fe00700 1 -- 172.21.15.67:0/1699596590 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f719c047070 con 0x5649b47865d0 2024-03-30T02:43:35.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.600+0000 7f71c4bf8b80 10 client.26969 fetch_fsmap finished waiting for FSMap version 37 2024-03-30T02:43:35.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.600+0000 7f71c4bf8b80 10 client.26969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:43:35.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.600+0000 7f71c4bf8b80 20 client.26969 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:35.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.600+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5649b47f64f0 0x5649b47f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:35.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.600+0000 7f71c4bf8b80 4 client.26969 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:43:35.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.600+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5649b4527cf0 con 0x5649b47f64f0 2024-03-30T02:43:35.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.601+0000 7f71a9400700 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5649b47f64f0 0x5649b47f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:35.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.602+0000 7f71a9400700 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5649b47f64f0 0x5649b47f8910 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85cac858712587dd server_cookie=516325f7c3183bc5 in_seq=0 out_seq=0 2024-03-30T02:43:35.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.602+0000 7f718fe00700 10 client.26969 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:35.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.602+0000 7f718fe00700 1 -- 172.21.15.67:0/1699596590 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5649b4527cf0 con 0x5649b47f64f0 2024-03-30T02:43:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.602+0000 7f718fe00700 10 client.26969 handle_command_reply: tid=0 2024-03-30T02:43:35.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.602+0000 7f71c4bf8b80 1 client.26969 shutdown 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 msgr2=0x5649b4796b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4796b70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f719c035600 tx=0x7f719c000bc0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 shutdown_connections 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5649b4791df0 0x5649b47ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5649b47865d0 0x5649b4796b70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5649b47f64f0 0x5649b47f8910 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f71c4bf8b80 1 --2- 172.21.15.67:0/1699596590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5649b4786ef0 0x5649b47970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 >> 172.21.15.67:0/1699596590 conn(0x5649b46e66a0 msgr2=0x5649b478d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f718fe00700 0 client.26969 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.603+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 shutdown_connections 2024-03-30T02:43:35.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.604+0000 7f71c4bf8b80 1 -- 172.21.15.67:0/1699596590 wait complete. 2024-03-30T02:43:35.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.613+0000 7f71c4bf8b80 20 client.26969 trim_cache size 0 max 16384 2024-03-30T02:43:35.613 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:43:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 -- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71900a1310 msgr2=0x7f71900a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71900a1310 0x7f71900a37d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007fd0 tx=0x7f71a4009040 comp rx=0 tx=0).stop 2024-03-30T02:43:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 -- 172.21.15.67:0/820760784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac11a590 msgr2=0x7f71ac1a31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac11a590 0x7f71ac1a31f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7194009230 tx=0x7f719400f9e0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 -- 172.21.15.67:0/820760784 shutdown_connections 2024-03-30T02:43:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f71900a1310 0x7f71900a37d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71ac11a590 0x7f71ac1a31f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71ac1082f0 0x7f71ac115ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 --2- 172.21.15.67:0/820760784 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f71ac10f540 0x7f71ac116020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.617+0000 7f71b1000700 1 -- 172.21.15.67:0/820760784 >> 172.21.15.67:0/820760784 conn(0x7f71ac06bdf0 msgr2=0x7f71ac11a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.618+0000 7f71b1000700 1 -- 172.21.15.67:0/820760784 shutdown_connections 2024-03-30T02:43:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:35.618+0000 7f71b1000700 1 -- 172.21.15.67:0/820760784 wait complete. 2024-03-30T02:43:35.763 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/list_lru.o 2024-03-30T02:43:35.911 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/intel/cstate.o 2024-03-30T02:43:35.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:35 smithi067 ceph-mon[37808]: pgmap v199: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.2 MiB/s wr, 261 op/s 2024-03-30T02:43:35.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:35 smithi170 ceph-mon[78798]: pgmap v199: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.2 MiB/s wr, 261 op/s 2024-03-30T02:43:35.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:35 smithi082 ceph-mon[79176]: pgmap v199: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.2 MiB/s wr, 261 op/s 2024-03-30T02:43:36.012 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/tlb.o 2024-03-30T02:43:36.110 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/mtrr/cleanup.o 2024-03-30T02:43:36.207 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/interrupts.o 2024-03-30T02:43:36.408 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/printk/printk_safe.o 2024-03-30T02:43:36.626 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/akcipher.o 2024-03-30T02:43:36.679 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:43:36.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:43:36.698 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/workingset.o 2024-03-30T02:43:36.793 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/ss/context.o 2024-03-30T02:43:36.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:36 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:36.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:36 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:36.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:36 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:37.011 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/loadavg.o 2024-03-30T02:43:37.258 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/cpu/mtrr/built-in.a 2024-03-30T02:43:37.402 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/microcode/core.o 2024-03-30T02:43:37.407 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/cpu_entry_area.o 2024-03-30T02:43:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.538+0000 7ff621000700 1 -- 172.21.15.67:0/259377972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff61c11a590 msgr2=0x7ff61c11c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.538+0000 7ff621000700 1 --2- 172.21.15.67:0/259377972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff61c11a590 0x7ff61c11c980 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff6080099c0 tx=0x7ff608009cd0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.542+0000 7ff621000700 1 -- 172.21.15.67:0/259377972 shutdown_connections 2024-03-30T02:43:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.542+0000 7ff621000700 1 --2- 172.21.15.67:0/259377972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff61c10f540 0x7ff61c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.542+0000 7ff621000700 1 --2- 172.21.15.67:0/259377972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff61c11a590 0x7ff61c11c980 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.542+0000 7ff621000700 1 --2- 172.21.15.67:0/259377972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff61c1082f0 0x7ff61c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.542+0000 7ff621000700 1 -- 172.21.15.67:0/259377972 >> 172.21.15.67:0/259377972 conn(0x7ff61c06bdf0 msgr2=0x7ff61c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.546+0000 7ff621000700 1 -- 172.21.15.67:0/259377972 shutdown_connections 2024-03-30T02:43:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.546+0000 7ff621000700 1 -- 172.21.15.67:0/259377972 wait complete. 2024-03-30T02:43:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.551+0000 7ff621000700 1 Processor -- start 2024-03-30T02:43:37.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.553+0000 7ff621000700 1 -- start start 2024-03-30T02:43:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff621000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff61c1082f0 0x7ff61c1a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff621000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff61c10f540 0x7ff61c1a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff621000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff61c11a590 0x7ff61c1aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff621000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c1aafa0 con 0x7ff61c10f540 2024-03-30T02:43:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff621000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c1ab110 con 0x7ff61c1082f0 2024-03-30T02:43:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff621000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c1ab2d0 con 0x7ff61c11a590 2024-03-30T02:43:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff619e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff61c1082f0 0x7ff61c1a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff619e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff61c1082f0 0x7ff61c1a6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:59026/0 (socket says 172.21.15.67:59026) 2024-03-30T02:43:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff619e00700 1 -- 172.21.15.67:0/3274654433 learned_addr learned my addr 172.21.15.67:0/3274654433 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.557+0000 7ff619e00700 1 -- 172.21.15.67:0/3274654433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff61c11a590 msgr2=0x7ff61c1aa960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:43:37.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.558+0000 7ff619e00700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff61c11a590 0x7ff61c1aa960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.558+0000 7ff619e00700 1 -- 172.21.15.67:0/3274654433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff61c10f540 msgr2=0x7ff61c1a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.558+0000 7ff619e00700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff61c10f540 0x7ff61c1a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.558+0000 7ff619e00700 1 -- 172.21.15.67:0/3274654433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080095f0 con 0x7ff61c1082f0 2024-03-30T02:43:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.558+0000 7ff619e00700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff61c1082f0 0x7ff61c1a6080 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b530 tx=0x7ff60c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.559+0000 7ff612a00700 1 -- 172.21.15.67:0/3274654433 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c011830 con 0x7ff61c1082f0 2024-03-30T02:43:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.559+0000 7ff621000700 1 -- 172.21.15.67:0/3274654433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c1ab5b0 con 0x7ff61c1082f0 2024-03-30T02:43:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.559+0000 7ff621000700 1 -- 172.21.15.67:0/3274654433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c1b2df0 con 0x7ff61c1082f0 2024-03-30T02:43:37.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.561+0000 7ff612a00700 1 -- 172.21.15.67:0/3274654433 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff60c011e50 con 0x7ff61c1082f0 2024-03-30T02:43:37.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.561+0000 7ff612a00700 1 -- 172.21.15.67:0/3274654433 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c010e70 con 0x7ff61c1082f0 2024-03-30T02:43:37.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.571+0000 7ff612a00700 1 -- 172.21.15.67:0/3274654433 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff60c01a400 con 0x7ff61c1082f0 2024-03-30T02:43:37.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.574+0000 7ff612a00700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff6000a13d0 0x7ff6000a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.574+0000 7ff612a00700 1 -- 172.21.15.67:0/3274654433 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff60c0cd970 con 0x7ff61c1082f0 2024-03-30T02:43:37.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.574+0000 7ff619400700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff6000a13d0 0x7ff6000a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.576+0000 7ff619400700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff6000a13d0 0x7ff6000a3890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff61c1a75d0 tx=0x7ff614009230 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:37.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.576+0000 7ff634ae1b80 1 Processor -- start 2024-03-30T02:43:37.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.586+0000 7ff634ae1b80 1 -- start start 2024-03-30T02:43:37.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.587+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd857b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.587+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.587+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.587+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8570ba10 con 0x55fd857b2650 2024-03-30T02:43:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.587+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd857093d0 con 0x55fd857b2f70 2024-03-30T02:43:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.587+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd85704cc0 con 0x55fd857bde70 2024-03-30T02:43:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.588+0000 7ff61a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.588+0000 7ff61a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857c0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47698/0 (socket says 172.21.15.67:47698) 2024-03-30T02:43:37.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.588+0000 7ff61a800700 1 -- 172.21.15.67:0/1729438865 learned_addr learned my addr 172.21.15.67:0/1729438865 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:37.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.588+0000 7ff619400700 1 --2- 172.21.15.67:0/1729438865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.589+0000 7ff61a800700 1 -- 172.21.15.67:0/1729438865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 msgr2=0x55fd857bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.589+0000 7ff61a800700 1 --2- 172.21.15.67:0/1729438865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.589+0000 7ff61a800700 1 -- 172.21.15.67:0/1729438865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 msgr2=0x55fd857b2a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:43:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.589+0000 7ff61a800700 1 --2- 172.21.15.67:0/1729438865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd857b2a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.589+0000 7ff61a800700 1 -- 172.21.15.67:0/1729438865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd85586430 con 0x55fd857bde70 2024-03-30T02:43:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.589+0000 7ff619400700 1 --2- 172.21.15.67:0/1729438865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:37.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.590+0000 7ff61a800700 1 --2- 172.21.15.67:0/1729438865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857c0260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff608005c20 tx=0x7ff608005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d26f38f5be742b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.594+0000 7ff607400700 1 -- 172.21.15.67:0/1729438865 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60801d070 con 0x55fd857bde70 2024-03-30T02:43:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.594+0000 7ff607400700 1 -- 172.21.15.67:0/1729438865 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff608005590 con 0x55fd857bde70 2024-03-30T02:43:37.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.594+0000 7ff607400700 1 -- 172.21.15.67:0/1729438865 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800fa70 con 0x55fd857bde70 2024-03-30T02:43:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.595+0000 7ff634ae1b80 1 -- 172.21.15.67:0/1729438865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 msgr2=0x55fd857c0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:37.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.595+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/1729438865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857c0260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff608005c20 tx=0x7ff608005c50 comp rx=0 tx=0).stop 2024-03-30T02:43:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.596+0000 7ff634ae1b80 1 -- 172.21.15.67:0/1729438865 shutdown_connections 2024-03-30T02:43:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.596+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/1729438865 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd857b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.596+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/1729438865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857c0260 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.596+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/1729438865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.596+0000 7ff634ae1b80 1 -- 172.21.15.67:0/1729438865 >> 172.21.15.67:0/1729438865 conn(0x55fd85712720 msgr2=0x55fd857aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:37.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.600+0000 7ff634ae1b80 1 -- 172.21.15.67:0/1729438865 shutdown_connections 2024-03-30T02:43:37.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.604+0000 7ff634ae1b80 1 -- 172.21.15.67:0/1729438865 wait complete. 2024-03-30T02:43:37.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.605+0000 7ff634ae1b80 1 -- start start 2024-03-30T02:43:37.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.605+0000 7ff634ae1b80 1 Processor -- start 2024-03-30T02:43:37.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.610+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2650 0x55fd857bd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.610+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.610+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857bde70 0x55fd857bb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.610+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd856a0e00 con 0x55fd857bde70 2024-03-30T02:43:37.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.610+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd856a0aa0 con 0x55fd857b2f70 2024-03-30T02:43:37.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.610+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd85588720 con 0x55fd857b2650 2024-03-30T02:43:37.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.610+0000 7ff634ae1b80 10 client.0 _ll_register_callbacks cb 0x55fd856aa4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:37.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.613+0000 7ff619400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.613+0000 7ff619400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:59034/0 (socket says 172.21.15.67:59034) 2024-03-30T02:43:37.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.613+0000 7ff619400700 1 -- 172.21.15.67:0/3303091011 learned_addr learned my addr 172.21.15.67:0/3303091011 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:37.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.613+0000 7ff619400700 1 -- 172.21.15.67:0/3303091011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2650 msgr2=0x55fd857bd8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:43:37.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.613+0000 7ff619400700 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2650 0x55fd857bd8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.613+0000 7ff619400700 1 -- 172.21.15.67:0/3303091011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857bde70 msgr2=0x55fd857bb370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:43:37.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.613+0000 7ff619400700 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857bde70 0x55fd857bb370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.613+0000 7ff619400700 1 -- 172.21.15.67:0/3303091011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c11d280 con 0x55fd857b2f70 2024-03-30T02:43:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.614+0000 7ff619400700 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bac30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff614019b00 tx=0x7ff614004bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e67e2229abf4db0f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.614+0000 7ff5fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:43:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.614+0000 7ff5fbe00700 1 -- 172.21.15.67:0/3303091011 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614009ad0 con 0x55fd857b2f70 2024-03-30T02:43:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.614+0000 7ff5fbe00700 1 -- 172.21.15.67:0/3303091011 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff614009c30 con 0x55fd857b2f70 2024-03-30T02:43:37.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.614+0000 7ff634ae1b80 1 -- 172.21.15.67:0/3303091011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd856c5400 con 0x55fd857b2f70 2024-03-30T02:43:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.615+0000 7ff5fbe00700 1 -- 172.21.15.67:0/3303091011 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff614009df0 con 0x55fd857b2f70 2024-03-30T02:43:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.615+0000 7ff634ae1b80 10 client.26978 fetch_fsmap learned FSMap version 37 2024-03-30T02:43:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.615+0000 7ff634ae1b80 1 -- 172.21.15.67:0/3303091011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55fd85532be0 con 0x55fd857b2f70 2024-03-30T02:43:37.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.616+0000 7ff5fbe00700 1 -- 172.21.15.67:0/3303091011 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61401e070 con 0x55fd857b2f70 2024-03-30T02:43:37.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.617+0000 7ff5fbe00700 1 -- 172.21.15.67:0/3303091011 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7ff61401a4c0 con 0x55fd857b2f70 2024-03-30T02:43:37.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.619+0000 7ff634ae1b80 10 client.26978 fetch_fsmap finished waiting for FSMap version 37 2024-03-30T02:43:37.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.619+0000 7ff634ae1b80 10 client.26978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:43:37.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.619+0000 7ff634ae1b80 20 client.26978 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:37.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.619+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fd857cd8d0 0x55fd857cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.619+0000 7ff634ae1b80 4 client.26978 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:43:37.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.619+0000 7ff634ae1b80 1 -- 172.21.15.67:0/3303091011 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd856c5400 con 0x55fd857cd8d0 2024-03-30T02:43:37.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.619+0000 7ff619e00700 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fd857cd8d0 0x55fd857cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.620+0000 7ff5fbe00700 10 client.26978 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:37.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.620+0000 7ff619e00700 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fd857cd8d0 0x55fd857cfcf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432c73bb2a13708e server_cookie=8cb547bf702017bb in_seq=0 out_seq=0 2024-03-30T02:43:37.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.620+0000 7ff5fbe00700 1 -- 172.21.15.67:0/3303091011 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fd856c5400 con 0x55fd857cd8d0 2024-03-30T02:43:37.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.621+0000 7ff5fbe00700 10 client.26978 handle_command_reply: tid=0 2024-03-30T02:43:37.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.621+0000 7ff634ae1b80 1 client.26978 shutdown 2024-03-30T02:43:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.623+0000 7ff634ae1b80 1 -- 172.21.15.67:0/3303091011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 msgr2=0x55fd857bac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:37.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.623+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bac30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff614019b00 tx=0x7ff614004bf0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.624+0000 7ff634ae1b80 1 -- 172.21.15.67:0/3303091011 shutdown_connections 2024-03-30T02:43:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.624+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fd857cd8d0 0x55fd857cfcf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.624+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857bde70 0x55fd857bb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.624+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2650 0x55fd857bd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.624+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/3303091011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857bac30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.624+0000 7ff5fbe00700 0 client.26978 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.624+0000 7ff634ae1b80 1 -- 172.21.15.67:0/3303091011 >> 172.21.15.67:0/3303091011 conn(0x55fd85712720 msgr2=0x55fd857b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:37.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.625+0000 7ff634ae1b80 1 -- 172.21.15.67:0/3303091011 shutdown_connections 2024-03-30T02:43:37.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.625+0000 7ff634ae1b80 1 -- 172.21.15.67:0/3303091011 wait complete. 2024-03-30T02:43:37.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.627+0000 7ff634ae1b80 20 client.26978 trim_cache size 0 max 16384 2024-03-30T02:43:37.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.687+0000 7ff634ae1b80 1 Processor -- start 2024-03-30T02:43:37.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.692+0000 7ff634ae1b80 1 -- start start 2024-03-30T02:43:37.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.692+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd857c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.692+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.692+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857baa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.692+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd857baf70 con 0x55fd857b2650 2024-03-30T02:43:37.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.692+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd857a8670 con 0x55fd857b2f70 2024-03-30T02:43:37.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.692+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd857bbe70 con 0x55fd857bde70 2024-03-30T02:43:37.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.693+0000 7ff61a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857baa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.693+0000 7ff61a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857baa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47724/0 (socket says 172.21.15.67:47724) 2024-03-30T02:43:37.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.693+0000 7ff61a800700 1 -- 172.21.15.67:0/2206965803 learned_addr learned my addr 172.21.15.67:0/2206965803 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:37.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.693+0000 7ff619400700 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.693+0000 7ff619e00700 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd857c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.693+0000 7ff619400700 1 -- 172.21.15.67:0/2206965803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 msgr2=0x55fd857baa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:37.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.693+0000 7ff619400700 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857baa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.694+0000 7ff619400700 1 -- 172.21.15.67:0/2206965803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 msgr2=0x55fd857c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.694+0000 7ff619400700 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd857c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.694+0000 7ff619400700 1 -- 172.21.15.67:0/2206965803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd857d4960 con 0x55fd857b2f70 2024-03-30T02:43:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.694+0000 7ff619e00700 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd857c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:37.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.694+0000 7ff61a800700 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857baa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:43:37.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.696+0000 7ff619400700 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857ba4f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff614009660 tx=0x7ff614009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1addc0413d7b74a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:37.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.696+0000 7ff607e00700 1 -- 172.21.15.67:0/2206965803 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614033070 con 0x55fd857b2f70 2024-03-30T02:43:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.700+0000 7ff607e00700 1 -- 172.21.15.67:0/2206965803 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61401e040 con 0x55fd857b2f70 2024-03-30T02:43:37.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.700+0000 7ff607e00700 1 -- 172.21.15.67:0/2206965803 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614009df0 con 0x55fd857b2f70 2024-03-30T02:43:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.702+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2206965803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 msgr2=0x55fd857ba4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:37.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.702+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857ba4f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff614009660 tx=0x7ff614009790 comp rx=0 tx=0).stop 2024-03-30T02:43:37.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.705+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2206965803 shutdown_connections 2024-03-30T02:43:37.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.705+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd857c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.705+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857bde70 0x55fd857baa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.705+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2206965803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857b2f70 0x55fd857ba4f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.705+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2206965803 >> 172.21.15.67:0/2206965803 conn(0x55fd85712720 msgr2=0x55fd857a7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:37.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.706+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2206965803 shutdown_connections 2024-03-30T02:43:37.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.706+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2206965803 wait complete. 2024-03-30T02:43:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.707+0000 7ff634ae1b80 1 -- start start 2024-03-30T02:43:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.707+0000 7ff634ae1b80 1 Processor -- start 2024-03-30T02:43:37.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.708+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd8581ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.711+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2f70 0x55fd857c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.711+0000 7ff634ae1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857bde70 0x55fd857c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.711+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8581d4a0 con 0x55fd857b2650 2024-03-30T02:43:37.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.711+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd857c44a0 con 0x55fd857bde70 2024-03-30T02:43:37.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.711+0000 7ff634ae1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd857c4660 con 0x55fd857b2f70 2024-03-30T02:43:37.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.711+0000 7ff634ae1b80 10 client.0 _ll_register_callbacks cb 0x55fd856aa4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.712+0000 7ff619400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2f70 0x55fd857c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.712+0000 7ff619400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2f70 0x55fd857c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47740/0 (socket says 172.21.15.67:47740) 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.712+0000 7ff619400700 1 -- 172.21.15.67:0/2299441142 learned_addr learned my addr 172.21.15.67:0/2299441142 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.712+0000 7ff61a800700 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857bde70 0x55fd857c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.712+0000 7ff619e00700 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd8581ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.713+0000 7ff619400700 1 -- 172.21.15.67:0/2299441142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857bde70 msgr2=0x55fd857c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.713+0000 7ff619400700 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857bde70 0x55fd857c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.713+0000 7ff619400700 1 -- 172.21.15.67:0/2299441142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 msgr2=0x55fd8581ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.713+0000 7ff619400700 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd8581ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.713+0000 7ff619400700 1 -- 172.21.15.67:0/2299441142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd85532be0 con 0x55fd857b2f70 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.713+0000 7ff619e00700 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd8581ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:43:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.713+0000 7ff619400700 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2f70 0x55fd857c3a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff614007140 tx=0x7ff61400ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=126459eeab075ef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:37.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.713+0000 7ff5fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:43:37.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.714+0000 7ff5fbe00700 1 -- 172.21.15.67:0/2299441142 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61403c070 con 0x55fd857b2f70 2024-03-30T02:43:37.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.714+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2299441142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff608005590 con 0x55fd857b2f70 2024-03-30T02:43:37.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.714+0000 7ff5fbe00700 1 -- 172.21.15.67:0/2299441142 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7ff614003ac0 con 0x55fd857b2f70 2024-03-30T02:43:37.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.715+0000 7ff5fbe00700 1 -- 172.21.15.67:0/2299441142 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff614033070 con 0x55fd857b2f70 2024-03-30T02:43:37.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.717+0000 7ff634ae1b80 10 client.26935 fetch_fsmap learned FSMap version 37 2024-03-30T02:43:37.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.717+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2299441142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55fd856caf20 con 0x55fd857b2f70 2024-03-30T02:43:37.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.718+0000 7ff5fbe00700 1 -- 172.21.15.67:0/2299441142 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff608005590 con 0x55fd857b2f70 2024-03-30T02:43:37.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.719+0000 7ff5fbe00700 1 -- 172.21.15.67:0/2299441142 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7ff61403f020 con 0x55fd857b2f70 2024-03-30T02:43:37.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.719+0000 7ff634ae1b80 10 client.26935 fetch_fsmap finished waiting for FSMap version 37 2024-03-30T02:43:37.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.719+0000 7ff634ae1b80 10 client.26935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:43:37.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.719+0000 7ff634ae1b80 20 client.26935 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:37.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.719+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fd858347f0 0x55fd85836be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:37.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.719+0000 7ff634ae1b80 4 client.26935 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:43:37.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.719+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2299441142 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fd856fd780 con 0x55fd858347f0 2024-03-30T02:43:37.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.719+0000 7ff619e00700 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fd858347f0 0x55fd85836be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:37.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.720+0000 7ff619e00700 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fd858347f0 0x55fd85836be0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866cafdaf96a3bbc server_cookie=2897b74b5ea74b43 in_seq=0 out_seq=0 2024-03-30T02:43:37.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.720+0000 7ff5fbe00700 10 client.26935 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff5fbe00700 1 -- 172.21.15.67:0/2299441142 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fd856fd780 con 0x55fd858347f0 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff5fbe00700 10 client.26935 handle_command_reply: tid=0 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff634ae1b80 1 client.26935 shutdown 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2299441142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2f70 msgr2=0x55fd857c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2f70 0x55fd857c3a20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff614007140 tx=0x7ff61400ca20 comp rx=0 tx=0).stop 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2299441142 shutdown_connections 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55fd858347f0 0x55fd85836be0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd857b2650 0x55fd8581ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd857bde70 0x55fd857c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff634ae1b80 1 --2- 172.21.15.67:0/2299441142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd857b2f70 0x55fd857c3a20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff5fbe00700 0 client.26935 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.724+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2299441142 >> 172.21.15.67:0/2299441142 conn(0x55fd85712720 msgr2=0x55fd857b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.726+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2299441142 shutdown_connections 2024-03-30T02:43:37.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.726+0000 7ff634ae1b80 1 -- 172.21.15.67:0/2299441142 wait complete. 2024-03-30T02:43:37.730 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:43:37.730 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:43:37.730 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "32d087aa-2761-4165-913b-6efa530e6563", 2024-03-30T02:43:37.730 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:43:37.730 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:43:37.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.729+0000 7ff634ae1b80 20 client.26935 trim_cache size 0 max 16384 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.737+0000 7ff621000700 1 -- 172.21.15.67:0/3274654433 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff6000a13d0 msgr2=0x7ff6000a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.737+0000 7ff621000700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff6000a13d0 0x7ff6000a3890 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff61c1a75d0 tx=0x7ff614009230 comp rx=0 tx=0).stop 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.737+0000 7ff621000700 1 -- 172.21.15.67:0/3274654433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff61c1082f0 msgr2=0x7ff61c1a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.737+0000 7ff621000700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff61c1082f0 0x7ff61c1a6080 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b530 tx=0x7ff60c00b8f0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.738+0000 7ff621000700 1 -- 172.21.15.67:0/3274654433 shutdown_connections 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.738+0000 7ff621000700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff6000a13d0 0x7ff6000a3890 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.738+0000 7ff621000700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff61c11a590 0x7ff61c1aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.738+0000 7ff621000700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff61c1082f0 0x7ff61c1a6080 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.738+0000 7ff621000700 1 --2- 172.21.15.67:0/3274654433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff61c10f540 0x7ff61c1a65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:37.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.738+0000 7ff621000700 1 -- 172.21.15.67:0/3274654433 >> 172.21.15.67:0/3274654433 conn(0x7ff61c06bdf0 msgr2=0x7ff61c118580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:37.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.743+0000 7ff621000700 1 -- 172.21.15.67:0/3274654433 shutdown_connections 2024-03-30T02:43:37.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:37.743+0000 7ff621000700 1 -- 172.21.15.67:0/3274654433 wait complete. 2024-03-30T02:43:37.744 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/events/intel/built-in.a 2024-03-30T02:43:37.758 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/printk/printk_ringbuffer.o 2024-03-30T02:43:37.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:43:37.909 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/zhaoxin/core.o 2024-03-30T02:43:38.006 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/sig.o 2024-03-30T02:43:38.060 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/meminfo.o 2024-03-30T02:43:38.279 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/debug.o 2024-03-30T02:43:38.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.386+0000 7f534b200700 1 -- 172.21.15.67:0/3462416358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c11a240 msgr2=0x7f534c11c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:38.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.386+0000 7f534b200700 1 --2- 172.21.15.67:0/3462416358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c11a240 0x7f534c11c630 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f534400b0a0 tx=0x7f534400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.388+0000 7f534b200700 1 -- 172.21.15.67:0/3462416358 shutdown_connections 2024-03-30T02:43:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.388+0000 7f534b200700 1 --2- 172.21.15.67:0/3462416358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534c10ce90 0x7f534c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.388+0000 7f534b200700 1 --2- 172.21.15.67:0/3462416358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c11a240 0x7f534c11c630 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.388+0000 7f534b200700 1 --2- 172.21.15.67:0/3462416358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f534c10d7b0 0x7f534c119d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.388+0000 7f534b200700 1 -- 172.21.15.67:0/3462416358 >> 172.21.15.67:0/3462416358 conn(0x7f534c06bc10 msgr2=0x7f534c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.390+0000 7f534b200700 1 -- 172.21.15.67:0/3462416358 shutdown_connections 2024-03-30T02:43:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.390+0000 7f534b200700 1 -- 172.21.15.67:0/3462416358 wait complete. 2024-03-30T02:43:38.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.391+0000 7f534b200700 1 Processor -- start 2024-03-30T02:43:38.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.393+0000 7f534b200700 1 -- start start 2024-03-30T02:43:38.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.394+0000 7f534b200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534c10ce90 0x7f534c1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.394+0000 7f534b200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c10d7b0 0x7f534c1a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.395+0000 7f534b200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f534c11a240 0x7f534c1aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.395+0000 7f5349400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c10d7b0 0x7f534c1a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.395+0000 7f534a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f534c11a240 0x7f534c1aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.395+0000 7f534a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f534c11a240 0x7f534c1aa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:59078/0 (socket says 172.21.15.67:59078) 2024-03-30T02:43:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.395+0000 7f5349400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c10d7b0 0x7f534c1a6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47766/0 (socket says 172.21.15.67:47766) 2024-03-30T02:43:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.395+0000 7f534a800700 1 -- 172.21.15.67:0/3586852514 learned_addr learned my addr 172.21.15.67:0/3586852514 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.395+0000 7f5349e00700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534c10ce90 0x7f534c1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.396+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c1aade0 con 0x7f534c10ce90 2024-03-30T02:43:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.396+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c1aaf50 con 0x7f534c11a240 2024-03-30T02:43:38.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.396+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c1ab110 con 0x7f534c10d7b0 2024-03-30T02:43:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.396+0000 7f534a800700 1 -- 172.21.15.67:0/3586852514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c10d7b0 msgr2=0x7f534c1a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.396+0000 7f534a800700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c10d7b0 0x7f534c1a6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.396+0000 7f534a800700 1 -- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534c10ce90 msgr2=0x7f534c1a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.396+0000 7f534a800700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534c10ce90 0x7f534c1a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.396+0000 7f534a800700 1 -- 172.21.15.67:0/3586852514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344009d00 con 0x7f534c11a240 2024-03-30T02:43:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.397+0000 7f534a800700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f534c11a240 0x7f534c1aa7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f534400b070 tx=0x7f534400bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.397+0000 7f5342a00700 1 -- 172.21.15.67:0/3586852514 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534400e040 con 0x7f534c11a240 2024-03-30T02:43:38.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.398+0000 7f5342a00700 1 -- 172.21.15.67:0/3586852514 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53440094b0 con 0x7f534c11a240 2024-03-30T02:43:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.399+0000 7f5342a00700 1 -- 172.21.15.67:0/3586852514 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534401b6c0 con 0x7f534c11a240 2024-03-30T02:43:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.399+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c1ab390 con 0x7f534c11a240 2024-03-30T02:43:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.399+0000 7f5349e00700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534c10ce90 0x7f534c1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:38.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.399+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c1ab5e0 con 0x7f534c11a240 2024-03-30T02:43:38.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.401+0000 7f5349400700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c10d7b0 0x7f534c1a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:38.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.411+0000 7f5364476b80 1 Processor -- start 2024-03-30T02:43:38.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.412+0000 7f5342a00700 1 -- 172.21.15.67:0/3586852514 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f534401d470 con 0x7f534c11a240 2024-03-30T02:43:38.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.416+0000 7f5342a00700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53300a11f0 0x7f53300a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.416+0000 7f5364476b80 1 -- start start 2024-03-30T02:43:38.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.417+0000 7f5364476b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b3eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.418+0000 7f5349e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b3eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.418+0000 7f5349e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b3eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41264/0 (socket says 172.21.15.67:41264) 2024-03-30T02:43:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.419+0000 7f5349e00700 1 -- 172.21.15.67:0/1717334149 learned_addr learned my addr 172.21.15.67:0/1717334149 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.419+0000 7f5364476b80 1 --2- 172.21.15.67:0/1717334149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b3f0c0 0x55f896b49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.420+0000 7f5364476b80 1 --2- 172.21.15.67:0/1717334149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b49fc0 0x55f896b4c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.420+0000 7f5364476b80 1 -- 172.21.15.67:0/1717334149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896a97990 con 0x55f896b3e7a0 2024-03-30T02:43:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.420+0000 7f5364476b80 1 -- 172.21.15.67:0/1717334149 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896a95350 con 0x55f896b3f0c0 2024-03-30T02:43:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.420+0000 7f5364476b80 1 -- 172.21.15.67:0/1717334149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896a90c40 con 0x55f896b49fc0 2024-03-30T02:43:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.421+0000 7f5349e00700 1 -- 172.21.15.67:0/1717334149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b49fc0 msgr2=0x55f896b4c3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:43:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.421+0000 7f5349e00700 1 --2- 172.21.15.67:0/1717334149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b49fc0 0x55f896b4c3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.421+0000 7f5349e00700 1 -- 172.21.15.67:0/1717334149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b3f0c0 msgr2=0x55f896b49a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:43:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.421+0000 7f5349e00700 1 --2- 172.21.15.67:0/1717334149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b3f0c0 0x55f896b49a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.421+0000 7f5349e00700 1 -- 172.21.15.67:0/1717334149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8969123b0 con 0x55f896b3e7a0 2024-03-30T02:43:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.423+0000 7f5349e00700 1 --2- 172.21.15.67:0/1717334149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b3eb80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f533800bca0 tx=0x7f533800d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b30ace3a49327d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.423+0000 7f5349400700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53300a11f0 0x7f53300a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.424+0000 7f5342a00700 1 -- 172.21.15.67:0/3586852514 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f53440cdae0 con 0x7f534c11a240 2024-03-30T02:43:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.425+0000 7f5337000700 1 -- 172.21.15.67:0/1717334149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533800de80 con 0x55f896b3e7a0 2024-03-30T02:43:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.425+0000 7f5337000700 1 -- 172.21.15.67:0/1717334149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53380178e0 con 0x55f896b3e7a0 2024-03-30T02:43:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.425+0000 7f5337000700 1 -- 172.21.15.67:0/1717334149 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533800fac0 con 0x55f896b3e7a0 2024-03-30T02:43:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.426+0000 7f5364476b80 1 -- 172.21.15.67:0/1717334149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 msgr2=0x55f896b3eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:38.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.426+0000 7f5364476b80 1 --2- 172.21.15.67:0/1717334149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b3eb80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f533800bca0 tx=0x7f533800d340 comp rx=0 tx=0).stop 2024-03-30T02:43:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.427+0000 7f5349400700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53300a11f0 0x7f53300a36b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f534c1a7410 tx=0x7f533c009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.429+0000 7f5364476b80 1 -- 172.21.15.67:0/1717334149 shutdown_connections 2024-03-30T02:43:38.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.429+0000 7f5364476b80 1 --2- 172.21.15.67:0/1717334149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b3eb80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.429+0000 7f5364476b80 1 --2- 172.21.15.67:0/1717334149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b49fc0 0x55f896b4c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.429+0000 7f5364476b80 1 --2- 172.21.15.67:0/1717334149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b3f0c0 0x55f896b49a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.429+0000 7f5364476b80 1 -- 172.21.15.67:0/1717334149 >> 172.21.15.67:0/1717334149 conn(0x55f896a9e6a0 msgr2=0x55f896b3ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:38.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.430+0000 7f5364476b80 1 -- 172.21.15.67:0/1717334149 shutdown_connections 2024-03-30T02:43:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.430+0000 7f5364476b80 1 -- 172.21.15.67:0/1717334149 wait complete. 2024-03-30T02:43:38.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.431+0000 7f5364476b80 1 -- start start 2024-03-30T02:43:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.432+0000 7f5364476b80 1 Processor -- start 2024-03-30T02:43:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.432+0000 7f5364476b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b49a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.433+0000 7f5364476b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3f0c0 0x55f896b46de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.433+0000 7f5349e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b49a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.433+0000 7f5364476b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b47520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.433+0000 7f5349e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b49a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41280/0 (socket says 172.21.15.67:41280) 2024-03-30T02:43:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.433+0000 7f5349400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3f0c0 0x55f896b46de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.433+0000 7f5349400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3f0c0 0x55f896b46de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47776/0 (socket says 172.21.15.67:47776) 2024-03-30T02:43:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.434+0000 7f5349e00700 1 -- 172.21.15.67:0/1513680129 learned_addr learned my addr 172.21.15.67:0/1513680129 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:38.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.434+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896a2cd80 con 0x55f896b3e7a0 2024-03-30T02:43:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.434+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896a2ca20 con 0x55f896b49fc0 2024-03-30T02:43:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.434+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8969146a0 con 0x55f896b3f0c0 2024-03-30T02:43:38.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.434+0000 7f5364476b80 10 client.0 _ll_register_callbacks cb 0x55f896a36470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.435+0000 7f5349e00700 1 -- 172.21.15.67:0/1513680129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3f0c0 msgr2=0x55f896b46de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.435+0000 7f5349e00700 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3f0c0 0x55f896b46de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.435+0000 7f5349e00700 1 -- 172.21.15.67:0/1513680129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 msgr2=0x55f896b47520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.435+0000 7f5349e00700 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b47520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.435+0000 7f5349e00700 1 -- 172.21.15.67:0/1513680129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533800b8d0 con 0x55f896b3e7a0 2024-03-30T02:43:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.438+0000 7f5349e00700 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b49a50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f533800d740 tx=0x7f5338017a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7d93371ca756b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.439+0000 7f532b400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:43:38.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.440+0000 7f532b400700 1 -- 172.21.15.67:0/1513680129 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5338017630 con 0x55f896b3e7a0 2024-03-30T02:43:38.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.440+0000 7f532b400700 1 -- 172.21.15.67:0/1513680129 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f5338013070 con 0x55f896b3e7a0 2024-03-30T02:43:38.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.441+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f896a51380 con 0x55f896b3e7a0 2024-03-30T02:43:38.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.442+0000 7f532b400700 1 -- 172.21.15.67:0/1513680129 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f533800f6e0 con 0x55f896b3e7a0 2024-03-30T02:43:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.444+0000 7f5364476b80 10 client.16878 fetch_fsmap learned FSMap version 37 2024-03-30T02:43:38.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.445+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f8968beb60 con 0x55f896b3e7a0 2024-03-30T02:43:38.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.449+0000 7f532b400700 1 -- 172.21.15.67:0/1513680129 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f896a51380 con 0x55f896b3e7a0 2024-03-30T02:43:38.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.449+0000 7f532b400700 1 -- 172.21.15.67:0/1513680129 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f533801f070 con 0x55f896b3e7a0 2024-03-30T02:43:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.450+0000 7f5364476b80 10 client.16878 fetch_fsmap finished waiting for FSMap version 37 2024-03-30T02:43:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.450+0000 7f5364476b80 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:43:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.450+0000 7f5364476b80 20 client.16878 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.450+0000 7f5364476b80 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f896b59b50 0x55f896b5bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.450+0000 7f5364476b80 4 client.16878 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:43:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.450+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f896a89700 con 0x55f896b59b50 2024-03-30T02:43:38.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.451+0000 7f5349400700 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f896b59b50 0x55f896b5bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.451+0000 7f5349400700 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f896b59b50 0x55f896b5bf70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17374b80019eac77 server_cookie=ae1d4f0553eb67e1 in_seq=0 out_seq=0 2024-03-30T02:43:38.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.451+0000 7f532b400700 10 client.16878 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:38.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.454+0000 7f532b400700 1 -- 172.21.15.67:0/1513680129 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f896a89700 con 0x55f896b59b50 2024-03-30T02:43:38.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.455+0000 7f532b400700 10 client.16878 handle_command_reply: tid=0 2024-03-30T02:43:38.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.455+0000 7f5364476b80 1 client.16878 shutdown 2024-03-30T02:43:38.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.456+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 msgr2=0x55f896b49a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:38.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.456+0000 7f5364476b80 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b49a50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f533800d740 tx=0x7f5338017a50 comp rx=0 tx=0).stop 2024-03-30T02:43:38.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.456+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 shutdown_connections 2024-03-30T02:43:38.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.456+0000 7f5364476b80 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f896b59b50 0x55f896b5bf70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.457+0000 7f5364476b80 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3e7a0 0x55f896b49a50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.457+0000 7f5364476b80 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b47520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.457+0000 7f5364476b80 1 --2- 172.21.15.67:0/1513680129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3f0c0 0x55f896b46de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.457+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 >> 172.21.15.67:0/1513680129 conn(0x55f896a9e6a0 msgr2=0x55f896b40840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:38.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.457+0000 7f532b400700 0 client.16878 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:38.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.457+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 shutdown_connections 2024-03-30T02:43:38.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.459+0000 7f5364476b80 1 -- 172.21.15.67:0/1513680129 wait complete. 2024-03-30T02:43:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.461+0000 7f5364476b80 20 client.16878 trim_cache size 0 max 16384 2024-03-30T02:43:38.490 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/microcode/intel.o 2024-03-30T02:43:38.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.515+0000 7f5364476b80 1 Processor -- start 2024-03-30T02:43:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.515+0000 7f5364476b80 1 -- start start 2024-03-30T02:43:38.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.516+0000 7f5364476b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.516+0000 7f5364476b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b49580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.517+0000 7f5349e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.517+0000 7f5349e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:47790/0 (socket says 172.21.15.67:47790) 2024-03-30T02:43:38.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.517+0000 7f5364476b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b46670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.517+0000 7f5349e00700 1 -- 172.21.15.67:0/2709278084 learned_addr learned my addr 172.21.15.67:0/2709278084 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:38.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.517+0000 7f5364476b80 1 -- 172.21.15.67:0/2709278084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896b3fa20 con 0x55f896b3f0c0 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.517+0000 7f5364476b80 1 -- 172.21.15.67:0/2709278084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896b47dd0 con 0x55f896b49fc0 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.517+0000 7f5364476b80 1 -- 172.21.15.67:0/2709278084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896b47b20 con 0x55f896b3e7a0 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.517+0000 7f534a800700 1 --2- 172.21.15.67:0/2709278084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b46670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.517+0000 7f5349400700 1 --2- 172.21.15.67:0/2709278084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b49580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.518+0000 7f534a800700 1 -- 172.21.15.67:0/2709278084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 msgr2=0x55f896b50c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.518+0000 7f534a800700 1 --2- 172.21.15.67:0/2709278084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.518+0000 7f534a800700 1 -- 172.21.15.67:0/2709278084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 msgr2=0x55f896b49580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.518+0000 7f534a800700 1 --2- 172.21.15.67:0/2709278084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b49580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.518+0000 7f534a800700 1 -- 172.21.15.67:0/2709278084 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f896a56ea0 con 0x55f896b49fc0 2024-03-30T02:43:38.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.518+0000 7f534a800700 1 --2- 172.21.15.67:0/2709278084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b46670 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53440caee0 tx=0x7f5344096d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cfd415f59ab2045 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:38.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.519+0000 7f5337a00700 1 -- 172.21.15.67:0/2709278084 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344085040 con 0x55f896b49fc0 2024-03-30T02:43:38.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.519+0000 7f5337a00700 1 -- 172.21.15.67:0/2709278084 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53440b8e90 con 0x55f896b49fc0 2024-03-30T02:43:38.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.519+0000 7f5337a00700 1 -- 172.21.15.67:0/2709278084 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344098db0 con 0x55f896b49fc0 2024-03-30T02:43:38.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.520+0000 7f5364476b80 1 -- 172.21.15.67:0/2709278084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 msgr2=0x55f896b46670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:38.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.520+0000 7f5364476b80 1 --2- 172.21.15.67:0/2709278084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b46670 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53440caee0 tx=0x7f5344096d80 comp rx=0 tx=0).stop 2024-03-30T02:43:38.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.521+0000 7f5364476b80 1 -- 172.21.15.67:0/2709278084 shutdown_connections 2024-03-30T02:43:38.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.521+0000 7f5364476b80 1 --2- 172.21.15.67:0/2709278084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b46670 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.521+0000 7f5364476b80 1 --2- 172.21.15.67:0/2709278084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.521+0000 7f5364476b80 1 --2- 172.21.15.67:0/2709278084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b49580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.521+0000 7f5364476b80 1 -- 172.21.15.67:0/2709278084 >> 172.21.15.67:0/2709278084 conn(0x55f896a9e6a0 msgr2=0x55f896b34010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:38.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.521+0000 7f5364476b80 1 -- 172.21.15.67:0/2709278084 shutdown_connections 2024-03-30T02:43:38.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.522+0000 7f5364476b80 1 -- 172.21.15.67:0/2709278084 wait complete. 2024-03-30T02:43:38.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.524+0000 7f5364476b80 1 -- start start 2024-03-30T02:43:38.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.524+0000 7f5364476b80 1 Processor -- start 2024-03-30T02:43:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.526+0000 7f5364476b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.526+0000 7f5364476b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b508c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.526+0000 7f5364476b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b4ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.527+0000 7f5364476b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896b4f360 con 0x55f896b3f0c0 2024-03-30T02:43:38.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.527+0000 7f5364476b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896b4f4d0 con 0x55f896b49fc0 2024-03-30T02:43:38.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.527+0000 7f5364476b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f896b4f690 con 0x55f896b3e7a0 2024-03-30T02:43:38.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.527+0000 7f5364476b80 10 client.0 _ll_register_callbacks cb 0x55f896a36470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:43:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.527+0000 7f5349400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.527+0000 7f5349400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41292/0 (socket says 172.21.15.67:41292) 2024-03-30T02:43:38.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.527+0000 7f5349400700 1 -- 172.21.15.67:0/2143806705 learned_addr learned my addr 172.21.15.67:0/2143806705 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:43:38.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.528+0000 7f5349e00700 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.529+0000 7f534a800700 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b4ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.528+0000 7f5349400700 1 -- 172.21.15.67:0/2143806705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 msgr2=0x55f896b50380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.529+0000 7f5349400700 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.529+0000 7f5349400700 1 -- 172.21.15.67:0/2143806705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 msgr2=0x55f896b4ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:43:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.529+0000 7f5349400700 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b4ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.529+0000 7f5349400700 1 -- 172.21.15.67:0/2143806705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c1ab5e0 con 0x55f896b3f0c0 2024-03-30T02:43:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.530+0000 7f5349e00700 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:38.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.530+0000 7f534a800700 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b4ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:43:38.534 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/maccess.o 2024-03-30T02:43:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.536+0000 7f5349400700 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b508c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55f896b49f90 tx=0x7f533c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d196c02739eb2965 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:43:38.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.536+0000 7f532b400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:43:38.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.536+0000 7f532b400700 1 -- 172.21.15.67:0/2143806705 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c025d80 con 0x55f896b3f0c0 2024-03-30T02:43:38.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.539+0000 7f532b400700 1 -- 172.21.15.67:0/2143806705 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f533c004030 con 0x55f896b3f0c0 2024-03-30T02:43:38.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.539+0000 7f5364476b80 1 -- 172.21.15.67:0/2143806705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53380178e0 con 0x55f896b3f0c0 2024-03-30T02:43:38.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.540+0000 7f532b400700 1 -- 172.21.15.67:0/2143806705 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f533c025590 con 0x55f896b3f0c0 2024-03-30T02:43:38.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.540+0000 7f5364476b80 10 client.16884 fetch_fsmap learned FSMap version 37 2024-03-30T02:43:38.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.540+0000 7f5364476b80 1 -- 172.21.15.67:0/2143806705 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f896916cd0 con 0x55f896b3f0c0 2024-03-30T02:43:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.541+0000 7f532b400700 1 -- 172.21.15.67:0/2143806705 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53380178e0 con 0x55f896b3f0c0 2024-03-30T02:43:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.541+0000 7f532b400700 1 -- 172.21.15.67:0/2143806705 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 37) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f533c011440 con 0x55f896b3f0c0 2024-03-30T02:43:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.541+0000 7f5364476b80 10 client.16884 fetch_fsmap finished waiting for FSMap version 37 2024-03-30T02:43:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.541+0000 7f5364476b80 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:43:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.541+0000 7f5364476b80 20 client.16884 populate_metadata read hostname 'smithi067' 2024-03-30T02:43:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.541+0000 7f5364476b80 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f896baed30 0x55f896bb1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:43:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.541+0000 7f5364476b80 4 client.16884 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:43:38.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.541+0000 7f5364476b80 1 -- 172.21.15.67:0/2143806705 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f8968dfcf0 con 0x55f896baed30 2024-03-30T02:43:38.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.544+0000 7f5349e00700 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f896baed30 0x55f896bb1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:43:38.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.550+0000 7f5349e00700 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f896baed30 0x55f896bb1150 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f42d9e2bed28656 server_cookie=5b4cb55824fa1955 in_seq=0 out_seq=0 2024-03-30T02:43:38.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.550+0000 7f532b400700 10 client.16884 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:38.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.551+0000 7f532b400700 1 -- 172.21.15.67:0/2143806705 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+269 (crc 0 0 0) 0x55f8968dfcf0 con 0x55f896baed30 2024-03-30T02:43:38.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.551+0000 7f532b400700 10 client.16884 handle_command_reply: tid=0 2024-03-30T02:43:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.552+0000 7f5364476b80 1 client.16884 shutdown 2024-03-30T02:43:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.556+0000 7f5364476b80 1 -- 172.21.15.67:0/2143806705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 msgr2=0x55f896b508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.556+0000 7f5364476b80 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b508c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55f896b49f90 tx=0x7f533c0047c0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.560+0000 7f5364476b80 1 -- 172.21.15.67:0/2143806705 shutdown_connections 2024-03-30T02:43:38.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.560+0000 7f5364476b80 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f896b49fc0 0x55f896b4ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.560+0000 7f5364476b80 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f896b3e7a0 0x55f896b50380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.560+0000 7f5364476b80 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55f896baed30 0x55f896bb1150 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.560+0000 7f5364476b80 1 --2- 172.21.15.67:0/2143806705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f896b3f0c0 0x55f896b508c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.560+0000 7f5364476b80 1 -- 172.21.15.67:0/2143806705 >> 172.21.15.67:0/2143806705 conn(0x55f896a9e6a0 msgr2=0x55f896b46670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:38.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.560+0000 7f532b400700 0 client.16884 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:43:38.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.564+0000 7f5364476b80 1 -- 172.21.15.67:0/2143806705 shutdown_connections 2024-03-30T02:43:38.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.565+0000 7f5364476b80 1 -- 172.21.15.67:0/2143806705 wait complete. 2024-03-30T02:43:38.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.568+0000 7f5364476b80 20 client.16884 trim_cache size 0 max 16384 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (16141 inodes in the stack)", 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout: "32d087aa-2761-4165-913b-6efa530e6563": { 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "32d087aa-2761-4165-913b-6efa530e6563", 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:43:38.574 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:43:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.580+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53300a11f0 msgr2=0x7f53300a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.580+0000 7f534b200700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53300a11f0 0x7f53300a36b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f534c1a7410 tx=0x7f533c009260 comp rx=0 tx=0).stop 2024-03-30T02:43:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.580+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f534c11a240 msgr2=0x7f534c1aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:43:38.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.580+0000 7f534b200700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f534c11a240 0x7f534c1aa7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f534400b070 tx=0x7f534400bd90 comp rx=0 tx=0).stop 2024-03-30T02:43:38.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.585+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 shutdown_connections 2024-03-30T02:43:38.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.585+0000 7f534b200700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53300a11f0 0x7f53300a36b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.585+0000 7f534b200700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f534c10ce90 0x7f534c1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.585+0000 7f534b200700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f534c11a240 0x7f534c1aa7a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.585+0000 7f534b200700 1 --2- 172.21.15.67:0/3586852514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f534c10d7b0 0x7f534c1a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:43:38.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.585+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 >> 172.21.15.67:0/3586852514 conn(0x7f534c06bc10 msgr2=0x7f534c118cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:43:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.593+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 shutdown_connections 2024-03-30T02:43:38.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:43:38.594+0000 7f534b200700 1 -- 172.21.15.67:0/3586852514 wait complete. 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: pgmap v200: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:38.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.644 INFO:tasks.cephfs.filesystem:scrub status for tag:32d087aa-2761-4165-913b-6efa530e6563 - {'path': '/', 'tag': '32d087aa-2761-4165-913b-6efa530e6563', 'options': 'recursive,force'} 2024-03-30T02:43:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: pgmap v200: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-03-30T02:43:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:38.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: pgmap v200: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-03-30T02:43:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:38.788 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/printk/sysctl.o 2024-03-30T02:43:38.795 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/selinux/netlabel.o 2024-03-30T02:43:39.085 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pgprot.o 2024-03-30T02:43:39.172 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/events/zhaoxin/built-in.a 2024-03-30T02:43:39.180 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/kpp.o 2024-03-30T02:43:39.229 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/core.o 2024-03-30T02:43:39.321 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/printk/built-in.a 2024-03-30T02:43:39.343 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/stat.o 2024-03-30T02:43:39.486 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/irqdesc.o 2024-03-30T02:43:39.573 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/gup.o 2024-03-30T02:43:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:43:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:39.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.678 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/microcode/amd.o 2024-03-30T02:43:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:43:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:39.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:39.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:39.925 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/hugetlbpage.o 2024-03-30T02:43:40.225 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/rsapubkey.asn1.[ch] 2024-03-30T02:43:40.241 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/uptime.o 2024-03-30T02:43:40.246 INFO:tasks.workunit.client.0.smithi067.stdout: ASN.1 crypto/rsaprivkey.asn1.[ch] 2024-03-30T02:43:40.261 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsa.o 2024-03-30T02:43:40.456 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/handle.o 2024-03-30T02:43:40.530 INFO:tasks.workunit.client.0.smithi067.stdout: AR security/selinux/built-in.a 2024-03-30T02:43:40.634 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/commoncap.o 2024-03-30T02:43:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: pgmap v201: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-30T02:43:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:40.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: pgmap v201: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:40.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: pgmap v201: 129 pgs: 129 active+clean; 1.8 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:40.810 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/kernel/cpu/microcode/built-in.a 2024-03-30T02:43:40.907 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/cacheinfo.o 2024-03-30T02:43:40.921 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/dump_pagetables.o 2024-03-30T02:43:41.100 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/util.o 2024-03-30T02:43:41.217 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsa_helper.o 2024-03-30T02:43:41.283 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/manage.o 2024-03-30T02:43:41.390 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/probe.o 2024-03-30T02:43:41.428 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsa-pkcs1pad.o 2024-03-30T02:43:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:41.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:41.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:41.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:41.864 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/version.o 2024-03-30T02:43:41.941 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmap_lock.o 2024-03-30T02:43:41.978 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/min_addr.o 2024-03-30T02:43:42.003 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/numa.o 2024-03-30T02:43:42.021 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/utils.o 2024-03-30T02:43:42.286 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/scattered.o 2024-03-30T02:43:42.556 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/acompress.o 2024-03-30T02:43:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: pgmap v202: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-03-30T02:43:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.669 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:42.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: pgmap v202: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-03-30T02:43:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:42.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.710 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/softirqs.o 2024-03-30T02:43:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: pgmap v202: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:42.755 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/spurious.o 2024-03-30T02:43:42.770 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/security.o 2024-03-30T02:43:42.897 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/topology.o 2024-03-30T02:43:42.982 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/highmem.o 2024-03-30T02:43:43.034 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/rapl.o 2024-03-30T02:43:43.420 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/numa_64.o 2024-03-30T02:43:43.432 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/namespaces.o 2024-03-30T02:43:43.570 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/resend.o 2024-03-30T02:43:43.604 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/scompress.o 2024-03-30T02:43:43.645 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/common.o 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:43.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:43:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.291 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/memory.o 2024-03-30T02:43:44.297 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/events/msr.o 2024-03-30T02:43:44.557 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/amdtopology.o 2024-03-30T02:43:44.572 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/chip.o 2024-03-30T02:43:44.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: pgmap v203: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.0 MiB/s wr, 436 op/s 2024-03-30T02:43:44.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.699 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.700 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.701 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.702 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.702 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:44.702 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: pgmap v203: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.0 MiB/s wr, 436 op/s 2024-03-30T02:43:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.907 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/algboss.o 2024-03-30T02:43:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: pgmap v203: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.0 MiB/s wr, 436 op/s 2024-03-30T02:43:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:44.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:44.956 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/self.o 2024-03-30T02:43:45.499 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/rdrand.o 2024-03-30T02:43:45.741 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/events/built-in.a 2024-03-30T02:43:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.886 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/match.o 2024-03-30T02:43:45.907 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/srat.o 2024-03-30T02:43:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.935 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/partitions/core.o 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:45.954 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/dummychip.o 2024-03-30T02:43:45.965 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/thread_self.o 2024-03-30T02:43:45.979 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/lsm_audit.o 2024-03-30T02:43:46.236 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/testmgr.o 2024-03-30T02:43:46.578 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/bugs.o 2024-03-30T02:43:46.709 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/devres.o 2024-03-30T02:43:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: pgmap v204: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 809 KiB/s wr, 711 op/s 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.904 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pkeys.o 2024-03-30T02:43:46.914 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/proc_sysctl.o 2024-03-30T02:43:46.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: pgmap v204: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 809 KiB/s wr, 711 op/s 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: pgmap v204: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 809 KiB/s wr, 711 op/s 2024-03-30T02:43:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.207 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/partitions/msdos.o 2024-03-30T02:43:47.215 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/cmac.o 2024-03-30T02:43:47.432 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/autoprobe.o 2024-03-30T02:43:47.658 INFO:tasks.workunit.client.0.smithi067.stdout: CC security/device_cgroup.o 2024-03-30T02:43:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: pgmap v205: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 893 KiB/s wr, 1.80k op/s 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:47.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.880 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/kaslr.o 2024-03-30T02:43:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: pgmap v205: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 893 KiB/s wr, 1.80k op/s 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:47.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: pgmap v205: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 893 KiB/s wr, 1.80k op/s 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:47.994 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/hmac.o 2024-03-30T02:43:48.037 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mincore.o 2024-03-30T02:43:48.124 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/irqdomain.o 2024-03-30T02:43:48.367 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/partitions/efi.o 2024-03-30T02:43:48.426 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/aperfmperf.o 2024-03-30T02:43:48.659 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/proc_net.o 2024-03-30T02:43:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:48.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.793 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/mm/pti.o 2024-03-30T02:43:48.877 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/crypto_null.o 2024-03-30T02:43:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:48.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.172 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/cpuid-deps.o 2024-03-30T02:43:49.218 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mlock.o 2024-03-30T02:43:49.270 INFO:tasks.workunit.client.0.smithi067.stdout: AR security/built-in.a 2024-03-30T02:43:49.408 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/io_uring.o 2024-03-30T02:43:49.505 INFO:tasks.workunit.client.0.smithi067.stdout: AR block/partitions/built-in.a 2024-03-30T02:43:49.544 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/proc.o 2024-03-30T02:43:49.571 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/bdev.o 2024-03-30T02:43:49.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/md5.o 2024-03-30T02:43:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: pgmap v206: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 734 KiB/s wr, 2.02k op/s 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:49.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.805 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/kcore.o 2024-03-30T02:43:49.842 INFO:tasks.workunit.client.0.smithi067.stdout: AR arch/x86/mm/built-in.a 2024-03-30T02:43:49.877 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/umwait.o 2024-03-30T02:43:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: pgmap v206: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 734 KiB/s wr, 2.02k op/s 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.928 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/migration.o 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: pgmap v206: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 734 KiB/s wr, 2.02k op/s 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.514 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/sha256_generic.o 2024-03-30T02:43:50.595 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmap.o 2024-03-30T02:43:50.656 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/cpuhotplug.o 2024-03-30T02:43:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:50.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.769 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/vmcore.o 2024-03-30T02:43:50.900 INFO:tasks.workunit.client.0.smithi067.stdout: CC arch/x86/kernel/cpu/proc.o 2024-03-30T02:43:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:50.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:50.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:50.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:50.959 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/update.o 2024-03-30T02:43:50.964 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/fops.o 2024-03-30T02:43:51.285 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/sha512_generic.o 2024-03-30T02:43:51.351 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/pm.o 2024-03-30T02:43:51.695 INFO:tasks.workunit.client.0.smithi067.stdout: MKCAP arch/x86/kernel/cpu/capflags.c 2024-03-30T02:43:51.835 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/kmsg.o 2024-03-30T02:43:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: pgmap v207: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 3.15k op/s 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:51.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:51 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: pgmap v207: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 3.15k op/s 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:51.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:51 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:51.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: pgmap v207: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 3.15k op/s 2024-03-30T02:43:52.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:51 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.085 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/bio.o 2024-03-30T02:43:52.101 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/sha3_generic.o 2024-03-30T02:43:52.159 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/msi.o 2024-03-30T02:43:52.438 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc/page.o 2024-03-30T02:43:52.740 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/cbc.o 2024-03-30T02:43:52.811 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/sync.o 2024-03-30T02:43:52.907 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmu_gather.o 2024-03-30T02:43:52.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:43:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:52 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:52 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:43:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:53.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:52 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:43:53.155 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/xattr.o 2024-03-30T02:43:53.208 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/srcutree.o 2024-03-30T02:43:53.277 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/affinity.o 2024-03-30T02:43:53.286 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/ctr.o 2024-03-30T02:43:53.356 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/proc/built-in.a 2024-03-30T02:43:53.439 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/mount.o 2024-03-30T02:43:53.605 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/elevator.o 2024-03-30T02:43:53.754 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mprotect.o 2024-03-30T02:43:53.838 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq/matrix.o 2024-03-30T02:43:53.856 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/gcm.o 2024-03-30T02:43:53.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:53 smithi067 ceph-mon[37808]: pgmap v208: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 3.60k op/s 2024-03-30T02:43:53.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:53 smithi170 ceph-mon[78798]: pgmap v208: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 3.60k op/s 2024-03-30T02:43:53.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:53 smithi082 ceph-mon[79176]: pgmap v208: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 3.60k op/s 2024-03-30T02:43:54.022 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/nop.o 2024-03-30T02:43:54.176 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/inode.o 2024-03-30T02:43:54.246 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/tree.o 2024-03-30T02:43:54.754 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-core.o 2024-03-30T02:43:54.828 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/fs.o 2024-03-30T02:43:54.879 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/ccm.o 2024-03-30T02:43:54.973 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/dir.o 2024-03-30T02:43:54.976 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/irq/built-in.a 2024-03-30T02:43:54.994 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mremap.o 2024-03-30T02:43:55.049 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/file.o 2024-03-30T02:43:55.699 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/splice.o 2024-03-30T02:43:55.842 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/dir.o 2024-03-30T02:43:55.847 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/aes_generic.o 2024-03-30T02:43:55.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:55 smithi067 ceph-mon[37808]: pgmap v209: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 3.60k op/s 2024-03-30T02:43:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:55 smithi170 ceph-mon[78798]: pgmap v209: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 3.60k op/s 2024-03-30T02:43:55.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:55 smithi082 ceph-mon[79176]: pgmap v209: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.3 MiB/s wr, 3.60k op/s 2024-03-30T02:43:56.036 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/file.o 2024-03-30T02:43:56.136 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/msync.o 2024-03-30T02:43:56.462 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/symlink.o 2024-03-30T02:43:56.537 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/sync.o 2024-03-30T02:43:56.596 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-sysfs.o 2024-03-30T02:43:56.736 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/crc32c_generic.o 2024-03-30T02:43:56.919 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page_vma_mapped.o 2024-03-30T02:43:56.975 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernfs/symlink.o 2024-03-30T02:43:57.149 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/mount.o 2024-03-30T02:43:57.253 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/authenc.o 2024-03-30T02:43:57.409 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/advise.o 2024-03-30T02:43:57.441 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rcu/rcu_segcblist.o 2024-03-30T02:43:57.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-flush.o 2024-03-30T02:43:57.644 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/kernfs/built-in.a 2024-03-30T02:43:57.685 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-settings.o 2024-03-30T02:43:57.778 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/pagewalk.o 2024-03-30T02:43:57.926 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysfs/group.o 2024-03-30T02:43:58.093 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/rcu/built-in.a 2024-03-30T02:43:58.156 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/livepatch/built-in.a 2024-03-30T02:43:58.197 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/mapping.o 2024-03-30T02:43:58.242 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/authencesn.o 2024-03-30T02:43:58.243 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/filetable.o 2024-03-30T02:43:58.600 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/div64.o 2024-03-30T02:43:58.601 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-ioc.o 2024-03-30T02:43:58.636 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/sysfs/built-in.a 2024-03-30T02:43:58.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:43:58 smithi067 ceph-mon[37808]: pgmap v210: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 3.58k op/s 2024-03-30T02:43:58.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:43:58 smithi170 ceph-mon[78798]: pgmap v210: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 3.58k op/s 2024-03-30T02:43:58.695 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/devpts/inode.o 2024-03-30T02:43:58.722 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/gcd.o 2024-03-30T02:43:58.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/pgtable-generic.o 2024-03-30T02:43:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:43:58 smithi082 ceph-mon[79176]: pgmap v210: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 3.58k op/s 2024-03-30T02:43:58.869 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/lcm.o 2024-03-30T02:43:58.968 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/int_pow.o 2024-03-30T02:43:59.062 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/int_sqrt.o 2024-03-30T02:43:59.180 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/reciprocal_div.o 2024-03-30T02:43:59.203 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/direct.o 2024-03-30T02:43:59.249 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rng.o 2024-03-30T02:43:59.315 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/math/rational.o 2024-03-30T02:43:59.382 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/openclose.o 2024-03-30T02:43:59.414 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-map.o 2024-03-30T02:43:59.508 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/rmap.o 2024-03-30T02:43:59.515 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/devpts/built-in.a 2024-03-30T02:43:59.571 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/buffered_read.o 2024-03-30T02:43:59.797 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/math/built-in.a 2024-03-30T02:43:59.871 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/memneq.o 2024-03-30T02:44:00.092 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/drbg.o 2024-03-30T02:44:00.143 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/ops_helpers.o 2024-03-30T02:44:00.365 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/utils.o 2024-03-30T02:44:00.446 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-merge.o 2024-03-30T02:44:00.480 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/io.o 2024-03-30T02:44:00.528 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/uring_cmd.o 2024-03-30T02:44:00.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:00 smithi067 ceph-mon[37808]: pgmap v211: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 921 KiB/s wr, 2.47k op/s 2024-03-30T02:44:00.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:00 smithi170 ceph-mon[78798]: pgmap v211: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 921 KiB/s wr, 2.47k op/s 2024-03-30T02:44:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:00 smithi082 ceph-mon[79176]: pgmap v211: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 921 KiB/s wr, 2.47k op/s 2024-03-30T02:44:00.870 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/dummy.o 2024-03-30T02:44:00.881 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/chacha.o 2024-03-30T02:44:01.152 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/aes.o 2024-03-30T02:44:01.157 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/jitterentropy.o 2024-03-30T02:44:01.277 INFO:tasks.workunit.client.0.smithi067.stderr:./arch/x86/kernel/cpu/mkcapflags.sh: line 46: printf: write error: Resource temporarily unavailable 2024-03-30T02:44:01.295 INFO:tasks.workunit.client.0.smithi067.stderr:make[5]: *** [arch/x86/kernel/cpu/Makefile:64: arch/x86/kernel/cpu/capflags.c] Error 1 2024-03-30T02:44:01.295 INFO:tasks.workunit.client.0.smithi067.stderr:make[4]: *** [scripts/Makefile.build:480: arch/x86/kernel/cpu] Error 2 2024-03-30T02:44:01.296 INFO:tasks.workunit.client.0.smithi067.stderr:make[3]: *** [scripts/Makefile.build:480: arch/x86/kernel] Error 2 2024-03-30T02:44:01.297 INFO:tasks.workunit.client.0.smithi067.stderr:make[2]: *** [scripts/Makefile.build:480: arch/x86] Error 2 2024-03-30T02:44:01.298 INFO:tasks.workunit.client.0.smithi067.stderr:make[2]: *** Waiting for unfinished jobs.... 2024-03-30T02:44:01.304 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/epoll.o 2024-03-30T02:44:01.319 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/vmalloc.o 2024-03-30T02:44:01.322 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/jitterentropy-kcapi.o 2024-03-30T02:44:01.523 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/iterator.o 2024-03-30T02:44:01.705 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/swiotlb.o 2024-03-30T02:44:01.753 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/statx.o 2024-03-30T02:44:02.102 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-timeout.o 2024-03-30T02:44:02.394 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/ghash-generic.o 2024-03-30T02:44:02.477 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/entry/common.o 2024-03-30T02:44:02.637 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/arc4.o 2024-03-30T02:44:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:02 smithi067 ceph-mon[37808]: pgmap v212: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 886 KiB/s wr, 2.25k op/s 2024-03-30T02:44:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:02.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:02.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:02 smithi170 ceph-mon[78798]: pgmap v212: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 886 KiB/s wr, 2.25k op/s 2024-03-30T02:44:02.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:02.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:02 smithi082 ceph-mon[79176]: pgmap v212: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 886 KiB/s wr, 2.25k op/s 2024-03-30T02:44:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:02.823 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/main.o 2024-03-30T02:44:03.107 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/hash_info.o 2024-03-30T02:44:03.207 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsapubkey.asn1.o 2024-03-30T02:44:03.218 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/net.o 2024-03-30T02:44:03.280 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/gf128mul.o 2024-03-30T02:44:03.285 INFO:tasks.workunit.client.0.smithi067.stdout: CC crypto/rsaprivkey.asn1.o 2024-03-30T02:44:03.355 INFO:tasks.workunit.client.0.smithi067.stdout: AR crypto/built-in.a 2024-03-30T02:44:03.390 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma/remap.o 2024-03-30T02:44:03.448 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/balloc.o 2024-03-30T02:44:03.568 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-lib.o 2024-03-30T02:44:03.649 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/entry/syscall_user_dispatch.o 2024-03-30T02:44:03.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:03 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:03.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:03 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:44:03.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:03 smithi067 ceph-mon[37808]: osdmap e89: 12 total, 12 up, 12 in 2024-03-30T02:44:03.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:03 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:03.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:03 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:44:03.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:03 smithi170 ceph-mon[78798]: osdmap e89: 12 total, 12 up, 12 in 2024-03-30T02:44:03.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:03 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:03.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:03 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:44:03.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:03 smithi082 ceph-mon[79176]: osdmap e89: 12 total, 12 up, 12 in 2024-03-30T02:44:03.848 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/netfs/objects.o 2024-03-30T02:44:03.962 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/blake2s.o 2024-03-30T02:44:04.079 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/dma/built-in.a 2024-03-30T02:44:04.140 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/inffast.o 2024-03-30T02:44:04.325 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/entry/built-in.a 2024-03-30T02:44:04.361 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/inflate.o 2024-03-30T02:44:04.388 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/main.o 2024-03-30T02:44:04.405 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq.o 2024-03-30T02:44:04.478 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/blake2s-generic.o 2024-03-30T02:44:04.547 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/process_vm_access.o 2024-03-30T02:44:04.640 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/netfs/built-in.a 2024-03-30T02:44:04.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:04 smithi170 ceph-mon[78798]: pgmap v214: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 237 KiB/s wr, 592 op/s 2024-03-30T02:44:04.690 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page_alloc.o 2024-03-30T02:44:04.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:04 smithi082 ceph-mon[79176]: pgmap v214: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 237 KiB/s wr, 592 op/s 2024-03-30T02:44:04.838 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/msg_ring.o 2024-03-30T02:44:04.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:04 smithi067 ceph-mon[37808]: pgmap v214: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 237 KiB/s wr, 592 op/s 2024-03-30T02:44:04.940 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/infutil.o 2024-03-30T02:44:04.953 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/sha1.o 2024-03-30T02:44:04.991 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/bitmap.o 2024-03-30T02:44:05.422 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/inftrees.o 2024-03-30T02:44:05.448 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_deflate/deflate.o 2024-03-30T02:44:05.501 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crypto/sha256.o 2024-03-30T02:44:05.659 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_inflate/inflate_syms.o 2024-03-30T02:44:06.105 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/block_validity.o 2024-03-30T02:44:06.339 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/timeout.o 2024-03-30T02:44:06.555 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/zlib_inflate/built-in.a 2024-03-30T02:44:06.619 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/init-mm.o 2024-03-30T02:44:06.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:06 smithi170 ceph-mon[78798]: pgmap v215: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 429 KiB/s wr, 468 op/s 2024-03-30T02:44:06.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:06 smithi082 ceph-mon[79176]: pgmap v215: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 429 KiB/s wr, 468 op/s 2024-03-30T02:44:06.808 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/crypto/built-in.a 2024-03-30T02:44:06.847 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_deflate/deftree.o 2024-03-30T02:44:06.878 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/strict_rwx.o 2024-03-30T02:44:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:06 smithi067 ceph-mon[37808]: pgmap v215: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 429 KiB/s wr, 468 op/s 2024-03-30T02:44:06.938 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/dir.o 2024-03-30T02:44:07.545 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zlib_deflate/deflate_syms.o 2024-03-30T02:44:07.549 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/sqpoll.o 2024-03-30T02:44:07.615 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-tag.o 2024-03-30T02:44:07.680 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/kmod.o 2024-03-30T02:44:07.797 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/time.o 2024-03-30T02:44:07.845 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/tree_lookup.o 2024-03-30T02:44:08.003 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/zlib_deflate/built-in.a 2024-03-30T02:44:08.070 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/lzo/lzo1x_compress.o 2024-03-30T02:44:08.098 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/ext4_jbd2.o 2024-03-30T02:44:08.337 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/memblock.o 2024-03-30T02:44:08.499 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/kallsyms.o 2024-03-30T02:44:08.541 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/transaction.o 2024-03-30T02:44:08.584 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/fdinfo.o 2024-03-30T02:44:08.607 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-stat.o 2024-03-30T02:44:08.644 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:44:08.656 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/lzo/lzo1x_decompress_safe.o 2024-03-30T02:44:08.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:08 smithi170 ceph-mon[78798]: pgmap v216: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 482 KiB/s wr, 327 op/s 2024-03-30T02:44:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:08 smithi082 ceph-mon[79176]: pgmap v216: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 482 KiB/s wr, 327 op/s 2024-03-30T02:44:08.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:08 smithi067 ceph-mon[37808]: pgmap v216: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 482 KiB/s wr, 327 op/s 2024-03-30T02:44:08.901 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timer.o 2024-03-30T02:44:09.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.109+0000 7f900fa00700 1 -- 172.21.15.67:0/1507927142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90081082f0 msgr2=0x7f9008108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.109+0000 7f900fa00700 1 --2- 172.21.15.67:0/1507927142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90081082f0 0x7f9008108770 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0099c0 tx=0x7f8ffc009cd0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.111+0000 7f900fa00700 1 -- 172.21.15.67:0/1507927142 shutdown_connections 2024-03-30T02:44:09.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.111+0000 7f900fa00700 1 --2- 172.21.15.67:0/1507927142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f900811a590 0x7f900811c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.111+0000 7f900fa00700 1 --2- 172.21.15.67:0/1507927142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f900810f540 0x7f9008107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.111+0000 7f900fa00700 1 --2- 172.21.15.67:0/1507927142 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90081082f0 0x7f9008108770 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.111+0000 7f900fa00700 1 -- 172.21.15.67:0/1507927142 >> 172.21.15.67:0/1507927142 conn(0x7f900806bdf0 msgr2=0x7f900806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:09.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.111+0000 7f900fa00700 1 -- 172.21.15.67:0/1507927142 shutdown_connections 2024-03-30T02:44:09.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.111+0000 7f900fa00700 1 -- 172.21.15.67:0/1507927142 wait complete. 2024-03-30T02:44:09.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.113+0000 7f900fa00700 1 Processor -- start 2024-03-30T02:44:09.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.117+0000 7f900fa00700 1 -- start start 2024-03-30T02:44:09.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.117+0000 7f900fa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90081082f0 0x7f9008118930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.117+0000 7f900fa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f900810f540 0x7f9008118eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.117+0000 7f900fa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f900811a590 0x7f9008111aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.118+0000 7f900fa00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9008112270 con 0x7f90081082f0 2024-03-30T02:44:09.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.118+0000 7f900fa00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90081123e0 con 0x7f900810f540 2024-03-30T02:44:09.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.118+0000 7f900fa00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90081125a0 con 0x7f900811a590 2024-03-30T02:44:09.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.118+0000 7f900d000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90081082f0 0x7f9008118930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.118+0000 7f900da00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f900811a590 0x7f9008111aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.118+0000 7f900d000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90081082f0 0x7f9008118930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45908/0 (socket says 172.21.15.67:45908) 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.118+0000 7f900d000700 1 -- 172.21.15.67:0/3148930394 learned_addr learned my addr 172.21.15.67:0/3148930394 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.118+0000 7f9007e00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f900810f540 0x7f9008118eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.118+0000 7f900da00700 1 -- 172.21.15.67:0/3148930394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f900810f540 msgr2=0x7f9008118eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.119+0000 7f900da00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f900810f540 0x7f9008118eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.119+0000 7f900da00700 1 -- 172.21.15.67:0/3148930394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90081082f0 msgr2=0x7f9008118930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.119+0000 7f900da00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90081082f0 0x7f9008118930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.119+0000 7f900da00700 1 -- 172.21.15.67:0/3148930394 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4009520 con 0x7f900811a590 2024-03-30T02:44:09.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.119+0000 7f900da00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f900811a590 0x7f9008111aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f900000be00 tx=0x7f9000009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.122+0000 7f9005600700 1 -- 172.21.15.67:0/3148930394 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900000cc10 con 0x7f900811a590 2024-03-30T02:44:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.122+0000 7f9005600700 1 -- 172.21.15.67:0/3148930394 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f900000cd70 con 0x7f900811a590 2024-03-30T02:44:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.122+0000 7f900fa00700 1 -- 172.21.15.67:0/3148930394 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc0095f0 con 0x7f900811a590 2024-03-30T02:44:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.122+0000 7f9005600700 1 -- 172.21.15.67:0/3148930394 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000012740 con 0x7f900811a590 2024-03-30T02:44:09.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.122+0000 7f900fa00700 1 -- 172.21.15.67:0/3148930394 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90081166f0 con 0x7f900811a590 2024-03-30T02:44:09.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.131+0000 7f9005600700 1 -- 172.21.15.67:0/3148930394 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f90000128a0 con 0x7f900811a590 2024-03-30T02:44:09.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.133+0000 7f9005600700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8ff00a1420 0x7f8ff00a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.134+0000 7f9005600700 1 -- 172.21.15.67:0/3148930394 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f90000cc950 con 0x7f900811a590 2024-03-30T02:44:09.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.134+0000 7f900d000700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8ff00a1420 0x7f8ff00a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.134+0000 7f90234deb80 1 Processor -- start 2024-03-30T02:44:09.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.135+0000 7f90234deb80 1 -- start start 2024-03-30T02:44:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.135+0000 7f900d000700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8ff00a1420 0x7f8ff00a38e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009f40 tx=0x7f8ff4009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.135+0000 7f90234deb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab87c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.135+0000 7f90234deb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87cef0 0x5585ab8878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.135+0000 7f90234deb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.135+0000 7f90234deb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab7d5990 con 0x5585ab87c5d0 2024-03-30T02:44:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.135+0000 7f90234deb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab7d3350 con 0x5585ab887df0 2024-03-30T02:44:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.135+0000 7f90234deb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab7cec40 con 0x5585ab87cef0 2024-03-30T02:44:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f900da00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f9007e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87cef0 0x5585ab8878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f900da00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48800/0 (socket says 172.21.15.67:48800) 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f900da00700 1 -- 172.21.15.67:0/4100871575 learned_addr learned my addr 172.21.15.67:0/4100871575 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f900d000700 1 --2- 172.21.15.67:0/4100871575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab87c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f9007e00700 1 -- 172.21.15.67:0/4100871575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 msgr2=0x5585ab88a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f9007e00700 1 --2- 172.21.15.67:0/4100871575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f9007e00700 1 -- 172.21.15.67:0/4100871575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 msgr2=0x5585ab87c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f9007e00700 1 --2- 172.21.15.67:0/4100871575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab87c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f9007e00700 1 -- 172.21.15.67:0/4100871575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ab6503b0 con 0x5585ab87cef0 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.136+0000 7f900d000700 1 --2- 172.21.15.67:0/4100871575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab87c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:09.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.137+0000 7f9007e00700 1 --2- 172.21.15.67:0/4100871575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87cef0 0x5585ab8878b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9008113130 tx=0x7f8ffc005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f840f85501f87b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:09.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.141+0000 7f8ffa000700 1 -- 172.21.15.67:0/4100871575 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc01d070 con 0x5585ab87cef0 2024-03-30T02:44:09.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.141+0000 7f8ffa000700 1 -- 172.21.15.67:0/4100871575 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ffc005610 con 0x5585ab87cef0 2024-03-30T02:44:09.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.141+0000 7f8ffa000700 1 -- 172.21.15.67:0/4100871575 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc0036f0 con 0x5585ab87cef0 2024-03-30T02:44:09.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.143+0000 7f90234deb80 1 -- 172.21.15.67:0/4100871575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87cef0 msgr2=0x5585ab8878b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.143+0000 7f90234deb80 1 --2- 172.21.15.67:0/4100871575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87cef0 0x5585ab8878b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9008113130 tx=0x7f8ffc005c00 comp rx=0 tx=0).stop 2024-03-30T02:44:09.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.147+0000 7f90234deb80 1 -- 172.21.15.67:0/4100871575 shutdown_connections 2024-03-30T02:44:09.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.147+0000 7f90234deb80 1 --2- 172.21.15.67:0/4100871575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab87c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.147+0000 7f90234deb80 1 --2- 172.21.15.67:0/4100871575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.147+0000 7f90234deb80 1 --2- 172.21.15.67:0/4100871575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87cef0 0x5585ab8878b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.147+0000 7f90234deb80 1 -- 172.21.15.67:0/4100871575 >> 172.21.15.67:0/4100871575 conn(0x5585ab7dc6a0 msgr2=0x5585ab878a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:09.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.147+0000 7f90234deb80 1 -- 172.21.15.67:0/4100871575 shutdown_connections 2024-03-30T02:44:09.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.147+0000 7f90234deb80 1 -- 172.21.15.67:0/4100871575 wait complete. 2024-03-30T02:44:09.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.148+0000 7f90234deb80 1 -- start start 2024-03-30T02:44:09.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.148+0000 7f90234deb80 1 Processor -- start 2024-03-30T02:44:09.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.151+0000 7f90234deb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab884b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.151+0000 7f90234deb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87cef0 0x5585ab885080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.151+0000 7f90234deb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab8855c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.152+0000 7f900d000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab884b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.152+0000 7f90234deb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab76ad80 con 0x5585ab87c5d0 2024-03-30T02:44:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.152+0000 7f900da00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab8855c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.152+0000 7f900da00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab8855c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37688/0 (socket says 172.21.15.67:37688) 2024-03-30T02:44:09.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.152+0000 7f900da00700 1 -- 172.21.15.67:0/3047474963 learned_addr learned my addr 172.21.15.67:0/3047474963 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:09.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.153+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab76aa20 con 0x5585ab87cef0 2024-03-30T02:44:09.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.153+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab6526a0 con 0x5585ab887df0 2024-03-30T02:44:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.153+0000 7f9007e00700 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87cef0 0x5585ab885080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.153+0000 7f900da00700 1 -- 172.21.15.67:0/3047474963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87cef0 msgr2=0x5585ab885080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.153+0000 7f900da00700 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87cef0 0x5585ab885080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.153+0000 7f900da00700 1 -- 172.21.15.67:0/3047474963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 msgr2=0x5585ab884b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.153+0000 7f900da00700 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab884b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.153+0000 7f900da00700 1 -- 172.21.15.67:0/3047474963 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff4017070 con 0x5585ab887df0 2024-03-30T02:44:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.154+0000 7f900d000700 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab884b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:09.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.154+0000 7f900da00700 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab8855c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90000c9e60 tx=0x7f9000099e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd4cdf368e7580dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:09.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.155+0000 7f8feea00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:09.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.155+0000 7f8feea00700 1 -- 172.21.15.67:0/3047474963 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000099bd0 con 0x5585ab887df0 2024-03-30T02:44:09.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.155+0000 7f8feea00700 1 -- 172.21.15.67:0/3047474963 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f900009d380 con 0x5585ab887df0 2024-03-30T02:44:09.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.155+0000 7f8feea00700 10 client.0 inactive, discarding osd_map(89..89 src has 1..89) v4 2024-03-30T02:44:09.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.157+0000 7f90234deb80 10 client.0 _ll_register_callbacks cb 0x5585ab774470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:09.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.157+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585ab78f380 con 0x5585ab887df0 2024-03-30T02:44:09.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.157+0000 7f8feea00700 1 -- 172.21.15.67:0/3047474963 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f900009d5a0 con 0x5585ab887df0 2024-03-30T02:44:09.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.157+0000 7f90234deb80 10 client.26956 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:09.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.157+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=39}) v3 -- 0x7f900811d280 con 0x5585ab887df0 2024-03-30T02:44:09.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.158+0000 7f8feea00700 1 -- 172.21.15.67:0/3047474963 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9000098c60 con 0x5585ab887df0 2024-03-30T02:44:09.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.158+0000 7f90234deb80 10 client.26956 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:09.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.158+0000 7f90234deb80 10 client.26956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:09.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.158+0000 7f90234deb80 20 client.26956 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:09.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.158+0000 7f90234deb80 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5585ab897760 0x5585ab899c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.158+0000 7f90234deb80 4 client.26956 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:09.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.158+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585ab7c7700 con 0x5585ab897760 2024-03-30T02:44:09.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.159+0000 7f900d000700 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5585ab897760 0x5585ab899c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.163+0000 7f8feea00700 10 client.26956 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:09.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.163+0000 7f900d000700 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5585ab897760 0x5585ab899c20 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df921c9264deafaa server_cookie=868c92abd3b27da1 in_seq=0 out_seq=0 2024-03-30T02:44:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.164+0000 7f8feea00700 1 -- 172.21.15.67:0/3047474963 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5585ab7c7700 con 0x5585ab897760 2024-03-30T02:44:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.164+0000 7f8feea00700 10 client.26956 handle_command_reply: tid=0 2024-03-30T02:44:09.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.164+0000 7f90234deb80 1 client.26956 shutdown 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.164+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 msgr2=0x5585ab8855c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.164+0000 7f90234deb80 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab8855c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90000c9e60 tx=0x7f9000099e70 comp rx=0 tx=0).stop 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 shutdown_connections 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f90234deb80 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5585ab897760 0x5585ab899c20 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f90234deb80 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87c5d0 0x5585ab884b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f8feea00700 0 client.26956 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f90234deb80 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab8855c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f90234deb80 1 --2- 172.21.15.67:0/3047474963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87cef0 0x5585ab885080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 >> 172.21.15.67:0/3047474963 conn(0x5585ab7dc6a0 msgr2=0x5585ab87e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 shutdown_connections 2024-03-30T02:44:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f90234deb80 1 -- 172.21.15.67:0/3047474963 wait complete. 2024-03-30T02:44:09.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.165+0000 7f90234deb80 20 client.26956 trim_cache size 0 max 16384 2024-03-30T02:44:09.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.214+0000 7f90234deb80 1 Processor -- start 2024-03-30T02:44:09.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.214+0000 7f90234deb80 1 -- start start 2024-03-30T02:44:09.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.215+0000 7f90234deb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87c5d0 0x5585ab88a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f90234deb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f90234deb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab883410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f90234deb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab893db0 con 0x5585ab87cef0 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f90234deb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab894160 con 0x5585ab87c5d0 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f90234deb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab88ef50 con 0x5585ab887df0 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f900d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87c5d0 0x5585ab88a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f900d000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87c5d0 0x5585ab88a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48810/0 (socket says 172.21.15.67:48810) 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f900d000700 1 -- 172.21.15.67:0/1643426351 learned_addr learned my addr 172.21.15.67:0/1643426351 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f9007e00700 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.216+0000 7f900da00700 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab883410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f900d000700 1 -- 172.21.15.67:0/1643426351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 msgr2=0x5585ab883410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f900d000700 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab883410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f900d000700 1 -- 172.21.15.67:0/1643426351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 msgr2=0x5585ab88ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:09.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f900d000700 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f900d000700 1 -- 172.21.15.67:0/1643426351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585ab5fcb60 con 0x5585ab87c5d0 2024-03-30T02:44:09.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f9007e00700 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:09.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f900da00700 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab883410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:09.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f900d000700 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87c5d0 0x5585ab88a1e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009fd0 tx=0x7f8ff4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5004892973b4c69 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:09.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f8ffaa00700 1 -- 172.21.15.67:0/1643426351 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4022040 con 0x5585ab87c5d0 2024-03-30T02:44:09.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.217+0000 7f8ffaa00700 1 -- 172.21.15.67:0/1643426351 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ff4023040 con 0x5585ab87c5d0 2024-03-30T02:44:09.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.218+0000 7f8ffaa00700 1 -- 172.21.15.67:0/1643426351 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4004490 con 0x5585ab87c5d0 2024-03-30T02:44:09.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.223+0000 7f90234deb80 1 -- 172.21.15.67:0/1643426351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87c5d0 msgr2=0x5585ab88a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.223+0000 7f90234deb80 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87c5d0 0x5585ab88a1e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009fd0 tx=0x7f8ff4014000 comp rx=0 tx=0).stop 2024-03-30T02:44:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.224+0000 7f90234deb80 1 -- 172.21.15.67:0/1643426351 shutdown_connections 2024-03-30T02:44:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.224+0000 7f90234deb80 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab887df0 0x5585ab883410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.224+0000 7f90234deb80 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab87c5d0 0x5585ab88a1e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.224+0000 7f90234deb80 1 --2- 172.21.15.67:0/1643426351 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.224+0000 7f90234deb80 1 -- 172.21.15.67:0/1643426351 >> 172.21.15.67:0/1643426351 conn(0x5585ab7dc6a0 msgr2=0x5585ab8715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:09.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.226+0000 7f90234deb80 1 -- 172.21.15.67:0/1643426351 shutdown_connections 2024-03-30T02:44:09.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.227+0000 7f90234deb80 1 -- 172.21.15.67:0/1643426351 wait complete. 2024-03-30T02:44:09.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.227+0000 7f90234deb80 1 -- start start 2024-03-30T02:44:09.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.228+0000 7f90234deb80 1 Processor -- start 2024-03-30T02:44:09.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.230+0000 7f90234deb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87c5d0 0x5585ab88e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.230+0000 7f90234deb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.231+0000 7f900d000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87c5d0 0x5585ab88e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.231+0000 7f900d000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87c5d0 0x5585ab88e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37716/0 (socket says 172.21.15.67:37716) 2024-03-30T02:44:09.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.231+0000 7f9007e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.232+0000 7f9007e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45940/0 (socket says 172.21.15.67:45940) 2024-03-30T02:44:09.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.232+0000 7f900d000700 1 -- 172.21.15.67:0/4166015471 learned_addr learned my addr 172.21.15.67:0/4166015471 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:09.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.232+0000 7f90234deb80 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.232+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab88cde0 con 0x5585ab87cef0 2024-03-30T02:44:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.233+0000 7f900da00700 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.233+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab88cf50 con 0x5585ab887df0 2024-03-30T02:44:09.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.233+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585ab88d110 con 0x5585ab87c5d0 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.234+0000 7f9007e00700 1 -- 172.21.15.67:0/4166015471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87c5d0 msgr2=0x5585ab88e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.234+0000 7f9007e00700 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87c5d0 0x5585ab88e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.234+0000 7f9007e00700 1 -- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 msgr2=0x5585ab88c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.234+0000 7f9007e00700 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.234+0000 7f9007e00700 1 -- 172.21.15.67:0/4166015471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff401cc90 con 0x5585ab87cef0 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.234+0000 7f9007e00700 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88e6c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009fd0 tx=0x7f8ffc00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e842a0e33b9a6afe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.235+0000 7f900da00700 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.235+0000 7f8feea00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.235+0000 7f8feea00700 1 -- 172.21.15.67:0/4166015471 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc01d040 con 0x5585ab87cef0 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.235+0000 7f8feea00700 1 -- 172.21.15.67:0/4166015471 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f8ffc0036b0 con 0x5585ab87cef0 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.235+0000 7f8feea00700 10 client.0 inactive, discarding osd_map(89..89 src has 1..89) v4 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.236+0000 7f90234deb80 10 client.0 _ll_register_callbacks cb 0x5585ab774470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:09.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.236+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ffc005610 con 0x5585ab87cef0 2024-03-30T02:44:09.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.239+0000 7f8feea00700 1 -- 172.21.15.67:0/4166015471 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ffc005da0 con 0x5585ab87cef0 2024-03-30T02:44:09.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.239+0000 7f90234deb80 10 client.16902 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:09.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.239+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=39}) v3 -- 0x5585ab794ea0 con 0x5585ab87cef0 2024-03-30T02:44:09.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.240+0000 7f8feea00700 1 -- 172.21.15.67:0/4166015471 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8ffc003870 con 0x5585ab87cef0 2024-03-30T02:44:09.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.240+0000 7f90234deb80 10 client.16902 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:09.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.240+0000 7f90234deb80 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:09.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.240+0000 7f90234deb80 20 client.16902 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:09.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.240+0000 7f90234deb80 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5585ab8ebf20 0x5585ab8ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.241+0000 7f90234deb80 4 client.16902 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:44:09.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.241+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5585ab61dcf0 con 0x5585ab8ebf20 2024-03-30T02:44:09.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.241+0000 7f900d000700 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5585ab8ebf20 0x5585ab8ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.242+0000 7f8feea00700 10 client.16902 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:09.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.242+0000 7f900d000700 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5585ab8ebf20 0x5585ab8ee3e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372ab94acf2bac91 server_cookie=a0796b0084dce171 in_seq=0 out_seq=0 2024-03-30T02:44:09.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.242+0000 7f8feea00700 1 -- 172.21.15.67:0/4166015471 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5585ab61dcf0 con 0x5585ab8ebf20 2024-03-30T02:44:09.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.243+0000 7f8feea00700 10 client.16902 handle_command_reply: tid=0 2024-03-30T02:44:09.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.243+0000 7f90234deb80 1 client.16902 shutdown 2024-03-30T02:44:09.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.244+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 msgr2=0x5585ab88e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.244+0000 7f90234deb80 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88e6c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009fd0 tx=0x7f8ffc00bd30 comp rx=0 tx=0).stop 2024-03-30T02:44:09.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.244+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 shutdown_connections 2024-03-30T02:44:09.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.244+0000 7f90234deb80 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5585ab887df0 0x5585ab88c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.244+0000 7f90234deb80 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5585ab87c5d0 0x5585ab88e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.244+0000 7f90234deb80 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5585ab8ebf20 0x5585ab8ee3e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.244+0000 7f90234deb80 1 --2- 172.21.15.67:0/4166015471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5585ab87cef0 0x5585ab88e6c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.244+0000 7f8feea00700 0 client.16902 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:09.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.244+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 >> 172.21.15.67:0/4166015471 conn(0x5585ab7dc6a0 msgr2=0x5585ab883350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:09.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.248+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 shutdown_connections 2024-03-30T02:44:09.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.248+0000 7f90234deb80 1 -- 172.21.15.67:0/4166015471 wait complete. 2024-03-30T02:44:09.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.248+0000 7f90234deb80 20 client.16902 trim_cache size 0 max 16384 2024-03-30T02:44:09.248 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:44:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:44:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:44:09.248 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:44:09.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 -- 172.21.15.67:0/3148930394 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8ff00a1420 msgr2=0x7f8ff00a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8ff00a1420 0x7f8ff00a38e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009f40 tx=0x7f8ff4009260 comp rx=0 tx=0).stop 2024-03-30T02:44:09.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 -- 172.21.15.67:0/3148930394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f900811a590 msgr2=0x7f9008111aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f900811a590 0x7f9008111aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f900000be00 tx=0x7f9000009ed0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 -- 172.21.15.67:0/3148930394 shutdown_connections 2024-03-30T02:44:09.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8ff00a1420 0x7f8ff00a38e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90081082f0 0x7f9008118930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f900811a590 0x7f9008111aa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 --2- 172.21.15.67:0/3148930394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f900810f540 0x7f9008118eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 -- 172.21.15.67:0/3148930394 >> 172.21.15.67:0/3148930394 conn(0x7f900806bdf0 msgr2=0x7f900806f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 -- 172.21.15.67:0/3148930394 shutdown_connections 2024-03-30T02:44:09.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.251+0000 7f900fa00700 1 -- 172.21.15.67:0/3148930394 wait complete. 2024-03-30T02:44:09.298 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/lzo/built-in.a 2024-03-30T02:44:09.306 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:44:09.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:44:09.376 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/lz4/lz4_decompress.o 2024-03-30T02:44:09.411 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/procfs.o 2024-03-30T02:44:09.528 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-sysfs.o 2024-03-30T02:44:09.549 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/extents.o 2024-03-30T02:44:09.602 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/madvise.o 2024-03-30T02:44:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.758+0000 7f7b78800700 1 -- 172.21.15.67:0/3626828492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b701086d0 msgr2=0x7f7b701139c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.758+0000 7f7b78800700 1 --2- 172.21.15.67:0/3626828492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b701086d0 0x7f7b701139c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009990 tx=0x7f7b5c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.762+0000 7f7b78800700 1 -- 172.21.15.67:0/3626828492 shutdown_connections 2024-03-30T02:44:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.762+0000 7f7b78800700 1 --2- 172.21.15.67:0/3626828492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b70107db0 0x7f7b70108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.762+0000 7f7b78800700 1 --2- 172.21.15.67:0/3626828492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b70113f00 0x7f7b701162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.762+0000 7f7b78800700 1 --2- 172.21.15.67:0/3626828492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b701086d0 0x7f7b701139c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.762+0000 7f7b78800700 1 -- 172.21.15.67:0/3626828492 >> 172.21.15.67:0/3626828492 conn(0x7f7b7006bd20 msgr2=0x7f7b7006c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.762+0000 7f7b78800700 1 -- 172.21.15.67:0/3626828492 shutdown_connections 2024-03-30T02:44:09.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.762+0000 7f7b78800700 1 -- 172.21.15.67:0/3626828492 wait complete. 2024-03-30T02:44:09.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.765+0000 7f7b78800700 1 Processor -- start 2024-03-30T02:44:09.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.765+0000 7f7b78800700 1 -- start start 2024-03-30T02:44:09.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.766+0000 7f7b78800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b70107db0 0x7f7b701aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.766+0000 7f7b78800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b701086d0 0x7f7b701ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.766+0000 7f7b78800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b70113f00 0x7f7b70112a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.766+0000 7f7b75400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b701086d0 0x7f7b701ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.767+0000 7f7b75400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b701086d0 0x7f7b701ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45950/0 (socket says 172.21.15.67:45950) 2024-03-30T02:44:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.769+0000 7f7b75400700 1 -- 172.21.15.67:0/3713406415 learned_addr learned my addr 172.21.15.67:0/3713406415 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.769+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b701131f0 con 0x7f7b701086d0 2024-03-30T02:44:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.769+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b70113360 con 0x7f7b70113f00 2024-03-30T02:44:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.769+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b70113520 con 0x7f7b70107db0 2024-03-30T02:44:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.769+0000 7f7b75e00700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b70107db0 0x7f7b701aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.769+0000 7f7b76800700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b70113f00 0x7f7b70112a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.769+0000 7f7b75e00700 1 -- 172.21.15.67:0/3713406415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b70113f00 msgr2=0x7f7b70112a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.769+0000 7f7b75e00700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b70113f00 0x7f7b70112a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.769+0000 7f7b75e00700 1 -- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b701086d0 msgr2=0x7f7b701ab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.770+0000 7f7b75e00700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b701086d0 0x7f7b701ab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.770+0000 7f7b75e00700 1 -- 172.21.15.67:0/3713406415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c0095f0 con 0x7f7b70107db0 2024-03-30T02:44:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.770+0000 7f7b75400700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b701086d0 0x7f7b701ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:09.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.770+0000 7f7b75e00700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b70107db0 0x7f7b701aab30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400e9f0 tx=0x7f7b6400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:09.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.770+0000 7f7b6aa00700 1 -- 172.21.15.67:0/3713406415 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400eee0 con 0x7f7b70107db0 2024-03-30T02:44:09.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.771+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b70113800 con 0x7f7b70107db0 2024-03-30T02:44:09.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.771+0000 7f7b6aa00700 1 -- 172.21.15.67:0/3713406415 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b6400cbd0 con 0x7f7b70107db0 2024-03-30T02:44:09.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.771+0000 7f7b6aa00700 1 -- 172.21.15.67:0/3713406415 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64018680 con 0x7f7b70107db0 2024-03-30T02:44:09.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.771+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b701b2ce0 con 0x7f7b70107db0 2024-03-30T02:44:09.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.774+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7010cf00 con 0x7f7b70107db0 2024-03-30T02:44:09.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.779+0000 7f7b6aa00700 1 -- 172.21.15.67:0/3713406415 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7b6400cd40 con 0x7f7b70107db0 2024-03-30T02:44:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.782+0000 7f7b6aa00700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7b580a1290 0x7f7b580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.782+0000 7f7b6aa00700 1 -- 172.21.15.67:0/3713406415 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f7b64014070 con 0x7f7b70107db0 2024-03-30T02:44:09.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.782+0000 7f7b75400700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7b580a1290 0x7f7b580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:09.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.786+0000 7f7b75400700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7b580a1290 0x7f7b580a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7b701a75c0 tx=0x7f7b5c005ab0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:09.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:09.791+0000 7f7b6aa00700 1 -- 172.21.15.67:0/3713406415 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7b640529f0 con 0x7f7b70107db0 2024-03-30T02:44:09.831 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/tctx.o 2024-03-30T02:44:10.176 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/module/sysfs.o 2024-03-30T02:44:10.187 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/commit.o 2024-03-30T02:44:10.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.354+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b701a7230 con 0x7f7b70107db0 2024-03-30T02:44:10.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.355+0000 7f7b6aa00700 1 -- 172.21.15.67:0/3713406415 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10916 (secure 0 0 0) 0x7f7b640ade60 con 0x7f7b70107db0 2024-03-30T02:44:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:44:10.357 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:44:02.385480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:44:10.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7b580a1290 msgr2=0x7f7b580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7b580a1290 0x7f7b580a3750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7b701a75c0 tx=0x7f7b5c005ab0 comp rx=0 tx=0).stop 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b70107db0 msgr2=0x7f7b701aab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b70107db0 0x7f7b701aab30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400e9f0 tx=0x7f7b6400ed00 comp rx=0 tx=0).stop 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 shutdown_connections 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f7b580a1290 0x7f7b580a3750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b70113f00 0x7f7b70112a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b70107db0 0x7f7b701aab30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 --2- 172.21.15.67:0/3713406415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b701086d0 0x7f7b701ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 >> 172.21.15.67:0/3713406415 conn(0x7f7b7006bd20 msgr2=0x7f7b7006f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 shutdown_connections 2024-03-30T02:44:10.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:10.365+0000 7f7b78800700 1 -- 172.21.15.67:0/3713406415 wait complete. 2024-03-30T02:44:10.366 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 39 2024-03-30T02:44:10.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:44:10.476 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-cpumap.o 2024-03-30T02:44:10.519 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/lz4/built-in.a 2024-03-30T02:44:10.589 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/zstd_decompress_module.o 2024-03-30T02:44:10.615 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/hrtimer.o 2024-03-30T02:44:10.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:10 smithi067 ceph-mon[37808]: pgmap v217: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 763 KiB/s wr, 341 op/s 2024-03-30T02:44:10.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:10 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3713406415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:44:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:10 smithi082 ceph-mon[79176]: pgmap v217: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 763 KiB/s wr, 341 op/s 2024-03-30T02:44:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:10 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3713406415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:44:10.934 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/poll.o 2024-03-30T02:44:10.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:10 smithi170 ceph-mon[78798]: pgmap v217: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 763 KiB/s wr, 341 op/s 2024-03-30T02:44:10.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:10 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3713406415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:44:11.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.017+0000 7f4cc1600700 1 -- 172.21.15.67:0/944772070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cbc107db0 msgr2=0x7f4cbc108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.017+0000 7f4cc1600700 1 --2- 172.21.15.67:0/944772070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cbc107db0 0x7f4cbc108230 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4cac0098f0 tx=0x7f4cac009c00 comp rx=0 tx=0).stop 2024-03-30T02:44:11.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.018+0000 7f4cc1600700 1 -- 172.21.15.67:0/944772070 shutdown_connections 2024-03-30T02:44:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.018+0000 7f4cc1600700 1 --2- 172.21.15.67:0/944772070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cbc108770 0x7f4cbc1145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.018+0000 7f4cc1600700 1 --2- 172.21.15.67:0/944772070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cbc10d770 0x7f4cbc10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.018+0000 7f4cc1600700 1 --2- 172.21.15.67:0/944772070 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cbc107db0 0x7f4cbc108230 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.018+0000 7f4cc1600700 1 -- 172.21.15.67:0/944772070 >> 172.21.15.67:0/944772070 conn(0x7f4cbc06bdf0 msgr2=0x7f4cbc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.018+0000 7f4cc1600700 1 -- 172.21.15.67:0/944772070 shutdown_connections 2024-03-30T02:44:11.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.018+0000 7f4cc1600700 1 -- 172.21.15.67:0/944772070 wait complete. 2024-03-30T02:44:11.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.020+0000 7f4cc1600700 1 Processor -- start 2024-03-30T02:44:11.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.020+0000 7f4cc1600700 1 -- start start 2024-03-30T02:44:11.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.021+0000 7f4cc1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cbc108770 0x7f4cbc1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.021+0000 7f4cc1600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cbc10d770 0x7f4cbc1a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.021+0000 7f4cc1600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cbc19fb10 0x7f4cbc19ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.021+0000 7f4cc1600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc1a0590 con 0x7f4cbc19fb10 2024-03-30T02:44:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.021+0000 7f4cc1600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc1a0700 con 0x7f4cbc108770 2024-03-30T02:44:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.021+0000 7f4cc1600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc1a08c0 con 0x7f4cbc10d770 2024-03-30T02:44:11.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.022+0000 7f4cbaa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cbc108770 0x7f4cbc1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.022+0000 7f4cbaa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cbc108770 0x7f4cbc1a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48856/0 (socket says 172.21.15.67:48856) 2024-03-30T02:44:11.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.022+0000 7f4cbaa00700 1 -- 172.21.15.67:0/4236704916 learned_addr learned my addr 172.21.15.67:0/4236704916 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:11.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.022+0000 7f4cba000700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cbc10d770 0x7f4cbc1a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.022+0000 7f4cbaa00700 1 -- 172.21.15.67:0/4236704916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cbc10d770 msgr2=0x7f4cbc1a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.022+0000 7f4cbaa00700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cbc10d770 0x7f4cbc1a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.022+0000 7f4cbaa00700 1 -- 172.21.15.67:0/4236704916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cbc19fb10 msgr2=0x7f4cbc19ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.022+0000 7f4cbaa00700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cbc19fb10 0x7f4cbc19ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.022+0000 7f4cbaa00700 1 -- 172.21.15.67:0/4236704916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cac009520 con 0x7f4cbc108770 2024-03-30T02:44:11.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.024+0000 7f4cbaa00700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cbc108770 0x7f4cbc1a5990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40060b0 tx=0x7f4cb4007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.024+0000 7f4cb3400700 1 -- 172.21.15.67:0/4236704916 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400de20 con 0x7f4cbc108770 2024-03-30T02:44:11.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.024+0000 7f4cb3400700 1 -- 172.21.15.67:0/4236704916 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb400f040 con 0x7f4cbc108770 2024-03-30T02:44:11.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.024+0000 7f4cb3400700 1 -- 172.21.15.67:0/4236704916 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb40146e0 con 0x7f4cbc108770 2024-03-30T02:44:11.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.024+0000 7f4cc1600700 1 -- 172.21.15.67:0/4236704916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cbc1ac860 con 0x7f4cbc108770 2024-03-30T02:44:11.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.024+0000 7f4cc1600700 1 -- 172.21.15.67:0/4236704916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc1accf0 con 0x7f4cbc108770 2024-03-30T02:44:11.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.030+0000 7f4cd519cb80 1 Processor -- start 2024-03-30T02:44:11.030 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/module/built-in.a 2024-03-30T02:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.030+0000 7f4cd519cb80 1 -- start start 2024-03-30T02:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.030+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c8e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.031+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c96df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.031+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c99720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.031+0000 7f4cd519cb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3bdc990 con 0x55e4c3c97330 2024-03-30T02:44:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.031+0000 7f4cd519cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3bda350 con 0x55e4c3c8e890 2024-03-30T02:44:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.031+0000 7f4cd519cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3bd5c40 con 0x55e4c3c8df70 2024-03-30T02:44:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.031+0000 7f4cbb400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c8e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.031+0000 7f4cbb400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c8e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37772/0 (socket says 172.21.15.67:37772) 2024-03-30T02:44:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.031+0000 7f4cbb400700 1 -- 172.21.15.67:0/903377769 learned_addr learned my addr 172.21.15.67:0/903377769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.032+0000 7f4cba000700 1 --2- 172.21.15.67:0/903377769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c99720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.032+0000 7f4cbb400700 1 -- 172.21.15.67:0/903377769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 msgr2=0x55e4c3c96df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.032+0000 7f4cbb400700 1 --2- 172.21.15.67:0/903377769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c96df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.032+0000 7f4cbb400700 1 -- 172.21.15.67:0/903377769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 msgr2=0x55e4c3c99720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.032+0000 7f4cbb400700 1 --2- 172.21.15.67:0/903377769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c99720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.032+0000 7f4cbb400700 1 -- 172.21.15.67:0/903377769 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4c3a573b0 con 0x55e4c3c8df70 2024-03-30T02:44:11.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.032+0000 7f4cb3400700 1 -- 172.21.15.67:0/4236704916 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4cb4014840 con 0x7f4cbc108770 2024-03-30T02:44:11.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.035+0000 7f4cba000700 1 --2- 172.21.15.67:0/903377769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c99720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:11.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.035+0000 7f4cb3400700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4ca00a12d0 0x7f4ca00a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.035+0000 7f4cb3400700 1 -- 172.21.15.67:0/4236704916 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f4cb40ccd80 con 0x7f4cbc108770 2024-03-30T02:44:11.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.036+0000 7f4cbb400700 1 --2- 172.21.15.67:0/903377769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c8e350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009bc0 tx=0x7f4ca8009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95249db2d97773c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.037+0000 7f4cbaa00700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4ca00a12d0 0x7f4ca00a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.038+0000 7f4cbaa00700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4ca00a12d0 0x7f4ca00a3790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55e4c3c84640 tx=0x7f4cb400b050 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.038+0000 7f4ca7400700 1 -- 172.21.15.67:0/903377769 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800eae0 con 0x55e4c3c8df70 2024-03-30T02:44:11.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.038+0000 7f4ca7400700 1 -- 172.21.15.67:0/903377769 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ca8004cf0 con 0x55e4c3c8df70 2024-03-30T02:44:11.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.038+0000 7f4ca7400700 1 -- 172.21.15.67:0/903377769 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8005650 con 0x55e4c3c8df70 2024-03-30T02:44:11.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.039+0000 7f4cd519cb80 1 -- 172.21.15.67:0/903377769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 msgr2=0x55e4c3c8e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.039+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/903377769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c8e350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009bc0 tx=0x7f4ca8009ed0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.040+0000 7f4cd519cb80 1 -- 172.21.15.67:0/903377769 shutdown_connections 2024-03-30T02:44:11.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.040+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/903377769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c99720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.040+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/903377769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c8e350 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.040+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/903377769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c96df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.040+0000 7f4cd519cb80 1 -- 172.21.15.67:0/903377769 >> 172.21.15.67:0/903377769 conn(0x55e4c3be36a0 msgr2=0x55e4c3c84c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:11.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.040+0000 7f4cd519cb80 1 -- 172.21.15.67:0/903377769 shutdown_connections 2024-03-30T02:44:11.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.040+0000 7f4cd519cb80 1 -- 172.21.15.67:0/903377769 wait complete. 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.041+0000 7f4cd519cb80 1 -- start start 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.041+0000 7f4cd519cb80 1 Processor -- start 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.042+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.042+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8e890 0x55e4c3c94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.042+0000 7f4cbb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.043+0000 7f4cbb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c94100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45988/0 (socket says 172.21.15.67:45988) 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.043+0000 7f4cbb400700 1 -- 172.21.15.67:0/3121303024 learned_addr learned my addr 172.21.15.67:0/3121303024 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.043+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c97330 0x55e4c3c94b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.043+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3b71d80 con 0x55e4c3c8df70 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.043+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3b71a20 con 0x55e4c3c97330 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.043+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3a596a0 con 0x55e4c3c8e890 2024-03-30T02:44:11.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.043+0000 7f4cd519cb80 10 client.0 _ll_register_callbacks cb 0x55e4c3b7b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:11.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.043+0000 7f4cba000700 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8e890 0x55e4c3c94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.044+0000 7f4cba000700 1 -- 172.21.15.67:0/3121303024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c97330 msgr2=0x55e4c3c94b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:11.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.044+0000 7f4cba000700 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c97330 0x55e4c3c94b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.044+0000 7f4cba000700 1 -- 172.21.15.67:0/3121303024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 msgr2=0x55e4c3c94100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:11.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.044+0000 7f4cba000700 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c94100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.044+0000 7f4cba000700 1 -- 172.21.15.67:0/3121303024 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca80098a0 con 0x55e4c3c8e890 2024-03-30T02:44:11.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.044+0000 7f4cbb400700 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c94100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:11.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.045+0000 7f4cba000700 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8e890 0x55e4c3c94640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cac008000 tx=0x7f4cac00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a912241d27bc01ff server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.045+0000 7f4c9be00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:11.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.045+0000 7f4c9be00700 1 -- 172.21.15.67:0/3121303024 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac01c040 con 0x55e4c3c8e890 2024-03-30T02:44:11.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.045+0000 7f4c9be00700 1 -- 172.21.15.67:0/3121303024 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f4cac0054f0 con 0x55e4c3c8e890 2024-03-30T02:44:11.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.045+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4c3b96380 con 0x55e4c3c8e890 2024-03-30T02:44:11.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.046+0000 7f4c9be00700 1 -- 172.21.15.67:0/3121303024 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cac0056f0 con 0x55e4c3c8e890 2024-03-30T02:44:11.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.048+0000 7f4cd519cb80 10 client.26974 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:11.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.048+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e4c3a03b60 con 0x55e4c3c8e890 2024-03-30T02:44:11.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.050+0000 7f4c9be00700 1 -- 172.21.15.67:0/3121303024 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4c3b96380 con 0x55e4c3c8e890 2024-03-30T02:44:11.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.050+0000 7f4c9be00700 1 -- 172.21.15.67:0/3121303024 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4cac015020 con 0x55e4c3c8e890 2024-03-30T02:44:11.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.050+0000 7f4cd519cb80 10 client.26974 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:11.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.050+0000 7f4cd519cb80 10 client.26974 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:44:11.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.051+0000 7f4cd519cb80 20 client.26974 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:11.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.052+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e4c3ca7290 0x55e4c3ca96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.052+0000 7f4cd519cb80 4 client.26974 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:11.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.052+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4c3bce700 con 0x55e4c3ca7290 2024-03-30T02:44:11.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.052+0000 7f4cbb400700 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e4c3ca7290 0x55e4c3ca96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.054+0000 7f4c9be00700 10 client.26974 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:44:11.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.054+0000 7f4cbb400700 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e4c3ca7290 0x55e4c3ca96b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7102f65f2eef90a0 server_cookie=4f46345fd7199baa in_seq=0 out_seq=0 2024-03-30T02:44:11.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4c9be00700 1 -- 172.21.15.67:0/3121303024 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e4c3bce700 con 0x55e4c3ca7290 2024-03-30T02:44:11.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4c9be00700 10 client.26974 handle_command_reply: tid=0 2024-03-30T02:44:11.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4cd519cb80 1 client.26974 shutdown 2024-03-30T02:44:11.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8e890 msgr2=0x55e4c3c94640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8e890 0x55e4c3c94640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4cac008000 tx=0x7f4cac00f690 comp rx=0 tx=0).stop 2024-03-30T02:44:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 shutdown_connections 2024-03-30T02:44:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c94100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e4c3ca7290 0x55e4c3ca96b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c97330 0x55e4c3c94b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/3121303024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8e890 0x55e4c3c94640 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4c9be00700 0 client.26974 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:44:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.058+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 >> 172.21.15.67:0/3121303024 conn(0x55e4c3be36a0 msgr2=0x55e4c3c81620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:11.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.059+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 shutdown_connections 2024-03-30T02:44:11.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.059+0000 7f4cd519cb80 1 -- 172.21.15.67:0/3121303024 wait complete. 2024-03-30T02:44:11.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.059+0000 7f4cd519cb80 20 client.26974 trim_cache size 0 max 16384 2024-03-30T02:44:11.085 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page_io.o 2024-03-30T02:44:11.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.092+0000 7f4cd519cb80 1 Processor -- start 2024-03-30T02:44:11.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.093+0000 7f4cd519cb80 1 -- start start 2024-03-30T02:44:11.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.094+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c7ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.094+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c9e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.094+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c97330 0x55e4c3c93960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.094+0000 7f4cd519cb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3ca7080 con 0x55e4c3c8df70 2024-03-30T02:44:11.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.094+0000 7f4cd519cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3c970d0 con 0x55e4c3c8e890 2024-03-30T02:44:11.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.094+0000 7f4cd519cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3ca9a40 con 0x55e4c3c97330 2024-03-30T02:44:11.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.096+0000 7f4cbb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c7ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.096+0000 7f4cbb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c7ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45990/0 (socket says 172.21.15.67:45990) 2024-03-30T02:44:11.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.096+0000 7f4cbb400700 1 -- 172.21.15.67:0/2363265212 learned_addr learned my addr 172.21.15.67:0/2363265212 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:11.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.097+0000 7f4cbb400700 1 -- 172.21.15.67:0/2363265212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c97330 msgr2=0x55e4c3c93960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:11.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.097+0000 7f4cbb400700 1 --2- 172.21.15.67:0/2363265212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c97330 0x55e4c3c93960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.097+0000 7f4cbb400700 1 -- 172.21.15.67:0/2363265212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 msgr2=0x55e4c3c9e2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:11.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.097+0000 7f4cbb400700 1 --2- 172.21.15.67:0/2363265212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c9e2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.097+0000 7f4cbb400700 1 -- 172.21.15.67:0/2363265212 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4c3b9bea0 con 0x55e4c3c8df70 2024-03-30T02:44:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.097+0000 7f4cbb400700 1 --2- 172.21.15.67:0/2363265212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c7ef10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8004790 tx=0x7f4ca8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1927365dd6282ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.098+0000 7f4ca7e00700 1 -- 172.21.15.67:0/2363265212 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8028b20 con 0x55e4c3c8df70 2024-03-30T02:44:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.098+0000 7f4ca7e00700 1 -- 172.21.15.67:0/2363265212 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ca8028c80 con 0x55e4c3c8df70 2024-03-30T02:44:11.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.098+0000 7f4ca7e00700 1 -- 172.21.15.67:0/2363265212 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8004040 con 0x55e4c3c8df70 2024-03-30T02:44:11.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.099+0000 7f4cd519cb80 1 -- 172.21.15.67:0/2363265212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 msgr2=0x55e4c3c7ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.099+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/2363265212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c7ef10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8004790 tx=0x7f4ca8005230 comp rx=0 tx=0).stop 2024-03-30T02:44:11.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.101+0000 7f4cd519cb80 1 -- 172.21.15.67:0/2363265212 shutdown_connections 2024-03-30T02:44:11.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.101+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/2363265212 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c8df70 0x55e4c3c7ef10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.101+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/2363265212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c97330 0x55e4c3c93960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.101+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/2363265212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c9e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.101+0000 7f4cd519cb80 1 -- 172.21.15.67:0/2363265212 >> 172.21.15.67:0/2363265212 conn(0x55e4c3be36a0 msgr2=0x55e4c3c99720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:11.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.101+0000 7f4cd519cb80 1 -- 172.21.15.67:0/2363265212 shutdown_connections 2024-03-30T02:44:11.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.101+0000 7f4cd519cb80 1 -- 172.21.15.67:0/2363265212 wait complete. 2024-03-30T02:44:11.101 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/cancel.o 2024-03-30T02:44:11.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.102+0000 7f4cd519cb80 1 -- start start 2024-03-30T02:44:11.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.102+0000 7f4cd519cb80 1 Processor -- start 2024-03-30T02:44:11.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.103+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c9de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.103+0000 7f4cd519cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c9c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.104+0000 7f4cbb400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c9de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.104+0000 7f4cbb400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c9de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37800/0 (socket says 172.21.15.67:37800) 2024-03-30T02:44:11.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.104+0000 7f4cbb400700 1 -- 172.21.15.67:0/1409471374 learned_addr learned my addr 172.21.15.67:0/1409471374 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:11.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.104+0000 7f4cba000700 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c9c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.104+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c9c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.104+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3c9cd90 con 0x55e4c3c97330 2024-03-30T02:44:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.104+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3c9cf00 con 0x55e4c3c8e890 2024-03-30T02:44:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.104+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4c3c9d0c0 con 0x55e4c3c8df70 2024-03-30T02:44:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.104+0000 7f4cd519cb80 10 client.0 _ll_register_callbacks cb 0x55e4c3b7b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.105+0000 7f4cba000700 1 -- 172.21.15.67:0/1409471374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 msgr2=0x55e4c3c9de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.105+0000 7f4cba000700 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c9de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.105+0000 7f4cba000700 1 -- 172.21.15.67:0/1409471374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 msgr2=0x55e4c3c9c850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:11.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.105+0000 7f4cbb400700 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c9c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.106+0000 7f4cba000700 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c9c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.106+0000 7f4cba000700 1 -- 172.21.15.67:0/1409471374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca801c040 con 0x55e4c3c8e890 2024-03-30T02:44:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.106+0000 7f4cba000700 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c9c310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55e4c3bce840 tx=0x7f4cac0038c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2bb597f68f68fd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.106+0000 7f4c9be00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.106+0000 7f4cbb400700 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c9c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:11.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.106+0000 7f4c9be00700 1 -- 172.21.15.67:0/1409471374 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac01c040 con 0x55e4c3c8e890 2024-03-30T02:44:11.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.107+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ca8004cf0 con 0x55e4c3c8e890 2024-03-30T02:44:11.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.107+0000 7f4c9be00700 1 -- 172.21.15.67:0/1409471374 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f4cac003a10 con 0x55e4c3c8e890 2024-03-30T02:44:11.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.107+0000 7f4c9be00700 1 -- 172.21.15.67:0/1409471374 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cac01c040 con 0x55e4c3c8e890 2024-03-30T02:44:11.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.107+0000 7f4cd519cb80 10 client.27017 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:11.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.107+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e4c3a5bcd0 con 0x55e4c3c8e890 2024-03-30T02:44:11.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.108+0000 7f4c9be00700 1 -- 172.21.15.67:0/1409471374 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ca8004cf0 con 0x55e4c3c8e890 2024-03-30T02:44:11.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.108+0000 7f4c9be00700 1 -- 172.21.15.67:0/1409471374 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4cac015070 con 0x55e4c3c8e890 2024-03-30T02:44:11.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.109+0000 7f4cd519cb80 10 client.27017 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:11.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.109+0000 7f4cd519cb80 10 client.27017 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:44:11.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.109+0000 7f4cd519cb80 20 client.27017 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:11.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.109+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e4c3cfc1b0 0x55e4c3cfe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.109+0000 7f4cd519cb80 4 client.27017 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:11.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.109+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e4c3a24cf0 con 0x55e4c3cfc1b0 2024-03-30T02:44:11.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.109+0000 7f4cbb400700 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e4c3cfc1b0 0x55e4c3cfe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.112 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/decompress/huf_decompress.o 2024-03-30T02:44:11.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.114+0000 7f4cbb400700 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e4c3cfc1b0 0x55e4c3cfe5d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb5394e20b5c1061 server_cookie=ea16be184ddcfd30 in_seq=0 out_seq=0 2024-03-30T02:44:11.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.114+0000 7f4c9be00700 10 client.27017 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:44:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.118+0000 7f4c9be00700 1 -- 172.21.15.67:0/1409471374 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e4c3a24cf0 con 0x55e4c3cfc1b0 2024-03-30T02:44:11.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.118+0000 7f4c9be00700 10 client.27017 handle_command_reply: tid=0 2024-03-30T02:44:11.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 client.27017 shutdown 2024-03-30T02:44:11.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 msgr2=0x55e4c3c9c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c9c310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55e4c3bce840 tx=0x7f4cac0038c0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 shutdown_connections 2024-03-30T02:44:11.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e4c3c97330 0x55e4c3c9c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e4c3cfc1b0 0x55e4c3cfe5d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e4c3c8df70 0x55e4c3c9de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 --2- 172.21.15.67:0/1409471374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4c3c8e890 0x55e4c3c9c310 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55e4c3bce840 tx=0x7f4cac0038c0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4c9be00700 0 client.27017 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:44:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 >> 172.21.15.67:0/1409471374 conn(0x55e4c3be36a0 msgr2=0x55e4c3c785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.120+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 shutdown_connections 2024-03-30T02:44:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.121+0000 7f4cd519cb80 1 -- 172.21.15.67:0/1409471374 wait complete. 2024-03-30T02:44:11.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.121+0000 7f4cd519cb80 20 client.27017 trim_cache size 0 max 16384 2024-03-30T02:44:11.121 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:11.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.122+0000 7f4cc1600700 1 -- 172.21.15.67:0/4236704916 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4ca00a12d0 msgr2=0x7f4ca00a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.122+0000 7f4cc1600700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4ca00a12d0 0x7f4ca00a3790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55e4c3c84640 tx=0x7f4cb400b050 comp rx=0 tx=0).stop 2024-03-30T02:44:11.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.122+0000 7f4cc1600700 1 -- 172.21.15.67:0/4236704916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cbc108770 msgr2=0x7f4cbc1a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.122+0000 7f4cc1600700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cbc108770 0x7f4cbc1a5990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40060b0 tx=0x7f4cb4007f90 comp rx=0 tx=0).stop 2024-03-30T02:44:11.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.126+0000 7f4cc1600700 1 -- 172.21.15.67:0/4236704916 shutdown_connections 2024-03-30T02:44:11.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.126+0000 7f4cc1600700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4ca00a12d0 0x7f4ca00a3790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.126+0000 7f4cc1600700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cbc19fb10 0x7f4cbc19ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.126+0000 7f4cc1600700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cbc108770 0x7f4cbc1a5990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.126+0000 7f4cc1600700 1 --2- 172.21.15.67:0/4236704916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cbc10d770 0x7f4cbc1a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.126+0000 7f4cc1600700 1 -- 172.21.15.67:0/4236704916 >> 172.21.15.67:0/4236704916 conn(0x7f4cbc06bdf0 msgr2=0x7f4cbc10b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:11.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.128+0000 7f4cc1600700 1 -- 172.21.15.67:0/4236704916 shutdown_connections 2024-03-30T02:44:11.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.128+0000 7f4cc1600700 1 -- 172.21.15.67:0/4236704916 wait complete. 2024-03-30T02:44:11.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:44:11.308 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-sched.o 2024-03-30T02:44:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.846+0000 7ff2d9400700 1 -- 172.21.15.67:0/3759343547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d41082f0 msgr2=0x7ff2d4108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.846+0000 7ff2d9400700 1 --2- 172.21.15.67:0/3759343547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d41082f0 0x7ff2d4108770 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc01c0d0 tx=0x7ff2cc01c3e0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.849+0000 7ff2d9400700 1 -- 172.21.15.67:0/3759343547 shutdown_connections 2024-03-30T02:44:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.849+0000 7ff2d9400700 1 --2- 172.21.15.67:0/3759343547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2d411a590 0x7ff2d411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.849+0000 7ff2d9400700 1 --2- 172.21.15.67:0/3759343547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d410f540 0x7ff2d4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.849+0000 7ff2d9400700 1 --2- 172.21.15.67:0/3759343547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d41082f0 0x7ff2d4108770 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.849+0000 7ff2d9400700 1 -- 172.21.15.67:0/3759343547 >> 172.21.15.67:0/3759343547 conn(0x7ff2d406bdf0 msgr2=0x7ff2d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:11.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.854+0000 7ff2d9400700 1 -- 172.21.15.67:0/3759343547 shutdown_connections 2024-03-30T02:44:11.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.858+0000 7ff2d9400700 1 -- 172.21.15.67:0/3759343547 wait complete. 2024-03-30T02:44:11.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.860+0000 7ff2d9400700 1 Processor -- start 2024-03-30T02:44:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.863+0000 7ff2d9400700 1 -- start start 2024-03-30T02:44:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.863+0000 7ff2d9400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d41082f0 0x7ff2d4119820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.863+0000 7ff2d9400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2d410f540 0x7ff2d4119d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.864+0000 7ff2d2800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d41082f0 0x7ff2d4119820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.864+0000 7ff2d2800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d41082f0 0x7ff2d4119820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37816/0 (socket says 172.21.15.67:37816) 2024-03-30T02:44:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.865+0000 7ff2d2800700 1 -- 172.21.15.67:0/108088770 learned_addr learned my addr 172.21.15.67:0/108088770 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.865+0000 7ff2d9400700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d411a590 0x7ff2d4115990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.865+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4116130 con 0x7ff2d410f540 2024-03-30T02:44:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.865+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d41162a0 con 0x7ff2d411a590 2024-03-30T02:44:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.865+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4116460 con 0x7ff2d41082f0 2024-03-30T02:44:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.866+0000 7ff2d2800700 1 -- 172.21.15.67:0/108088770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d411a590 msgr2=0x7ff2d4115990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T02:44:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.866+0000 7ff2d2800700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d411a590 0x7ff2d4115990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.866+0000 7ff2d2800700 1 -- 172.21.15.67:0/108088770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2d410f540 msgr2=0x7ff2d4119d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.866+0000 7ff2d2800700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2d410f540 0x7ff2d4119d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.866+0000 7ff2d2800700 1 -- 172.21.15.67:0/108088770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc01ad10 con 0x7ff2d41082f0 2024-03-30T02:44:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.866+0000 7ff2d2800700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d41082f0 0x7ff2d4119820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000d950 tx=0x7ff2c000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.867+0000 7ff2cb400700 1 -- 172.21.15.67:0/108088770 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0009b40 con 0x7ff2d41082f0 2024-03-30T02:44:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.867+0000 7ff2cb400700 1 -- 172.21.15.67:0/108088770 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c0009ca0 con 0x7ff2d41082f0 2024-03-30T02:44:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.867+0000 7ff2cb400700 1 -- 172.21.15.67:0/108088770 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c000f620 con 0x7ff2d41082f0 2024-03-30T02:44:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.869+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d4116740 con 0x7ff2d41082f0 2024-03-30T02:44:11.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.869+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d41aa410 con 0x7ff2d41082f0 2024-03-30T02:44:11.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.883+0000 7ff2cb400700 1 -- 172.21.15.67:0/108088770 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff2c0010460 con 0x7ff2d41082f0 2024-03-30T02:44:11.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.886+0000 7ff2cb400700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff2b80a1420 0x7ff2b80a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.889+0000 7ff2eceefb80 1 Processor -- start 2024-03-30T02:44:11.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.891+0000 7ff2cb400700 1 -- 172.21.15.67:0/108088770 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7ff2c00cce60 con 0x7ff2d41082f0 2024-03-30T02:44:11.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.895+0000 7ff2d1e00700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff2b80a1420 0x7ff2b80a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.896+0000 7ff2eceefb80 1 -- start start 2024-03-30T02:44:11.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.898+0000 7ff2eceefb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f6485d0 0x55642f6489b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.898+0000 7ff2eceefb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f648ef0 0x55642f6538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.898+0000 7ff2eceefb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f653df0 0x55642f6561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.899+0000 7ff2d1e00700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff2b80a1420 0x7ff2b80a38e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc01c860 tx=0x7ff2cc01c040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.899+0000 7ff2eceefb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f5a1990 con 0x55642f653df0 2024-03-30T02:44:11.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.899+0000 7ff2eceefb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f59f350 con 0x55642f648ef0 2024-03-30T02:44:11.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.899+0000 7ff2eceefb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f59ac40 con 0x55642f6485d0 2024-03-30T02:44:11.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.901+0000 7ff2d2800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f6485d0 0x55642f6489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.901+0000 7ff2d2800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f6485d0 0x55642f6489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37818/0 (socket says 172.21.15.67:37818) 2024-03-30T02:44:11.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.901+0000 7ff2d2800700 1 -- 172.21.15.67:0/1350483441 learned_addr learned my addr 172.21.15.67:0/1350483441 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:11.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.901+0000 7ff2d2800700 1 -- 172.21.15.67:0/1350483441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f648ef0 msgr2=0x55642f6538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:11.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.901+0000 7ff2d2800700 1 --2- 172.21.15.67:0/1350483441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f648ef0 0x55642f6538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.901+0000 7ff2d2800700 1 -- 172.21.15.67:0/1350483441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f653df0 msgr2=0x55642f6561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:11.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.901+0000 7ff2d2800700 1 --2- 172.21.15.67:0/1350483441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f653df0 0x55642f6561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.901+0000 7ff2d2800700 1 -- 172.21.15.67:0/1350483441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55642f41c3b0 con 0x55642f6485d0 2024-03-30T02:44:11.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.902+0000 7ff2d2800700 1 --2- 172.21.15.67:0/1350483441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f6485d0 0x55642f6489b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00ca810 tx=0x7ff2c0099530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbc4778b0ffa2118 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.902+0000 7ff2bf400700 1 -- 172.21.15.67:0/1350483441 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c009da00 con 0x55642f6485d0 2024-03-30T02:44:11.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.902+0000 7ff2bf400700 1 -- 172.21.15.67:0/1350483441 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c00a2050 con 0x55642f6485d0 2024-03-30T02:44:11.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.902+0000 7ff2bf400700 1 -- 172.21.15.67:0/1350483441 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c009ae40 con 0x55642f6485d0 2024-03-30T02:44:11.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.903+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1350483441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f6485d0 msgr2=0x55642f6489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.903+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1350483441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f6485d0 0x55642f6489b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00ca810 tx=0x7ff2c0099530 comp rx=0 tx=0).stop 2024-03-30T02:44:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.906+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1350483441 shutdown_connections 2024-03-30T02:44:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.906+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1350483441 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f653df0 0x55642f6561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.906+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1350483441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f6485d0 0x55642f6489b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.906+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1350483441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f648ef0 0x55642f6538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.906+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1350483441 >> 172.21.15.67:0/1350483441 conn(0x55642f5a86a0 msgr2=0x55642f644a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:11.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.912+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1350483441 shutdown_connections 2024-03-30T02:44:11.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.916+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1350483441 wait complete. 2024-03-30T02:44:11.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.917+0000 7ff2eceefb80 1 -- start start 2024-03-30T02:44:11.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.918+0000 7ff2eceefb80 1 Processor -- start 2024-03-30T02:44:11.922 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/recovery.o 2024-03-30T02:44:11.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.921+0000 7ff2eceefb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f650df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.923+0000 7ff2eceefb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f651330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.923+0000 7ff2eceefb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.924+0000 7ff2d2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f650df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.924+0000 7ff2d3200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.924+0000 7ff2d3200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37830/0 (socket says 172.21.15.67:37830) 2024-03-30T02:44:11.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.924+0000 7ff2d3200700 1 -- 172.21.15.67:0/620721952 learned_addr learned my addr 172.21.15.67:0/620721952 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:11.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.924+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f536d80 con 0x55642f6485d0 2024-03-30T02:44:11.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.924+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f536a20 con 0x55642f653df0 2024-03-30T02:44:11.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.924+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f41e6a0 con 0x55642f651870 2024-03-30T02:44:11.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.925+0000 7ff2d1e00700 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f651330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.925+0000 7ff2d3200700 1 -- 172.21.15.67:0/620721952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 msgr2=0x55642f651330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:11.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.925+0000 7ff2d3200700 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f651330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.925+0000 7ff2d3200700 1 -- 172.21.15.67:0/620721952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 msgr2=0x55642f650df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:11.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.925+0000 7ff2d3200700 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f650df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.925+0000 7ff2d3200700 1 -- 172.21.15.67:0/620721952 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d41aa410 con 0x55642f651870 2024-03-30T02:44:11.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.925+0000 7ff2d1e00700 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f651330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:11.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.925+0000 7ff2d3200700 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65f930 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009e10 tx=0x7ff2c400b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86d0b42cc4893031 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:11.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.926+0000 7ff2b3e00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:11.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.926+0000 7ff2b3e00700 1 -- 172.21.15.67:0/620721952 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4011420 con 0x55642f651870 2024-03-30T02:44:11.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.926+0000 7ff2b3e00700 1 -- 172.21.15.67:0/620721952 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7ff2c4011580 con 0x55642f651870 2024-03-30T02:44:11.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.926+0000 7ff2b3e00700 10 client.0 inactive, discarding osd_map(90..90 src has 1..90) v4 2024-03-30T02:44:11.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.926+0000 7ff2eceefb80 10 client.0 _ll_register_callbacks cb 0x55642f540470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:11.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.928+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55642f55b380 con 0x55642f651870 2024-03-30T02:44:11.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.928+0000 7ff2b3e00700 1 -- 172.21.15.67:0/620721952 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2c4011740 con 0x55642f651870 2024-03-30T02:44:11.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.928+0000 7ff2eceefb80 10 client.26989 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:11.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.928+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=39}) v3 -- 0x55642f3c8b60 con 0x55642f651870 2024-03-30T02:44:11.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.929+0000 7ff2b3e00700 1 -- 172.21.15.67:0/620721952 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff2c40118a0 con 0x55642f651870 2024-03-30T02:44:11.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.929+0000 7ff2eceefb80 10 client.26989 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:11.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.929+0000 7ff2eceefb80 10 client.26989 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:44:11.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.929+0000 7ff2eceefb80 20 client.26989 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:11.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.929+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55642f663af0 0x55642f665f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:11.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.929+0000 7ff2eceefb80 4 client.26989 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:11.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.929+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55642f593700 con 0x55642f663af0 2024-03-30T02:44:11.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.933+0000 7ff2d2800700 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55642f663af0 0x55642f665f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:11.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.937+0000 7ff2d2800700 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55642f663af0 0x55642f665f50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62e262ea6e100350 server_cookie=4efcb677266cfb04 in_seq=0 out_seq=0 2024-03-30T02:44:11.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.938+0000 7ff2b3e00700 10 client.26989 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:44:11.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.938+0000 7ff2b3e00700 1 -- 172.21.15.67:0/620721952 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55642f593700 con 0x55642f663af0 2024-03-30T02:44:11.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.938+0000 7ff2b3e00700 10 client.26989 handle_command_reply: tid=0 2024-03-30T02:44:11.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.938+0000 7ff2eceefb80 1 client.26989 shutdown 2024-03-30T02:44:11.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.939+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 msgr2=0x55642f65f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:11.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.939+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65f930 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009e10 tx=0x7ff2c400b3c0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.940+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 shutdown_connections 2024-03-30T02:44:11.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.940+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55642f663af0 0x55642f665f50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.940+0000 7ff2b3e00700 0 client.26989 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:44:11.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.940+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f650df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.940+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65f930 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.940+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/620721952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f651330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:11.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.940+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 >> 172.21.15.67:0/620721952 conn(0x55642f5a86a0 msgr2=0x55642f64a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:11.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.942+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 shutdown_connections 2024-03-30T02:44:11.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.943+0000 7ff2eceefb80 1 -- 172.21.15.67:0/620721952 wait complete. 2024-03-30T02:44:11.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:11.944+0000 7ff2eceefb80 20 client.26989 trim_cache size 0 max 16384 2024-03-30T02:44:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.011+0000 7ff2eceefb80 1 Processor -- start 2024-03-30T02:44:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.011+0000 7ff2eceefb80 1 -- start start 2024-03-30T02:44:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.013+0000 7ff2eceefb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f6489b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.014+0000 7ff2d2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f6489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.014+0000 7ff2d2800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f6489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46046/0 (socket says 172.21.15.67:46046) 2024-03-30T02:44:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.014+0000 7ff2d2800700 1 -- 172.21.15.67:0/1576343313 learned_addr learned my addr 172.21.15.67:0/1576343313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.014+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.015+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f650310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.015+0000 7ff2d1e00700 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.016+0000 7ff2d3200700 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f650310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.016+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1576343313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f6516a0 con 0x55642f6485d0 2024-03-30T02:44:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.017+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1576343313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f653b30 con 0x55642f653df0 2024-03-30T02:44:12.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.017+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1576343313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f65adc0 con 0x55642f651870 2024-03-30T02:44:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.023+0000 7ff2d2800700 1 -- 172.21.15.67:0/1576343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 msgr2=0x55642f65a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.023+0000 7ff2d2800700 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.023+0000 7ff2d2800700 1 -- 172.21.15.67:0/1576343313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 msgr2=0x55642f650310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.023+0000 7ff2d2800700 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f650310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.023+0000 7ff2d2800700 1 -- 172.21.15.67:0/1576343313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55642f560ea0 con 0x55642f6485d0 2024-03-30T02:44:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.024+0000 7ff2d2800700 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f6489b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0012040 tx=0x7ff2c0099d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c7f07c71e20a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.024+0000 7ff2d1e00700 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.024+0000 7ff2d3200700 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f650310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:12.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.029+0000 7ff2bfe00700 1 -- 172.21.15.67:0/1576343313 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0098030 con 0x55642f6485d0 2024-03-30T02:44:12.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.029+0000 7ff2bfe00700 1 -- 172.21.15.67:0/1576343313 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c0010710 con 0x55642f6485d0 2024-03-30T02:44:12.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.030+0000 7ff2bfe00700 1 -- 172.21.15.67:0/1576343313 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00990c0 con 0x55642f6485d0 2024-03-30T02:44:12.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.034+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1576343313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 msgr2=0x55642f6489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.034+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f6489b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0012040 tx=0x7ff2c0099d50 comp rx=0 tx=0).stop 2024-03-30T02:44:12.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.035+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1576343313 shutdown_connections 2024-03-30T02:44:12.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.035+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f6485d0 0x55642f6489b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.035+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f653df0 0x55642f650310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.035+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/1576343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f651870 0x55642f65a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.035+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1576343313 >> 172.21.15.67:0/1576343313 conn(0x55642f5a86a0 msgr2=0x55642f63d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:12.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.038+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1576343313 shutdown_connections 2024-03-30T02:44:12.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.038+0000 7ff2eceefb80 1 -- 172.21.15.67:0/1576343313 wait complete. 2024-03-30T02:44:12.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.041+0000 7ff2eceefb80 1 -- start start 2024-03-30T02:44:12.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.042+0000 7ff2eceefb80 1 Processor -- start 2024-03-30T02:44:12.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.046+0000 7ff2eceefb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f6485d0 0x55642f658040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.047+0000 7ff2d2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f6485d0 0x55642f658040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.047+0000 7ff2d2800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f6485d0 0x55642f658040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48938/0 (socket says 172.21.15.67:48938) 2024-03-30T02:44:12.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.047+0000 7ff2eceefb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f651870 0x55642f658580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.048+0000 7ff2d1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f651870 0x55642f658580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.048+0000 7ff2d1e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f651870 0x55642f658580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46058/0 (socket says 172.21.15.67:46058) 2024-03-30T02:44:12.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.048+0000 7ff2d2800700 1 -- 172.21.15.67:0/3382869313 learned_addr learned my addr 172.21.15.67:0/3382869313 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:12.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.048+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f653df0 0x55642f658b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.048+0000 7ff2d3200700 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f653df0 0x55642f658b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.051+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f6b6b30 con 0x55642f651870 2024-03-30T02:44:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.051+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f6b6ca0 con 0x55642f6485d0 2024-03-30T02:44:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.051+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55642f6b6e60 con 0x55642f653df0 2024-03-30T02:44:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.051+0000 7ff2eceefb80 10 client.0 _ll_register_callbacks cb 0x7ff2c4009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.052+0000 7ff2d1e00700 1 -- 172.21.15.67:0/3382869313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f653df0 msgr2=0x55642f658b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.053+0000 7ff2d1e00700 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f653df0 0x55642f658b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.053+0000 7ff2d1e00700 1 -- 172.21.15.67:0/3382869313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f6485d0 msgr2=0x55642f658040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.053+0000 7ff2d1e00700 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f6485d0 0x55642f658040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.053+0000 7ff2d1e00700 1 -- 172.21.15.67:0/3382869313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55642f41c3b0 con 0x55642f651870 2024-03-30T02:44:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.053+0000 7ff2d1e00700 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f651870 0x55642f658580 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55642f649610 tx=0x7ff2cc01a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb229f954fb30c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:12.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.054+0000 7ff2b3e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:44:12.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.054+0000 7ff2b3e00700 1 -- 172.21.15.67:0/3382869313 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc01f040 con 0x55642f651870 2024-03-30T02:44:12.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.054+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2c00a2050 con 0x55642f651870 2024-03-30T02:44:12.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.055+0000 7ff2b3e00700 1 -- 172.21.15.67:0/3382869313 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7ff2cc018cb0 con 0x55642f651870 2024-03-30T02:44:12.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.059+0000 7ff2b3e00700 1 -- 172.21.15.67:0/3382869313 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2cc018ad0 con 0x55642f651870 2024-03-30T02:44:12.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.059+0000 7ff2eceefb80 10 client.16929 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:12.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.059+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55642f420cd0 con 0x55642f651870 2024-03-30T02:44:12.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.061+0000 7ff2b3e00700 1 -- 172.21.15.67:0/3382869313 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c00a2050 con 0x55642f651870 2024-03-30T02:44:12.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.061+0000 7ff2b3e00700 1 -- 172.21.15.67:0/3382869313 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff2cc005320 con 0x55642f651870 2024-03-30T02:44:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.061+0000 7ff2eceefb80 10 client.16929 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.061+0000 7ff2eceefb80 10 client.16929 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:44:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.061+0000 7ff2eceefb80 20 client.16929 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.061+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55642f663af0 0x55642f6ba4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.061+0000 7ff2d2800700 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55642f663af0 0x55642f6ba4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.062+0000 7ff2eceefb80 4 client.16929 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:12.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.062+0000 7ff2d2800700 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55642f663af0 0x55642f6ba4c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e12c8272dda205c server_cookie=afdd962139e4941 in_seq=0 out_seq=0 2024-03-30T02:44:12.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.062+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55642f3e9cf0 con 0x55642f663af0 2024-03-30T02:44:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.064+0000 7ff2b3e00700 10 client.16929 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:44:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.064+0000 7ff2b3e00700 1 -- 172.21.15.67:0/3382869313 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55642f3e9cf0 con 0x55642f663af0 2024-03-30T02:44:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.064+0000 7ff2b3e00700 10 client.16929 handle_command_reply: tid=0 2024-03-30T02:44:12.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.064+0000 7ff2eceefb80 1 client.16929 shutdown 2024-03-30T02:44:12.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.070+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f651870 msgr2=0x55642f658580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.070+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f651870 0x55642f658580 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55642f649610 tx=0x7ff2cc01a9f0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.070+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 shutdown_connections 2024-03-30T02:44:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.070+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55642f663af0 0x55642f6ba4c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.070+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55642f653df0 0x55642f658b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.070+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55642f6485d0 0x55642f658040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.070+0000 7ff2eceefb80 1 --2- 172.21.15.67:0/3382869313 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55642f651870 0x55642f658580 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.070+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 >> 172.21.15.67:0/3382869313 conn(0x55642f5a86a0 msgr2=0x55642f650310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:12.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.074+0000 7ff2b3e00700 0 client.16929 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:44:12.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.078+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 shutdown_connections 2024-03-30T02:44:12.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.078+0000 7ff2eceefb80 1 -- 172.21.15.67:0/3382869313 wait complete. 2024-03-30T02:44:12.082 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:12.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.081+0000 7ff2eceefb80 20 client.16929 trim_cache size 0 max 16384 2024-03-30T02:44:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.087+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff2b80a1420 msgr2=0x7ff2b80a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.087+0000 7ff2d9400700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff2b80a1420 0x7ff2b80a38e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc01c860 tx=0x7ff2cc01c040 comp rx=0 tx=0).stop 2024-03-30T02:44:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.087+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d41082f0 msgr2=0x7ff2d4119820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.087+0000 7ff2d9400700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d41082f0 0x7ff2d4119820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000d950 tx=0x7ff2c000dd10 comp rx=0 tx=0).stop 2024-03-30T02:44:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.088+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 shutdown_connections 2024-03-30T02:44:12.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.088+0000 7ff2d9400700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff2b80a1420 0x7ff2b80a38e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.088+0000 7ff2d9400700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d411a590 0x7ff2d4115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.088+0000 7ff2d9400700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d41082f0 0x7ff2d4119820 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.088+0000 7ff2d9400700 1 --2- 172.21.15.67:0/108088770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff2d410f540 0x7ff2d4119d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.088+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 >> 172.21.15.67:0/108088770 conn(0x7ff2d406bdf0 msgr2=0x7ff2d411a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:12.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.093+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 shutdown_connections 2024-03-30T02:44:12.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.094+0000 7ff2d9400700 1 -- 172.21.15.67:0/108088770 wait complete. 2024-03-30T02:44:12.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:44:12.422 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timekeeping.o 2024-03-30T02:44:12.424 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:12 smithi067 ceph-mon[37808]: pgmap v218: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.5 MiB/s wr, 406 op/s 2024-03-30T02:44:12.424 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:12 smithi067 ceph-mon[37808]: osdmap e90: 12 total, 12 up, 12 in 2024-03-30T02:44:12.668 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/swap_state.o 2024-03-30T02:44:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:12 smithi170 ceph-mon[78798]: pgmap v218: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.5 MiB/s wr, 406 op/s 2024-03-30T02:44:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:12 smithi170 ceph-mon[78798]: osdmap e90: 12 total, 12 up, 12 in 2024-03-30T02:44:12.709 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/ioctl.o 2024-03-30T02:44:12.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.728+0000 7f6a2ca00700 1 -- 172.21.15.67:0/1169607750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a28067e30 msgr2=0x7f6a280682b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.728+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/1169607750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a28067e30 0x7f6a280682b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009fc0 tx=0x7f6a2000b330 comp rx=0 tx=0).stop 2024-03-30T02:44:12.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.728+0000 7f6a2ca00700 1 -- 172.21.15.67:0/1169607750 shutdown_connections 2024-03-30T02:44:12.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.728+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/1169607750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a280687f0 0x7f6a2810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.728+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/1169607750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a28105ce0 0x7f6a281060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.728+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/1169607750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a28067e30 0x7f6a280682b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.728+0000 7f6a2ca00700 1 -- 172.21.15.67:0/1169607750 >> 172.21.15.67:0/1169607750 conn(0x7f6a28076d40 msgr2=0x7f6a28077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.729+0000 7f6a2ca00700 1 -- 172.21.15.67:0/1169607750 shutdown_connections 2024-03-30T02:44:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.729+0000 7f6a2ca00700 1 -- 172.21.15.67:0/1169607750 wait complete. 2024-03-30T02:44:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.733+0000 7f6a2ca00700 1 Processor -- start 2024-03-30T02:44:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.733+0000 7f6a2ca00700 1 -- start start 2024-03-30T02:44:12.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.734+0000 7f6a2ca00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a28067e30 0x7f6a28195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.734+0000 7f6a2ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a280687f0 0x7f6a28195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.734+0000 7f6a2ca00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28105ce0 0x7f6a28199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.734+0000 7f6a2ca00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28199f70 con 0x7f6a28105ce0 2024-03-30T02:44:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.734+0000 7f6a2ca00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2819a0e0 con 0x7f6a280687f0 2024-03-30T02:44:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.734+0000 7f6a2ca00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2819a2a0 con 0x7f6a28067e30 2024-03-30T02:44:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.734+0000 7f6a25400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a280687f0 0x7f6a28195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.734+0000 7f6a25400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a280687f0 0x7f6a28195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48956/0 (socket says 172.21.15.67:48956) 2024-03-30T02:44:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.734+0000 7f6a25400700 1 -- 172.21.15.67:0/4204104454 learned_addr learned my addr 172.21.15.67:0/4204104454 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.735+0000 7f6a25400700 1 -- 172.21.15.67:0/4204104454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a28067e30 msgr2=0x7f6a28195050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.735+0000 7f6a25400700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a28067e30 0x7f6a28195050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.735+0000 7f6a25400700 1 -- 172.21.15.67:0/4204104454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28105ce0 msgr2=0x7f6a28199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.735+0000 7f6a25400700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28105ce0 0x7f6a28199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.735+0000 7f6a25400700 1 -- 172.21.15.67:0/4204104454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a18009520 con 0x7f6a280687f0 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.735+0000 7f6a25400700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a280687f0 0x7f6a28195590 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b6c0 tx=0x7f6a20008ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.735+0000 7f6a1ea00700 1 -- 172.21.15.67:0/4204104454 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000e040 con 0x7f6a280687f0 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.736+0000 7f6a1ea00700 1 -- 172.21.15.67:0/4204104454 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a2000bbf0 con 0x7f6a280687f0 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.736+0000 7f6a1ea00700 1 -- 172.21.15.67:0/4204104454 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2001b6c0 con 0x7f6a280687f0 2024-03-30T02:44:12.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.736+0000 7f6a2ca00700 1 -- 172.21.15.67:0/4204104454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1400f750 con 0x7f6a280687f0 2024-03-30T02:44:12.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.738+0000 7f6a2ca00700 1 -- 172.21.15.67:0/4204104454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a20009c40 con 0x7f6a280687f0 2024-03-30T02:44:12.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.746+0000 7f6a1ea00700 1 -- 172.21.15.67:0/4204104454 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6a2001d470 con 0x7f6a280687f0 2024-03-30T02:44:12.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.746+0000 7f6a405d1b80 1 Processor -- start 2024-03-30T02:44:12.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.748+0000 7f6a1ea00700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6a0c0a9cb0 0x7f6a0c0ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.748+0000 7f6a1ea00700 1 -- 172.21.15.67:0/4204104454 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f6a200cdb90 con 0x7f6a280687f0 2024-03-30T02:44:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:12 smithi082 ceph-mon[79176]: pgmap v218: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 1.5 MiB/s wr, 406 op/s 2024-03-30T02:44:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:12 smithi082 ceph-mon[79176]: osdmap e90: 12 total, 12 up, 12 in 2024-03-30T02:44:12.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.751+0000 7f6a26800700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6a0c0a9cb0 0x7f6a0c0ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.753+0000 7f6a26800700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6a0c0a9cb0 0x7f6a0c0ac170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005d90 tx=0x7f6a18005ce0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:12.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.754+0000 7f6a405d1b80 1 -- start start 2024-03-30T02:44:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.754+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.754+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda3ef0 0x557f5bdae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.754+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdaedf0 0x557f5bdb11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.754+0000 7f6a405d1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5bcfc990 con 0x557f5bda3ef0 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.754+0000 7f6a405d1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5bcfa350 con 0x557f5bdaedf0 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.754+0000 7f6a405d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5bcf5c40 con 0x557f5bda35d0 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a25e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a25e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37882/0 (socket says 172.21.15.67:37882) 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a25e00700 1 -- 172.21.15.67:0/3735413238 learned_addr learned my addr 172.21.15.67:0/3735413238 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a26800700 1 --2- 172.21.15.67:0/3735413238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdaedf0 0x557f5bdb11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a25e00700 1 -- 172.21.15.67:0/3735413238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdaedf0 msgr2=0x557f5bdb11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a25e00700 1 --2- 172.21.15.67:0/3735413238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdaedf0 0x557f5bdb11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a25e00700 1 -- 172.21.15.67:0/3735413238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda3ef0 msgr2=0x557f5bdae8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a25e00700 1 --2- 172.21.15.67:0/3735413238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda3ef0 0x557f5bdae8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a25e00700 1 -- 172.21.15.67:0/3735413238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f5bb773b0 con 0x557f5bda35d0 2024-03-30T02:44:12.757 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/kbuf.o 2024-03-30T02:44:12.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.755+0000 7f6a25e00700 1 --2- 172.21.15.67:0/3735413238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400fbd0 tx=0x7f6a14009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31bca3c6e856e2d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:12.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.756+0000 7f6a13000700 1 -- 172.21.15.67:0/3735413238 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14009c00 con 0x557f5bda35d0 2024-03-30T02:44:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.756+0000 7f6a13000700 1 -- 172.21.15.67:0/3735413238 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a140044f0 con 0x557f5bda35d0 2024-03-30T02:44:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.756+0000 7f6a13000700 1 -- 172.21.15.67:0/3735413238 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14009c00 con 0x557f5bda35d0 2024-03-30T02:44:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.757+0000 7f6a405d1b80 1 -- 172.21.15.67:0/3735413238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bda35d0 msgr2=0x557f5bda39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.757+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/3735413238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400fbd0 tx=0x7f6a14009310 comp rx=0 tx=0).stop 2024-03-30T02:44:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.758+0000 7f6a405d1b80 1 -- 172.21.15.67:0/3735413238 shutdown_connections 2024-03-30T02:44:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.758+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/3735413238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdaedf0 0x557f5bdb11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.758+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/3735413238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.758+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/3735413238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda3ef0 0x557f5bdae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.758+0000 7f6a405d1b80 1 -- 172.21.15.67:0/3735413238 >> 172.21.15.67:0/3735413238 conn(0x557f5bd036a0 msgr2=0x557f5bd9fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.762+0000 7f6a405d1b80 1 -- 172.21.15.67:0/3735413238 shutdown_connections 2024-03-30T02:44:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.762+0000 7f6a405d1b80 1 -- 172.21.15.67:0/3735413238 wait complete. 2024-03-30T02:44:12.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.762+0000 7f6a405d1b80 1 -- start start 2024-03-30T02:44:12.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.763+0000 7f6a405d1b80 1 Processor -- start 2024-03-30T02:44:12.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.764+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda35d0 0x557f5bdabca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.764+0000 7f6a25e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda35d0 0x557f5bdabca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.764+0000 7f6a25e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda35d0 0x557f5bdabca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46070/0 (socket says 172.21.15.67:46070) 2024-03-30T02:44:12.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.764+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdaedf0 0x557f5bdac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.765+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdac720 0x557f5bdbab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.765+0000 7f6a25e00700 1 -- 172.21.15.67:0/447022444 learned_addr learned my addr 172.21.15.67:0/447022444 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:12.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.765+0000 7f6a25400700 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdaedf0 0x557f5bdac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.765+0000 7f6a26800700 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdac720 0x557f5bdbab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.765+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5bc91d80 con 0x557f5bda35d0 2024-03-30T02:44:12.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.765+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5bc91a20 con 0x557f5bdac720 2024-03-30T02:44:12.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.765+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5bb796a0 con 0x557f5bdaedf0 2024-03-30T02:44:12.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.765+0000 7f6a405d1b80 10 client.0 _ll_register_callbacks cb 0x557f5bc9b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.766+0000 7f6a25400700 1 -- 172.21.15.67:0/447022444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdac720 msgr2=0x557f5bdbab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.766+0000 7f6a25400700 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdac720 0x557f5bdbab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.766+0000 7f6a25400700 1 -- 172.21.15.67:0/447022444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda35d0 msgr2=0x557f5bdabca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.766+0000 7f6a25400700 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda35d0 0x557f5bdabca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.766+0000 7f6a25400700 1 -- 172.21.15.67:0/447022444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1400f480 con 0x557f5bdaedf0 2024-03-30T02:44:12.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.766+0000 7f6a25e00700 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda35d0 0x557f5bdabca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.766+0000 7f6a25400700 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdaedf0 0x557f5bdac1e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6a200caf70 tx=0x7f6a2009a3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bdfb82289e53fb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.766+0000 7f6a07400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.767+0000 7f6a07400700 1 -- 172.21.15.67:0/447022444 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2001d920 con 0x557f5bdaedf0 2024-03-30T02:44:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.767+0000 7f6a07400700 1 -- 172.21.15.67:0/447022444 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f6a2009a9c0 con 0x557f5bdaedf0 2024-03-30T02:44:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.767+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f5bcb6380 con 0x557f5bdaedf0 2024-03-30T02:44:12.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.767+0000 7f6a07400700 1 -- 172.21.15.67:0/447022444 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a2009e7f0 con 0x557f5bdaedf0 2024-03-30T02:44:12.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a405d1b80 10 client.27001 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:12.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557f5bb23b60 con 0x557f5bdaedf0 2024-03-30T02:44:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a07400700 1 -- 172.21.15.67:0/447022444 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f5bcb6380 con 0x557f5bdaedf0 2024-03-30T02:44:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a07400700 1 -- 172.21.15.67:0/447022444 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6a20099eb0 con 0x557f5bdaedf0 2024-03-30T02:44:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a405d1b80 10 client.27001 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a405d1b80 10 client.27001 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:44:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a405d1b80 20 client.27001 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557f5bdbebe0 0x557f5bdc0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a405d1b80 4 client.27001 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:12.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.771+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f5bcee700 con 0x557f5bdbebe0 2024-03-30T02:44:12.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.772+0000 7f6a25e00700 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557f5bdbebe0 0x557f5bdc0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.775+0000 7f6a25e00700 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557f5bdbebe0 0x557f5bdc0fd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f42214b20280683 server_cookie=ab37a2e9767550d0 in_seq=0 out_seq=0 2024-03-30T02:44:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a07400700 10 client.27001 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:44:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a07400700 1 -- 172.21.15.67:0/447022444 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557f5bcee700 con 0x557f5bdbebe0 2024-03-30T02:44:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a07400700 10 client.27001 handle_command_reply: tid=0 2024-03-30T02:44:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a405d1b80 1 client.27001 shutdown 2024-03-30T02:44:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdaedf0 msgr2=0x557f5bdac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdaedf0 0x557f5bdac1e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6a200caf70 tx=0x7f6a2009a3d0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 shutdown_connections 2024-03-30T02:44:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557f5bdbebe0 0x557f5bdc0fd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bda35d0 0x557f5bdabca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bdac720 0x557f5bdbab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/447022444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdaedf0 0x557f5bdac1e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a07400700 0 client.27001 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:44:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.778+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 >> 172.21.15.67:0/447022444 conn(0x557f5bd036a0 msgr2=0x557f5bda56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:12.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.779+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 shutdown_connections 2024-03-30T02:44:12.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.779+0000 7f6a405d1b80 1 -- 172.21.15.67:0/447022444 wait complete. 2024-03-30T02:44:12.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.780+0000 7f6a405d1b80 20 client.27001 trim_cache size 0 max 16384 2024-03-30T02:44:12.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.828+0000 7f6a405d1b80 1 Processor -- start 2024-03-30T02:44:12.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.828+0000 7f6a405d1b80 1 -- start start 2024-03-30T02:44:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.828+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.829+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 0x557f5bdab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.829+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5bdb5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.829+0000 7f6a405d1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5bda4850 con 0x557f5bdaedf0 2024-03-30T02:44:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.829+0000 7f6a405d1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5bdaba70 con 0x557f5bda35d0 2024-03-30T02:44:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.829+0000 7f6a405d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5bdb5fb0 con 0x557f5bdac720 2024-03-30T02:44:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.829+0000 7f6a25e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.829+0000 7f6a25e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48978/0 (socket says 172.21.15.67:48978) 2024-03-30T02:44:12.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.829+0000 7f6a25e00700 1 -- 172.21.15.67:0/2825703131 learned_addr learned my addr 172.21.15.67:0/2825703131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.830+0000 7f6a26800700 1 --2- 172.21.15.67:0/2825703131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5bdb5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.830+0000 7f6a25e00700 1 -- 172.21.15.67:0/2825703131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 msgr2=0x557f5bdab530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:12.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.830+0000 7f6a25e00700 1 --2- 172.21.15.67:0/2825703131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 0x557f5bdab530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.830+0000 7f6a25e00700 1 -- 172.21.15.67:0/2825703131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 msgr2=0x557f5bdb5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.830+0000 7f6a25e00700 1 --2- 172.21.15.67:0/2825703131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5bdb5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.830+0000 7f6a25e00700 1 -- 172.21.15.67:0/2825703131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f5bcbbea0 con 0x557f5bda35d0 2024-03-30T02:44:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.831+0000 7f6a26800700 1 --2- 172.21.15.67:0/2825703131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5bdb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.831+0000 7f6a25e00700 1 --2- 172.21.15.67:0/2825703131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400f450 tx=0x7f6a14005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e28da04df605781 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.831+0000 7f6a13a00700 1 -- 172.21.15.67:0/2825703131 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14029070 con 0x557f5bda35d0 2024-03-30T02:44:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.832+0000 7f6a13a00700 1 -- 172.21.15.67:0/2825703131 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a14005c00 con 0x557f5bda35d0 2024-03-30T02:44:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.832+0000 7f6a13a00700 1 -- 172.21.15.67:0/2825703131 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1402ae90 con 0x557f5bda35d0 2024-03-30T02:44:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.833+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2825703131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 msgr2=0x557f5bda39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.833+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2825703131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400f450 tx=0x7f6a14005fb0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.834+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2825703131 shutdown_connections 2024-03-30T02:44:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.834+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2825703131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5bdb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.834+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2825703131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bda39b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.834+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2825703131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 0x557f5bdab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.834+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2825703131 >> 172.21.15.67:0/2825703131 conn(0x557f5bd036a0 msgr2=0x557f5bdc58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.834+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2825703131 shutdown_connections 2024-03-30T02:44:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.834+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2825703131 wait complete. 2024-03-30T02:44:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.834+0000 7f6a405d1b80 1 -- start start 2024-03-30T02:44:12.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.835+0000 7f6a405d1b80 1 Processor -- start 2024-03-30T02:44:12.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.841+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bdb3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.841+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 0x557f5bdb4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.841+0000 7f6a405d1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5be119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.841+0000 7f6a405d1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5be11ee0 con 0x557f5bdaedf0 2024-03-30T02:44:12.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.841+0000 7f6a405d1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5be12020 con 0x557f5bda35d0 2024-03-30T02:44:12.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.841+0000 7f6a405d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f5be12160 con 0x557f5bdac720 2024-03-30T02:44:12.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.841+0000 7f6a405d1b80 10 client.0 _ll_register_callbacks cb 0x557f5bc9b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:12.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.842+0000 7f6a25e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bdb3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.842+0000 7f6a25400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 0x557f5bdb4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.842+0000 7f6a25e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bdb3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48994/0 (socket says 172.21.15.67:48994) 2024-03-30T02:44:12.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.842+0000 7f6a25e00700 1 -- 172.21.15.67:0/2514093391 learned_addr learned my addr 172.21.15.67:0/2514093391 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:12.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.842+0000 7f6a26800700 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5be119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.843+0000 7f6a25400700 1 -- 172.21.15.67:0/2514093391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 msgr2=0x557f5bdb3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.843+0000 7f6a25400700 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bdb3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.843+0000 7f6a25400700 1 -- 172.21.15.67:0/2514093391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 msgr2=0x557f5be119a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:12.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.843+0000 7f6a25400700 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5be119a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.843+0000 7f6a25400700 1 -- 172.21.15.67:0/2514093391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1800eb50 con 0x557f5bdac720 2024-03-30T02:44:12.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.843+0000 7f6a26800700 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5be119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.845+0000 7f6a25400700 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 0x557f5bdb4180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x557f5bda6be0 tx=0x7f6a2000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f48217b93fc29510 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.845+0000 7f6a07400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.845+0000 7f6a07400700 1 -- 172.21.15.67:0/2514093391 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2009ae70 con 0x557f5bdac720 2024-03-30T02:44:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.845+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2514093391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a140044f0 con 0x557f5bdac720 2024-03-30T02:44:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.845+0000 7f6a07400700 1 -- 172.21.15.67:0/2514093391 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f6a2009a9c0 con 0x557f5bdac720 2024-03-30T02:44:12.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.845+0000 7f6a07400700 1 -- 172.21.15.67:0/2514093391 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a2009e7f0 con 0x557f5bdac720 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.845+0000 7f6a405d1b80 10 client.27004 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.845+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2514093391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x557f5bb773b0 con 0x557f5bdac720 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.846+0000 7f6a07400700 1 -- 172.21.15.67:0/2514093391 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a140044f0 con 0x557f5bdac720 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.846+0000 7f6a07400700 1 -- 172.21.15.67:0/2514093391 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6a200a2920 con 0x557f5bdac720 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.846+0000 7f6a405d1b80 10 client.27004 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.846+0000 7f6a405d1b80 10 client.27004 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.846+0000 7f6a405d1b80 20 client.27004 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.846+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557f5be13390 0x557f5be15780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.846+0000 7f6a405d1b80 4 client.27004 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:12.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.846+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2514093391 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f5bb44cf0 con 0x557f5be13390 2024-03-30T02:44:12.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.847+0000 7f6a25e00700 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557f5be13390 0x557f5be15780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:12.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.847+0000 7f6a07400700 10 client.27004 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:44:12.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.848+0000 7f6a25e00700 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557f5be13390 0x557f5be15780 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11c33460063a979f server_cookie=ccc6983dd1531e76 in_seq=0 out_seq=0 2024-03-30T02:44:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.848+0000 7f6a07400700 1 -- 172.21.15.67:0/2514093391 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f5bb44cf0 con 0x557f5be13390 2024-03-30T02:44:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.848+0000 7f6a07400700 10 client.27004 handle_command_reply: tid=0 2024-03-30T02:44:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.848+0000 7f6a405d1b80 1 client.27004 shutdown 2024-03-30T02:44:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.848+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2514093391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 msgr2=0x557f5bdb4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.848+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 0x557f5bdb4180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x557f5bda6be0 tx=0x7f6a2000b870 comp rx=0 tx=0).stop 2024-03-30T02:44:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.849+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2514093391 shutdown_connections 2024-03-30T02:44:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.849+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x557f5be13390 0x557f5be15780 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.849+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557f5bdaedf0 0x557f5be119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.849+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f5bda35d0 0x557f5bdb3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.849+0000 7f6a405d1b80 1 --2- 172.21.15.67:0/2514093391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557f5bdac720 0x557f5bdb4180 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.849+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2514093391 >> 172.21.15.67:0/2514093391 conn(0x557f5bd036a0 msgr2=0x557f5bda6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:12.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.852+0000 7f6a07400700 0 client.27004 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:44:12.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.852+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2514093391 shutdown_connections 2024-03-30T02:44:12.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.852+0000 7f6a405d1b80 1 -- 172.21.15.67:0/2514093391 wait complete. 2024-03-30T02:44:12.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.853+0000 7f6a405d1b80 20 client.27004 trim_cache size 0 max 16384 2024-03-30T02:44:12.854 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:12.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 -- 172.21.15.67:0/4204104454 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6a0c0a9cb0 msgr2=0x7f6a0c0ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6a0c0a9cb0 0x7f6a0c0ac170 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005d90 tx=0x7f6a18005ce0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 -- 172.21.15.67:0/4204104454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a280687f0 msgr2=0x7f6a28195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:12.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a280687f0 0x7f6a28195590 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b6c0 tx=0x7f6a20008ff0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 -- 172.21.15.67:0/4204104454 shutdown_connections 2024-03-30T02:44:12.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6a0c0a9cb0 0x7f6a0c0ac170 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a28105ce0 0x7f6a28199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a28067e30 0x7f6a28195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 --2- 172.21.15.67:0/4204104454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a280687f0 0x7f6a28195590 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:12.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.856+0000 7f6a2ca00700 1 -- 172.21.15.67:0/4204104454 >> 172.21.15.67:0/4204104454 conn(0x7f6a28076d40 msgr2=0x7f6a280fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:12.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.858+0000 7f6a2ca00700 1 -- 172.21.15.67:0/4204104454 shutdown_connections 2024-03-30T02:44:12.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:12.858+0000 7f6a2ca00700 1 -- 172.21.15.67:0/4204104454 wait complete. 2024-03-30T02:44:12.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:44:12.998 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ramfs/inode.o 2024-03-30T02:44:13.280 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/checkpoint.o 2024-03-30T02:44:13.374 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/extents_status.o 2024-03-30T02:44:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.416+0000 7f640b000700 1 -- 172.21.15.67:0/3982973172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6404107560 msgr2=0x7f6404107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.416+0000 7f640b000700 1 --2- 172.21.15.67:0/3982973172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6404107560 0x7f6404107940 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f63f400b0a0 tx=0x7f63f400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.420+0000 7f640b000700 1 -- 172.21.15.67:0/3982973172 shutdown_connections 2024-03-30T02:44:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.420+0000 7f640b000700 1 --2- 172.21.15.67:0/3982973172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f640410c530 0x7f64041140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.420+0000 7f640b000700 1 --2- 172.21.15.67:0/3982973172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6404107560 0x7f6404107940 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.420+0000 7f640b000700 1 --2- 172.21.15.67:0/3982973172 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6404107f10 0x7f640410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.420+0000 7f640b000700 1 -- 172.21.15.67:0/3982973172 >> 172.21.15.67:0/3982973172 conn(0x7f6404076bd0 msgr2=0x7f6404105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.420+0000 7f640b000700 1 -- 172.21.15.67:0/3982973172 shutdown_connections 2024-03-30T02:44:13.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.420+0000 7f640b000700 1 -- 172.21.15.67:0/3982973172 wait complete. 2024-03-30T02:44:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.421+0000 7f640b000700 1 Processor -- start 2024-03-30T02:44:13.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.422+0000 7f640b000700 1 -- start start 2024-03-30T02:44:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.422+0000 7f640b000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6404107560 0x7f640419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.422+0000 7f640b000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f640410c530 0x7f640419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.422+0000 7f640b000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64041a1f40 0x7f64041a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.422+0000 7f640b000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64041a29c0 con 0x7f640410c530 2024-03-30T02:44:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.422+0000 7f640b000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64041a2b30 con 0x7f64041a1f40 2024-03-30T02:44:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.422+0000 7f640b000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64041a2cf0 con 0x7f6404107560 2024-03-30T02:44:13.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.423+0000 7f640a600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64041a1f40 0x7f64041a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.423+0000 7f6409c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6404107560 0x7f640419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.423+0000 7f640a600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64041a1f40 0x7f64041a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49016/0 (socket says 172.21.15.67:49016) 2024-03-30T02:44:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.423+0000 7f640a600700 1 -- 172.21.15.67:0/1265197858 learned_addr learned my addr 172.21.15.67:0/1265197858 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.423+0000 7f640a600700 1 -- 172.21.15.67:0/1265197858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6404107560 msgr2=0x7f640419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.423+0000 7f640a600700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6404107560 0x7f640419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.423+0000 7f640a600700 1 -- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f640410c530 msgr2=0x7f640419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.423+0000 7f6409200700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f640410c530 0x7f640419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.424+0000 7f640a600700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f640410c530 0x7f640419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.424+0000 7f640a600700 1 -- 172.21.15.67:0/1265197858 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f4009d00 con 0x7f64041a1f40 2024-03-30T02:44:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.424+0000 7f6409200700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f640410c530 0x7f640419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.424+0000 7f640a600700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64041a1f40 0x7f64041a23c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f63f000e970 tx=0x7f63f000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.424+0000 7f6402000700 1 -- 172.21.15.67:0/1265197858 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000c9d0 con 0x7f64041a1f40 2024-03-30T02:44:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.424+0000 7f6402000700 1 -- 172.21.15.67:0/1265197858 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f0004830 con 0x7f64041a1f40 2024-03-30T02:44:13.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.424+0000 7f6402000700 1 -- 172.21.15.67:0/1265197858 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000c9d0 con 0x7f64041a1f40 2024-03-30T02:44:13.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.426+0000 7f640b000700 1 -- 172.21.15.67:0/1265197858 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64041aa250 con 0x7f64041a1f40 2024-03-30T02:44:13.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.426+0000 7f640b000700 1 -- 172.21.15.67:0/1265197858 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64041aa6e0 con 0x7f64041a1f40 2024-03-30T02:44:13.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.435+0000 7f6402000700 1 -- 172.21.15.67:0/1265197858 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f63f0005280 con 0x7f64041a1f40 2024-03-30T02:44:13.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.437+0000 7f6402000700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63e80a1390 0x7f63e80a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.437+0000 7f6409c00700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63e80a1390 0x7f63e80a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.439+0000 7f6402000700 1 -- 172.21.15.67:0/1265197858 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f63f0014070 con 0x7f64041a1f40 2024-03-30T02:44:13.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.440+0000 7f6409c00700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63e80a1390 0x7f63e80a3850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f63f400b780 tx=0x7f63f40096e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.443+0000 7f642010bb80 1 Processor -- start 2024-03-30T02:44:13.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.443+0000 7f642010bb80 1 -- start start 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.443+0000 7f642010bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a356b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.443+0000 7f642010bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a372a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.444+0000 7f642010bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a3753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.444+0000 7f642010bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a2af990 con 0x55db3a3567a0 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.444+0000 7f642010bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a2ad350 con 0x55db3a3570c0 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.444+0000 7f642010bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a2a8c40 con 0x55db3a372fd0 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.444+0000 7f640a600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a3753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.444+0000 7f640a600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a3753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37926/0 (socket says 172.21.15.67:37926) 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.444+0000 7f640a600700 1 -- 172.21.15.67:0/131203646 learned_addr learned my addr 172.21.15.67:0/131203646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.444+0000 7f6409200700 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a372a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.444+0000 7f6409c00700 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a356b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.445+0000 7f6409200700 1 -- 172.21.15.67:0/131203646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 msgr2=0x55db3a3753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.445+0000 7f6409200700 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a3753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.445+0000 7f6409200700 1 -- 172.21.15.67:0/131203646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 msgr2=0x55db3a356b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.445+0000 7f6409200700 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a356b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.445+0000 7f6409200700 1 -- 172.21.15.67:0/131203646 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db3a12a3b0 con 0x55db3a3570c0 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.445+0000 7f6409200700 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a372a90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f640406a210 tx=0x7f63fc009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e5ebcb3222b632 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.446+0000 7f640a600700 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a3753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.446+0000 7f6409c00700 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a356b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.446+0000 7f63faa00700 1 -- 172.21.15.67:0/131203646 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc030b10 con 0x55db3a3570c0 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.446+0000 7f63faa00700 1 -- 172.21.15.67:0/131203646 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63fc004cf0 con 0x55db3a3570c0 2024-03-30T02:44:13.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.446+0000 7f63faa00700 1 -- 172.21.15.67:0/131203646 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0056a0 con 0x55db3a3570c0 2024-03-30T02:44:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.447+0000 7f642010bb80 1 -- 172.21.15.67:0/131203646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 msgr2=0x55db3a372a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.447+0000 7f642010bb80 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a372a90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f640406a210 tx=0x7f63fc009ed0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.447+0000 7f642010bb80 1 -- 172.21.15.67:0/131203646 shutdown_connections 2024-03-30T02:44:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.447+0000 7f642010bb80 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a356b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.447+0000 7f642010bb80 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a3753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.448+0000 7f642010bb80 1 --2- 172.21.15.67:0/131203646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a372a90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.448+0000 7f642010bb80 1 -- 172.21.15.67:0/131203646 >> 172.21.15.67:0/131203646 conn(0x55db3a2b66a0 msgr2=0x55db3a352c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.448+0000 7f642010bb80 1 -- 172.21.15.67:0/131203646 shutdown_connections 2024-03-30T02:44:13.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.448+0000 7f642010bb80 1 -- 172.21.15.67:0/131203646 wait complete. 2024-03-30T02:44:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.448+0000 7f642010bb80 1 -- start start 2024-03-30T02:44:13.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.449+0000 7f642010bb80 1 Processor -- start 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.451+0000 7f642010bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3567a0 0x55db3a366210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.451+0000 7f642010bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3570c0 0x55db3a366750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.451+0000 7f642010bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a361550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.451+0000 7f642010bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a244d80 con 0x55db3a3570c0 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.451+0000 7f642010bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a244a20 con 0x55db3a3567a0 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.451+0000 7f642010bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a12c6a0 con 0x55db3a372fd0 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f642010bb80 10 client.0 _ll_register_callbacks cb 0x55db3a24e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f6409c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3567a0 0x55db3a366210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f6409c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3567a0 0x55db3a366210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49046/0 (socket says 172.21.15.67:49046) 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f6409c00700 1 -- 172.21.15.67:0/3645960025 learned_addr learned my addr 172.21.15.67:0/3645960025 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:13.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f640a600700 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a361550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f640a600700 1 -- 172.21.15.67:0/3645960025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3567a0 msgr2=0x55db3a366210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f6409200700 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3570c0 0x55db3a366750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f640a600700 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3567a0 0x55db3a366210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f640a600700 1 -- 172.21.15.67:0/3645960025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3570c0 msgr2=0x55db3a366750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f640a600700 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3570c0 0x55db3a366750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f640a600700 1 -- 172.21.15.67:0/3645960025 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63fc009850 con 0x55db3a372fd0 2024-03-30T02:44:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.452+0000 7f6409c00700 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3567a0 0x55db3a366210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:13.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.453+0000 7f6409200700 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3570c0 0x55db3a366750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:13.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.453+0000 7f640a600700 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a361550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c5b0 tx=0x7f63f00a29d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f34baca00d3991f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:13.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.454+0000 7f63ef400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:13.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.454+0000 7f63ef400700 1 -- 172.21.15.67:0/3645960025 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000c9d0 con 0x55db3a372fd0 2024-03-30T02:44:13.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.454+0000 7f63ef400700 1 -- 172.21.15.67:0/3645960025 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f63f0016330 con 0x55db3a372fd0 2024-03-30T02:44:13.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.454+0000 7f642010bb80 1 -- 172.21.15.67:0/3645960025 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db3a269380 con 0x55db3a372fd0 2024-03-30T02:44:13.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.457+0000 7f63ef400700 1 -- 172.21.15.67:0/3645960025 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f00b8080 con 0x55db3a372fd0 2024-03-30T02:44:13.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.458+0000 7f642010bb80 10 client.27016 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:13.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.458+0000 7f642010bb80 1 -- 172.21.15.67:0/3645960025 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55db3a0d6b60 con 0x55db3a372fd0 2024-03-30T02:44:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.458+0000 7f63ef400700 1 -- 172.21.15.67:0/3645960025 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55db3a269380 con 0x55db3a372fd0 2024-03-30T02:44:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.458+0000 7f63ef400700 1 -- 172.21.15.67:0/3645960025 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f63f00b3220 con 0x55db3a372fd0 2024-03-30T02:44:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.458+0000 7f642010bb80 10 client.27016 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.458+0000 7f642010bb80 10 client.27016 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:44:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.458+0000 7f642010bb80 20 client.27016 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.458+0000 7f642010bb80 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55db3a371670 0x55db3a371a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.459+0000 7f642010bb80 4 client.27016 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.459+0000 7f642010bb80 1 -- 172.21.15.67:0/3645960025 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db3a2a1700 con 0x55db3a371670 2024-03-30T02:44:13.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.460+0000 7f6409c00700 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55db3a371670 0x55db3a371a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.463+0000 7f63ef400700 10 client.27016 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:44:13.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.463+0000 7f6409c00700 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55db3a371670 0x55db3a371a80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4eabfcccc04d38f server_cookie=45e74bfe90c0815e in_seq=0 out_seq=0 2024-03-30T02:44:13.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.464+0000 7f63ef400700 1 -- 172.21.15.67:0/3645960025 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55db3a2a1700 con 0x55db3a371670 2024-03-30T02:44:13.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.464+0000 7f63ef400700 10 client.27016 handle_command_reply: tid=0 2024-03-30T02:44:13.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.464+0000 7f642010bb80 1 client.27016 shutdown 2024-03-30T02:44:13.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.467+0000 7f642010bb80 1 -- 172.21.15.67:0/3645960025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 msgr2=0x55db3a361550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:13.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.467+0000 7f642010bb80 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a361550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c5b0 tx=0x7f63f00a29d0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.467+0000 7f642010bb80 1 -- 172.21.15.67:0/3645960025 shutdown_connections 2024-03-30T02:44:13.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.467+0000 7f642010bb80 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a361550 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.467+0000 7f642010bb80 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3567a0 0x55db3a366210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.467+0000 7f642010bb80 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55db3a371670 0x55db3a371a80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.467+0000 7f642010bb80 1 --2- 172.21.15.67:0/3645960025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3570c0 0x55db3a366750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.467+0000 7f642010bb80 1 -- 172.21.15.67:0/3645960025 >> 172.21.15.67:0/3645960025 conn(0x55db3a2b66a0 msgr2=0x55db3a358890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:13.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.467+0000 7f63ef400700 0 client.27016 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:44:13.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.469+0000 7f642010bb80 1 -- 172.21.15.67:0/3645960025 shutdown_connections 2024-03-30T02:44:13.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.469+0000 7f642010bb80 1 -- 172.21.15.67:0/3645960025 wait complete. 2024-03-30T02:44:13.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.469+0000 7f642010bb80 20 client.27016 trim_cache size 0 max 16384 2024-03-30T02:44:13.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.529+0000 7f642010bb80 1 Processor -- start 2024-03-30T02:44:13.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.529+0000 7f642010bb80 1 -- start start 2024-03-30T02:44:13.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.529+0000 7f642010bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a3567a0 0x55db3a361dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.529+0000 7f642010bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a35fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.529+0000 7f642010bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a372fd0 0x55db3a3603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.529+0000 7f642010bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a357a90 con 0x55db3a372fd0 2024-03-30T02:44:13.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.529+0000 7f642010bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a36ee50 con 0x55db3a3570c0 2024-03-30T02:44:13.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.529+0000 7f642010bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a36dd90 con 0x55db3a3567a0 2024-03-30T02:44:13.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f6409200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a35fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f6409c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a3567a0 0x55db3a361dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f640a600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a372fd0 0x55db3a3603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f6409200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a35fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49054/0 (socket says 172.21.15.67:49054) 2024-03-30T02:44:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f6409200700 1 -- 172.21.15.67:0/286732961 learned_addr learned my addr 172.21.15.67:0/286732961 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f6409c00700 1 -- 172.21.15.67:0/286732961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 msgr2=0x55db3a35fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f6409c00700 1 --2- 172.21.15.67:0/286732961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a35fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f6409c00700 1 -- 172.21.15.67:0/286732961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a372fd0 msgr2=0x55db3a3603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f6409c00700 1 --2- 172.21.15.67:0/286732961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a372fd0 0x55db3a3603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f6409c00700 1 -- 172.21.15.67:0/286732961 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db3a26eea0 con 0x55db3a3567a0 2024-03-30T02:44:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.530+0000 7f640a600700 1 --2- 172.21.15.67:0/286732961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a372fd0 0x55db3a3603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.531+0000 7f6409c00700 1 --2- 172.21.15.67:0/286732961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a3567a0 0x55db3a361dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55db3a372390 tx=0x7f63f401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ca4c4b85855c70 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:13.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.533+0000 7f63fb400700 1 -- 172.21.15.67:0/286732961 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f402e070 con 0x55db3a3567a0 2024-03-30T02:44:13.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.533+0000 7f63fb400700 1 -- 172.21.15.67:0/286732961 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f4003e40 con 0x55db3a3567a0 2024-03-30T02:44:13.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.533+0000 7f63fb400700 1 -- 172.21.15.67:0/286732961 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f400e040 con 0x55db3a3567a0 2024-03-30T02:44:13.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.534+0000 7f642010bb80 1 -- 172.21.15.67:0/286732961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a3567a0 msgr2=0x55db3a361dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:13.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.534+0000 7f642010bb80 1 --2- 172.21.15.67:0/286732961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a3567a0 0x55db3a361dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55db3a372390 tx=0x7f63f401e000 comp rx=0 tx=0).stop 2024-03-30T02:44:13.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.534+0000 7f642010bb80 1 -- 172.21.15.67:0/286732961 shutdown_connections 2024-03-30T02:44:13.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.534+0000 7f642010bb80 1 --2- 172.21.15.67:0/286732961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a372fd0 0x55db3a3603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.534+0000 7f642010bb80 1 --2- 172.21.15.67:0/286732961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a3567a0 0x55db3a361dd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.534+0000 7f642010bb80 1 --2- 172.21.15.67:0/286732961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a35fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.534+0000 7f642010bb80 1 -- 172.21.15.67:0/286732961 >> 172.21.15.67:0/286732961 conn(0x55db3a2b66a0 msgr2=0x55db3a3723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:13.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.534+0000 7f642010bb80 1 -- 172.21.15.67:0/286732961 shutdown_connections 2024-03-30T02:44:13.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.534+0000 7f642010bb80 1 -- 172.21.15.67:0/286732961 wait complete. 2024-03-30T02:44:13.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.535+0000 7f642010bb80 1 -- start start 2024-03-30T02:44:13.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.536+0000 7f642010bb80 1 Processor -- start 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.539+0000 7f642010bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a36cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.539+0000 7f642010bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a36d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.540+0000 7f642010bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a378aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.540+0000 7f642010bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a378fe0 con 0x55db3a3567a0 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.540+0000 7f642010bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a379150 con 0x55db3a3570c0 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.540+0000 7f642010bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db3a379310 con 0x55db3a372fd0 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.540+0000 7f642010bb80 10 client.0 _ll_register_callbacks cb 0x55db3a24e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.540+0000 7f6409200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a36d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.540+0000 7f6409200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a36d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49068/0 (socket says 172.21.15.67:49068) 2024-03-30T02:44:13.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.540+0000 7f6409200700 1 -- 172.21.15.67:0/1721994962 learned_addr learned my addr 172.21.15.67:0/1721994962 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:13.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.541+0000 7f6409200700 1 -- 172.21.15.67:0/1721994962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 msgr2=0x55db3a378aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:13.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.541+0000 7f6409200700 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a378aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.541+0000 7f6409200700 1 -- 172.21.15.67:0/1721994962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 msgr2=0x55db3a36cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.541+0000 7f6409c00700 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a36cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.541+0000 7f6409200700 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a36cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.541+0000 7f6409200700 1 -- 172.21.15.67:0/1721994962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db3a0d6b60 con 0x55db3a3570c0 2024-03-30T02:44:13.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.541+0000 7f6409c00700 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a36cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.541+0000 7f6409200700 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a36d4b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f63fc030b40 tx=0x7f63fc030c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a100749745cdaafc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.542+0000 7f63ef400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.542+0000 7f63ef400700 1 -- 172.21.15.67:0/1721994962 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc004220 con 0x55db3a3570c0 2024-03-30T02:44:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.542+0000 7f63ef400700 1 -- 172.21.15.67:0/1721994962 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f63fc004380 con 0x55db3a3570c0 2024-03-30T02:44:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.543+0000 7f642010bb80 1 -- 172.21.15.67:0/1721994962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63fc004cf0 con 0x55db3a3570c0 2024-03-30T02:44:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.544+0000 7f63ef400700 1 -- 172.21.15.67:0/1721994962 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63fc004540 con 0x55db3a3570c0 2024-03-30T02:44:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.544+0000 7f642010bb80 10 client.27059 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:13.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.544+0000 7f642010bb80 1 -- 172.21.15.67:0/1721994962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f64041148f0 con 0x55db3a3570c0 2024-03-30T02:44:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.544+0000 7f63ef400700 1 -- 172.21.15.67:0/1721994962 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63fc004cf0 con 0x55db3a3570c0 2024-03-30T02:44:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.545+0000 7f63ef400700 1 -- 172.21.15.67:0/1721994962 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f63fc0051d0 con 0x55db3a3570c0 2024-03-30T02:44:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.545+0000 7f642010bb80 10 client.27059 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:13.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.545+0000 7f642010bb80 10 client.27059 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:44:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.545+0000 7f642010bb80 20 client.27059 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.545+0000 7f642010bb80 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55db3a3c6630 0x55db3a3c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.545+0000 7f642010bb80 4 client.27059 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.545+0000 7f642010bb80 1 -- 172.21.15.67:0/1721994962 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55db3a0f7cf0 con 0x55db3a3c6630 2024-03-30T02:44:13.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.546+0000 7f6409c00700 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55db3a3c6630 0x55db3a3c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.547+0000 7f63ef400700 10 client.27059 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:44:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.547+0000 7f6409c00700 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55db3a3c6630 0x55db3a3c8a50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be7e6500be3b85e1 server_cookie=614d0a7e0ca52e79 in_seq=0 out_seq=0 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.547+0000 7f63ef400700 1 -- 172.21.15.67:0/1721994962 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55db3a0f7cf0 con 0x55db3a3c6630 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.547+0000 7f63ef400700 10 client.27059 handle_command_reply: tid=0 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.547+0000 7f642010bb80 1 client.27059 shutdown 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.548+0000 7f642010bb80 1 -- 172.21.15.67:0/1721994962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 msgr2=0x55db3a36d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.548+0000 7f642010bb80 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a36d4b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f63fc030b40 tx=0x7f63fc030c20 comp rx=0 tx=0).stop 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.548+0000 7f642010bb80 1 -- 172.21.15.67:0/1721994962 shutdown_connections 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.548+0000 7f642010bb80 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55db3a3c6630 0x55db3a3c8a50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.548+0000 7f642010bb80 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55db3a3567a0 0x55db3a36cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.548+0000 7f642010bb80 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55db3a372fd0 0x55db3a378aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.548+0000 7f63ef400700 0 client.27059 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.548+0000 7f642010bb80 1 --2- 172.21.15.67:0/1721994962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db3a3570c0 0x55db3a36d4b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.548+0000 7f642010bb80 1 -- 172.21.15.67:0/1721994962 >> 172.21.15.67:0/1721994962 conn(0x55db3a2b66a0 msgr2=0x55db3a35fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:13.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.549+0000 7f642010bb80 1 -- 172.21.15.67:0/1721994962 shutdown_connections 2024-03-30T02:44:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.549+0000 7f642010bb80 1 -- 172.21.15.67:0/1721994962 wait complete. 2024-03-30T02:44:13.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.550+0000 7f642010bb80 20 client.27059 trim_cache size 0 max 16384 2024-03-30T02:44:13.550 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 -- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63e80a1390 msgr2=0x7f63e80a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63e80a1390 0x7f63e80a3850 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f63f400b780 tx=0x7f63f40096e0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 -- 172.21.15.67:0/1265197858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64041a1f40 msgr2=0x7f64041a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64041a1f40 0x7f64041a23c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f63f000e970 tx=0x7f63f000ec80 comp rx=0 tx=0).stop 2024-03-30T02:44:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 -- 172.21.15.67:0/1265197858 shutdown_connections 2024-03-30T02:44:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f63e80a1390 0x7f63e80a3850 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64041a1f40 0x7f64041a23c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6404107560 0x7f640419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 --2- 172.21.15.67:0/1265197858 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f640410c530 0x7f640419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 -- 172.21.15.67:0/1265197858 >> 172.21.15.67:0/1265197858 conn(0x7f6404076bd0 msgr2=0x7f6404077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 -- 172.21.15.67:0/1265197858 shutdown_connections 2024-03-30T02:44:13.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:13.558+0000 7f640b000700 1 -- 172.21.15.67:0/1265197858 wait complete. 2024-03-30T02:44:13.636 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:44:13.770 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/genhd.o 2024-03-30T02:44:13.785 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/ntp.o 2024-03-30T02:44:13.788 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ramfs/file-mmu.o 2024-03-30T02:44:13.822 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/decompress/zstd_ddict.o 2024-03-30T02:44:13.838 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/rsrc.o 2024-03-30T02:44:14.041 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/swapfile.o 2024-03-30T02:44:14.083 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/decompress/zstd_decompress.o 2024-03-30T02:44:14.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.089+0000 7efcf2000700 1 -- 172.21.15.67:0/2090231947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec10bf70 msgr2=0x7efcec10c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:14.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.089+0000 7efcf2000700 1 --2- 172.21.15.67:0/2090231947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec10bf70 0x7efcec10c3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efcd800baf0 tx=0x7efcd800be00 comp rx=0 tx=0).stop 2024-03-30T02:44:14.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.092+0000 7efcf2000700 1 -- 172.21.15.67:0/2090231947 shutdown_connections 2024-03-30T02:44:14.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.092+0000 7efcf2000700 1 --2- 172.21.15.67:0/2090231947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcec108470 0x7efcec108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.092+0000 7efcf2000700 1 --2- 172.21.15.67:0/2090231947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec10cb20 0x7efcec0790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.092+0000 7efcf2000700 1 --2- 172.21.15.67:0/2090231947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec10bf70 0x7efcec10c3f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.092+0000 7efcf2000700 1 -- 172.21.15.67:0/2090231947 >> 172.21.15.67:0/2090231947 conn(0x7efcec06bdf0 msgr2=0x7efcec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:14.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.093+0000 7efcf2000700 1 -- 172.21.15.67:0/2090231947 shutdown_connections 2024-03-30T02:44:14.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.093+0000 7efcf2000700 1 -- 172.21.15.67:0/2090231947 wait complete. 2024-03-30T02:44:14.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.095+0000 7efcf2000700 1 Processor -- start 2024-03-30T02:44:14.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.095+0000 7efcf2000700 1 -- start start 2024-03-30T02:44:14.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.096+0000 7efcf2000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec108470 0x7efcec078c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.096+0000 7efcf2000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcec10bf70 0x7efcec073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.096+0000 7efcf2000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec10cb20 0x7efcec0735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.096+0000 7efcf2000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec073bb0 con 0x7efcec10bf70 2024-03-30T02:44:14.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.096+0000 7efceb400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec108470 0x7efcec078c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.096+0000 7efceb400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec108470 0x7efcec078c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37986/0 (socket says 172.21.15.67:37986) 2024-03-30T02:44:14.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.096+0000 7efceb400700 1 -- 172.21.15.67:0/763533515 learned_addr learned my addr 172.21.15.67:0/763533515 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:14.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.097+0000 7efcebe00700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec10cb20 0x7efcec0735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.097+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec073d20 con 0x7efcec10cb20 2024-03-30T02:44:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.097+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec073e90 con 0x7efcec108470 2024-03-30T02:44:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.097+0000 7efcebe00700 1 -- 172.21.15.67:0/763533515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec108470 msgr2=0x7efcec078c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.097+0000 7efcebe00700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec108470 0x7efcec078c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.097+0000 7efcebe00700 1 -- 172.21.15.67:0/763533515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcec10bf70 msgr2=0x7efcec073070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:44:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.097+0000 7efcebe00700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcec10bf70 0x7efcec073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.098+0000 7efcebe00700 1 -- 172.21.15.67:0/763533515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd80095f0 con 0x7efcec10cb20 2024-03-30T02:44:14.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.102+0000 7efcebe00700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec10cb20 0x7efcec0735b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00e970 tx=0x7efcdc00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:14.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.103+0000 7efce3e00700 1 -- 172.21.15.67:0/763533515 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00cbd0 con 0x7efcec10cb20 2024-03-30T02:44:14.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.103+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcec1bb010 con 0x7efcec10cb20 2024-03-30T02:44:14.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.103+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec1bb4f0 con 0x7efcec10cb20 2024-03-30T02:44:14.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.103+0000 7efce3e00700 1 -- 172.21.15.67:0/763533515 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcdc0044e0 con 0x7efcec10cb20 2024-03-30T02:44:14.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.103+0000 7efce3e00700 1 -- 172.21.15.67:0/763533515 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00cbd0 con 0x7efcec10cb20 2024-03-30T02:44:14.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.114+0000 7efce3e00700 1 -- 172.21.15.67:0/763533515 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efcdc00cd30 con 0x7efcec10cb20 2024-03-30T02:44:14.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.116+0000 7efce3e00700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efcd00a1370 0x7efcd00a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.117+0000 7efceb400700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efcd00a1370 0x7efcd00a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.117+0000 7efd05affb80 1 Processor -- start 2024-03-30T02:44:14.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.118+0000 7efd05affb80 1 -- start start 2024-03-30T02:44:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.118+0000 7efceb400700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efcd00a1370 0x7efcd00a3830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efce4007fd0 tx=0x7efce4009040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.118+0000 7efd05affb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25e5d0 0x555dda25e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.118+0000 7efd05affb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda2698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.118+0000 7efd05affb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda269df0 0x555dda26c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.118+0000 7efce3e00700 1 -- 172.21.15.67:0/763533515 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7efcdc014070 con 0x7efcec10cb20 2024-03-30T02:44:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.119+0000 7efceaa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda2698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.119+0000 7efcebe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda269df0 0x555dda26c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efceaa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda2698b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:37994/0 (socket says 172.21.15.67:37994) 2024-03-30T02:44:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efceaa00700 1 -- 172.21.15.67:0/3263449618 learned_addr learned my addr 172.21.15.67:0/3263449618 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efd05affb80 1 -- 172.21.15.67:0/3263449618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda1b7990 con 0x555dda269df0 2024-03-30T02:44:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efd05affb80 1 -- 172.21.15.67:0/3263449618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda1b5350 con 0x555dda25e5d0 2024-03-30T02:44:14.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efd05affb80 1 -- 172.21.15.67:0/3263449618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda1b0c40 con 0x555dda25eef0 2024-03-30T02:44:14.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efcebe00700 1 -- 172.21.15.67:0/3263449618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 msgr2=0x555dda2698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:14.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efcebe00700 1 --2- 172.21.15.67:0/3263449618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda2698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efcebe00700 1 -- 172.21.15.67:0/3263449618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25e5d0 msgr2=0x555dda25e9b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:14.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efcebe00700 1 --2- 172.21.15.67:0/3263449618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25e5d0 0x555dda25e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.120+0000 7efcebe00700 1 -- 172.21.15.67:0/3263449618 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dda0323b0 con 0x555dda269df0 2024-03-30T02:44:14.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.121+0000 7efcebe00700 1 --2- 172.21.15.67:0/3263449618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda269df0 0x555dda26c1e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0c9ef0 tx=0x7efcdc09e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ae15af236c3118 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:14.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.122+0000 7efcd7e00700 1 -- 172.21.15.67:0/3263449618 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0b7ea0 con 0x555dda269df0 2024-03-30T02:44:14.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.122+0000 7efcd7e00700 1 -- 172.21.15.67:0/3263449618 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcdc099fb0 con 0x555dda269df0 2024-03-30T02:44:14.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.122+0000 7efcd7e00700 1 -- 172.21.15.67:0/3263449618 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0ca170 con 0x555dda269df0 2024-03-30T02:44:14.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.123+0000 7efd05affb80 1 -- 172.21.15.67:0/3263449618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda269df0 msgr2=0x555dda26c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:14.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.123+0000 7efd05affb80 1 --2- 172.21.15.67:0/3263449618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda269df0 0x555dda26c1e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0c9ef0 tx=0x7efcdc09e590 comp rx=0 tx=0).stop 2024-03-30T02:44:14.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.127+0000 7efd05affb80 1 -- 172.21.15.67:0/3263449618 shutdown_connections 2024-03-30T02:44:14.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.127+0000 7efd05affb80 1 --2- 172.21.15.67:0/3263449618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda269df0 0x555dda26c1e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.127+0000 7efd05affb80 1 --2- 172.21.15.67:0/3263449618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25e5d0 0x555dda25e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.127+0000 7efd05affb80 1 --2- 172.21.15.67:0/3263449618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda2698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.127+0000 7efd05affb80 1 -- 172.21.15.67:0/3263449618 >> 172.21.15.67:0/3263449618 conn(0x555dda1be6a0 msgr2=0x555dda25aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:14.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.128+0000 7efd05affb80 1 -- 172.21.15.67:0/3263449618 shutdown_connections 2024-03-30T02:44:14.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.128+0000 7efd05affb80 1 -- 172.21.15.67:0/3263449618 wait complete. 2024-03-30T02:44:14.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.129+0000 7efd05affb80 1 -- start start 2024-03-30T02:44:14.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.129+0000 7efd05affb80 1 Processor -- start 2024-03-30T02:44:14.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.130+0000 7efd05affb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25e5d0 0x555dda269890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.130+0000 7efd05affb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25eef0 0x555dda266b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.130+0000 7efd05affb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.130+0000 7efd05affb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda14cd80 con 0x555dda25eef0 2024-03-30T02:44:14.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.130+0000 7efd05affb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda14ca20 con 0x555dda25e5d0 2024-03-30T02:44:14.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.130+0000 7efd05affb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda0346a0 con 0x555dda269df0 2024-03-30T02:44:14.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.130+0000 7efd05affb80 10 client.0 _ll_register_callbacks cb 0x555dda156470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:14.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efcebe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efcebe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2672c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:38008/0 (socket says 172.21.15.67:38008) 2024-03-30T02:44:14.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efcebe00700 1 -- 172.21.15.67:0/3264378910 learned_addr learned my addr 172.21.15.67:0/3264378910 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:14.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efceaa00700 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25eef0 0x555dda266b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efcebe00700 1 -- 172.21.15.67:0/3264378910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25e5d0 msgr2=0x555dda269890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:14.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efcebe00700 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25e5d0 0x555dda269890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efcebe00700 1 -- 172.21.15.67:0/3264378910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25eef0 msgr2=0x555dda266b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:14.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efcebe00700 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25eef0 0x555dda266b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efcebe00700 1 -- 172.21.15.67:0/3264378910 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec1bb4f0 con 0x555dda269df0 2024-03-30T02:44:14.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.131+0000 7efceaa00700 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25eef0 0x555dda266b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:14.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.132+0000 7efcebe00700 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2672c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0959a0 tx=0x7efcdc0ca3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679a6554747d3a71 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:14.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.134+0000 7efcd4c00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:14.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.134+0000 7efcd4c00700 1 -- 172.21.15.67:0/3264378910 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0998b0 con 0x555dda269df0 2024-03-30T02:44:14.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.134+0000 7efcd4c00700 1 -- 172.21.15.67:0/3264378910 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7efcdc0a1d30 con 0x555dda269df0 2024-03-30T02:44:14.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.137+0000 7efd05affb80 1 -- 172.21.15.67:0/3264378910 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555dda171380 con 0x555dda269df0 2024-03-30T02:44:14.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.137+0000 7efcd4c00700 1 -- 172.21.15.67:0/3264378910 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcdc0b2230 con 0x555dda269df0 2024-03-30T02:44:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.137+0000 7efd05affb80 10 client.27031 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.137+0000 7efd05affb80 1 -- 172.21.15.67:0/3264378910 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555dd9fdeb60 con 0x555dda269df0 2024-03-30T02:44:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.138+0000 7efcd4c00700 1 -- 172.21.15.67:0/3264378910 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcdc07f030 con 0x555dda269df0 2024-03-30T02:44:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.138+0000 7efcd4c00700 1 -- 172.21.15.67:0/3264378910 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efcdc0988a0 con 0x555dda269df0 2024-03-30T02:44:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.138+0000 7efd05affb80 10 client.27031 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.138+0000 7efd05affb80 10 client.27031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.138+0000 7efd05affb80 20 client.27031 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.138+0000 7efd05affb80 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555dda279420 0x555dda27b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.138+0000 7efd05affb80 4 client.27031 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:14.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.138+0000 7efd05affb80 1 -- 172.21.15.67:0/3264378910 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dda1a9700 con 0x555dda279420 2024-03-30T02:44:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.142+0000 7efceb400700 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555dda279420 0x555dda27b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.142+0000 7efceb400700 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555dda279420 0x555dda27b840 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9036571dfe4be824 server_cookie=c44b9358ea423aeb in_seq=0 out_seq=0 2024-03-30T02:44:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.142+0000 7efcd4c00700 10 client.27031 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.143+0000 7efcd4c00700 1 -- 172.21.15.67:0/3264378910 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555dda1a9700 con 0x555dda279420 2024-03-30T02:44:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.143+0000 7efcd4c00700 10 client.27031 handle_command_reply: tid=0 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.143+0000 7efd05affb80 1 client.27031 shutdown 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.144+0000 7efd05affb80 1 -- 172.21.15.67:0/3264378910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 msgr2=0x555dda2672c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.144+0000 7efd05affb80 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2672c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0959a0 tx=0x7efcdc0ca3a0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.146+0000 7efd05affb80 1 -- 172.21.15.67:0/3264378910 shutdown_connections 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.146+0000 7efd05affb80 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2672c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.146+0000 7efd05affb80 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25e5d0 0x555dda269890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.146+0000 7efd05affb80 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555dda279420 0x555dda27b840 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.146+0000 7efd05affb80 1 --2- 172.21.15.67:0/3264378910 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25eef0 0x555dda266b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.146+0000 7efcd4c00700 0 client.27031 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.146+0000 7efd05affb80 1 -- 172.21.15.67:0/3264378910 >> 172.21.15.67:0/3264378910 conn(0x555dda1be6a0 msgr2=0x555dda2606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.146+0000 7efd05affb80 1 -- 172.21.15.67:0/3264378910 shutdown_connections 2024-03-30T02:44:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.146+0000 7efd05affb80 1 -- 172.21.15.67:0/3264378910 wait complete. 2024-03-30T02:44:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.149+0000 7efd05affb80 20 client.27031 trim_cache size 0 max 16384 2024-03-30T02:44:14.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.191+0000 7efd05affb80 1 Processor -- start 2024-03-30T02:44:14.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.191+0000 7efd05affb80 1 -- start start 2024-03-30T02:44:14.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.191+0000 7efd05affb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25e5d0 0x555dda266b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.191+0000 7efd05affb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25eef0 0x555dda2708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.191+0000 7efd05affb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.191+0000 7efd05affb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda260130 con 0x555dda25e5d0 2024-03-30T02:44:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.191+0000 7efd05affb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda270e00 con 0x555dda25eef0 2024-03-30T02:44:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.191+0000 7efd05affb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda2650d0 con 0x555dda269df0 2024-03-30T02:44:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25e5d0 0x555dda266b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceaa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25eef0 0x555dda2708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25e5d0 0x555dda266b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46202/0 (socket says 172.21.15.67:46202) 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceb400700 1 -- 172.21.15.67:0/1697382695 learned_addr learned my addr 172.21.15.67:0/1697382695 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efcebe00700 1 --2- 172.21.15.67:0/1697382695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceb400700 1 -- 172.21.15.67:0/1697382695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 msgr2=0x555dda2535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceb400700 1 --2- 172.21.15.67:0/1697382695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceb400700 1 -- 172.21.15.67:0/1697382695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25eef0 msgr2=0x555dda2708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceb400700 1 --2- 172.21.15.67:0/1697382695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25eef0 0x555dda2708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceb400700 1 -- 172.21.15.67:0/1697382695 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dda176ea0 con 0x555dda25e5d0 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.192+0000 7efceaa00700 1 --2- 172.21.15.67:0/1697382695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25eef0 0x555dda2708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:14.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.193+0000 7efceb400700 1 --2- 172.21.15.67:0/1697382695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25e5d0 0x555dda266b40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x555dda27c110 tx=0x7efce4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d8afbf5493276f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.193+0000 7efce1200700 1 -- 172.21.15.67:0/1697382695 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4023130 con 0x555dda25e5d0 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.193+0000 7efce1200700 1 -- 172.21.15.67:0/1697382695 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efce4023290 con 0x555dda25e5d0 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.193+0000 7efce1200700 1 -- 172.21.15.67:0/1697382695 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4022400 con 0x555dda25e5d0 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.195+0000 7efd05affb80 1 -- 172.21.15.67:0/1697382695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25e5d0 msgr2=0x555dda266b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.195+0000 7efd05affb80 1 --2- 172.21.15.67:0/1697382695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25e5d0 0x555dda266b40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x555dda27c110 tx=0x7efce4011000 comp rx=0 tx=0).stop 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.197+0000 7efd05affb80 1 -- 172.21.15.67:0/1697382695 shutdown_connections 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.197+0000 7efd05affb80 1 --2- 172.21.15.67:0/1697382695 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda25e5d0 0x555dda266b40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.197+0000 7efd05affb80 1 --2- 172.21.15.67:0/1697382695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda269df0 0x555dda2535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.197+0000 7efd05affb80 1 --2- 172.21.15.67:0/1697382695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda25eef0 0x555dda2708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.197+0000 7efd05affb80 1 -- 172.21.15.67:0/1697382695 >> 172.21.15.67:0/1697382695 conn(0x555dda1be6a0 msgr2=0x555dda266410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:14.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.198+0000 7efd05affb80 1 -- 172.21.15.67:0/1697382695 shutdown_connections 2024-03-30T02:44:14.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.198+0000 7efd05affb80 1 -- 172.21.15.67:0/1697382695 wait complete. 2024-03-30T02:44:14.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.199+0000 7efd05affb80 1 -- start start 2024-03-30T02:44:14.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.199+0000 7efd05affb80 1 Processor -- start 2024-03-30T02:44:14.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.203+0000 7efd05affb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda270540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.203+0000 7efd05affb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda269df0 0x555dda26e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.203+0000 7efd05affb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda26ee80 0x555dda26f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.203+0000 7efd05affb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda270c20 con 0x555dda26ee80 2024-03-30T02:44:14.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.203+0000 7efd05affb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda2cd220 con 0x555dda269df0 2024-03-30T02:44:14.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.203+0000 7efd05affb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dda2cd390 con 0x555dda25eef0 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.203+0000 7efd05affb80 10 client.0 _ll_register_callbacks cb 0x555dda156470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.204+0000 7efceaa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda269df0 0x555dda26e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.204+0000 7efceb400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda270540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.204+0000 7efceaa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda269df0 0x555dda26e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49098/0 (socket says 172.21.15.67:49098) 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.204+0000 7efceaa00700 1 -- 172.21.15.67:0/737233103 learned_addr learned my addr 172.21.15.67:0/737233103 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.204+0000 7efceb400700 1 -- 172.21.15.67:0/737233103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda269df0 msgr2=0x555dda26e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.204+0000 7efceb400700 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda269df0 0x555dda26e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.204+0000 7efceb400700 1 -- 172.21.15.67:0/737233103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda26ee80 msgr2=0x555dda26f300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.205+0000 7efcebe00700 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda26ee80 0x555dda26f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.205+0000 7efceb400700 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda26ee80 0x555dda26f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.205+0000 7efceb400700 1 -- 172.21.15.67:0/737233103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec079d30 con 0x555dda25eef0 2024-03-30T02:44:14.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.205+0000 7efcebe00700 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda26ee80 0x555dda26f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.206+0000 7efceb400700 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda270540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efce4016000 tx=0x7efce4023130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a0ec5a417a1162 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.206+0000 7efcd4c00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.206+0000 7efcd4c00700 1 -- 172.21.15.67:0/737233103 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4031070 con 0x555dda25eef0 2024-03-30T02:44:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.206+0000 7efcd4c00700 1 -- 172.21.15.67:0/737233103 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7efce4023500 con 0x555dda25eef0 2024-03-30T02:44:14.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.206+0000 7efd05affb80 1 -- 172.21.15.67:0/737233103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcdc099fb0 con 0x555dda25eef0 2024-03-30T02:44:14.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.211+0000 7efcd4c00700 1 -- 172.21.15.67:0/737233103 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7efce4015df0 con 0x555dda25eef0 2024-03-30T02:44:14.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.211+0000 7efd05affb80 10 client.27034 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:14.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.211+0000 7efd05affb80 1 -- 172.21.15.67:0/737233103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7efce400f040 con 0x555dda25eef0 2024-03-30T02:44:14.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.211+0000 7efcd4c00700 1 -- 172.21.15.67:0/737233103 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcdc099fb0 con 0x555dda25eef0 2024-03-30T02:44:14.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.211+0000 7efcd4c00700 1 -- 172.21.15.67:0/737233103 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efce4005040 con 0x555dda25eef0 2024-03-30T02:44:14.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.211+0000 7efd05affb80 10 client.27034 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:14.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.211+0000 7efd05affb80 10 client.27034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:14.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.211+0000 7efd05affb80 20 client.27034 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:14.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.212+0000 7efd05affb80 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555dda2cea70 0x555dda2d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:14.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.212+0000 7efd05affb80 4 client.27034 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:14.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.212+0000 7efd05affb80 1 -- 172.21.15.67:0/737233103 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555dd9fffcf0 con 0x555dda2cea70 2024-03-30T02:44:14.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.212+0000 7efceaa00700 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555dda2cea70 0x555dda2d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:14.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.214+0000 7efcd4c00700 10 client.27034 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:14.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.214+0000 7efceaa00700 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555dda2cea70 0x555dda2d0e90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d17323d276c18e6 server_cookie=273b97e6d5f587ab in_seq=0 out_seq=0 2024-03-30T02:44:14.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.214+0000 7efcd4c00700 1 -- 172.21.15.67:0/737233103 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555dd9fffcf0 con 0x555dda2cea70 2024-03-30T02:44:14.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.214+0000 7efcd4c00700 10 client.27034 handle_command_reply: tid=0 2024-03-30T02:44:14.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.214+0000 7efd05affb80 1 client.27034 shutdown 2024-03-30T02:44:14.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 -- 172.21.15.67:0/737233103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 msgr2=0x555dda270540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:14.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda270540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efce4016000 tx=0x7efce4023130 comp rx=0 tx=0).stop 2024-03-30T02:44:14.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 -- 172.21.15.67:0/737233103 shutdown_connections 2024-03-30T02:44:14.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x555dda2cea70 0x555dda2d0e90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x555dda26ee80 0x555dda26f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555dda25eef0 0x555dda270540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efcd4c00700 0 client.27034 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:14.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 --2- 172.21.15.67:0/737233103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555dda269df0 0x555dda26e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 -- 172.21.15.67:0/737233103 >> 172.21.15.67:0/737233103 conn(0x555dda1be6a0 msgr2=0x555dda266410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:14.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 -- 172.21.15.67:0/737233103 shutdown_connections 2024-03-30T02:44:14.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.215+0000 7efd05affb80 1 -- 172.21.15.67:0/737233103 wait complete. 2024-03-30T02:44:14.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.219+0000 7efd05affb80 20 client.27034 trim_cache size 0 max 16384 2024-03-30T02:44:14.222 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efcd00a1370 msgr2=0x7efcd00a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efcd00a1370 0x7efcd00a3830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efce4007fd0 tx=0x7efce4009040 comp rx=0 tx=0).stop 2024-03-30T02:44:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec10cb20 msgr2=0x7efcec0735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:14.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec10cb20 0x7efcec0735b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00e970 tx=0x7efcdc00ed30 comp rx=0 tx=0).stop 2024-03-30T02:44:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 shutdown_connections 2024-03-30T02:44:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efcd00a1370 0x7efcd00a3830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec10cb20 0x7efcec0735b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcec108470 0x7efcec078c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 --2- 172.21.15.67:0/763533515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efcec10bf70 0x7efcec073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:14.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.225+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 >> 172.21.15.67:0/763533515 conn(0x7efcec06bdf0 msgr2=0x7efcec1090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.227+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 shutdown_connections 2024-03-30T02:44:14.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:14.227+0000 7efcf2000700 1 -- 172.21.15.67:0/763533515 wait complete. 2024-03-30T02:44:14.317 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/revoke.o 2024-03-30T02:44:14.478 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/ramfs/built-in.a 2024-03-30T02:44:14.537 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/hugetlbfs/inode.o 2024-03-30T02:44:14.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:14 smithi067 ceph-mon[37808]: pgmap v220: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 416 op/s 2024-03-30T02:44:14.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:14 smithi170 ceph-mon[78798]: pgmap v220: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 416 op/s 2024-03-30T02:44:14.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/clocksource.o 2024-03-30T02:44:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:14 smithi082 ceph-mon[79176]: pgmap v220: 129 pgs: 129 active+clean; 1.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 416 op/s 2024-03-30T02:44:14.791 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/decompress/zstd_decompress_block.o 2024-03-30T02:44:15.010 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/ioprio.o 2024-03-30T02:44:15.109 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/file.o 2024-03-30T02:44:15.198 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/jbd2/journal.o 2024-03-30T02:44:15.270 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:44:15.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:44:15.530 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/rw.o 2024-03-30T02:44:15.697 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/jiffies.o 2024-03-30T02:44:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.788+0000 7f6782000700 1 -- 172.21.15.67:0/3680934912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c11a240 msgr2=0x7f677c11c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:15.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.788+0000 7f6782000700 1 --2- 172.21.15.67:0/3680934912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c11a240 0x7f677c11c630 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6774009990 tx=0x7f6774009ca0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.788+0000 7f6782000700 1 -- 172.21.15.67:0/3680934912 shutdown_connections 2024-03-30T02:44:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.789+0000 7f6782000700 1 --2- 172.21.15.67:0/3680934912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f677c10ce90 0x7f677c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.789+0000 7f6782000700 1 --2- 172.21.15.67:0/3680934912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c11a240 0x7f677c11c630 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.789+0000 7f6782000700 1 --2- 172.21.15.67:0/3680934912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c10d7b0 0x7f677c119d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.789+0000 7f6782000700 1 -- 172.21.15.67:0/3680934912 >> 172.21.15.67:0/3680934912 conn(0x7f677c06bc10 msgr2=0x7f677c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.789+0000 7f6782000700 1 -- 172.21.15.67:0/3680934912 shutdown_connections 2024-03-30T02:44:15.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.790+0000 7f6782000700 1 -- 172.21.15.67:0/3680934912 wait complete. 2024-03-30T02:44:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.795+0000 7f6782000700 1 Processor -- start 2024-03-30T02:44:15.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.795+0000 7f6782000700 1 -- start start 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.796+0000 7f6782000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c10ce90 0x7f677c1a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.796+0000 7f6782000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f677c10d7b0 0x7f677c1a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.796+0000 7f6782000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c11a240 0x7f677c1aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.796+0000 7f6782000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c1aad20 con 0x7f677c10d7b0 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.796+0000 7f6782000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c1aae90 con 0x7f677c10ce90 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.796+0000 7f6782000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c1ab050 con 0x7f677c11a240 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.796+0000 7f6780c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c10ce90 0x7f677c1a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.797+0000 7f6780c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c10ce90 0x7f677c1a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49114/0 (socket says 172.21.15.67:49114) 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.797+0000 7f6780c00700 1 -- 172.21.15.67:0/3922650434 learned_addr learned my addr 172.21.15.67:0/3922650434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.797+0000 7f6781600700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c11a240 0x7f677c1aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.797+0000 7f6773e00700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f677c10d7b0 0x7f677c1a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.797+0000 7f6780c00700 1 -- 172.21.15.67:0/3922650434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c11a240 msgr2=0x7f677c1aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.797+0000 7f6780c00700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c11a240 0x7f677c1aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.797+0000 7f6780c00700 1 -- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f677c10d7b0 msgr2=0x7f677c1a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.798+0000 7f6780c00700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f677c10d7b0 0x7f677c1a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.798+0000 7f6780c00700 1 -- 172.21.15.67:0/3922650434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c009c40 con 0x7f677c10ce90 2024-03-30T02:44:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.798+0000 7f6781600700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c11a240 0x7f677c1aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:15.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.798+0000 7f6773e00700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f677c10d7b0 0x7f677c1a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:15.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.800+0000 7f6780c00700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c10ce90 0x7f677c1a5e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f676c00c370 tx=0x7f676c00c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.800+0000 7f677a000700 1 -- 172.21.15.67:0/3922650434 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c017070 con 0x7f677c10ce90 2024-03-30T02:44:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.801+0000 7f677a000700 1 -- 172.21.15.67:0/3922650434 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f676c00f040 con 0x7f677c10ce90 2024-03-30T02:44:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.801+0000 7f677a000700 1 -- 172.21.15.67:0/3922650434 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c00e050 con 0x7f677c10ce90 2024-03-30T02:44:15.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.801+0000 7f6782000700 1 -- 172.21.15.67:0/3922650434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67740095f0 con 0x7f677c10ce90 2024-03-30T02:44:15.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.801+0000 7f6782000700 1 -- 172.21.15.67:0/3922650434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c1b2990 con 0x7f677c10ce90 2024-03-30T02:44:15.806 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/hugetlbfs/built-in.a 2024-03-30T02:44:15.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.808+0000 7f677a000700 1 -- 172.21.15.67:0/3922650434 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f676c004440 con 0x7f677c10ce90 2024-03-30T02:44:15.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.811+0000 7f677a000700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67600a1310 0x7f67600a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.812+0000 7f677a000700 1 -- 172.21.15.67:0/3922650434 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f676c0cc990 con 0x7f677c10ce90 2024-03-30T02:44:15.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.812+0000 7f6773e00700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67600a1310 0x7f67600a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.812 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.812+0000 7f67970afb80 1 Processor -- start 2024-03-30T02:44:15.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.813+0000 7f6773e00700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67600a1310 0x7f67600a37d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f677c1a7350 tx=0x7f6764009190 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:15.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.814+0000 7f67970afb80 1 -- start start 2024-03-30T02:44:15.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.814+0000 7f67970afb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 0x55712f304c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.814+0000 7f67970afb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f305140 0x55712f30fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.815+0000 7f67970afb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f310040 0x55712f312430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.815+0000 7f6773e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f305140 0x55712f30fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.815+0000 7f6773e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f305140 0x55712f30fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:38054/0 (socket says 172.21.15.67:38054) 2024-03-30T02:44:15.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.815+0000 7f6773e00700 1 -- 172.21.15.67:0/3631258244 learned_addr learned my addr 172.21.15.67:0/3631258244 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:15.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.815+0000 7f67970afb80 1 -- 172.21.15.67:0/3631258244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f25da10 con 0x55712f304820 2024-03-30T02:44:15.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.815+0000 7f67970afb80 1 -- 172.21.15.67:0/3631258244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f25b3d0 con 0x55712f310040 2024-03-30T02:44:15.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.815+0000 7f67970afb80 1 -- 172.21.15.67:0/3631258244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f256cc0 con 0x55712f305140 2024-03-30T02:44:15.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.817+0000 7f6773e00700 1 -- 172.21.15.67:0/3631258244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f310040 msgr2=0x55712f312430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:15.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.817+0000 7f6773e00700 1 --2- 172.21.15.67:0/3631258244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f310040 0x55712f312430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.817+0000 7f6773e00700 1 -- 172.21.15.67:0/3631258244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 msgr2=0x55712f304c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:15.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.817+0000 7f6773e00700 1 --2- 172.21.15.67:0/3631258244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 0x55712f304c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.817+0000 7f6773e00700 1 -- 172.21.15.67:0/3631258244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55712f0d8430 con 0x55712f305140 2024-03-30T02:44:15.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.820+0000 7f6773e00700 1 --2- 172.21.15.67:0/3631258244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f305140 0x55712f30fb00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f676400ede0 tx=0x7f6764011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96d0bb14683af24 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:15.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.820+0000 7f6771c00700 1 -- 172.21.15.67:0/3631258244 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764015890 con 0x55712f305140 2024-03-30T02:44:15.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.820+0000 7f6771c00700 1 -- 172.21.15.67:0/3631258244 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6764013450 con 0x55712f305140 2024-03-30T02:44:15.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.820+0000 7f6771c00700 1 -- 172.21.15.67:0/3631258244 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676401f650 con 0x55712f305140 2024-03-30T02:44:15.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.821+0000 7f67970afb80 1 -- 172.21.15.67:0/3631258244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f305140 msgr2=0x55712f30fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:15.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.821+0000 7f67970afb80 1 --2- 172.21.15.67:0/3631258244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f305140 0x55712f30fb00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f676400ede0 tx=0x7f6764011000 comp rx=0 tx=0).stop 2024-03-30T02:44:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.825+0000 7f67970afb80 1 -- 172.21.15.67:0/3631258244 shutdown_connections 2024-03-30T02:44:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.825+0000 7f67970afb80 1 --2- 172.21.15.67:0/3631258244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 0x55712f304c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.825+0000 7f67970afb80 1 --2- 172.21.15.67:0/3631258244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f310040 0x55712f312430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.825+0000 7f67970afb80 1 --2- 172.21.15.67:0/3631258244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f305140 0x55712f30fb00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.825+0000 7f67970afb80 1 -- 172.21.15.67:0/3631258244 >> 172.21.15.67:0/3631258244 conn(0x55712f264720 msgr2=0x55712f300ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:15.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.827+0000 7f67970afb80 1 -- 172.21.15.67:0/3631258244 shutdown_connections 2024-03-30T02:44:15.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.827+0000 7f67970afb80 1 -- 172.21.15.67:0/3631258244 wait complete. 2024-03-30T02:44:15.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.828+0000 7f67970afb80 1 -- start start 2024-03-30T02:44:15.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.829+0000 7f67970afb80 1 Processor -- start 2024-03-30T02:44:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.831+0000 7f67970afb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 0x55712f30fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.831+0000 7f67970afb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f30ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.832+0000 7f6780c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 0x55712f30fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.832+0000 7f6780c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 0x55712f30fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58930/0 (socket says 172.21.15.67:58930) 2024-03-30T02:44:15.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.832+0000 7f6773e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f30ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f6773e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f30ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57916/0 (socket says 172.21.15.67:57916) 2024-03-30T02:44:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f6780c00700 1 -- 172.21.15.67:0/829246527 learned_addr learned my addr 172.21.15.67:0/829246527 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f67970afb80 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f310040 0x55712f30d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f1f2e00 con 0x55712f304820 2024-03-30T02:44:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f1f2aa0 con 0x55712f305140 2024-03-30T02:44:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f0da720 con 0x55712f310040 2024-03-30T02:44:15.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f67970afb80 10 client.0 _ll_register_callbacks cb 0x55712f1fc4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:15.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f6781600700 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f310040 0x55712f30d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f6780c00700 1 -- 172.21.15.67:0/829246527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f310040 msgr2=0x55712f30d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:15.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f6780c00700 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f310040 0x55712f30d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f6780c00700 1 -- 172.21.15.67:0/829246527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 msgr2=0x55712f30ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:15.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f6780c00700 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f30ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f6780c00700 1 -- 172.21.15.67:0/829246527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676400eac0 con 0x55712f304820 2024-03-30T02:44:15.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.833+0000 7f6781600700 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f310040 0x55712f30d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:15.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.836+0000 7f6780c00700 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 0x55712f30fa00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f676c096170 tx=0x7f676c0995a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9b1aa281784323 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:15.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.836+0000 7f676a000700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:44:15.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.837+0000 7f676a000700 1 -- 172.21.15.67:0/829246527 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c099e00 con 0x55712f304820 2024-03-30T02:44:15.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.837+0000 7f676a000700 1 -- 172.21.15.67:0/829246527 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f676c099700 con 0x55712f304820 2024-03-30T02:44:15.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.837+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55712f217400 con 0x55712f304820 2024-03-30T02:44:15.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.837+0000 7f676a000700 1 -- 172.21.15.67:0/829246527 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f676c09d530 con 0x55712f304820 2024-03-30T02:44:15.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.837+0000 7f67970afb80 10 client.16965 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:15.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.837+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55712f084be0 con 0x55712f304820 2024-03-30T02:44:15.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.838+0000 7f676a000700 1 -- 172.21.15.67:0/829246527 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f676c0a1b80 con 0x55712f304820 2024-03-30T02:44:15.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.838+0000 7f676a000700 1 -- 172.21.15.67:0/829246527 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f676c0b3020 con 0x55712f304820 2024-03-30T02:44:15.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.838+0000 7f67970afb80 10 client.16965 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:15.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.838+0000 7f67970afb80 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:15.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.838+0000 7f67970afb80 20 client.16965 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:15.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.838+0000 7f67970afb80 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55712f31fd70 0x55712f322190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.838+0000 7f67970afb80 4 client.16965 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:15.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.838+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55712f217400 con 0x55712f31fd70 2024-03-30T02:44:15.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.839+0000 7f6773e00700 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55712f31fd70 0x55712f322190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.841+0000 7f676a000700 10 client.16965 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:15.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.841+0000 7f6773e00700 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55712f31fd70 0x55712f322190 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08034a3a002aa97 server_cookie=cfb4cb508e505c50 in_seq=0 out_seq=0 2024-03-30T02:44:15.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.843+0000 7f676a000700 1 -- 172.21.15.67:0/829246527 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55712f217400 con 0x55712f31fd70 2024-03-30T02:44:15.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.843+0000 7f676a000700 10 client.16965 handle_command_reply: tid=0 2024-03-30T02:44:15.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.846+0000 7f67970afb80 1 client.16965 shutdown 2024-03-30T02:44:15.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.846+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 msgr2=0x55712f30fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:15.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.846+0000 7f67970afb80 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 0x55712f30fa00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f676c096170 tx=0x7f676c0995a0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.850+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 shutdown_connections 2024-03-30T02:44:15.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.850+0000 7f67970afb80 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55712f31fd70 0x55712f322190 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.850+0000 7f67970afb80 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f304820 0x55712f30fa00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.850+0000 7f67970afb80 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f310040 0x55712f30d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.850+0000 7f67970afb80 1 --2- 172.21.15.67:0/829246527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f30ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.850+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 >> 172.21.15.67:0/829246527 conn(0x55712f264720 msgr2=0x55712f306910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:15.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.850+0000 7f676a000700 0 client.16965 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:15.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.850+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 shutdown_connections 2024-03-30T02:44:15.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.850+0000 7f67970afb80 1 -- 172.21.15.67:0/829246527 wait complete. 2024-03-30T02:44:15.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.851+0000 7f67970afb80 20 client.16965 trim_cache size 0 max 16384 2024-03-30T02:44:15.872 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timer_list.o 2024-03-30T02:44:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.892+0000 7f67970afb80 1 Processor -- start 2024-03-30T02:44:15.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.892+0000 7f67970afb80 1 -- start start 2024-03-30T02:44:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.893+0000 7f67970afb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 0x55712f316cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.893+0000 7f67970afb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f31bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.893+0000 7f67970afb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f30c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.894+0000 7f67970afb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f317230 con 0x55712f310040 2024-03-30T02:44:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.894+0000 7f67970afb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f3001e0 con 0x55712f305140 2024-03-30T02:44:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.894+0000 7f67970afb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f31c090 con 0x55712f304820 2024-03-30T02:44:15.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.894+0000 7f6781600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f30c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.894+0000 7f6781600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f30c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58932/0 (socket says 172.21.15.67:58932) 2024-03-30T02:44:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.894+0000 7f6781600700 1 -- 172.21.15.67:0/3654451694 learned_addr learned my addr 172.21.15.67:0/3654451694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.895+0000 7f6781600700 1 -- 172.21.15.67:0/3654451694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 msgr2=0x55712f316cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:15.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.895+0000 7f6781600700 1 --2- 172.21.15.67:0/3654451694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 0x55712f316cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.895+0000 7f6781600700 1 -- 172.21.15.67:0/3654451694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 msgr2=0x55712f31bb50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.895+0000 7f6781600700 1 --2- 172.21.15.67:0/3654451694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f31bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.895+0000 7f6781600700 1 -- 172.21.15.67:0/3654451694 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55712f326dc0 con 0x55712f310040 2024-03-30T02:44:15.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.896+0000 7f6781600700 1 --2- 172.21.15.67:0/3654451694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f30c5d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6774005da0 tx=0x7f6774005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4954ea0f2652a727 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:15.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.897+0000 7f6772600700 1 -- 172.21.15.67:0/3654451694 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774005300 con 0x55712f310040 2024-03-30T02:44:15.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.897+0000 7f6772600700 1 -- 172.21.15.67:0/3654451694 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f677401e050 con 0x55712f310040 2024-03-30T02:44:15.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.897+0000 7f6772600700 1 -- 172.21.15.67:0/3654451694 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774004ea0 con 0x55712f310040 2024-03-30T02:44:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.898+0000 7f67970afb80 1 -- 172.21.15.67:0/3654451694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 msgr2=0x55712f30c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:15.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.898+0000 7f67970afb80 1 --2- 172.21.15.67:0/3654451694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f30c5d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6774005da0 tx=0x7f6774005dd0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.899+0000 7f67970afb80 1 -- 172.21.15.67:0/3654451694 shutdown_connections 2024-03-30T02:44:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.899+0000 7f67970afb80 1 --2- 172.21.15.67:0/3654451694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f30c5d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.899+0000 7f67970afb80 1 --2- 172.21.15.67:0/3654451694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 0x55712f316cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.899+0000 7f67970afb80 1 --2- 172.21.15.67:0/3654451694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f31bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.899+0000 7f67970afb80 1 -- 172.21.15.67:0/3654451694 >> 172.21.15.67:0/3654451694 conn(0x55712f264720 msgr2=0x55712f31b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.899+0000 7f67970afb80 1 -- 172.21.15.67:0/3654451694 shutdown_connections 2024-03-30T02:44:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.899+0000 7f67970afb80 1 -- 172.21.15.67:0/3654451694 wait complete. 2024-03-30T02:44:15.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.900+0000 7f67970afb80 1 -- start start 2024-03-30T02:44:15.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.901+0000 7f67970afb80 1 Processor -- start 2024-03-30T02:44:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.902+0000 7f67970afb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 0x55712f36df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.902+0000 7f67970afb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f36e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.902+0000 7f6780c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 0x55712f36df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.902+0000 7f6780c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 0x55712f36df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:33116/0 (socket says 172.21.15.67:33116) 2024-03-30T02:44:15.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.902+0000 7f6780c00700 1 -- 172.21.15.67:0/2962231294 learned_addr learned my addr 172.21.15.67:0/2962231294 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.903+0000 7f67970afb80 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f315a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.903+0000 7f6773e00700 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f36e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.903+0000 7f6781600700 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f315a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.903+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f316030 con 0x55712f310040 2024-03-30T02:44:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.903+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f316170 con 0x55712f305140 2024-03-30T02:44:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.903+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712f316330 con 0x55712f304820 2024-03-30T02:44:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.905+0000 7f6780c00700 1 -- 172.21.15.67:0/2962231294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 msgr2=0x55712f36e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:15.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.905+0000 7f6780c00700 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f36e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.905+0000 7f6780c00700 1 -- 172.21.15.67:0/2962231294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 msgr2=0x55712f315a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.905+0000 7f6780c00700 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f315a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.905+0000 7f6780c00700 1 -- 172.21.15.67:0/2962231294 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c11d090 con 0x55712f304820 2024-03-30T02:44:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.905+0000 7f6781600700 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f315a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.905+0000 7f6780c00700 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 0x55712f36df20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ca340 tx=0x7f676c00e1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b152baca9788471 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.906+0000 7f67970afb80 10 client.0 _ll_register_callbacks cb 0x55712f1fc4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.906+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6764013450 con 0x55712f304820 2024-03-30T02:44:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.906+0000 7f676a000700 10 client.27046 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:15.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.906+0000 7f676a000700 1 -- 172.21.15.67:0/2962231294 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c09d530 con 0x55712f304820 2024-03-30T02:44:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.906+0000 7f676a000700 1 -- 172.21.15.67:0/2962231294 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f676c097d60 con 0x55712f304820 2024-03-30T02:44:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.906+0000 7f676a000700 1 -- 172.21.15.67:0/2962231294 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f676c099060 con 0x55712f304820 2024-03-30T02:44:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.906+0000 7f67970afb80 10 client.27046 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.906+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=39}) v3 -- 0x55712f21cf20 con 0x55712f304820 2024-03-30T02:44:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.907+0000 7f676a000700 1 -- 172.21.15.67:0/2962231294 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f676c0b3020 con 0x55712f304820 2024-03-30T02:44:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.907+0000 7f67970afb80 10 client.27046 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.907+0000 7f67970afb80 10 client.27046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.907+0000 7f67970afb80 20 client.27046 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:15.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.907+0000 7f67970afb80 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55712f375660 0x55712f377b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.908+0000 7f67970afb80 4 client.27046 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:44:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.908+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f6764013450 con 0x55712f375660 2024-03-30T02:44:15.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.908+0000 7f6773e00700 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55712f375660 0x55712f377b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.910+0000 7f6773e00700 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55712f375660 0x55712f377b00 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657fac322c070ccf server_cookie=cfedf2e9de195b6a in_seq=0 out_seq=0 2024-03-30T02:44:15.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.910+0000 7f676a000700 10 client.27046 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.911+0000 7f676a000700 1 -- 172.21.15.67:0/2962231294 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f6764013450 con 0x55712f375660 2024-03-30T02:44:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.911+0000 7f676a000700 10 client.27046 handle_command_reply: tid=0 2024-03-30T02:44:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.911+0000 7f67970afb80 1 client.27046 shutdown 2024-03-30T02:44:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.912+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 msgr2=0x55712f36df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.912+0000 7f67970afb80 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 0x55712f36df20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ca340 tx=0x7f676c00e1b0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.912+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 shutdown_connections 2024-03-30T02:44:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.912+0000 7f67970afb80 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55712f375660 0x55712f377b00 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.912+0000 7f67970afb80 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55712f310040 0x55712f315a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.912+0000 7f67970afb80 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55712f304820 0x55712f36df20 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ca340 tx=0x7f676c00e1b0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.912+0000 7f67970afb80 1 --2- 172.21.15.67:0/2962231294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55712f305140 0x55712f36e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.912+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 >> 172.21.15.67:0/2962231294 conn(0x55712f264720 msgr2=0x55712f30e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.912+0000 7f676a000700 0 client.27046 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.913+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 shutdown_connections 2024-03-30T02:44:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.913+0000 7f67970afb80 1 -- 172.21.15.67:0/2962231294 wait complete. 2024-03-30T02:44:15.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.913+0000 7f67970afb80 20 client.27046 trim_cache size 0 max 16384 2024-03-30T02:44:15.916 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:44:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:44:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "2a7394b5-5ef0-4cb8-8666-7015ebc1d011", 2024-03-30T02:44:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:44:15.916 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:44:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.919+0000 7f6782000700 1 -- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67600a1310 msgr2=0x7f67600a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.919+0000 7f6782000700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67600a1310 0x7f67600a37d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f677c1a7350 tx=0x7f6764009190 comp rx=0 tx=0).stop 2024-03-30T02:44:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.919+0000 7f6782000700 1 -- 172.21.15.67:0/3922650434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c10ce90 msgr2=0x7f677c1a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:15.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.919+0000 7f6782000700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c10ce90 0x7f677c1a5e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f676c00c370 tx=0x7f676c00c680 comp rx=0 tx=0).stop 2024-03-30T02:44:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.924+0000 7f6782000700 1 -- 172.21.15.67:0/3922650434 shutdown_connections 2024-03-30T02:44:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.924+0000 7f6782000700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67600a1310 0x7f67600a37d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.924+0000 7f6782000700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f677c11a240 0x7f677c1aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.924+0000 7f6782000700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f677c10ce90 0x7f677c1a5e00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.924+0000 7f6782000700 1 --2- 172.21.15.67:0/3922650434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f677c10d7b0 0x7f677c1a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.924+0000 7f6782000700 1 -- 172.21.15.67:0/3922650434 >> 172.21.15.67:0/3922650434 conn(0x7f677c06bc10 msgr2=0x7f677c118d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.924+0000 7f6782000700 1 -- 172.21.15.67:0/3922650434 shutdown_connections 2024-03-30T02:44:15.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:15.924+0000 7f6782000700 1 -- 172.21.15.67:0/3922650434 wait complete. 2024-03-30T02:44:16.001 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:44:16.021 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/badblocks.o 2024-03-30T02:44:16.153 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/swap_slots.o 2024-03-30T02:44:16.286 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/fsmap.o 2024-03-30T02:44:16.385 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/zstd_common_module.o 2024-03-30T02:44:16.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: pgmap v221: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.1 MiB/s wr, 374 op/s 2024-03-30T02:44:16.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:16.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:16.393 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:16.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:16.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:16.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:16.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:16 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: pgmap v221: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.1 MiB/s wr, 374 op/s 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:16 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: pgmap v221: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.1 MiB/s wr, 374 op/s 2024-03-30T02:44:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:16.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:16.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:16.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:16 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:16.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.829+0000 7fbc1c000700 1 -- 172.21.15.67:0/272256616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 msgr2=0x7fbc14107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:16.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.829+0000 7fbc1c000700 1 --2- 172.21.15.67:0/272256616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 0x7fbc14107db0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009990 tx=0x7fbc00009ca0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.831+0000 7fbc1a000700 1 --2- 172.21.15.67:0/272256616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1411a590 0x7fbc1411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:16.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.831+0000 7fbc1c000700 1 -- 172.21.15.67:0/272256616 shutdown_connections 2024-03-30T02:44:16.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.831+0000 7fbc1c000700 1 --2- 172.21.15.67:0/272256616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1411a590 0x7fbc1411c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.831+0000 7fbc1c000700 1 --2- 172.21.15.67:0/272256616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 0x7fbc14107db0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.831+0000 7fbc1c000700 1 --2- 172.21.15.67:0/272256616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc141082f0 0x7fbc14108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.831+0000 7fbc1c000700 1 -- 172.21.15.67:0/272256616 >> 172.21.15.67:0/272256616 conn(0x7fbc1406bdf0 msgr2=0x7fbc1406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:16.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.831+0000 7fbc1c000700 1 -- 172.21.15.67:0/272256616 shutdown_connections 2024-03-30T02:44:16.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.831+0000 7fbc1c000700 1 -- 172.21.15.67:0/272256616 wait complete. 2024-03-30T02:44:16.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.838+0000 7fbc1c000700 1 Processor -- start 2024-03-30T02:44:16.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.839+0000 7fbc1c000700 1 -- start start 2024-03-30T02:44:16.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.841+0000 7fbc1c000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc141082f0 0x7fbc141a6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.841+0000 7fbc1c000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 0x7fbc141a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.842+0000 7fbc1c000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1411a590 0x7fbc141aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.842+0000 7fbc1c000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc141aafa0 con 0x7fbc141082f0 2024-03-30T02:44:16.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.842+0000 7fbc1c000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc141ab110 con 0x7fbc1410f540 2024-03-30T02:44:16.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.842+0000 7fbc1c000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc141ab2d0 con 0x7fbc1411a590 2024-03-30T02:44:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.843+0000 7fbc18c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 0x7fbc141a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.843+0000 7fbc18c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 0x7fbc141a65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:57950/0 (socket says 172.21.15.67:57950) 2024-03-30T02:44:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.843+0000 7fbc18c00700 1 -- 172.21.15.67:0/3309699221 learned_addr learned my addr 172.21.15.67:0/3309699221 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.843+0000 7fbc19600700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc141082f0 0x7fbc141a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.843+0000 7fbc18c00700 1 -- 172.21.15.67:0/3309699221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1411a590 msgr2=0x7fbc141aa960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:44:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.843+0000 7fbc18c00700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1411a590 0x7fbc141aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.843+0000 7fbc18c00700 1 -- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc141082f0 msgr2=0x7fbc141a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.844+0000 7fbc18c00700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc141082f0 0x7fbc141a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.844+0000 7fbc18c00700 1 -- 172.21.15.67:0/3309699221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c009c40 con 0x7fbc1410f540 2024-03-30T02:44:16.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.844+0000 7fbc18c00700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 0x7fbc141a65c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c370 tx=0x7fbc0c00c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.845+0000 7fbc12000700 1 -- 172.21.15.67:0/3309699221 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c017070 con 0x7fbc1410f540 2024-03-30T02:44:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.845+0000 7fbc12000700 1 -- 172.21.15.67:0/3309699221 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc0c00f040 con 0x7fbc1410f540 2024-03-30T02:44:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.845+0000 7fbc12000700 1 -- 172.21.15.67:0/3309699221 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00e050 con 0x7fbc1410f540 2024-03-30T02:44:16.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.845+0000 7fbc1c000700 1 -- 172.21.15.67:0/3309699221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc08009520 con 0x7fbc1410f540 2024-03-30T02:44:16.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.846+0000 7fbc1c000700 1 -- 172.21.15.67:0/3309699221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc000095f0 con 0x7fbc1410f540 2024-03-30T02:44:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.864+0000 7fbc12000700 1 -- 172.21.15.67:0/3309699221 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fbc0c0090d0 con 0x7fbc1410f540 2024-03-30T02:44:16.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.868+0000 7fbc12000700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbbfc0a1310 0x7fbbfc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.868+0000 7fbc2fbc8b80 1 Processor -- start 2024-03-30T02:44:16.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.868+0000 7fbc12000700 1 -- 172.21.15.67:0/3309699221 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fbc0c0cc4b0 con 0x7fbc1410f540 2024-03-30T02:44:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.871+0000 7fbc2fbc8b80 1 -- start start 2024-03-30T02:44:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.871+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.871+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b34ef0 0x5588f7b3f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.873+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 0x5588f7b421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.873+0000 7fbc2fbc8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7a8d990 con 0x5588f7b34ef0 2024-03-30T02:44:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.873+0000 7fbc2fbc8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7a8b350 con 0x5588f7b3fdf0 2024-03-30T02:44:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.873+0000 7fbc2fbc8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7a86c40 con 0x5588f7b345d0 2024-03-30T02:44:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.873+0000 7fbc18c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b34ef0 0x5588f7b3f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.873+0000 7fbc18c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b34ef0 0x5588f7b3f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58964/0 (socket says 172.21.15.67:58964) 2024-03-30T02:44:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.873+0000 7fbc18c00700 1 -- 172.21.15.67:0/1130966435 learned_addr learned my addr 172.21.15.67:0/1130966435 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.873+0000 7fbc1a000700 1 --2- 172.21.15.67:0/1130966435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 0x5588f7b421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.873+0000 7fbc19600700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbbfc0a1310 0x7fbbfc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.874+0000 7fbc18c00700 1 -- 172.21.15.67:0/1130966435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 msgr2=0x5588f7b349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.874+0000 7fbc18c00700 1 --2- 172.21.15.67:0/1130966435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.874+0000 7fbc18c00700 1 -- 172.21.15.67:0/1130966435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 msgr2=0x5588f7b421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.874+0000 7fbc18c00700 1 --2- 172.21.15.67:0/1130966435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 0x5588f7b421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.874+0000 7fbc18c00700 1 -- 172.21.15.67:0/1130966435 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588f79083b0 con 0x5588f7b34ef0 2024-03-30T02:44:16.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.875+0000 7fbc18c00700 1 --2- 172.21.15.67:0/1130966435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b34ef0 0x5588f7b3f8b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c095c90 tx=0x7fbc0c098c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da39b02299e2ad0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:16.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.875+0000 7fbc19600700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbbfc0a1310 0x7fbbfc0a37d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006010 tx=0x7fbc00005040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:16.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.876+0000 7fbc06600700 1 -- 172.21.15.67:0/1130966435 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c09d050 con 0x5588f7b34ef0 2024-03-30T02:44:16.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.877+0000 7fbc06600700 1 -- 172.21.15.67:0/1130966435 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc0c0a16a0 con 0x5588f7b34ef0 2024-03-30T02:44:16.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.877+0000 7fbc06600700 1 -- 172.21.15.67:0/1130966435 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c098710 con 0x5588f7b34ef0 2024-03-30T02:44:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.879+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1130966435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b34ef0 msgr2=0x5588f7b3f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:16.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.879+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1130966435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b34ef0 0x5588f7b3f8b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c095c90 tx=0x7fbc0c098c00 comp rx=0 tx=0).stop 2024-03-30T02:44:16.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.892+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1130966435 shutdown_connections 2024-03-30T02:44:16.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.892+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1130966435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 0x5588f7b421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.892+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1130966435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.892+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1130966435 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b34ef0 0x5588f7b3f8b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.892+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1130966435 >> 172.21.15.67:0/1130966435 conn(0x5588f7a946a0 msgr2=0x5588f7b30a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:16.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.893+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1130966435 shutdown_connections 2024-03-30T02:44:16.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.893+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1130966435 wait complete. 2024-03-30T02:44:16.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.895+0000 7fbc2fbc8b80 1 -- start start 2024-03-30T02:44:16.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.896+0000 7fbc2fbc8b80 1 Processor -- start 2024-03-30T02:44:16.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.898+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b345d0 0x5588f7b3ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.899+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b34ef0 0x5588f7b3d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.899+0000 7fbc19600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b345d0 0x5588f7b3ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.899+0000 7fbc19600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b345d0 0x5588f7b3ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58970/0 (socket says 172.21.15.67:58970) 2024-03-30T02:44:16.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.902+0000 7fbc19600700 1 -- 172.21.15.67:0/2621445855 learned_addr learned my addr 172.21.15.67:0/2621445855 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:16.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.902+0000 7fbc18c00700 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b34ef0 0x5588f7b3d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.902+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 0x5588f7b3d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.902+0000 7fbc1a000700 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 0x5588f7b3d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.902+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7a22d80 con 0x5588f7b345d0 2024-03-30T02:44:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.902+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7a22a20 con 0x5588f7b3fdf0 2024-03-30T02:44:16.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.902+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f790a6a0 con 0x5588f7b34ef0 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.904+0000 7fbc1a000700 1 -- 172.21.15.67:0/2621445855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b34ef0 msgr2=0x5588f7b3d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.904+0000 7fbc2fbc8b80 10 client.0 _ll_register_callbacks cb 0x5588f7a2c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.905+0000 7fbc1a000700 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b34ef0 0x5588f7b3d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.905+0000 7fbc1a000700 1 -- 172.21.15.67:0/2621445855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b345d0 msgr2=0x5588f7b3ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.905+0000 7fbc1a000700 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b345d0 0x5588f7b3ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.905+0000 7fbc1a000700 1 -- 172.21.15.67:0/2621445855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc000095f0 con 0x5588f7b3fdf0 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.905+0000 7fbc19600700 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b345d0 0x5588f7b3ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.906+0000 7fbc1a000700 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 0x5588f7b3d740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbc08011670 tx=0x7fbc08011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40fe42bbe5a685f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.906+0000 7fbbfaa00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.906+0000 7fbbfaa00700 1 -- 172.21.15.67:0/2621445855 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08004500 con 0x5588f7b3fdf0 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.906+0000 7fbbfaa00700 1 -- 172.21.15.67:0/2621445855 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fbc08003680 con 0x5588f7b3fdf0 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.906+0000 7fbc18c00700 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b34ef0 0x5588f7b3d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:16.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.907+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588f7a47380 con 0x5588f7b3fdf0 2024-03-30T02:44:16.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.907+0000 7fbbfaa00700 1 -- 172.21.15.67:0/2621445855 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc08003840 con 0x5588f7b3fdf0 2024-03-30T02:44:16.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.908+0000 7fbc2fbc8b80 10 client.27092 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:16.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.908+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=39}) v3 -- 0x5588f78b4b60 con 0x5588f7b3fdf0 2024-03-30T02:44:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.911+0000 7fbbfaa00700 1 -- 172.21.15.67:0/2621445855 <== mon.2 v2:172.21.15.82:3300/0 4 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbc080039a0 con 0x5588f7b3fdf0 2024-03-30T02:44:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.911+0000 7fbc2fbc8b80 10 client.27092 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.911+0000 7fbc2fbc8b80 10 client.27092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.911+0000 7fbc2fbc8b80 20 client.27092 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.912+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5588f7b4f9a0 0x5588f7b51dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.912+0000 7fbc2fbc8b80 4 client.27092 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.912+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588f7a47380 con 0x5588f7b4f9a0 2024-03-30T02:44:16.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.912+0000 7fbc19600700 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5588f7b4f9a0 0x5588f7b51dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.914+0000 7fbbfaa00700 10 client.27092 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:16.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.914+0000 7fbc19600700 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5588f7b4f9a0 0x5588f7b51dc0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d444331641e207b server_cookie=5e155c482069ffc in_seq=0 out_seq=0 2024-03-30T02:44:16.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.915+0000 7fbbfaa00700 1 -- 172.21.15.67:0/2621445855 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5588f7a47380 con 0x5588f7b4f9a0 2024-03-30T02:44:16.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.915+0000 7fbbfaa00700 10 client.27092 handle_command_reply: tid=0 2024-03-30T02:44:16.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.915+0000 7fbc2fbc8b80 1 client.27092 shutdown 2024-03-30T02:44:16.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.917+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 msgr2=0x5588f7b3d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:16.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.917+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 0x5588f7b3d740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbc08011670 tx=0x7fbc08011980 comp rx=0 tx=0).stop 2024-03-30T02:44:16.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.917+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 shutdown_connections 2024-03-30T02:44:16.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.917+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5588f7b4f9a0 0x5588f7b51dc0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.917+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b345d0 0x5588f7b3ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.917+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b3fdf0 0x5588f7b3d740 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.917+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2621445855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b34ef0 0x5588f7b3d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.917+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 >> 172.21.15.67:0/2621445855 conn(0x5588f7a946a0 msgr2=0x5588f7b366c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.917+0000 7fbbfaa00700 0 client.27092 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.918+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 shutdown_connections 2024-03-30T02:44:16.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.918+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2621445855 wait complete. 2024-03-30T02:44:16.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.921+0000 7fbc2fbc8b80 20 client.27092 trim_cache size 0 max 16384 2024-03-30T02:44:16.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.965+0000 7fbc2fbc8b80 1 Processor -- start 2024-03-30T02:44:16.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.966+0000 7fbc2fbc8b80 1 -- start start 2024-03-30T02:44:16.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.967+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b4bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.968+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b34ef0 0x5588f7b46ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.968+0000 7fbc19600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b4bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.969+0000 7fbc19600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b4bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:33168/0 (socket says 172.21.15.67:33168) 2024-03-30T02:44:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.969+0000 7fbc19600700 1 -- 172.21.15.67:0/2001012446 learned_addr learned my addr 172.21.15.67:0/2001012446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.969+0000 7fbc18c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b34ef0 0x5588f7b46ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.969+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b295a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.970+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2001012446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7b4c660 con 0x5588f7b3fdf0 2024-03-30T02:44:16.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.970+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2001012446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7b47000 con 0x5588f7b34ef0 2024-03-30T02:44:16.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.970+0000 7fbc1a000700 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b295a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.970+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2001012446 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7b29ae0 con 0x5588f7b345d0 2024-03-30T02:44:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.971+0000 7fbc19600700 1 -- 172.21.15.67:0/2001012446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b34ef0 msgr2=0x5588f7b46ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.971+0000 7fbc19600700 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b34ef0 0x5588f7b46ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.971+0000 7fbc19600700 1 -- 172.21.15.67:0/2001012446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 msgr2=0x5588f7b295a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.971+0000 7fbc19600700 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b295a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.972+0000 7fbc19600700 1 -- 172.21.15.67:0/2001012446 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588f7a4cea0 con 0x5588f7b345d0 2024-03-30T02:44:16.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.972+0000 7fbc1a000700 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b295a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:16.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.972+0000 7fbc19600700 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b4bc10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5588f7b526a0 tx=0x7fbc00021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=902192f757eb816e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:16.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.973+0000 7fbc07000700 1 -- 172.21.15.67:0/2001012446 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0002c070 con 0x5588f7b345d0 2024-03-30T02:44:16.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.973+0000 7fbc07000700 1 -- 172.21.15.67:0/2001012446 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc0002d040 con 0x5588f7b345d0 2024-03-30T02:44:16.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.973+0000 7fbc07000700 1 -- 172.21.15.67:0/2001012446 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc00030730 con 0x5588f7b345d0 2024-03-30T02:44:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.975+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2001012446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 msgr2=0x5588f7b4bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:16.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.975+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b4bc10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5588f7b526a0 tx=0x7fbc00021000 comp rx=0 tx=0).stop 2024-03-30T02:44:16.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.975+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2001012446 shutdown_connections 2024-03-30T02:44:16.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.975+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b295a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.975+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b4bc10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.975+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/2001012446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b34ef0 0x5588f7b46ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.975+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2001012446 >> 172.21.15.67:0/2001012446 conn(0x5588f7a946a0 msgr2=0x5588f7b56720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:16.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.976+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2001012446 shutdown_connections 2024-03-30T02:44:16.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.976+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/2001012446 wait complete. 2024-03-30T02:44:16.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.978+0000 7fbc2fbc8b80 1 -- start start 2024-03-30T02:44:16.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.979+0000 7fbc2fbc8b80 1 Processor -- start 2024-03-30T02:44:16.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.982+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b46720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.982+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b34ef0 0x5588f7b44c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.983+0000 7fbc2fbc8b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b45150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.983+0000 7fbc2fbc8b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7b45690 con 0x5588f7b3fdf0 2024-03-30T02:44:16.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.983+0000 7fbc2fbc8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7b45800 con 0x5588f7b34ef0 2024-03-30T02:44:16.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.983+0000 7fbc2fbc8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588f7b459c0 con 0x5588f7b345d0 2024-03-30T02:44:16.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.983+0000 7fbc2fbc8b80 10 client.0 _ll_register_callbacks cb 0x7fbc08009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:16.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.984+0000 7fbc1a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b45150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:16.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.984+0000 7fbc1a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b45150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58990/0 (socket says 172.21.15.67:58990) 2024-03-30T02:44:16.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.984+0000 7fbc1a000700 1 -- 172.21.15.67:0/1523179148 learned_addr learned my addr 172.21.15.67:0/1523179148 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:16.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.985+0000 7fbc1a000700 1 -- 172.21.15.67:0/1523179148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 msgr2=0x5588f7b46720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:16.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.985+0000 7fbc1a000700 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b46720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.985+0000 7fbc1a000700 1 -- 172.21.15.67:0/1523179148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b34ef0 msgr2=0x5588f7b44c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:16.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.985+0000 7fbc1a000700 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b34ef0 0x5588f7b44c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:16.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.985+0000 7fbc1a000700 1 -- 172.21.15.67:0/1523179148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc00020040 con 0x5588f7b3fdf0 2024-03-30T02:44:16.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.986+0000 7fbc1a000700 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b45150 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000c00 tx=0x7fbc08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b541824d6aa01a04 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:16.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.987+0000 7fbbfaa00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:44:16.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.987+0000 7fbbfaa00700 1 -- 172.21.15.67:0/1523179148 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08003840 con 0x5588f7b3fdf0 2024-03-30T02:44:16.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.987+0000 7fbbfaa00700 1 -- 172.21.15.67:0/1523179148 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fbc08000e40 con 0x5588f7b3fdf0 2024-03-30T02:44:16.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.987+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1523179148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc0c0a16a0 con 0x5588f7b3fdf0 2024-03-30T02:44:16.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.988+0000 7fbbfaa00700 1 -- 172.21.15.67:0/1523179148 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc08003680 con 0x5588f7b3fdf0 2024-03-30T02:44:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.990+0000 7fbc2fbc8b80 10 client.16989 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:16.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.990+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1523179148 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5588f790ccd0 con 0x5588f7b3fdf0 2024-03-30T02:44:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.991+0000 7fbbfaa00700 1 -- 172.21.15.67:0/1523179148 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc0c0a16a0 con 0x5588f7b3fdf0 2024-03-30T02:44:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.992+0000 7fbbfaa00700 1 -- 172.21.15.67:0/1523179148 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbc08024970 con 0x5588f7b3fdf0 2024-03-30T02:44:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.993+0000 7fbc2fbc8b80 10 client.16989 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.993+0000 7fbc2fbc8b80 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.993+0000 7fbc2fbc8b80 20 client.16989 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.993+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5588f7ba4d20 0x5588f7ba7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.993+0000 7fbc2fbc8b80 4 client.16989 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:44:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.993+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1523179148 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5588f7a7f700 con 0x5588f7ba4d20 2024-03-30T02:44:16.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:16.994+0000 7fbc19600700 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5588f7ba4d20 0x5588f7ba7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.003+0000 7fbbfaa00700 10 client.16989 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.003+0000 7fbc19600700 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5588f7ba4d20 0x5588f7ba7140 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98b3b9aa1c2fb71 server_cookie=95e6806548577325 in_seq=0 out_seq=0 2024-03-30T02:44:17.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.004+0000 7fbbfaa00700 1 -- 172.21.15.67:0/1523179148 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+269 (crc 0 0 0) 0x5588f7a7f700 con 0x5588f7ba4d20 2024-03-30T02:44:17.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.004+0000 7fbbfaa00700 10 client.16989 handle_command_reply: tid=0 2024-03-30T02:44:17.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.004+0000 7fbc2fbc8b80 1 client.16989 shutdown 2024-03-30T02:44:17.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.005+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1523179148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 msgr2=0x5588f7b45150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:17.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.005+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b45150 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000c00 tx=0x7fbc08004750 comp rx=0 tx=0).stop 2024-03-30T02:44:17.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.006+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1523179148 shutdown_connections 2024-03-30T02:44:17.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.006+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5588f7ba4d20 0x5588f7ba7140 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:17.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.006+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5588f7b3fdf0 0x5588f7b45150 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:17.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.006+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5588f7b345d0 0x5588f7b46720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:17.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.006+0000 7fbc2fbc8b80 1 --2- 172.21.15.67:0/1523179148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5588f7b34ef0 0x5588f7b44c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:17.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.006+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1523179148 >> 172.21.15.67:0/1523179148 conn(0x5588f7a946a0 msgr2=0x5588f7b4bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:17.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.006+0000 7fbbfaa00700 0 client.16989 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:17.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.009+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1523179148 shutdown_connections 2024-03-30T02:44:17.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.012+0000 7fbc2fbc8b80 1 -- 172.21.15.67:0/1523179148 wait complete. 2024-03-30T02:44:17.013 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:44:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (15359 inodes in the stack)", 2024-03-30T02:44:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:44:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "2a7394b5-5ef0-4cb8-8666-7015ebc1d011": { 2024-03-30T02:44:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:44:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "2a7394b5-5ef0-4cb8-8666-7015ebc1d011", 2024-03-30T02:44:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:44:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:44:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:44:17.014 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:44:17.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.012+0000 7fbc2fbc8b80 20 client.16989 trim_cache size 0 max 16384 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 -- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbbfc0a1310 msgr2=0x7fbbfc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbbfc0a1310 0x7fbbfc0a37d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006010 tx=0x7fbc00005040 comp rx=0 tx=0).stop 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 -- 172.21.15.67:0/3309699221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 msgr2=0x7fbc141a65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 0x7fbc141a65c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c370 tx=0x7fbc0c00c730 comp rx=0 tx=0).stop 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 -- 172.21.15.67:0/3309699221 shutdown_connections 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fbbfc0a1310 0x7fbbfc0a37d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc141082f0 0x7fbc141a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc1411a590 0x7fbc141aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 --2- 172.21.15.67:0/3309699221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1410f540 0x7fbc141a65c0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c370 tx=0x7fbc0c00c730 comp rx=0 tx=0).stop 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.015+0000 7fbc1c000700 1 -- 172.21.15.67:0/3309699221 >> 172.21.15.67:0/3309699221 conn(0x7fbc1406bdf0 msgr2=0x7fbc14118580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:17.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.016+0000 7fbc1c000700 1 -- 172.21.15.67:0/3309699221 shutdown_connections 2024-03-30T02:44:17.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:17.016+0000 7fbc1c000700 1 -- 172.21.15.67:0/3309699221 wait complete. 2024-03-30T02:44:17.073 INFO:tasks.cephfs.filesystem:scrub status for tag:2a7394b5-5ef0-4cb8-8666-7015ebc1d011 - {'path': '/', 'tag': '2a7394b5-5ef0-4cb8-8666-7015ebc1d011', 'options': 'recursive,force'} 2024-03-30T02:44:17.354 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timeconv.o 2024-03-30T02:44:17.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:44:17.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.410 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.411 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.549 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-rq-qos.o 2024-03-30T02:44:17.552 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xz/xz_dec_syms.o 2024-03-30T02:44:17.562 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/debug.o 2024-03-30T02:44:17.568 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/dmapool.o 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:17.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:17 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:17.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:17 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:44:17.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:17.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:17 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:17.798 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/entropy_common.o 2024-03-30T02:44:17.840 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/opdef.o 2024-03-30T02:44:17.853 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/jbd2/built-in.a 2024-03-30T02:44:17.903 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/error_private.o 2024-03-30T02:44:18.139 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timecounter.o 2024-03-30T02:44:18.156 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/fsync.o 2024-03-30T02:44:18.618 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/alarmtimer.o 2024-03-30T02:44:18.636 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/fse_decompress.o 2024-03-30T02:44:18.647 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/core.o 2024-03-30T02:44:18.663 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xz/xz_dec_stream.o 2024-03-30T02:44:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: pgmap v222: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 2.1 MiB/s wr, 295 op/s 2024-03-30T02:44:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:44:18.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:18.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:18 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.669 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/disk-events.o 2024-03-30T02:44:18.677 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/hugetlb.o 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: pgmap v222: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 2.1 MiB/s wr, 295 op/s 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:44:18.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:18.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:18 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: pgmap v222: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 2.1 MiB/s wr, 295 op/s 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:18.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:18 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.083 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/notif.o 2024-03-30T02:44:19.441 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/hash.o 2024-03-30T02:44:19.496 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xz/xz_dec_lzma2.o 2024-03-30T02:44:19.610 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/zstd/common/zstd_common.o 2024-03-30T02:44:19.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:19.696 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:19 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:19.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:19 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.803 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/zstd/built-in.a 2024-03-30T02:44:19.907 INFO:tasks.workunit.client.0.smithi067.stdout: CC io_uring/io-wq.o 2024-03-30T02:44:19.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:19.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:19 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:19.957 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-ia-ranges.o 2024-03-30T02:44:19.984 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/syscalls.o 2024-03-30T02:44:20.220 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/posix-timers.o 2024-03-30T02:44:20.359 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xz/xz_dec_bcj.o 2024-03-30T02:44:20.503 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/ialloc.o 2024-03-30T02:44:20.623 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/early-lookup.o 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: pgmap v223: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.9 MiB/s wr, 295 op/s 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.690 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.692 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.693 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.694 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:20.695 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:20 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: pgmap v223: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.9 MiB/s wr, 295 op/s 2024-03-30T02:44:20.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:20.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:20 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: pgmap v223: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.9 MiB/s wr, 295 op/s 2024-03-30T02:44:20.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:20.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:20 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:20.996 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/xz/built-in.a 2024-03-30T02:44:20.999 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/cache.o 2024-03-30T02:44:21.090 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-lshift.o 2024-03-30T02:44:21.249 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/pi.o 2024-03-30T02:44:21.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/bsg.o 2024-03-30T02:44:21.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:21.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:21 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.758 INFO:tasks.workunit.client.0.smithi067.stdout: AR io_uring/built-in.a 2024-03-30T02:44:21.798 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/posix-cpu-timers.o 2024-03-30T02:44:21.899 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/indirect.o 2024-03-30T02:44:21.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:21.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:21 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:21.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:21 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.169 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/dir.o 2024-03-30T02:44:22.200 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-mul1.o 2024-03-30T02:44:22.214 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/requeue.o 2024-03-30T02:44:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: pgmap v224: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 665 op/s 2024-03-30T02:44:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:22.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:22.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:22.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:22.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:22.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:22.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:22 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: pgmap v224: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 665 op/s 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:22.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:22.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:22 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: pgmap v224: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 665 op/s 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:22.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:22.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:22.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:22.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:22.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:22 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.044 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/futex/waitwake.o 2024-03-30T02:44:23.107 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/hugetlb_vmemmap.o 2024-03-30T02:44:23.133 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/namei.o 2024-03-30T02:44:23.142 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-cgroup.o 2024-03-30T02:44:23.218 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-mul2.o 2024-03-30T02:44:23.256 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/posix-clock.o 2024-03-30T02:44:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:23 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.824 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/futex/built-in.a 2024-03-30T02:44:23.902 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/client.o 2024-03-30T02:44:23.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:23.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:23 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:23 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:23.969 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/fatent.o 2024-03-30T02:44:24.106 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/inline.o 2024-03-30T02:44:24.116 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-mul3.o 2024-03-30T02:44:24.136 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/inode.o 2024-03-30T02:44:24.276 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/itimer.o 2024-03-30T02:44:24.296 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mempolicy.o 2024-03-30T02:44:24.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: pgmap v225: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 689 KiB/s wr, 901 op/s 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:24.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:24 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: pgmap v225: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 689 KiB/s wr, 901 op/s 2024-03-30T02:44:24.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:24.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:24 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: pgmap v225: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 689 KiB/s wr, 901 op/s 2024-03-30T02:44:24.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:24.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:24 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:24.969 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-ioprio.o 2024-03-30T02:44:24.981 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-rshift.o 2024-03-30T02:44:25.327 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/file.o 2024-03-30T02:44:25.458 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/clockevents.o 2024-03-30T02:44:25.738 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/dir.o 2024-03-30T02:44:25.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:25 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.805 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-sub1.o 2024-03-30T02:44:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:25.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:25 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.921 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/dir.o 2024-03-30T02:44:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:25 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:25.968 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/inode.o 2024-03-30T02:44:25.996 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-iolatency.o 2024-03-30T02:44:26.383 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-common.o 2024-03-30T02:44:26.410 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/sparse.o 2024-03-30T02:44:26.572 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/inode.o 2024-03-30T02:44:26.609 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/generic_mpih-add1.o 2024-03-30T02:44:26.739 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/util.o 2024-03-30T02:44:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: pgmap v226: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 582 KiB/s wr, 1.13k op/s 2024-03-30T02:44:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:26.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:26 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: pgmap v226: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 582 KiB/s wr, 1.13k op/s 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:26.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:26 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: pgmap v226: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 582 KiB/s wr, 1.13k op/s 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:26.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:26.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:26 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.498 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-broadcast.o 2024-03-30T02:44:27.512 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/ec.o 2024-03-30T02:44:27.567 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/sparse-vmemmap.o 2024-03-30T02:44:27.599 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/rock.o 2024-03-30T02:44:27.654 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-iocost.o 2024-03-30T02:44:27.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: pgmap v227: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1023 KiB/s wr, 2.20k op/s 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:27.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:27 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: pgmap v227: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1023 KiB/s wr, 2.20k op/s 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:27.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:27 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: pgmap v227: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1023 KiB/s wr, 2.20k op/s 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:27.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:27.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:27 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.102 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/misc.o 2024-03-30T02:44:28.399 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-broadcast-hrtimer.o 2024-03-30T02:44:28.741 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/export.o 2024-03-30T02:44:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:28.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:28 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.755 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/mmu_notifier.o 2024-03-30T02:44:28.856 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpicoder.o 2024-03-30T02:44:28.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:28.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:28 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:28.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:28.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:28 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.024 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/file.o 2024-03-30T02:44:29.045 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-oneshot.o 2024-03-30T02:44:29.144 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/nfs.o 2024-03-30T02:44:29.590 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/joliet.o 2024-03-30T02:44:29.636 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/tick-sched.o 2024-03-30T02:44:29.728 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/slub.o 2024-03-30T02:44:29.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: pgmap v228: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.0 MiB/s wr, 2.38k op/s 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:29.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:29 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.775 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-add.o 2024-03-30T02:44:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: pgmap v228: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.0 MiB/s wr, 2.38k op/s 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:29.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:29 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.937 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/ioctl.o 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: pgmap v228: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.0 MiB/s wr, 2.38k op/s 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:29.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:29 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:29.972 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/namei_vfat.o 2024-03-30T02:44:29.992 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/mq-deadline.o 2024-03-30T02:44:30.298 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/isofs/compress.o 2024-03-30T02:44:30.491 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-bit.o 2024-03-30T02:44:30.624 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/vsyscall.o 2024-03-30T02:44:30.785 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/getroot.o 2024-03-30T02:44:30.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:30.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:30.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:30.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:30 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:30.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:30.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:30 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:30.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:31.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:31.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:31.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:31.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:31.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:31.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:31.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:31.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:31.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:31.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:31.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:31.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:30 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:31.080 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fat/namei_msdos.o 2024-03-30T02:44:31.144 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/timekeeping_debug.o 2024-03-30T02:44:31.190 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/isofs/built-in.a 2024-03-30T02:44:31.210 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/kyber-iosched.o 2024-03-30T02:44:31.250 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-cmp.o 2024-03-30T02:44:31.271 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/exportfs/expfs.o 2024-03-30T02:44:31.618 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/mballoc.o 2024-03-30T02:44:31.847 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/inode.o 2024-03-30T02:44:31.904 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/time/namespace.o 2024-03-30T02:44:31.913 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-sub-ui.o 2024-03-30T02:44:31.982 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/fat/built-in.a 2024-03-30T02:44:31.985 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/exportfs/built-in.a 2024-03-30T02:44:32.038 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/migrate.o 2024-03-30T02:44:32.058 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/fonts/fonts.o 2024-03-30T02:44:32.240 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/migrate.o 2024-03-30T02:44:32.525 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/fonts/font_8x16.o 2024-03-30T02:44:32.582 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-pci.o 2024-03-30T02:44:32.597 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-div.o 2024-03-30T02:44:32.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:32 smithi067 ceph-mon[37808]: pgmap v229: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 995 KiB/s wr, 3.68k op/s 2024-03-30T02:44:32.667 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/time/built-in.a 2024-03-30T02:44:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:32 smithi170 ceph-mon[78798]: pgmap v229: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 995 KiB/s wr, 3.68k op/s 2024-03-30T02:44:32.742 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/cgroup.o 2024-03-30T02:44:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:32 smithi082 ceph-mon[79176]: pgmap v229: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 995 KiB/s wr, 3.68k op/s 2024-03-30T02:44:32.988 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/fonts/built-in.a 2024-03-30T02:44:33.030 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-inv.o 2024-03-30T02:44:33.170 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/super.o 2024-03-30T02:44:33.398 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-virtio.o 2024-03-30T02:44:33.412 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/clntlock.o 2024-03-30T02:44:33.631 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/memory-tiers.o 2024-03-30T02:44:33.732 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-mod.o 2024-03-30T02:44:33.971 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/io.o 2024-03-30T02:44:34.146 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-mq-debugfs.o 2024-03-30T02:44:34.427 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-mul.o 2024-03-30T02:44:34.546 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/page_counter.o 2024-03-30T02:44:34.571 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/blk-pm.o 2024-03-30T02:44:34.575 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/clntproc.o 2024-03-30T02:44:34.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:34 smithi067 ceph-mon[37808]: pgmap v230: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.0 MiB/s wr, 3.75k op/s 2024-03-30T02:44:34.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:34 smithi170 ceph-mon[78798]: pgmap v230: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.0 MiB/s wr, 3.75k op/s 2024-03-30T02:44:34.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:34 smithi082 ceph-mon[79176]: pgmap v230: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.0 MiB/s wr, 3.75k op/s 2024-03-30T02:44:34.943 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/hugetlb_cgroup.o 2024-03-30T02:44:35.030 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/mmp.o 2024-03-30T02:44:35.042 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/direct.o 2024-03-30T02:44:35.112 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpih-cmp.o 2024-03-30T02:44:35.152 INFO:tasks.workunit.client.0.smithi067.stdout: CC block/holder.o 2024-03-30T02:44:35.358 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_base.o 2024-03-30T02:44:35.771 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpih-div.o 2024-03-30T02:44:35.864 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/rstat.o 2024-03-30T02:44:35.912 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/move_extent.o 2024-03-30T02:44:35.914 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_cp437.o 2024-03-30T02:44:35.915 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/clntxdr.o 2024-03-30T02:44:36.029 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/early_ioremap.o 2024-03-30T02:44:36.039 INFO:tasks.workunit.client.0.smithi067.stdout: AR block/built-in.a 2024-03-30T02:44:36.145 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/secretmem.o 2024-03-30T02:44:36.377 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_ascii.o 2024-03-30T02:44:36.609 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpih-mul.o 2024-03-30T02:44:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:36 smithi067 ceph-mon[37808]: pgmap v231: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 752 KiB/s wr, 3.48k op/s 2024-03-30T02:44:36.670 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/namespace.o 2024-03-30T02:44:36.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:36 smithi170 ceph-mon[78798]: pgmap v231: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 752 KiB/s wr, 3.48k op/s 2024-03-30T02:44:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:36 smithi082 ceph-mon[79176]: pgmap v231: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 752 KiB/s wr, 3.48k op/s 2024-03-30T02:44:36.807 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/pagelist.o 2024-03-30T02:44:36.838 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/memfd.o 2024-03-30T02:44:36.938 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_iso8859-1.o 2024-03-30T02:44:37.147 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/namei.o 2024-03-30T02:44:37.304 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/host.o 2024-03-30T02:44:37.317 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nls/nls_utf8.o 2024-03-30T02:44:37.605 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpi-pow.o 2024-03-30T02:44:37.714 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/unicode/built-in.a 2024-03-30T02:44:37.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/cgroup-v1.o 2024-03-30T02:44:37.758 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_clock.o 2024-03-30T02:44:37.890 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/nls/built-in.a 2024-03-30T02:44:38.144 INFO:tasks.workunit.client.0.smithi067.stdout: CC mm/ptdump.o 2024-03-30T02:44:38.239 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/ring_buffer.o 2024-03-30T02:44:38.352 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/bpf/core.o 2024-03-30T02:44:38.430 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/mpi/mpiutil.o 2024-03-30T02:44:38.601 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svc.o 2024-03-30T02:44:38.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:38 smithi067 ceph-mon[37808]: pgmap v232: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 778 KiB/s wr, 3.17k op/s 2024-03-30T02:44:38.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:38 smithi170 ceph-mon[78798]: pgmap v232: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 778 KiB/s wr, 3.17k op/s 2024-03-30T02:44:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:38 smithi082 ceph-mon[79176]: pgmap v232: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 778 KiB/s wr, 3.17k op/s 2024-03-30T02:44:38.960 INFO:tasks.workunit.client.0.smithi067.stdout: AR mm/built-in.a 2024-03-30T02:44:39.020 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/freezer.o 2024-03-30T02:44:39.053 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/argv_split.o 2024-03-30T02:44:39.124 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/read.o 2024-03-30T02:44:39.259 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/mpi/built-in.a 2024-03-30T02:44:39.319 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/symlink.o 2024-03-30T02:44:39.475 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bug.o 2024-03-30T02:44:39.838 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/page-io.o 2024-03-30T02:44:39.892 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/legacy_freezer.o 2024-03-30T02:44:39.920 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svclock.o 2024-03-30T02:44:40.237 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/buildid.o 2024-03-30T02:44:40.263 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace.o 2024-03-30T02:44:40.432 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/readpage.o 2024-03-30T02:44:40.523 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/bpf/built-in.a 2024-03-30T02:44:40.605 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/unlink.o 2024-03-30T02:44:40.639 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/pids.o 2024-03-30T02:44:40.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:40 smithi067 ceph-mon[37808]: pgmap v233: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 430 KiB/s wr, 2.06k op/s 2024-03-30T02:44:40.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:40 smithi170 ceph-mon[78798]: pgmap v233: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 430 KiB/s wr, 2.06k op/s 2024-03-30T02:44:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:40 smithi082 ceph-mon[79176]: pgmap v233: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 430 KiB/s wr, 2.06k op/s 2024-03-30T02:44:40.784 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/resize.o 2024-03-30T02:44:40.864 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/init.o 2024-03-30T02:44:40.944 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/clz_tab.o 2024-03-30T02:44:41.014 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/cmdline.o 2024-03-30T02:44:41.162 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svcshare.o 2024-03-30T02:44:41.200 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/cpumask.o 2024-03-30T02:44:41.304 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/rdma.o 2024-03-30T02:44:41.386 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/ctype.o 2024-03-30T02:44:41.462 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/inode.o 2024-03-30T02:44:41.480 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svcproc.o 2024-03-30T02:44:41.859 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/dec_and_lock.o 2024-03-30T02:44:42.047 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/cpuset.o 2024-03-30T02:44:42.137 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress.o 2024-03-30T02:44:42.151 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/write.o 2024-03-30T02:44:42.208 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/root.o 2024-03-30T02:44:42.235 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_super.o 2024-03-30T02:44:42.256 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_bunzip2.o 2024-03-30T02:44:42.288 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/super.o 2024-03-30T02:44:42.616 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svcsubs.o 2024-03-30T02:44:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:42 smithi170 ceph-mon[78798]: pgmap v234: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 KiB/s wr, 1.83k op/s 2024-03-30T02:44:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:42 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:42 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:42 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:42 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:42.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:42 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:44:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:42 smithi082 ceph-mon[79176]: pgmap v234: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 KiB/s wr, 1.83k op/s 2024-03-30T02:44:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:42 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:42 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:42 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:42 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:42 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:44:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:42 smithi067 ceph-mon[37808]: pgmap v234: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 909 KiB/s wr, 1.83k op/s 2024-03-30T02:44:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:42 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:42 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:42 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:42 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:42 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:44:42.998 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_inode.o 2024-03-30T02:44:43.000 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/symlink.o 2024-03-30T02:44:43.024 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_inflate.o 2024-03-30T02:44:43.236 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_output.o 2024-03-30T02:44:43.577 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/waitq.o 2024-03-30T02:44:43.654 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unlz4.o 2024-03-30T02:44:43.760 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/mon.o 2024-03-30T02:44:43.851 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/misc.o 2024-03-30T02:44:44.039 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_inode_dotl.o 2024-03-30T02:44:44.262 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/expire.o 2024-03-30T02:44:44.265 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unlzma.o 2024-03-30T02:44:44.343 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/namespace.o 2024-03-30T02:44:44.486 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cgroup/debug.o 2024-03-30T02:44:44.502 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_seq.o 2024-03-30T02:44:44.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:44 smithi170 ceph-mon[78798]: pgmap v235: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.2 MiB/s wr, 528 op/s 2024-03-30T02:44:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:44 smithi082 ceph-mon[79176]: pgmap v235: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.2 MiB/s wr, 528 op/s 2024-03-30T02:44:44.906 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/trace.o 2024-03-30T02:44:44.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:44 smithi067 ceph-mon[37808]: pgmap v235: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.2 MiB/s wr, 528 op/s 2024-03-30T02:44:44.967 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/autofs/dev-ioctl.o 2024-03-30T02:44:44.975 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_addr.o 2024-03-30T02:44:45.059 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unlzo.o 2024-03-30T02:44:45.153 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_stat.o 2024-03-30T02:44:45.242 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/cgroup/built-in.a 2024-03-30T02:44:45.342 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_file.o 2024-03-30T02:44:45.476 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/mount_clnt.o 2024-03-30T02:44:45.696 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unxz.o 2024-03-30T02:44:45.723 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/autofs/built-in.a 2024-03-30T02:44:45.775 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_dir.o 2024-03-30T02:44:45.781 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/xdr.o 2024-03-30T02:44:46.027 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_printk.o 2024-03-30T02:44:46.034 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/symlink.o 2024-03-30T02:44:46.182 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/core.o 2024-03-30T02:44:46.304 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/decompress_unzstd.o 2024-03-30T02:44:46.551 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/vfs_dentry.o 2024-03-30T02:44:46.589 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfstrace.o 2024-03-30T02:44:46.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:46 smithi170 ceph-mon[78798]: pgmap v236: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-03-30T02:44:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:46 smithi082 ceph-mon[79176]: pgmap v236: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-03-30T02:44:46.857 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/sysfs.o 2024-03-30T02:44:46.875 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/pid_list.o 2024-03-30T02:44:46.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:46 smithi067 ceph-mon[37808]: pgmap v236: 129 pgs: 129 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-03-30T02:44:46.940 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/dump_stack.o 2024-03-30T02:44:46.972 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/clnt4xdr.o 2024-03-30T02:44:47.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:44:47.321 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/v9fs.o 2024-03-30T02:44:47.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.618+0000 7fa813e00700 1 -- 172.21.15.67:0/2104717519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c108770 msgr2=0x7fa80c1145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:47.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.618+0000 7fa813e00700 1 --2- 172.21.15.67:0/2104717519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c108770 0x7fa80c1145c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa8040074e0 tx=0x7fa80400af90 comp rx=0 tx=0).stop 2024-03-30T02:44:47.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.621+0000 7fa813e00700 1 -- 172.21.15.67:0/2104717519 shutdown_connections 2024-03-30T02:44:47.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.621+0000 7fa813e00700 1 --2- 172.21.15.67:0/2104717519 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c108770 0x7fa80c1145c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.621+0000 7fa813e00700 1 --2- 172.21.15.67:0/2104717519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa80c10d770 0x7fa80c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.621+0000 7fa813e00700 1 --2- 172.21.15.67:0/2104717519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa80c107db0 0x7fa80c108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.621+0000 7fa813e00700 1 -- 172.21.15.67:0/2104717519 >> 172.21.15.67:0/2104717519 conn(0x7fa80c06bdf0 msgr2=0x7fa80c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:47.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.621+0000 7fa813e00700 1 -- 172.21.15.67:0/2104717519 shutdown_connections 2024-03-30T02:44:47.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.621+0000 7fa813e00700 1 -- 172.21.15.67:0/2104717519 wait complete. 2024-03-30T02:44:47.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.624+0000 7fa813e00700 1 Processor -- start 2024-03-30T02:44:47.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.624+0000 7fa813e00700 1 -- start start 2024-03-30T02:44:47.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa813e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa80c107db0 0x7fa80c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa813e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa80c108770 0x7fa80c1a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa813e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c10d770 0x7fa80c1a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa813e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c1a6c70 con 0x7fa80c10d770 2024-03-30T02:44:47.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa813e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c1a6de0 con 0x7fa80c107db0 2024-03-30T02:44:47.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa813e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c1a6fa0 con 0x7fa80c108770 2024-03-30T02:44:47.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa811400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa80c107db0 0x7fa80c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa811400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa80c107db0 0x7fa80c1a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:54732/0 (socket says 172.21.15.67:54732) 2024-03-30T02:44:47.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa811400700 1 -- 172.21.15.67:0/1111264261 learned_addr learned my addr 172.21.15.67:0/1111264261 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:47.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa810a00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa80c108770 0x7fa80c1a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.625+0000 7fa811e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c10d770 0x7fa80c1a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.627+0000 7fa811e00700 1 -- 172.21.15.67:0/1111264261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa80c108770 msgr2=0x7fa80c1a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:47.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.627+0000 7fa811e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa80c108770 0x7fa80c1a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.627+0000 7fa811e00700 1 -- 172.21.15.67:0/1111264261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa80c107db0 msgr2=0x7fa80c1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:47.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.627+0000 7fa811e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa80c107db0 0x7fa80c1a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.627+0000 7fa811e00700 1 -- 172.21.15.67:0/1111264261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8040071c0 con 0x7fa80c10d770 2024-03-30T02:44:47.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.628+0000 7fa811e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c10d770 0x7fa80c1a6670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa804006010 tx=0x7fa804003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:47.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.629+0000 7fa80a000700 1 -- 172.21.15.67:0/1111264261 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80400f070 con 0x7fa80c10d770 2024-03-30T02:44:47.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.629+0000 7fa80a000700 1 -- 172.21.15.67:0/1111264261 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa804004210 con 0x7fa80c10d770 2024-03-30T02:44:47.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.629+0000 7fa80a000700 1 -- 172.21.15.67:0/1111264261 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804004cd0 con 0x7fa80c10d770 2024-03-30T02:44:47.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.631+0000 7fa813e00700 1 -- 172.21.15.67:0/1111264261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c1a7220 con 0x7fa80c10d770 2024-03-30T02:44:47.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.631+0000 7fa813e00700 1 -- 172.21.15.67:0/1111264261 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c10e6e0 con 0x7fa80c10d770 2024-03-30T02:44:47.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.637+0000 7fa80a000700 1 -- 172.21.15.67:0/1111264261 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fa804005050 con 0x7fa80c10d770 2024-03-30T02:44:47.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.639+0000 7fa80a000700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7f40a13e0 0x7fa7f40a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.640+0000 7fa811400700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7f40a13e0 0x7fa7f40a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.640+0000 7fa827983b80 1 Processor -- start 2024-03-30T02:44:47.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.641+0000 7fa80a000700 1 -- 172.21.15.67:0/1111264261 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fa8040ccf00 con 0x7fa80c10d770 2024-03-30T02:44:47.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.641+0000 7fa827983b80 1 -- start start 2024-03-30T02:44:47.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.641+0000 7fa811400700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7f40a13e0 0x7fa7f40a38a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa800013a50 tx=0x7fa800015040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.642+0000 7fa827983b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.642+0000 7fa827983b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 0x56365ac438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.642+0000 7fa827983b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.643+0000 7fa811e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.643+0000 7fa811400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.643+0000 7fa811e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36270/0 (socket says 172.21.15.67:36270) 2024-03-30T02:44:47.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.643+0000 7fa811e00700 1 -- 172.21.15.67:0/2544553606 learned_addr learned my addr 172.21.15.67:0/2544553606 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:47.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.643+0000 7fa827983b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365ab91990 con 0x56365ac385d0 2024-03-30T02:44:47.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.643+0000 7fa810a00700 1 --2- 172.21.15.67:0/2544553606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 0x56365ac438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.643+0000 7fa827983b80 1 -- 172.21.15.67:0/2544553606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365ab8f350 con 0x56365ac38ef0 2024-03-30T02:44:47.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.643+0000 7fa827983b80 1 -- 172.21.15.67:0/2544553606 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365ab8ac40 con 0x56365ac43df0 2024-03-30T02:44:47.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.644+0000 7fa811e00700 1 -- 172.21.15.67:0/2544553606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 msgr2=0x56365ac438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:47.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.644+0000 7fa811e00700 1 --2- 172.21.15.67:0/2544553606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 0x56365ac438b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.644+0000 7fa811e00700 1 -- 172.21.15.67:0/2544553606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 msgr2=0x56365ac389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:47.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.644+0000 7fa811e00700 1 --2- 172.21.15.67:0/2544553606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.644+0000 7fa811e00700 1 -- 172.21.15.67:0/2544553606 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56365aa0c3b0 con 0x56365ac43df0 2024-03-30T02:44:47.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.644+0000 7fa811400700 1 --2- 172.21.15.67:0/2544553606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:47.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.645+0000 7fa811e00700 1 --2- 172.21.15.67:0/2544553606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac461e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa804096770 tx=0x7fa8040998c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13450852e5373669 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.645+0000 7fa7fe600700 1 -- 172.21.15.67:0/2544553606 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804099d00 con 0x56365ac43df0 2024-03-30T02:44:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.645+0000 7fa7fe600700 1 -- 172.21.15.67:0/2544553606 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa80409db30 con 0x56365ac43df0 2024-03-30T02:44:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.645+0000 7fa7fe600700 1 -- 172.21.15.67:0/2544553606 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80400c560 con 0x56365ac43df0 2024-03-30T02:44:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.646+0000 7fa827983b80 1 -- 172.21.15.67:0/2544553606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 msgr2=0x56365ac461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:47.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.646+0000 7fa827983b80 1 --2- 172.21.15.67:0/2544553606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac461e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa804096770 tx=0x7fa8040998c0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.646+0000 7fa827983b80 1 -- 172.21.15.67:0/2544553606 shutdown_connections 2024-03-30T02:44:47.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.646+0000 7fa827983b80 1 --2- 172.21.15.67:0/2544553606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.646+0000 7fa827983b80 1 --2- 172.21.15.67:0/2544553606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac461e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.646+0000 7fa827983b80 1 --2- 172.21.15.67:0/2544553606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 0x56365ac438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.646+0000 7fa827983b80 1 -- 172.21.15.67:0/2544553606 >> 172.21.15.67:0/2544553606 conn(0x56365ab986a0 msgr2=0x56365ac34a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:47.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.646+0000 7fa827983b80 1 -- 172.21.15.67:0/2544553606 shutdown_connections 2024-03-30T02:44:47.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.646+0000 7fa827983b80 1 -- 172.21.15.67:0/2544553606 wait complete. 2024-03-30T02:44:47.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.647+0000 7fa827983b80 1 -- start start 2024-03-30T02:44:47.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.648+0000 7fa827983b80 1 Processor -- start 2024-03-30T02:44:47.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.648+0000 7fa827983b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.649+0000 7fa827983b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 0x56365ac40b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.649+0000 7fa827983b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac41250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.649+0000 7fa827983b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365ab26d80 con 0x56365ac385d0 2024-03-30T02:44:47.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.649+0000 7fa827983b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365ab26a20 con 0x56365ac38ef0 2024-03-30T02:44:47.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.649+0000 7fa827983b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365aa0e6a0 con 0x56365ac43df0 2024-03-30T02:44:47.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.649+0000 7fa827983b80 10 client.0 _ll_register_callbacks cb 0x56365ab30470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:47.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.650+0000 7fa810a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 0x56365ac40b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.650+0000 7fa810a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 0x56365ac40b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:54748/0 (socket says 172.21.15.67:54748) 2024-03-30T02:44:47.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.651+0000 7fa810a00700 1 -- 172.21.15.67:0/1558405538 learned_addr learned my addr 172.21.15.67:0/1558405538 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:47.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.651+0000 7fa811e00700 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac41250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.651+0000 7fa811400700 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.651+0000 7fa811400700 1 -- 172.21.15.67:0/1558405538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 msgr2=0x56365ac41250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.651+0000 7fa811400700 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac41250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.651+0000 7fa811400700 1 -- 172.21.15.67:0/1558405538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 msgr2=0x56365ac40b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.651+0000 7fa811400700 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 0x56365ac40b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.651+0000 7fa811400700 1 -- 172.21.15.67:0/1558405538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa800007860 con 0x56365ac385d0 2024-03-30T02:44:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.652+0000 7fa811400700 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac43890 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa800009c40 tx=0x7fa800004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd5e55b6834f0ea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.653+0000 7fa7f2a00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:44:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.653+0000 7fa7f2a00700 1 -- 172.21.15.67:0/1558405538 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800018040 con 0x56365ac385d0 2024-03-30T02:44:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.653+0000 7fa7f2a00700 1 -- 172.21.15.67:0/1558405538 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fa80002a850 con 0x56365ac385d0 2024-03-30T02:44:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.654+0000 7fa827983b80 1 -- 172.21.15.67:0/1558405538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56365ab4b380 con 0x56365ac385d0 2024-03-30T02:44:47.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.654+0000 7fa7f2a00700 1 -- 172.21.15.67:0/1558405538 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa800018040 con 0x56365ac385d0 2024-03-30T02:44:47.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.656+0000 7fa827983b80 10 client.17001 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:47.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.656+0000 7fa827983b80 1 -- 172.21.15.67:0/1558405538 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fa80c10e6e0 con 0x56365ac385d0 2024-03-30T02:44:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.656+0000 7fa7f2a00700 1 -- 172.21.15.67:0/1558405538 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa800024040 con 0x56365ac385d0 2024-03-30T02:44:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.659+0000 7fa7f2a00700 1 -- 172.21.15.67:0/1558405538 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa800033cd0 con 0x56365ac385d0 2024-03-30T02:44:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.659+0000 7fa827983b80 10 client.17001 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.659+0000 7fa827983b80 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.659+0000 7fa827983b80 20 client.17001 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.659+0000 7fa827983b80 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56365ac53840 0x56365ac55ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.659+0000 7fa827983b80 4 client.17001 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:47.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.659+0000 7fa827983b80 1 -- 172.21.15.67:0/1558405538 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56365ab4b380 con 0x56365ac53840 2024-03-30T02:44:47.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.659+0000 7fa810a00700 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56365ac53840 0x56365ac55ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.662+0000 7fa810a00700 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56365ac53840 0x56365ac55ca0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff07a424a138396e server_cookie=4afc5f45d1b56e1a in_seq=0 out_seq=0 2024-03-30T02:44:47.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.662+0000 7fa7f2a00700 10 client.17001 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.663+0000 7fa7f2a00700 1 -- 172.21.15.67:0/1558405538 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56365ab4b380 con 0x56365ac53840 2024-03-30T02:44:47.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.663+0000 7fa7f2a00700 10 client.17001 handle_command_reply: tid=0 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.663+0000 7fa827983b80 1 client.17001 shutdown 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.663+0000 7fa827983b80 1 -- 172.21.15.67:0/1558405538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 msgr2=0x56365ac43890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.663+0000 7fa827983b80 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac43890 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa800009c40 tx=0x7fa800004b60 comp rx=0 tx=0).stop 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.664+0000 7fa827983b80 1 -- 172.21.15.67:0/1558405538 shutdown_connections 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.664+0000 7fa827983b80 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56365ac53840 0x56365ac55ca0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.664+0000 7fa827983b80 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac385d0 0x56365ac43890 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.664+0000 7fa827983b80 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac43df0 0x56365ac41250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.664+0000 7fa827983b80 1 --2- 172.21.15.67:0/1558405538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac38ef0 0x56365ac40b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.664+0000 7fa827983b80 1 -- 172.21.15.67:0/1558405538 >> 172.21.15.67:0/1558405538 conn(0x56365ab986a0 msgr2=0x56365ac3a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.664+0000 7fa7f2a00700 0 client.17001 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.665+0000 7fa827983b80 1 -- 172.21.15.67:0/1558405538 shutdown_connections 2024-03-30T02:44:47.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.668+0000 7fa827983b80 1 -- 172.21.15.67:0/1558405538 wait complete. 2024-03-30T02:44:47.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.672+0000 7fa827983b80 20 client.17001 trim_cache size 0 max 16384 2024-03-30T02:44:47.711 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/earlycpio.o 2024-03-30T02:44:47.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.711+0000 7fa827983b80 1 Processor -- start 2024-03-30T02:44:47.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.712+0000 7fa827983b80 1 -- start start 2024-03-30T02:44:47.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.713+0000 7fa827983b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac3b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.713+0000 7fa827983b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 0x56365ac4a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.713+0000 7fa827983b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 0x56365ac34530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.713+0000 7fa827983b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365ac56020 con 0x56365ac43df0 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.713+0000 7fa827983b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365ac41f70 con 0x56365ac2df70 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.713+0000 7fa827983b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365ac4ad20 con 0x56365ac38ef0 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.713+0000 7fa811400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac3b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.713+0000 7fa811400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac3b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36284/0 (socket says 172.21.15.67:36284) 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.713+0000 7fa811400700 1 -- 172.21.15.67:0/2303261226 learned_addr learned my addr 172.21.15.67:0/2303261226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.714+0000 7fa811400700 1 -- 172.21.15.67:0/2303261226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 msgr2=0x56365ac34530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.714+0000 7fa811400700 1 --2- 172.21.15.67:0/2303261226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 0x56365ac34530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.714+0000 7fa810a00700 1 --2- 172.21.15.67:0/2303261226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 0x56365ac4a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.714+0000 7fa811400700 1 -- 172.21.15.67:0/2303261226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 msgr2=0x56365ac4a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:47.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.714+0000 7fa811400700 1 --2- 172.21.15.67:0/2303261226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 0x56365ac4a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.714+0000 7fa811400700 1 -- 172.21.15.67:0/2303261226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56365a9b8b60 con 0x56365ac38ef0 2024-03-30T02:44:47.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.714+0000 7fa810a00700 1 --2- 172.21.15.67:0/2303261226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 0x56365ac4a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:47.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.715+0000 7fa811400700 1 --2- 172.21.15.67:0/2303261226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac3b520 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa800000f80 tx=0x7fa80001ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f26a031252d2f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:47.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.715+0000 7fa7ff000700 1 -- 172.21.15.67:0/2303261226 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800018040 con 0x56365ac38ef0 2024-03-30T02:44:47.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.717+0000 7fa7ff000700 1 -- 172.21.15.67:0/2303261226 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa800024040 con 0x56365ac38ef0 2024-03-30T02:44:47.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.717+0000 7fa7ff000700 1 -- 172.21.15.67:0/2303261226 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa800029850 con 0x56365ac38ef0 2024-03-30T02:44:47.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.718+0000 7fa827983b80 1 -- 172.21.15.67:0/2303261226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 msgr2=0x56365ac3b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:47.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.718+0000 7fa827983b80 1 --2- 172.21.15.67:0/2303261226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac3b520 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa800000f80 tx=0x7fa80001ef90 comp rx=0 tx=0).stop 2024-03-30T02:44:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.723+0000 7fa827983b80 1 -- 172.21.15.67:0/2303261226 shutdown_connections 2024-03-30T02:44:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.723+0000 7fa827983b80 1 --2- 172.21.15.67:0/2303261226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 0x56365ac34530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.723+0000 7fa827983b80 1 --2- 172.21.15.67:0/2303261226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac3b520 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.723+0000 7fa827983b80 1 --2- 172.21.15.67:0/2303261226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 0x56365ac4a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.723+0000 7fa827983b80 1 -- 172.21.15.67:0/2303261226 >> 172.21.15.67:0/2303261226 conn(0x56365ab986a0 msgr2=0x56365ac2d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.726+0000 7fa827983b80 1 -- 172.21.15.67:0/2303261226 shutdown_connections 2024-03-30T02:44:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.726+0000 7fa827983b80 1 -- 172.21.15.67:0/2303261226 wait complete. 2024-03-30T02:44:47.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.726+0000 7fa827983b80 1 -- start start 2024-03-30T02:44:47.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.727+0000 7fa827983b80 1 Processor -- start 2024-03-30T02:44:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.730+0000 7fa827983b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 0x56365ac48e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa827983b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac49340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa827983b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 0x56365ac49880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa827983b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365aca76c0 con 0x56365ac43df0 2024-03-30T02:44:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa827983b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365aca7800 con 0x56365ac2df70 2024-03-30T02:44:47.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa827983b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56365aca7940 con 0x56365ac38ef0 2024-03-30T02:44:47.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa827983b80 10 client.0 _ll_register_callbacks cb 0x56365ab30470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:47.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa810a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac49340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa811400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 0x56365ac48e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa810a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac49340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36292/0 (socket says 172.21.15.67:36292) 2024-03-30T02:44:47.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa810a00700 1 -- 172.21.15.67:0/3230333542 learned_addr learned my addr 172.21.15.67:0/3230333542 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:47.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.731+0000 7fa811e00700 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 0x56365ac49880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.732+0000 7fa810a00700 1 -- 172.21.15.67:0/3230333542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 msgr2=0x56365ac48e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.732+0000 7fa810a00700 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 0x56365ac48e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.732+0000 7fa810a00700 1 -- 172.21.15.67:0/3230333542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 msgr2=0x56365ac49880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.732+0000 7fa810a00700 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 0x56365ac49880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.732+0000 7fa810a00700 1 -- 172.21.15.67:0/3230333542 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c10e6e0 con 0x56365ac38ef0 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.732+0000 7fa811400700 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 0x56365ac48e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.732+0000 7fa810a00700 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac49340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8000c00 tx=0x7fa7f8009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc805a866b70710c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.732+0000 7fa7f2a00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.733+0000 7fa7f2a00700 1 -- 172.21.15.67:0/3230333542 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f801cc70 con 0x56365ac38ef0 2024-03-30T02:44:47.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.733+0000 7fa7f2a00700 1 -- 172.21.15.67:0/3230333542 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fa7f801cdd0 con 0x56365ac38ef0 2024-03-30T02:44:47.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.733+0000 7fa827983b80 1 -- 172.21.15.67:0/3230333542 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa80409db30 con 0x56365ac38ef0 2024-03-30T02:44:47.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.733+0000 7fa7f2a00700 1 -- 172.21.15.67:0/3230333542 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7f8025470 con 0x56365ac38ef0 2024-03-30T02:44:47.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.733+0000 7fa827983b80 10 client.27070 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.733+0000 7fa827983b80 1 -- 172.21.15.67:0/3230333542 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56365ab50ea0 con 0x56365ac38ef0 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.734+0000 7fa7f2a00700 1 -- 172.21.15.67:0/3230333542 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa80409db30 con 0x56365ac38ef0 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.734+0000 7fa7f2a00700 1 -- 172.21.15.67:0/3230333542 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa7f8016700 con 0x56365ac38ef0 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.734+0000 7fa827983b80 10 client.27070 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.734+0000 7fa827983b80 10 client.27070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.734+0000 7fa827983b80 20 client.27070 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.734+0000 7fa827983b80 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56365aca9150 0x56365acab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.734+0000 7fa827983b80 4 client.27070 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.734+0000 7fa827983b80 1 -- 172.21.15.67:0/3230333542 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56365ab83700 con 0x56365aca9150 2024-03-30T02:44:47.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.734+0000 7fa811400700 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56365aca9150 0x56365acab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:47.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.735+0000 7fa7f2a00700 10 client.27070 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:47.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.735+0000 7fa811400700 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56365aca9150 0x56365acab570 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e77a3f5277d7d60 server_cookie=c4b62f1d96c9114b in_seq=0 out_seq=0 2024-03-30T02:44:47.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.736+0000 7fa7f2a00700 1 -- 172.21.15.67:0/3230333542 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56365ab83700 con 0x56365aca9150 2024-03-30T02:44:47.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.736+0000 7fa7f2a00700 10 client.27070 handle_command_reply: tid=0 2024-03-30T02:44:47.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 client.27070 shutdown 2024-03-30T02:44:47.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 -- 172.21.15.67:0/3230333542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 msgr2=0x56365ac49340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:47.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac49340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8000c00 tx=0x7fa7f8009f40 comp rx=0 tx=0).stop 2024-03-30T02:44:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 -- 172.21.15.67:0/3230333542 shutdown_connections 2024-03-30T02:44:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56365aca9150 0x56365acab570 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56365ac43df0 0x56365ac49880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56365ac2df70 0x56365ac48e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 --2- 172.21.15.67:0/3230333542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56365ac38ef0 0x56365ac49340 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa7f2a00700 0 client.27070 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 -- 172.21.15.67:0/3230333542 >> 172.21.15.67:0/3230333542 conn(0x56365ab986a0 msgr2=0x56365ac403a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 -- 172.21.15.67:0/3230333542 shutdown_connections 2024-03-30T02:44:47.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.737+0000 7fa827983b80 1 -- 172.21.15.67:0/3230333542 wait complete. 2024-03-30T02:44:47.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.740+0000 7fa827983b80 20 client.27070 trim_cache size 0 max 16384 2024-03-30T02:44:47.739 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:44:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:44:47.739 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:44:47.740 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:44:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 -- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7f40a13e0 msgr2=0x7fa7f40a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7f40a13e0 0x7fa7f40a38a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa800013a50 tx=0x7fa800015040 comp rx=0 tx=0).stop 2024-03-30T02:44:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 -- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c10d770 msgr2=0x7fa80c1a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:47.744 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c10d770 0x7fa80c1a6670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa804006010 tx=0x7fa804003d60 comp rx=0 tx=0).stop 2024-03-30T02:44:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 -- 172.21.15.67:0/1111264261 shutdown_connections 2024-03-30T02:44:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa7f40a13e0 0x7fa7f40a38a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa80c10d770 0x7fa80c1a6670 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa80c107db0 0x7fa80c1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 --2- 172.21.15.67:0/1111264261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa80c108770 0x7fa80c1a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:47.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.745+0000 7fa813e00700 1 -- 172.21.15.67:0/1111264261 >> 172.21.15.67:0/1111264261 conn(0x7fa80c06bdf0 msgr2=0x7fa80c112ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:47.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.746+0000 7fa813e00700 1 -- 172.21.15.67:0/1111264261 shutdown_connections 2024-03-30T02:44:47.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:47.746+0000 7fa813e00700 1 -- 172.21.15.67:0/1111264261 wait complete. 2024-03-30T02:44:47.780 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_sched_switch.o 2024-03-30T02:44:47.799 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:44:47.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:44:47.897 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/xdr4.o 2024-03-30T02:44:47.901 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/extable.o 2024-03-30T02:44:48.156 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/export.o 2024-03-30T02:44:48.281 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/fid.o 2024-03-30T02:44:48.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.313+0000 7f2f1f400700 1 -- 172.21.15.67:0/1494247713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f18113af0 msgr2=0x7f2f18115ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:48.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.313+0000 7f2f1f400700 1 --2- 172.21.15.67:0/1494247713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f18113af0 0x7f2f18115ee0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009990 tx=0x7f2f08009ca0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.316+0000 7f2f1f400700 1 -- 172.21.15.67:0/1494247713 shutdown_connections 2024-03-30T02:44:48.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.316+0000 7f2f1f400700 1 --2- 172.21.15.67:0/1494247713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f18113af0 0x7f2f18115ee0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.316+0000 7f2f1f400700 1 --2- 172.21.15.67:0/1494247713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f1806d100 0x7f2f18107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.316+0000 7f2f1f400700 1 --2- 172.21.15.67:0/1494247713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f18108380 0x7f2f18108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.316+0000 7f2f1f400700 1 -- 172.21.15.67:0/1494247713 >> 172.21.15.67:0/1494247713 conn(0x7f2f1806bcf0 msgr2=0x7f2f1806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:48.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.320+0000 7f2f1f400700 1 -- 172.21.15.67:0/1494247713 shutdown_connections 2024-03-30T02:44:48.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.320+0000 7f2f1f400700 1 -- 172.21.15.67:0/1494247713 wait complete. 2024-03-30T02:44:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.321+0000 7f2f1f400700 1 Processor -- start 2024-03-30T02:44:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.323+0000 7f2f1f400700 1 -- start start 2024-03-30T02:44:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.323+0000 7f2f1f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f1806d100 0x7f2f18113450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.323+0000 7f2f1f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f18108380 0x7f2f1810f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.323+0000 7f2f1f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f18113af0 0x7f2f1810fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1f400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181103b0 con 0x7f2f18113af0 2024-03-30T02:44:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1f400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181b2290 con 0x7f2f1806d100 2024-03-30T02:44:48.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f1806d100 0x7f2f18113450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:48.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f1806d100 0x7f2f18113450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:54764/0 (socket says 172.21.15.67:54764) 2024-03-30T02:44:48.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1ca00700 1 -- 172.21.15.67:0/2237854131 learned_addr learned my addr 172.21.15.67:0/2237854131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:48.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181b2450 con 0x7f2f18108380 2024-03-30T02:44:48.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1ca00700 1 -- 172.21.15.67:0/2237854131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f18108380 msgr2=0x7f2f1810f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:48.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1ca00700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f18108380 0x7f2f1810f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1ca00700 1 -- 172.21.15.67:0/2237854131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f18113af0 msgr2=0x7f2f1810fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:48.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1ca00700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f18113af0 0x7f2f1810fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.324+0000 7f2f1ca00700 1 -- 172.21.15.67:0/2237854131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f080095f0 con 0x7f2f1806d100 2024-03-30T02:44:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.325+0000 7f2f1ca00700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f1806d100 0x7f2f18113450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000b840 tx=0x7f2f1000bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.325+0000 7f2f15600700 1 -- 172.21.15.67:0/2237854131 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000d890 con 0x7f2f1806d100 2024-03-30T02:44:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.325+0000 7f2f15600700 1 -- 172.21.15.67:0/2237854131 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1000e430 con 0x7f2f1806d100 2024-03-30T02:44:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.325+0000 7f2f15600700 1 -- 172.21.15.67:0/2237854131 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10017660 con 0x7f2f1806d100 2024-03-30T02:44:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.325+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f181b2730 con 0x7f2f1806d100 2024-03-30T02:44:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.326+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f181b2bf0 con 0x7f2f1806d100 2024-03-30T02:44:48.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.328+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f18109490 con 0x7f2f1806d100 2024-03-30T02:44:48.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.334+0000 7f2f15600700 1 -- 172.21.15.67:0/2237854131 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2f1000e810 con 0x7f2f1806d100 2024-03-30T02:44:48.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.336+0000 7f2f15600700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2efc0a12d0 0x7f2efc0a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:48.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.337+0000 7f2f15600700 1 -- 172.21.15.67:0/2237854131 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2f100ccbe0 con 0x7f2f1806d100 2024-03-30T02:44:48.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.340+0000 7f2f17e00700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2efc0a12d0 0x7f2efc0a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:48.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.341+0000 7f2f17e00700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2efc0a12d0 0x7f2efc0a3790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f18110c40 tx=0x7f2f040096c0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:48.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.344+0000 7f2f15600700 1 -- 172.21.15.67:0/2237854131 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2f10095c90 con 0x7f2f1806d100 2024-03-30T02:44:48.592 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/flex_proportions.o 2024-03-30T02:44:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:48 smithi170 ceph-mon[78798]: pgmap v237: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.3 MiB/s wr, 71 op/s 2024-03-30T02:44:48.708 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr.o 2024-03-30T02:44:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:48 smithi082 ceph-mon[79176]: pgmap v237: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.3 MiB/s wr, 71 op/s 2024-03-30T02:44:48.819 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_nop.o 2024-03-30T02:44:48.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:48 smithi067 ceph-mon[37808]: pgmap v237: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.3 MiB/s wr, 71 op/s 2024-03-30T02:44:48.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.956+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f180679a0 con 0x7f2f1806d100 2024-03-30T02:44:48.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.961+0000 7f2f15600700 1 -- 172.21.15.67:0/2237854131 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10916 (secure 0 0 0) 0x7f2f1000e5a0 con 0x7f2f1806d100 2024-03-30T02:44:48.967 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:44:48.967 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:44:02.385480+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:44:48.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2efc0a12d0 msgr2=0x7f2efc0a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:48.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2efc0a12d0 0x7f2efc0a3790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f18110c40 tx=0x7f2f040096c0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f1806d100 msgr2=0x7f2f18113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:48.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f1806d100 0x7f2f18113450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000b840 tx=0x7f2f1000bc00 comp rx=0 tx=0).stop 2024-03-30T02:44:48.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 shutdown_connections 2024-03-30T02:44:48.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2efc0a12d0 0x7f2efc0a3790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f18113af0 0x7f2f1810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f1806d100 0x7f2f18113450 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 --2- 172.21.15.67:0/2237854131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f18108380 0x7f2f1810f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:48.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 >> 172.21.15.67:0/2237854131 conn(0x7f2f1806bcf0 msgr2=0x7f2f18072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:48.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.971+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 shutdown_connections 2024-03-30T02:44:48.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:48.972+0000 7f2f1f400700 1 -- 172.21.15.67:0/2237854131 wait complete. 2024-03-30T02:44:48.973 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 39 2024-03-30T02:44:49.019 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:44:49.068 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/idr.o 2024-03-30T02:44:49.128 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/svc4proc.o 2024-03-30T02:44:49.150 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/9p/xattr.o 2024-03-30T02:44:49.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.490+0000 7f8d29800700 1 -- 172.21.15.67:0/1285889319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24107db0 msgr2=0x7f8d24108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:49.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.490+0000 7f8d29800700 1 --2- 172.21.15.67:0/1285889319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24107db0 0x7f8d24108190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000fb90 tx=0x7f8d1000fea0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.492+0000 7f8d29800700 1 -- 172.21.15.67:0/1285889319 shutdown_connections 2024-03-30T02:44:49.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.492+0000 7f8d29800700 1 --2- 172.21.15.67:0/1285889319 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d24113f00 0x7f8d241162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.492+0000 7f8d29800700 1 --2- 172.21.15.67:0/1285889319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24107db0 0x7f8d24108190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.492+0000 7f8d29800700 1 --2- 172.21.15.67:0/1285889319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d241086d0 0x7f8d241139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.492+0000 7f8d29800700 1 -- 172.21.15.67:0/1285889319 >> 172.21.15.67:0/1285889319 conn(0x7f8d2406bdf0 msgr2=0x7f8d2406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:49.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.496+0000 7f8d29800700 1 -- 172.21.15.67:0/1285889319 shutdown_connections 2024-03-30T02:44:49.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.496+0000 7f8d29800700 1 -- 172.21.15.67:0/1285889319 wait complete. 2024-03-30T02:44:49.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.498+0000 7f8d29800700 1 Processor -- start 2024-03-30T02:44:49.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.498+0000 7f8d29800700 1 -- start start 2024-03-30T02:44:49.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.498+0000 7f8d29800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d24107db0 0x7f8d241a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.498+0000 7f8d29800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d241086d0 0x7f8d241a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.498+0000 7f8d29800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24113f00 0x7f8d2419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.498+0000 7f8d29800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2419df60 con 0x7f8d24107db0 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.498+0000 7f8d29800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2419e0d0 con 0x7f8d241086d0 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.498+0000 7f8d29800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2419e290 con 0x7f8d24113f00 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d23400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24113f00 0x7f8d2419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d22000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d241086d0 0x7f8d241a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d23400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24113f00 0x7f8d2419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36336/0 (socket says 172.21.15.67:36336) 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d23400700 1 -- 172.21.15.67:0/3985873596 learned_addr learned my addr 172.21.15.67:0/3985873596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d22a00700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d24107db0 0x7f8d241a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d23400700 1 -- 172.21.15.67:0/3985873596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d241086d0 msgr2=0x7f8d241a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:49.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d23400700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d241086d0 0x7f8d241a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d23400700 1 -- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d24107db0 msgr2=0x7f8d241a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:49.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d23400700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d24107db0 0x7f8d241a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.499+0000 7f8d23400700 1 -- 172.21.15.67:0/3985873596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140095f0 con 0x7f8d24113f00 2024-03-30T02:44:49.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.500+0000 7f8d23400700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24113f00 0x7f8d2419d7d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400ebe0 tx=0x7f8d1400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:49.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.500+0000 7f8d22a00700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d24107db0 0x7f8d241a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:49.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.504+0000 7f8d1b400700 1 -- 172.21.15.67:0/3985873596 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400cd40 con 0x7f8d24113f00 2024-03-30T02:44:49.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.504+0000 7f8d1b400700 1 -- 172.21.15.67:0/3985873596 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d140044f0 con 0x7f8d24113f00 2024-03-30T02:44:49.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.504+0000 7f8d1b400700 1 -- 172.21.15.67:0/3985873596 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14010430 con 0x7f8d24113f00 2024-03-30T02:44:49.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.504+0000 7f8d29800700 1 -- 172.21.15.67:0/3985873596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1000f790 con 0x7f8d24113f00 2024-03-30T02:44:49.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.504+0000 7f8d29800700 1 -- 172.21.15.67:0/3985873596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d241a2310 con 0x7f8d24113f00 2024-03-30T02:44:49.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.511+0000 7f8d1b400700 1 -- 172.21.15.67:0/3985873596 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8d14010590 con 0x7f8d24113f00 2024-03-30T02:44:49.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.513+0000 7f8d1b400700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8d080a1310 0x7f8d080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.514+0000 7f8d1b400700 1 -- 172.21.15.67:0/3985873596 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f8d14014070 con 0x7f8d24113f00 2024-03-30T02:44:49.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.514+0000 7f8d22a00700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8d080a1310 0x7f8d080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.517+0000 7f8d22a00700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8d080a1310 0x7f8d080a37d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005bf0 tx=0x7f8d10005b60 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.518+0000 7f8d3d3cfb80 1 Processor -- start 2024-03-30T02:44:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.518+0000 7f8d3d3cfb80 1 -- start start 2024-03-30T02:44:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e66d5d0 0x55758e66d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e6788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e678df0 0x55758e67b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e5c6990 con 0x55758e66d5d0 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e5c4350 con 0x55758e678df0 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e5bfc40 con 0x55758e66def0 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d22a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e66d5d0 0x55758e66d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d23400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e678df0 0x55758e67b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d23400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e678df0 0x55758e67b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:54798/0 (socket says 172.21.15.67:54798) 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.519+0000 7f8d23400700 1 -- 172.21.15.67:0/1051907240 learned_addr learned my addr 172.21.15.67:0/1051907240 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.520+0000 7f8d23400700 1 -- 172.21.15.67:0/1051907240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 msgr2=0x55758e6788b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.520+0000 7f8d23400700 1 --2- 172.21.15.67:0/1051907240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e6788b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.520+0000 7f8d23400700 1 -- 172.21.15.67:0/1051907240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e66d5d0 msgr2=0x55758e66d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.520+0000 7f8d23400700 1 --2- 172.21.15.67:0/1051907240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e66d5d0 0x55758e66d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.520+0000 7f8d23400700 1 -- 172.21.15.67:0/1051907240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55758e4413b0 con 0x55758e678df0 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.520+0000 7f8d22a00700 1 --2- 172.21.15.67:0/1051907240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e66d5d0 0x55758e66d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:49.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.520+0000 7f8d23400700 1 --2- 172.21.15.67:0/1051907240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e678df0 0x55758e67b1e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8d14005950 tx=0x7f8d1409abb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83d9ea54a015684 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:49.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.521+0000 7f8d0fe00700 1 -- 172.21.15.67:0/1051907240 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1409b180 con 0x55758e678df0 2024-03-30T02:44:49.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.521+0000 7f8d0fe00700 1 -- 172.21.15.67:0/1051907240 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1409efb0 con 0x55758e678df0 2024-03-30T02:44:49.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.521+0000 7f8d0fe00700 1 -- 172.21.15.67:0/1051907240 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140038e0 con 0x55758e678df0 2024-03-30T02:44:49.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.523+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1051907240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e678df0 msgr2=0x55758e67b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:49.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.523+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1051907240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e678df0 0x55758e67b1e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8d14005950 tx=0x7f8d1409abb0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.527+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1051907240 shutdown_connections 2024-03-30T02:44:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.527+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1051907240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e66d5d0 0x55758e66d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.527+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1051907240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e678df0 0x55758e67b1e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.527+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1051907240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e6788b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.527+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1051907240 >> 172.21.15.67:0/1051907240 conn(0x55758e5cd6a0 msgr2=0x55758e669a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.527+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1051907240 shutdown_connections 2024-03-30T02:44:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.527+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1051907240 wait complete. 2024-03-30T02:44:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.528+0000 7f8d3d3cfb80 1 -- start start 2024-03-30T02:44:49.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.532+0000 7f8d3d3cfb80 1 Processor -- start 2024-03-30T02:44:49.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.532+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66d5d0 0x55758e678820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.532+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66def0 0x55758e675b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.532+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e6762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.532+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e55bd80 con 0x55758e678df0 2024-03-30T02:44:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.532+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e55ba20 con 0x55758e66def0 2024-03-30T02:44:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.532+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e4436a0 con 0x55758e66d5d0 2024-03-30T02:44:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.533+0000 7f8d3d3cfb80 10 client.0 _ll_register_callbacks cb 0x55758e565470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.533+0000 7f8d22a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66d5d0 0x55758e678820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.533+0000 7f8d23400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e6762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.533+0000 7f8d23400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e6762a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39070/0 (socket says 172.21.15.67:39070) 2024-03-30T02:44:49.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.533+0000 7f8d23400700 1 -- 172.21.15.67:0/523734376 learned_addr learned my addr 172.21.15.67:0/523734376 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:49.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.533+0000 7f8d22a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66d5d0 0x55758e678820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36350/0 (socket says 172.21.15.67:36350) 2024-03-30T02:44:49.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.534+0000 7f8d22000700 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66def0 0x55758e675b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.534+0000 7f8d23400700 1 -- 172.21.15.67:0/523734376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66d5d0 msgr2=0x55758e678820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.534+0000 7f8d23400700 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66d5d0 0x55758e678820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.534+0000 7f8d23400700 1 -- 172.21.15.67:0/523734376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66def0 msgr2=0x55758e675b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.534+0000 7f8d23400700 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66def0 0x55758e675b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.534+0000 7f8d23400700 1 -- 172.21.15.67:0/523734376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d24116cc0 con 0x55758e678df0 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.534+0000 7f8d22000700 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66def0 0x55758e675b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.534+0000 7f8d22a00700 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66d5d0 0x55758e678820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.535+0000 7f8d23400700 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e6762a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8d1409ae80 tx=0x7f8d140975f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4ebd9eb9ccf4b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.535+0000 7f8d0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.535+0000 7f8d0cc00700 1 -- 172.21.15.67:0/523734376 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14003890 con 0x55758e678df0 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.535+0000 7f8d0cc00700 1 -- 172.21.15.67:0/523734376 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f8d1409d070 con 0x55758e678df0 2024-03-30T02:44:49.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.535+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/523734376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55758e580380 con 0x55758e678df0 2024-03-30T02:44:49.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.540+0000 7f8d0cc00700 1 -- 172.21.15.67:0/523734376 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d14003890 con 0x55758e678df0 2024-03-30T02:44:49.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.540+0000 7f8d3d3cfb80 10 client.17019 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:49.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.540+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/523734376 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f8d241a2310 con 0x55758e678df0 2024-03-30T02:44:49.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.540+0000 7f8d0cc00700 1 -- 172.21.15.67:0/523734376 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55758e580380 con 0x55758e678df0 2024-03-30T02:44:49.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.541+0000 7f8d0cc00700 1 -- 172.21.15.67:0/523734376 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8d140a3170 con 0x55758e678df0 2024-03-30T02:44:49.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.544+0000 7f8d3d3cfb80 10 client.17019 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:49.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.544+0000 7f8d3d3cfb80 10 client.17019 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:44:49.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.544+0000 7f8d3d3cfb80 20 client.17019 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:49.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.544+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55758e688590 0x55758e68a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.544+0000 7f8d3d3cfb80 4 client.17019 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:49.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.544+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/523734376 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55758e5b8700 con 0x55758e688590 2024-03-30T02:44:49.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.546+0000 7f8d22a00700 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55758e688590 0x55758e68a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.547+0000 7f8d0cc00700 10 client.17019 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:44:49.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.547+0000 7f8d22a00700 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55758e688590 0x55758e68a9b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1f55470fb22ff65 server_cookie=dcfa703187d135f7 in_seq=0 out_seq=0 2024-03-30T02:44:49.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.548+0000 7f8d0cc00700 1 -- 172.21.15.67:0/523734376 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55758e5b8700 con 0x55758e688590 2024-03-30T02:44:49.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.548+0000 7f8d0cc00700 10 client.17019 handle_command_reply: tid=0 2024-03-30T02:44:49.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.548+0000 7f8d3d3cfb80 1 client.17019 shutdown 2024-03-30T02:44:49.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.548+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/523734376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 msgr2=0x55758e6762a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:49.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.548+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e6762a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8d1409ae80 tx=0x7f8d140975f0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.549+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/523734376 shutdown_connections 2024-03-30T02:44:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.549+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e6762a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.549+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55758e688590 0x55758e68a9b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.549+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66d5d0 0x55758e678820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.549+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/523734376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66def0 0x55758e675b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.549+0000 7f8d0cc00700 0 client.17019 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:44:49.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.549+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/523734376 >> 172.21.15.67:0/523734376 conn(0x55758e5cd6a0 msgr2=0x55758e66f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:49.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.550+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/523734376 shutdown_connections 2024-03-30T02:44:49.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.550+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/523734376 wait complete. 2024-03-30T02:44:49.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.551+0000 7f8d3d3cfb80 20 client.17019 trim_cache size 0 max 16384 2024-03-30T02:44:49.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.597+0000 7f8d3d3cfb80 1 Processor -- start 2024-03-30T02:44:49.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.597+0000 7f8d3d3cfb80 1 -- start start 2024-03-30T02:44:49.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.598+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 0x55758e68a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.598+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e676220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.598+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e668f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.598+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e66a660 con 0x55758e678df0 2024-03-30T02:44:49.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.598+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e66e850 con 0x55758e66d5d0 2024-03-30T02:44:49.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.598+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e6709a0 con 0x55758e66def0 2024-03-30T02:44:49.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.598+0000 7f8d22a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 0x55758e68a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.598+0000 7f8d22a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 0x55758e68a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:54820/0 (socket says 172.21.15.67:54820) 2024-03-30T02:44:49.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.598+0000 7f8d22a00700 1 -- 172.21.15.67:0/309383830 learned_addr learned my addr 172.21.15.67:0/309383830 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:49.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.599+0000 7f8d23400700 1 --2- 172.21.15.67:0/309383830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e668f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.600+0000 7f8d23400700 1 -- 172.21.15.67:0/309383830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 msgr2=0x55758e676220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:49.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.600+0000 7f8d23400700 1 --2- 172.21.15.67:0/309383830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e676220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.600+0000 7f8d23400700 1 -- 172.21.15.67:0/309383830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 msgr2=0x55758e68a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:49.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.600+0000 7f8d23400700 1 --2- 172.21.15.67:0/309383830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 0x55758e68a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.600+0000 7f8d23400700 1 -- 172.21.15.67:0/309383830 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55758e3edb60 con 0x55758e678df0 2024-03-30T02:44:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.600+0000 7f8d23400700 1 --2- 172.21.15.67:0/309383830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e668f10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8d14097ba0 tx=0x7f8d14016e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2008ecb9d3e1cb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.601+0000 7f8d18c00700 1 -- 172.21.15.67:0/309383830 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14003890 con 0x55758e678df0 2024-03-30T02:44:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.601+0000 7f8d18c00700 1 -- 172.21.15.67:0/309383830 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1409b180 con 0x55758e678df0 2024-03-30T02:44:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.601+0000 7f8d18c00700 1 -- 172.21.15.67:0/309383830 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1409a670 con 0x55758e678df0 2024-03-30T02:44:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.602+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/309383830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 msgr2=0x55758e668f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.602+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/309383830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e668f10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8d14097ba0 tx=0x7f8d14016e60 comp rx=0 tx=0).stop 2024-03-30T02:44:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.603+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/309383830 shutdown_connections 2024-03-30T02:44:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.603+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/309383830 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e668f10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.603+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/309383830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 0x55758e68a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.603+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/309383830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e676220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.603+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/309383830 >> 172.21.15.67:0/309383830 conn(0x55758e5cd6a0 msgr2=0x55758e6625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:49.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.606+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/309383830 shutdown_connections 2024-03-30T02:44:49.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.606+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/309383830 wait complete. 2024-03-30T02:44:49.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.607+0000 7f8d3d3cfb80 1 -- start start 2024-03-30T02:44:49.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.607+0000 7f8d3d3cfb80 1 Processor -- start 2024-03-30T02:44:49.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.612+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 0x55758e67fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.612+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e67dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.612+0000 7f8d3d3cfb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e67e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.612+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e67e7a0 con 0x55758e678df0 2024-03-30T02:44:49.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.612+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e67e910 con 0x55758e66d5d0 2024-03-30T02:44:49.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.612+0000 7f8d3d3cfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55758e6db3b0 con 0x55758e66def0 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.612+0000 7f8d3d3cfb80 10 client.0 _ll_register_callbacks cb 0x55758e565470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.613+0000 7f8d22000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e67dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.613+0000 7f8d22000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e67dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36356/0 (socket says 172.21.15.67:36356) 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.613+0000 7f8d22000700 1 -- 172.21.15.67:0/1559533121 learned_addr learned my addr 172.21.15.67:0/1559533121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.613+0000 7f8d22a00700 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 0x55758e67fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.613+0000 7f8d23400700 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e67e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.613+0000 7f8d22000700 1 -- 172.21.15.67:0/1559533121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 msgr2=0x55758e67fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.613+0000 7f8d22000700 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 0x55758e67fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.614+0000 7f8d22000700 1 -- 172.21.15.67:0/1559533121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 msgr2=0x55758e67e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.614+0000 7f8d22000700 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e67e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.614+0000 7f8d22000700 1 -- 172.21.15.67:0/1559533121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d241a2310 con 0x55758e66def0 2024-03-30T02:44:49.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.614+0000 7f8d23400700 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e67e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:49.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.618+0000 7f8d22000700 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e67dab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009fd0 tx=0x7f8d1c004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f23dc619223dd8d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:49.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.618+0000 7f8d0cc00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:49.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.618+0000 7f8d0cc00700 1 -- 172.21.15.67:0/1559533121 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c004bb0 con 0x55758e66def0 2024-03-30T02:44:49.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.618+0000 7f8d0cc00700 1 -- 172.21.15.67:0/1559533121 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f8d1c004d10 con 0x55758e66def0 2024-03-30T02:44:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.618+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1559533121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d1409efb0 con 0x55758e66def0 2024-03-30T02:44:49.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.619+0000 7f8d0cc00700 1 -- 172.21.15.67:0/1559533121 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d1c007710 con 0x55758e66def0 2024-03-30T02:44:49.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.620+0000 7f8d3d3cfb80 10 client.27085 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:49.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.620+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1559533121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55758e585ea0 con 0x55758e66def0 2024-03-30T02:44:49.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.621+0000 7f8d0cc00700 1 -- 172.21.15.67:0/1559533121 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d1c00f070 con 0x55758e66def0 2024-03-30T02:44:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.621+0000 7f8d0cc00700 1 -- 172.21.15.67:0/1559533121 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8d1c014070 con 0x55758e66def0 2024-03-30T02:44:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.621+0000 7f8d3d3cfb80 10 client.27085 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.621+0000 7f8d3d3cfb80 10 client.27085 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:44:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.621+0000 7f8d3d3cfb80 20 client.27085 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:49.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.621+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55758e6dcc10 0x55758e6df030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.621+0000 7f8d3d3cfb80 4 client.27085 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.621+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1559533121 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f8d1409efb0 con 0x55758e6dcc10 2024-03-30T02:44:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.622+0000 7f8d22a00700 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55758e6dcc10 0x55758e6df030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.622+0000 7f8d22a00700 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55758e6dcc10 0x55758e6df030 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f4acf88d3d5878a server_cookie=619c28cd20065ea0 in_seq=0 out_seq=0 2024-03-30T02:44:49.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.622+0000 7f8d0cc00700 10 client.27085 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:44:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.623+0000 7f8d0cc00700 1 -- 172.21.15.67:0/1559533121 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f8d1409efb0 con 0x55758e6dcc10 2024-03-30T02:44:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.623+0000 7f8d0cc00700 10 client.27085 handle_command_reply: tid=0 2024-03-30T02:44:49.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.623+0000 7f8d3d3cfb80 1 client.27085 shutdown 2024-03-30T02:44:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.623+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1559533121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 msgr2=0x55758e67dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:49.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.624+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e67dab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009fd0 tx=0x7f8d1c004320 comp rx=0 tx=0).stop 2024-03-30T02:44:49.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.625+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1559533121 shutdown_connections 2024-03-30T02:44:49.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.626+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55758e678df0 0x55758e67e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.626+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55758e6dcc10 0x55758e6df030 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.626+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55758e66d5d0 0x55758e67fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.626+0000 7f8d0cc00700 0 client.27085 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:44:49.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.626+0000 7f8d3d3cfb80 1 --2- 172.21.15.67:0/1559533121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55758e66def0 0x55758e67dab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.626+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1559533121 >> 172.21.15.67:0/1559533121 conn(0x55758e5cd6a0 msgr2=0x55758e670e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:49.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.626+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1559533121 shutdown_connections 2024-03-30T02:44:49.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.628+0000 7f8d3d3cfb80 1 -- 172.21.15.67:0/1559533121 wait complete. 2024-03-30T02:44:49.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.628+0000 7f8d3d3cfb80 20 client.27085 trim_cache size 0 max 16384 2024-03-30T02:44:49.627 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 -- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8d080a1310 msgr2=0x7f8d080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8d080a1310 0x7f8d080a37d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005bf0 tx=0x7f8d10005b60 comp rx=0 tx=0).stop 2024-03-30T02:44:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 -- 172.21.15.67:0/3985873596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24113f00 msgr2=0x7f8d2419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:49.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24113f00 0x7f8d2419d7d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400ebe0 tx=0x7f8d1400eef0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 -- 172.21.15.67:0/3985873596 shutdown_connections 2024-03-30T02:44:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8d080a1310 0x7f8d080a37d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8d24107db0 0x7f8d241a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d24113f00 0x7f8d2419d7d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 --2- 172.21.15.67:0/3985873596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d241086d0 0x7f8d241a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.632+0000 7f8d29800700 1 -- 172.21.15.67:0/3985873596 >> 172.21.15.67:0/3985873596 conn(0x7f8d2406bdf0 msgr2=0x7f8d2406fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.633+0000 7f8d29800700 1 -- 172.21.15.67:0/3985873596 shutdown_connections 2024-03-30T02:44:49.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:49.633+0000 7f8d29800700 1 -- 172.21.15.67:0/3985873596 wait complete. 2024-03-30T02:44:49.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:44:49.716 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/irq_regs.o 2024-03-30T02:44:49.734 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/is_single_threaded.o 2024-03-30T02:44:49.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:49 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2237854131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:44:49.835 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/blktrace.o 2024-03-30T02:44:49.846 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:49 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2237854131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:44:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:49 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2237854131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:44:50.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.097+0000 7fd424000700 1 -- 172.21.15.67:0/1474248514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c0ff440 msgr2=0x7fd41c0ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.097+0000 7fd424000700 1 --2- 172.21.15.67:0/1474248514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c0ff440 0x7fd41c0ff820 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd41000b0a0 tx=0x7fd41000b3b0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.097+0000 7fd424000700 1 -- 172.21.15.67:0/1474248514 shutdown_connections 2024-03-30T02:44:50.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.097+0000 7fd424000700 1 --2- 172.21.15.67:0/1474248514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c0ff440 0x7fd41c0ff820 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.097+0000 7fd424000700 1 --2- 172.21.15.67:0/1474248514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c1042e0 0x7fd41c10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.097+0000 7fd424000700 1 --2- 172.21.15.67:0/1474248514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c0ffdf0 0x7fd41c103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.097+0000 7fd424000700 1 -- 172.21.15.67:0/1474248514 >> 172.21.15.67:0/1474248514 conn(0x7fd41c074b20 msgr2=0x7fd41c074f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.098+0000 7fd424000700 1 -- 172.21.15.67:0/1474248514 shutdown_connections 2024-03-30T02:44:50.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.098+0000 7fd424000700 1 -- 172.21.15.67:0/1474248514 wait complete. 2024-03-30T02:44:50.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.099+0000 7fd424000700 1 Processor -- start 2024-03-30T02:44:50.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.100+0000 7fd424000700 1 -- start start 2024-03-30T02:44:50.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.100+0000 7fd424000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c0ff440 0x7fd41c195000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.100+0000 7fd424000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1042e0 0x7fd41c195540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.100+0000 7fd424000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c1998e0 0x7fd41c199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.100+0000 7fd424000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19a360 con 0x7fd41c1042e0 2024-03-30T02:44:50.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.100+0000 7fd424000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19a4d0 con 0x7fd41c1998e0 2024-03-30T02:44:50.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.100+0000 7fd424000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19a690 con 0x7fd41c0ff440 2024-03-30T02:44:50.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.101+0000 7fd420c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1042e0 0x7fd41c195540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.101+0000 7fd420c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1042e0 0x7fd41c195540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39104/0 (socket says 172.21.15.67:39104) 2024-03-30T02:44:50.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.101+0000 7fd420c00700 1 -- 172.21.15.67:0/2468968370 learned_addr learned my addr 172.21.15.67:0/2468968370 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.101+0000 7fd421600700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c0ff440 0x7fd41c195000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.101+0000 7fd422000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c1998e0 0x7fd41c199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.102+0000 7fd422000700 1 -- 172.21.15.67:0/2468968370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c0ff440 msgr2=0x7fd41c195000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.102+0000 7fd422000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c0ff440 0x7fd41c195000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.102+0000 7fd422000700 1 -- 172.21.15.67:0/2468968370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1042e0 msgr2=0x7fd41c195540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.102+0000 7fd422000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1042e0 0x7fd41c195540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.102+0000 7fd422000700 1 -- 172.21.15.67:0/2468968370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd410009d00 con 0x7fd41c1998e0 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.102+0000 7fd422000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c1998e0 0x7fd41c199d60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ab50 tx=0x7fd414007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.103+0000 7fd41a000700 1 -- 172.21.15.67:0/2468968370 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400f040 con 0x7fd41c1998e0 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.103+0000 7fd41a000700 1 -- 172.21.15.67:0/2468968370 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd414004cf0 con 0x7fd41c1998e0 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.103+0000 7fd41a000700 1 -- 172.21.15.67:0/2468968370 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414005550 con 0x7fd41c1998e0 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.103+0000 7fd424000700 1 -- 172.21.15.67:0/2468968370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c06a7d0 con 0x7fd41c1998e0 2024-03-30T02:44:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.103+0000 7fd424000700 1 -- 172.21.15.67:0/2468968370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c06ac90 con 0x7fd41c1998e0 2024-03-30T02:44:50.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.108+0000 7fd437b3ab80 1 Processor -- start 2024-03-30T02:44:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.109+0000 7fd41a000700 1 -- 172.21.15.67:0/2468968370 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd4140042b0 con 0x7fd41c1998e0 2024-03-30T02:44:50.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.110+0000 7fd437b3ab80 1 -- start start 2024-03-30T02:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.112+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.112+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.112+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 0x55a6680411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.112+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a667f8c990 con 0x55a66803edf0 2024-03-30T02:44:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.112+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a667f8a350 con 0x55a668033ef0 2024-03-30T02:44:50.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.112+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a667f85c40 con 0x55a6680335d0 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.113+0000 7fd421600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.113+0000 7fd421600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:54854/0 (socket says 172.21.15.67:54854) 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.113+0000 7fd420c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 0x55a6680411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.113+0000 7fd421600700 1 -- 172.21.15.67:0/866181436 learned_addr learned my addr 172.21.15.67:0/866181436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.113+0000 7fd420c00700 1 --2- 172.21.15.67:0/866181436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.113+0000 7fd41a000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4040a13e0 0x7fd4040a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.114+0000 7fd420c00700 1 -- 172.21.15.67:0/866181436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 msgr2=0x55a6680339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.114+0000 7fd420c00700 1 --2- 172.21.15.67:0/866181436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.114+0000 7fd420c00700 1 -- 172.21.15.67:0/866181436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 msgr2=0x55a66803e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.114+0000 7fd420c00700 1 --2- 172.21.15.67:0/866181436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.114+0000 7fd420c00700 1 -- 172.21.15.67:0/866181436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a667e073b0 con 0x55a66803edf0 2024-03-30T02:44:50.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.114+0000 7fd41a000700 1 -- 172.21.15.67:0/2468968370 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fd4140dd410 con 0x7fd41c1998e0 2024-03-30T02:44:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.115+0000 7fd420c00700 1 --2- 172.21.15.67:0/866181436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 0x55a6680411e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd41c196550 tx=0x7fd40800b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2f52da93162d62 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.115+0000 7fd40e600700 1 -- 172.21.15.67:0/866181436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408010040 con 0x55a66803edf0 2024-03-30T02:44:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.115+0000 7fd422000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4040a13e0 0x7fd4040a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.115+0000 7fd40e600700 1 -- 172.21.15.67:0/866181436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40800ba40 con 0x55a66803edf0 2024-03-30T02:44:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.116+0000 7fd437b3ab80 1 -- 172.21.15.67:0/866181436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 msgr2=0x55a6680411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.116+0000 7fd422000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4040a13e0 0x7fd4040a38a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd4140a9e80 tx=0x7fd414004560 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.116+0000 7fd40e600700 1 -- 172.21.15.67:0/866181436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40801c730 con 0x55a66803edf0 2024-03-30T02:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.116+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/866181436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 0x55a6680411e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd41c196550 tx=0x7fd40800b470 comp rx=0 tx=0).stop 2024-03-30T02:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.117+0000 7fd437b3ab80 1 -- 172.21.15.67:0/866181436 shutdown_connections 2024-03-30T02:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.117+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/866181436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 0x55a6680411e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.117+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/866181436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.117+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/866181436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.117+0000 7fd437b3ab80 1 -- 172.21.15.67:0/866181436 >> 172.21.15.67:0/866181436 conn(0x55a667f936a0 msgr2=0x55a66802fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.118+0000 7fd437b3ab80 1 -- 172.21.15.67:0/866181436 shutdown_connections 2024-03-30T02:44:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.118+0000 7fd437b3ab80 1 -- 172.21.15.67:0/866181436 wait complete. 2024-03-30T02:44:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.118+0000 7fd437b3ab80 1 -- start start 2024-03-30T02:44:50.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.119+0000 7fd437b3ab80 1 Processor -- start 2024-03-30T02:44:50.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.120+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6680335d0 0x55a66803e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.120+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a668033ef0 0x55a66803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.120+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 0x55a66803c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.120+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a667f21d80 con 0x55a668033ef0 2024-03-30T02:44:50.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.120+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a667f21a20 con 0x55a6680335d0 2024-03-30T02:44:50.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.120+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a667e096a0 con 0x55a66803edf0 2024-03-30T02:44:50.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.120+0000 7fd437b3ab80 10 client.0 _ll_register_callbacks cb 0x55a667f2b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:50.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.121+0000 7fd421600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a668033ef0 0x55a66803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.121+0000 7fd420c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6680335d0 0x55a66803e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.121+0000 7fd421600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a668033ef0 0x55a66803bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39132/0 (socket says 172.21.15.67:39132) 2024-03-30T02:44:50.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.121+0000 7fd421600700 1 -- 172.21.15.67:0/1538624813 learned_addr learned my addr 172.21.15.67:0/1538624813 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.121+0000 7fd420c00700 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 0x55a66803c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd420c00700 1 -- 172.21.15.67:0/1538624813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 msgr2=0x55a66803c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd420c00700 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 0x55a66803c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd420c00700 1 -- 172.21.15.67:0/1538624813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a668033ef0 msgr2=0x55a66803bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd420c00700 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a668033ef0 0x55a66803bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd420c00700 1 -- 172.21.15.67:0/1538624813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd408019620 con 0x55a6680335d0 2024-03-30T02:44:50.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd421600700 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a668033ef0 0x55a66803bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:50.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd420c00700 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6680335d0 0x55a66803e820 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd4080092b0 tx=0x7fd408003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33be12ea38a02fa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd402a00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:50.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd402a00700 1 -- 172.21.15.67:0/1538624813 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408010070 con 0x55a6680335d0 2024-03-30T02:44:50.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd402a00700 1 -- 172.21.15.67:0/1538624813 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fd408003680 con 0x55a6680335d0 2024-03-30T02:44:50.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.122+0000 7fd437b3ab80 1 -- 172.21.15.67:0/1538624813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a667f46380 con 0x55a6680335d0 2024-03-30T02:44:50.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.123+0000 7fd402a00700 1 -- 172.21.15.67:0/1538624813 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd408003840 con 0x55a6680335d0 2024-03-30T02:44:50.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.123+0000 7fd437b3ab80 10 client.27137 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.123+0000 7fd437b3ab80 1 -- 172.21.15.67:0/1538624813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a667db3b60 con 0x55a6680335d0 2024-03-30T02:44:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.123+0000 7fd402a00700 1 -- 172.21.15.67:0/1538624813 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a667f46380 con 0x55a6680335d0 2024-03-30T02:44:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.124+0000 7fd402a00700 1 -- 172.21.15.67:0/1538624813 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd408019940 con 0x55a6680335d0 2024-03-30T02:44:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.124+0000 7fd437b3ab80 10 client.27137 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.124+0000 7fd437b3ab80 10 client.27137 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:44:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.124+0000 7fd437b3ab80 20 client.27137 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.124+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a66804e7f0 0x55a668050c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.124+0000 7fd437b3ab80 4 client.27137 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:50.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.124+0000 7fd437b3ab80 1 -- 172.21.15.67:0/1538624813 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a667f7e700 con 0x55a66804e7f0 2024-03-30T02:44:50.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.125+0000 7fd421600700 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a66804e7f0 0x55a668050c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.125+0000 7fd421600700 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a66804e7f0 0x55a668050c10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc68c6ee1419617d server_cookie=a334ee76435604b5 in_seq=0 out_seq=0 2024-03-30T02:44:50.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.126+0000 7fd402a00700 10 client.27137 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:44:50.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.126+0000 7fd402a00700 1 -- 172.21.15.67:0/1538624813 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a667f7e700 con 0x55a66804e7f0 2024-03-30T02:44:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.126+0000 7fd402a00700 10 client.27137 handle_command_reply: tid=0 2024-03-30T02:44:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.126+0000 7fd437b3ab80 1 client.27137 shutdown 2024-03-30T02:44:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 -- 172.21.15.67:0/1538624813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6680335d0 msgr2=0x55a66803e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6680335d0 0x55a66803e820 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd4080092b0 tx=0x7fd408003f20 comp rx=0 tx=0).stop 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 -- 172.21.15.67:0/1538624813 shutdown_connections 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a66804e7f0 0x55a668050c10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 0x55a66803c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6680335d0 0x55a66803e820 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/1538624813 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a668033ef0 0x55a66803bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 -- 172.21.15.67:0/1538624813 >> 172.21.15.67:0/1538624813 conn(0x55a667f936a0 msgr2=0x55a6680356c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd402a00700 0 client.27137 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 -- 172.21.15.67:0/1538624813 shutdown_connections 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.127+0000 7fd437b3ab80 1 -- 172.21.15.67:0/1538624813 wait complete. 2024-03-30T02:44:50.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.128+0000 7fd437b3ab80 20 client.27137 trim_cache size 0 max 16384 2024-03-30T02:44:50.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.142+0000 7fd437b3ab80 1 Processor -- start 2024-03-30T02:44:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.142+0000 7fd437b3ab80 1 -- start start 2024-03-30T02:44:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.143+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.143+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.143+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 0x55a6680285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.143+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a668034570 con 0x55a66803edf0 2024-03-30T02:44:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.143+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66803a120 con 0x55a668033ef0 2024-03-30T02:44:50.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.143+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a668028ae0 con 0x55a6680335d0 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36412/0 (socket says 172.21.15.67:36412) 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 -- 172.21.15.67:0/3949699275 learned_addr learned my addr 172.21.15.67:0/3949699275 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 0x55a6680285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd421600700 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 -- 172.21.15.67:0/3949699275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 msgr2=0x55a66803b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 -- 172.21.15.67:0/3949699275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 msgr2=0x55a6680285a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 0x55a6680285a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 -- 172.21.15.67:0/3949699275 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a667f4bea0 con 0x55a6680335d0 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd421600700 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.144+0000 7fd420c00700 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd40800bbd0 tx=0x7fd40800d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5e8b2c488c1f844 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.145+0000 7fd40f000700 1 -- 172.21.15.67:0/3949699275 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408010070 con 0x55a6680335d0 2024-03-30T02:44:50.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.145+0000 7fd40f000700 1 -- 172.21.15.67:0/3949699275 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd408003b40 con 0x55a6680335d0 2024-03-30T02:44:50.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.145+0000 7fd40f000700 1 -- 172.21.15.67:0/3949699275 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408004a50 con 0x55a6680335d0 2024-03-30T02:44:50.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.145+0000 7fd437b3ab80 1 -- 172.21.15.67:0/3949699275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 msgr2=0x55a6680339b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.145+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd40800bbd0 tx=0x7fd40800d040 comp rx=0 tx=0).stop 2024-03-30T02:44:50.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.146+0000 7fd437b3ab80 1 -- 172.21.15.67:0/3949699275 shutdown_connections 2024-03-30T02:44:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.146+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a66803edf0 0x55a6680285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.146+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a6680335d0 0x55a6680339b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.146+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/3949699275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a66803b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.146+0000 7fd437b3ab80 1 -- 172.21.15.67:0/3949699275 >> 172.21.15.67:0/3949699275 conn(0x55a667f936a0 msgr2=0x55a668055570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.146+0000 7fd437b3ab80 1 -- 172.21.15.67:0/3949699275 shutdown_connections 2024-03-30T02:44:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.146+0000 7fd437b3ab80 1 -- 172.21.15.67:0/3949699275 wait complete. 2024-03-30T02:44:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.147+0000 7fd437b3ab80 1 -- start start 2024-03-30T02:44:50.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.147+0000 7fd437b3ab80 1 Processor -- start 2024-03-30T02:44:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.148+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a6680335d0 0x55a668045ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.148+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a668043aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.148+0000 7fd437b3ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 0x55a668043fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.148+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a668044560 con 0x55a6680335d0 2024-03-30T02:44:50.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.148+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6680446d0 con 0x55a668033ef0 2024-03-30T02:44:50.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.148+0000 7fd437b3ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a668044890 con 0x55a66803edf0 2024-03-30T02:44:50.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.148+0000 7fd437b3ab80 10 client.0 _ll_register_callbacks cb 0x55a667f2b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.149+0000 7fd420c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a6680335d0 0x55a668045ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.149+0000 7fd420c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a6680335d0 0x55a668045ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39142/0 (socket says 172.21.15.67:39142) 2024-03-30T02:44:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.149+0000 7fd420c00700 1 -- 172.21.15.67:0/2084405381 learned_addr learned my addr 172.21.15.67:0/2084405381 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.149+0000 7fd421600700 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a668043aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.149+0000 7fd420c00700 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 0x55a668043fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.150+0000 7fd421600700 1 -- 172.21.15.67:0/2084405381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 msgr2=0x55a668043fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.150+0000 7fd421600700 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 0x55a668043fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.150+0000 7fd421600700 1 -- 172.21.15.67:0/2084405381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a6680335d0 msgr2=0x55a668045ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.150+0000 7fd421600700 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a6680335d0 0x55a668045ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.150+0000 7fd421600700 1 -- 172.21.15.67:0/2084405381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a667db3b60 con 0x55a668033ef0 2024-03-30T02:44:50.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.150+0000 7fd420c00700 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 0x55a668043fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.150+0000 7fd421600700 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a668043aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55a668051520 tx=0x7fd410009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1adc7f1598c55b91 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.150+0000 7fd402a00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.151+0000 7fd402a00700 1 -- 172.21.15.67:0/2084405381 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410022070 con 0x55a668033ef0 2024-03-30T02:44:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.151+0000 7fd402a00700 1 -- 172.21.15.67:0/2084405381 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fd410023270 con 0x55a668033ef0 2024-03-30T02:44:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.151+0000 7fd437b3ab80 1 -- 172.21.15.67:0/2084405381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd40800ba40 con 0x55a668033ef0 2024-03-30T02:44:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.151+0000 7fd402a00700 1 -- 172.21.15.67:0/2084405381 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd41000e040 con 0x55a668033ef0 2024-03-30T02:44:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.151+0000 7fd437b3ab80 10 client.27140 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:50.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.151+0000 7fd437b3ab80 1 -- 172.21.15.67:0/2084405381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a667e0bcd0 con 0x55a668033ef0 2024-03-30T02:44:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.152+0000 7fd402a00700 1 -- 172.21.15.67:0/2084405381 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40800ba40 con 0x55a668033ef0 2024-03-30T02:44:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.152+0000 7fd402a00700 1 -- 172.21.15.67:0/2084405381 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd41000b0d0 con 0x55a668033ef0 2024-03-30T02:44:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.152+0000 7fd437b3ab80 10 client.27140 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.152+0000 7fd437b3ab80 10 client.27140 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:44:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.152+0000 7fd437b3ab80 20 client.27140 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.152+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a6680a2a40 0x55a6680a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.152+0000 7fd437b3ab80 4 client.27140 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:50.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.152+0000 7fd437b3ab80 1 -- 172.21.15.67:0/2084405381 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a667dd4cf0 con 0x55a6680a2a40 2024-03-30T02:44:50.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.152+0000 7fd420c00700 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a6680a2a40 0x55a6680a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.153+0000 7fd402a00700 10 client.27140 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.153+0000 7fd420c00700 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a6680a2a40 0x55a6680a4e60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c11ee8c6240ff1d1 server_cookie=dca0025027734f91 in_seq=0 out_seq=0 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.153+0000 7fd402a00700 1 -- 172.21.15.67:0/2084405381 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a667dd4cf0 con 0x55a6680a2a40 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.153+0000 7fd402a00700 10 client.27140 handle_command_reply: tid=0 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.153+0000 7fd437b3ab80 1 client.27140 shutdown 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 -- 172.21.15.67:0/2084405381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 msgr2=0x55a668043aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a668043aa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55a668051520 tx=0x7fd410009760 comp rx=0 tx=0).stop 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 -- 172.21.15.67:0/2084405381 shutdown_connections 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55a6680a2a40 0x55a6680a4e60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a6680335d0 0x55a668045ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a66803edf0 0x55a668043fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd402a00700 0 client.27140 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:44:50.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 --2- 172.21.15.67:0/2084405381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a668033ef0 0x55a668043aa0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 -- 172.21.15.67:0/2084405381 >> 172.21.15.67:0/2084405381 conn(0x55a667f936a0 msgr2=0x55a668028f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 -- 172.21.15.67:0/2084405381 shutdown_connections 2024-03-30T02:44:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 1 -- 172.21.15.67:0/2084405381 wait complete. 2024-03-30T02:44:50.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.154+0000 7fd437b3ab80 20 client.27140 trim_cache size 0 max 16384 2024-03-30T02:44:50.155 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 -- 172.21.15.67:0/2468968370 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4040a13e0 msgr2=0x7fd4040a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4040a13e0 0x7fd4040a38a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd4140a9e80 tx=0x7fd414004560 comp rx=0 tx=0).stop 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 -- 172.21.15.67:0/2468968370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c1998e0 msgr2=0x7fd41c199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c1998e0 0x7fd41c199d60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ab50 tx=0x7fd414007bd0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 -- 172.21.15.67:0/2468968370 shutdown_connections 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd4040a13e0 0x7fd4040a38a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c1998e0 0x7fd41c199d60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd41c0ff440 0x7fd41c195000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 --2- 172.21.15.67:0/2468968370 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd41c1042e0 0x7fd41c195540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.155+0000 7fd424000700 1 -- 172.21.15.67:0/2468968370 >> 172.21.15.67:0/2468968370 conn(0x7fd41c074b20 msgr2=0x7fd41c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.156+0000 7fd424000700 1 -- 172.21.15.67:0/2468968370 shutdown_connections 2024-03-30T02:44:50.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.156+0000 7fd424000700 1 -- 172.21.15.67:0/2468968370 wait complete. 2024-03-30T02:44:50.176 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:44:50.290 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/sysfs.o 2024-03-30T02:44:50.292 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/9p/built-in.a 2024-03-30T02:44:50.298 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/lockd/procfs.o 2024-03-30T02:44:50.350 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/klist.o 2024-03-30T02:44:50.363 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/hostfs/built-in.a 2024-03-30T02:44:50.411 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/fork.o 2024-03-30T02:44:50.433 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/debugfs/inode.o 2024-03-30T02:44:50.449 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:50 smithi067 ceph-mon[37808]: pgmap v238: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-03-30T02:44:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.641+0000 7f134f200700 1 -- 172.21.15.67:0/2567869764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135010f8b0 msgr2=0x7f1350116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.641+0000 7f134f200700 1 --2- 172.21.15.67:0/2567869764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135010f8b0 0x7f1350116270 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1348009990 tx=0x7f1348009ca0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.646+0000 7f134f200700 1 -- 172.21.15.67:0/2567869764 shutdown_connections 2024-03-30T02:44:50.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.646+0000 7f134f200700 1 --2- 172.21.15.67:0/2567869764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f135010f8b0 0x7f1350116270 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.646+0000 7f134f200700 1 --2- 172.21.15.67:0/2567869764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1350108020 0x7f1350108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.646+0000 7f134f200700 1 --2- 172.21.15.67:0/2567869764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13501089d0 0x7f135010f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.646+0000 7f134f200700 1 -- 172.21.15.67:0/2567869764 >> 172.21.15.67:0/2567869764 conn(0x7f135006bc10 msgr2=0x7f135006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.646+0000 7f134f200700 1 -- 172.21.15.67:0/2567869764 shutdown_connections 2024-03-30T02:44:50.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.646+0000 7f134f200700 1 -- 172.21.15.67:0/2567869764 wait complete. 2024-03-30T02:44:50.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.648+0000 7f134f200700 1 Processor -- start 2024-03-30T02:44:50.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.651+0000 7f134f200700 1 -- start start 2024-03-30T02:44:50.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.651+0000 7f134f200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1350108020 0x7f1350108f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.651+0000 7f134f200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f135010f8b0 0x7f13501094d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.652+0000 7f1345400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f135010f8b0 0x7f13501094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.653+0000 7f1345400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f135010f8b0 0x7f13501094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39152/0 (socket says 172.21.15.67:39152) 2024-03-30T02:44:50.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.653+0000 7f134f200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1350109a10 0x7f13501adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.653+0000 7f134de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1350108020 0x7f1350108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.653+0000 7f134de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1350108020 0x7f1350108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36432/0 (socket says 172.21.15.67:36432) 2024-03-30T02:44:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.653+0000 7f134de00700 1 -- 172.21.15.67:0/247579040 learned_addr learned my addr 172.21.15.67:0/247579040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.654+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501ae0e0 con 0x7f135010f8b0 2024-03-30T02:44:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.654+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501ae250 con 0x7f1350109a10 2024-03-30T02:44:50.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.654+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13501ae410 con 0x7f1350108020 2024-03-30T02:44:50.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.655+0000 7f1345400700 1 -- 172.21.15.67:0/247579040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1350108020 msgr2=0x7f1350108f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.655+0000 7f1345400700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1350108020 0x7f1350108f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.655+0000 7f1345400700 1 -- 172.21.15.67:0/247579040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1350109a10 msgr2=0x7f13501adb10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T02:44:50.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.655+0000 7f1345400700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1350109a10 0x7f13501adb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.655+0000 7f1345400700 1 -- 172.21.15.67:0/247579040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13480095f0 con 0x7f135010f8b0 2024-03-30T02:44:50.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.656+0000 7f1345400700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f135010f8b0 0x7f13501094d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f135006a2a0 tx=0x7f133800bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.657+0000 7f1347400700 1 -- 172.21.15.67:0/247579040 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338010680 con 0x7f135010f8b0 2024-03-30T02:44:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.657+0000 7f1347400700 1 -- 172.21.15.67:0/247579040 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1338010ca0 con 0x7f135010f8b0 2024-03-30T02:44:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.657+0000 7f1347400700 1 -- 172.21.15.67:0/247579040 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133800fe40 con 0x7f135010f8b0 2024-03-30T02:44:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.657+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13501ae6f0 con 0x7f135010f8b0 2024-03-30T02:44:50.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.657+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13501aebb0 con 0x7f135010f8b0 2024-03-30T02:44:50.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.668+0000 7f1347400700 1 -- 172.21.15.67:0/247579040 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f133800f4e0 con 0x7f135010f8b0 2024-03-30T02:44:50.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.675+0000 7f1347400700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f13340a1350 0x7f13340a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.676+0000 7f1347400700 1 -- 172.21.15.67:0/247579040 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f13380cc980 con 0x7f135010f8b0 2024-03-30T02:44:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.676+0000 7f134de00700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f13340a1350 0x7f13340a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.677+0000 7f134de00700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f13340a1350 0x7f13340a3810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1340009fa0 tx=0x7f13400095b0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.679+0000 7f1368af4b80 1 Processor -- start 2024-03-30T02:44:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.681+0000 7f1368af4b80 1 -- start start 2024-03-30T02:44:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.681+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x56455998cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.681+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998d0c0 0x564559997a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.681+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x56455999a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.682+0000 7f1368af4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645598e5990 con 0x564559997fc0 2024-03-30T02:44:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.682+0000 7f1368af4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645598e3350 con 0x56455998c7a0 2024-03-30T02:44:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.682+0000 7f1368af4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645598dec40 con 0x56455998d0c0 2024-03-30T02:44:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.682+0000 7f134e800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x56455999a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.682+0000 7f134e800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x56455999a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39156/0 (socket says 172.21.15.67:39156) 2024-03-30T02:44:50.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.682+0000 7f134e800700 1 -- 172.21.15.67:0/4054448139 learned_addr learned my addr 172.21.15.67:0/4054448139 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.682+0000 7f1345400700 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998d0c0 0x564559997a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.683+0000 7f134e800700 1 -- 172.21.15.67:0/4054448139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998d0c0 msgr2=0x564559997a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.683+0000 7f134e800700 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998d0c0 0x564559997a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.683+0000 7f134e800700 1 -- 172.21.15.67:0/4054448139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 msgr2=0x56455998cb80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:50.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.683+0000 7f134de00700 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x56455998cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.683+0000 7f134e800700 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x56455998cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.683+0000 7f134e800700 1 -- 172.21.15.67:0/4054448139 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645597603b0 con 0x564559997fc0 2024-03-30T02:44:50.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.684+0000 7f134de00700 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x56455998cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:50.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.684+0000 7f1345400700 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998d0c0 0x564559997a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:50.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.684+0000 7f134e800700 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x56455999a3b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f13480098e0 tx=0x7f1348003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e72f0b0de6903f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.686+0000 7f133f400700 1 -- 172.21.15.67:0/4054448139 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134801d070 con 0x564559997fc0 2024-03-30T02:44:50.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.686+0000 7f133f400700 1 -- 172.21.15.67:0/4054448139 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f134800be00 con 0x564559997fc0 2024-03-30T02:44:50.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.686+0000 7f133f400700 1 -- 172.21.15.67:0/4054448139 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348003a90 con 0x564559997fc0 2024-03-30T02:44:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.688+0000 7f1368af4b80 1 -- 172.21.15.67:0/4054448139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 msgr2=0x56455999a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.688+0000 7f1368af4b80 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x56455999a3b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f13480098e0 tx=0x7f1348003680 comp rx=0 tx=0).stop 2024-03-30T02:44:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.689+0000 7f1368af4b80 1 -- 172.21.15.67:0/4054448139 shutdown_connections 2024-03-30T02:44:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.689+0000 7f1368af4b80 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x56455999a3b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.689+0000 7f1368af4b80 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x56455998cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.689+0000 7f1368af4b80 1 --2- 172.21.15.67:0/4054448139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998d0c0 0x564559997a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.689+0000 7f1368af4b80 1 -- 172.21.15.67:0/4054448139 >> 172.21.15.67:0/4054448139 conn(0x5645598ec6a0 msgr2=0x564559988c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.689+0000 7f1368af4b80 1 -- 172.21.15.67:0/4054448139 shutdown_connections 2024-03-30T02:44:50.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.689+0000 7f1368af4b80 1 -- 172.21.15.67:0/4054448139 wait complete. 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.690+0000 7f1368af4b80 1 -- start start 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.691+0000 7f1368af4b80 1 Processor -- start 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.693+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.693+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x564559994d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.693+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559995340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.694+0000 7f134de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.694+0000 7f134de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36450/0 (socket says 172.21.15.67:36450) 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.694+0000 7f1345400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x564559994d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.694+0000 7f1345400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x564559994d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:54900/0 (socket says 172.21.15.67:54900) 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.694+0000 7f134de00700 1 -- 172.21.15.67:0/2941465430 learned_addr learned my addr 172.21.15.67:0/2941465430 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.694+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455987ad80 con 0x564559997fc0 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.694+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455987aa20 con 0x56455998d0c0 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.694+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645597626a0 con 0x56455998c7a0 2024-03-30T02:44:50.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.694+0000 7f1368af4b80 10 client.0 _ll_register_callbacks cb 0x564559884470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.695+0000 7f1345400700 1 -- 172.21.15.67:0/2941465430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 msgr2=0x5645599979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.695+0000 7f1345400700 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.695+0000 7f1345400700 1 -- 172.21.15.67:0/2941465430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 msgr2=0x564559995340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.695+0000 7f134e800700 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559995340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.695+0000 7f1345400700 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559995340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.695+0000 7f1345400700 1 -- 172.21.15.67:0/2941465430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1350116ab0 con 0x56455998d0c0 2024-03-30T02:44:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.695+0000 7f134e800700 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559995340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:50.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.696+0000 7f1333e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:50.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.696+0000 7f1345400700 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x564559994d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f13380c9e40 tx=0x7f1338099f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e60637b3ad81dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.697 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr_hurd.o 2024-03-30T02:44:50.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.696+0000 7f1333e00700 1 -- 172.21.15.67:0/2941465430 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380b92e0 con 0x56455998d0c0 2024-03-30T02:44:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.697+0000 7f1333e00700 1 -- 172.21.15.67:0/2941465430 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f133809d6c0 con 0x56455998d0c0 2024-03-30T02:44:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.699+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56455989f380 con 0x56455998d0c0 2024-03-30T02:44:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.699+0000 7f1333e00700 1 -- 172.21.15.67:0/2941465430 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13380a1c80 con 0x56455998d0c0 2024-03-30T02:44:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.699+0000 7f1368af4b80 10 client.27146 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.699+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56455970cb60 con 0x56455998d0c0 2024-03-30T02:44:50.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.700+0000 7f1333e00700 1 -- 172.21.15.67:0/2941465430 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56455989f380 con 0x56455998d0c0 2024-03-30T02:44:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.701+0000 7f1333e00700 1 -- 172.21.15.67:0/2941465430 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f13380a17f0 con 0x56455998d0c0 2024-03-30T02:44:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.701+0000 7f1368af4b80 10 client.27146 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.701+0000 7f1368af4b80 10 client.27146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:44:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.701+0000 7f1368af4b80 20 client.27146 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.701+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5645599b0270 0x5645599b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.701+0000 7f1368af4b80 4 client.27146 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:50.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.701+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645598d7700 con 0x5645599b0270 2024-03-30T02:44:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.702+0000 7f134de00700 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5645599b0270 0x5645599b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.702+0000 7f134de00700 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5645599b0270 0x5645599b2690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4258aeb08f081b2c server_cookie=5543df6f60f602ca in_seq=0 out_seq=0 2024-03-30T02:44:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.702+0000 7f1333e00700 10 client.27146 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:44:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.703+0000 7f1333e00700 1 -- 172.21.15.67:0/2941465430 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5645598d7700 con 0x5645599b0270 2024-03-30T02:44:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.703+0000 7f1333e00700 10 client.27146 handle_command_reply: tid=0 2024-03-30T02:44:50.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.704+0000 7f1368af4b80 1 client.27146 shutdown 2024-03-30T02:44:50.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.704+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 msgr2=0x564559994d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.704+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x564559994d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f13380c9e40 tx=0x7f1338099f90 comp rx=0 tx=0).stop 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.705+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 shutdown_connections 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.705+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x5645599b0270 0x5645599b2690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.705+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559995340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.705+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.705+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2941465430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x564559994d30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.705+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 >> 172.21.15.67:0/2941465430 conn(0x5645598ec6a0 msgr2=0x56455998e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.705+0000 7f1333e00700 0 client.27146 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.705+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 shutdown_connections 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.706+0000 7f1368af4b80 1 -- 172.21.15.67:0/2941465430 wait complete. 2024-03-30T02:44:50.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.708+0000 7f1368af4b80 20 client.27146 trim_cache size 0 max 16384 2024-03-30T02:44:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:50 smithi082 ceph-mon[79176]: pgmap v238: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-03-30T02:44:50.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.765+0000 7f1368af4b80 1 Processor -- start 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.766+0000 7f1368af4b80 1 -- start start 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.770+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x5645599a7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.770+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56455998d0c0 0x5645599977a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.770+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564559997fc0 0x5645599902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.770+0000 7f1368af4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645599956d0 con 0x56455998d0c0 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.770+0000 7f1368af4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645599a79b0 con 0x56455998c7a0 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.770+0000 7f1368af4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564559995bf0 con 0x564559997fc0 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.771+0000 7f134e800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564559997fc0 0x5645599902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.771+0000 7f134e800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564559997fc0 0x5645599902c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36460/0 (socket says 172.21.15.67:36460) 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.771+0000 7f134de00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x5645599a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.771+0000 7f134e800700 1 -- 172.21.15.67:0/2628646297 learned_addr learned my addr 172.21.15.67:0/2628646297 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.771+0000 7f134e800700 1 -- 172.21.15.67:0/2628646297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 msgr2=0x5645599a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.771+0000 7f134e800700 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x5645599a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.771+0000 7f134e800700 1 -- 172.21.15.67:0/2628646297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56455998d0c0 msgr2=0x5645599977a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.772+0000 7f1345400700 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56455998d0c0 0x5645599977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.772+0000 7f134e800700 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56455998d0c0 0x5645599977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.772+0000 7f134e800700 1 -- 172.21.15.67:0/2628646297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645598a4ea0 con 0x564559997fc0 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.772+0000 7f134de00700 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x5645599a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.772+0000 7f1345400700 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56455998d0c0 0x5645599977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.773+0000 7f134e800700 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564559997fc0 0x5645599902c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1348009ce0 tx=0x7f1348005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95b1d665c6209094 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.773+0000 7f133fe00700 1 -- 172.21.15.67:0/2628646297 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134801d040 con 0x564559997fc0 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.773+0000 7f133fe00700 1 -- 172.21.15.67:0/2628646297 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f134800b9d0 con 0x564559997fc0 2024-03-30T02:44:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.773+0000 7f133fe00700 1 -- 172.21.15.67:0/2628646297 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134800f460 con 0x564559997fc0 2024-03-30T02:44:50.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.774+0000 7f1368af4b80 1 -- 172.21.15.67:0/2628646297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564559997fc0 msgr2=0x5645599902c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.774+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564559997fc0 0x5645599902c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1348009ce0 tx=0x7f1348005bc0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.774+0000 7f1368af4b80 1 -- 172.21.15.67:0/2628646297 shutdown_connections 2024-03-30T02:44:50.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.774+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564559997fc0 0x5645599902c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.774+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998c7a0 0x5645599a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.774+0000 7f1368af4b80 1 --2- 172.21.15.67:0/2628646297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56455998d0c0 0x5645599977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.774+0000 7f1368af4b80 1 -- 172.21.15.67:0/2628646297 >> 172.21.15.67:0/2628646297 conn(0x5645598ec6a0 msgr2=0x5645599817f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.775+0000 7f1368af4b80 1 -- 172.21.15.67:0/2628646297 shutdown_connections 2024-03-30T02:44:50.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.775+0000 7f1368af4b80 1 -- 172.21.15.67:0/2628646297 wait complete. 2024-03-30T02:44:50.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.776+0000 7f1368af4b80 1 -- start start 2024-03-30T02:44:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.777+0000 7f1368af4b80 1 Processor -- start 2024-03-30T02:44:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x5645599a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1368af4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559a030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1368af4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645599a5b40 con 0x564559997fc0 2024-03-30T02:44:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1368af4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564559a035e0 con 0x56455998d0c0 2024-03-30T02:44:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1368af4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455999ac30 con 0x56455998c7a0 2024-03-30T02:44:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1368af4b80 10 client.0 _ll_register_callbacks cb 0x564559884470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1345400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x5645599a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1345400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x5645599a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:54926/0 (socket says 172.21.15.67:54926) 2024-03-30T02:44:50.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.780+0000 7f1345400700 1 -- 172.21.15.67:0/3967231605 learned_addr learned my addr 172.21.15.67:0/3967231605 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:50.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.781+0000 7f1345400700 1 -- 172.21.15.67:0/3967231605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 msgr2=0x5645599a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.781+0000 7f134de00700 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.782+0000 7f134e800700 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559a030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.782+0000 7f1345400700 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.782+0000 7f1345400700 1 -- 172.21.15.67:0/3967231605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 msgr2=0x564559a030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:50.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.782+0000 7f1345400700 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559a030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.783+0000 7f1345400700 1 -- 172.21.15.67:0/3967231605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1340016420 con 0x56455998d0c0 2024-03-30T02:44:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.783+0000 7f134de00700 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.783+0000 7f134e800700 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559a030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.783+0000 7f1345400700 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x5645599a5280 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1338006fd0 tx=0x7f133800f8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e0f13f3a793ed7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.783+0000 7f1333e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.783+0000 7f1333e00700 1 -- 172.21.15.67:0/3967231605 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380a1c80 con 0x56455998d0c0 2024-03-30T02:44:50.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.783+0000 7f1333e00700 1 -- 172.21.15.67:0/3967231605 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f133809d6c0 con 0x56455998d0c0 2024-03-30T02:44:50.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.787+0000 7f1368af4b80 1 -- 172.21.15.67:0/3967231605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f134800be00 con 0x56455998d0c0 2024-03-30T02:44:50.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.788+0000 7f1333e00700 1 -- 172.21.15.67:0/3967231605 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1338098d80 con 0x56455998d0c0 2024-03-30T02:44:50.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.789+0000 7f1368af4b80 10 client.27152 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:50.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.789+0000 7f1368af4b80 1 -- 172.21.15.67:0/3967231605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5645597603b0 con 0x56455998d0c0 2024-03-30T02:44:50.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.789+0000 7f1333e00700 1 -- 172.21.15.67:0/3967231605 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1338063070 con 0x56455998d0c0 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.793+0000 7f1333e00700 1 -- 172.21.15.67:0/3967231605 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f13380107e0 con 0x56455998d0c0 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.793+0000 7f1368af4b80 10 client.27152 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.793+0000 7f1368af4b80 10 client.27152 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.793+0000 7f1368af4b80 20 client.27152 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.793+0000 7f1368af4b80 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x56455999be70 0x56455999e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.793+0000 7f1368af4b80 4 client.27152 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.793+0000 7f1368af4b80 1 -- 172.21.15.67:0/3967231605 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56455972dcf0 con 0x56455999be70 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.793+0000 7f134de00700 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x56455999be70 0x56455999e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.794+0000 7f1333e00700 10 client.27152 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.794+0000 7f134de00700 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x56455999be70 0x56455999e260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46c2703773a225b9 server_cookie=6b386af4dd0c000c in_seq=0 out_seq=0 2024-03-30T02:44:50.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1333e00700 1 -- 172.21.15.67:0/3967231605 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56455972dcf0 con 0x56455999be70 2024-03-30T02:44:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1333e00700 10 client.27152 handle_command_reply: tid=0 2024-03-30T02:44:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 client.27152 shutdown 2024-03-30T02:44:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 -- 172.21.15.67:0/3967231605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 msgr2=0x5645599a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x5645599a5280 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1338006fd0 tx=0x7f133800f8e0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 -- 172.21.15.67:0/3967231605 shutdown_connections 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x56455999be70 0x56455999e260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x564559997fc0 0x564559a030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1333e00700 0 client.27152 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56455998c7a0 0x5645599a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 --2- 172.21.15.67:0/3967231605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56455998d0c0 0x5645599a5280 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 -- 172.21.15.67:0/3967231605 >> 172.21.15.67:0/3967231605 conn(0x5645598ec6a0 msgr2=0x5645599902c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.795+0000 7f1368af4b80 1 -- 172.21.15.67:0/3967231605 shutdown_connections 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.796+0000 7f1368af4b80 1 -- 172.21.15.67:0/3967231605 wait complete. 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.797+0000 7f1368af4b80 20 client.27152 trim_cache size 0 max 16384 2024-03-30T02:44:50.796 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:50.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.801+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f13340a1350 msgr2=0x7f13340a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.801+0000 7f134f200700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f13340a1350 0x7f13340a3810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1340009fa0 tx=0x7f13400095b0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.801+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f135010f8b0 msgr2=0x7f13501094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:50.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.801+0000 7f134f200700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f135010f8b0 0x7f13501094d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f135006a2a0 tx=0x7f133800bae0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.801+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 shutdown_connections 2024-03-30T02:44:50.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.801+0000 7f134f200700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f13340a1350 0x7f13340a3810 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.801+0000 7f134f200700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1350109a10 0x7f13501adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.801+0000 7f134f200700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1350108020 0x7f1350108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.801+0000 7f134f200700 1 --2- 172.21.15.67:0/247579040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f135010f8b0 0x7f13501094d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:50.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.802+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 >> 172.21.15.67:0/247579040 conn(0x7f135006bc10 msgr2=0x7f1350071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:50.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.802+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 shutdown_connections 2024-03-30T02:44:50.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:50.806+0000 7f134f200700 1 -- 172.21.15.67:0/247579040 wait complete. 2024-03-30T02:44:50.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:44:50.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:50 smithi170 ceph-mon[78798]: pgmap v238: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-03-30T02:44:51.143 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kobject.o 2024-03-30T02:44:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.242+0000 7fe5fd000700 1 -- 172.21.15.67:0/3210860809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5f8101900 msgr2=0x7fe5f8105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.242+0000 7fe5fd000700 1 --2- 172.21.15.67:0/3210860809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5f8101900 0x7fe5f8105940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009990 tx=0x7fe5e4009ca0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.242+0000 7fe5fd000700 1 -- 172.21.15.67:0/3210860809 shutdown_connections 2024-03-30T02:44:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.242+0000 7fe5fd000700 1 --2- 172.21.15.67:0/3210860809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5f8100f50 0x7fe5f8101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.242+0000 7fe5fd000700 1 --2- 172.21.15.67:0/3210860809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f8106070 0x7fe5f810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.242+0000 7fe5fd000700 1 --2- 172.21.15.67:0/3210860809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5f8101900 0x7fe5f8105940 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.242+0000 7fe5fd000700 1 -- 172.21.15.67:0/3210860809 >> 172.21.15.67:0/3210860809 conn(0x7fe5f80fc7f0 msgr2=0x7fe5f80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.242+0000 7fe5fd000700 1 -- 172.21.15.67:0/3210860809 shutdown_connections 2024-03-30T02:44:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.243+0000 7fe5fd000700 1 -- 172.21.15.67:0/3210860809 wait complete. 2024-03-30T02:44:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.244+0000 7fe5fd000700 1 Processor -- start 2024-03-30T02:44:51.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.244+0000 7fe5fd000700 1 -- start start 2024-03-30T02:44:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.245+0000 7fe5fd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f8100f50 0x7fe5f8195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.245+0000 7fe5fd000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5f8106070 0x7fe5f8195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.245+0000 7fe5fd000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5f8199990 0x7fe5f8199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.245+0000 7fe5fd000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819a410 con 0x7fe5f8199990 2024-03-30T02:44:51.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.245+0000 7fe5fd000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819a580 con 0x7fe5f8100f50 2024-03-30T02:44:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.245+0000 7fe5fd000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819a740 con 0x7fe5f8106070 2024-03-30T02:44:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5f8106070 0x7fe5f8195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f8100f50 0x7fe5f8195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5f8106070 0x7fe5f8195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36486/0 (socket says 172.21.15.67:36486) 2024-03-30T02:44:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f5400700 1 -- 172.21.15.67:0/2330940511 learned_addr learned my addr 172.21.15.67:0/2330940511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f6800700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5f8199990 0x7fe5f8199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f5e00700 1 -- 172.21.15.67:0/2330940511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5f8106070 msgr2=0x7fe5f8195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5f8106070 0x7fe5f8195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f5e00700 1 -- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5f8199990 msgr2=0x7fe5f8199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5f8199990 0x7fe5f8199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f5e00700 1 -- 172.21.15.67:0/2330940511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8009520 con 0x7fe5f8100f50 2024-03-30T02:44:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.246+0000 7fe5f6800700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5f8199990 0x7fe5f8199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:51.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.247+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f8100f50 0x7fe5f8195100 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800e9f0 tx=0x7fe5e800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.247+0000 7fe5eea00700 1 -- 172.21.15.67:0/2330940511 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800cb70 con 0x7fe5f8100f50 2024-03-30T02:44:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.247+0000 7fe5fd000700 1 -- 172.21.15.67:0/2330940511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f0009c40 con 0x7fe5f8100f50 2024-03-30T02:44:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.248+0000 7fe5eea00700 1 -- 172.21.15.67:0/2330940511 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e80044e0 con 0x7fe5f8100f50 2024-03-30T02:44:51.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.248+0000 7fe5eea00700 1 -- 172.21.15.67:0/2330940511 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8003d00 con 0x7fe5f8100f50 2024-03-30T02:44:51.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.248+0000 7fe5fd000700 1 -- 172.21.15.67:0/2330940511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e40095f0 con 0x7fe5f8100f50 2024-03-30T02:44:51.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.253+0000 7fe5eea00700 1 -- 172.21.15.67:0/2330940511 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe5e8004020 con 0x7fe5f8100f50 2024-03-30T02:44:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.255+0000 7fe610a7bb80 1 Processor -- start 2024-03-30T02:44:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.255+0000 7fe5eea00700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5dc0a1310 0x7fe5dc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.256+0000 7fe5eea00700 1 -- 172.21.15.67:0/2330940511 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fe5e8014070 con 0x7fe5f8100f50 2024-03-30T02:44:51.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.256+0000 7fe5f6800700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5dc0a1310 0x7fe5dc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.256+0000 7fe610a7bb80 1 -- start start 2024-03-30T02:44:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.257+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777be40 0x56030777c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.257+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777c760 0x5603077870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.258+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x560307789a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.258+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603076cc990 con 0x56030777c760 2024-03-30T02:44:51.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.258+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603076ca350 con 0x560307787610 2024-03-30T02:44:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.258+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603076c5c40 con 0x56030777be40 2024-03-30T02:44:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.258+0000 7fe5f5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777be40 0x56030777c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.258+0000 7fe5f5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777be40 0x56030777c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36496/0 (socket says 172.21.15.67:36496) 2024-03-30T02:44:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.258+0000 7fe5f5e00700 1 -- 172.21.15.67:0/3605360462 learned_addr learned my addr 172.21.15.67:0/3605360462 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.259+0000 7fe5f5e00700 1 -- 172.21.15.67:0/3605360462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 msgr2=0x560307789a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:44:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.259+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/3605360462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x560307789a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.259+0000 7fe5f5e00700 1 -- 172.21.15.67:0/3605360462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777c760 msgr2=0x5603077870d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.259+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/3605360462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777c760 0x5603077870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.259+0000 7fe5f5e00700 1 -- 172.21.15.67:0/3605360462 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603075473b0 con 0x56030777be40 2024-03-30T02:44:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.259+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/3605360462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777be40 0x56030777c220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8096bd0 tx=0x7fe5e8099ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9c2066f7b63b676 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.260+0000 7fe5e3400700 1 -- 172.21.15.67:0/3605360462 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e809a160 con 0x56030777be40 2024-03-30T02:44:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.260+0000 7fe5e3400700 1 -- 172.21.15.67:0/3605360462 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e809df90 con 0x56030777be40 2024-03-30T02:44:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.260+0000 7fe5e3400700 1 -- 172.21.15.67:0/3605360462 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e801fda0 con 0x56030777be40 2024-03-30T02:44:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3605360462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777be40 msgr2=0x56030777c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3605360462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777be40 0x56030777c220 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8096bd0 tx=0x7fe5e8099ca0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe5f6800700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5dc0a1310 0x7fe5dc0a37d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00094f0 tx=0x7fe5f0009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3605360462 shutdown_connections 2024-03-30T02:44:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3605360462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x560307789a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3605360462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777be40 0x56030777c220 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3605360462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777c760 0x5603077870d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3605360462 >> 172.21.15.67:0/3605360462 conn(0x5603076d36a0 msgr2=0x56030776fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3605360462 shutdown_connections 2024-03-30T02:44:51.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.261+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3605360462 wait complete. 2024-03-30T02:44:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.262+0000 7fe610a7bb80 1 -- start start 2024-03-30T02:44:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.262+0000 7fe610a7bb80 1 Processor -- start 2024-03-30T02:44:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.263+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x5603077870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.263+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.263+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x560307784c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.263+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560307661d80 con 0x56030777be40 2024-03-30T02:44:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.263+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560307661a20 con 0x560307787610 2024-03-30T02:44:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.263+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603075496a0 con 0x56030777c760 2024-03-30T02:44:51.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.263+0000 7fe610a7bb80 10 client.0 _ll_register_callbacks cb 0x56030766b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.264+0000 7fe5f5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.265+0000 7fe5f5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36508/0 (socket says 172.21.15.67:36508) 2024-03-30T02:44:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.265+0000 7fe5f5400700 1 -- 172.21.15.67:0/2544046610 learned_addr learned my addr 172.21.15.67:0/2544046610 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.265+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x5603077870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.265+0000 7fe5f6800700 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x560307784c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.265+0000 7fe5f5e00700 1 -- 172.21.15.67:0/2544046610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 msgr2=0x560307784510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.265+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.265+0000 7fe5f5e00700 1 -- 172.21.15.67:0/2544046610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 msgr2=0x560307784c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.265+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x560307784c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.265+0000 7fe5f5e00700 1 -- 172.21.15.67:0/2544046610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e40095f0 con 0x56030777be40 2024-03-30T02:44:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.266+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x5603077870d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8096bd0 tx=0x7fe5e80a26d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a74c93c48bd3250 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.266+0000 7fe5d7e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:44:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.267+0000 7fe5d7e00700 1 -- 172.21.15.67:0/2544046610 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e801fda0 con 0x56030777be40 2024-03-30T02:44:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.267+0000 7fe5d7e00700 1 -- 172.21.15.67:0/2544046610 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fe5e809a3c0 con 0x56030777be40 2024-03-30T02:44:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.267+0000 7fe610a7bb80 1 -- 172.21.15.67:0/2544046610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560307686380 con 0x56030777be40 2024-03-30T02:44:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.267+0000 7fe5d7e00700 1 -- 172.21.15.67:0/2544046610 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5e801ff00 con 0x56030777be40 2024-03-30T02:44:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.267+0000 7fe610a7bb80 10 client.17055 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:51.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.267+0000 7fe610a7bb80 1 -- 172.21.15.67:0/2544046610 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5603074f3b60 con 0x56030777be40 2024-03-30T02:44:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.268+0000 7fe5d7e00700 1 -- 172.21.15.67:0/2544046610 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e80a6080 con 0x56030777be40 2024-03-30T02:44:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.268+0000 7fe5d7e00700 1 -- 172.21.15.67:0/2544046610 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe5e80a21c0 con 0x56030777be40 2024-03-30T02:44:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.268+0000 7fe610a7bb80 10 client.17055 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.268+0000 7fe610a7bb80 10 client.17055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:44:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.268+0000 7fe610a7bb80 20 client.17055 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.268+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5603077a7ef0 0x5603077aa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.268+0000 7fe610a7bb80 4 client.17055 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.268+0000 7fe610a7bb80 1 -- 172.21.15.67:0/2544046610 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603076be700 con 0x5603077a7ef0 2024-03-30T02:44:51.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.268+0000 7fe5f5400700 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5603077a7ef0 0x5603077aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.269+0000 7fe5f5400700 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5603077a7ef0 0x5603077aa310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa00de5696094413 server_cookie=f8bd244f3044fec3 in_seq=0 out_seq=0 2024-03-30T02:44:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.269+0000 7fe5d7e00700 10 client.17055 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:44:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.270+0000 7fe5d7e00700 1 -- 172.21.15.67:0/2544046610 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5603076be700 con 0x5603077a7ef0 2024-03-30T02:44:51.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.270+0000 7fe5d7e00700 10 client.17055 handle_command_reply: tid=0 2024-03-30T02:44:51.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.271+0000 7fe610a7bb80 1 client.17055 shutdown 2024-03-30T02:44:51.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.271+0000 7fe610a7bb80 1 -- 172.21.15.67:0/2544046610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 msgr2=0x5603077870d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.271+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x5603077870d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8096bd0 tx=0x7fe5e80a26d0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.271+0000 7fe610a7bb80 1 -- 172.21.15.67:0/2544046610 shutdown_connections 2024-03-30T02:44:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.272+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5603077a7ef0 0x5603077aa310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.272+0000 7fe5d7e00700 0 client.17055 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:44:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.272+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x5603077870d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.272+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x560307784c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.272+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/2544046610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.272+0000 7fe610a7bb80 1 -- 172.21.15.67:0/2544046610 >> 172.21.15.67:0/2544046610 conn(0x5603076d36a0 msgr2=0x56030777ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.272+0000 7fe610a7bb80 1 -- 172.21.15.67:0/2544046610 shutdown_connections 2024-03-30T02:44:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.272+0000 7fe610a7bb80 1 -- 172.21.15.67:0/2544046610 wait complete. 2024-03-30T02:44:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.272+0000 7fe610a7bb80 20 client.17055 trim_cache size 0 max 16384 2024-03-30T02:44:51.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.301+0000 7fe610a7bb80 1 Processor -- start 2024-03-30T02:44:51.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.301+0000 7fe610a7bb80 1 -- start start 2024-03-30T02:44:51.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.301+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x560307783dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.301 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.302+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.302+0000 7fe5f5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.302+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x56030778e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.302+0000 7fe5f5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36524/0 (socket says 172.21.15.67:36524) 2024-03-30T02:44:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.302+0000 7fe5f5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x560307783dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.302+0000 7fe5f5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x560307783dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39220/0 (socket says 172.21.15.67:39220) 2024-03-30T02:44:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.302+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560307782ad0 con 0x56030777be40 2024-03-30T02:44:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.302+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56030778e820 con 0x560307787610 2024-03-30T02:44:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.302+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603077685a0 con 0x56030777c760 2024-03-30T02:44:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.303+0000 7fe5f5400700 1 -- 172.21.15.67:0/1921139111 learned_addr learned my addr 172.21.15.67:0/1921139111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.303+0000 7fe5f6800700 1 --2- 172.21.15.67:0/1921139111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x56030778e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.303+0000 7fe5f5400700 1 -- 172.21.15.67:0/1921139111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 msgr2=0x56030778e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.303+0000 7fe5f5400700 1 --2- 172.21.15.67:0/1921139111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x56030778e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.303+0000 7fe5f5400700 1 -- 172.21.15.67:0/1921139111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 msgr2=0x560307783dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.303+0000 7fe5f5400700 1 --2- 172.21.15.67:0/1921139111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x560307783dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.303+0000 7fe5f5400700 1 -- 172.21.15.67:0/1921139111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56030768bea0 con 0x56030777c760 2024-03-30T02:44:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.304+0000 7fe5f5400700 1 --2- 172.21.15.67:0/1921139111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4005b40 tx=0x7fe5e400b280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf5f12ef182f2b49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.304+0000 7fe5e3e00700 1 -- 172.21.15.67:0/1921139111 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e401d070 con 0x56030777c760 2024-03-30T02:44:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.304+0000 7fe5e3e00700 1 -- 172.21.15.67:0/1921139111 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e401e040 con 0x56030777c760 2024-03-30T02:44:51.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.304+0000 7fe5e3e00700 1 -- 172.21.15.67:0/1921139111 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e400b950 con 0x56030777c760 2024-03-30T02:44:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.305+0000 7fe610a7bb80 1 -- 172.21.15.67:0/1921139111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 msgr2=0x560307784c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.305+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/1921139111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4005b40 tx=0x7fe5e400b280 comp rx=0 tx=0).stop 2024-03-30T02:44:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.305+0000 7fe610a7bb80 1 -- 172.21.15.67:0/1921139111 shutdown_connections 2024-03-30T02:44:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.305+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/1921139111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x560307783dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.305+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/1921139111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560307787610 0x56030778e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.305+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/1921139111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56030777c760 0x560307784c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.305+0000 7fe610a7bb80 1 -- 172.21.15.67:0/1921139111 >> 172.21.15.67:0/1921139111 conn(0x5603076d36a0 msgr2=0x5603077aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.306+0000 7fe610a7bb80 1 -- 172.21.15.67:0/1921139111 shutdown_connections 2024-03-30T02:44:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.306+0000 7fe610a7bb80 1 -- 172.21.15.67:0/1921139111 wait complete. 2024-03-30T02:44:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.307+0000 7fe610a7bb80 1 -- start start 2024-03-30T02:44:51.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.307+0000 7fe610a7bb80 1 Processor -- start 2024-03-30T02:44:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.309+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x56030778e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.309+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56030777c760 0x56030778c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.309+0000 7fe610a7bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560307787610 0x56030778ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.309+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56030778d160 con 0x56030777be40 2024-03-30T02:44:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.309+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56030778d2d0 con 0x56030777c760 2024-03-30T02:44:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.309+0000 7fe610a7bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603077fbcf0 con 0x560307787610 2024-03-30T02:44:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.309+0000 7fe610a7bb80 10 client.0 _ll_register_callbacks cb 0x56030766b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.310+0000 7fe5f5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56030777c760 0x56030778c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.310+0000 7fe5f5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56030777c760 0x56030778c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55004/0 (socket says 172.21.15.67:55004) 2024-03-30T02:44:51.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.310+0000 7fe5f5400700 1 -- 172.21.15.67:0/3273341355 learned_addr learned my addr 172.21.15.67:0/3273341355 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.310+0000 7fe5f6800700 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560307787610 0x56030778ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.310+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x56030778e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.311+0000 7fe5f5400700 1 -- 172.21.15.67:0/3273341355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560307787610 msgr2=0x56030778ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.311+0000 7fe5f5400700 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560307787610 0x56030778ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.311+0000 7fe5f5400700 1 -- 172.21.15.67:0/3273341355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 msgr2=0x56030778e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.311+0000 7fe5f5400700 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x56030778e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.311+0000 7fe5f5400700 1 -- 172.21.15.67:0/3273341355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f810e460 con 0x56030777c760 2024-03-30T02:44:51.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.311+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x56030778e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:51.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.312+0000 7fe5f5400700 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56030777c760 0x56030778c510 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4006010 tx=0x7fe5e400b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510a531ab583eee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.312+0000 7fe5d7e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.313+0000 7fe5d7e00700 1 -- 172.21.15.67:0/3273341355 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e401d040 con 0x56030777c760 2024-03-30T02:44:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.313+0000 7fe5d7e00700 1 -- 172.21.15.67:0/3273341355 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fe5e4005400 con 0x56030777c760 2024-03-30T02:44:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.313+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3273341355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5e809df90 con 0x56030777c760 2024-03-30T02:44:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.313+0000 7fe5d7e00700 1 -- 172.21.15.67:0/3273341355 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5e40273c0 con 0x56030777c760 2024-03-30T02:44:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.313+0000 7fe610a7bb80 10 client.27167 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:51.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.313+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3273341355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56030754bcd0 con 0x56030777c760 2024-03-30T02:44:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.313+0000 7fe5d7e00700 1 -- 172.21.15.67:0/3273341355 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5e809df90 con 0x56030777c760 2024-03-30T02:44:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.314+0000 7fe5d7e00700 1 -- 172.21.15.67:0/3273341355 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe5e403d070 con 0x56030777c760 2024-03-30T02:44:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.314+0000 7fe610a7bb80 10 client.27167 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.314+0000 7fe610a7bb80 10 client.27167 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:44:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.314+0000 7fe610a7bb80 20 client.27167 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:51.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.314+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5603077a7ef0 0x5603077fee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.314+0000 7fe610a7bb80 4 client.27167 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.314+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3273341355 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560307514cf0 con 0x5603077a7ef0 2024-03-30T02:44:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.314+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5603077a7ef0 0x5603077fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.315+0000 7fe5f5e00700 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5603077a7ef0 0x5603077fee00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39f0f9a28f670f34 server_cookie=4172b1999c0a2c9a in_seq=0 out_seq=0 2024-03-30T02:44:51.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.315+0000 7fe5d7e00700 10 client.27167 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.316+0000 7fe5d7e00700 1 -- 172.21.15.67:0/3273341355 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560307514cf0 con 0x5603077a7ef0 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.316+0000 7fe5d7e00700 10 client.27167 handle_command_reply: tid=0 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.316+0000 7fe610a7bb80 1 client.27167 shutdown 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.316+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3273341355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56030777c760 msgr2=0x56030778c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.316+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56030777c760 0x56030778c510 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4006010 tx=0x7fe5e400b640 comp rx=0 tx=0).stop 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.316+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3273341355 shutdown_connections 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.316+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5603077a7ef0 0x5603077fee00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.317+0000 7fe5d7e00700 0 client.27167 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.317+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56030777be40 0x56030778e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.317+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560307787610 0x56030778ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.317+0000 7fe610a7bb80 1 --2- 172.21.15.67:0/3273341355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56030777c760 0x56030778c510 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.317+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3273341355 >> 172.21.15.67:0/3273341355 conn(0x5603076d36a0 msgr2=0x56030777f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.317+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3273341355 shutdown_connections 2024-03-30T02:44:51.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.317+0000 7fe610a7bb80 1 -- 172.21.15.67:0/3273341355 wait complete. 2024-03-30T02:44:51.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.317+0000 7fe610a7bb80 20 client.27167 trim_cache size 0 max 16384 2024-03-30T02:44:51.317 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.321+0000 7fe5fd000700 1 -- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5dc0a1310 msgr2=0x7fe5dc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.321+0000 7fe5fd000700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5dc0a1310 0x7fe5dc0a37d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00094f0 tx=0x7fe5f0009920 comp rx=0 tx=0).stop 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.321+0000 7fe5fd000700 1 -- 172.21.15.67:0/2330940511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f8100f50 msgr2=0x7fe5f8195100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.321+0000 7fe5fd000700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f8100f50 0x7fe5f8195100 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800e9f0 tx=0x7fe5e800ed00 comp rx=0 tx=0).stop 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.322+0000 7fe5fd000700 1 -- 172.21.15.67:0/2330940511 shutdown_connections 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.322+0000 7fe5fd000700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5dc0a1310 0x7fe5dc0a37d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.322+0000 7fe5fd000700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe5f8199990 0x7fe5f8199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.322+0000 7fe5fd000700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f8100f50 0x7fe5f8195100 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.322+0000 7fe5fd000700 1 --2- 172.21.15.67:0/2330940511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5f8106070 0x7fe5f8195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.322+0000 7fe5fd000700 1 -- 172.21.15.67:0/2330940511 >> 172.21.15.67:0/2330940511 conn(0x7fe5f80fc7f0 msgr2=0x7fe5f80fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.322+0000 7fe5fd000700 1 -- 172.21.15.67:0/2330940511 shutdown_connections 2024-03-30T02:44:51.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.322+0000 7fe5fd000700 1 -- 172.21.15.67:0/2330940511 wait complete. 2024-03-30T02:44:51.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:44:51.364 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_events.o 2024-03-30T02:44:51.579 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/lockd/built-in.a 2024-03-30T02:44:51.582 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/ring_buffer.o 2024-03-30T02:44:51.585 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/fs_context.o 2024-03-30T02:44:51.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.630+0000 7f4a0f400700 1 -- 172.21.15.67:0/3468100605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 msgr2=0x7f4a0811c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.630+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3468100605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 0x7f4a0811c660 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009990 tx=0x7f49f4009ca0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.631+0000 7f4a0f400700 1 -- 172.21.15.67:0/3468100605 shutdown_connections 2024-03-30T02:44:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.631+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3468100605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 0x7f4a0811c660 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.631+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3468100605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806c580 0x7f4a0806c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.631+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3468100605 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0806cea0 0x7f4a08119d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.631+0000 7f4a0f400700 1 -- 172.21.15.67:0/3468100605 >> 172.21.15.67:0/3468100605 conn(0x7f4a0806ba00 msgr2=0x7f4a08109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.631+0000 7f4a0f400700 1 -- 172.21.15.67:0/3468100605 shutdown_connections 2024-03-30T02:44:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.632+0000 7f4a0f400700 1 -- 172.21.15.67:0/3468100605 wait complete. 2024-03-30T02:44:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.633+0000 7f4a0f400700 1 Processor -- start 2024-03-30T02:44:51.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.634+0000 7f4a0f400700 1 -- start start 2024-03-30T02:44:51.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.635+0000 7f4a0f400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806c580 0x7f4a081abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.635+0000 7f4a0f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0806cea0 0x7f4a081ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.635+0000 7f4a0f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 0x7f4a081a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.636+0000 7f4a0f400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a081a6330 con 0x7f4a0806cea0 2024-03-30T02:44:51.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.636+0000 7f4a0f400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a081a64a0 con 0x7f4a0811a270 2024-03-30T02:44:51.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.636+0000 7f4a0f400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a081a6660 con 0x7f4a0806c580 2024-03-30T02:44:51.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.636+0000 7f4a0d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 0x7f4a081a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.636+0000 7f4a0d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 0x7f4a081a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55026/0 (socket says 172.21.15.67:55026) 2024-03-30T02:44:51.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.636+0000 7f4a0d400700 1 -- 172.21.15.67:0/3291841390 learned_addr learned my addr 172.21.15.67:0/3291841390 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.636+0000 7f4a0d400700 1 -- 172.21.15.67:0/3291841390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806c580 msgr2=0x7f4a081abac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:44:51.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.636+0000 7f4a0d400700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806c580 0x7f4a081abac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.636+0000 7f4a0d400700 1 -- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0806cea0 msgr2=0x7f4a081ac000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:44:51.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.637+0000 7f4a07e00700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0806cea0 0x7f4a081ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.637+0000 7f4a0d400700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0806cea0 0x7f4a081ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.637+0000 7f4a0d400700 1 -- 172.21.15.67:0/3291841390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f40095f0 con 0x7f4a0811a270 2024-03-30T02:44:51.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.637+0000 7f4a07e00700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0806cea0 0x7f4a081ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:51.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.637+0000 7f4a0d400700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 0x7f4a081a5b60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f49f4003730 tx=0x7f49f4003fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.638+0000 7f4a05600700 1 -- 172.21.15.67:0/3291841390 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f401d070 con 0x7f4a0811a270 2024-03-30T02:44:51.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.638+0000 7f4a05600700 1 -- 172.21.15.67:0/3291841390 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f4004380 con 0x7f4a0811a270 2024-03-30T02:44:51.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.638+0000 7f4a05600700 1 -- 172.21.15.67:0/3291841390 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f40054d0 con 0x7f4a0811a270 2024-03-30T02:44:51.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.638+0000 7f4a0f400700 1 -- 172.21.15.67:0/3291841390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a081a68e0 con 0x7f4a0811a270 2024-03-30T02:44:51.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.638+0000 7f4a0f400700 1 -- 172.21.15.67:0/3291841390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a081b2a40 con 0x7f4a0811a270 2024-03-30T02:44:51.645 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/debugfs/file.o 2024-03-30T02:44:51.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.649+0000 7f4a05600700 1 -- 172.21.15.67:0/3291841390 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f49f4004830 con 0x7f4a0811a270 2024-03-30T02:44:51.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.649+0000 7f4a22e78b80 1 Processor -- start 2024-03-30T02:44:51.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.650+0000 7f4a22e78b80 1 -- start start 2024-03-30T02:44:51.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.651+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b417a0 0x563e57b41b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.651+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b420c0 0x563e57b4ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.652+0000 7f4a0ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b420c0 0x563e57b4ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.652+0000 7f4a0ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b420c0 0x563e57b4ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55030/0 (socket says 172.21.15.67:55030) 2024-03-30T02:44:51.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.652+0000 7f4a07e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b417a0 0x563e57b41b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.652+0000 7f4a07e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b417a0 0x563e57b41b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39264/0 (socket says 172.21.15.67:39264) 2024-03-30T02:44:51.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.652+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.652+0000 7f4a0ca00700 1 -- 172.21.15.67:0/1198412463 learned_addr learned my addr 172.21.15.67:0/1198412463 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.652+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1198412463 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57a9a990 con 0x563e57b417a0 2024-03-30T02:44:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.652+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1198412463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57a98350 con 0x563e57b420c0 2024-03-30T02:44:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.652+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1198412463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57a93c40 con 0x563e57b4cfc0 2024-03-30T02:44:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f4a0ca00700 1 -- 172.21.15.67:0/1198412463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 msgr2=0x563e57b4f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f4a07e00700 1 --2- 172.21.15.67:0/1198412463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/1198412463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f4a0ca00700 1 -- 172.21.15.67:0/1198412463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b417a0 msgr2=0x563e57b41b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/1198412463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b417a0 0x563e57b41b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f4a0ca00700 1 -- 172.21.15.67:0/1198412463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e579153b0 con 0x563e57b420c0 2024-03-30T02:44:51.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f4a07e00700 1 --2- 172.21.15.67:0/1198412463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/1198412463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b420c0 0x563e57b4ca80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009c60 tx=0x7f4a00009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8612cc8ee01e61c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f49fdc00700 1 -- 172.21.15.67:0/1198412463 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00004bb0 con 0x563e57b420c0 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f49fdc00700 1 -- 172.21.15.67:0/1198412463 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a0000f040 con 0x563e57b420c0 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.653+0000 7f49fdc00700 1 -- 172.21.15.67:0/1198412463 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00013650 con 0x563e57b420c0 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.655+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1198412463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b420c0 msgr2=0x563e57b4ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.655+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1198412463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b420c0 0x563e57b4ca80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009c60 tx=0x7f4a00009f70 comp rx=0 tx=0).stop 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.655+0000 7f4a05600700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f49ec0a1350 0x7f49ec0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.655+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1198412463 shutdown_connections 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.655+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1198412463 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b417a0 0x563e57b41b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.655+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1198412463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.655+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1198412463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b420c0 0x563e57b4ca80 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009c60 tx=0x7f4a00009f70 comp rx=0 tx=0).stop 2024-03-30T02:44:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.655+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1198412463 >> 172.21.15.67:0/1198412463 conn(0x563e57aa16a0 msgr2=0x563e57b3dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.655+0000 7f4a05600700 1 -- 172.21.15.67:0/3291841390 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f49f40cde00 con 0x7f4a0811a270 2024-03-30T02:44:51.655 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfsroot.o 2024-03-30T02:44:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.656+0000 7f4a07e00700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f49ec0a1350 0x7f49ec0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.657+0000 7f4a07e00700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f49ec0a1350 0x7f49ec0a3810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a081a72d0 tx=0x7f49f800b7e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.658+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1198412463 shutdown_connections 2024-03-30T02:44:51.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.658+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1198412463 wait complete. 2024-03-30T02:44:51.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.659+0000 7f4a22e78b80 1 -- start start 2024-03-30T02:44:51.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.659+0000 7f4a22e78b80 1 Processor -- start 2024-03-30T02:44:51.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.660+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b49eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4a930 0x563e57b58cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a0ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b49eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a07e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a07e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36570/0 (socket says 172.21.15.67:36570) 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a0ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b49eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55046/0 (socket says 172.21.15.67:55046) 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a0ca00700 1 -- 172.21.15.67:0/3686043365 learned_addr learned my addr 172.21.15.67:0/3686043365 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57a2fd80 con 0x563e57b4a930 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57a2fa20 con 0x563e57b417a0 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e579176a0 con 0x563e57b4cfc0 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.661+0000 7f4a22e78b80 10 client.0 _ll_register_callbacks cb 0x563e57a39470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:51.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.662+0000 7f4a0d400700 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4a930 0x563e57b58cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.662+0000 7f4a07e00700 1 -- 172.21.15.67:0/3686043365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 msgr2=0x563e57b49eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.662+0000 7f4a07e00700 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b49eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.662+0000 7f4a07e00700 1 -- 172.21.15.67:0/3686043365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4a930 msgr2=0x563e57b58cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.662+0000 7f4a07e00700 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4a930 0x563e57b58cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.662+0000 7f4a07e00700 1 -- 172.21.15.67:0/3686043365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a000098c0 con 0x563e57b4cfc0 2024-03-30T02:44:51.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.662+0000 7f4a0d400700 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4a930 0x563e57b58cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:51.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.663+0000 7f4a07e00700 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4a3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49f8010fd0 tx=0x7f49f80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e4de395fdfae247 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.663+0000 7f49f2000700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:51.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.664+0000 7f49f2000700 1 -- 172.21.15.67:0/3686043365 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f801e820 con 0x563e57b4cfc0 2024-03-30T02:44:51.664 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr_trusted.o 2024-03-30T02:44:51.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.666+0000 7f49f2000700 1 -- 172.21.15.67:0/3686043365 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f49f8018070 con 0x563e57b4cfc0 2024-03-30T02:44:51.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.666+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e57a54380 con 0x563e57b4cfc0 2024-03-30T02:44:51.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.666+0000 7f49f2000700 1 -- 172.21.15.67:0/3686043365 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49f801e980 con 0x563e57b4cfc0 2024-03-30T02:44:51.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.666+0000 7f4a22e78b80 10 client.27133 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.666+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563e578c1b60 con 0x563e57b4cfc0 2024-03-30T02:44:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.666+0000 7f49f2000700 1 -- 172.21.15.67:0/3686043365 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e57a54380 con 0x563e57b4cfc0 2024-03-30T02:44:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.667+0000 7f49f2000700 1 -- 172.21.15.67:0/3686043365 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f49f8024020 con 0x563e57b4cfc0 2024-03-30T02:44:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.667+0000 7f4a22e78b80 10 client.27133 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:51.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.667+0000 7f4a22e78b80 10 client.27133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:51.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.667+0000 7f4a22e78b80 20 client.27133 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:51.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.667+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563e57b5d1d0 0x563e57b5f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.667+0000 7f4a22e78b80 4 client.27133 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:51.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.667+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e57a8c700 con 0x563e57b5d1d0 2024-03-30T02:44:51.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.667+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563e57b5d1d0 0x563e57b5f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.669+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563e57b5d1d0 0x563e57b5f5f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b316adde61ad63 server_cookie=9c658bae2bd717e9 in_seq=0 out_seq=0 2024-03-30T02:44:51.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.669+0000 7f49f2000700 10 client.27133 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:51.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f49f2000700 1 -- 172.21.15.67:0/3686043365 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563e57a8c700 con 0x563e57b5d1d0 2024-03-30T02:44:51.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f49f2000700 10 client.27133 handle_command_reply: tid=0 2024-03-30T02:44:51.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f4a22e78b80 1 client.27133 shutdown 2024-03-30T02:44:51.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 msgr2=0x563e57b4a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4a3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f49f8010fd0 tx=0x7f49f80047c0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 shutdown_connections 2024-03-30T02:44:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563e57b5d1d0 0x563e57b5f5f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4a930 0x563e57b58cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b49eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/3686043365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4cfc0 0x563e57b4a3f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 >> 172.21.15.67:0/3686043365 conn(0x563e57aa16a0 msgr2=0x563e57b43890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.670+0000 7f49f2000700 0 client.27133 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.671+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 shutdown_connections 2024-03-30T02:44:51.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.671+0000 7f4a22e78b80 1 -- 172.21.15.67:0/3686043365 wait complete. 2024-03-30T02:44:51.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.671+0000 7f4a22e78b80 20 client.27133 trim_cache size 0 max 16384 2024-03-30T02:44:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.712+0000 7f4a22e78b80 1 Processor -- start 2024-03-30T02:44:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.712+0000 7f4a22e78b80 1 -- start start 2024-03-30T02:44:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.712+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b53c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.712+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4a930 0x563e57b58390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 0x563e57b588d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a22e78b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57b42ff0 con 0x563e57b4cfc0 2024-03-30T02:44:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a22e78b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57b5f980 con 0x563e57b417a0 2024-03-30T02:44:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a22e78b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57b3e1a0 con 0x563e57b4a930 2024-03-30T02:44:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a07e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4a930 0x563e57b58390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a07e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4a930 0x563e57b58390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36572/0 (socket says 172.21.15.67:36572) 2024-03-30T02:44:51.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a07e00700 1 -- 172.21.15.67:0/743038595 learned_addr learned my addr 172.21.15.67:0/743038595 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/743038595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b53c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a07e00700 1 -- 172.21.15.67:0/743038595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 msgr2=0x563e57b53c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a07e00700 1 --2- 172.21.15.67:0/743038595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b53c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a07e00700 1 -- 172.21.15.67:0/743038595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 msgr2=0x563e57b588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a07e00700 1 --2- 172.21.15.67:0/743038595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 0x563e57b588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.713+0000 7f4a07e00700 1 -- 172.21.15.67:0/743038595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e57a59ea0 con 0x563e57b4a930 2024-03-30T02:44:51.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.714+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/743038595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b53c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:51.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.714+0000 7f4a07e00700 1 --2- 172.21.15.67:0/743038595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4a930 0x563e57b58390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f49f8010b10 tx=0x7f49f800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75e7a0679c0e297 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.715+0000 7f49fe600700 1 -- 172.21.15.67:0/743038595 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f801d930 con 0x563e57b4a930 2024-03-30T02:44:51.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.715+0000 7f49fe600700 1 -- 172.21.15.67:0/743038595 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f8016070 con 0x563e57b4a930 2024-03-30T02:44:51.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.715+0000 7f49fe600700 1 -- 172.21.15.67:0/743038595 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8026400 con 0x563e57b4a930 2024-03-30T02:44:51.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.716+0000 7f4a22e78b80 1 -- 172.21.15.67:0/743038595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4a930 msgr2=0x563e57b58390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.716+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/743038595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4a930 0x563e57b58390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f49f8010b10 tx=0x7f49f800bc80 comp rx=0 tx=0).stop 2024-03-30T02:44:51.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.719+0000 7f4a22e78b80 1 -- 172.21.15.67:0/743038595 shutdown_connections 2024-03-30T02:44:51.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.719+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/743038595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 0x563e57b588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.719+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/743038595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b417a0 0x563e57b53c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.719+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/743038595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b4a930 0x563e57b58390 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f49f8010b10 tx=0x7f49f800bc80 comp rx=0 tx=0).stop 2024-03-30T02:44:51.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.719+0000 7f4a22e78b80 1 -- 172.21.15.67:0/743038595 >> 172.21.15.67:0/743038595 conn(0x563e57aa16a0 msgr2=0x563e57b48780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.719+0000 7f4a22e78b80 1 -- 172.21.15.67:0/743038595 shutdown_connections 2024-03-30T02:44:51.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.719+0000 7f4a22e78b80 1 -- 172.21.15.67:0/743038595 wait complete. 2024-03-30T02:44:51.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.720+0000 7f4a22e78b80 1 -- start start 2024-03-30T02:44:51.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.720+0000 7f4a22e78b80 1 Processor -- start 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.723+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b417a0 0x563e57b51c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.723+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 0x563e57b521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.723+0000 7f4a22e78b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b53130 0x563e57b535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.723+0000 7f4a22e78b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57b53bb0 con 0x563e57b4cfc0 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.723+0000 7f4a22e78b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57b53d20 con 0x563e57b53130 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.723+0000 7f4a22e78b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e57b53ee0 con 0x563e57b417a0 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.723+0000 7f4a22e78b80 10 client.0 _ll_register_callbacks cb 0x563e57a39470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a0ca00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b417a0 0x563e57b51c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a0ca00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b417a0 0x563e57b51c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36578/0 (socket says 172.21.15.67:36578) 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a0ca00700 1 -- 172.21.15.67:0/1003109293 learned_addr learned my addr 172.21.15.67:0/1003109293 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a07e00700 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 0x563e57b521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a0ca00700 1 -- 172.21.15.67:0/1003109293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b53130 msgr2=0x563e57b535b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:51.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b53130 0x563e57b535b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a0ca00700 1 -- 172.21.15.67:0/1003109293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 msgr2=0x563e57b521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:51.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 0x563e57b521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a0ca00700 1 -- 172.21.15.67:0/1003109293 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e578c1b60 con 0x563e57b417a0 2024-03-30T02:44:51.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.724+0000 7f4a07e00700 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 0x563e57b521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:51.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.727+0000 7f4a0ca00700 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b417a0 0x563e57b51c90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4a00015040 tx=0x7f4a000060b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4718e592caada35f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:51.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.727+0000 7f49f2000700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:44:51.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.727+0000 7f49f2000700 1 -- 172.21.15.67:0/1003109293 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00024070 con 0x563e57b417a0 2024-03-30T02:44:51.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.727+0000 7f49f2000700 1 -- 172.21.15.67:0/1003109293 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f4a00005280 con 0x563e57b417a0 2024-03-30T02:44:51.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.728+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1003109293 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a0000f040 con 0x563e57b417a0 2024-03-30T02:44:51.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.728+0000 7f49f2000700 1 -- 172.21.15.67:0/1003109293 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a00003a00 con 0x563e57b417a0 2024-03-30T02:44:51.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.728+0000 7f4a22e78b80 10 client.27139 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:51.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.728+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1003109293 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x563e57919cd0 con 0x563e57b417a0 2024-03-30T02:44:51.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.729+0000 7f49f2000700 1 -- 172.21.15.67:0/1003109293 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a0000f040 con 0x563e57b417a0 2024-03-30T02:44:51.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.731+0000 7f49f2000700 1 -- 172.21.15.67:0/1003109293 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4a00009c90 con 0x563e57b417a0 2024-03-30T02:44:51.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.731+0000 7f4a22e78b80 10 client.27139 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:51.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.731+0000 7f4a22e78b80 10 client.27139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:51.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.731+0000 7f4a22e78b80 20 client.27139 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.731+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563e57bc36a0 0x563e57bc5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.731+0000 7f4a22e78b80 4 client.27139 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:44:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.731+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1003109293 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563e578e2cf0 con 0x563e57bc36a0 2024-03-30T02:44:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.731+0000 7f4a07e00700 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563e57bc36a0 0x563e57bc5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.732+0000 7f49f2000700 10 client.27139 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:51.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.732+0000 7f4a07e00700 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563e57bc36a0 0x563e57bc5ac0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a492792086df94 server_cookie=1ad1dfb1a6231c9e in_seq=0 out_seq=0 2024-03-30T02:44:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f49f2000700 1 -- 172.21.15.67:0/1003109293 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563e578e2cf0 con 0x563e57bc36a0 2024-03-30T02:44:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f49f2000700 10 client.27139 handle_command_reply: tid=0 2024-03-30T02:44:51.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f4a22e78b80 1 client.27139 shutdown 2024-03-30T02:44:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1003109293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b417a0 msgr2=0x563e57b51c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b417a0 0x563e57b51c90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4a00015040 tx=0x7f4a000060b0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1003109293 shutdown_connections 2024-03-30T02:44:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e57b53130 0x563e57b535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563e57b417a0 0x563e57b51c90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563e57bc36a0 0x563e57bc5ac0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f4a22e78b80 1 --2- 172.21.15.67:0/1003109293 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563e57b4cfc0 0x563e57b521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f49f2000700 0 client.27139 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.735+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1003109293 >> 172.21.15.67:0/1003109293 conn(0x563e57aa16a0 msgr2=0x563e57b58390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.736+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1003109293 shutdown_connections 2024-03-30T02:44:51.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.736+0000 7f4a22e78b80 1 -- 172.21.15.67:0/1003109293 wait complete. 2024-03-30T02:44:51.736 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:44:51.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.736+0000 7f4a22e78b80 20 client.27139 trim_cache size 0 max 16384 2024-03-30T02:44:51.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.737+0000 7f4a0f400700 1 -- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f49ec0a1350 msgr2=0x7f49ec0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.737+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f49ec0a1350 0x7f49ec0a3810 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4a081a72d0 tx=0x7f49f800b7e0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.737+0000 7f4a0f400700 1 -- 172.21.15.67:0/3291841390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 msgr2=0x7f4a081a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.737+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 0x7f4a081a5b60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f49f4003730 tx=0x7f49f4003fa0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.738+0000 7f4a0f400700 1 -- 172.21.15.67:0/3291841390 shutdown_connections 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.738+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f49ec0a1350 0x7f49ec0a3810 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.738+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a0811a270 0x7f4a081a5b60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.738+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a0806c580 0x7f4a081abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.738+0000 7f4a0f400700 1 --2- 172.21.15.67:0/3291841390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a0806cea0 0x7f4a081ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.738+0000 7f4a0f400700 1 -- 172.21.15.67:0/3291841390 >> 172.21.15.67:0/3291841390 conn(0x7f4a0806ba00 msgr2=0x7f4a0806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.738+0000 7f4a0f400700 1 -- 172.21.15.67:0/3291841390 shutdown_connections 2024-03-30T02:44:51.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:51.739+0000 7f4a0f400700 1 -- 172.21.15.67:0/3291841390 wait complete. 2024-03-30T02:44:52.044 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kobject_uevent.o 2024-03-30T02:44:52.744 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/callchain.o 2024-03-30T02:44:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:52 smithi082 ceph-mon[79176]: pgmap v239: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-03-30T02:44:52.774 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:44:52.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:44:52.794 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr_user.o 2024-03-30T02:44:52.823 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/logic_pio.o 2024-03-30T02:44:52.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:52 smithi067 ceph-mon[37808]: pgmap v239: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-03-30T02:44:52.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:52 smithi170 ceph-mon[78798]: pgmap v239: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-03-30T02:44:52.962 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/debugfs/built-in.a 2024-03-30T02:44:53.069 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/exec_domain.o 2024-03-30T02:44:53.100 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/hw_breakpoint.o 2024-03-30T02:44:53.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.279+0000 7f2db2400700 1 -- 172.21.15.67:0/4201826628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac107db0 msgr2=0x7f2dac108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:53.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.279+0000 7f2db2400700 1 --2- 172.21.15.67:0/4201826628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac107db0 0x7f2dac108190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2d940099c0 tx=0x7f2d94009cd0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.283+0000 7f2db2400700 1 -- 172.21.15.67:0/4201826628 shutdown_connections 2024-03-30T02:44:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.283+0000 7f2db2400700 1 --2- 172.21.15.67:0/4201826628 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac107db0 0x7f2dac108190 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.283+0000 7f2db2400700 1 --2- 172.21.15.67:0/4201826628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac113f00 0x7f2dac1162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.283+0000 7f2db2400700 1 --2- 172.21.15.67:0/4201826628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dac1086d0 0x7f2dac1139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.283+0000 7f2db2400700 1 -- 172.21.15.67:0/4201826628 >> 172.21.15.67:0/4201826628 conn(0x7f2dac06bdf0 msgr2=0x7f2dac06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.283+0000 7f2db2400700 1 -- 172.21.15.67:0/4201826628 shutdown_connections 2024-03-30T02:44:53.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.283+0000 7f2db2400700 1 -- 172.21.15.67:0/4201826628 wait complete. 2024-03-30T02:44:53.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.286+0000 7f2db2400700 1 Processor -- start 2024-03-30T02:44:53.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.286+0000 7f2db2400700 1 -- start start 2024-03-30T02:44:53.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2db2400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dac107db0 0x7f2dac1a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2db2400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac1086d0 0x7f2dac1a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2db2400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac113f00 0x7f2dac1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2db2400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac1a1fc0 con 0x7f2dac1086d0 2024-03-30T02:44:53.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2db2400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac1a2130 con 0x7f2dac113f00 2024-03-30T02:44:53.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2db2400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac1a22f0 con 0x7f2dac107db0 2024-03-30T02:44:53.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2dab400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dac107db0 0x7f2dac1a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2dab400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dac107db0 0x7f2dac1a7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36590/0 (socket says 172.21.15.67:36590) 2024-03-30T02:44:53.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2dab400700 1 -- 172.21.15.67:0/4226086480 learned_addr learned my addr 172.21.15.67:0/4226086480 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:53.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2dabe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac113f00 0x7f2dac1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.287+0000 7f2daaa00700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac1086d0 0x7f2dac1a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.288+0000 7f2dab400700 1 -- 172.21.15.67:0/4226086480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac113f00 msgr2=0x7f2dac1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:53.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.288+0000 7f2dab400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac113f00 0x7f2dac1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.288+0000 7f2dab400700 1 -- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac1086d0 msgr2=0x7f2dac1a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:53.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.288+0000 7f2dab400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac1086d0 0x7f2dac1a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.288+0000 7f2dab400700 1 -- 172.21.15.67:0/4226086480 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d940095f0 con 0x7f2dac107db0 2024-03-30T02:44:53.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.288+0000 7f2dabe00700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac113f00 0x7f2dac1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:44:53.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.288+0000 7f2daaa00700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac1086d0 0x7f2dac1a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:53.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.288+0000 7f2dab400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dac107db0 0x7f2dac1a7830 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400b580 tx=0x7f2d94005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:53.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.290+0000 7f2da3e00700 1 -- 172.21.15.67:0/4226086480 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94022ce0 con 0x7f2dac107db0 2024-03-30T02:44:53.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.290+0000 7f2da3e00700 1 -- 172.21.15.67:0/4226086480 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d94026070 con 0x7f2dac107db0 2024-03-30T02:44:53.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.290+0000 7f2da3e00700 1 -- 172.21.15.67:0/4226086480 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d940179a0 con 0x7f2dac107db0 2024-03-30T02:44:53.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.290+0000 7f2db2400700 1 -- 172.21.15.67:0/4226086480 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dac1a2570 con 0x7f2dac107db0 2024-03-30T02:44:53.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.290+0000 7f2db2400700 1 -- 172.21.15.67:0/4226086480 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dac10bb40 con 0x7f2dac107db0 2024-03-30T02:44:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.303+0000 7f2dc5fafb80 1 Processor -- start 2024-03-30T02:44:53.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.305+0000 7f2da3e00700 1 -- 172.21.15.67:0/4226086480 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2d94017b00 con 0x7f2dac107db0 2024-03-30T02:44:53.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.306+0000 7f2dc5fafb80 1 -- start start 2024-03-30T02:44:53.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.306+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.306+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 0x560a545d5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.306+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545d5e70 0x560a545d8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.306+0000 7f2dc5fafb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a54523a10 con 0x560a545ca650 2024-03-30T02:44:53.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.306+0000 7f2dc5fafb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a545213d0 con 0x560a545caf70 2024-03-30T02:44:53.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.306+0000 7f2dc5fafb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a5451ccc0 con 0x560a545d5e70 2024-03-30T02:44:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.307+0000 7f2dabe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.307+0000 7f2dabe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39326/0 (socket says 172.21.15.67:39326) 2024-03-30T02:44:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.307+0000 7f2dabe00700 1 -- 172.21.15.67:0/591229528 learned_addr learned my addr 172.21.15.67:0/591229528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.307+0000 7f2daaa00700 1 --2- 172.21.15.67:0/591229528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545d5e70 0x560a545d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.308+0000 7f2dabe00700 1 -- 172.21.15.67:0/591229528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545d5e70 msgr2=0x560a545d8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.308+0000 7f2dabe00700 1 --2- 172.21.15.67:0/591229528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545d5e70 0x560a545d8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.308+0000 7f2dabe00700 1 -- 172.21.15.67:0/591229528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 msgr2=0x560a545d5930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:53.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.308+0000 7f2dabe00700 1 --2- 172.21.15.67:0/591229528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 0x560a545d5930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.308+0000 7f2dabe00700 1 -- 172.21.15.67:0/591229528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a5439e430 con 0x560a545ca650 2024-03-30T02:44:53.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.310+0000 7f2da3e00700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2d900a13e0 0x7f2d900a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.310+0000 7f2da3e00700 1 -- 172.21.15.67:0/4226086480 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2d94028070 con 0x7f2dac107db0 2024-03-30T02:44:53.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.311+0000 7f2dab400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2d900a13e0 0x7f2d900a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.311+0000 7f2dabe00700 1 --2- 172.21.15.67:0/591229528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545caa30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00f470 tx=0x7f2d9c00f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470eefbbaeff8e2f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.311+0000 7f2da0c00700 1 -- 172.21.15.67:0/591229528 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00ab30 con 0x560a545ca650 2024-03-30T02:44:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.312+0000 7f2da0c00700 1 -- 172.21.15.67:0/591229528 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d9c00feb0 con 0x560a545ca650 2024-03-30T02:44:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.312+0000 7f2da0c00700 1 -- 172.21.15.67:0/591229528 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0108c0 con 0x560a545ca650 2024-03-30T02:44:53.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.312+0000 7f2dab400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2d900a13e0 0x7f2d900a38a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d94022fd0 tx=0x7f2d94022e40 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:53.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.313+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/591229528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 msgr2=0x560a545caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:53.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.313+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/591229528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545caa30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00f470 tx=0x7f2d9c00f780 comp rx=0 tx=0).stop 2024-03-30T02:44:53.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.314+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/591229528 shutdown_connections 2024-03-30T02:44:53.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.314+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/591229528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545caa30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.314+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/591229528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545d5e70 0x560a545d8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.314+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/591229528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 0x560a545d5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.314+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/591229528 >> 172.21.15.67:0/591229528 conn(0x560a5452a720 msgr2=0x560a545c6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:53.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.316+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/591229528 shutdown_connections 2024-03-30T02:44:53.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.316+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/591229528 wait complete. 2024-03-30T02:44:53.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.318+0000 7f2dc5fafb80 1 -- start start 2024-03-30T02:44:53.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.318+0000 7f2dc5fafb80 1 Processor -- start 2024-03-30T02:44:53.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.319+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545d5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.321+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545caf70 0x560a545d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.321+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 0x560a545d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.321+0000 7f2dc5fafb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a544b8e00 con 0x560a545ca650 2024-03-30T02:44:53.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.321+0000 7f2dc5fafb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a544b8aa0 con 0x560a545d5e70 2024-03-30T02:44:53.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.321+0000 7f2dc5fafb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a543a0720 con 0x560a545caf70 2024-03-30T02:44:53.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.321+0000 7f2dc5fafb80 10 client.0 _ll_register_callbacks cb 0x560a544c24f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:53.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.322+0000 7f2dabe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545d5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.322+0000 7f2dabe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545d5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39334/0 (socket says 172.21.15.67:39334) 2024-03-30T02:44:53.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.322+0000 7f2dabe00700 1 -- 172.21.15.67:0/3091526478 learned_addr learned my addr 172.21.15.67:0/3091526478 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:53.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.323+0000 7f2dabe00700 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 0x560a545d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.323+0000 7f2dabe00700 1 -- 172.21.15.67:0/3091526478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545caf70 msgr2=0x560a545d2b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:53.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.323+0000 7f2dabe00700 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545caf70 0x560a545d2b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.323+0000 7f2dabe00700 1 -- 172.21.15.67:0/3091526478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 msgr2=0x560a545d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:53.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.323+0000 7f2dabe00700 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 0x560a545d31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.323+0000 7f2dabe00700 1 -- 172.21.15.67:0/3091526478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c00ac90 con 0x560a545ca650 2024-03-30T02:44:53.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.324+0000 7f2dabe00700 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545d5890 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c01a040 tx=0x7f2d9c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d2bda031f6657b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:53.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.324+0000 7f2d99600700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:44:53.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.324+0000 7f2d99600700 1 -- 172.21.15.67:0/3091526478 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c004c20 con 0x560a545ca650 2024-03-30T02:44:53.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.324+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3091526478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a544dd400 con 0x560a545ca650 2024-03-30T02:44:53.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.325+0000 7f2d99600700 1 -- 172.21.15.67:0/3091526478 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2d9c004d80 con 0x560a545ca650 2024-03-30T02:44:53.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.328+0000 7f2d99600700 1 -- 172.21.15.67:0/3091526478 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d9c02e700 con 0x560a545ca650 2024-03-30T02:44:53.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.328+0000 7f2dc5fafb80 10 client.17079 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:53.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.328+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3091526478 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x560a5434abe0 con 0x560a545ca650 2024-03-30T02:44:53.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.329+0000 7f2d99600700 1 -- 172.21.15.67:0/3091526478 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d9c02ed20 con 0x560a545ca650 2024-03-30T02:44:53.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.329+0000 7f2d99600700 1 -- 172.21.15.67:0/3091526478 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2d9c020070 con 0x560a545ca650 2024-03-30T02:44:53.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.329+0000 7f2dc5fafb80 10 client.17079 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:53.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.329+0000 7f2dc5fafb80 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:53.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.329+0000 7f2dc5fafb80 20 client.17079 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:53.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.329+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560a545e5840 0x560a545e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.329+0000 7f2dc5fafb80 4 client.17079 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:53.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.329+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3091526478 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a544dd400 con 0x560a545e5840 2024-03-30T02:44:53.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.329+0000 7f2daaa00700 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560a545e5840 0x560a545e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.331+0000 7f2daaa00700 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560a545e5840 0x560a545e7c60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7345949f6d2c5317 server_cookie=aef09bc1b143cb11 in_seq=0 out_seq=0 2024-03-30T02:44:53.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.331+0000 7f2d99600700 10 client.17079 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:53.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.332+0000 7f2d99600700 1 -- 172.21.15.67:0/3091526478 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560a544dd400 con 0x560a545e5840 2024-03-30T02:44:53.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.332+0000 7f2d99600700 10 client.17079 handle_command_reply: tid=0 2024-03-30T02:44:53.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.332+0000 7f2dc5fafb80 1 client.17079 shutdown 2024-03-30T02:44:53.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.334+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3091526478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 msgr2=0x560a545d5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:53.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.334+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545d5890 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c01a040 tx=0x7f2d9c005730 comp rx=0 tx=0).stop 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.336+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3091526478 shutdown_connections 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.336+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560a545e5840 0x560a545e7c60 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.336+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545ca650 0x560a545d5890 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.336+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 0x560a545d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.336+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3091526478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545caf70 0x560a545d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.336+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3091526478 >> 172.21.15.67:0/3091526478 conn(0x560a5452a720 msgr2=0x560a545cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.336+0000 7f2d99600700 0 client.17079 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.336+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3091526478 shutdown_connections 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.336+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3091526478 wait complete. 2024-03-30T02:44:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.337+0000 7f2dc5fafb80 20 client.17079 trim_cache size 0 max 16384 2024-03-30T02:44:53.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.374+0000 7f2dc5fafb80 1 Processor -- start 2024-03-30T02:44:53.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.375+0000 7f2dc5fafb80 1 -- start start 2024-03-30T02:44:53.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.375+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a545c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.375+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 0x560a545dcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.376+0000 7f2dabe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a545c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.376+0000 7f2dabe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a545c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36602/0 (socket says 172.21.15.67:36602) 2024-03-30T02:44:53.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.376+0000 7f2dabe00700 1 -- 172.21.15.67:0/2565016859 learned_addr learned my addr 172.21.15.67:0/2565016859 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:53.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.376+0000 7f2daaa00700 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 0x560a545dcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.376+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545d5e70 0x560a545d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.376+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/2565016859 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a545cd5e0 con 0x560a545d5e70 2024-03-30T02:44:53.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.376+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/2565016859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a545d3790 con 0x560a545caf70 2024-03-30T02:44:53.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.377+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/2565016859 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a545dd0c0 con 0x560a545ca650 2024-03-30T02:44:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.377+0000 7f2daaa00700 1 -- 172.21.15.67:0/2565016859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 msgr2=0x560a545c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.377+0000 7f2daaa00700 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a545c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.377+0000 7f2daaa00700 1 -- 172.21.15.67:0/2565016859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545d5e70 msgr2=0x560a545d53c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.377+0000 7f2dabe00700 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545d5e70 0x560a545d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.378+0000 7f2daaa00700 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545d5e70 0x560a545d53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.378+0000 7f2daaa00700 1 -- 172.21.15.67:0/2565016859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a545bf8f0 con 0x560a545caf70 2024-03-30T02:44:53.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.378+0000 7f2dabe00700 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545d5e70 0x560a545d53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.382+0000 7f2daaa00700 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 0x560a545dcb80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x560a545e8510 tx=0x7f2da400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406e2e12816e1189 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.382+0000 7f2da1600700 1 -- 172.21.15.67:0/2565016859 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da400e040 con 0x560a545caf70 2024-03-30T02:44:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.382+0000 7f2da1600700 1 -- 172.21.15.67:0/2565016859 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da400f040 con 0x560a545caf70 2024-03-30T02:44:53.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.382+0000 7f2da1600700 1 -- 172.21.15.67:0/2565016859 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40088f0 con 0x560a545caf70 2024-03-30T02:44:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.383+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/2565016859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 msgr2=0x560a545dcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.383+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 0x560a545dcb80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x560a545e8510 tx=0x7f2da400b040 comp rx=0 tx=0).stop 2024-03-30T02:44:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.384+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/2565016859 shutdown_connections 2024-03-30T02:44:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.384+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545d5e70 0x560a545d53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.384+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a545c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.384+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/2565016859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545caf70 0x560a545dcb80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.384+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/2565016859 >> 172.21.15.67:0/2565016859 conn(0x560a5452a720 msgr2=0x560a545c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:53.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.386+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/2565016859 shutdown_connections 2024-03-30T02:44:53.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.386+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/2565016859 wait complete. 2024-03-30T02:44:53.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.386+0000 7f2dc5fafb80 1 -- start start 2024-03-30T02:44:53.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.387+0000 7f2dc5fafb80 1 Processor -- start 2024-03-30T02:44:53.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.388+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a54633cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.392+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545caf70 0x560a54634230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.392+0000 7f2dc5fafb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 0x560a545dbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.392+0000 7f2dc5fafb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a545dc2b0 con 0x560a545caf70 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.392+0000 7f2dc5fafb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a545dc420 con 0x560a545d5e70 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.392+0000 7f2dc5fafb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a545dc5e0 con 0x560a545ca650 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.392+0000 7f2dc5fafb80 10 client.0 _ll_register_callbacks cb 0x560a544c24f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.392+0000 7f2dabe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a54633cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.393+0000 7f2dabe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a54633cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36610/0 (socket says 172.21.15.67:36610) 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.393+0000 7f2dabe00700 1 -- 172.21.15.67:0/3440081135 learned_addr learned my addr 172.21.15.67:0/3440081135 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.393+0000 7f2daaa00700 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545caf70 0x560a54634230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.393+0000 7f2dabe00700 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 0x560a545dbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.393+0000 7f2daaa00700 1 -- 172.21.15.67:0/3440081135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 msgr2=0x560a54633cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.393+0000 7f2daaa00700 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a54633cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.393+0000 7f2daaa00700 1 -- 172.21.15.67:0/3440081135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 msgr2=0x560a545dbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.393+0000 7f2daaa00700 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 0x560a545dbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.393+0000 7f2daaa00700 1 -- 172.21.15.67:0/3440081135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da401aaf0 con 0x560a545caf70 2024-03-30T02:44:53.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.394+0000 7f2daaa00700 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545caf70 0x560a54634230 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x560a545ec850 tx=0x7f2da401a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940441ee76fa7dd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:53.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.394+0000 7f2d99600700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:44:53.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.395+0000 7f2d99600700 1 -- 172.21.15.67:0/3440081135 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da400e070 con 0x560a545caf70 2024-03-30T02:44:53.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.395+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3440081135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d9c00feb0 con 0x560a545caf70 2024-03-30T02:44:53.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.395+0000 7f2d99600700 1 -- 172.21.15.67:0/3440081135 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2da401a500 con 0x560a545caf70 2024-03-30T02:44:53.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.395+0000 7f2d99600700 1 -- 172.21.15.67:0/3440081135 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2da4003a00 con 0x560a545caf70 2024-03-30T02:44:53.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.395+0000 7f2dc5fafb80 10 client.17082 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:53.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.395+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3440081135 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x560a544e2f20 con 0x560a545caf70 2024-03-30T02:44:53.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.396+0000 7f2d99600700 1 -- 172.21.15.67:0/3440081135 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d9c00feb0 con 0x560a545caf70 2024-03-30T02:44:53.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.397+0000 7f2d99600700 1 -- 172.21.15.67:0/3440081135 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2da402a060 con 0x560a545caf70 2024-03-30T02:44:53.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.397+0000 7f2dc5fafb80 10 client.17082 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:53.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.397+0000 7f2dc5fafb80 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:53.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.397+0000 7f2dc5fafb80 20 client.17082 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:53.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.397+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560a5463bb40 0x560a5463df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:53.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.397+0000 7f2dc5fafb80 4 client.17082 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:44:53.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.397+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3440081135 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560a54515780 con 0x560a5463bb40 2024-03-30T02:44:53.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.397+0000 7f2dabe00700 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560a5463bb40 0x560a5463df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:53.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.398+0000 7f2dabe00700 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560a5463bb40 0x560a5463df60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aed93e35b5c94b9 server_cookie=a0fab9745562a9d8 in_seq=0 out_seq=0 2024-03-30T02:44:53.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.398+0000 7f2d99600700 10 client.17082 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:53.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.401+0000 7f2d99600700 1 -- 172.21.15.67:0/3440081135 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560a54515780 con 0x560a5463bb40 2024-03-30T02:44:53.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.401+0000 7f2d99600700 10 client.17082 handle_command_reply: tid=0 2024-03-30T02:44:53.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.401+0000 7f2dc5fafb80 1 client.17082 shutdown 2024-03-30T02:44:53.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.401+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3440081135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545caf70 msgr2=0x560a54634230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:53.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.401+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545caf70 0x560a54634230 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x560a545ec850 tx=0x7f2da401a180 comp rx=0 tx=0).stop 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3440081135 shutdown_connections 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560a545d5e70 0x560a545dbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a545ca650 0x560a54633cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560a5463bb40 0x560a5463df60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2dc5fafb80 1 --2- 172.21.15.67:0/3440081135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560a545caf70 0x560a54634230 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2d99600700 0 client.17082 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3440081135 >> 172.21.15.67:0/3440081135 conn(0x560a5452a720 msgr2=0x560a545cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3440081135 shutdown_connections 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2dc5fafb80 1 -- 172.21.15.67:0/3440081135 wait complete. 2024-03-30T02:44:53.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.403+0000 7f2dc5fafb80 20 client.17082 trim_cache size 0 max 16384 2024-03-30T02:44:53.406 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:44:53.406 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:44:53.406 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "3ec72e7d-817d-43eb-b8e7-d621f690652d", 2024-03-30T02:44:53.406 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:44:53.406 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:44:53.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 -- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2d900a13e0 msgr2=0x7f2d900a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:53.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2d900a13e0 0x7f2d900a38a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d94022fd0 tx=0x7f2d94022e40 comp rx=0 tx=0).stop 2024-03-30T02:44:53.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 -- 172.21.15.67:0/4226086480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dac107db0 msgr2=0x7f2dac1a7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:53.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dac107db0 0x7f2dac1a7830 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400b580 tx=0x7f2d94005ef0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 -- 172.21.15.67:0/4226086480 shutdown_connections 2024-03-30T02:44:53.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2d900a13e0 0x7f2d900a38a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dac113f00 0x7f2dac1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dac107db0 0x7f2dac1a7830 secure :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400b580 tx=0x7f2d94005ef0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 --2- 172.21.15.67:0/4226086480 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2dac1086d0 0x7f2dac1a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:53.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.409+0000 7f2db2400700 1 -- 172.21.15.67:0/4226086480 >> 172.21.15.67:0/4226086480 conn(0x7f2dac06bdf0 msgr2=0x7f2dac109750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:53.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.410+0000 7f2db2400700 1 -- 172.21.15.67:0/4226086480 shutdown_connections 2024-03-30T02:44:53.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:53.410+0000 7f2db2400700 1 -- 172.21.15.67:0/4226086480 wait complete. 2024-03-30T02:44:53.419 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_export.o 2024-03-30T02:44:53.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:44:53.633 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/maple_tree.o 2024-03-30T02:44:53.790 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/fast_commit.o 2024-03-30T02:44:53.880 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/sysctl.o 2024-03-30T02:44:53.891 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/panic.o 2024-03-30T02:44:53.935 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/tracefs/inode.o 2024-03-30T02:44:54.211 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs2super.o 2024-03-30T02:44:54.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.275+0000 7f22afc00700 1 -- 172.21.15.67:0/3892404058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a8112d00 msgr2=0x7f22a81150f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:54.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.275+0000 7f22afc00700 1 --2- 172.21.15.67:0/3892404058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a8112d00 0x7f22a81150f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f22a000dd20 tx=0x7f22a000f330 comp rx=0 tx=0).stop 2024-03-30T02:44:54.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.276+0000 7f22afc00700 1 -- 172.21.15.67:0/3892404058 shutdown_connections 2024-03-30T02:44:54.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.276+0000 7f22afc00700 1 --2- 172.21.15.67:0/3892404058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a8109750 0x7f22a8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.276+0000 7f22afc00700 1 --2- 172.21.15.67:0/3892404058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a8112d00 0x7f22a81150f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.276+0000 7f22afc00700 1 --2- 172.21.15.67:0/3892404058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a810a070 0x7f22a81127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.276+0000 7f22afc00700 1 -- 172.21.15.67:0/3892404058 >> 172.21.15.67:0/3892404058 conn(0x7f22a806bdf0 msgr2=0x7f22a806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:54.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.276+0000 7f22afc00700 1 -- 172.21.15.67:0/3892404058 shutdown_connections 2024-03-30T02:44:54.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.276+0000 7f22afc00700 1 -- 172.21.15.67:0/3892404058 wait complete. 2024-03-30T02:44:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.281+0000 7f22afc00700 1 Processor -- start 2024-03-30T02:44:54.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.281+0000 7f22afc00700 1 -- start start 2024-03-30T02:44:54.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22afc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a8109750 0x7f22a81a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22afc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a810a070 0x7f22a81a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22afc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a8112d00 0x7f22a81aa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22afc00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a81aaf70 con 0x7f22a810a070 2024-03-30T02:44:54.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22afc00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a81ab0e0 con 0x7f22a8109750 2024-03-30T02:44:54.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22afc00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a81ab2a0 con 0x7f22a8112d00 2024-03-30T02:44:54.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22ad200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a8109750 0x7f22a81a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22ad200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a8109750 0x7f22a81a6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55138/0 (socket says 172.21.15.67:55138) 2024-03-30T02:44:54.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22ad200700 1 -- 172.21.15.67:0/100257986 learned_addr learned my addr 172.21.15.67:0/100257986 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:54.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f22adc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a8112d00 0x7f22a81aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.284+0000 7f229fe00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a810a070 0x7f22a81a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.285+0000 7f22ad200700 1 -- 172.21.15.67:0/100257986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a8112d00 msgr2=0x7f22a81aa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:54.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.286+0000 7f22ad200700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a8112d00 0x7f22a81aa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.286+0000 7f22ad200700 1 -- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a810a070 msgr2=0x7f22a81a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:54.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.286+0000 7f22ad200700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a810a070 0x7f22a81a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.286+0000 7f22ad200700 1 -- 172.21.15.67:0/100257986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a000da00 con 0x7f22a8109750 2024-03-30T02:44:54.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.286+0000 7f229fe00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a810a070 0x7f22a81a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:54.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.286+0000 7f22adc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a8112d00 0x7f22a81aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:54.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.286+0000 7f22ad200700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a8109750 0x7f22a81a6050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f229800ef10 tx=0x7f229800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:54.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.289+0000 7f22a6000700 1 -- 172.21.15.67:0/100257986 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800cad0 con 0x7f22a8109750 2024-03-30T02:44:54.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.289+0000 7f22a6000700 1 -- 172.21.15.67:0/100257986 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f229800f040 con 0x7f22a8109750 2024-03-30T02:44:54.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.289+0000 7f22a6000700 1 -- 172.21.15.67:0/100257986 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980146e0 con 0x7f22a8109750 2024-03-30T02:44:54.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.292+0000 7f22afc00700 1 -- 172.21.15.67:0/100257986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a81ab520 con 0x7f22a8109750 2024-03-30T02:44:54.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.292+0000 7f22afc00700 1 -- 172.21.15.67:0/100257986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a81b2dc0 con 0x7f22a8109750 2024-03-30T02:44:54.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.309+0000 7f22a6000700 1 -- 172.21.15.67:0/100257986 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2298007500 con 0x7f22a8109750 2024-03-30T02:44:54.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.312+0000 7f22c36f3b80 1 Processor -- start 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.314+0000 7f22c36f3b80 1 -- start start 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.314+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5cdc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.314+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ce78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.314+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.314+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5c35990 con 0x5570a5cdc5d0 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.314+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5c33350 con 0x5570a5cdcef0 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.314+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5c2ec40 con 0x5570a5ce7df0 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.315+0000 7f229fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.315+0000 7f229fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cea1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:36624/0 (socket says 172.21.15.67:36624) 2024-03-30T02:44:54.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.315+0000 7f229fe00700 1 -- 172.21.15.67:0/3531937581 learned_addr learned my addr 172.21.15.67:0/3531937581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:54.316 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.315+0000 7f22ad200700 1 --2- 172.21.15.67:0/3531937581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ce78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.316+0000 7f22ad200700 1 -- 172.21.15.67:0/3531937581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 msgr2=0x5570a5cea1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.316+0000 7f22ad200700 1 --2- 172.21.15.67:0/3531937581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cea1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.316+0000 7f22ad200700 1 -- 172.21.15.67:0/3531937581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 msgr2=0x5570a5cdc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.316+0000 7f22ad200700 1 --2- 172.21.15.67:0/3531937581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5cdc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.316+0000 7f22ad200700 1 -- 172.21.15.67:0/3531937581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570a5ab03b0 con 0x5570a5cdcef0 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.316+0000 7f229fe00700 1 --2- 172.21.15.67:0/3531937581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cea1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.317+0000 7f22ad200700 1 --2- 172.21.15.67:0/3531937581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ce78b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f22980ca690 tx=0x7f22980d55d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f14e6372dd95414 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.317+0000 7f229e000700 1 -- 172.21.15.67:0/3531937581 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980cf2d0 con 0x5570a5cdcef0 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.317+0000 7f229e000700 1 -- 172.21.15.67:0/3531937581 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22980d0e40 con 0x5570a5cdcef0 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.317+0000 7f229e000700 1 -- 172.21.15.67:0/3531937581 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980db7c0 con 0x5570a5cdcef0 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.317+0000 7f22a6000700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f228c0a14a0 0x7f228c0a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.318+0000 7f22adc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f228c0a14a0 0x7f228c0a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.322+0000 7f22a6000700 1 -- 172.21.15.67:0/100257986 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f22980cd340 con 0x7f22a8109750 2024-03-30T02:44:54.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.323+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3531937581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 msgr2=0x5570a5ce78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:54.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.323+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3531937581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ce78b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f22980ca690 tx=0x7f22980d55d0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.324+0000 7f22adc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f228c0a14a0 0x7f228c0a3960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f22a000f700 tx=0x7f22a00058e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:54.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.325+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3531937581 shutdown_connections 2024-03-30T02:44:54.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.325+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3531937581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5cdc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.325+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3531937581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cea1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.325+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3531937581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ce78b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.325+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3531937581 >> 172.21.15.67:0/3531937581 conn(0x5570a5c3c6a0 msgr2=0x5570a5cd8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:54.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.327+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3531937581 shutdown_connections 2024-03-30T02:44:54.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.327+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3531937581 wait complete. 2024-03-30T02:44:54.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.331+0000 7f22c36f3b80 1 -- start start 2024-03-30T02:44:54.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.331+0000 7f22c36f3b80 1 Processor -- start 2024-03-30T02:44:54.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.335+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5ce7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.335+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5cdcef0 0x5570a5ce4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.335+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5ce7df0 0x5570a5ce52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.335+0000 7f22ad200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5ce7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.335+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5bcad80 con 0x5570a5cdc5d0 2024-03-30T02:44:54.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.336+0000 7f22ad200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5ce7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39378/0 (socket says 172.21.15.67:39378) 2024-03-30T02:44:54.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.336+0000 7f22ad200700 1 -- 172.21.15.67:0/3955299770 learned_addr learned my addr 172.21.15.67:0/3955299770 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:54.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.336+0000 7f22adc00700 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5ce7df0 0x5570a5ce52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.336+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5bcaa20 con 0x5570a5ce7df0 2024-03-30T02:44:54.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.336+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5ab26a0 con 0x5570a5cdcef0 2024-03-30T02:44:54.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.336+0000 7f22c36f3b80 10 client.0 _ll_register_callbacks cb 0x5570a5bd4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:54.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.337+0000 7f229fe00700 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5cdcef0 0x5570a5ce4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.337+0000 7f22adc00700 1 -- 172.21.15.67:0/3955299770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5cdcef0 msgr2=0x5570a5ce4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:54.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.337+0000 7f22adc00700 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5cdcef0 0x5570a5ce4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.337+0000 7f22adc00700 1 -- 172.21.15.67:0/3955299770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 msgr2=0x5570a5ce7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:54.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.337+0000 7f22adc00700 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5ce7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.337+0000 7f22adc00700 1 -- 172.21.15.67:0/3955299770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a81b2dc0 con 0x5570a5ce7df0 2024-03-30T02:44:54.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.338+0000 7f22adc00700 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5ce7df0 0x5570a5ce52a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005950 tx=0x7f22a0004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9575108b9b508fac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:54.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.338+0000 7f2296a00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:54.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.339+0000 7f2296a00700 1 -- 172.21.15.67:0/3955299770 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0025070 con 0x5570a5ce7df0 2024-03-30T02:44:54.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.339+0000 7f2296a00700 1 -- 172.21.15.67:0/3955299770 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f22a0003890 con 0x5570a5ce7df0 2024-03-30T02:44:54.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.340+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570a5c27700 con 0x5570a5ce7df0 2024-03-30T02:44:54.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.341+0000 7f2296a00700 1 -- 172.21.15.67:0/3955299770 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22a0003a50 con 0x5570a5ce7df0 2024-03-30T02:44:54.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.342+0000 7f22c36f3b80 10 client.27206 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:54.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.342+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5570a5a5cb60 con 0x5570a5ce7df0 2024-03-30T02:44:54.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.342+0000 7f2296a00700 1 -- 172.21.15.67:0/3955299770 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570a5c27700 con 0x5570a5ce7df0 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.346+0000 7f2296a00700 1 -- 172.21.15.67:0/3955299770 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f22a0032070 con 0x5570a5ce7df0 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.346+0000 7f22c36f3b80 10 client.27206 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.346+0000 7f22c36f3b80 10 client.27206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.346+0000 7f22c36f3b80 20 client.27206 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.346+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570a5cf79a0 0x5570a5cf9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.346+0000 7f22c36f3b80 4 client.27206 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.346+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570a5a7dcf0 con 0x5570a5cf79a0 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.347+0000 7f22ad200700 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570a5cf79a0 0x5570a5cf9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.348+0000 7f22ad200700 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570a5cf79a0 0x5570a5cf9dc0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8060c09a9eb10c9 server_cookie=2fa2b03c05365981 in_seq=0 out_seq=0 2024-03-30T02:44:54.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.348+0000 7f2296a00700 10 client.27206 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.349+0000 7f2296a00700 1 -- 172.21.15.67:0/3955299770 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5570a5a7dcf0 con 0x5570a5cf79a0 2024-03-30T02:44:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.349+0000 7f2296a00700 10 client.27206 handle_command_reply: tid=0 2024-03-30T02:44:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.350+0000 7f22c36f3b80 1 client.27206 shutdown 2024-03-30T02:44:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.355+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5ce7df0 msgr2=0x5570a5ce52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.355+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5ce7df0 0x5570a5ce52a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f22a0005950 tx=0x7f22a0004320 comp rx=0 tx=0).stop 2024-03-30T02:44:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.355+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 shutdown_connections 2024-03-30T02:44:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.355+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570a5cf79a0 0x5570a5cf9dc0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.355+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5ce7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.355+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5ce7df0 0x5570a5ce52a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.355+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/3955299770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5cdcef0 0x5570a5ce4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.355+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 >> 172.21.15.67:0/3955299770 conn(0x5570a5c3c6a0 msgr2=0x5570a5cde6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:54.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.355+0000 7f2296a00700 0 client.27206 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:54.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.358+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 shutdown_connections 2024-03-30T02:44:54.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.358+0000 7f22c36f3b80 1 -- 172.21.15.67:0/3955299770 wait complete. 2024-03-30T02:44:54.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.359+0000 7f22c36f3b80 20 client.27206 trim_cache size 0 max 16384 2024-03-30T02:44:54.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.457+0000 7f22c36f3b80 1 Processor -- start 2024-03-30T02:44:54.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.458+0000 7f22c36f3b80 1 -- start start 2024-03-30T02:44:54.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.459+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5cdef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.459+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ceea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.460+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cd15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.460+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5ce6160 con 0x5570a5cdc5d0 2024-03-30T02:44:54.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.460+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5cf50e0 con 0x5570a5cdcef0 2024-03-30T02:44:54.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.460+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5ce4f30 con 0x5570a5ce7df0 2024-03-30T02:44:54.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.460+0000 7f229fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ceea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.460+0000 7f229fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ceea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55160/0 (socket says 172.21.15.67:55160) 2024-03-30T02:44:54.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.460+0000 7f229fe00700 1 -- 172.21.15.67:0/2193712431 learned_addr learned my addr 172.21.15.67:0/2193712431 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:54.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.462+0000 7f22adc00700 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cd15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.460+0000 7f22ad200700 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5cdef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.462+0000 7f229fe00700 1 -- 172.21.15.67:0/2193712431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 msgr2=0x5570a5cd15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.462+0000 7f229fe00700 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cd15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.462+0000 7f229fe00700 1 -- 172.21.15.67:0/2193712431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 msgr2=0x5570a5cdef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.462+0000 7f229fe00700 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5cdef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.462+0000 7f229fe00700 1 -- 172.21.15.67:0/2193712431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570a5bf4ea0 con 0x5570a5cdcef0 2024-03-30T02:44:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.462+0000 7f22ad200700 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5cdef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.462+0000 7f22adc00700 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cd15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:44:54.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.462+0000 7f229fe00700 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ceea70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f229000bdc0 tx=0x7f2290013740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7e7eca837608c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:54.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.463+0000 7f229ea00700 1 -- 172.21.15.67:0/2193712431 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2290004230 con 0x5570a5cdcef0 2024-03-30T02:44:54.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.465+0000 7f229ea00700 1 -- 172.21.15.67:0/2193712431 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2290004390 con 0x5570a5cdcef0 2024-03-30T02:44:54.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.465+0000 7f229ea00700 1 -- 172.21.15.67:0/2193712431 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2290011930 con 0x5570a5cdcef0 2024-03-30T02:44:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.467+0000 7f22c36f3b80 1 -- 172.21.15.67:0/2193712431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 msgr2=0x5570a5ceea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.467+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ceea70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f229000bdc0 tx=0x7f2290013740 comp rx=0 tx=0).stop 2024-03-30T02:44:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.467+0000 7f22c36f3b80 1 -- 172.21.15.67:0/2193712431 shutdown_connections 2024-03-30T02:44:54.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.467+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5cdc5d0 0x5570a5cdef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.467+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5ce7df0 0x5570a5cd15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.467+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/2193712431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cdcef0 0x5570a5ceea70 secure :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f229000bdc0 tx=0x7f2290013740 comp rx=0 tx=0).stop 2024-03-30T02:44:54.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.467+0000 7f22c36f3b80 1 -- 172.21.15.67:0/2193712431 >> 172.21.15.67:0/2193712431 conn(0x5570a5c3c6a0 msgr2=0x5570a5cd1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:54.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.470+0000 7f22c36f3b80 1 -- 172.21.15.67:0/2193712431 shutdown_connections 2024-03-30T02:44:54.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.470+0000 7f22c36f3b80 1 -- 172.21.15.67:0/2193712431 wait complete. 2024-03-30T02:44:54.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.471+0000 7f22c36f3b80 1 -- start start 2024-03-30T02:44:54.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.473+0000 7f22c36f3b80 1 Processor -- start 2024-03-30T02:44:54.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.478+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5cdc5d0 0x5570a5ced210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.478+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5ce7df0 0x5570a5ced750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.478+0000 7f22c36f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cedc90 0x5570a5d4b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.478+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5d4be00 con 0x5570a5ce7df0 2024-03-30T02:44:54.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.478+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5d4bf40 con 0x5570a5cedc90 2024-03-30T02:44:54.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.478+0000 7f22c36f3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570a5d4c100 con 0x5570a5cdc5d0 2024-03-30T02:44:54.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.478+0000 7f22c36f3b80 10 client.0 _ll_register_callbacks cb 0x5570a5bd4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:44:54.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.479+0000 7f22adc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cedc90 0x5570a5d4b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.479+0000 7f22adc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cedc90 0x5570a5d4b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:55170/0 (socket says 172.21.15.67:55170) 2024-03-30T02:44:54.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.479+0000 7f22adc00700 1 -- 172.21.15.67:0/1493301434 learned_addr learned my addr 172.21.15.67:0/1493301434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:44:54.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.479+0000 7f229fe00700 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5ce7df0 0x5570a5ced750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.480+0000 7f22adc00700 1 -- 172.21.15.67:0/1493301434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5cdc5d0 msgr2=0x5570a5ced210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:44:54.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.480+0000 7f22adc00700 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5cdc5d0 0x5570a5ced210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.480+0000 7f22adc00700 1 -- 172.21.15.67:0/1493301434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5ce7df0 msgr2=0x5570a5ced750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:44:54.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.480+0000 7f22adc00700 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5ce7df0 0x5570a5ced750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.480+0000 7f22adc00700 1 -- 172.21.15.67:0/1493301434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229000b9c0 con 0x5570a5cedc90 2024-03-30T02:44:54.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.480+0000 7f22adc00700 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cedc90 0x5570a5d4b7e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f22a000fb20 tx=0x7f22a0004720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba252488d074074 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:44:54.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.481+0000 7f2296a00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:44:54.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.481+0000 7f2296a00700 1 -- 172.21.15.67:0/1493301434 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0025070 con 0x5570a5cedc90 2024-03-30T02:44:54.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.481+0000 7f22c36f3b80 1 -- 172.21.15.67:0/1493301434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570a5bef380 con 0x5570a5cedc90 2024-03-30T02:44:54.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.481+0000 7f2296a00700 1 -- 172.21.15.67:0/1493301434 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f22a001f440 con 0x5570a5cedc90 2024-03-30T02:44:54.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.481+0000 7f2296a00700 1 -- 172.21.15.67:0/1493301434 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22a0028580 con 0x5570a5cedc90 2024-03-30T02:44:54.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.484+0000 7f22c36f3b80 10 client.27212 fetch_fsmap learned FSMap version 39 2024-03-30T02:44:54.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.484+0000 7f22c36f3b80 1 -- 172.21.15.67:0/1493301434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5570a5ab4cd0 con 0x5570a5cedc90 2024-03-30T02:44:54.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.485+0000 7f2296a00700 1 -- 172.21.15.67:0/1493301434 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570a5bef380 con 0x5570a5cedc90 2024-03-30T02:44:54.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.486+0000 7f2296a00700 1 -- 172.21.15.67:0/1493301434 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f22a0004040 con 0x5570a5cedc90 2024-03-30T02:44:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.487+0000 7f22c36f3b80 10 client.27212 fetch_fsmap finished waiting for FSMap version 39 2024-03-30T02:44:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.487+0000 7f22c36f3b80 10 client.27212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:44:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.487+0000 7f22c36f3b80 20 client.27212 populate_metadata read hostname 'smithi067' 2024-03-30T02:44:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.487+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570a5d4d5a0 0x5570a5d4f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:44:54.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.487+0000 7f22c36f3b80 4 client.27212 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:44:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.487+0000 7f22c36f3b80 1 -- 172.21.15.67:0/1493301434 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5570a5ab2ce0 con 0x5570a5d4d5a0 2024-03-30T02:44:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.487+0000 7f22ad200700 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570a5d4d5a0 0x5570a5d4f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:44:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.488+0000 7f22ad200700 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570a5d4d5a0 0x5570a5d4f9c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d56e03ceff4c50 server_cookie=aee9bec92c34c1b5 in_seq=0 out_seq=0 2024-03-30T02:44:54.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.488+0000 7f2296a00700 10 client.27212 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:54.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.490+0000 7f2296a00700 1 -- 172.21.15.67:0/1493301434 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+269 (crc 0 0 0) 0x5570a5ab2ce0 con 0x5570a5d4d5a0 2024-03-30T02:44:54.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.490+0000 7f2296a00700 10 client.27212 handle_command_reply: tid=0 2024-03-30T02:44:54.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.490+0000 7f22c36f3b80 1 client.27212 shutdown 2024-03-30T02:44:54.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.490+0000 7f22c36f3b80 1 -- 172.21.15.67:0/1493301434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cedc90 msgr2=0x5570a5d4b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:54.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.490+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cedc90 0x5570a5d4b7e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f22a000fb20 tx=0x7f22a0004720 comp rx=0 tx=0).stop 2024-03-30T02:44:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.491+0000 7f22c36f3b80 1 -- 172.21.15.67:0/1493301434 shutdown_connections 2024-03-30T02:44:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.491+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570a5cedc90 0x5570a5d4b7e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.491+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570a5cdc5d0 0x5570a5ced210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.491+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570a5d4d5a0 0x5570a5d4f9c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.491+0000 7f22c36f3b80 1 --2- 172.21.15.67:0/1493301434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570a5ce7df0 0x5570a5ced750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.491+0000 7f22c36f3b80 1 -- 172.21.15.67:0/1493301434 >> 172.21.15.67:0/1493301434 conn(0x5570a5c3c6a0 msgr2=0x5570a5cd87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:54.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.491+0000 7f2296a00700 0 client.27212 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:44:54.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.494+0000 7f22c36f3b80 1 -- 172.21.15.67:0/1493301434 shutdown_connections 2024-03-30T02:44:54.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.494+0000 7f22c36f3b80 1 -- 172.21.15.67:0/1493301434 wait complete. 2024-03-30T02:44:54.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.498+0000 7f22c36f3b80 20 client.27212 trim_cache size 0 max 16384 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (16596 inodes in the stack)", 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout: "3ec72e7d-817d-43eb-b8e7-d621f690652d": { 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "3ec72e7d-817d-43eb-b8e7-d621f690652d", 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:44:54.498 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:44:54.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.502+0000 7f22afc00700 1 -- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f228c0a14a0 msgr2=0x7f228c0a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:54.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.502+0000 7f22afc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f228c0a14a0 0x7f228c0a3960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f22a000f700 tx=0x7f22a00058e0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.505+0000 7f22afc00700 1 -- 172.21.15.67:0/100257986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a8109750 msgr2=0x7f22a81a6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:44:54.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.505+0000 7f22afc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a8109750 0x7f22a81a6050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f229800ef10 tx=0x7f229800c3a0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.505+0000 7f22afc00700 1 -- 172.21.15.67:0/100257986 shutdown_connections 2024-03-30T02:44:54.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.505+0000 7f22afc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f228c0a14a0 0x7f228c0a3960 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.505+0000 7f22afc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22a8112d00 0x7f22a81aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.505+0000 7f22afc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a8109750 0x7f22a81a6050 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.505+0000 7f22afc00700 1 --2- 172.21.15.67:0/100257986 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22a810a070 0x7f22a81a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:44:54.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.505+0000 7f22afc00700 1 -- 172.21.15.67:0/100257986 >> 172.21.15.67:0/100257986 conn(0x7f22a806bdf0 msgr2=0x7f22a806fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:44:54.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.507+0000 7f22afc00700 1 -- 172.21.15.67:0/100257986 shutdown_connections 2024-03-30T02:44:54.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:44:54.507+0000 7f22afc00700 1 -- 172.21.15.67:0/100257986 wait complete. 2024-03-30T02:44:54.598 INFO:tasks.cephfs.filesystem:scrub status for tag:3ec72e7d-817d-43eb-b8e7-d621f690652d - {'path': '/', 'tag': '3ec72e7d-817d-43eb-b8e7-d621f690652d', 'options': 'recursive,force'} 2024-03-30T02:44:54.723 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_event_perf.o 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: pgmap v240: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 670 KiB/s wr, 38 op/s 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:54.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:54 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.787 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/events/uprobes.o 2024-03-30T02:44:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: pgmap v240: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 670 KiB/s wr, 38 op/s 2024-03-30T02:44:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:44:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:54.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:54 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: pgmap v240: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 670 KiB/s wr, 38 op/s 2024-03-30T02:44:54.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:44:54.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:54.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:54.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:54 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.261 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/tracefs/built-in.a 2024-03-30T02:44:55.311 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/proc.o 2024-03-30T02:44:55.401 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] fs/efivarfs/inode.o 2024-03-30T02:44:55.558 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/memcat_p.o 2024-03-30T02:44:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:55 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:55.805 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_events_filter.o 2024-03-30T02:44:55.814 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/orphan.o 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:55 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:55.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:55 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.049 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cpu.o 2024-03-30T02:44:56.315 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] fs/efivarfs/file.o 2024-03-30T02:44:56.423 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] fs/efivarfs/super.o 2024-03-30T02:44:56.670 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/events/built-in.a 2024-03-30T02:44:56.735 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/open.o 2024-03-30T02:44:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: pgmap v241: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 625 KiB/s wr, 39 op/s 2024-03-30T02:44:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:56.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:56 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: pgmap v241: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 625 KiB/s wr, 39 op/s 2024-03-30T02:44:56.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:56.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:56 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: pgmap v241: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 625 KiB/s wr, 39 op/s 2024-03-30T02:44:56.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:56.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:56 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:56.974 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs2xdr.o 2024-03-30T02:44:57.106 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/acl.o 2024-03-30T02:44:57.210 INFO:tasks.workunit.client.0.smithi067.stdout: CC [M] fs/efivarfs/vars.o 2024-03-30T02:44:57.475 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_events_trigger.o 2024-03-30T02:44:57.523 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ext4/xattr_security.o 2024-03-30T02:44:57.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: pgmap v242: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-03-30T02:44:57.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:57.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:57 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: pgmap v242: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:57.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:57 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: pgmap v242: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:57.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:57.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:57 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.127 INFO:tasks.workunit.client.0.smithi067.stdout: LD [M] fs/efivarfs/efivarfs.o 2024-03-30T02:44:58.211 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3super.o 2024-03-30T02:44:58.265 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_eprobe.o 2024-03-30T02:44:58.334 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_kprobe.o 2024-03-30T02:44:58.494 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/nmi_backtrace.o 2024-03-30T02:44:58.523 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/ext4/built-in.a 2024-03-30T02:44:58.639 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/error_report-traces.o 2024-03-30T02:44:58.648 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/read_write.o 2024-03-30T02:44:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:58.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:58 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:58.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:58 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:58.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:58 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:58.991 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/file_table.o 2024-03-30T02:44:59.131 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3client.o 2024-03-30T02:44:59.451 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/plist.o 2024-03-30T02:44:59.590 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/exit.o 2024-03-30T02:44:59.618 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/radix-tree.o 2024-03-30T02:44:59.659 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/ratelimit.o 2024-03-30T02:44:59.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: pgmap v243: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:44:59.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:44:59 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/super.o 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: pgmap v243: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:44:59.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:44:59 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:59.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:44:59.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:44:59.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: pgmap v243: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.2 MiB/s wr, 190 op/s 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:00.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:44:59 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.075 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/char_dev.o 2024-03-30T02:45:00.113 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/softirq.o 2024-03-30T02:45:00.621 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3proc.o 2024-03-30T02:45:00.645 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/power-traces.o 2024-03-30T02:45:00.664 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3xdr.o 2024-03-30T02:45:00.828 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/rbtree.o 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:00.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:00 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:00.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:00 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:00.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:01.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:00 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.172 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/stat.o 2024-03-30T02:45:01.357 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/seq_buf.o 2024-03-30T02:45:01.379 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/siphash.o 2024-03-30T02:45:01.673 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs3acl.o 2024-03-30T02:45:01.750 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/string.o 2024-03-30T02:45:01.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: pgmap v244: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 1.09k op/s 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:01.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:01 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: pgmap v244: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 1.09k op/s 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:01.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:01.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:01 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: pgmap v244: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 1.09k op/s 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:02.005 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:01 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.113 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/rpm-traces.o 2024-03-30T02:45:02.187 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/exec.o 2024-03-30T02:45:02.248 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_dynevent.o 2024-03-30T02:45:02.488 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_probe.o 2024-03-30T02:45:02.511 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/timerqueue.o 2024-03-30T02:45:02.515 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/pipe.o 2024-03-30T02:45:02.834 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/vsprintf.o 2024-03-30T02:45:02.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:02.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:02 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:02.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:02.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:02 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:02 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:45:03.092 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/win_minmax.o 2024-03-30T02:45:03.204 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/resource.o 2024-03-30T02:45:03.244 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/trace_uprobe.o 2024-03-30T02:45:03.263 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4proc.o 2024-03-30T02:45:03.765 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/namei.o 2024-03-30T02:45:03.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: pgmap v245: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 1.71k op/s 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:03.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:03 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: pgmap v245: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 1.71k op/s 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:03.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:03 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:03.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:03.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: pgmap v245: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 1.71k op/s 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:03 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.147 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xarray.o 2024-03-30T02:45:04.237 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sysctl.o 2024-03-30T02:45:04.343 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fcntl.o 2024-03-30T02:45:04.564 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/ioctl.o 2024-03-30T02:45:04.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:04.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:04 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:04.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:04 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:04.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:04.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:04 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.287 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4xdr.o 2024-03-30T02:45:05.340 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/trace/rethook.o 2024-03-30T02:45:05.547 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/lockref.o 2024-03-30T02:45:05.640 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/capability.o 2024-03-30T02:45:05.828 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/ptrace.o 2024-03-30T02:45:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: pgmap v246: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 2.15k op/s 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: pgmap v246: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 2.15k op/s 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.975 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bcd.o 2024-03-30T02:45:05.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:05.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:05.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: pgmap v246: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 2.15k op/s 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:06.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:06.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.073 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/sort.o 2024-03-30T02:45:06.193 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/parser.o 2024-03-30T02:45:06.225 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/trace/built-in.a 2024-03-30T02:45:06.228 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/user.o 2024-03-30T02:45:06.296 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/debug_locks.o 2024-03-30T02:45:06.555 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4state.o 2024-03-30T02:45:06.580 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/readdir.o 2024-03-30T02:45:06.646 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/random32.o 2024-03-30T02:45:06.726 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/select.o 2024-03-30T02:45:06.815 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4renewd.o 2024-03-30T02:45:06.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:06.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:06.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:07.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.003 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/signal.o 2024-03-30T02:45:07.115 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bust_spinlocks.o 2024-03-30T02:45:07.319 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/dcache.o 2024-03-30T02:45:07.619 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kasprintf.o 2024-03-30T02:45:07.822 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sys.o 2024-03-30T02:45:07.908 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/inode.o 2024-03-30T02:45:07.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:07 smithi067 ceph-mon[37808]: pgmap v247: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1016 KiB/s wr, 3.59k op/s 2024-03-30T02:45:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:07 smithi067 ceph-mon[37808]: osdmap e91: 12 total, 12 up, 12 in 2024-03-30T02:45:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:07 smithi067 ceph-mon[37808]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:45:07.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:07 smithi170 ceph-mon[78798]: pgmap v247: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1016 KiB/s wr, 3.59k op/s 2024-03-30T02:45:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:07 smithi170 ceph-mon[78798]: osdmap e91: 12 total, 12 up, 12 in 2024-03-30T02:45:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:07 smithi170 ceph-mon[78798]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:45:07.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:07.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:07 smithi082 ceph-mon[79176]: pgmap v247: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1016 KiB/s wr, 3.59k op/s 2024-03-30T02:45:07.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:07 smithi082 ceph-mon[79176]: osdmap e91: 12 total, 12 up, 12 in 2024-03-30T02:45:08.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:07 smithi082 ceph-mon[79176]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-30T02:45:08.010 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bitmap.o 2024-03-30T02:45:08.160 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4super.o 2024-03-30T02:45:08.316 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/umh.o 2024-03-30T02:45:08.540 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/attr.o 2024-03-30T02:45:08.881 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/scatterlist.o 2024-03-30T02:45:09.082 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4file.o 2024-03-30T02:45:09.213 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/list_sort.o 2024-03-30T02:45:09.274 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/delegation.o 2024-03-30T02:45:09.289 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/uuid.o 2024-03-30T02:45:09.379 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/bad_inode.o 2024-03-30T02:45:09.379 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/file.o 2024-03-30T02:45:09.407 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4idmap.o 2024-03-30T02:45:09.556 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/iov_iter.o 2024-03-30T02:45:09.637 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/workqueue.o 2024-03-30T02:45:09.897 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/clz_ctz.o 2024-03-30T02:45:09.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:09 smithi067 ceph-mon[37808]: pgmap v249: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 442 KiB/s wr, 4.37k op/s 2024-03-30T02:45:09.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:09 smithi170 ceph-mon[78798]: pgmap v249: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 442 KiB/s wr, 4.37k op/s 2024-03-30T02:45:09.987 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/pid.o 2024-03-30T02:45:09.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:09 smithi082 ceph-mon[79176]: pgmap v249: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 442 KiB/s wr, 4.37k op/s 2024-03-30T02:45:10.028 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/task_work.o 2024-03-30T02:45:10.496 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bsearch.o 2024-03-30T02:45:10.774 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/filesystems.o 2024-03-30T02:45:10.789 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/find_bit.o 2024-03-30T02:45:10.844 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/llist.o 2024-03-30T02:45:10.963 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/callback.o 2024-03-30T02:45:11.016 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/namespace.o 2024-03-30T02:45:11.088 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/seq_file.o 2024-03-30T02:45:11.161 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/extable.o 2024-03-30T02:45:11.224 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/callback_xdr.o 2024-03-30T02:45:11.487 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/memweight.o 2024-03-30T02:45:11.621 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/xattr.o 2024-03-30T02:45:11.641 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kfifo.o 2024-03-30T02:45:11.995 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/callback_proc.o 2024-03-30T02:45:12.045 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/percpu-refcount.o 2024-03-30T02:45:12.086 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/params.o 2024-03-30T02:45:12.179 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/rhashtable.o 2024-03-30T02:45:12.458 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kthread.o 2024-03-30T02:45:12.609 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/libfs.o 2024-03-30T02:45:12.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:12 smithi067 ceph-mon[37808]: pgmap v250: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 436 KiB/s wr, 3.70k op/s 2024-03-30T02:45:12.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:12 smithi170 ceph-mon[78798]: pgmap v250: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 436 KiB/s wr, 3.70k op/s 2024-03-30T02:45:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:12 smithi082 ceph-mon[79176]: pgmap v250: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 436 KiB/s wr, 3.70k op/s 2024-03-30T02:45:12.776 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs-writeback.o 2024-03-30T02:45:12.787 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/base64.o 2024-03-30T02:45:12.942 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/once.o 2024-03-30T02:45:13.025 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/sys_ni.o 2024-03-30T02:45:13.286 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/refcount.o 2024-03-30T02:45:13.441 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4namespace.o 2024-03-30T02:45:13.526 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/pnode.o 2024-03-30T02:45:13.738 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/rcuref.o 2024-03-30T02:45:13.898 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/usercopy.o 2024-03-30T02:45:13.993 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/nsproxy.o 2024-03-30T02:45:14.066 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/splice.o 2024-03-30T02:45:14.104 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4getroot.o 2024-03-30T02:45:14.181 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sync.o 2024-03-30T02:45:14.336 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/errseq.o 2024-03-30T02:45:14.456 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4client.o 2024-03-30T02:45:14.507 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bucket_locks.o 2024-03-30T02:45:14.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:14 smithi170 ceph-mon[78798]: pgmap v251: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 419 KiB/s wr, 2.97k op/s 2024-03-30T02:45:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:14 smithi082 ceph-mon[79176]: pgmap v251: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 419 KiB/s wr, 2.97k op/s 2024-03-30T02:45:14.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:14 smithi067 ceph-mon[37808]: pgmap v251: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 419 KiB/s wr, 2.97k op/s 2024-03-30T02:45:14.926 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/utimes.o 2024-03-30T02:45:14.981 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/generic-radix-tree.o 2024-03-30T02:45:15.039 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/notifier.o 2024-03-30T02:45:15.110 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/string_helpers.o 2024-03-30T02:45:15.176 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/ksysfs.o 2024-03-30T02:45:15.181 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4session.o 2024-03-30T02:45:15.478 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/hexdump.o 2024-03-30T02:45:15.552 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/dns_resolve.o 2024-03-30T02:45:15.703 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/d_path.o 2024-03-30T02:45:15.795 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/cred.o 2024-03-30T02:45:15.820 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4trace.o 2024-03-30T02:45:15.939 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/kstrtox.o 2024-03-30T02:45:15.951 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/reboot.o 2024-03-30T02:45:15.965 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/stack.o 2024-03-30T02:45:16.324 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/async.o 2024-03-30T02:45:16.491 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/iomap.o 2024-03-30T02:45:16.575 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_struct.o 2024-03-30T02:45:16.595 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nfs/nfs4sysctl.o 2024-03-30T02:45:16.624 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/range.o 2024-03-30T02:45:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:16 smithi170 ceph-mon[78798]: pgmap v252: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 640 KiB/s wr, 2.44k op/s 2024-03-30T02:45:16.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:16 smithi170 ceph-mon[78798]: osdmap e92: 12 total, 12 up, 12 in 2024-03-30T02:45:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:16 smithi082 ceph-mon[79176]: pgmap v252: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 640 KiB/s wr, 2.44k op/s 2024-03-30T02:45:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:16 smithi082 ceph-mon[79176]: osdmap e92: 12 total, 12 up, 12 in 2024-03-30T02:45:16.893 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/statfs.o 2024-03-30T02:45:16.910 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_pin.o 2024-03-30T02:45:16.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:16 smithi067 ceph-mon[37808]: pgmap v252: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 640 KiB/s wr, 2.44k op/s 2024-03-30T02:45:16.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:16 smithi067 ceph-mon[37808]: osdmap e92: 12 total, 12 up, 12 in 2024-03-30T02:45:17.009 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/pci_iomap.o 2024-03-30T02:45:17.065 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/smpboot.o 2024-03-30T02:45:17.268 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/ucount.o 2024-03-30T02:45:17.462 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/iomap_copy.o 2024-03-30T02:45:17.523 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/nsfs.o 2024-03-30T02:45:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:17 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:17 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:17 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:17 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:17.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:17 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:17.738 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/devres.o 2024-03-30T02:45:17.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:17 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:17.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:17 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:17.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:17 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:17.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:17 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:17.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:17 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:17.750 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/check_signature.o 2024-03-30T02:45:17.753 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_types.o 2024-03-30T02:45:17.766 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/interval_tree.o 2024-03-30T02:45:17.836 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/regset.o 2024-03-30T02:45:17.856 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_context.o 2024-03-30T02:45:17.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:17 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:17.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:17 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:17.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:17 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:17.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:17 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:17.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:17 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:18.021 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fs_parser.o 2024-03-30T02:45:18.078 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/assoc_array.o 2024-03-30T02:45:18.234 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fsopen.o 2024-03-30T02:45:18.325 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/ksyms_common.o 2024-03-30T02:45:18.354 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/bitrev.o 2024-03-30T02:45:18.539 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/init.o 2024-03-30T02:45:18.616 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/nfs/built-in.a 2024-03-30T02:45:18.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:18 smithi170 ceph-mon[78798]: pgmap v254: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.2 MiB/s wr, 529 op/s 2024-03-30T02:45:18.727 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/kernel_read_file.o 2024-03-30T02:45:18.733 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/mnt_idmapping.o 2024-03-30T02:45:18.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:18 smithi082 ceph-mon[79176]: pgmap v254: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.2 MiB/s wr, 529 op/s 2024-03-30T02:45:18.794 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crc-ccitt.o 2024-03-30T02:45:18.809 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/remap_range.o 2024-03-30T02:45:18.851 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/buffer.o 2024-03-30T02:45:18.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:18 smithi067 ceph-mon[37808]: pgmap v254: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.2 MiB/s wr, 529 op/s 2024-03-30T02:45:18.983 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/groups.o 2024-03-30T02:45:19.120 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/mpage.o 2024-03-30T02:45:19.258 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crc16.o 2024-03-30T02:45:19.268 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kcmp.o 2024-03-30T02:45:19.339 INFO:tasks.workunit.client.0.smithi067.stdout: HOSTCC lib/gen_crc32table 2024-03-30T02:45:19.409 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/proc_namespace.o 2024-03-30T02:45:19.499 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/xxhash.o 2024-03-30T02:45:19.727 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/freezer.o 2024-03-30T02:45:19.758 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/direct-io.o 2024-03-30T02:45:19.802 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/genalloc.o 2024-03-30T02:45:20.102 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/profile.o 2024-03-30T02:45:20.136 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/eventpoll.o 2024-03-30T02:45:20.160 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/percpu_counter.o 2024-03-30T02:45:20.204 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/syscall.o 2024-03-30T02:45:20.586 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/anon_inodes.o 2024-03-30T02:45:20.617 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/errname.o 2024-03-30T02:45:20.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:20 smithi170 ceph-mon[78798]: pgmap v255: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 1.7 MiB/s wr, 490 op/s 2024-03-30T02:45:20.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:20 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:45:20.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:20 smithi082 ceph-mon[79176]: pgmap v255: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 1.7 MiB/s wr, 490 op/s 2024-03-30T02:45:20.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:20 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:45:20.815 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/signalfd.o 2024-03-30T02:45:20.841 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/nlattr.o 2024-03-30T02:45:20.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:20 smithi067 ceph-mon[37808]: pgmap v255: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1024 KiB/s rd, 1.7 MiB/s wr, 490 op/s 2024-03-30T02:45:20.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:20 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:45:20.914 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/timerfd.o 2024-03-30T02:45:20.975 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/stacktrace.o 2024-03-30T02:45:21.149 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/eventfd.o 2024-03-30T02:45:21.350 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/dma.o 2024-03-30T02:45:21.383 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/cpu_rmap.o 2024-03-30T02:45:21.765 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/smp.o 2024-03-30T02:45:21.780 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/dynamic_queue_limits.o 2024-03-30T02:45:21.842 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/aio.o 2024-03-30T02:45:21.920 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/glob.o 2024-03-30T02:45:21.988 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/uid16.o 2024-03-30T02:45:21.992 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/locks.o 2024-03-30T02:45:22.005 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/binfmt_misc.o 2024-03-30T02:45:22.076 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/strncpy_from_user.o 2024-03-30T02:45:22.105 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kallsyms.o 2024-03-30T02:45:22.399 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/binfmt_script.o 2024-03-30T02:45:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:22 smithi082 ceph-mon[79176]: pgmap v256: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-03-30T02:45:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:22 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:45:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:22 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:45:22.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:22 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:45:22.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:22 smithi067 ceph-mon[37808]: pgmap v256: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-03-30T02:45:22.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:22 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:45:22.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:22 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:45:22.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:22 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:45:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:22 smithi170 ceph-mon[78798]: pgmap v256: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.7 MiB/s wr, 95 op/s 2024-03-30T02:45:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:22 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:45:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:22 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:45:22.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:22 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:45:23.012 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/strnlen_user.o 2024-03-30T02:45:23.133 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/net_utils.o 2024-03-30T02:45:23.269 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/binfmt_elf.o 2024-03-30T02:45:23.276 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/compat_binfmt_elf.o 2024-03-30T02:45:23.397 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/mbcache.o 2024-03-30T02:45:23.415 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/acct.o 2024-03-30T02:45:23.561 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/sg_pool.o 2024-03-30T02:45:23.675 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/posix_acl.o 2024-03-30T02:45:23.686 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/crash_core.o 2024-03-30T02:45:23.924 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/stackdepot.o 2024-03-30T02:45:24.333 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/asn1_decoder.o 2024-03-30T02:45:24.541 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kexec_core.o 2024-03-30T02:45:24.555 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/coredump.o 2024-03-30T02:45:24.559 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/drop_caches.o 2024-03-30T02:45:24.560 INFO:tasks.workunit.client.0.smithi067.stdout: GEN lib/oid_registry_data.c 2024-03-30T02:45:24.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:45:24.708 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kexec.o 2024-03-30T02:45:24.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:24 smithi082 ceph-mon[79176]: pgmap v257: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.9 MiB/s wr, 72 op/s 2024-03-30T02:45:24.749 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/ucs2_string.o 2024-03-30T02:45:24.759 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/sysctls.o 2024-03-30T02:45:24.773 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/compat.o 2024-03-30T02:45:24.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:24 smithi067 ceph-mon[37808]: pgmap v257: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.9 MiB/s wr, 72 op/s 2024-03-30T02:45:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.922+0000 7fad12e00700 1 -- 172.21.15.67:0/3150237041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad0c107db0 msgr2=0x7fad0c108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.922+0000 7fad12e00700 1 --2- 172.21.15.67:0/3150237041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad0c107db0 0x7fad0c108230 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7facf80099c0 tx=0x7facf8009cd0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.929+0000 7fad12e00700 1 -- 172.21.15.67:0/3150237041 shutdown_connections 2024-03-30T02:45:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.929+0000 7fad12e00700 1 --2- 172.21.15.67:0/3150237041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad0c10d630 0x7fad0c10da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.929+0000 7fad12e00700 1 --2- 172.21.15.67:0/3150237041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c108770 0x7fad0c1143f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.929+0000 7fad12e00700 1 --2- 172.21.15.67:0/3150237041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad0c107db0 0x7fad0c108230 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.929+0000 7fad12e00700 1 -- 172.21.15.67:0/3150237041 >> 172.21.15.67:0/3150237041 conn(0x7fad0c06bc10 msgr2=0x7fad0c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:24.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.929+0000 7fad12e00700 1 -- 172.21.15.67:0/3150237041 shutdown_connections 2024-03-30T02:45:24.932 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/sbitmap.o 2024-03-30T02:45:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.929+0000 7fad12e00700 1 -- 172.21.15.67:0/3150237041 wait complete. 2024-03-30T02:45:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.931+0000 7fad12e00700 1 Processor -- start 2024-03-30T02:45:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.932+0000 7fad12e00700 1 -- start start 2024-03-30T02:45:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.933+0000 7fad12e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c107db0 0x7fad0c1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.934+0000 7fad12e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad0c108770 0x7fad0c1a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.934+0000 7fad12e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad0c10d630 0x7fad0c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.934+0000 7fad12e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c19dda0 con 0x7fad0c10d630 2024-03-30T02:45:24.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.934+0000 7fad12e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c19df10 con 0x7fad0c108770 2024-03-30T02:45:24.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.934+0000 7fad11a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c107db0 0x7fad0c1a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:24.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.934+0000 7fad12e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c19e0d0 con 0x7fad0c107db0 2024-03-30T02:45:24.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.934+0000 7fad11a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c107db0 0x7fad0c1a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52284/0 (socket says 172.21.15.67:52284) 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.934+0000 7fad11a00700 1 -- 172.21.15.67:0/2766537124 learned_addr learned my addr 172.21.15.67:0/2766537124 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.934+0000 7fad12400700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad0c10d630 0x7fad0c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad11a00700 1 -- 172.21.15.67:0/2766537124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad0c108770 msgr2=0x7fad0c1a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad11a00700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad0c108770 0x7fad0c1a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad11a00700 1 -- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad0c10d630 msgr2=0x7fad0c19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad11a00700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad0c10d630 0x7fad0c19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad11a00700 1 -- 172.21.15.67:0/2766537124 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf80095f0 con 0x7fad0c107db0 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad11a00700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c107db0 0x7fad0c1a4400 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad0000b550 tx=0x7fad0000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad0a000700 1 -- 172.21.15.67:0/2766537124 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00011830 con 0x7fad0c107db0 2024-03-30T02:45:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad0a000700 1 -- 172.21.15.67:0/2766537124 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad00011e50 con 0x7fad0c107db0 2024-03-30T02:45:24.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad0a000700 1 -- 172.21.15.67:0/2766537124 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0000f2e0 con 0x7fad0c107db0 2024-03-30T02:45:24.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.935+0000 7fad12e00700 1 -- 172.21.15.67:0/2766537124 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c1a1f90 con 0x7fad0c107db0 2024-03-30T02:45:24.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.936+0000 7fad12e00700 1 -- 172.21.15.67:0/2766537124 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c1a2450 con 0x7fad0c107db0 2024-03-30T02:45:24.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:24 smithi170 ceph-mon[78798]: pgmap v257: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.9 MiB/s wr, 72 op/s 2024-03-30T02:45:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.950+0000 7fad0a000700 1 -- 172.21.15.67:0/2766537124 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fad0000f440 con 0x7fad0c107db0 2024-03-30T02:45:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.953+0000 7fad0a000700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7facf40a1340 0x7facf40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.953+0000 7fad27ee7b80 1 Processor -- start 2024-03-30T02:45:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.953+0000 7fad0a000700 1 -- 172.21.15.67:0/2766537124 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fad000cc9b0 con 0x7fad0c107db0 2024-03-30T02:45:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.954+0000 7fad11000700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7facf40a1340 0x7facf40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.955+0000 7fad27ee7b80 1 -- start start 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.955+0000 7fad27ee7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008077a0 0x556c00807b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.955+0000 7fad27ee7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008080c0 0x556c00812a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.955+0000 7fad27ee7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c008153b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.955+0000 7fad27ee7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c00760990 con 0x556c008080c0 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.955+0000 7fad27ee7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0075e350 con 0x556c008077a0 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.955+0000 7fad27ee7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c00759c40 con 0x556c00812fc0 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad11a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008077a0 0x556c00807b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad11a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008077a0 0x556c00807b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:53640/0 (socket says 172.21.15.67:53640) 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad11a00700 1 -- 172.21.15.67:0/2873476902 learned_addr learned my addr 172.21.15.67:0/2873476902 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad12400700 1 --2- 172.21.15.67:0/2873476902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c008153b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:24.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad11000700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7facf40a1340 0x7facf40a3800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facf80098e0 tx=0x7facf8000bc0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad11a00700 1 -- 172.21.15.67:0/2873476902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 msgr2=0x556c008153b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad11a00700 1 --2- 172.21.15.67:0/2873476902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c008153b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad11a00700 1 -- 172.21.15.67:0/2873476902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008080c0 msgr2=0x556c00812a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:45:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad11a00700 1 --2- 172.21.15.67:0/2873476902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008080c0 0x556c00812a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.956+0000 7fad11a00700 1 -- 172.21.15.67:0/2873476902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c005db3b0 con 0x556c008077a0 2024-03-30T02:45:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.958+0000 7fad11a00700 1 --2- 172.21.15.67:0/2873476902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008077a0 0x556c00807b80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fad00095d30 tx=0x7fad00099fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd019aa7349dde1b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.958+0000 7facfea00700 1 -- 172.21.15.67:0/2873476902 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000a1cb0 con 0x556c008077a0 2024-03-30T02:45:24.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.958+0000 7facfea00700 1 -- 172.21.15.67:0/2873476902 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad000b90b0 con 0x556c008077a0 2024-03-30T02:45:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.961+0000 7facfea00700 1 -- 172.21.15.67:0/2873476902 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00098f80 con 0x556c008077a0 2024-03-30T02:45:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.962+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2873476902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008077a0 msgr2=0x556c00807b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.962+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2873476902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008077a0 0x556c00807b80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fad00095d30 tx=0x7fad00099fb0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.965+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2873476902 shutdown_connections 2024-03-30T02:45:24.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.965+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2873476902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c008153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.965+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2873476902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008077a0 0x556c00807b80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.965+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2873476902 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008080c0 0x556c00812a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.965+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2873476902 >> 172.21.15.67:0/2873476902 conn(0x556c007676a0 msgr2=0x556c00803c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:24.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.965+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2873476902 shutdown_connections 2024-03-30T02:45:24.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.968+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2873476902 wait complete. 2024-03-30T02:45:24.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.968+0000 7fad27ee7b80 1 -- start start 2024-03-30T02:45:24.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.969+0000 7fad27ee7b80 1 Processor -- start 2024-03-30T02:45:24.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.970+0000 7fad27ee7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c008129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.971+0000 7fad11a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c008129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.971+0000 7fad11a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c008129f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56556/0 (socket says 172.21.15.67:56556) 2024-03-30T02:45:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.971+0000 7fad11a00700 1 -- 172.21.15.67:0/1395037781 learned_addr learned my addr 172.21.15.67:0/1395037781 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:24.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.971+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c0080fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.971+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00810420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.972+0000 7fad11000700 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c0080fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.972+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c006f5d80 con 0x556c008077a0 2024-03-30T02:45:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.972+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c006f5a20 con 0x556c008080c0 2024-03-30T02:45:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.972+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c005dd6a0 con 0x556c00812fc0 2024-03-30T02:45:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.972+0000 7fad12400700 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00810420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:24.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.973+0000 7fad12400700 1 -- 172.21.15.67:0/1395037781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 msgr2=0x556c0080fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:24.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.973+0000 7fad12400700 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c0080fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.973+0000 7fad12400700 1 -- 172.21.15.67:0/1395037781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 msgr2=0x556c008129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:24.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.973+0000 7fad27ee7b80 10 client.0 _ll_register_callbacks cb 0x556c006ff470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:24.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.973+0000 7fad12400700 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c008129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.973+0000 7fad12400700 1 -- 172.21.15.67:0/1395037781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c1a2450 con 0x556c00812fc0 2024-03-30T02:45:24.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.973+0000 7fad11a00700 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c008129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:45:24.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.974+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c0071a380 con 0x556c00812fc0 2024-03-30T02:45:24.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.974+0000 7fad12400700 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00810420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad04007ab0 tx=0x7fad04007ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9701653d5d6f038 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:24.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.974+0000 7facf3400700 10 client.27175 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:45:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.977+0000 7facf3400700 1 -- 172.21.15.67:0/1395037781 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0400e030 con 0x556c00812fc0 2024-03-30T02:45:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.977+0000 7facf3400700 1 -- 172.21.15.67:0/1395037781 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fad04016840 con 0x556c00812fc0 2024-03-30T02:45:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.978+0000 7facf3400700 1 -- 172.21.15.67:0/1395037781 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad04016a60 con 0x556c00812fc0 2024-03-30T02:45:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.978+0000 7fad27ee7b80 10 client.27175 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.978+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=41}) v3 -- 0x556c00587b60 con 0x556c00812fc0 2024-03-30T02:45:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.982+0000 7facf3400700 1 -- 172.21.15.67:0/1395037781 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fad04016bc0 con 0x556c00812fc0 2024-03-30T02:45:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.982+0000 7fad27ee7b80 10 client.27175 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.982+0000 7fad27ee7b80 10 client.27175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:45:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.982+0000 7fad27ee7b80 20 client.27175 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.982+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556c00822ad0 0x556c00824ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.982+0000 7fad27ee7b80 4 client.27175 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:45:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.982+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c00752700 con 0x556c00822ad0 2024-03-30T02:45:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.985+0000 7fad11a00700 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556c00822ad0 0x556c00824ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:24.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.986+0000 7facf3400700 10 client.27175 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:24.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.986+0000 7fad11a00700 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556c00822ad0 0x556c00824ef0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3566eb6c1a390b4b server_cookie=1909c5403cc3617b in_seq=0 out_seq=0 2024-03-30T02:45:24.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.986+0000 7facf3400700 1 -- 172.21.15.67:0/1395037781 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c00752700 con 0x556c00822ad0 2024-03-30T02:45:24.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.986+0000 7facf3400700 10 client.27175 handle_command_reply: tid=0 2024-03-30T02:45:24.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.987+0000 7fad27ee7b80 1 client.27175 shutdown 2024-03-30T02:45:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.988+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 msgr2=0x556c00810420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.988+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00810420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad04007ab0 tx=0x7fad04007ae0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.988+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 shutdown_connections 2024-03-30T02:45:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.988+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556c00822ad0 0x556c00824ef0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.988+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c008129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.988+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00810420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.988+0000 7facf3400700 0 client.27175 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.988+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/1395037781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c0080fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.988+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 >> 172.21.15.67:0/1395037781 conn(0x556c007676a0 msgr2=0x556c00809740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.989+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 shutdown_connections 2024-03-30T02:45:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.989+0000 7fad27ee7b80 1 -- 172.21.15.67:0/1395037781 wait complete. 2024-03-30T02:45:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:24.990+0000 7fad27ee7b80 20 client.27175 trim_cache size 0 max 16384 2024-03-30T02:45:25.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.031+0000 7fad27ee7b80 1 Processor -- start 2024-03-30T02:45:25.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.031+0000 7fad27ee7b80 1 -- start start 2024-03-30T02:45:25.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.032+0000 7fad27ee7b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c00819c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.032+0000 7fad27ee7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c0080a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.032+0000 7fad27ee7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00812600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.032+0000 7fad27ee7b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c00810060 con 0x556c008077a0 2024-03-30T02:45:25.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.032+0000 7fad27ee7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c00804830 con 0x556c008080c0 2024-03-30T02:45:25.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.032+0000 7fad27ee7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0081a180 con 0x556c00812fc0 2024-03-30T02:45:25.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.033+0000 7fad12400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00812600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.033+0000 7fad12400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00812600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52304/0 (socket says 172.21.15.67:52304) 2024-03-30T02:45:25.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.033+0000 7fad12400700 1 -- 172.21.15.67:0/924912585 learned_addr learned my addr 172.21.15.67:0/924912585 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:25.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.033+0000 7fad11000700 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c0080a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.033+0000 7fad12400700 1 -- 172.21.15.67:0/924912585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 msgr2=0x556c0080a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:25.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.033+0000 7fad12400700 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c0080a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.033+0000 7fad12400700 1 -- 172.21.15.67:0/924912585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 msgr2=0x556c00819c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:25.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.033+0000 7fad11a00700 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c00819c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.034+0000 7fad12400700 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c00819c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.034+0000 7fad12400700 1 -- 172.21.15.67:0/924912585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c0071fea0 con 0x556c00812fc0 2024-03-30T02:45:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.034+0000 7fad11000700 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c0080a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.034+0000 7fad11a00700 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c00819c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:25.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.035+0000 7fad12400700 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00812600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fad04027ff0 tx=0x7fad04010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47cc44aa48302762 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:25.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.037+0000 7facff400700 1 -- 172.21.15.67:0/924912585 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0400e070 con 0x556c00812fc0 2024-03-30T02:45:25.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.037+0000 7facff400700 1 -- 172.21.15.67:0/924912585 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c0071a380 con 0x556c00812fc0 2024-03-30T02:45:25.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.037+0000 7facff400700 1 -- 172.21.15.67:0/924912585 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0401eae0 con 0x556c00812fc0 2024-03-30T02:45:25.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.039+0000 7fad27ee7b80 1 -- 172.21.15.67:0/924912585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 msgr2=0x556c00812600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:25.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.039+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00812600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fad04027ff0 tx=0x7fad04010000 comp rx=0 tx=0).stop 2024-03-30T02:45:25.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.039+0000 7fad27ee7b80 1 -- 172.21.15.67:0/924912585 shutdown_connections 2024-03-30T02:45:25.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.039+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c008077a0 0x556c00819c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.039+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c00812fc0 0x556c00812600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.039+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/924912585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c0080a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.039+0000 7fad27ee7b80 1 -- 172.21.15.67:0/924912585 >> 172.21.15.67:0/924912585 conn(0x556c007676a0 msgr2=0x556c0080b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.039+0000 7fad27ee7b80 1 -- 172.21.15.67:0/924912585 shutdown_connections 2024-03-30T02:45:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.039+0000 7fad27ee7b80 1 -- 172.21.15.67:0/924912585 wait complete. 2024-03-30T02:45:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.040+0000 7fad27ee7b80 1 -- start start 2024-03-30T02:45:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.040+0000 7fad27ee7b80 1 Processor -- start 2024-03-30T02:45:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.041+0000 7fad27ee7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c008077a0 0x556c00819b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.042+0000 7fad27ee7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c00818040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.042+0000 7fad11a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c008077a0 0x556c00819b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.042+0000 7fad11a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c008077a0 0x556c00819b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52308/0 (socket says 172.21.15.67:52308) 2024-03-30T02:45:25.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.042+0000 7fad11a00700 1 -- 172.21.15.67:0/2652384617 learned_addr learned my addr 172.21.15.67:0/2652384617 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:25.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.043+0000 7fad11000700 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c00818040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.043+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c00812fc0 0x556c008185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.044+0000 7fad12400700 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c00812fc0 0x556c008185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.044+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c00818cf0 con 0x556c00812fc0 2024-03-30T02:45:25.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.044+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c00818e60 con 0x556c008080c0 2024-03-30T02:45:25.044 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.044+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c008768d0 con 0x556c008077a0 2024-03-30T02:45:25.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.045+0000 7fad12400700 1 -- 172.21.15.67:0/2652384617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c008077a0 msgr2=0x556c00819b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.045+0000 7fad12400700 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c008077a0 0x556c00819b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.045+0000 7fad12400700 1 -- 172.21.15.67:0/2652384617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 msgr2=0x556c00818040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.045+0000 7fad12400700 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c00818040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.045+0000 7fad12400700 1 -- 172.21.15.67:0/2652384617 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c00587b60 con 0x556c00812fc0 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.046+0000 7fad12400700 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c00812fc0 0x556c008185e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x556c00810e30 tx=0x7fad04016c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11afe5d91f76cf85 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.046+0000 7facf3400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.046+0000 7facf3400700 1 -- 172.21.15.67:0/2652384617 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0400e040 con 0x556c00812fc0 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.046+0000 7facf3400700 1 -- 172.21.15.67:0/2652384617 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fad0401e8c0 con 0x556c00812fc0 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.046+0000 7fad27ee7b80 10 client.0 _ll_register_callbacks cb 0x556c006ff470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.047+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad000b90b0 con 0x556c00812fc0 2024-03-30T02:45:25.049 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/group_cpus.o 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7facf3400700 1 -- 172.21.15.67:0/2652384617 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad0401e4a0 con 0x556c00812fc0 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7fad27ee7b80 10 client.17103 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=41}) v3 -- 0x556c005dfcd0 con 0x556c00812fc0 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7facf3400700 1 -- 172.21.15.67:0/2652384617 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fad0403c050 con 0x556c00812fc0 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7fad27ee7b80 10 client.17103 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7fad27ee7b80 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7fad27ee7b80 20 client.17103 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556c008776d0 0x556c00879b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7fad27ee7b80 4 client.17103 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:45:25.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.049+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fad000b90b0 con 0x556c008776d0 2024-03-30T02:45:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.050+0000 7fad11a00700 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556c008776d0 0x556c00879b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.051+0000 7facf3400700 10 client.17103 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.051+0000 7fad11a00700 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556c008776d0 0x556c00879b90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50dee35eeec8b003 server_cookie=3e3221cfa1cbf939 in_seq=0 out_seq=0 2024-03-30T02:45:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.051+0000 7facf3400700 1 -- 172.21.15.67:0/2652384617 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fad000b90b0 con 0x556c008776d0 2024-03-30T02:45:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.051+0000 7facf3400700 10 client.17103 handle_command_reply: tid=0 2024-03-30T02:45:25.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.051+0000 7fad27ee7b80 1 client.17103 shutdown 2024-03-30T02:45:25.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.052+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c00812fc0 msgr2=0x556c008185e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:25.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.052+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c00812fc0 0x556c008185e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x556c00810e30 tx=0x7fad04016c90 comp rx=0 tx=0).stop 2024-03-30T02:45:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.052+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 shutdown_connections 2024-03-30T02:45:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.052+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556c008776d0 0x556c00879b90 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.052+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556c00812fc0 0x556c008185e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.052+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556c008077a0 0x556c00819b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.052+0000 7fad27ee7b80 1 --2- 172.21.15.67:0/2652384617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c008080c0 0x556c00818040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.052+0000 7facf3400700 0 client.17103 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.052+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 >> 172.21.15.67:0/2652384617 conn(0x556c007676a0 msgr2=0x556c0080f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.055+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 shutdown_connections 2024-03-30T02:45:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.055+0000 7fad27ee7b80 1 -- 172.21.15.67:0/2652384617 wait complete. 2024-03-30T02:45:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.055+0000 7fad27ee7b80 20 client.17103 trim_cache size 0 max 16384 2024-03-30T02:45:25.056 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:45:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:45:25.056 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:45:25.056 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:45:25.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.060+0000 7fad12e00700 1 -- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7facf40a1340 msgr2=0x7facf40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:25.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.060+0000 7fad12e00700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7facf40a1340 0x7facf40a3800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facf80098e0 tx=0x7facf8000bc0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.060+0000 7fad12e00700 1 -- 172.21.15.67:0/2766537124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c107db0 msgr2=0x7fad0c1a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:25.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.060+0000 7fad12e00700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c107db0 0x7fad0c1a4400 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad0000b550 tx=0x7fad0000b910 comp rx=0 tx=0).stop 2024-03-30T02:45:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.062+0000 7fad12e00700 1 -- 172.21.15.67:0/2766537124 shutdown_connections 2024-03-30T02:45:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.062+0000 7fad12e00700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7facf40a1340 0x7facf40a3800 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.062+0000 7fad12e00700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad0c10d630 0x7fad0c19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.062+0000 7fad12e00700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad0c107db0 0x7fad0c1a4400 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.062+0000 7fad12e00700 1 --2- 172.21.15.67:0/2766537124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad0c108770 0x7fad0c1a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.062+0000 7fad12e00700 1 -- 172.21.15.67:0/2766537124 >> 172.21.15.67:0/2766537124 conn(0x7fad0c06bc10 msgr2=0x7fad0c071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.062+0000 7fad12e00700 1 -- 172.21.15.67:0/2766537124 shutdown_connections 2024-03-30T02:45:25.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.062+0000 7fad12e00700 1 -- 172.21.15.67:0/2766537124 wait complete. 2024-03-30T02:45:25.098 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:45:25.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:45:25.320 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/utsname.o 2024-03-30T02:45:25.395 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/lib.a 2024-03-30T02:45:25.542 INFO:tasks.workunit.client.0.smithi067.stdout: GEN lib/crc32table.h 2024-03-30T02:45:25.567 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/oid_registry.o 2024-03-30T02:45:25.579 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/pid_namespace.o 2024-03-30T02:45:25.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.597+0000 7f016c200700 1 -- 172.21.15.67:0/3756958692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 msgr2=0x7f0164107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:25.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.597+0000 7f016c200700 1 --2- 172.21.15.67:0/3756958692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 0x7f0164107db0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f01500098f0 tx=0x7f0150009c00 comp rx=0 tx=0).stop 2024-03-30T02:45:25.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.599+0000 7f016c200700 1 -- 172.21.15.67:0/3756958692 shutdown_connections 2024-03-30T02:45:25.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.599+0000 7f016c200700 1 --2- 172.21.15.67:0/3756958692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 0x7f0164107db0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.599+0000 7f016c200700 1 --2- 172.21.15.67:0/3756958692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0164113af0 0x7f0164115ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.599+0000 7f016c200700 1 --2- 172.21.15.67:0/3756958692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164108380 0x7f0164108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.599+0000 7f016c200700 1 -- 172.21.15.67:0/3756958692 >> 172.21.15.67:0/3756958692 conn(0x7f016406bcf0 msgr2=0x7f016406c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:25.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.603+0000 7f016c200700 1 -- 172.21.15.67:0/3756958692 shutdown_connections 2024-03-30T02:45:25.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.603+0000 7f016c200700 1 -- 172.21.15.67:0/3756958692 wait complete. 2024-03-30T02:45:25.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.610+0000 7f016c200700 1 Processor -- start 2024-03-30T02:45:25.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.611+0000 7f016c200700 1 -- start start 2024-03-30T02:45:25.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.612+0000 7f016c200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 0x7f01641a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.612+0000 7f016c200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0164108380 0x7f01641a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.612+0000 7f016c200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164113af0 0x7f016419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.613+0000 7f0169800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 0x7f01641a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.613+0000 7f0169800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 0x7f01641a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56590/0 (socket says 172.21.15.67:56590) 2024-03-30T02:45:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.613+0000 7f0168e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0164108380 0x7f01641a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.613+0000 7f0168e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0164108380 0x7f01641a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:52332/0 (socket says 172.21.15.67:52332) 2024-03-30T02:45:25.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.613+0000 7f016a200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164113af0 0x7f016419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.613+0000 7f016a200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164113af0 0x7f016419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:53674/0 (socket says 172.21.15.67:53674) 2024-03-30T02:45:25.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.614+0000 7f0169800700 1 -- 172.21.15.67:0/1450141238 learned_addr learned my addr 172.21.15.67:0/1450141238 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:25.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.614+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419fe40 con 0x7f016406d100 2024-03-30T02:45:25.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.614+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419ffb0 con 0x7f0164113af0 2024-03-30T02:45:25.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.615+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01641a0170 con 0x7f0164108380 2024-03-30T02:45:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.615+0000 7f016a200700 1 -- 172.21.15.67:0/1450141238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0164108380 msgr2=0x7f01641a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.615+0000 7f016a200700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0164108380 0x7f01641a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.616+0000 7f016a200700 1 -- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 msgr2=0x7f01641a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.616+0000 7f016a200700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 0x7f01641a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.616+0000 7f016a200700 1 -- 172.21.15.67:0/1450141238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150009520 con 0x7f0164113af0 2024-03-30T02:45:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.616+0000 7f0169800700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 0x7f01641a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:45:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.616+0000 7f016a200700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164113af0 0x7f016419f7d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f015c007fd0 tx=0x7f015c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.620+0000 7f0162000700 1 -- 172.21.15.67:0/1450141238 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c004bb0 con 0x7f0164113af0 2024-03-30T02:45:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.620+0000 7f0162000700 1 -- 172.21.15.67:0/1450141238 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f015c00f040 con 0x7f0164113af0 2024-03-30T02:45:25.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.620+0000 7f0162000700 1 -- 172.21.15.67:0/1450141238 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015c005800 con 0x7f0164113af0 2024-03-30T02:45:25.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.622+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01641a0450 con 0x7f0164113af0 2024-03-30T02:45:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.622+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0164109930 con 0x7f0164113af0 2024-03-30T02:45:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.627+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01641a05e0 con 0x7f0164113af0 2024-03-30T02:45:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.630+0000 7f0162000700 1 -- 172.21.15.67:0/1450141238 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f015c00db20 con 0x7f0164113af0 2024-03-30T02:45:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.632+0000 7f0162000700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f014c0a1310 0x7f014c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.633+0000 7f0162000700 1 -- 172.21.15.67:0/1450141238 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f015c0cc840 con 0x7f0164113af0 2024-03-30T02:45:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.634+0000 7f0169800700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f014c0a1310 0x7f014c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.635+0000 7f0169800700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f014c0a1310 0x7f014c0a37d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f015000b2e0 tx=0x7f0150005de0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:25.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:25.636+0000 7f0162000700 1 -- 172.21.15.67:0/1450141238 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f015c0958f0 con 0x7f0164113af0 2024-03-30T02:45:25.730 INFO:tasks.workunit.client.0.smithi067.stdout: CC lib/crc32.o 2024-03-30T02:45:25.745 INFO:tasks.workunit.client.0.smithi067.stdout: CC fs/fhandle.o 2024-03-30T02:45:25.836 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/stop_machine.o 2024-03-30T02:45:25.897 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/audit.o 2024-03-30T02:45:25.925 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/auditfilter.o 2024-03-30T02:45:25.928 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/auditsc.o 2024-03-30T02:45:26.160 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/audit_watch.o 2024-03-30T02:45:26.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.218+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01640679a0 con 0x7f0164113af0 2024-03-30T02:45:26.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.223+0000 7f0162000700 1 -- 172.21.15.67:0/1450141238 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f015c098f10 con 0x7f0164113af0 2024-03-30T02:45:26.222 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:45:26.222 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:45:07.400846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:45:26.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.233+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f014c0a1310 msgr2=0x7f014c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.233+0000 7f016c200700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f014c0a1310 0x7f014c0a37d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f015000b2e0 tx=0x7f0150005de0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.236+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164113af0 msgr2=0x7f016419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.236+0000 7f016c200700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164113af0 0x7f016419f7d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f015c007fd0 tx=0x7f015c00d3b0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.236+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 shutdown_connections 2024-03-30T02:45:26.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.236+0000 7f016c200700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f014c0a1310 0x7f014c0a37d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.236+0000 7f016c200700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f016406d100 0x7f01641a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.236+0000 7f016c200700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164113af0 0x7f016419f7d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.236+0000 7f016c200700 1 --2- 172.21.15.67:0/1450141238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0164108380 0x7f01641a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.236+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 >> 172.21.15.67:0/1450141238 conn(0x7f016406bcf0 msgr2=0x7f016410b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:26.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.238+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 shutdown_connections 2024-03-30T02:45:26.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.238+0000 7f016c200700 1 -- 172.21.15.67:0/1450141238 wait complete. 2024-03-30T02:45:26.243 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-03-30T02:45:26.309 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:45:26.450 INFO:tasks.workunit.client.0.smithi067.stdout: AR lib/built-in.a 2024-03-30T02:45:26.464 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/audit_fsnotify.o 2024-03-30T02:45:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:26 smithi067 ceph-mon[37808]: pgmap v258: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.7 MiB/s wr, 65 op/s 2024-03-30T02:45:26.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:26 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1450141238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:45:26.689 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/audit_tree.o 2024-03-30T02:45:26.730 INFO:tasks.workunit.client.0.smithi067.stdout: AR fs/built-in.a 2024-03-30T02:45:26.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.759+0000 7f34ba000700 1 -- 172.21.15.67:0/3602701325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 msgr2=0x7f34b41145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.759+0000 7f34ba000700 1 --2- 172.21.15.67:0/3602701325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 0x7f34b41145c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009f90 tx=0x7f34ac00b330 comp rx=0 tx=0).stop 2024-03-30T02:45:26.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.760+0000 7f34ba000700 1 -- 172.21.15.67:0/3602701325 shutdown_connections 2024-03-30T02:45:26.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.760+0000 7f34ba000700 1 --2- 172.21.15.67:0/3602701325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 0x7f34b41145c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.760+0000 7f34ba000700 1 --2- 172.21.15.67:0/3602701325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b410d770 0x7f34b410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.760+0000 7f34ba000700 1 --2- 172.21.15.67:0/3602701325 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b4107db0 0x7f34b4108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.760+0000 7f34ba000700 1 -- 172.21.15.67:0/3602701325 >> 172.21.15.67:0/3602701325 conn(0x7f34b406bdf0 msgr2=0x7f34b406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.760+0000 7f34ba000700 1 -- 172.21.15.67:0/3602701325 shutdown_connections 2024-03-30T02:45:26.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.760+0000 7f34ba000700 1 -- 172.21.15.67:0/3602701325 wait complete. 2024-03-30T02:45:26.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.762+0000 7f34ba000700 1 Processor -- start 2024-03-30T02:45:26.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.762+0000 7f34ba000700 1 -- start start 2024-03-30T02:45:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34ba000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b4107db0 0x7f34b41a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34ba000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 0x7f34b41a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34ba000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b410d770 0x7f34b419d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.762 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34b2a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 0x7f34b41a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34ba000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b419ddd0 con 0x7f34b4107db0 2024-03-30T02:45:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34b2a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 0x7f34b41a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56686/0 (socket says 172.21.15.67:56686) 2024-03-30T02:45:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34b3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b410d770 0x7f34b419d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34b2a00700 1 -- 172.21.15.67:0/321914373 learned_addr learned my addr 172.21.15.67:0/321914373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34ba000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b419df40 con 0x7f34b410d770 2024-03-30T02:45:26.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.763+0000 7f34ba000700 1 -- 172.21.15.67:0/321914373 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b419e100 con 0x7f34b4108770 2024-03-30T02:45:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.764+0000 7f34b2a00700 1 -- 172.21.15.67:0/321914373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b410d770 msgr2=0x7f34b419d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.764+0000 7f34b2a00700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b410d770 0x7f34b419d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.764+0000 7f34b2a00700 1 -- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b4107db0 msgr2=0x7f34b41a3640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T02:45:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.764+0000 7f34b3400700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b4107db0 0x7f34b41a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.764+0000 7f34b2a00700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b4107db0 0x7f34b41a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.765+0000 7f34b2a00700 1 -- 172.21.15.67:0/321914373 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a40071c0 con 0x7f34b4108770 2024-03-30T02:45:26.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.765+0000 7f34b3400700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b4107db0 0x7f34b41a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:26.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.766+0000 7f34b2a00700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 0x7f34b41a3b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f34a000e9a0 tx=0x7f34a000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:26.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.767+0000 7f34abe00700 1 -- 172.21.15.67:0/321914373 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0009790 con 0x7f34b4108770 2024-03-30T02:45:26.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.767+0000 7f34abe00700 1 -- 172.21.15.67:0/321914373 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34a000cbd0 con 0x7f34b4108770 2024-03-30T02:45:26.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.767+0000 7f34abe00700 1 -- 172.21.15.67:0/321914373 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0018680 con 0x7f34b4108770 2024-03-30T02:45:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.768+0000 7f34ba000700 1 -- 172.21.15.67:0/321914373 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac009c40 con 0x7f34b4108770 2024-03-30T02:45:26.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.768+0000 7f34ba000700 1 -- 172.21.15.67:0/321914373 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b41aa450 con 0x7f34b4108770 2024-03-30T02:45:26.769 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/kprobes.o 2024-03-30T02:45:26.773 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:26 smithi082 ceph-mon[79176]: pgmap v258: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.7 MiB/s wr, 65 op/s 2024-03-30T02:45:26.774 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:26 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1450141238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:45:26.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.775+0000 7f34abe00700 1 -- 172.21.15.67:0/321914373 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f34a000cd40 con 0x7f34b4108770 2024-03-30T02:45:26.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.776+0000 7f34cdbb2b80 1 Processor -- start 2024-03-30T02:45:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.777+0000 7f34cdbb2b80 1 -- start start 2024-03-30T02:45:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.777+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf845d0 0x5577abf849b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.777+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 0x5577abf8f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.777+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf8fdf0 0x5577abf921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.777+0000 7f34cdbb2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abedd990 con 0x5577abf84ef0 2024-03-30T02:45:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.778+0000 7f34cdbb2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abedb350 con 0x5577abf845d0 2024-03-30T02:45:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.778+0000 7f34cdbb2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abed6c40 con 0x5577abf8fdf0 2024-03-30T02:45:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.778+0000 7f34b3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf845d0 0x5577abf849b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.778+0000 7f34b3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf845d0 0x5577abf849b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39322/0 (socket says 172.21.15.67:39322) 2024-03-30T02:45:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.778+0000 7f34b3e00700 1 -- 172.21.15.67:0/1764710127 learned_addr learned my addr 172.21.15.67:0/1764710127 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.778+0000 7f34b3e00700 1 -- 172.21.15.67:0/1764710127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf8fdf0 msgr2=0x5577abf921e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:45:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.778+0000 7f34b3e00700 1 --2- 172.21.15.67:0/1764710127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf8fdf0 0x5577abf921e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.779+0000 7f34b3e00700 1 -- 172.21.15.67:0/1764710127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 msgr2=0x5577abf8f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.779+0000 7f34b3400700 1 --2- 172.21.15.67:0/1764710127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 0x5577abf8f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.779+0000 7f34b3e00700 1 --2- 172.21.15.67:0/1764710127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 0x5577abf8f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.779+0000 7f34b3e00700 1 -- 172.21.15.67:0/1764710127 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577abd583b0 con 0x5577abf845d0 2024-03-30T02:45:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.779+0000 7f34b3400700 1 --2- 172.21.15.67:0/1764710127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 0x5577abf8f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.779+0000 7f34b3e00700 1 --2- 172.21.15.67:0/1764710127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf845d0 0x5577abf849b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009540 tx=0x7f34ac009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd05da7ea282f4cb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.780+0000 7f34abe00700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f34980a5840 0x7f34980a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.780+0000 7f349fe00700 1 -- 172.21.15.67:0/1764710127 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00e040 con 0x5577abf845d0 2024-03-30T02:45:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.780+0000 7f34abe00700 1 -- 172.21.15.67:0/321914373 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f34a0014070 con 0x7f34b4108770 2024-03-30T02:45:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.780+0000 7f349fe00700 1 -- 172.21.15.67:0/1764710127 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ac0077e0 con 0x5577abf845d0 2024-03-30T02:45:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.780+0000 7f349fe00700 1 -- 172.21.15.67:0/1764710127 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00e040 con 0x5577abf845d0 2024-03-30T02:45:26.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.781+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/1764710127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf845d0 msgr2=0x5577abf849b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.781+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/1764710127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf845d0 0x5577abf849b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009540 tx=0x7f34ac009ac0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.783+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/1764710127 shutdown_connections 2024-03-30T02:45:26.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.783+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/1764710127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf8fdf0 0x5577abf921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.783+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/1764710127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf845d0 0x5577abf849b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.783+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/1764710127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 0x5577abf8f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.783+0000 7f34b3400700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f34980a5840 0x7f34980a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.783+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/1764710127 >> 172.21.15.67:0/1764710127 conn(0x5577abee46a0 msgr2=0x5577abf80a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:26.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.784+0000 7f34b3400700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f34980a5840 0x7f34980a7d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f34a4005e50 tx=0x7f34a4009040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:26.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.784+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/1764710127 shutdown_connections 2024-03-30T02:45:26.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.785+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/1764710127 wait complete. 2024-03-30T02:45:26.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.788+0000 7f34cdbb2b80 1 -- start start 2024-03-30T02:45:26.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.788+0000 7f34cdbb2b80 1 Processor -- start 2024-03-30T02:45:26.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.789+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf8f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.789+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 0x5577abf8c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.789+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 0x5577abf8cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.790+0000 7f34b3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf8f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.791+0000 7f34b2a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 0x5577abf8c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.791+0000 7f34b2a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 0x5577abf8c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56704/0 (socket says 172.21.15.67:56704) 2024-03-30T02:45:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.791+0000 7f34b3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf8f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44302/0 (socket says 172.21.15.67:44302) 2024-03-30T02:45:26.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.792+0000 7f34b3e00700 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 0x5577abf8cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.792+0000 7f34b2a00700 1 -- 172.21.15.67:0/3872848793 learned_addr learned my addr 172.21.15.67:0/3872848793 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.792+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abe72d80 con 0x5577abf845d0 2024-03-30T02:45:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.792+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abe72a20 con 0x5577abf8fdf0 2024-03-30T02:45:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.792+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abd5a6a0 con 0x5577abf84ef0 2024-03-30T02:45:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.792+0000 7f34cdbb2b80 10 client.0 _ll_register_callbacks cb 0x5577abe7c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.792+0000 7f34b3e00700 1 -- 172.21.15.67:0/3872848793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 msgr2=0x5577abf8c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.792+0000 7f34b3e00700 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 0x5577abf8c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.792+0000 7f34b3e00700 1 -- 172.21.15.67:0/3872848793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 msgr2=0x5577abf8f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.793+0000 7f34b2a00700 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 0x5577abf8c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-30T02:45:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.793+0000 7f34b3e00700 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf8f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.793+0000 7f34b3e00700 1 -- 172.21.15.67:0/3872848793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b4114eb0 con 0x5577abf8fdf0 2024-03-30T02:45:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.793+0000 7f34b3400700 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf8f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:45:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.794+0000 7f349cc00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:45:26.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.794+0000 7f34b3e00700 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 0x5577abf8cf00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00d010 tx=0x7f34ac009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9fe930008d249f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:26.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.795+0000 7f349cc00700 1 -- 172.21.15.67:0/3872848793 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00e040 con 0x5577abf8fdf0 2024-03-30T02:45:26.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.795+0000 7f349cc00700 1 -- 172.21.15.67:0/3872848793 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f34ac02a070 con 0x5577abf8fdf0 2024-03-30T02:45:26.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.796+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577abe97380 con 0x5577abf8fdf0 2024-03-30T02:45:26.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.796+0000 7f349cc00700 1 -- 172.21.15.67:0/3872848793 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34ac003dd0 con 0x5577abf8fdf0 2024-03-30T02:45:26.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.796+0000 7f34cdbb2b80 10 client.27242 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:26.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.797+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5577abd04b60 con 0x5577abf8fdf0 2024-03-30T02:45:26.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.798+0000 7f349cc00700 1 -- 172.21.15.67:0/3872848793 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ac028070 con 0x5577abf8fdf0 2024-03-30T02:45:26.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.799+0000 7f349cc00700 1 -- 172.21.15.67:0/3872848793 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34ac019070 con 0x5577abf8fdf0 2024-03-30T02:45:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.800+0000 7f34cdbb2b80 10 client.27242 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.800+0000 7f34cdbb2b80 10 client.27242 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:45:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.800+0000 7f34cdbb2b80 20 client.27242 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.800+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5577abfa7b60 0x5577abfa9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.800+0000 7f34cdbb2b80 4 client.27242 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:45:26.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.800+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577abe97380 con 0x5577abfa7b60 2024-03-30T02:45:26.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.801+0000 7f34b3400700 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5577abfa7b60 0x5577abfa9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.802+0000 7f34b3400700 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5577abfa7b60 0x5577abfa9f80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2cc538eddd25736 server_cookie=eedfb18a0486b70d in_seq=0 out_seq=0 2024-03-30T02:45:26.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.802+0000 7f349cc00700 10 client.27242 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:45:26.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.803+0000 7f349cc00700 1 -- 172.21.15.67:0/3872848793 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5577abe97380 con 0x5577abfa7b60 2024-03-30T02:45:26.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.803+0000 7f349cc00700 10 client.27242 handle_command_reply: tid=0 2024-03-30T02:45:26.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.803+0000 7f34cdbb2b80 1 client.27242 shutdown 2024-03-30T02:45:26.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.804+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 msgr2=0x5577abf8cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.804+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 0x5577abf8cf00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00d010 tx=0x7f34ac009920 comp rx=0 tx=0).stop 2024-03-30T02:45:26.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.804+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 shutdown_connections 2024-03-30T02:45:26.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.804+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf8f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.804+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5577abfa7b60 0x5577abfa9f80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.804+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 0x5577abf8cf00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.804+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/3872848793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 0x5577abf8c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.804+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 >> 172.21.15.67:0/3872848793 conn(0x5577abee46a0 msgr2=0x5577abf86530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:26.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.804+0000 7f349cc00700 0 client.27242 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:45:26.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.805+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 shutdown_connections 2024-03-30T02:45:26.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.805+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/3872848793 wait complete. 2024-03-30T02:45:26.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.806+0000 7f34cdbb2b80 20 client.27242 trim_cache size 0 max 16384 2024-03-30T02:45:26.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.843+0000 7f34cdbb2b80 1 Processor -- start 2024-03-30T02:45:26.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.843+0000 7f34cdbb2b80 1 -- start start 2024-03-30T02:45:26.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.844+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf87340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.844+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 0x5577abf9f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.844+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 0x5577abfa3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.844+0000 7f34cdbb2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abf8fc30 con 0x5577abf845d0 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.844+0000 7f34cdbb2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abf86110 con 0x5577abf8fdf0 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.844+0000 7f34cdbb2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abf9f870 con 0x5577abf84ef0 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.845+0000 7f34b3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf87340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.845+0000 7f34b3400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf87340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44306/0 (socket says 172.21.15.67:44306) 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.845+0000 7f34b3400700 1 -- 172.21.15.67:0/4227229570 learned_addr learned my addr 172.21.15.67:0/4227229570 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.845+0000 7f34b3400700 1 -- 172.21.15.67:0/4227229570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 msgr2=0x5577abf9f330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.845+0000 7f34b3400700 1 --2- 172.21.15.67:0/4227229570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 0x5577abf9f330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.845+0000 7f34b3400700 1 -- 172.21.15.67:0/4227229570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 msgr2=0x5577abfa3a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.845+0000 7f34b3400700 1 --2- 172.21.15.67:0/4227229570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 0x5577abfa3a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.845+0000 7f34b3400700 1 -- 172.21.15.67:0/4227229570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577abe9cea0 con 0x5577abf845d0 2024-03-30T02:45:26.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.846+0000 7f34b3400700 1 --2- 172.21.15.67:0/4227229570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf87340 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f34a400c010 tx=0x7f34a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07c38c534f88de0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:26.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.847+0000 7f34a9200700 1 -- 172.21.15.67:0/4227229570 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4020040 con 0x5577abf845d0 2024-03-30T02:45:26.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.847+0000 7f34a9200700 1 -- 172.21.15.67:0/4227229570 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34a4012040 con 0x5577abf845d0 2024-03-30T02:45:26.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.847+0000 7f34a9200700 1 -- 172.21.15.67:0/4227229570 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4020040 con 0x5577abf845d0 2024-03-30T02:45:26.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.848+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/4227229570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 msgr2=0x5577abf87340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.848+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/4227229570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf87340 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f34a400c010 tx=0x7f34a4004750 comp rx=0 tx=0).stop 2024-03-30T02:45:26.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.848+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/4227229570 shutdown_connections 2024-03-30T02:45:26.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.848+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/4227229570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf845d0 0x5577abf87340 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.848+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/4227229570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf8fdf0 0x5577abfa3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.848+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/4227229570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf84ef0 0x5577abf9f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.848+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/4227229570 >> 172.21.15.67:0/4227229570 conn(0x5577abee46a0 msgr2=0x5577abf87c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:26.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.848+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/4227229570 shutdown_connections 2024-03-30T02:45:26.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.849+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/4227229570 wait complete. 2024-03-30T02:45:26.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.849+0000 7f34cdbb2b80 1 -- start start 2024-03-30T02:45:26.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.849+0000 7f34cdbb2b80 1 Processor -- start 2024-03-30T02:45:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.850+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 0x5577abf97000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.850+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf8fdf0 0x5577abf97540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.850+0000 7f34cdbb2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf9b490 0x5577abf9b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.850+0000 7f34cdbb2b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abf9be50 con 0x5577abf84ef0 2024-03-30T02:45:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.850+0000 7f34cdbb2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abf9bfc0 con 0x5577abf9b490 2024-03-30T02:45:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.850+0000 7f34cdbb2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577abf9c180 con 0x5577abf8fdf0 2024-03-30T02:45:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.850+0000 7f34cdbb2b80 10 client.0 _ll_register_callbacks cb 0x5577abe7c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.851+0000 7f34b3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf9b490 0x5577abf9b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.851+0000 7f34b3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf9b490 0x5577abf9b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39330/0 (socket says 172.21.15.67:39330) 2024-03-30T02:45:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.851+0000 7f34b3e00700 1 -- 172.21.15.67:0/2549877402 learned_addr learned my addr 172.21.15.67:0/2549877402 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.851+0000 7f34b3e00700 1 -- 172.21.15.67:0/2549877402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf8fdf0 msgr2=0x5577abf97540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:45:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.851+0000 7f34b3e00700 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf8fdf0 0x5577abf97540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.851+0000 7f34b3e00700 1 -- 172.21.15.67:0/2549877402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 msgr2=0x5577abf97000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:45:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.852+0000 7f34b3400700 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 0x5577abf97000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.852+0000 7f34b3e00700 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 0x5577abf97000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.852+0000 7f34b3e00700 1 -- 172.21.15.67:0/2549877402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a4007c50 con 0x5577abf9b490 2024-03-30T02:45:26.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.854+0000 7f349cc00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:45:26.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.854+0000 7f34b3e00700 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf9b490 0x5577abf9b910 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009320 tx=0x7f34ac008fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e5e91f3cc0e70e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:26.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.854+0000 7f349cc00700 1 -- 172.21.15.67:0/2549877402 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00e070 con 0x5577abf9b490 2024-03-30T02:45:26.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.854+0000 7f349cc00700 1 -- 172.21.15.67:0/2549877402 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f34ac02a040 con 0x5577abf9b490 2024-03-30T02:45:26.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.854+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/2549877402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34ac0077e0 con 0x5577abf9b490 2024-03-30T02:45:26.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f349cc00700 1 -- 172.21.15.67:0/2549877402 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34ac00e070 con 0x5577abf9b490 2024-03-30T02:45:26.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f34cdbb2b80 10 client.27245 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:26.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/2549877402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5577abd5ccd0 con 0x5577abf9b490 2024-03-30T02:45:26.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f349cc00700 1 -- 172.21.15.67:0/2549877402 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ac03f020 con 0x5577abf9b490 2024-03-30T02:45:26.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f349cc00700 1 -- 172.21.15.67:0/2549877402 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34ac00b050 con 0x5577abf9b490 2024-03-30T02:45:26.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f34cdbb2b80 10 client.27245 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:26.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f34cdbb2b80 10 client.27245 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:45:26.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f34cdbb2b80 20 client.27245 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:26.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5577abff48b0 0x5577abff6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:26.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f34cdbb2b80 4 client.27245 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:45:26.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.855+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/2549877402 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f34ac0077e0 con 0x5577abff48b0 2024-03-30T02:45:26.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.856+0000 7f34b3400700 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5577abff48b0 0x5577abff6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:26.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.857+0000 7f349cc00700 10 client.27245 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:45:26.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.857+0000 7f34b3400700 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5577abff48b0 0x5577abff6cd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=94b3b58af1dbc44 server_cookie=78fcf82e6c214d4a in_seq=0 out_seq=0 2024-03-30T02:45:26.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.858+0000 7f349cc00700 1 -- 172.21.15.67:0/2549877402 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f34ac0077e0 con 0x5577abff48b0 2024-03-30T02:45:26.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.858+0000 7f349cc00700 10 client.27245 handle_command_reply: tid=0 2024-03-30T02:45:26.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.858+0000 7f34cdbb2b80 1 client.27245 shutdown 2024-03-30T02:45:26.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.858+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/2549877402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf9b490 msgr2=0x5577abf9b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.858+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf9b490 0x5577abf9b910 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f34ac009320 tx=0x7f34ac008fd0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.859+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/2549877402 shutdown_connections 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.859+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5577abf84ef0 0x5577abf97000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.859+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5577abff48b0 0x5577abff6cd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.859+0000 7f349cc00700 0 client.27245 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.859+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5577abf9b490 0x5577abf9b910 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.859+0000 7f34cdbb2b80 1 --2- 172.21.15.67:0/2549877402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577abf8fdf0 0x5577abf97540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.859+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/2549877402 >> 172.21.15.67:0/2549877402 conn(0x5577abee46a0 msgr2=0x5577abfa3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.859+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/2549877402 shutdown_connections 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.859+0000 7f34cdbb2b80 1 -- 172.21.15.67:0/2549877402 wait complete. 2024-03-30T02:45:26.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.860+0000 7f34cdbb2b80 20 client.27245 trim_cache size 0 max 16384 2024-03-30T02:45:26.860 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:45:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.861+0000 7f34ba000700 1 -- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f34980a5840 msgr2=0x7f34980a7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.861+0000 7f34ba000700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f34980a5840 0x7f34980a7d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f34a4005e50 tx=0x7f34a4009040 comp rx=0 tx=0).stop 2024-03-30T02:45:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.861+0000 7f34ba000700 1 -- 172.21.15.67:0/321914373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 msgr2=0x7f34b41a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.861+0000 7f34ba000700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 0x7f34b41a3b80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f34a000e9a0 tx=0x7f34a000ecb0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.862+0000 7f34ba000700 1 -- 172.21.15.67:0/321914373 shutdown_connections 2024-03-30T02:45:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.862+0000 7f34ba000700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f34980a5840 0x7f34980a7d00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.862+0000 7f34ba000700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34b4107db0 0x7f34b41a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.862+0000 7f34ba000700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34b410d770 0x7f34b419d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.862+0000 7f34ba000700 1 --2- 172.21.15.67:0/321914373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b4108770 0x7f34b41a3b80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.862+0000 7f34ba000700 1 -- 172.21.15.67:0/321914373 >> 172.21.15.67:0/321914373 conn(0x7f34b406bdf0 msgr2=0x7f34b4071640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.862+0000 7f34ba000700 1 -- 172.21.15.67:0/321914373 shutdown_connections 2024-03-30T02:45:26.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:26.862+0000 7f34ba000700 1 -- 172.21.15.67:0/321914373 wait complete. 2024-03-30T02:45:26.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:45:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:26 smithi170 ceph-mon[78798]: pgmap v258: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 1.7 MiB/s wr, 65 op/s 2024-03-30T02:45:26.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:26 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1450141238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:45:26.995 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/seccomp.o 2024-03-30T02:45:27.169 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/relay.o 2024-03-30T02:45:27.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.406+0000 7f6725800700 1 -- 172.21.15.67:0/3432087074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6720107db0 msgr2=0x7f6720108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:27.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.406+0000 7f6725800700 1 --2- 172.21.15.67:0/3432087074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6720107db0 0x7f6720108190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f671000dfb0 tx=0x7f671000f330 comp rx=0 tx=0).stop 2024-03-30T02:45:27.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.406+0000 7f6725800700 1 -- 172.21.15.67:0/3432087074 shutdown_connections 2024-03-30T02:45:27.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.406+0000 7f6725800700 1 --2- 172.21.15.67:0/3432087074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6720107db0 0x7f6720108190 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.406+0000 7f6725800700 1 --2- 172.21.15.67:0/3432087074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6720113f00 0x7f67201162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.406+0000 7f6725800700 1 --2- 172.21.15.67:0/3432087074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67201086d0 0x7f67201139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.406+0000 7f6725800700 1 -- 172.21.15.67:0/3432087074 >> 172.21.15.67:0/3432087074 conn(0x7f672006bdf0 msgr2=0x7f672006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:27.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.411+0000 7f6725800700 1 -- 172.21.15.67:0/3432087074 shutdown_connections 2024-03-30T02:45:27.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.411+0000 7f6725800700 1 -- 172.21.15.67:0/3432087074 wait complete. 2024-03-30T02:45:27.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.412+0000 7f6725800700 1 Processor -- start 2024-03-30T02:45:27.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.413+0000 7f6725800700 1 -- start start 2024-03-30T02:45:27.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.413+0000 7f6725800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6720107db0 0x7f672019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.413+0000 7f6725800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67201086d0 0x7f672019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.413+0000 7f6725800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6720113f00 0x7f67201a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.413+0000 7f6725800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67201a1610 con 0x7f6720113f00 2024-03-30T02:45:27.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.413+0000 7f6725800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67201a1780 con 0x7f6720107db0 2024-03-30T02:45:27.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.413+0000 7f6725800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67201a1940 con 0x7f67201086d0 2024-03-30T02:45:27.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.414+0000 7f671ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6720107db0 0x7f672019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.414+0000 7f671ea00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6720107db0 0x7f672019d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39352/0 (socket says 172.21.15.67:39352) 2024-03-30T02:45:27.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.414+0000 7f671ea00700 1 -- 172.21.15.67:0/133018562 learned_addr learned my addr 172.21.15.67:0/133018562 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:27.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.414+0000 7f671f400700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6720113f00 0x7f67201a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.414+0000 7f671e000700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67201086d0 0x7f672019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.415+0000 7f671f400700 1 -- 172.21.15.67:0/133018562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67201086d0 msgr2=0x7f672019dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.415+0000 7f671f400700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67201086d0 0x7f672019dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.415+0000 7f671f400700 1 -- 172.21.15.67:0/133018562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6720107db0 msgr2=0x7f672019d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.415+0000 7f671f400700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6720107db0 0x7f672019d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.415+0000 7f671f400700 1 -- 172.21.15.67:0/133018562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671000da00 con 0x7f6720113f00 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.415+0000 7f671e000700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67201086d0 0x7f672019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.415+0000 7f671f400700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6720113f00 0x7f67201a1010 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6718007ec0 tx=0x7f671800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.417+0000 7f6717400700 1 -- 172.21.15.67:0/133018562 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718004db0 con 0x7f6720113f00 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.417+0000 7f6725800700 1 -- 172.21.15.67:0/133018562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67201a1c20 con 0x7f6720113f00 2024-03-30T02:45:27.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.417+0000 7f6725800700 1 -- 172.21.15.67:0/133018562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67201aa580 con 0x7f6720113f00 2024-03-30T02:45:27.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.421+0000 7f6717400700 1 -- 172.21.15.67:0/133018562 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671800f040 con 0x7f6720113f00 2024-03-30T02:45:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.421+0000 7f6717400700 1 -- 172.21.15.67:0/133018562 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718005980 con 0x7f6720113f00 2024-03-30T02:45:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.425+0000 7f67393beb80 1 Processor -- start 2024-03-30T02:45:27.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.427+0000 7f6717400700 1 -- 172.21.15.67:0/133018562 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6718015400 con 0x7f6720113f00 2024-03-30T02:45:27.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.430+0000 7f6717400700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67040a1310 0x7f67040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.430+0000 7f6717400700 1 -- 172.21.15.67:0/133018562 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f67180cc7b0 con 0x7f6720113f00 2024-03-30T02:45:27.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.430+0000 7f671e000700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67040a1310 0x7f67040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.432+0000 7f67393beb80 1 -- start start 2024-03-30T02:45:27.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.432+0000 7f67393beb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f19895d0 0x55c5f19899b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.432+0000 7f67393beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 0x55c5f19948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.432+0000 7f67393beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f19971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.432+0000 7f67393beb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f18e2990 con 0x55c5f19895d0 2024-03-30T02:45:27.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.432+0000 7f67393beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f18e0350 con 0x55c5f1994df0 2024-03-30T02:45:27.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.432+0000 7f67393beb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f18dbc40 con 0x55c5f1989ef0 2024-03-30T02:45:27.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.433+0000 7f671ea00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f19895d0 0x55c5f19899b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.433+0000 7f671f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f19971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.433+0000 7f671f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f19971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39366/0 (socket says 172.21.15.67:39366) 2024-03-30T02:45:27.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.433+0000 7f671f400700 1 -- 172.21.15.67:0/3549235354 learned_addr learned my addr 172.21.15.67:0/3549235354 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:27.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.433+0000 7f671e000700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67040a1310 0x7f67040a37d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f672019ed30 tx=0x7f670c009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:27.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.433+0000 7f671ea00700 1 -- 172.21.15.67:0/3549235354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 msgr2=0x55c5f19948b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:45:27.433 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.433+0000 7f671ea00700 1 --2- 172.21.15.67:0/3549235354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 0x55c5f19948b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.434+0000 7f671ea00700 1 -- 172.21.15.67:0/3549235354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 msgr2=0x55c5f19971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:27.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.434+0000 7f671ea00700 1 --2- 172.21.15.67:0/3549235354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f19971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.434+0000 7f671ea00700 1 -- 172.21.15.67:0/3549235354 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5f175d3b0 con 0x55c5f19895d0 2024-03-30T02:45:27.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.434+0000 7f671f400700 1 --2- 172.21.15.67:0/3549235354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f19971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:45:27.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.436+0000 7f671ea00700 1 --2- 172.21.15.67:0/3549235354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f19895d0 0x55c5f19899b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6710019040 tx=0x7f6710005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ca3c62f35f7a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:27.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.437+0000 7f670be00700 1 -- 172.21.15.67:0/3549235354 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710012040 con 0x55c5f19895d0 2024-03-30T02:45:27.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.437+0000 7f670be00700 1 -- 172.21.15.67:0/3549235354 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671000fb60 con 0x55c5f19895d0 2024-03-30T02:45:27.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.437+0000 7f670be00700 1 -- 172.21.15.67:0/3549235354 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710003ba0 con 0x55c5f19895d0 2024-03-30T02:45:27.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.439+0000 7f67393beb80 1 -- 172.21.15.67:0/3549235354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f19895d0 msgr2=0x55c5f19899b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:27.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.439+0000 7f67393beb80 1 --2- 172.21.15.67:0/3549235354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f19895d0 0x55c5f19899b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6710019040 tx=0x7f6710005c70 comp rx=0 tx=0).stop 2024-03-30T02:45:27.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.439+0000 7f67393beb80 1 -- 172.21.15.67:0/3549235354 shutdown_connections 2024-03-30T02:45:27.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.439+0000 7f67393beb80 1 --2- 172.21.15.67:0/3549235354 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f19895d0 0x55c5f19899b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.439+0000 7f67393beb80 1 --2- 172.21.15.67:0/3549235354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f19971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.439+0000 7f67393beb80 1 --2- 172.21.15.67:0/3549235354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 0x55c5f19948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.439+0000 7f67393beb80 1 -- 172.21.15.67:0/3549235354 >> 172.21.15.67:0/3549235354 conn(0x55c5f18e96a0 msgr2=0x55c5f1985a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:27.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.439+0000 7f67393beb80 1 -- 172.21.15.67:0/3549235354 shutdown_connections 2024-03-30T02:45:27.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.439+0000 7f67393beb80 1 -- 172.21.15.67:0/3549235354 wait complete. 2024-03-30T02:45:27.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.440+0000 7f67393beb80 1 -- start start 2024-03-30T02:45:27.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.440+0000 7f67393beb80 1 Processor -- start 2024-03-30T02:45:27.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.442+0000 7f67393beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1989ef0 0x55c5f1991cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.443+0000 7f67393beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1994df0 0x55c5f19921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.443+0000 7f67393beb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1992730 0x55c5f19a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.443+0000 7f67393beb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1877d80 con 0x55c5f1992730 2024-03-30T02:45:27.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.443+0000 7f67393beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1877a20 con 0x55c5f1989ef0 2024-03-30T02:45:27.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.443+0000 7f67393beb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f175f6a0 con 0x55c5f1994df0 2024-03-30T02:45:27.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.444+0000 7f67393beb80 10 client.0 _ll_register_callbacks cb 0x55c5f1881470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:27.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.444+0000 7f671f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1992730 0x55c5f19a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.444+0000 7f671f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1992730 0x55c5f19a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44348/0 (socket says 172.21.15.67:44348) 2024-03-30T02:45:27.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.444+0000 7f671f400700 1 -- 172.21.15.67:0/2015105997 learned_addr learned my addr 172.21.15.67:0/2015105997 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.444+0000 7f671ea00700 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1989ef0 0x55c5f1991cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.444+0000 7f671e000700 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1994df0 0x55c5f19921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.445+0000 7f671f400700 1 -- 172.21.15.67:0/2015105997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1994df0 msgr2=0x55c5f19921f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.445+0000 7f671f400700 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1994df0 0x55c5f19921f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.445+0000 7f671f400700 1 -- 172.21.15.67:0/2015105997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1989ef0 msgr2=0x55c5f1991cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.445+0000 7f671f400700 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1989ef0 0x55c5f1991cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.445+0000 7f671f400700 1 -- 172.21.15.67:0/2015105997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67201aa580 con 0x55c5f1992730 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.445+0000 7f671e000700 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1994df0 0x55c5f19921f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.445+0000 7f671f400700 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1992730 0x55c5f19a0b50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6718095f90 tx=0x7f67180993e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1c7c7707f0f7d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.446+0000 7f6708c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.446+0000 7f6708c00700 1 -- 172.21.15.67:0/2015105997 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718095040 con 0x55c5f1992730 2024-03-30T02:45:27.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.446+0000 7f6708c00700 1 -- 172.21.15.67:0/2015105997 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f6718099520 con 0x55c5f1992730 2024-03-30T02:45:27.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.447+0000 7f67393beb80 1 -- 172.21.15.67:0/2015105997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5f189c380 con 0x55c5f1992730 2024-03-30T02:45:27.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.447+0000 7f6708c00700 1 -- 172.21.15.67:0/2015105997 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f671809d170 con 0x55c5f1992730 2024-03-30T02:45:27.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.447+0000 7f67393beb80 10 client.17130 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:27.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.448+0000 7f67393beb80 1 -- 172.21.15.67:0/2015105997 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f6720116cc0 con 0x55c5f1992730 2024-03-30T02:45:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.448+0000 7f6708c00700 1 -- 172.21.15.67:0/2015105997 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5f189c380 con 0x55c5f1992730 2024-03-30T02:45:27.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.448+0000 7f6708c00700 1 -- 172.21.15.67:0/2015105997 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67180b7020 con 0x55c5f1992730 2024-03-30T02:45:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.448+0000 7f67393beb80 10 client.17130 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.448+0000 7f67393beb80 10 client.17130 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:45:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.448+0000 7f67393beb80 20 client.17130 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.448+0000 7f67393beb80 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55c5f19a4f10 0x55c5f19a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.448+0000 7f67393beb80 4 client.17130 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:45:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.448+0000 7f67393beb80 1 -- 172.21.15.67:0/2015105997 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5f18d4700 con 0x55c5f19a4f10 2024-03-30T02:45:27.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.449+0000 7f671ea00700 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55c5f19a4f10 0x55c5f19a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.451+0000 7f671ea00700 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55c5f19a4f10 0x55c5f19a7330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=47b66a3913172fa2 server_cookie=a70684e61de6d500 in_seq=0 out_seq=0 2024-03-30T02:45:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.451+0000 7f6708c00700 10 client.17130 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:45:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.452+0000 7f6708c00700 1 -- 172.21.15.67:0/2015105997 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c5f18d4700 con 0x55c5f19a4f10 2024-03-30T02:45:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.452+0000 7f6708c00700 10 client.17130 handle_command_reply: tid=0 2024-03-30T02:45:27.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.452+0000 7f67393beb80 1 client.17130 shutdown 2024-03-30T02:45:27.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.452+0000 7f67393beb80 1 -- 172.21.15.67:0/2015105997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1992730 msgr2=0x55c5f19a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:27.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.452+0000 7f67393beb80 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1992730 0x55c5f19a0b50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6718095f90 tx=0x7f67180993e0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.452 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/utsname_sysctl.o 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.453+0000 7f67393beb80 1 -- 172.21.15.67:0/2015105997 shutdown_connections 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.453+0000 7f67393beb80 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55c5f19a4f10 0x55c5f19a7330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.453+0000 7f67393beb80 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1992730 0x55c5f19a0b50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.453+0000 7f67393beb80 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1989ef0 0x55c5f1991cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.453+0000 7f6708c00700 0 client.17130 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.453+0000 7f67393beb80 1 --2- 172.21.15.67:0/2015105997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1994df0 0x55c5f19921f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.453+0000 7f67393beb80 1 -- 172.21.15.67:0/2015105997 >> 172.21.15.67:0/2015105997 conn(0x55c5f18e96a0 msgr2=0x55c5f198b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.453+0000 7f67393beb80 1 -- 172.21.15.67:0/2015105997 shutdown_connections 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.453+0000 7f67393beb80 1 -- 172.21.15.67:0/2015105997 wait complete. 2024-03-30T02:45:27.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.454+0000 7f67393beb80 20 client.17130 trim_cache size 0 max 16384 2024-03-30T02:45:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.498+0000 7f67393beb80 1 Processor -- start 2024-03-30T02:45:27.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.499+0000 7f67393beb80 1 -- start start 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.499+0000 7f67393beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 0x55c5f199ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.499+0000 7f67393beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1992730 0x55c5f198ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.500+0000 7f671ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 0x55c5f199ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.500+0000 7f671ea00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 0x55c5f199ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56768/0 (socket says 172.21.15.67:56768) 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.500+0000 7f671ea00700 1 -- 172.21.15.67:0/1407983995 learned_addr learned my addr 172.21.15.67:0/1407983995 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.500+0000 7f67393beb80 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1994df0 0x55c5f198f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.500+0000 7f671e000700 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1992730 0x55c5f198ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.501+0000 7f671f400700 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1994df0 0x55c5f198f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.501+0000 7f67393beb80 1 -- 172.21.15.67:0/1407983995 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f19872b0 con 0x55c5f1994df0 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.501+0000 7f67393beb80 1 -- 172.21.15.67:0/1407983995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f199bfd0 con 0x55c5f1992730 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.501+0000 7f67393beb80 1 -- 172.21.15.67:0/1407983995 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1990200 con 0x55c5f1989ef0 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.502+0000 7f671f400700 1 -- 172.21.15.67:0/1407983995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 msgr2=0x55c5f199ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:27.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.502+0000 7f671f400700 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 0x55c5f199ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.502+0000 7f671f400700 1 -- 172.21.15.67:0/1407983995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1992730 msgr2=0x55c5f198ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.502+0000 7f671f400700 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1992730 0x55c5f198ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.502+0000 7f671f400700 1 -- 172.21.15.67:0/1407983995 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5f1709b60 con 0x55c5f1994df0 2024-03-30T02:45:27.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.502+0000 7f671f400700 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1994df0 0x55c5f198f450 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6718097870 tx=0x7f6718005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f6bf73074de8b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:27.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.507+0000 7f6714c00700 1 -- 172.21.15.67:0/1407983995 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718095040 con 0x55c5f1994df0 2024-03-30T02:45:27.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.507+0000 7f6714c00700 1 -- 172.21.15.67:0/1407983995 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671809d170 con 0x55c5f1994df0 2024-03-30T02:45:27.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.507+0000 7f6714c00700 1 -- 172.21.15.67:0/1407983995 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67180cc5f0 con 0x55c5f1994df0 2024-03-30T02:45:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.508+0000 7f67393beb80 1 -- 172.21.15.67:0/1407983995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1994df0 msgr2=0x55c5f198f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:27.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.508+0000 7f67393beb80 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1994df0 0x55c5f198f450 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6718097870 tx=0x7f6718005bf0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.511+0000 7f67393beb80 1 -- 172.21.15.67:0/1407983995 shutdown_connections 2024-03-30T02:45:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.511+0000 7f67393beb80 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1994df0 0x55c5f198f450 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.511+0000 7f67393beb80 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1989ef0 0x55c5f199ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.511+0000 7f67393beb80 1 --2- 172.21.15.67:0/1407983995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1992730 0x55c5f198ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.511+0000 7f67393beb80 1 -- 172.21.15.67:0/1407983995 >> 172.21.15.67:0/1407983995 conn(0x55c5f18e96a0 msgr2=0x55c5f1990580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.511+0000 7f67393beb80 1 -- 172.21.15.67:0/1407983995 shutdown_connections 2024-03-30T02:45:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.511+0000 7f67393beb80 1 -- 172.21.15.67:0/1407983995 wait complete. 2024-03-30T02:45:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.512+0000 7f67393beb80 1 -- start start 2024-03-30T02:45:27.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.512+0000 7f67393beb80 1 Processor -- start 2024-03-30T02:45:27.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.514+0000 7f67393beb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1989ef0 0x55c5f199b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f67393beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1992730 0x55c5f199b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f67393beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f1999800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f67393beb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1999dd0 con 0x55c5f1989ef0 2024-03-30T02:45:27.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f67393beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f1999f40 con 0x55c5f1994df0 2024-03-30T02:45:27.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f67393beb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5f199a100 con 0x55c5f1992730 2024-03-30T02:45:27.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f67393beb80 10 client.0 _ll_register_callbacks cb 0x55c5f1881470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f671f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f1999800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f671f400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f1999800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39398/0 (socket says 172.21.15.67:39398) 2024-03-30T02:45:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f671f400700 1 -- 172.21.15.67:0/3455194993 learned_addr learned my addr 172.21.15.67:0/3455194993 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.518+0000 7f671e000700 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1992730 0x55c5f199b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.519+0000 7f671f400700 1 -- 172.21.15.67:0/3455194993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1992730 msgr2=0x55c5f199b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:27.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.519+0000 7f671f400700 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1992730 0x55c5f199b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.519+0000 7f671f400700 1 -- 172.21.15.67:0/3455194993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1989ef0 msgr2=0x55c5f199b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.519+0000 7f671ea00700 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1989ef0 0x55c5f199b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.519+0000 7f671f400700 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1989ef0 0x55c5f199b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.519+0000 7f671f400700 1 -- 172.21.15.67:0/3455194993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6720116cc0 con 0x55c5f1994df0 2024-03-30T02:45:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.519+0000 7f671e000700 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1992730 0x55c5f199b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:45:27.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.519+0000 7f671ea00700 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1989ef0 0x55c5f199b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.520+0000 7f671f400700 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f1999800 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f67180ca160 tx=0x7f67180cc640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3529d5cac67f6783 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.520+0000 7f6708c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:45:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.520+0000 7f6708c00700 1 -- 172.21.15.67:0/3455194993 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718095040 con 0x55c5f1994df0 2024-03-30T02:45:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.520+0000 7f6708c00700 1 -- 172.21.15.67:0/3455194993 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f671809d350 con 0x55c5f1994df0 2024-03-30T02:45:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.520+0000 7f67393beb80 1 -- 172.21.15.67:0/3455194993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5f172acf0 con 0x55c5f1994df0 2024-03-30T02:45:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.521+0000 7f6708c00700 1 -- 172.21.15.67:0/3455194993 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6718005000 con 0x55c5f1994df0 2024-03-30T02:45:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.521+0000 7f67393beb80 10 client.27263 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:27.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.521+0000 7f67393beb80 1 -- 172.21.15.67:0/3455194993 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c5f18a1ea0 con 0x55c5f1994df0 2024-03-30T02:45:27.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.521+0000 7f6708c00700 1 -- 172.21.15.67:0/3455194993 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5f172acf0 con 0x55c5f1994df0 2024-03-30T02:45:27.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.522+0000 7f6708c00700 1 -- 172.21.15.67:0/3455194993 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67180b7040 con 0x55c5f1994df0 2024-03-30T02:45:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.522+0000 7f67393beb80 10 client.27263 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:27.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.522+0000 7f67393beb80 10 client.27263 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:45:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.522+0000 7f67393beb80 20 client.27263 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.522+0000 7f67393beb80 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55c5f19f99f0 0x55c5f19fbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.522+0000 7f67393beb80 4 client.27263 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:45:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.522+0000 7f67393beb80 1 -- 172.21.15.67:0/3455194993 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5f175fce0 con 0x55c5f19f99f0 2024-03-30T02:45:27.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.522+0000 7f671ea00700 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55c5f19f99f0 0x55c5f19fbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.525+0000 7f671ea00700 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55c5f19f99f0 0x55c5f19fbde0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f5c674d5994f3e9 server_cookie=374e72e7217a27cd in_seq=0 out_seq=0 2024-03-30T02:45:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.525+0000 7f6708c00700 10 client.27263 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:45:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.525+0000 7f6708c00700 1 -- 172.21.15.67:0/3455194993 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5f175fce0 con 0x55c5f19f99f0 2024-03-30T02:45:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.525+0000 7f6708c00700 10 client.27263 handle_command_reply: tid=0 2024-03-30T02:45:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.525+0000 7f67393beb80 1 client.27263 shutdown 2024-03-30T02:45:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.526+0000 7f67393beb80 1 -- 172.21.15.67:0/3455194993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 msgr2=0x55c5f1999800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:27.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.526+0000 7f67393beb80 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f1999800 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f67180ca160 tx=0x7f67180cc640 comp rx=0 tx=0).stop 2024-03-30T02:45:27.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.529+0000 7f67393beb80 1 -- 172.21.15.67:0/3455194993 shutdown_connections 2024-03-30T02:45:27.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.529+0000 7f67393beb80 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55c5f19f99f0 0x55c5f19fbde0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.529+0000 7f67393beb80 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c5f1989ef0 0x55c5f199b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.529+0000 7f67393beb80 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5f1994df0 0x55c5f1999800 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.529+0000 7f67393beb80 1 --2- 172.21.15.67:0/3455194993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c5f1992730 0x55c5f199b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.529+0000 7f67393beb80 1 -- 172.21.15.67:0/3455194993 >> 172.21.15.67:0/3455194993 conn(0x55c5f18e96a0 msgr2=0x55c5f1984f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.529+0000 7f6708c00700 0 client.27263 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:45:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.529+0000 7f67393beb80 1 -- 172.21.15.67:0/3455194993 shutdown_connections 2024-03-30T02:45:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.529+0000 7f67393beb80 1 -- 172.21.15.67:0/3455194993 wait complete. 2024-03-30T02:45:27.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.530+0000 7f67393beb80 20 client.27263 trim_cache size 0 max 16384 2024-03-30T02:45:27.530 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:45:27.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.532+0000 7f6725800700 1 -- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67040a1310 msgr2=0x7f67040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:27.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.532+0000 7f6725800700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67040a1310 0x7f67040a37d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f672019ed30 tx=0x7f670c009260 comp rx=0 tx=0).stop 2024-03-30T02:45:27.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.532+0000 7f6725800700 1 -- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6720113f00 msgr2=0x7f67201a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:27.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.532+0000 7f6725800700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6720113f00 0x7f67201a1010 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6718007ec0 tx=0x7f671800d3b0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.533+0000 7f6725800700 1 -- 172.21.15.67:0/133018562 shutdown_connections 2024-03-30T02:45:27.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.533+0000 7f6725800700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f67040a1310 0x7f67040a37d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.533+0000 7f6725800700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6720113f00 0x7f67201a1010 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.533+0000 7f6725800700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6720107db0 0x7f672019d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.533+0000 7f6725800700 1 --2- 172.21.15.67:0/133018562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67201086d0 0x7f672019dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:27.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.533+0000 7f6725800700 1 -- 172.21.15.67:0/133018562 >> 172.21.15.67:0/133018562 conn(0x7f672006bdf0 msgr2=0x7f672006fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:27.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.533+0000 7f6725800700 1 -- 172.21.15.67:0/133018562 shutdown_connections 2024-03-30T02:45:27.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:27.533+0000 7f6725800700 1 -- 172.21.15.67:0/133018562 wait complete. 2024-03-30T02:45:27.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:45:27.617 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/delayacct.o 2024-03-30T02:45:27.644 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/taskstats.o 2024-03-30T02:45:27.720 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/tsacct.o 2024-03-30T02:45:27.850 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/tracepoint.o 2024-03-30T02:45:27.999 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/irq_work.o 2024-03-30T02:45:28.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.079+0000 7f83df200700 1 -- 172.21.15.67:0/4236475337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e01139c0 msgr2=0x7f83e0115db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:28.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.079+0000 7f83df200700 1 --2- 172.21.15.67:0/4236475337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e01139c0 0x7f83e0115db0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009990 tx=0x7f83c8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.083+0000 7f83df200700 1 -- 172.21.15.67:0/4236475337 shutdown_connections 2024-03-30T02:45:28.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.083+0000 7f83df200700 1 --2- 172.21.15.67:0/4236475337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e01139c0 0x7f83e0115db0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.083+0000 7f83df200700 1 --2- 172.21.15.67:0/4236475337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83e006ced0 0x7f83e0107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.083+0000 7f83df200700 1 --2- 172.21.15.67:0/4236475337 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e01082f0 0x7f83e0108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.083+0000 7f83df200700 1 -- 172.21.15.67:0/4236475337 >> 172.21.15.67:0/4236475337 conn(0x7f83e006bbc0 msgr2=0x7f83e006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.083+0000 7f83df200700 1 -- 172.21.15.67:0/4236475337 shutdown_connections 2024-03-30T02:45:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.083+0000 7f83df200700 1 -- 172.21.15.67:0/4236475337 wait complete. 2024-03-30T02:45:28.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.085+0000 7f83df200700 1 Processor -- start 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.085+0000 7f83df200700 1 -- start start 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.087+0000 7f83df200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e006ced0 0x7f83e0109820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.087+0000 7f83df200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e01082f0 0x7f83e0109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.087+0000 7f83df200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83e01139c0 0x7f83e010a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.087+0000 7f83df200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e010b7f0 con 0x7f83e01082f0 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.087+0000 7f83df200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e01a3950 con 0x7f83e01139c0 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.087+0000 7f83df200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e01a3b10 con 0x7f83e006ced0 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.087+0000 7f83dde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e006ced0 0x7f83e0109820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83dde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e006ced0 0x7f83e0109820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56788/0 (socket says 172.21.15.67:56788) 2024-03-30T02:45:28.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83dde00700 1 -- 172.21.15.67:0/3389989596 learned_addr learned my addr 172.21.15.67:0/3389989596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83de800700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83e01139c0 0x7f83e010a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83dd400700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e01082f0 0x7f83e0109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83dde00700 1 -- 172.21.15.67:0/3389989596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83e01139c0 msgr2=0x7f83e010a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83dde00700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83e01139c0 0x7f83e010a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83dde00700 1 -- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e01082f0 msgr2=0x7f83e0109d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83dde00700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e01082f0 0x7f83e0109d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83dde00700 1 -- 172.21.15.67:0/3389989596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d0009520 con 0x7f83e006ced0 2024-03-30T02:45:28.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.088+0000 7f83dd400700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e01082f0 0x7f83e0109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:28.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.092+0000 7f83dde00700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e006ced0 0x7f83e0109820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f83d000eb30 tx=0x7f83d000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:28.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.093+0000 7f83d6a00700 1 -- 172.21.15.67:0/3389989596 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0009880 con 0x7f83e006ced0 2024-03-30T02:45:28.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.093+0000 7f83d6a00700 1 -- 172.21.15.67:0/3389989596 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83d00044e0 con 0x7f83e006ced0 2024-03-30T02:45:28.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.093+0000 7f83d6a00700 1 -- 172.21.15.67:0/3389989596 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0003df0 con 0x7f83e006ced0 2024-03-30T02:45:28.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.093+0000 7f83df200700 1 -- 172.21.15.67:0/3389989596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c80095f0 con 0x7f83e006ced0 2024-03-30T02:45:28.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.093+0000 7f83df200700 1 -- 172.21.15.67:0/3389989596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e01a4120 con 0x7f83e006ced0 2024-03-30T02:45:28.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.098+0000 7f83d6a00700 1 -- 172.21.15.67:0/3389989596 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f83d0003f50 con 0x7f83e006ced0 2024-03-30T02:45:28.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.101+0000 7f83d6a00700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f83c40a1310 0x7f83c40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.101+0000 7f83d6a00700 1 -- 172.21.15.67:0/3389989596 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f83d0014070 con 0x7f83e006ced0 2024-03-30T02:45:28.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.104+0000 7f83dd400700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f83c40a1310 0x7f83c40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.105+0000 7f83dd400700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f83c40a1310 0x7f83c40a37d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009790 tx=0x7f83d8009040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:28.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.107+0000 7f83f89d5b80 1 Processor -- start 2024-03-30T02:45:28.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.107+0000 7f83f89d5b80 1 -- start start 2024-03-30T02:45:28.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4b2fc0 0x55a42b4b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83f89d5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b400990 con 0x55a42b4a80c0 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83f89d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b3fe350 con 0x55a42b4b2fc0 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83f89d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b3f9c40 con 0x55a42b4a77a0 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83dde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83dde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4a7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56796/0 (socket says 172.21.15.67:56796) 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83dde00700 1 -- 172.21.15.67:0/776668797 learned_addr learned my addr 172.21.15.67:0/776668797 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83de800700 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4b2fc0 0x55a42b4b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83dd400700 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83dde00700 1 -- 172.21.15.67:0/776668797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4b2fc0 msgr2=0x55a42b4b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83dde00700 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4b2fc0 0x55a42b4b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.108+0000 7f83dde00700 1 -- 172.21.15.67:0/776668797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 msgr2=0x55a42b4b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.109+0000 7f83dde00700 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.109+0000 7f83dde00700 1 -- 172.21.15.67:0/776668797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42b27b3b0 con 0x55a42b4a77a0 2024-03-30T02:45:28.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.109+0000 7f83dd400700 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:45:28.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.109+0000 7f83dde00700 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4a7b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f83d00a20b0 tx=0x7f83d0099ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8674003939a790ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:28.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.110+0000 7f83cf400700 1 -- 172.21.15.67:0/776668797 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d00b1070 con 0x55a42b4a77a0 2024-03-30T02:45:28.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.113+0000 7f83cf400700 1 -- 172.21.15.67:0/776668797 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83d009a160 con 0x55a42b4a77a0 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.113+0000 7f83cf400700 1 -- 172.21.15.67:0/776668797 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d00cd160 con 0x55a42b4a77a0 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.114+0000 7f83f89d5b80 1 -- 172.21.15.67:0/776668797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 msgr2=0x55a42b4a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.114+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4a7b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f83d00a20b0 tx=0x7f83d0099ac0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.118+0000 7f83f89d5b80 1 -- 172.21.15.67:0/776668797 shutdown_connections 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.118+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4b2fc0 0x55a42b4b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.118+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4a7b80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.118+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/776668797 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.118+0000 7f83f89d5b80 1 -- 172.21.15.67:0/776668797 >> 172.21.15.67:0/776668797 conn(0x55a42b4076a0 msgr2=0x55a42b4a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.118+0000 7f83f89d5b80 1 -- 172.21.15.67:0/776668797 shutdown_connections 2024-03-30T02:45:28.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.118+0000 7f83f89d5b80 1 -- 172.21.15.67:0/776668797 wait complete. 2024-03-30T02:45:28.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.119+0000 7f83f89d5b80 1 -- start start 2024-03-30T02:45:28.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.119+0000 7f83f89d5b80 1 Processor -- start 2024-03-30T02:45:28.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.123+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a77a0 0x55a42b4b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.123+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.123+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 0x55a42b4b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.123+0000 7f83f89d5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b395d80 con 0x55a42b4a80c0 2024-03-30T02:45:28.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.123+0000 7f83f89d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b395a20 con 0x55a42b4a77a0 2024-03-30T02:45:28.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.123+0000 7f83f89d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b27d6a0 con 0x55a42b4b2fc0 2024-03-30T02:45:28.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83f89d5b80 10 client.0 _ll_register_callbacks cb 0x55a42b39f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83dde00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a77a0 0x55a42b4b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83dde00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a77a0 0x55a42b4b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39436/0 (socket says 172.21.15.67:39436) 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83dde00700 1 -- 172.21.15.67:0/3855948539 learned_addr learned my addr 172.21.15.67:0/3855948539 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83de800700 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 0x55a42b4b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83dde00700 1 -- 172.21.15.67:0/3855948539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 msgr2=0x55a42b4b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83dde00700 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 0x55a42b4b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83dde00700 1 -- 172.21.15.67:0/3855948539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 msgr2=0x55a42b4afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83dd400700 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83dde00700 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.124+0000 7f83dde00700 1 -- 172.21.15.67:0/3855948539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e01a4120 con 0x55a42b4a77a0 2024-03-30T02:45:28.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.125+0000 7f83dd400700 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:28.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.125+0000 7f83dde00700 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a77a0 0x55a42b4b2990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f83d0003730 tx=0x7f83d00ca790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4faf478eba49f306 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:28.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.125+0000 7f83c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:45:28.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.125+0000 7f83c3e00700 1 -- 172.21.15.67:0/3855948539 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d00b1030 con 0x55a42b4a77a0 2024-03-30T02:45:28.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.126+0000 7f83c3e00700 1 -- 172.21.15.67:0/3855948539 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f83d00b2070 con 0x55a42b4a77a0 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.126+0000 7f83f89d5b80 1 -- 172.21.15.67:0/3855948539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a42b3ba380 con 0x55a42b4a77a0 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.126+0000 7f83c3e00700 1 -- 172.21.15.67:0/3855948539 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83d00a25e0 con 0x55a42b4a77a0 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.126+0000 7f83f89d5b80 10 client.27275 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.126+0000 7f83f89d5b80 1 -- 172.21.15.67:0/3855948539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a42b227b60 con 0x55a42b4a77a0 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.126+0000 7f83c3e00700 1 -- 172.21.15.67:0/3855948539 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a42b3ba380 con 0x55a42b4a77a0 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.127+0000 7f83c3e00700 1 -- 172.21.15.67:0/3855948539 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f83d0099650 con 0x55a42b4a77a0 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.127+0000 7f83f89d5b80 10 client.27275 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.127+0000 7f83f89d5b80 10 client.27275 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.127+0000 7f83f89d5b80 20 client.27275 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:28.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.127+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55a42b4c2860 0x55a42b4c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.130+0000 7f83f89d5b80 4 client.27275 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:45:28.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.130+0000 7f83f89d5b80 1 -- 172.21.15.67:0/3855948539 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a42b3f2700 con 0x55a42b4c2860 2024-03-30T02:45:28.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.130+0000 7f83dd400700 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55a42b4c2860 0x55a42b4c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.131+0000 7f83dd400700 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55a42b4c2860 0x55a42b4c4c80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53671058d929a0e0 server_cookie=e2a1a17138593844 in_seq=0 out_seq=0 2024-03-30T02:45:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.131+0000 7f83c3e00700 10 client.27275 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:45:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.135+0000 7f83c3e00700 1 -- 172.21.15.67:0/3855948539 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a42b3f2700 con 0x55a42b4c2860 2024-03-30T02:45:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.135+0000 7f83c3e00700 10 client.27275 handle_command_reply: tid=0 2024-03-30T02:45:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.135+0000 7f83f89d5b80 1 client.27275 shutdown 2024-03-30T02:45:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.135+0000 7f83f89d5b80 1 -- 172.21.15.67:0/3855948539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a77a0 msgr2=0x55a42b4b2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:28.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.135+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a77a0 0x55a42b4b2990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f83d0003730 tx=0x7f83d00ca790 comp rx=0 tx=0).stop 2024-03-30T02:45:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.138+0000 7f83f89d5b80 1 -- 172.21.15.67:0/3855948539 shutdown_connections 2024-03-30T02:45:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.138+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55a42b4c2860 0x55a42b4c4c80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.138+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 0x55a42b4b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.138+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a77a0 0x55a42b4b2990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.138+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/3855948539 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a80c0 0x55a42b4afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.138+0000 7f83c3e00700 0 client.27275 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:45:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.138+0000 7f83f89d5b80 1 -- 172.21.15.67:0/3855948539 >> 172.21.15.67:0/3855948539 conn(0x55a42b4076a0 msgr2=0x55a42b4a9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.138+0000 7f83f89d5b80 1 -- 172.21.15.67:0/3855948539 shutdown_connections 2024-03-30T02:45:28.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.138+0000 7f83f89d5b80 1 -- 172.21.15.67:0/3855948539 wait complete. 2024-03-30T02:45:28.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.142+0000 7f83f89d5b80 20 client.27275 trim_cache size 0 max 16384 2024-03-30T02:45:28.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.196+0000 7f83f89d5b80 1 Processor -- start 2024-03-30T02:45:28.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.197+0000 7f83f89d5b80 1 -- start start 2024-03-30T02:45:28.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4b2fc0 0x55a42b49c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83dde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83dde00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56814/0 (socket says 172.21.15.67:56814) 2024-03-30T02:45:28.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83dd400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83dd400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39442/0 (socket says 172.21.15.67:39442) 2024-03-30T02:45:28.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83de800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4b2fc0 0x55a42b49c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83de800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4b2fc0 0x55a42b49c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44426/0 (socket says 172.21.15.67:44426) 2024-03-30T02:45:28.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.198+0000 7f83dde00700 1 -- 172.21.15.67:0/1861662392 learned_addr learned my addr 172.21.15.67:0/1861662392 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:28.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.199+0000 7f83f89d5b80 1 -- 172.21.15.67:0/1861662392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b4b10a0 con 0x55a42b4b2fc0 2024-03-30T02:45:28.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.199+0000 7f83f89d5b80 1 -- 172.21.15.67:0/1861662392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b4a84a0 con 0x55a42b4a80c0 2024-03-30T02:45:28.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.199+0000 7f83f89d5b80 1 -- 172.21.15.67:0/1861662392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b4a8ff0 con 0x55a42b4a77a0 2024-03-30T02:45:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.199+0000 7f83de800700 1 -- 172.21.15.67:0/1861662392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 msgr2=0x55a42b4c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.199+0000 7f83de800700 1 --2- 172.21.15.67:0/1861662392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.199+0000 7f83de800700 1 -- 172.21.15.67:0/1861662392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 msgr2=0x55a42b4b9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.199+0000 7f83de800700 1 --2- 172.21.15.67:0/1861662392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.199+0000 7f83de800700 1 -- 172.21.15.67:0/1861662392 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42b3bfea0 con 0x55a42b4b2fc0 2024-03-30T02:45:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.200+0000 7f83de800700 1 --2- 172.21.15.67:0/1861662392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4b2fc0 0x55a42b49c7f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f83c80098e0 tx=0x7f83c8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771e7be7883909d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.200+0000 7f83cfe00700 1 -- 172.21.15.67:0/1861662392 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c801d030 con 0x55a42b4b2fc0 2024-03-30T02:45:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.200+0000 7f83cfe00700 1 -- 172.21.15.67:0/1861662392 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83c801e040 con 0x55a42b4b2fc0 2024-03-30T02:45:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.200+0000 7f83cfe00700 1 -- 172.21.15.67:0/1861662392 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8004870 con 0x55a42b4b2fc0 2024-03-30T02:45:28.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.201+0000 7f83dde00700 1 --2- 172.21.15.67:0/1861662392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:28.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.202+0000 7f83f89d5b80 1 -- 172.21.15.67:0/1861662392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4b2fc0 msgr2=0x55a42b49c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:28.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.202+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/1861662392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4b2fc0 0x55a42b49c7f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f83c80098e0 tx=0x7f83c8005bc0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.203+0000 7f83f89d5b80 1 -- 172.21.15.67:0/1861662392 shutdown_connections 2024-03-30T02:45:28.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.203+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/1861662392 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4b2fc0 0x55a42b49c7f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.203+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/1861662392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4a77a0 0x55a42b4c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.203+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/1861662392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.203+0000 7f83f89d5b80 1 -- 172.21.15.67:0/1861662392 >> 172.21.15.67:0/1861662392 conn(0x55a42b4076a0 msgr2=0x55a42b4af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:28.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.203+0000 7f83f89d5b80 1 -- 172.21.15.67:0/1861662392 shutdown_connections 2024-03-30T02:45:28.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.203+0000 7f83f89d5b80 1 -- 172.21.15.67:0/1861662392 wait complete. 2024-03-30T02:45:28.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.204+0000 7f83f89d5b80 1 -- start start 2024-03-30T02:45:28.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.204+0000 7f83f89d5b80 1 Processor -- start 2024-03-30T02:45:28.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.206+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a77a0 0x55a42b4b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.206+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.206+0000 7f83dde00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a77a0 0x55a42b4b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.207+0000 7f83f89d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 0x55a42b50f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.207+0000 7f83dd400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.207+0000 7f83dd400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39446/0 (socket says 172.21.15.67:39446) 2024-03-30T02:45:28.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.207+0000 7f83dd400700 1 -- 172.21.15.67:0/2606782530 learned_addr learned my addr 172.21.15.67:0/2606782530 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:28.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.207+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b50fc10 con 0x55a42b4a77a0 2024-03-30T02:45:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.208+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b50fd80 con 0x55a42b4a80c0 2024-03-30T02:45:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.208+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42b50ff40 con 0x55a42b4b2fc0 2024-03-30T02:45:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.208+0000 7f83de800700 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 0x55a42b50f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.208+0000 7f83f89d5b80 10 client.0 _ll_register_callbacks cb 0x55a42b39f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.209+0000 7f83de800700 1 -- 172.21.15.67:0/2606782530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 msgr2=0x55a42b4b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.209+0000 7f83de800700 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.209+0000 7f83de800700 1 -- 172.21.15.67:0/2606782530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a77a0 msgr2=0x55a42b4b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:28.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.209+0000 7f83de800700 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a77a0 0x55a42b4b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.209+0000 7f83de800700 1 -- 172.21.15.67:0/2606782530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e0116a20 con 0x55a42b4b2fc0 2024-03-30T02:45:28.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.209+0000 7f83dde00700 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a77a0 0x55a42b4b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:45:28.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.210+0000 7f83de800700 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 0x55a42b50f6d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f83c8005f50 tx=0x7f83c8005ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dee71eec33a56aff server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:28.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.210+0000 7f83c3e00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:45:28.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.211+0000 7f83c3e00700 1 -- 172.21.15.67:0/2606782530 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c801d070 con 0x55a42b4b2fc0 2024-03-30T02:45:28.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.211+0000 7f83c3e00700 1 -- 172.21.15.67:0/2606782530 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f83c8005690 con 0x55a42b4b2fc0 2024-03-30T02:45:28.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.211+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83d009a160 con 0x55a42b4b2fc0 2024-03-30T02:45:28.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.212+0000 7f83c3e00700 1 -- 172.21.15.67:0/2606782530 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83c800f870 con 0x55a42b4b2fc0 2024-03-30T02:45:28.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.214+0000 7f83f89d5b80 10 client.27223 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:28.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.214+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a42b27fcd0 con 0x55a42b4b2fc0 2024-03-30T02:45:28.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.214+0000 7f83c3e00700 1 -- 172.21.15.67:0/2606782530 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83d009a160 con 0x55a42b4b2fc0 2024-03-30T02:45:28.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.214+0000 7f83c3e00700 1 -- 172.21.15.67:0/2606782530 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f83c8025c30 con 0x55a42b4b2fc0 2024-03-30T02:45:28.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.217+0000 7f83f89d5b80 10 client.27223 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:28.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.217+0000 7f83f89d5b80 10 client.27223 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:45:28.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.217+0000 7f83f89d5b80 20 client.27223 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:28.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.217+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55a42b5171b0 0x55a42b5195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:28.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.217+0000 7f83f89d5b80 4 client.27223 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:45:28.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.217+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a42b248cf0 con 0x55a42b5171b0 2024-03-30T02:45:28.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.218+0000 7f83dde00700 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55a42b5171b0 0x55a42b5195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:28.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.219+0000 7f83dde00700 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55a42b5171b0 0x55a42b5195a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10b691f541772443 server_cookie=faec183658a6b6c6 in_seq=0 out_seq=0 2024-03-30T02:45:28.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.219+0000 7f83c3e00700 10 client.27223 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:45:28.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83c3e00700 1 -- 172.21.15.67:0/2606782530 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a42b248cf0 con 0x55a42b5171b0 2024-03-30T02:45:28.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83c3e00700 10 client.27223 handle_command_reply: tid=0 2024-03-30T02:45:28.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83f89d5b80 1 client.27223 shutdown 2024-03-30T02:45:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 msgr2=0x55a42b50f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 0x55a42b50f6d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f83c8005f50 tx=0x7f83c8005ac0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 shutdown_connections 2024-03-30T02:45:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55a42b5171b0 0x55a42b5195a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a42b4a77a0 0x55a42b4b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a42b4b2fc0 0x55a42b50f6d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83f89d5b80 1 --2- 172.21.15.67:0/2606782530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a42b4a80c0 0x55a42b4b8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83c3e00700 0 client.27223 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:45:28.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.222+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 >> 172.21.15.67:0/2606782530 conn(0x55a42b4076a0 msgr2=0x55a42b4a3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:28.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.226+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 shutdown_connections 2024-03-30T02:45:28.226 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:45:28.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.226+0000 7f83f89d5b80 1 -- 172.21.15.67:0/2606782530 wait complete. 2024-03-30T02:45:28.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.226+0000 7f83f89d5b80 20 client.27223 trim_cache size 0 max 16384 2024-03-30T02:45:28.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 -- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f83c40a1310 msgr2=0x7f83c40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:28.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f83c40a1310 0x7f83c40a37d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009790 tx=0x7f83d8009040 comp rx=0 tx=0).stop 2024-03-30T02:45:28.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 -- 172.21.15.67:0/3389989596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e006ced0 msgr2=0x7f83e0109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:28.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e006ced0 0x7f83e0109820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f83d000eb30 tx=0x7f83d000ee40 comp rx=0 tx=0).stop 2024-03-30T02:45:28.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 -- 172.21.15.67:0/3389989596 shutdown_connections 2024-03-30T02:45:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f83c40a1310 0x7f83c40a37d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83e01139c0 0x7f83e010a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e006ced0 0x7f83e0109820 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 --2- 172.21.15.67:0/3389989596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83e01082f0 0x7f83e0109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 -- 172.21.15.67:0/3389989596 >> 172.21.15.67:0/3389989596 conn(0x7f83e006bbc0 msgr2=0x7f83e006f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 -- 172.21.15.67:0/3389989596 shutdown_connections 2024-03-30T02:45:28.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:28.232+0000 7f83df200700 1 -- 172.21.15.67:0/3389989596 wait complete. 2024-03-30T02:45:28.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:45:28.295 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/static_call.o 2024-03-30T02:45:28.312 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/static_call_inline.o 2024-03-30T02:45:28.331 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/crash_dump.o 2024-03-30T02:45:28.463 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/jump_label.o 2024-03-30T02:45:28.565 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/context_tracking.o 2024-03-30T02:45:28.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:28 smithi067 ceph-mon[37808]: pgmap v259: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 964 KiB/s wr, 31 op/s 2024-03-30T02:45:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:28 smithi082 ceph-mon[79176]: pgmap v259: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 964 KiB/s wr, 31 op/s 2024-03-30T02:45:28.889 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/iomem.o 2024-03-30T02:45:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:28 smithi170 ceph-mon[78798]: pgmap v259: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 964 KiB/s wr, 31 op/s 2024-03-30T02:45:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.007+0000 7f880ca00700 1 -- 172.21.15.67:0/3573074353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f880810ce90 msgr2=0x7f880810d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.007+0000 7f880ca00700 1 --2- 172.21.15.67:0/3573074353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f880810ce90 0x7f880810d270 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f880000b0a0 tx=0x7f880000b3b0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.017 INFO:tasks.workunit.client.0.smithi067.stdout: CC kernel/rseq.o 2024-03-30T02:45:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.015+0000 7f880ca00700 1 -- 172.21.15.67:0/3573074353 shutdown_connections 2024-03-30T02:45:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.015+0000 7f880ca00700 1 --2- 172.21.15.67:0/3573074353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f880811a240 0x7f880811c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.016+0000 7f880ca00700 1 --2- 172.21.15.67:0/3573074353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f880810ce90 0x7f880810d270 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.016+0000 7f880ca00700 1 --2- 172.21.15.67:0/3573074353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880810d7b0 0x7f8808119d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.016+0000 7f880ca00700 1 -- 172.21.15.67:0/3573074353 >> 172.21.15.67:0/3573074353 conn(0x7f880806bc10 msgr2=0x7f880806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.023+0000 7f880ca00700 1 -- 172.21.15.67:0/3573074353 shutdown_connections 2024-03-30T02:45:29.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.027+0000 7f880ca00700 1 -- 172.21.15.67:0/3573074353 wait complete. 2024-03-30T02:45:29.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.029+0000 7f880ca00700 1 Processor -- start 2024-03-30T02:45:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.029+0000 7f880ca00700 1 -- start start 2024-03-30T02:45:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.030+0000 7f880ca00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880810ce90 0x7f88081a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.030+0000 7f880ca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f880810d7b0 0x7f88081a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.030+0000 7f880ca00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f880811a240 0x7f88081aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.030+0000 7f880ca00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081aad00 con 0x7f880810ce90 2024-03-30T02:45:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.030+0000 7f880ca00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081aae70 con 0x7f880810d7b0 2024-03-30T02:45:29.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.030+0000 7f880ca00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081ab030 con 0x7f880811a240 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.031+0000 7f8805e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f880810d7b0 0x7f88081a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.031+0000 7f8805e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f880810d7b0 0x7f88081a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39456/0 (socket says 172.21.15.67:39456) 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.031+0000 7f8805e00700 1 -- 172.21.15.67:0/3132231439 learned_addr learned my addr 172.21.15.67:0/3132231439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.031+0000 7f8807200700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f880811a240 0x7f88081aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.032+0000 7f8807200700 1 -- 172.21.15.67:0/3132231439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f880810d7b0 msgr2=0x7f88081a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.032+0000 7f8807200700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f880810d7b0 0x7f88081a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.032+0000 7f8807200700 1 -- 172.21.15.67:0/3132231439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880810ce90 msgr2=0x7f88081a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.032+0000 7f8807200700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880810ce90 0x7f88081a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.032+0000 7f8807200700 1 -- 172.21.15.67:0/3132231439 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800009d00 con 0x7f880811a240 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.033+0000 7f8807200700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f880811a240 0x7f88081aa6c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f87f800c770 tx=0x7f87f800cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.035+0000 7f87ff400700 1 -- 172.21.15.67:0/3132231439 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8004a30 con 0x7f880811a240 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.035+0000 7f87ff400700 1 -- 172.21.15.67:0/3132231439 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f80054e0 con 0x7f880811a240 2024-03-30T02:45:29.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.035+0000 7f87ff400700 1 -- 172.21.15.67:0/3132231439 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f800ebe0 con 0x7f880811a240 2024-03-30T02:45:29.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.037+0000 7f880ca00700 1 -- 172.21.15.67:0/3132231439 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88081ab310 con 0x7f880811a240 2024-03-30T02:45:29.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.038+0000 7f880ca00700 1 -- 172.21.15.67:0/3132231439 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880810f6c0 con 0x7f880811a240 2024-03-30T02:45:29.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.056+0000 7f87ff400700 1 -- 172.21.15.67:0/3132231439 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f87f800ed40 con 0x7f880811a240 2024-03-30T02:45:29.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.058+0000 7f87ff400700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f87ec0a14a0 0x7f87ec0a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.058+0000 7f87ff400700 1 -- 172.21.15.67:0/3132231439 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f87f80cc1f0 con 0x7f880811a240 2024-03-30T02:45:29.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.059+0000 7f8806800700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f87ec0a14a0 0x7f87ec0a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.061+0000 7f8806800700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f87ec0a14a0 0x7f87ec0a3960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f880000b780 tx=0x7f88000094d0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.063+0000 7f8820f05b80 1 Processor -- start 2024-03-30T02:45:29.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.065+0000 7f8820f05b80 1 -- start start 2024-03-30T02:45:29.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.066+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c6382b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.066 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.066+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 0x5627c638da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.067+0000 7f8806800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c6382b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.067+0000 7f8806800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c6382b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39466/0 (socket says 172.21.15.67:39466) 2024-03-30T02:45:29.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.067+0000 7f8806800700 1 -- 172.21.15.67:0/2572779709 learned_addr learned my addr 172.21.15.67:0/2572779709 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.067+0000 7f8820f05b80 1 --2- 172.21.15.67:0/2572779709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 0x5627c63903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.067+0000 7f8820f05b80 1 -- 172.21.15.67:0/2572779709 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c62db990 con 0x5627c63830c0 2024-03-30T02:45:29.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.067+0000 7f8820f05b80 1 -- 172.21.15.67:0/2572779709 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c62d9350 con 0x5627c63827a0 2024-03-30T02:45:29.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.067+0000 7f8820f05b80 1 -- 172.21.15.67:0/2572779709 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c62d4c40 con 0x5627c638dfc0 2024-03-30T02:45:29.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.068+0000 7f8807200700 1 --2- 172.21.15.67:0/2572779709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 0x5627c63903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.068+0000 7f8806800700 1 -- 172.21.15.67:0/2572779709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 msgr2=0x5627c63903b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.068+0000 7f8806800700 1 --2- 172.21.15.67:0/2572779709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 0x5627c63903b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.068+0000 7f8806800700 1 -- 172.21.15.67:0/2572779709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 msgr2=0x5627c638da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.068+0000 7f8806800700 1 --2- 172.21.15.67:0/2572779709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 0x5627c638da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.068+0000 7f8806800700 1 -- 172.21.15.67:0/2572779709 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627c61563b0 con 0x5627c63827a0 2024-03-30T02:45:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.071+0000 7f8806800700 1 --2- 172.21.15.67:0/2572779709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c6382b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f880000ce30 tx=0x7f8800019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41519e5ea5a36c6a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.071+0000 7f87f3400700 1 -- 172.21.15.67:0/2572779709 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800004ea0 con 0x5627c63827a0 2024-03-30T02:45:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.071+0000 7f87f3400700 1 -- 172.21.15.67:0/2572779709 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88000079a0 con 0x5627c63827a0 2024-03-30T02:45:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.071+0000 7f87f3400700 1 -- 172.21.15.67:0/2572779709 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800005a70 con 0x5627c63827a0 2024-03-30T02:45:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.072+0000 7f8820f05b80 1 -- 172.21.15.67:0/2572779709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 msgr2=0x5627c6382b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.072+0000 7f8820f05b80 1 --2- 172.21.15.67:0/2572779709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c6382b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f880000ce30 tx=0x7f8800019000 comp rx=0 tx=0).stop 2024-03-30T02:45:29.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.075+0000 7f8820f05b80 1 -- 172.21.15.67:0/2572779709 shutdown_connections 2024-03-30T02:45:29.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.075+0000 7f8820f05b80 1 --2- 172.21.15.67:0/2572779709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 0x5627c63903b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.075+0000 7f8820f05b80 1 --2- 172.21.15.67:0/2572779709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c6382b80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.075+0000 7f8820f05b80 1 --2- 172.21.15.67:0/2572779709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 0x5627c638da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.075+0000 7f8820f05b80 1 -- 172.21.15.67:0/2572779709 >> 172.21.15.67:0/2572779709 conn(0x5627c62e26a0 msgr2=0x5627c637ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.088+0000 7f8820f05b80 1 -- 172.21.15.67:0/2572779709 shutdown_connections 2024-03-30T02:45:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.088+0000 7f8820f05b80 1 -- 172.21.15.67:0/2572779709 wait complete. 2024-03-30T02:45:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.089+0000 7f8820f05b80 1 -- start start 2024-03-30T02:45:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.089+0000 7f8820f05b80 1 Processor -- start 2024-03-30T02:45:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.090+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.090+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 0x5627c638ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.090+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 0x5627c638b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.090+0000 7f8820f05b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c6270d80 con 0x5627c63830c0 2024-03-30T02:45:29.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.090+0000 7f8820f05b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c6270a20 con 0x5627c63827a0 2024-03-30T02:45:29.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.090+0000 7f8820f05b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c61586a0 con 0x5627c638dfc0 2024-03-30T02:45:29.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.090+0000 7f8820f05b80 10 client.0 _ll_register_callbacks cb 0x5627c627a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:29.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.091+0000 7f8806800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.091+0000 7f8806800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39468/0 (socket says 172.21.15.67:39468) 2024-03-30T02:45:29.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.091+0000 7f8806800700 1 -- 172.21.15.67:0/3664342117 learned_addr learned my addr 172.21.15.67:0/3664342117 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.091+0000 7f8807200700 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 0x5627c638b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.091+0000 7f8805e00700 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 0x5627c638ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.092+0000 7f8807200700 1 -- 172.21.15.67:0/3664342117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 msgr2=0x5627c638da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.092+0000 7f8807200700 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.092+0000 7f8807200700 1 -- 172.21.15.67:0/3664342117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 msgr2=0x5627c638ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.092+0000 7f8807200700 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 0x5627c638ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.093+0000 7f8807200700 1 -- 172.21.15.67:0/3664342117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880811d090 con 0x5627c638dfc0 2024-03-30T02:45:29.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.096+0000 7f8807200700 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 0x5627c638b540 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f87f80c9540 tx=0x7f87f8098a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e9faf769108caa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.096+0000 7f87e7e00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:45:29.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.097+0000 7f87e7e00700 1 -- 172.21.15.67:0/3664342117 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80993e0 con 0x5627c638dfc0 2024-03-30T02:45:29.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.097+0000 7f87e7e00700 1 -- 172.21.15.67:0/3664342117 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f87f8098f80 con 0x5627c638dfc0 2024-03-30T02:45:29.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.097+0000 7f8820f05b80 1 -- 172.21.15.67:0/3664342117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627c6295380 con 0x5627c638dfc0 2024-03-30T02:45:29.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.098+0000 7f8805e00700 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 0x5627c638ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:45:29.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.098+0000 7f87e7e00700 1 -- 172.21.15.67:0/3664342117 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f80c99d0 con 0x5627c638dfc0 2024-03-30T02:45:29.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.098+0000 7f8820f05b80 10 client.27232 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:29.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.098+0000 7f8820f05b80 1 -- 172.21.15.67:0/3664342117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5627c6102b60 con 0x5627c638dfc0 2024-03-30T02:45:29.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.099+0000 7f87e7e00700 1 -- 172.21.15.67:0/3664342117 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f80a5030 con 0x5627c638dfc0 2024-03-30T02:45:29.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.099+0000 7f87e7e00700 1 -- 172.21.15.67:0/3664342117 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87f80a0f70 con 0x5627c638dfc0 2024-03-30T02:45:29.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.099+0000 7f8820f05b80 10 client.27232 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:29.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.099+0000 7f8820f05b80 10 client.27232 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:45:29.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.099+0000 7f8820f05b80 20 client.27232 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:29.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.099+0000 7f8820f05b80 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5627c639d990 0x5627c639fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.099+0000 7f8820f05b80 4 client.27232 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:45:29.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.099+0000 7f8820f05b80 1 -- 172.21.15.67:0/3664342117 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627c62cd700 con 0x5627c639d990 2024-03-30T02:45:29.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.105+0000 7f8806800700 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5627c639d990 0x5627c639fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.109+0000 7f87e7e00700 10 client.27232 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:45:29.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.109+0000 7f8806800700 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5627c639d990 0x5627c639fdb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b32427e948f99db server_cookie=4129765d0d8f1752 in_seq=0 out_seq=0 2024-03-30T02:45:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.118+0000 7f87e7e00700 1 -- 172.21.15.67:0/3664342117 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5627c62cd700 con 0x5627c639d990 2024-03-30T02:45:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.118+0000 7f87e7e00700 10 client.27232 handle_command_reply: tid=0 2024-03-30T02:45:29.117 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.118+0000 7f8820f05b80 1 client.27232 shutdown 2024-03-30T02:45:29.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.122+0000 7f8820f05b80 1 -- 172.21.15.67:0/3664342117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 msgr2=0x5627c638b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.122+0000 7f8820f05b80 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 0x5627c638b540 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f87f80c9540 tx=0x7f87f8098a00 comp rx=0 tx=0).stop 2024-03-30T02:45:29.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.122+0000 7f8820f05b80 1 -- 172.21.15.67:0/3664342117 shutdown_connections 2024-03-30T02:45:29.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.122+0000 7f8820f05b80 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c638dfc0 0x5627c638b540 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.122+0000 7f8820f05b80 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.122+0000 7f8820f05b80 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5627c639d990 0x5627c639fdb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.122+0000 7f8820f05b80 1 --2- 172.21.15.67:0/3664342117 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c63830c0 0x5627c638ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.122+0000 7f8820f05b80 1 -- 172.21.15.67:0/3664342117 >> 172.21.15.67:0/3664342117 conn(0x5627c62e26a0 msgr2=0x5627c6384890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.122+0000 7f87e7e00700 0 client.27232 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:45:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.123+0000 7f8820f05b80 1 -- 172.21.15.67:0/3664342117 shutdown_connections 2024-03-30T02:45:29.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.123+0000 7f8820f05b80 1 -- 172.21.15.67:0/3664342117 wait complete. 2024-03-30T02:45:29.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.124+0000 7f8820f05b80 20 client.27232 trim_cache size 0 max 16384 2024-03-30T02:45:29.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.186+0000 7f8820f05b80 1 Processor -- start 2024-03-30T02:45:29.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.189+0000 7f8820f05b80 1 -- start start 2024-03-30T02:45:29.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.189+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.189+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 0x5627c6394c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.190+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.190+0000 7f8820f05b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c638bb40 con 0x5627c638dfc0 2024-03-30T02:45:29.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.190+0000 7f8820f05b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c637f9a0 con 0x5627c63827a0 2024-03-30T02:45:29.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.190+0000 7f8820f05b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c6395150 con 0x5627c63830c0 2024-03-30T02:45:29.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.190+0000 7f8806800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.190+0000 7f8806800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39478/0 (socket says 172.21.15.67:39478) 2024-03-30T02:45:29.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.190+0000 7f8806800700 1 -- 172.21.15.67:0/4136469817 learned_addr learned my addr 172.21.15.67:0/4136469817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.190+0000 7f8805e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 0x5627c6394c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.190+0000 7f8807200700 1 --2- 172.21.15.67:0/4136469817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.191+0000 7f8806800700 1 -- 172.21.15.67:0/4136469817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 msgr2=0x5627c6394c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.191+0000 7f8806800700 1 --2- 172.21.15.67:0/4136469817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 0x5627c6394c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.191+0000 7f8806800700 1 -- 172.21.15.67:0/4136469817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 msgr2=0x5627c63777f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.191+0000 7f8806800700 1 --2- 172.21.15.67:0/4136469817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63777f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.191+0000 7f8806800700 1 -- 172.21.15.67:0/4136469817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627c629aea0 con 0x5627c63827a0 2024-03-30T02:45:29.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.191+0000 7f8805e00700 1 --2- 172.21.15.67:0/4136469817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 0x5627c6394c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:45:29.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.193+0000 7f8806800700 1 --2- 172.21.15.67:0/4136469817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638adc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f880000d010 tx=0x7f8800005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cae3af8928a73934 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.194+0000 7f87f3e00700 1 -- 172.21.15.67:0/4136469817 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800007660 con 0x5627c63827a0 2024-03-30T02:45:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.194+0000 7f87f3e00700 1 -- 172.21.15.67:0/4136469817 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88000077c0 con 0x5627c63827a0 2024-03-30T02:45:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.194+0000 7f87f3e00700 1 -- 172.21.15.67:0/4136469817 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800007660 con 0x5627c63827a0 2024-03-30T02:45:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.195+0000 7f8820f05b80 1 -- 172.21.15.67:0/4136469817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 msgr2=0x5627c638adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.195+0000 7f8820f05b80 1 --2- 172.21.15.67:0/4136469817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638adc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f880000d010 tx=0x7f8800005650 comp rx=0 tx=0).stop 2024-03-30T02:45:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.198+0000 7f8820f05b80 1 -- 172.21.15.67:0/4136469817 shutdown_connections 2024-03-30T02:45:29.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.198+0000 7f8820f05b80 1 --2- 172.21.15.67:0/4136469817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63777f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.198+0000 7f8820f05b80 1 --2- 172.21.15.67:0/4136469817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c638adc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.198+0000 7f8820f05b80 1 --2- 172.21.15.67:0/4136469817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 0x5627c6394c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.198+0000 7f8820f05b80 1 -- 172.21.15.67:0/4136469817 >> 172.21.15.67:0/4136469817 conn(0x5627c62e26a0 msgr2=0x5627c638a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.199+0000 7f8820f05b80 1 -- 172.21.15.67:0/4136469817 shutdown_connections 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.199+0000 7f8820f05b80 1 -- 172.21.15.67:0/4136469817 wait complete. 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.200+0000 7f8820f05b80 1 -- start start 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.200+0000 7f8820f05b80 1 Processor -- start 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c6392970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 0x5627c6392eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8820f05b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8820f05b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c63eac90 con 0x5627c638dfc0 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8820f05b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c63eae00 con 0x5627c63827a0 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8820f05b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627c63eafc0 con 0x5627c63830c0 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8820f05b80 10 client.0 _ll_register_callbacks cb 0x5627c627a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8807200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8807200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63ea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44502/0 (socket says 172.21.15.67:44502) 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.202+0000 7f8807200700 1 -- 172.21.15.67:0/1052772302 learned_addr learned my addr 172.21.15.67:0/1052772302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.203+0000 7f8807200700 1 -- 172.21.15.67:0/1052772302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 msgr2=0x5627c6392eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.203+0000 7f8807200700 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 0x5627c6392eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.203+0000 7f8807200700 1 -- 172.21.15.67:0/1052772302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 msgr2=0x5627c6392970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.203+0000 7f8807200700 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c6392970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.203+0000 7f8807200700 1 -- 172.21.15.67:0/1052772302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f4009c90 con 0x5627c638dfc0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.204+0000 7f8807200700 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63ea750 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f87f8098f10 tx=0x7f87f80988e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4d1bc96383ec7e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.204+0000 7f87e7e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.207+0000 7f87e7e00700 1 -- 172.21.15.67:0/1052772302 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80b82b0 con 0x5627c638dfc0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.207+0000 7f87e7e00700 1 -- 172.21.15.67:0/1052772302 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f87f8098f80 con 0x5627c638dfc0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.208+0000 7f8820f05b80 1 -- 172.21.15.67:0/1052772302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88000079a0 con 0x5627c638dfc0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.208+0000 7f87e7e00700 1 -- 172.21.15.67:0/1052772302 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f80cc020 con 0x5627c638dfc0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.211+0000 7f8820f05b80 10 client.17157 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.211+0000 7f8820f05b80 1 -- 172.21.15.67:0/1052772302 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5627c61563b0 con 0x5627c638dfc0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.211+0000 7f87e7e00700 1 -- 172.21.15.67:0/1052772302 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88000079a0 con 0x5627c638dfc0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.211+0000 7f87e7e00700 1 -- 172.21.15.67:0/1052772302 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f87f80a0f70 con 0x5627c638dfc0 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.212+0000 7f8820f05b80 10 client.17157 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:29.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.212+0000 7f8820f05b80 10 client.17157 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:45:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.212+0000 7f8820f05b80 20 client.17157 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.212+0000 7f8820f05b80 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5627c63f2370 0x5627c63f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.212+0000 7f8820f05b80 4 client.17157 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:45:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.212+0000 7f8820f05b80 1 -- 172.21.15.67:0/1052772302 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627c6123cf0 con 0x5627c63f2370 2024-03-30T02:45:29.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.213+0000 7f8806800700 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5627c63f2370 0x5627c63f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.214+0000 7f8806800700 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5627c63f2370 0x5627c63f4760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4cc31e860906ba12 server_cookie=c6d2c48dd38495d5 in_seq=0 out_seq=0 2024-03-30T02:45:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.214+0000 7f87e7e00700 10 client.17157 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:45:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.215+0000 7f87e7e00700 1 -- 172.21.15.67:0/1052772302 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627c6123cf0 con 0x5627c63f2370 2024-03-30T02:45:29.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.215+0000 7f87e7e00700 10 client.17157 handle_command_reply: tid=0 2024-03-30T02:45:29.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.215+0000 7f8820f05b80 1 client.17157 shutdown 2024-03-30T02:45:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.218+0000 7f8820f05b80 1 -- 172.21.15.67:0/1052772302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 msgr2=0x5627c63ea750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.218+0000 7f8820f05b80 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63ea750 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f87f8098f10 tx=0x7f87f80988e0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.218+0000 7f8820f05b80 1 -- 172.21.15.67:0/1052772302 shutdown_connections 2024-03-30T02:45:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.218+0000 7f8820f05b80 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5627c63f2370 0x5627c63f4760 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.218+0000 7f8820f05b80 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5627c638dfc0 0x5627c63ea750 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.219+0000 7f8820f05b80 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627c63827a0 0x5627c6392970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.219+0000 7f8820f05b80 1 --2- 172.21.15.67:0/1052772302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627c63830c0 0x5627c6392eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.219+0000 7f8820f05b80 1 -- 172.21.15.67:0/1052772302 >> 172.21.15.67:0/1052772302 conn(0x5627c62e26a0 msgr2=0x5627c6399360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.218+0000 7f87e7e00700 0 client.17157 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:45:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.219+0000 7f8820f05b80 1 -- 172.21.15.67:0/1052772302 shutdown_connections 2024-03-30T02:45:29.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.220+0000 7f8820f05b80 1 -- 172.21.15.67:0/1052772302 wait complete. 2024-03-30T02:45:29.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.225+0000 7f8820f05b80 20 client.17157 trim_cache size 0 max 16384 2024-03-30T02:45:29.227 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:45:29.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 -- 172.21.15.67:0/3132231439 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f87ec0a14a0 msgr2=0x7f87ec0a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f87ec0a14a0 0x7f87ec0a3960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f880000b780 tx=0x7f88000094d0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 -- 172.21.15.67:0/3132231439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f880811a240 msgr2=0x7f88081aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f880811a240 0x7f88081aa6c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f87f800c770 tx=0x7f87f800cb30 comp rx=0 tx=0).stop 2024-03-30T02:45:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 -- 172.21.15.67:0/3132231439 shutdown_connections 2024-03-30T02:45:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f87ec0a14a0 0x7f87ec0a3960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f880810ce90 0x7f88081a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f880811a240 0x7f88081aa6c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 --2- 172.21.15.67:0/3132231439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f880810d7b0 0x7f88081a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.230+0000 7f880ca00700 1 -- 172.21.15.67:0/3132231439 >> 172.21.15.67:0/3132231439 conn(0x7f880806bc10 msgr2=0x7f88080716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.231+0000 7f880ca00700 1 -- 172.21.15.67:0/3132231439 shutdown_connections 2024-03-30T02:45:29.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.231+0000 7f880ca00700 1 -- 172.21.15.67:0/3132231439 wait complete. 2024-03-30T02:45:29.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:45:29.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.635+0000 7f3abca00700 1 -- 172.21.15.67:0/1795448795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab8106080 msgr2=0x7f3ab810dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.635+0000 7f3abca00700 1 --2- 172.21.15.67:0/1795448795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab8106080 0x7f3ab810dc00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009f90 tx=0x7f3ab000b330 comp rx=0 tx=0).stop 2024-03-30T02:45:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.635+0000 7f3abca00700 1 -- 172.21.15.67:0/1795448795 shutdown_connections 2024-03-30T02:45:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.635+0000 7f3abca00700 1 --2- 172.21.15.67:0/1795448795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8100f60 0x7f3ab8101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.635+0000 7f3abca00700 1 --2- 172.21.15.67:0/1795448795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab8106080 0x7f3ab810dc00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.635+0000 7f3abca00700 1 --2- 172.21.15.67:0/1795448795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ab8101910 0x7f3ab8105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.635+0000 7f3abca00700 1 -- 172.21.15.67:0/1795448795 >> 172.21.15.67:0/1795448795 conn(0x7f3ab80fc7c0 msgr2=0x7f3ab80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.635+0000 7f3abca00700 1 -- 172.21.15.67:0/1795448795 shutdown_connections 2024-03-30T02:45:29.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.635+0000 7f3abca00700 1 -- 172.21.15.67:0/1795448795 wait complete. 2024-03-30T02:45:29.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.636+0000 7f3abca00700 1 Processor -- start 2024-03-30T02:45:29.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.636+0000 7f3abca00700 1 -- start start 2024-03-30T02:45:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.636+0000 7f3abca00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8100f60 0x7f3ab8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.636+0000 7f3abca00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab8106080 0x7f3ab8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.636+0000 7f3abca00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ab819dba0 0x7f3ab819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.636+0000 7f3abca00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab819e620 con 0x7f3ab8100f60 2024-03-30T02:45:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.636+0000 7f3abca00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab819e790 con 0x7f3ab8106080 2024-03-30T02:45:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.636+0000 7f3abca00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab819e950 con 0x7f3ab819dba0 2024-03-30T02:45:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8100f60 0x7f3ab8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8100f60 0x7f3ab8199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44508/0 (socket says 172.21.15.67:44508) 2024-03-30T02:45:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5e00700 1 -- 172.21.15.67:0/2601180112 learned_addr learned my addr 172.21.15.67:0/2601180112 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab6800700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ab819dba0 0x7f3ab819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5400700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab8106080 0x7f3ab8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5e00700 1 -- 172.21.15.67:0/2601180112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ab819dba0 msgr2=0x7f3ab819e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ab819dba0 0x7f3ab819e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5e00700 1 -- 172.21.15.67:0/2601180112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab8106080 msgr2=0x7f3ab8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab8106080 0x7f3ab8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5e00700 1 -- 172.21.15.67:0/2601180112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab0009c40 con 0x7f3ab8100f60 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.637+0000 7f3ab5400700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab8106080 0x7f3ab8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.638+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8100f60 0x7f3ab8199310 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e9c0 tx=0x7f3aa800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.638+0000 7f3aaea00700 1 -- 172.21.15.67:0/2601180112 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8004a30 con 0x7f3ab8100f60 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.638+0000 7f3aaea00700 1 -- 172.21.15.67:0/2601180112 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aa800cc80 con 0x7f3ab8100f60 2024-03-30T02:45:29.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.638+0000 7f3abca00700 1 -- 172.21.15.67:0/2601180112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab81a5eb0 con 0x7f3ab8100f60 2024-03-30T02:45:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.638+0000 7f3aaea00700 1 -- 172.21.15.67:0/2601180112 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80054e0 con 0x7f3ab8100f60 2024-03-30T02:45:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.638+0000 7f3abca00700 1 -- 172.21.15.67:0/2601180112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab81a6380 con 0x7f3ab8100f60 2024-03-30T02:45:29.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.643+0000 7f3aaea00700 1 -- 172.21.15.67:0/2601180112 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3aa8004b90 con 0x7f3ab8100f60 2024-03-30T02:45:29.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.643+0000 7f3ad05b1b80 1 Processor -- start 2024-03-30T02:45:29.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.644+0000 7f3ad05b1b80 1 -- start start 2024-03-30T02:45:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.644+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cb5d0 0x560ad29cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.644+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.644+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.644+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad2924990 con 0x560ad29cb5d0 2024-03-30T02:45:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.644+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad2922350 con 0x560ad29cbef0 2024-03-30T02:45:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.644+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad291dc40 con 0x560ad29d6df0 2024-03-30T02:45:29.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56884/0 (socket says 172.21.15.67:56884) 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5400700 1 -- 172.21.15.67:0/3595105779 learned_addr learned my addr 172.21.15.67:0/3595105779 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/3595105779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab6800700 1 --2- 172.21.15.67:0/3595105779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cb5d0 0x560ad29cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5400700 1 -- 172.21.15.67:0/3595105779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 msgr2=0x560ad29d68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5400700 1 --2- 172.21.15.67:0/3595105779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5400700 1 -- 172.21.15.67:0/3595105779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cb5d0 msgr2=0x560ad29cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5400700 1 --2- 172.21.15.67:0/3595105779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cb5d0 0x560ad29cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5400700 1 -- 172.21.15.67:0/3595105779 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ad279f3b0 con 0x560ad29d6df0 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.645+0000 7f3ab5400700 1 --2- 172.21.15.67:0/3595105779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d91e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40103d0 tx=0x7f3aa40106e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f73b7446e3cc1de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.646+0000 7f3aaea00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a9c0a1310 0x7f3a9c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.646+0000 7f3aa3000700 1 -- 172.21.15.67:0/3595105779 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4011640 con 0x560ad29d6df0 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.646+0000 7f3aa3000700 1 -- 172.21.15.67:0/3595105779 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aa4015070 con 0x560ad29d6df0 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.646+0000 7f3aa3000700 1 -- 172.21.15.67:0/3595105779 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4019960 con 0x560ad29d6df0 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.646+0000 7f3aaea00700 1 -- 172.21.15.67:0/2601180112 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f3aa8014070 con 0x7f3ab8100f60 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.646+0000 7f3ab6800700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a9c0a1310 0x7f3a9c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/3595105779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 msgr2=0x560ad29d91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/3595105779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d91e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40103d0 tx=0x7f3aa40106e0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ab6800700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a9c0a1310 0x7f3a9c0a37d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009550 tx=0x7f3ab00096c0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/3595105779 shutdown_connections 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/3595105779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cb5d0 0x560ad29cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/3595105779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d91e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/3595105779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/3595105779 >> 172.21.15.67:0/3595105779 conn(0x560ad292b6a0 msgr2=0x560ad29c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/3595105779 shutdown_connections 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.647+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/3595105779 wait complete. 2024-03-30T02:45:29.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.648+0000 7f3ad05b1b80 1 -- start start 2024-03-30T02:45:29.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.648+0000 7f3ad05b1b80 1 Processor -- start 2024-03-30T02:45:29.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.649+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 0x560ad29d6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.649+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cbef0 0x560ad29d3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.649+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.649+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad28b9d80 con 0x560ad29cbef0 2024-03-30T02:45:29.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.649+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad28b9a20 con 0x560ad29cb5d0 2024-03-30T02:45:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.649+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad27a16a0 con 0x560ad29d6df0 2024-03-30T02:45:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.649+0000 7f3ad05b1b80 10 client.0 _ll_register_callbacks cb 0x560ad28c3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.650+0000 7f3ab6800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.650+0000 7f3ab6800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56896/0 (socket says 172.21.15.67:56896) 2024-03-30T02:45:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.650+0000 7f3ab6800700 1 -- 172.21.15.67:0/4054548795 learned_addr learned my addr 172.21.15.67:0/4054548795 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.650+0000 7f3ab5400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cbef0 0x560ad29d3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.651+0000 7f3ab6800700 1 -- 172.21.15.67:0/4054548795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 msgr2=0x560ad29d6780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.651+0000 7f3ab6800700 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 0x560ad29d6780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.651+0000 7f3ab6800700 1 -- 172.21.15.67:0/4054548795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cbef0 msgr2=0x560ad29d3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.651+0000 7f3ab6800700 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cbef0 0x560ad29d3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.651+0000 7f3ab6800700 1 -- 172.21.15.67:0/4054548795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa4007c10 con 0x560ad29d6df0 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.651+0000 7f3ab6800700 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d4060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009f90 tx=0x7f3ab000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f92e529ebc4285 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.651+0000 7f3a97400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.652+0000 7f3a97400700 1 -- 172.21.15.67:0/4054548795 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab000e040 con 0x560ad29d6df0 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.652+0000 7f3a97400700 1 -- 172.21.15.67:0/4054548795 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f3ab0007880 con 0x560ad29d6df0 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.652+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/4054548795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ad28de380 con 0x560ad29d6df0 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.652+0000 7f3a97400700 1 -- 172.21.15.67:0/4054548795 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ab0032950 con 0x560ad29d6df0 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.652+0000 7f3ad05b1b80 10 client.27247 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:29.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.652+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/4054548795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560ad274bb60 con 0x560ad29d6df0 2024-03-30T02:45:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.653+0000 7f3a97400700 1 -- 172.21.15.67:0/4054548795 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ad28de380 con 0x560ad29d6df0 2024-03-30T02:45:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.653+0000 7f3a97400700 1 -- 172.21.15.67:0/4054548795 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ab0024070 con 0x560ad29d6df0 2024-03-30T02:45:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.653+0000 7f3ad05b1b80 10 client.27247 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.653+0000 7f3ad05b1b80 10 client.27247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:45:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.653+0000 7f3ad05b1b80 20 client.27247 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.653+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560ad29e6910 0x560ad29e8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.653+0000 7f3ad05b1b80 4 client.27247 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:45:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.653+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/4054548795 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ad2916700 con 0x560ad29e6910 2024-03-30T02:45:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.654+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560ad29e6910 0x560ad29e8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.654+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560ad29e6910 0x560ad29e8d30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7092e49635709113 server_cookie=229cef2e4fc25419 in_seq=0 out_seq=0 2024-03-30T02:45:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.654+0000 7f3a97400700 10 client.27247 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.655+0000 7f3a97400700 1 -- 172.21.15.67:0/4054548795 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560ad2916700 con 0x560ad29e6910 2024-03-30T02:45:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.655+0000 7f3a97400700 10 client.27247 handle_command_reply: tid=0 2024-03-30T02:45:29.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.655+0000 7f3ad05b1b80 1 client.27247 shutdown 2024-03-30T02:45:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.657+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/4054548795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 msgr2=0x560ad29d4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.657+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d4060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009f90 tx=0x7f3ab000b330 comp rx=0 tx=0).stop 2024-03-30T02:45:29.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.658+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/4054548795 shutdown_connections 2024-03-30T02:45:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.658+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29d6df0 0x560ad29d4060 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.658+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 0x560ad29d6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.658+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560ad29e6910 0x560ad29e8d30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.658+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/4054548795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29cbef0 0x560ad29d3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.658+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/4054548795 >> 172.21.15.67:0/4054548795 conn(0x560ad292b6a0 msgr2=0x560ad29cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.658+0000 7f3a97400700 0 client.27247 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.660+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/4054548795 shutdown_connections 2024-03-30T02:45:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.660+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/4054548795 wait complete. 2024-03-30T02:45:29.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.661+0000 7f3ad05b1b80 20 client.27247 trim_cache size 0 max 16384 2024-03-30T02:45:29.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.689+0000 7f3ad05b1b80 1 Processor -- start 2024-03-30T02:45:29.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.689+0000 7f3ad05b1b80 1 -- start start 2024-03-30T02:45:29.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.689+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cb5d0 0x560ad29ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.689+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.689+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad29e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.689+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad29e34e0 con 0x560ad29d6df0 2024-03-30T02:45:29.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.689+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad29d50a0 con 0x560ad29cbef0 2024-03-30T02:45:29.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.689+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad29cd140 con 0x560ad29cb5d0 2024-03-30T02:45:29.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.689+0000 7f3ab5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.690+0000 7f3ab5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39526/0 (socket says 172.21.15.67:39526) 2024-03-30T02:45:29.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.690+0000 7f3ab5400700 1 -- 172.21.15.67:0/145656934 learned_addr learned my addr 172.21.15.67:0/145656934 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.690+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cb5d0 0x560ad29ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.690+0000 7f3ab5400700 1 -- 172.21.15.67:0/145656934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cb5d0 msgr2=0x560ad29ddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.690+0000 7f3ab5400700 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cb5d0 0x560ad29ddaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.690+0000 7f3ab5400700 1 -- 172.21.15.67:0/145656934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 msgr2=0x560ad29e21f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.690+0000 7f3ab6800700 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad29e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.690+0000 7f3ab5400700 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad29e21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.691+0000 7f3ab5400700 1 -- 172.21.15.67:0/145656934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ad28e3ea0 con 0x560ad29cbef0 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.691+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cb5d0 0x560ad29ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.691+0000 7f3ab6800700 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad29e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.691+0000 7f3ab5400700 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d62b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40076a0 tx=0x7f3aa40058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9316cd713385bb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.691+0000 7f3aa3a00700 1 -- 172.21.15.67:0/145656934 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4005300 con 0x560ad29cbef0 2024-03-30T02:45:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.691+0000 7f3aa3a00700 1 -- 172.21.15.67:0/145656934 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aa4005460 con 0x560ad29cbef0 2024-03-30T02:45:29.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.691+0000 7f3aa3a00700 1 -- 172.21.15.67:0/145656934 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40197b0 con 0x560ad29cbef0 2024-03-30T02:45:29.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.692+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/145656934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 msgr2=0x560ad29d62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.692+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d62b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40076a0 tx=0x7f3aa40058e0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.693+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/145656934 shutdown_connections 2024-03-30T02:45:29.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.693+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad29e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.693+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cb5d0 0x560ad29ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.693+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/145656934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cbef0 0x560ad29d62b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.693+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/145656934 >> 172.21.15.67:0/145656934 conn(0x560ad292b6a0 msgr2=0x560ad29d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.694+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/145656934 shutdown_connections 2024-03-30T02:45:29.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.695+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/145656934 wait complete. 2024-03-30T02:45:29.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.695+0000 7f3ad05b1b80 1 -- start start 2024-03-30T02:45:29.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.696+0000 7f3ad05b1b80 1 Processor -- start 2024-03-30T02:45:29.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 0x560ad29db7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cbef0 0x560ad29dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ad05b1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad2a52f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad29dc3e0 con 0x560ad29d6df0 2024-03-30T02:45:29.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad2a534a0 con 0x560ad29cb5d0 2024-03-30T02:45:29.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ad05b1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ad2a535e0 con 0x560ad29cbef0 2024-03-30T02:45:29.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ad05b1b80 10 client.0 _ll_register_callbacks cb 0x560ad28c3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:29.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ab5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 0x560ad29db7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ab5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 0x560ad29db7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39536/0 (socket says 172.21.15.67:39536) 2024-03-30T02:45:29.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.698+0000 7f3ab5e00700 1 -- 172.21.15.67:0/1647085777 learned_addr learned my addr 172.21.15.67:0/1647085777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:29.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.699+0000 7f3ab5e00700 1 -- 172.21.15.67:0/1647085777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cbef0 msgr2=0x560ad29dbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.699+0000 7f3ab6800700 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad2a52f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.699+0000 7f3ab5400700 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cbef0 0x560ad29dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.700+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cbef0 0x560ad29dbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.700+0000 7f3ab5e00700 1 -- 172.21.15.67:0/1647085777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 msgr2=0x560ad2a52f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:29.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.700+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad2a52f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.700+0000 7f3ab5e00700 1 -- 172.21.15.67:0/1647085777 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ad279f3b0 con 0x560ad29cb5d0 2024-03-30T02:45:29.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.700+0000 7f3ab5400700 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cbef0 0x560ad29dbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:29.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.700+0000 7f3ab6800700 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad2a52f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:29.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.701+0000 7f3ab5e00700 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 0x560ad29db7e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c5b0 tx=0x7f3aa8009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eba59d6add7105c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:29.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.702+0000 7f3a97400700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:45:29.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.702+0000 7f3a97400700 1 -- 172.21.15.67:0/1647085777 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa809dc60 con 0x560ad29cb5d0 2024-03-30T02:45:29.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.702+0000 7f3a97400700 1 -- 172.21.15.67:0/1647085777 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f3aa809ae40 con 0x560ad29cb5d0 2024-03-30T02:45:29.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.702+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/1647085777 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3aa4015070 con 0x560ad29cb5d0 2024-03-30T02:45:29.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.702+0000 7f3a97400700 1 -- 172.21.15.67:0/1647085777 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aa80a22b0 con 0x560ad29cb5d0 2024-03-30T02:45:29.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.704+0000 7f3ad05b1b80 10 client.27308 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:29.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.704+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/1647085777 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560ad27a3cd0 con 0x560ad29cb5d0 2024-03-30T02:45:29.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.705+0000 7f3a97400700 1 -- 172.21.15.67:0/1647085777 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3aa4015070 con 0x560ad29cb5d0 2024-03-30T02:45:29.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.705+0000 7f3a97400700 1 -- 172.21.15.67:0/1647085777 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3aa8099320 con 0x560ad29cb5d0 2024-03-30T02:45:29.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.705+0000 7f3ad05b1b80 10 client.27308 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:29.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.705+0000 7f3ad05b1b80 10 client.27308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:45:29.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.705+0000 7f3ad05b1b80 20 client.27308 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:29.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.705+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560ad2a54c80 0x560ad2a570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:29.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.705+0000 7f3ad05b1b80 4 client.27308 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:45:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.705+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/1647085777 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560ad276ccf0 con 0x560ad2a54c80 2024-03-30T02:45:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.705+0000 7f3ab5400700 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560ad2a54c80 0x560ad2a570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.706+0000 7f3ab5400700 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560ad2a54c80 0x560ad2a570a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995c5711154146c4 server_cookie=3c7ab15444364143 in_seq=0 out_seq=0 2024-03-30T02:45:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.706+0000 7f3a97400700 10 client.27308 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.706+0000 7f3a97400700 1 -- 172.21.15.67:0/1647085777 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560ad276ccf0 con 0x560ad2a54c80 2024-03-30T02:45:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.706+0000 7f3a97400700 10 client.27308 handle_command_reply: tid=0 2024-03-30T02:45:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.706+0000 7f3ad05b1b80 1 client.27308 shutdown 2024-03-30T02:45:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.706+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/1647085777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 msgr2=0x560ad29db7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.706+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 0x560ad29db7e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c5b0 tx=0x7f3aa8009550 comp rx=0 tx=0).stop 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/1647085777 shutdown_connections 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560ad2a54c80 0x560ad2a570a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560ad29d6df0 0x560ad2a52f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560ad29cb5d0 0x560ad29db7e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3ad05b1b80 1 --2- 172.21.15.67:0/1647085777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ad29cbef0 0x560ad29dbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/1647085777 >> 172.21.15.67:0/1647085777 conn(0x560ad292b6a0 msgr2=0x560ad29cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3a97400700 0 client.27308 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/1647085777 shutdown_connections 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3ad05b1b80 1 -- 172.21.15.67:0/1647085777 wait complete. 2024-03-30T02:45:29.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.707+0000 7f3ad05b1b80 20 client.27308 trim_cache size 0 max 16384 2024-03-30T02:45:29.708 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:45:29.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.708+0000 7f3abca00700 1 -- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a9c0a1310 msgr2=0x7f3a9c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.708+0000 7f3abca00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a9c0a1310 0x7f3a9c0a37d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009550 tx=0x7f3ab00096c0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.708+0000 7f3abca00700 1 -- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8100f60 msgr2=0x7f3ab8199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.708+0000 7f3abca00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8100f60 0x7f3ab8199310 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e9c0 tx=0x7f3aa800ecd0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.709+0000 7f3abca00700 1 -- 172.21.15.67:0/2601180112 shutdown_connections 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.709+0000 7f3abca00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3a9c0a1310 0x7f3a9c0a37d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.709+0000 7f3abca00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ab8100f60 0x7f3ab8199310 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.709+0000 7f3abca00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ab819dba0 0x7f3ab819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.709+0000 7f3abca00700 1 --2- 172.21.15.67:0/2601180112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab8106080 0x7f3ab8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.709+0000 7f3abca00700 1 -- 172.21.15.67:0/2601180112 >> 172.21.15.67:0/2601180112 conn(0x7f3ab80fc7c0 msgr2=0x7f3ab80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.709+0000 7f3abca00700 1 -- 172.21.15.67:0/2601180112 shutdown_connections 2024-03-30T02:45:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:29.709+0000 7f3abca00700 1 -- 172.21.15.67:0/2601180112 wait complete. 2024-03-30T02:45:29.844 INFO:tasks.workunit.client.0.smithi067.stdout: AR kernel/built-in.a 2024-03-30T02:45:30.012 INFO:tasks.workunit.client.0.smithi067.stderr:make[1]: *** [/home/ubuntu/cephtest/mnt.0/client.0/tmp/t/linux-6.5.11/Makefile:2034: .] Error 2 2024-03-30T02:45:30.013 INFO:tasks.workunit.client.0.smithi067.stderr:make: *** [Makefile:234: __sub-make] Error 2 2024-03-30T02:45:30.014 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-30T02:45:30.014 INFO:tasks.workunit:Stopping ['kernel_untar_build.sh'] on client.0... 2024-03-30T02:45:30.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-30T02:45:30.728 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:45:30.728 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:45:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:30 smithi082 ceph-mon[79176]: pgmap v260: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 932 KiB/s wr, 30 op/s 2024-03-30T02:45:30.786 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:30 smithi067 ceph-mon[37808]: pgmap v260: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 932 KiB/s wr, 30 op/s 2024-03-30T02:45:30.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:30 smithi170 ceph-mon[78798]: pgmap v260: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 932 KiB/s wr, 30 op/s 2024-03-30T02:45:30.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.951+0000 7f0323600700 1 -- 172.21.15.67:0/2468209787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 msgr2=0x7f031c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:30.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.951+0000 7f0323600700 1 --2- 172.21.15.67:0/2468209787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 0x7f031c100230 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0310009990 tx=0x7f0310009ca0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.951+0000 7f0323600700 1 -- 172.21.15.67:0/2468209787 shutdown_connections 2024-03-30T02:45:30.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.951+0000 7f0323600700 1 --2- 172.21.15.67:0/2468209787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031c104f60 0x7f031c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.951+0000 7f0323600700 1 --2- 172.21.15.67:0/2468209787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 0x7f031c100230 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.951+0000 7f0323600700 1 --2- 172.21.15.67:0/2468209787 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f031c100800 0x7f031c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.951+0000 7f0323600700 1 -- 172.21.15.67:0/2468209787 >> 172.21.15.67:0/2468209787 conn(0x7f031c076d40 msgr2=0x7f031c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:30.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.952+0000 7f0323600700 1 -- 172.21.15.67:0/2468209787 shutdown_connections 2024-03-30T02:45:30.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.952+0000 7f0323600700 1 -- 172.21.15.67:0/2468209787 wait complete. 2024-03-30T02:45:30.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.953+0000 7f0323600700 1 Processor -- start 2024-03-30T02:45:30.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.954+0000 7f0323600700 1 -- start start 2024-03-30T02:45:30.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.954+0000 7f0323600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 0x7f031c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.954+0000 7f0323600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031c104f60 0x7f031c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.954+0000 7f0323600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f031c19db10 0x7f031c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.954+0000 7f0323600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c19e590 con 0x7f031c19db10 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.954+0000 7f0323600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c19e700 con 0x7f031c104f60 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.954+0000 7f0323600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c19e8c0 con 0x7f031c0ffe50 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.954+0000 7f0320c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 0x7f031c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.955+0000 7f0320c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 0x7f031c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56950/0 (socket says 172.21.15.67:56950) 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.955+0000 7f0320c00700 1 -- 172.21.15.67:0/1531647649 learned_addr learned my addr 172.21.15.67:0/1531647649 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.955+0000 7f0321600700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f031c19db10 0x7f031c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.955+0000 7f0320c00700 1 -- 172.21.15.67:0/1531647649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031c104f60 msgr2=0x7f031c199770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.955+0000 7f0320c00700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031c104f60 0x7f031c199770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.955+0000 7f0320c00700 1 -- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f031c19db10 msgr2=0x7f031c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.955+0000 7f0320c00700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f031c19db10 0x7f031c19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.955+0000 7f0320c00700 1 -- 172.21.15.67:0/1531647649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03100095f0 con 0x7f031c0ffe50 2024-03-30T02:45:30.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.956+0000 7f0320c00700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 0x7f031c199230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f031000b2e0 tx=0x7f03100057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.960+0000 7f0319600700 1 -- 172.21.15.67:0/1531647649 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031001d070 con 0x7f031c0ffe50 2024-03-30T02:45:30.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.960+0000 7f0319600700 1 -- 172.21.15.67:0/1531647649 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f031000fcc0 con 0x7f031c0ffe50 2024-03-30T02:45:30.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.960+0000 7f0319600700 1 -- 172.21.15.67:0/1531647649 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0310017650 con 0x7f031c0ffe50 2024-03-30T02:45:30.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.960+0000 7f0323600700 1 -- 172.21.15.67:0/1531647649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031c1069e0 con 0x7f031c0ffe50 2024-03-30T02:45:30.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.960+0000 7f0323600700 1 -- 172.21.15.67:0/1531647649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031c106de0 con 0x7f031c0ffe50 2024-03-30T02:45:30.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.972+0000 7f0319600700 1 -- 172.21.15.67:0/1531647649 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f03100177d0 con 0x7f031c0ffe50 2024-03-30T02:45:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.976+0000 7f0319600700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a1460 0x7f03040a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.977+0000 7f03371cfb80 1 Processor -- start 2024-03-30T02:45:30.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.977+0000 7f03371cfb80 1 -- start start 2024-03-30T02:45:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 0x55ee4c61aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61af70 0x55ee4c625930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c625e70 0x55ee4c628260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f03371cfb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c573a10 con 0x55ee4c61af70 2024-03-30T02:45:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f03371cfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c5713d0 con 0x55ee4c61a650 2024-03-30T02:45:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f03371cfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c56ccc0 con 0x55ee4c625e70 2024-03-30T02:45:30.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f031be00700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a1460 0x7f03040a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f031be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61af70 0x55ee4c625930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f031be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61af70 0x55ee4c625930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44576/0 (socket says 172.21.15.67:44576) 2024-03-30T02:45:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f031be00700 1 -- 172.21.15.67:0/4198189036 learned_addr learned my addr 172.21.15.67:0/4198189036 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.978+0000 7f0321600700 1 --2- 172.21.15.67:0/4198189036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c625e70 0x55ee4c628260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:30.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.979+0000 7f0319600700 1 -- 172.21.15.67:0/1531647649 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f03100ce670 con 0x7f031c0ffe50 2024-03-30T02:45:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.979+0000 7f031be00700 1 -- 172.21.15.67:0/4198189036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c625e70 msgr2=0x55ee4c628260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.979+0000 7f031be00700 1 --2- 172.21.15.67:0/4198189036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c625e70 0x55ee4c628260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.979+0000 7f031be00700 1 -- 172.21.15.67:0/4198189036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 msgr2=0x55ee4c61aa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T02:45:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.979+0000 7f031be00700 1 --2- 172.21.15.67:0/4198189036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 0x55ee4c61aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.980+0000 7f031be00700 1 -- 172.21.15.67:0/4198189036 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee4c3ee430 con 0x55ee4c61af70 2024-03-30T02:45:30.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.980+0000 7f031be00700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a1460 0x7f03040a3920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03080124b0 tx=0x7f030800aa00 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.981+0000 7f031be00700 1 --2- 172.21.15.67:0/4198189036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61af70 0x55ee4c625930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0308012c00 tx=0x7f0308017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3840daba72ad6669 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.981+0000 7f030dc00700 1 -- 172.21.15.67:0/4198189036 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308017070 con 0x55ee4c61af70 2024-03-30T02:45:30.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.981+0000 7f030dc00700 1 -- 172.21.15.67:0/4198189036 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f030800fec0 con 0x55ee4c61af70 2024-03-30T02:45:30.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.981+0000 7f030dc00700 1 -- 172.21.15.67:0/4198189036 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308005180 con 0x55ee4c61af70 2024-03-30T02:45:30.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.982+0000 7f03371cfb80 1 -- 172.21.15.67:0/4198189036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61af70 msgr2=0x55ee4c625930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:30.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.982+0000 7f03371cfb80 1 --2- 172.21.15.67:0/4198189036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61af70 0x55ee4c625930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0308012c00 tx=0x7f0308017000 comp rx=0 tx=0).stop 2024-03-30T02:45:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.983+0000 7f03371cfb80 1 -- 172.21.15.67:0/4198189036 shutdown_connections 2024-03-30T02:45:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.983+0000 7f03371cfb80 1 --2- 172.21.15.67:0/4198189036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c625e70 0x55ee4c628260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.984+0000 7f03371cfb80 1 --2- 172.21.15.67:0/4198189036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 0x55ee4c61aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.984+0000 7f03371cfb80 1 --2- 172.21.15.67:0/4198189036 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61af70 0x55ee4c625930 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.984+0000 7f03371cfb80 1 -- 172.21.15.67:0/4198189036 >> 172.21.15.67:0/4198189036 conn(0x55ee4c57a720 msgr2=0x55ee4c616b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.984+0000 7f03371cfb80 1 -- 172.21.15.67:0/4198189036 shutdown_connections 2024-03-30T02:45:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.984+0000 7f03371cfb80 1 -- 172.21.15.67:0/4198189036 wait complete. 2024-03-30T02:45:30.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.985+0000 7f03371cfb80 1 -- start start 2024-03-30T02:45:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.985+0000 7f03371cfb80 1 Processor -- start 2024-03-30T02:45:30.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.986+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 0x55ee4c6258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.986+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c622c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.986+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c6233a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.987+0000 7f03371cfb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c508e00 con 0x55ee4c61a650 2024-03-30T02:45:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.987+0000 7f03371cfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c508aa0 con 0x55ee4c625e70 2024-03-30T02:45:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.987+0000 7f03371cfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c3f0720 con 0x55ee4c61af70 2024-03-30T02:45:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.987+0000 7f03371cfb80 10 client.0 _ll_register_callbacks cb 0x55ee4c5124f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.987+0000 7f0320c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 0x55ee4c6258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:30.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.987+0000 7f0321600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c6233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.987+0000 7f0321600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c6233a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39568/0 (socket says 172.21.15.67:39568) 2024-03-30T02:45:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.987+0000 7f0321600700 1 -- 172.21.15.67:0/247330164 learned_addr learned my addr 172.21.15.67:0/247330164 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.988+0000 7f031be00700 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c622c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.988+0000 7f0321600700 1 -- 172.21.15.67:0/247330164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 msgr2=0x55ee4c622c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.988+0000 7f0321600700 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c622c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.988+0000 7f0321600700 1 -- 172.21.15.67:0/247330164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 msgr2=0x55ee4c6258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.988+0000 7f0321600700 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 0x55ee4c6258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.988+0000 7f0321600700 1 -- 172.21.15.67:0/247330164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0308012e70 con 0x55ee4c625e70 2024-03-30T02:45:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.988+0000 7f0320c00700 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 0x55ee4c6258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.989+0000 7f0321600700 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c6233a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0314007bf0 tx=0x7f0314012240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e1e77616c0a002 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.989+0000 7f0302000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:45:30.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.989+0000 7f0302000700 1 -- 172.21.15.67:0/247330164 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031400e070 con 0x55ee4c625e70 2024-03-30T02:45:30.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.989+0000 7f0302000700 1 -- 172.21.15.67:0/247330164 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f0314012ca0 con 0x55ee4c625e70 2024-03-30T02:45:30.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.989+0000 7f03371cfb80 1 -- 172.21.15.67:0/247330164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee4c565780 con 0x55ee4c625e70 2024-03-30T02:45:30.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.990+0000 7f0302000700 1 -- 172.21.15.67:0/247330164 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0314012e60 con 0x55ee4c625e70 2024-03-30T02:45:30.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.990+0000 7f03371cfb80 10 client.27314 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:30.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.990+0000 7f03371cfb80 1 -- 172.21.15.67:0/247330164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ee4c39abe0 con 0x55ee4c625e70 2024-03-30T02:45:30.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.990+0000 7f0302000700 1 -- 172.21.15.67:0/247330164 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee4c565780 con 0x55ee4c625e70 2024-03-30T02:45:30.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.991+0000 7f0302000700 1 -- 172.21.15.67:0/247330164 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0314014040 con 0x55ee4c625e70 2024-03-30T02:45:30.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.991+0000 7f03371cfb80 10 client.27314 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:30.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.991+0000 7f03371cfb80 10 client.27314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:45:30.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.991+0000 7f03371cfb80 20 client.27314 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:30.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.991+0000 7f03371cfb80 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ee4c635ad0 0x55ee4c637ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:30.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.991+0000 7f03371cfb80 4 client.27314 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:45:30.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.991+0000 7f03371cfb80 1 -- 172.21.15.67:0/247330164 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee4c3bbd70 con 0x55ee4c635ad0 2024-03-30T02:45:30.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.991+0000 7f0320c00700 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ee4c635ad0 0x55ee4c637ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:30.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.992+0000 7f0320c00700 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ee4c635ad0 0x55ee4c637ef0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2375d1658b071f server_cookie=74f7118b3d67350c in_seq=0 out_seq=0 2024-03-30T02:45:30.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.992+0000 7f0302000700 10 client.27314 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.993+0000 7f0302000700 1 -- 172.21.15.67:0/247330164 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ee4c3bbd70 con 0x55ee4c635ad0 2024-03-30T02:45:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.993+0000 7f0302000700 10 client.27314 handle_command_reply: tid=0 2024-03-30T02:45:30.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.993+0000 7f03371cfb80 1 client.27314 shutdown 2024-03-30T02:45:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.994+0000 7f03371cfb80 1 -- 172.21.15.67:0/247330164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 msgr2=0x55ee4c6233a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.994+0000 7f03371cfb80 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c6233a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0314007bf0 tx=0x7f0314012240 comp rx=0 tx=0).stop 2024-03-30T02:45:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.994+0000 7f03371cfb80 1 -- 172.21.15.67:0/247330164 shutdown_connections 2024-03-30T02:45:30.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.994+0000 7f03371cfb80 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ee4c635ad0 0x55ee4c637ef0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.994+0000 7f03371cfb80 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 0x55ee4c6258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.994+0000 7f03371cfb80 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c6233a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.994+0000 7f03371cfb80 1 --2- 172.21.15.67:0/247330164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c622c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.994+0000 7f03371cfb80 1 -- 172.21.15.67:0/247330164 >> 172.21.15.67:0/247330164 conn(0x55ee4c57a720 msgr2=0x55ee4c61c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.994+0000 7f0302000700 0 client.27314 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.995+0000 7f03371cfb80 1 -- 172.21.15.67:0/247330164 shutdown_connections 2024-03-30T02:45:30.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.995+0000 7f03371cfb80 1 -- 172.21.15.67:0/247330164 wait complete. 2024-03-30T02:45:30.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:30.996+0000 7f03371cfb80 20 client.27314 trim_cache size 0 max 16384 2024-03-30T02:45:31.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.034+0000 7f03371cfb80 1 Processor -- start 2024-03-30T02:45:31.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f03371cfb80 1 -- start start 2024-03-30T02:45:31.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 0x55ee4c62cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c631210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c61df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f03371cfb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c623f10 con 0x55ee4c61a650 2024-03-30T02:45:31.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f03371cfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c61b5f0 con 0x55ee4c625e70 2024-03-30T02:45:31.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f03371cfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c61c170 con 0x55ee4c61af70 2024-03-30T02:45:31.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f0320c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 0x55ee4c62cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f031be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c631210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f031be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c631210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56994/0 (socket says 172.21.15.67:56994) 2024-03-30T02:45:31.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.035+0000 7f031be00700 1 -- 172.21.15.67:0/1603224899 learned_addr learned my addr 172.21.15.67:0/1603224899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:31.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.036+0000 7f0321600700 1 --2- 172.21.15.67:0/1603224899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c61df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.036+0000 7f031be00700 1 -- 172.21.15.67:0/1603224899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 msgr2=0x55ee4c61df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.036+0000 7f031be00700 1 --2- 172.21.15.67:0/1603224899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c61df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.036+0000 7f031be00700 1 -- 172.21.15.67:0/1603224899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 msgr2=0x55ee4c62cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.036+0000 7f031be00700 1 --2- 172.21.15.67:0/1603224899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 0x55ee4c62cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.036+0000 7f031be00700 1 -- 172.21.15.67:0/1603224899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee4c63cb20 con 0x55ee4c61af70 2024-03-30T02:45:31.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.036+0000 7f0321600700 1 --2- 172.21.15.67:0/1603224899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c61df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:45:31.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.037+0000 7f031be00700 1 --2- 172.21.15.67:0/1603224899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c631210 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f030800a3a0 tx=0x7f030800ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f358e5b4d7d93a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:31.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.038+0000 7f030e600700 1 -- 172.21.15.67:0/1603224899 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030800fa90 con 0x55ee4c61af70 2024-03-30T02:45:31.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.038+0000 7f030e600700 1 -- 172.21.15.67:0/1603224899 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f030800fbf0 con 0x55ee4c61af70 2024-03-30T02:45:31.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.039+0000 7f030e600700 1 -- 172.21.15.67:0/1603224899 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308004c50 con 0x55ee4c61af70 2024-03-30T02:45:31.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.040+0000 7f03371cfb80 1 -- 172.21.15.67:0/1603224899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 msgr2=0x55ee4c631210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.040+0000 7f03371cfb80 1 --2- 172.21.15.67:0/1603224899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c631210 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f030800a3a0 tx=0x7f030800ae50 comp rx=0 tx=0).stop 2024-03-30T02:45:31.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.041+0000 7f03371cfb80 1 -- 172.21.15.67:0/1603224899 shutdown_connections 2024-03-30T02:45:31.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.041+0000 7f03371cfb80 1 --2- 172.21.15.67:0/1603224899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c61a650 0x55ee4c62cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.041+0000 7f03371cfb80 1 --2- 172.21.15.67:0/1603224899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c625e70 0x55ee4c61df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.041+0000 7f03371cfb80 1 --2- 172.21.15.67:0/1603224899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c631210 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.041+0000 7f03371cfb80 1 -- 172.21.15.67:0/1603224899 >> 172.21.15.67:0/1603224899 conn(0x55ee4c57a720 msgr2=0x55ee4c637ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:31.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.042+0000 7f03371cfb80 1 -- 172.21.15.67:0/1603224899 shutdown_connections 2024-03-30T02:45:31.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.042+0000 7f03371cfb80 1 -- 172.21.15.67:0/1603224899 wait complete. 2024-03-30T02:45:31.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.043+0000 7f03371cfb80 1 -- start start 2024-03-30T02:45:31.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.043+0000 7f03371cfb80 1 Processor -- start 2024-03-30T02:45:31.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.044+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 0x55ee4c62b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.044+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c62b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.044+0000 7f03371cfb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c625e70 0x55ee4c62bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.044+0000 7f03371cfb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c6899a0 con 0x55ee4c625e70 2024-03-30T02:45:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.044+0000 7f03371cfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c689b10 con 0x55ee4c61a650 2024-03-30T02:45:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.044+0000 7f03371cfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee4c689cd0 con 0x55ee4c61af70 2024-03-30T02:45:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.045+0000 7f031be00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c62b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.045+0000 7f0320c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 0x55ee4c62b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.046+0000 7f0320c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 0x55ee4c62b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39576/0 (socket says 172.21.15.67:39576) 2024-03-30T02:45:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.046+0000 7f0320c00700 1 -- 172.21.15.67:0/2203135208 learned_addr learned my addr 172.21.15.67:0/2203135208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.046+0000 7f0320c00700 1 -- 172.21.15.67:0/2203135208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 msgr2=0x55ee4c62b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.046+0000 7f0320c00700 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c62b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.047+0000 7f0320c00700 1 -- 172.21.15.67:0/2203135208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c625e70 msgr2=0x55ee4c62bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.047+0000 7f0320c00700 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c625e70 0x55ee4c62bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.047+0000 7f0320c00700 1 -- 172.21.15.67:0/2203135208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0308027040 con 0x55ee4c61a650 2024-03-30T02:45:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.047+0000 7f03371cfb80 10 client.0 _ll_register_callbacks cb 0x55ee4c5124f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.047+0000 7f03371cfb80 1 -- 172.21.15.67:0/2203135208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee4c52d400 con 0x55ee4c61a650 2024-03-30T02:45:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.047+0000 7f0320c00700 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 0x55ee4c62b200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0310098dc0 tx=0x7f031000b310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19cc35caf4c2553 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:31.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.047+0000 7f0302000700 10 client.27320 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:45:31.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.048+0000 7f0302000700 1 -- 172.21.15.67:0/2203135208 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03100a38f0 con 0x55ee4c61a650 2024-03-30T02:45:31.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.048+0000 7f0302000700 1 -- 172.21.15.67:0/2203135208 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f031009b840 con 0x55ee4c61a650 2024-03-30T02:45:31.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.048+0000 7f0302000700 1 -- 172.21.15.67:0/2203135208 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f031009ba40 con 0x55ee4c61a650 2024-03-30T02:45:31.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.048+0000 7f03371cfb80 10 client.27320 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:31.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.048+0000 7f03371cfb80 1 -- 172.21.15.67:0/2203135208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=41}) v3 -- 0x55ee4c532f20 con 0x55ee4c61a650 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.048+0000 7f0302000700 1 -- 172.21.15.67:0/2203135208 <== mon.2 v2:172.21.15.82:3300/0 4 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f031009c5b0 con 0x55ee4c61a650 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.049+0000 7f03371cfb80 10 client.27320 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.049+0000 7f03371cfb80 10 client.27320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.049+0000 7f03371cfb80 20 client.27320 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.049+0000 7f03371cfb80 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ee4c68af10 0x55ee4c68d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.049+0000 7f03371cfb80 4 client.27320 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.049+0000 7f03371cfb80 1 -- 172.21.15.67:0/2203135208 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ee4c3f0d60 con 0x55ee4c68af10 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.049+0000 7f031be00700 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ee4c68af10 0x55ee4c68d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.050+0000 7f031be00700 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ee4c68af10 0x55ee4c68d3b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed90f53dd96963be server_cookie=ca1c6daff85c24ae in_seq=0 out_seq=0 2024-03-30T02:45:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.050+0000 7f0302000700 10 client.27320 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:31.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.051+0000 7f0302000700 1 -- 172.21.15.67:0/2203135208 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ee4c3f0d60 con 0x55ee4c68af10 2024-03-30T02:45:31.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.051+0000 7f0302000700 10 client.27320 handle_command_reply: tid=0 2024-03-30T02:45:31.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.052+0000 7f03371cfb80 1 client.27320 shutdown 2024-03-30T02:45:31.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.052+0000 7f03371cfb80 1 -- 172.21.15.67:0/2203135208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 msgr2=0x55ee4c62b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.052+0000 7f03371cfb80 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 0x55ee4c62b200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0310098dc0 tx=0x7f031000b310 comp rx=0 tx=0).stop 2024-03-30T02:45:31.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.053+0000 7f03371cfb80 1 -- 172.21.15.67:0/2203135208 shutdown_connections 2024-03-30T02:45:31.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.053+0000 7f03371cfb80 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55ee4c68af10 0x55ee4c68d3b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.053+0000 7f03371cfb80 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee4c625e70 0x55ee4c62bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.053+0000 7f03371cfb80 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee4c61a650 0x55ee4c62b200 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.053+0000 7f0302000700 0 client.27320 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:31.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.053+0000 7f03371cfb80 1 --2- 172.21.15.67:0/2203135208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee4c61af70 0x55ee4c62b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.053+0000 7f03371cfb80 1 -- 172.21.15.67:0/2203135208 >> 172.21.15.67:0/2203135208 conn(0x55ee4c57a720 msgr2=0x55ee4c631210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:31.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.053+0000 7f03371cfb80 1 -- 172.21.15.67:0/2203135208 shutdown_connections 2024-03-30T02:45:31.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.053+0000 7f03371cfb80 1 -- 172.21.15.67:0/2203135208 wait complete. 2024-03-30T02:45:31.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.055+0000 7f03371cfb80 20 client.27320 trim_cache size 0 max 16384 2024-03-30T02:45:31.054 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:45:31.055 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:45:31.055 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "e0ee3515-7862-4a47-98fc-539b676c76f0", 2024-03-30T02:45:31.055 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:45:31.055 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:45:31.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 -- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a1460 msgr2=0x7f03040a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a1460 0x7f03040a3920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03080124b0 tx=0x7f030800aa00 comp rx=0 tx=0).stop 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 -- 172.21.15.67:0/1531647649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 msgr2=0x7f031c199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 0x7f031c199230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f031000b2e0 tx=0x7f03100057f0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 -- 172.21.15.67:0/1531647649 shutdown_connections 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f03040a1460 0x7f03040a3920 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f031c19db10 0x7f031c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f031c0ffe50 0x7f031c199230 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 --2- 172.21.15.67:0/1531647649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031c104f60 0x7f031c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 -- 172.21.15.67:0/1531647649 >> 172.21.15.67:0/1531647649 conn(0x7f031c076d40 msgr2=0x7f031c10c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.056+0000 7f0323600700 1 -- 172.21.15.67:0/1531647649 shutdown_connections 2024-03-30T02:45:31.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.057+0000 7f0323600700 1 -- 172.21.15.67:0/1531647649 wait complete. 2024-03-30T02:45:31.085 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi067 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=470100a4ccbcf7769052bf09a5413881ccf1b97f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-03-30T02:45:31.286 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=27bc5f1402394ff68594dc9c389b3116 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_470100a4ccbcf7769052bf09a5413881ccf1b97f/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test kernel_untar_build.sh) on smithi067 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=470100a4ccbcf7769052bf09a5413881ccf1b97f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2024-03-30T02:45:31.289 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-30T02:45:31.300 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-30T02:45:31.300 INFO:teuthology.task.exec:Executing custom commands... 2024-03-30T02:45:31.300 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:45:31.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-30T02:45:31.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:45:31.427 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-30T02:45:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.591+0000 7f96cce00700 1 -- 172.21.15.67:0/2873220305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c8105ce0 msgr2=0x7f96c81060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.591+0000 7f96cce00700 1 --2- 172.21.15.67:0/2873220305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c8105ce0 0x7f96c81060c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f96b80099c0 tx=0x7f96b8009cd0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.591+0000 7f96cce00700 1 -- 172.21.15.67:0/2873220305 shutdown_connections 2024-03-30T02:45:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.591+0000 7f96cce00700 1 --2- 172.21.15.67:0/2873220305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c80687f0 0x7f96c810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.591+0000 7f96cce00700 1 --2- 172.21.15.67:0/2873220305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c8105ce0 0x7f96c81060c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.591+0000 7f96cce00700 1 --2- 172.21.15.67:0/2873220305 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c8067e30 0x7f96c80682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.591+0000 7f96cce00700 1 -- 172.21.15.67:0/2873220305 >> 172.21.15.67:0/2873220305 conn(0x7f96c8076d40 msgr2=0x7f96c8077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:31.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.591+0000 7f96cce00700 1 -- 172.21.15.67:0/2873220305 shutdown_connections 2024-03-30T02:45:31.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.591+0000 7f96cce00700 1 -- 172.21.15.67:0/2873220305 wait complete. 2024-03-30T02:45:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.593+0000 7f96cce00700 1 Processor -- start 2024-03-30T02:45:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.593+0000 7f96cce00700 1 -- start start 2024-03-30T02:45:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.593+0000 7f96cce00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c8067e30 0x7f96c81a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.593+0000 7f96cce00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80687f0 0x7f96c81a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96cce00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8105ce0 0x7f96c819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96cce00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c819dcd0 con 0x7f96c80687f0 2024-03-30T02:45:31.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96cce00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c819de40 con 0x7f96c8067e30 2024-03-30T02:45:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96cce00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c819e000 con 0x7f96c8105ce0 2024-03-30T02:45:31.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96c5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c8067e30 0x7f96c81a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96c5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c8067e30 0x7f96c81a3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39594/0 (socket says 172.21.15.67:39594) 2024-03-30T02:45:31.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96c5e00700 1 -- 172.21.15.67:0/1590820276 learned_addr learned my addr 172.21.15.67:0/1590820276 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:31.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96c6800700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8105ce0 0x7f96c819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96c5400700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80687f0 0x7f96c81a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96c6800700 1 -- 172.21.15.67:0/1590820276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c8067e30 msgr2=0x7f96c81a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.594+0000 7f96c6800700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c8067e30 0x7f96c81a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.595+0000 7f96c6800700 1 -- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80687f0 msgr2=0x7f96c81a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.595+0000 7f96c6800700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80687f0 0x7f96c81a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.595+0000 7f96c6800700 1 -- 172.21.15.67:0/1590820276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b80095f0 con 0x7f96c8105ce0 2024-03-30T02:45:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.595+0000 7f96c6800700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8105ce0 0x7f96c819d670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f96c000efc0 tx=0x7f96c000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.595+0000 7f96bea00700 1 -- 172.21.15.67:0/1590820276 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0018070 con 0x7f96c8105ce0 2024-03-30T02:45:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.595+0000 7f96bea00700 1 -- 172.21.15.67:0/1590820276 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96c000f040 con 0x7f96c8105ce0 2024-03-30T02:45:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.595+0000 7f96cce00700 1 -- 172.21.15.67:0/1590820276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c819e280 con 0x7f96c8105ce0 2024-03-30T02:45:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.595+0000 7f96bea00700 1 -- 172.21.15.67:0/1590820276 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c000cd40 con 0x7f96c8105ce0 2024-03-30T02:45:31.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.596+0000 7f96cce00700 1 -- 172.21.15.67:0/1590820276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c81aa530 con 0x7f96c8105ce0 2024-03-30T02:45:31.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.600+0000 7f96bea00700 1 -- 172.21.15.67:0/1590820276 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f96c0003a00 con 0x7f96c8105ce0 2024-03-30T02:45:31.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.601+0000 7f96e089ab80 1 Processor -- start 2024-03-30T02:45:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.602+0000 7f96e089ab80 1 -- start start 2024-03-30T02:45:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.602+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x5619894309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.602+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x56198943b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.602+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x56198943e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.602+0000 7f96e089ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561989389990 con 0x56198943bdf0 2024-03-30T02:45:31.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.602+0000 7f96e089ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561989387350 con 0x5619894305d0 2024-03-30T02:45:31.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.602+0000 7f96e089ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561989382c40 con 0x561989430ef0 2024-03-30T02:45:31.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x5619894309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x5619894309b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39596/0 (socket says 172.21.15.67:39596) 2024-03-30T02:45:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5400700 1 -- 172.21.15.67:0/2995617465 learned_addr learned my addr 172.21.15.67:0/2995617465 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5400700 1 --2- 172.21.15.67:0/2995617465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x56198943e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5e00700 1 --2- 172.21.15.67:0/2995617465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x56198943b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96bea00700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f96ac0a12f0 0x7f96ac0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5400700 1 -- 172.21.15.67:0/2995617465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 msgr2=0x56198943b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5400700 1 --2- 172.21.15.67:0/2995617465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x56198943b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5400700 1 -- 172.21.15.67:0/2995617465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 msgr2=0x56198943e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5400700 1 --2- 172.21.15.67:0/2995617465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x56198943e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.603+0000 7f96c5400700 1 -- 172.21.15.67:0/2995617465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619892043b0 con 0x5619894305d0 2024-03-30T02:45:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.604+0000 7f96c6800700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f96ac0a12f0 0x7f96ac0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.604+0000 7f96bea00700 1 -- 172.21.15.67:0/1590820276 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f96c00ccc00 con 0x7f96c8105ce0 2024-03-30T02:45:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.604+0000 7f96c5400700 1 --2- 172.21.15.67:0/2995617465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x5619894309b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f96b4012840 tx=0x7f96b400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1757be7ca5d760bb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.604+0000 7f96b3400700 1 -- 172.21.15.67:0/2995617465 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4012e60 con 0x5619894305d0 2024-03-30T02:45:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.604+0000 7f96c6800700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f96ac0a12f0 0x7f96ac0a37b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f96c000ef90 tx=0x7f96c00092e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.604+0000 7f96b3400700 1 -- 172.21.15.67:0/2995617465 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b40044f0 con 0x5619894305d0 2024-03-30T02:45:31.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.604+0000 7f96b3400700 1 -- 172.21.15.67:0/2995617465 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4005310 con 0x5619894305d0 2024-03-30T02:45:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.605+0000 7f96e089ab80 1 -- 172.21.15.67:0/2995617465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 msgr2=0x5619894309b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.605+0000 7f96e089ab80 1 --2- 172.21.15.67:0/2995617465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x5619894309b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f96b4012840 tx=0x7f96b400b860 comp rx=0 tx=0).stop 2024-03-30T02:45:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.605+0000 7f96e089ab80 1 -- 172.21.15.67:0/2995617465 shutdown_connections 2024-03-30T02:45:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.605+0000 7f96e089ab80 1 --2- 172.21.15.67:0/2995617465 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x56198943e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.605+0000 7f96e089ab80 1 --2- 172.21.15.67:0/2995617465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x5619894309b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.605+0000 7f96e089ab80 1 --2- 172.21.15.67:0/2995617465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x56198943b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.605+0000 7f96e089ab80 1 -- 172.21.15.67:0/2995617465 >> 172.21.15.67:0/2995617465 conn(0x5619893906a0 msgr2=0x56198942ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.605+0000 7f96e089ab80 1 -- 172.21.15.67:0/2995617465 shutdown_connections 2024-03-30T02:45:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.605+0000 7f96e089ab80 1 -- 172.21.15.67:0/2995617465 wait complete. 2024-03-30T02:45:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.606+0000 7f96e089ab80 1 -- start start 2024-03-30T02:45:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.606+0000 7f96e089ab80 1 Processor -- start 2024-03-30T02:45:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x56198943b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989438b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x561989439230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96e089ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198931ed80 con 0x56198943bdf0 2024-03-30T02:45:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96e089ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198931ea20 con 0x5619894305d0 2024-03-30T02:45:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96e089ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619892066a0 con 0x561989430ef0 2024-03-30T02:45:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96e089ab80 10 client.0 _ll_register_callbacks cb 0x561989328470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96c5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x56198943b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96c5400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x56198943b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39598/0 (socket says 172.21.15.67:39598) 2024-03-30T02:45:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96c5400700 1 -- 172.21.15.67:0/1031466121 learned_addr learned my addr 172.21.15.67:0/1031466121 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.607+0000 7f96c5400700 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x561989439230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96c5e00700 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989438b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96c5400700 1 -- 172.21.15.67:0/1031466121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 msgr2=0x561989438b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96c5400700 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989438b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96c5400700 1 -- 172.21.15.67:0/1031466121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 msgr2=0x561989439230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96c5400700 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x561989439230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96c5400700 1 -- 172.21.15.67:0/1031466121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b401a040 con 0x5619894305d0 2024-03-30T02:45:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96c5e00700 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989438b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:45:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96c5400700 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x56198943b870 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96b40046e0 tx=0x7f96b4017040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ebb282aa7cb208 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:31.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96a7e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:45:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.608+0000 7f96a7e00700 1 -- 172.21.15.67:0/1031466121 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4005000 con 0x5619894305d0 2024-03-30T02:45:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.609+0000 7f96a7e00700 1 -- 172.21.15.67:0/1031466121 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f96b4005160 con 0x5619894305d0 2024-03-30T02:45:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.609+0000 7f96e089ab80 1 -- 172.21.15.67:0/1031466121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561989343380 con 0x5619894305d0 2024-03-30T02:45:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.609+0000 7f96a7e00700 1 -- 172.21.15.67:0/1031466121 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96b4005320 con 0x5619894305d0 2024-03-30T02:45:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.609+0000 7f96e089ab80 10 client.27332 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:31.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.609+0000 7f96e089ab80 1 -- 172.21.15.67:0/1031466121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5619891b0b60 con 0x5619894305d0 2024-03-30T02:45:31.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.609+0000 7f96a7e00700 1 -- 172.21.15.67:0/1031466121 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561989343380 con 0x5619894305d0 2024-03-30T02:45:31.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.610+0000 7f96a7e00700 1 -- 172.21.15.67:0/1031466121 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f96b402b070 con 0x5619894305d0 2024-03-30T02:45:31.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.610+0000 7f96e089ab80 10 client.27332 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.610+0000 7f96e089ab80 10 client.27332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:45:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.610+0000 7f96e089ab80 20 client.27332 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.610+0000 7f96e089ab80 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56198944b890 0x56198944dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.610+0000 7f96e089ab80 4 client.27332 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:45:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.610+0000 7f96e089ab80 1 -- 172.21.15.67:0/1031466121 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56198937b700 con 0x56198944b890 2024-03-30T02:45:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.610+0000 7f96c5e00700 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56198944b890 0x56198944dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.611+0000 7f96c5e00700 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56198944b890 0x56198944dcb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f3478aa7eead481 server_cookie=50e8f2451750b089 in_seq=0 out_seq=0 2024-03-30T02:45:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.611+0000 7f96a7e00700 10 client.27332 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.613+0000 7f96a7e00700 1 -- 172.21.15.67:0/1031466121 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56198937b700 con 0x56198944b890 2024-03-30T02:45:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.613+0000 7f96a7e00700 10 client.27332 handle_command_reply: tid=0 2024-03-30T02:45:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.613+0000 7f96e089ab80 1 client.27332 shutdown 2024-03-30T02:45:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 -- 172.21.15.67:0/1031466121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 msgr2=0x56198943b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x56198943b870 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96b40046e0 tx=0x7f96b4017040 comp rx=0 tx=0).stop 2024-03-30T02:45:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 -- 172.21.15.67:0/1031466121 shutdown_connections 2024-03-30T02:45:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x56198944b890 0x56198944dcb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x561989439230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619894305d0 0x56198943b870 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96a7e00700 0 client.27332 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:31.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 --2- 172.21.15.67:0/1031466121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989438b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 -- 172.21.15.67:0/1031466121 >> 172.21.15.67:0/1031466121 conn(0x5619893906a0 msgr2=0x5619894326c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:31.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 -- 172.21.15.67:0/1031466121 shutdown_connections 2024-03-30T02:45:31.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 1 -- 172.21.15.67:0/1031466121 wait complete. 2024-03-30T02:45:31.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.614+0000 7f96e089ab80 20 client.27332 trim_cache size 0 max 16384 2024-03-30T02:45:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.637+0000 7f96e089ab80 1 Processor -- start 2024-03-30T02:45:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.637+0000 7f96e089ab80 1 -- start start 2024-03-30T02:45:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619894305d0 0x56198942bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561989430ef0 0x561989442a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x5619894255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96e089ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561989430430 con 0x56198943bdf0 2024-03-30T02:45:31.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96e089ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561989431850 con 0x561989430ef0 2024-03-30T02:45:31.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96e089ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198943a130 con 0x5619894305d0 2024-03-30T02:45:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96c5400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619894305d0 0x56198942bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96c5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561989430ef0 0x561989442a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96c5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561989430ef0 0x561989442a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:39608/0 (socket says 172.21.15.67:39608) 2024-03-30T02:45:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96c5e00700 1 -- 172.21.15.67:0/863779934 learned_addr learned my addr 172.21.15.67:0/863779934 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.638+0000 7f96c5400700 1 --2- 172.21.15.67:0/863779934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x5619894255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96c5e00700 1 -- 172.21.15.67:0/863779934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619894305d0 msgr2=0x56198942bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96c5e00700 1 --2- 172.21.15.67:0/863779934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619894305d0 0x56198942bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96c5e00700 1 -- 172.21.15.67:0/863779934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 msgr2=0x5619894255a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96c5e00700 1 --2- 172.21.15.67:0/863779934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x5619894255a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96c5e00700 1 -- 172.21.15.67:0/863779934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561989348ea0 con 0x561989430ef0 2024-03-30T02:45:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96c5400700 1 --2- 172.21.15.67:0/863779934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x5619894255a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:45:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96c5e00700 1 --2- 172.21.15.67:0/863779934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561989430ef0 0x561989442a70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f96b80058e0 tx=0x7f96b8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cc2fa48967119da server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96b3e00700 1 -- 172.21.15.67:0/863779934 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b801d040 con 0x561989430ef0 2024-03-30T02:45:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96b3e00700 1 -- 172.21.15.67:0/863779934 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b8005680 con 0x561989430ef0 2024-03-30T02:45:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.639+0000 7f96b3e00700 1 -- 172.21.15.67:0/863779934 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b80255d0 con 0x561989430ef0 2024-03-30T02:45:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.640+0000 7f96e089ab80 1 -- 172.21.15.67:0/863779934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561989430ef0 msgr2=0x561989442a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.640+0000 7f96e089ab80 1 --2- 172.21.15.67:0/863779934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561989430ef0 0x561989442a70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f96b80058e0 tx=0x7f96b8009310 comp rx=0 tx=0).stop 2024-03-30T02:45:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.640+0000 7f96e089ab80 1 -- 172.21.15.67:0/863779934 shutdown_connections 2024-03-30T02:45:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.640+0000 7f96e089ab80 1 --2- 172.21.15.67:0/863779934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56198943bdf0 0x5619894255a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.640+0000 7f96e089ab80 1 --2- 172.21.15.67:0/863779934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619894305d0 0x56198942bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.640+0000 7f96e089ab80 1 --2- 172.21.15.67:0/863779934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561989430ef0 0x561989442a70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.640+0000 7f96e089ab80 1 -- 172.21.15.67:0/863779934 >> 172.21.15.67:0/863779934 conn(0x5619893906a0 msgr2=0x561989452610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.640+0000 7f96e089ab80 1 -- 172.21.15.67:0/863779934 shutdown_connections 2024-03-30T02:45:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.640+0000 7f96e089ab80 1 -- 172.21.15.67:0/863779934 wait complete. 2024-03-30T02:45:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.641+0000 7f96e089ab80 1 -- start start 2024-03-30T02:45:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.641+0000 7f96e089ab80 1 Processor -- start 2024-03-30T02:45:31.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.641+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619894305d0 0x561989442ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.641+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989441030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96e089ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56198943bdf0 0x561989441570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96e089ab80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561989441c50 con 0x5619894305d0 2024-03-30T02:45:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96e089ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561989441dc0 con 0x56198943bdf0 2024-03-30T02:45:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96e089ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198949f6d0 con 0x561989430ef0 2024-03-30T02:45:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96e089ab80 10 client.0 _ll_register_callbacks cb 0x561989328470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:45:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619894305d0 0x561989442ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619894305d0 0x561989442ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44648/0 (socket says 172.21.15.67:44648) 2024-03-30T02:45:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5400700 1 -- 172.21.15.67:0/944624805 learned_addr learned my addr 172.21.15.67:0/944624805 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:45:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5e00700 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989441030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5400700 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56198943bdf0 0x561989441570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5400700 1 -- 172.21.15.67:0/944624805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 msgr2=0x561989441030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5400700 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989441030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5400700 1 -- 172.21.15.67:0/944624805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56198943bdf0 msgr2=0x561989441570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:45:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5400700 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56198943bdf0 0x561989441570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.642+0000 7f96c5400700 1 -- 172.21.15.67:0/944624805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c810e550 con 0x5619894305d0 2024-03-30T02:45:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.643+0000 7f96c5e00700 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989441030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:45:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.643+0000 7f96c5400700 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619894305d0 0x561989442ac0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f96b4012840 tx=0x7f96b4004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d217e93ef97408 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:45:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.643+0000 7f96a7e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:45:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.643+0000 7f96a7e00700 1 -- 172.21.15.67:0/944624805 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b400bc90 con 0x5619894305d0 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.643+0000 7f96a7e00700 1 -- 172.21.15.67:0/944624805 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f96b401f460 con 0x5619894305d0 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.643+0000 7f96e089ab80 1 -- 172.21.15.67:0/944624805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96b40044f0 con 0x5619894305d0 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.643+0000 7f96a7e00700 1 -- 172.21.15.67:0/944624805 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96b401f620 con 0x5619894305d0 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.643+0000 7f96e089ab80 10 client.17199 fetch_fsmap learned FSMap version 41 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.643+0000 7f96e089ab80 1 -- 172.21.15.67:0/944624805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561989208cd0 con 0x5619894305d0 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.644+0000 7f96a7e00700 1 -- 172.21.15.67:0/944624805 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b40044f0 con 0x5619894305d0 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.644+0000 7f96a7e00700 1 -- 172.21.15.67:0/944624805 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f96b402b070 con 0x5619894305d0 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.644+0000 7f96e089ab80 10 client.17199 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.644+0000 7f96e089ab80 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:45:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.644+0000 7f96e089ab80 20 client.17199 populate_metadata read hostname 'smithi067' 2024-03-30T02:45:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.645+0000 7f96e089ab80 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619894a0ac0 0x5619894a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:45:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.645+0000 7f96e089ab80 4 client.17199 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:45:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.645+0000 7f96e089ab80 1 -- 172.21.15.67:0/944624805 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5619891d1cf0 con 0x5619894a0ac0 2024-03-30T02:45:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.645+0000 7f96c5e00700 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619894a0ac0 0x5619894a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:45:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.645+0000 7f96c5e00700 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619894a0ac0 0x5619894a2ee0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d52cf41f9fee1ac2 server_cookie=64213ca473a56401 in_seq=0 out_seq=0 2024-03-30T02:45:31.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.645+0000 7f96a7e00700 10 client.17199 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.647+0000 7f96a7e00700 1 -- 172.21.15.67:0/944624805 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+269 (crc 0 0 0) 0x5619891d1cf0 con 0x5619894a0ac0 2024-03-30T02:45:31.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.647+0000 7f96a7e00700 10 client.17199 handle_command_reply: tid=0 2024-03-30T02:45:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.647+0000 7f96e089ab80 1 client.17199 shutdown 2024-03-30T02:45:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.647+0000 7f96e089ab80 1 -- 172.21.15.67:0/944624805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619894305d0 msgr2=0x561989442ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.647+0000 7f96e089ab80 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619894305d0 0x561989442ac0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f96b4012840 tx=0x7f96b4004e70 comp rx=0 tx=0).stop 2024-03-30T02:45:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.647+0000 7f96e089ab80 1 -- 172.21.15.67:0/944624805 shutdown_connections 2024-03-30T02:45:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.647+0000 7f96e089ab80 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5619894a0ac0 0x5619894a2ee0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.648+0000 7f96e089ab80 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5619894305d0 0x561989442ac0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.648+0000 7f96e089ab80 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56198943bdf0 0x561989441570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.648+0000 7f96e089ab80 1 --2- 172.21.15.67:0/944624805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561989430ef0 0x561989441030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.648+0000 7f96e089ab80 1 -- 172.21.15.67:0/944624805 >> 172.21.15.67:0/944624805 conn(0x5619893906a0 msgr2=0x5619894255a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.648+0000 7f96a7e00700 0 client.17199 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:45:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.648+0000 7f96e089ab80 1 -- 172.21.15.67:0/944624805 shutdown_connections 2024-03-30T02:45:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.648+0000 7f96e089ab80 1 -- 172.21.15.67:0/944624805 wait complete. 2024-03-30T02:45:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.648+0000 7f96e089ab80 20 client.17199 trim_cache size 0 max 16384 2024-03-30T02:45:31.652 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:45:31.653 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (15213 inodes in the stack)", 2024-03-30T02:45:31.653 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:45:31.653 INFO:teuthology.orchestra.run.smithi067.stdout: "e0ee3515-7862-4a47-98fc-539b676c76f0": { 2024-03-30T02:45:31.653 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:45:31.653 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "e0ee3515-7862-4a47-98fc-539b676c76f0", 2024-03-30T02:45:31.653 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:45:31.653 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:45:31.653 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:45:31.653 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:45:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 -- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f96ac0a12f0 msgr2=0x7f96ac0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f96ac0a12f0 0x7f96ac0a37b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f96c000ef90 tx=0x7f96c00092e0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 -- 172.21.15.67:0/1590820276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8105ce0 msgr2=0x7f96c819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:45:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8105ce0 0x7f96c819d670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f96c000efc0 tx=0x7f96c000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 -- 172.21.15.67:0/1590820276 shutdown_connections 2024-03-30T02:45:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f96ac0a12f0 0x7f96ac0a37b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8105ce0 0x7f96c819d670 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96c8067e30 0x7f96c81a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 --2- 172.21.15.67:0/1590820276 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96c80687f0 0x7f96c81a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:45:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 -- 172.21.15.67:0/1590820276 >> 172.21.15.67:0/1590820276 conn(0x7f96c8076d40 msgr2=0x7f96c80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:45:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 -- 172.21.15.67:0/1590820276 shutdown_connections 2024-03-30T02:45:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:45:31.649+0000 7f96cce00700 1 -- 172.21.15.67:0/1590820276 wait complete. 2024-03-30T02:45:31.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:31.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:31.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:31 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.672 INFO:tasks.cephfs.filesystem:scrub status for tag:e0ee3515-7862-4a47-98fc-539b676c76f0 - {'path': '/', 'tag': 'e0ee3515-7862-4a47-98fc-539b676c76f0', 'options': 'recursive,force'} 2024-03-30T02:45:31.767 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:31.767 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:31.767 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:31.768 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.768 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.768 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.768 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:31.768 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.768 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:31.768 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.768 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:31.768 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:31 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:31.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:31.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:31.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:31.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:31.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:31.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:31.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:31 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.779 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: pgmap v261: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.2 MiB/s wr, 50 op/s 2024-03-30T02:45:32.779 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:45:32.779 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.780 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.780 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.780 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.780 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.780 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.780 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.780 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.780 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.780 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.781 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.781 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.781 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.781 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.781 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.781 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.781 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.781 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.781 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.782 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.782 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.782 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.782 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.782 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.782 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.782 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.782 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.782 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.783 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.783 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.783 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.783 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.783 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.783 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.783 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.783 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.784 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.784 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.784 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.784 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.784 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.784 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.784 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.784 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.784 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.785 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.786 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.786 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.786 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.786 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.786 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.786 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.786 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.786 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.787 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.788 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.789 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.791 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.792 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.792 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.792 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.792 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.792 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.792 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:32 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: pgmap v261: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.2 MiB/s wr, 50 op/s 2024-03-30T02:45:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:45:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:32 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: pgmap v261: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.2 MiB/s wr, 50 op/s 2024-03-30T02:45:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:45:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:32.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:32.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:32 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:33.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:33 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:33.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:33 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:33.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:33.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:33 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: pgmap v262: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-03-30T02:45:34.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:34.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:34 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: pgmap v262: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-03-30T02:45:34.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:34.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:34 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: pgmap v262: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-03-30T02:45:34.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:34.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:34.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:34 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: pgmap v263: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-03-30T02:45:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:35.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:35 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: pgmap v263: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-03-30T02:45:35.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:35.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:35 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: pgmap v263: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-03-30T02:45:35.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:35.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:35 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:36.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:36 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:36.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:36 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:36.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:36.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:36 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: pgmap v264: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 830 op/s 2024-03-30T02:45:37.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:37 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:37.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: pgmap v264: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 830 op/s 2024-03-30T02:45:37.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:37 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: pgmap v264: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 830 op/s 2024-03-30T02:45:37.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:37.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:37.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:37 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:38.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:38 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:38.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:38 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:38.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:38.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:38 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: pgmap v265: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.12k op/s 2024-03-30T02:45:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:39.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: pgmap v265: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.12k op/s 2024-03-30T02:45:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:39.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:39.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:39.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: pgmap v265: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 1.12k op/s 2024-03-30T02:45:40.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:40.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:40.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: pgmap v266: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 2.99k op/s 2024-03-30T02:45:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:45:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: pgmap v266: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 2.99k op/s 2024-03-30T02:45:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:45:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:41.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:41.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: pgmap v266: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 2.99k op/s 2024-03-30T02:45:42.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:45:42.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:42.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:45:43.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:43 smithi067 ceph-mon[37808]: pgmap v267: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 612 KiB/s wr, 3.79k op/s 2024-03-30T02:45:43.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:43 smithi170 ceph-mon[78798]: pgmap v267: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 612 KiB/s wr, 3.79k op/s 2024-03-30T02:45:43.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:43 smithi082 ceph-mon[79176]: pgmap v267: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 612 KiB/s wr, 3.79k op/s 2024-03-30T02:45:45.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:45 smithi067 ceph-mon[37808]: pgmap v268: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 189 KiB/s wr, 3.94k op/s 2024-03-30T02:45:45.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:45 smithi170 ceph-mon[78798]: pgmap v268: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 189 KiB/s wr, 3.94k op/s 2024-03-30T02:45:45.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:45 smithi082 ceph-mon[79176]: pgmap v268: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 189 KiB/s wr, 3.94k op/s 2024-03-30T02:45:48.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:48 smithi170 ceph-mon[78798]: pgmap v269: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 25 KiB/s wr, 3.96k op/s 2024-03-30T02:45:48.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:48 smithi082 ceph-mon[79176]: pgmap v269: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 25 KiB/s wr, 3.96k op/s 2024-03-30T02:45:48.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:48 smithi067 ceph-mon[37808]: pgmap v269: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 25 KiB/s wr, 3.96k op/s 2024-03-30T02:45:50.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:50 smithi170 ceph-mon[78798]: pgmap v270: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.9 KiB/s wr, 3.34k op/s 2024-03-30T02:45:50.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:50 smithi082 ceph-mon[79176]: pgmap v270: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.9 KiB/s wr, 3.34k op/s 2024-03-30T02:45:50.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:50 smithi067 ceph-mon[37808]: pgmap v270: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.9 KiB/s wr, 3.34k op/s 2024-03-30T02:45:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:52 smithi170 ceph-mon[78798]: pgmap v271: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 48 KiB/s wr, 3.05k op/s 2024-03-30T02:45:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:52 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:52 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:52 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:52 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:52.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:52 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:45:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:52 smithi082 ceph-mon[79176]: pgmap v271: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 48 KiB/s wr, 3.05k op/s 2024-03-30T02:45:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:52 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:52 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:52 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:52.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:52 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:52.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:52 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:45:52.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:52 smithi067 ceph-mon[37808]: pgmap v271: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 48 KiB/s wr, 3.05k op/s 2024-03-30T02:45:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:52 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:52 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:52 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:52 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:52.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:52 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:45:54.691 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:54 smithi170 ceph-mon[78798]: pgmap v272: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 70 KiB/s wr, 1.18k op/s 2024-03-30T02:45:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:54 smithi082 ceph-mon[79176]: pgmap v272: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 70 KiB/s wr, 1.18k op/s 2024-03-30T02:45:54.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:54 smithi067 ceph-mon[37808]: pgmap v272: 129 pgs: 129 active+clean; 1.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 70 KiB/s wr, 1.18k op/s 2024-03-30T02:45:56.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:56 smithi170 ceph-mon[78798]: pgmap v273: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 69 KiB/s wr, 357 op/s 2024-03-30T02:45:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:56 smithi082 ceph-mon[79176]: pgmap v273: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 69 KiB/s wr, 357 op/s 2024-03-30T02:45:56.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:56 smithi067 ceph-mon[37808]: pgmap v273: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 69 KiB/s wr, 357 op/s 2024-03-30T02:45:58.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:45:58 smithi170 ceph-mon[78798]: pgmap v274: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 86 KiB/s wr, 184 op/s 2024-03-30T02:45:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:45:58 smithi082 ceph-mon[79176]: pgmap v274: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 86 KiB/s wr, 184 op/s 2024-03-30T02:45:58.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:45:58 smithi067 ceph-mon[37808]: pgmap v274: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 86 KiB/s wr, 184 op/s 2024-03-30T02:46:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:00 smithi082 ceph-mon[79176]: pgmap v275: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 83 KiB/s wr, 14 op/s 2024-03-30T02:46:00.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:00 smithi067 ceph-mon[37808]: pgmap v275: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 83 KiB/s wr, 14 op/s 2024-03-30T02:46:00.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:00 smithi170 ceph-mon[78798]: pgmap v275: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 83 KiB/s wr, 14 op/s 2024-03-30T02:46:01.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:46:01.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.887+0000 7f2b00c00700 1 -- 172.21.15.67:0/2917293418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 msgr2=0x7f2afc10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.887+0000 7f2b00c00700 1 --2- 172.21.15.67:0/2917293418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 0x7f2afc10dc00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2af400b0a0 tx=0x7f2af400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.887+0000 7f2b00c00700 1 -- 172.21.15.67:0/2917293418 shutdown_connections 2024-03-30T02:46:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.887+0000 7f2b00c00700 1 --2- 172.21.15.67:0/2917293418 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 0x7f2afc10dc00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.887+0000 7f2b00c00700 1 --2- 172.21.15.67:0/2917293418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2afc100f60 0x7f2afc101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.887+0000 7f2b00c00700 1 --2- 172.21.15.67:0/2917293418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2afc101910 0x7f2afc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.887+0000 7f2b00c00700 1 -- 172.21.15.67:0/2917293418 >> 172.21.15.67:0/2917293418 conn(0x7f2afc0fc7c0 msgr2=0x7f2afc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.887+0000 7f2b00c00700 1 -- 172.21.15.67:0/2917293418 shutdown_connections 2024-03-30T02:46:01.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.887+0000 7f2b00c00700 1 -- 172.21.15.67:0/2917293418 wait complete. 2024-03-30T02:46:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.889+0000 7f2b00c00700 1 Processor -- start 2024-03-30T02:46:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.889+0000 7f2b00c00700 1 -- start start 2024-03-30T02:46:01.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.889+0000 7f2b00c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2afc100f60 0x7f2afc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.889+0000 7f2b00c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 0x7f2afc199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.889+0000 7f2b00c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2afc19de30 0x7f2afc19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.889+0000 7f2b00c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc19e8b0 con 0x7f2afc106080 2024-03-30T02:46:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.889+0000 7f2b00c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc19ea20 con 0x7f2afc100f60 2024-03-30T02:46:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.889+0000 7f2b00c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc19ebe0 con 0x7f2afc19de30 2024-03-30T02:46:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 0x7f2afc199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2afc100f60 0x7f2afc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 0x7f2afc199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37422/0 (socket says 172.21.15.67:37422) 2024-03-30T02:46:01.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af3e00700 1 -- 172.21.15.67:0/3819350747 learned_addr learned my addr 172.21.15.67:0/3819350747 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:01.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2afa800700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2afc19de30 0x7f2afc19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af3e00700 1 -- 172.21.15.67:0/3819350747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2afc19de30 msgr2=0x7f2afc19e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:01.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af3e00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2afc19de30 0x7f2afc19e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af3e00700 1 -- 172.21.15.67:0/3819350747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2afc100f60 msgr2=0x7f2afc199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:01.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af3e00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2afc100f60 0x7f2afc199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af3e00700 1 -- 172.21.15.67:0/3819350747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af4009d00 con 0x7f2afc106080 2024-03-30T02:46:01.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af9e00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2afc100f60 0x7f2afc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:01.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.890+0000 7f2af3e00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 0x7f2afc199b80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400d950 tx=0x7f2ae400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.891+0000 7f2af2a00700 1 -- 172.21.15.67:0/3819350747 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4004220 con 0x7f2afc106080 2024-03-30T02:46:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.891+0000 7f2af2a00700 1 -- 172.21.15.67:0/3819350747 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae4007620 con 0x7f2afc106080 2024-03-30T02:46:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.891+0000 7f2b00c00700 1 -- 172.21.15.67:0/3819350747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc1a6140 con 0x7f2afc106080 2024-03-30T02:46:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.891+0000 7f2af2a00700 1 -- 172.21.15.67:0/3819350747 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4005670 con 0x7f2afc106080 2024-03-30T02:46:01.894 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.891+0000 7f2b00c00700 1 -- 172.21.15.67:0/3819350747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc1a6620 con 0x7f2afc106080 2024-03-30T02:46:01.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.894+0000 7f2af2a00700 1 -- 172.21.15.67:0/3819350747 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2ae4003680 con 0x7f2afc106080 2024-03-30T02:46:01.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.895+0000 7f2b146f6b80 1 Processor -- start 2024-03-30T02:46:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.895+0000 7f2b146f6b80 1 -- start start 2024-03-30T02:46:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.895+0000 7f2af2a00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ae00a1290 0x7f2ae00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd4f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af2a00700 1 -- 172.21.15.67:0/3819350747 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2ae40ccd50 con 0x7f2afc106080 2024-03-30T02:46:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 0x561eedd5d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedca8990 con 0x561eedd4f5d0 2024-03-30T02:46:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedca6350 con 0x561eedd5adf0 2024-03-30T02:46:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedca1c40 con 0x561eedd4fef0 2024-03-30T02:46:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2afa800700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ae00a1290 0x7f2ae00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd4f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd4f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37436/0 (socket says 172.21.15.67:37436) 2024-03-30T02:46:01.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af9e00700 1 -- 172.21.15.67:0/4143462450 learned_addr learned my addr 172.21.15.67:0/4143462450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af3e00700 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2afa800700 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 0x561eedd5d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af3e00700 1 -- 172.21.15.67:0/4143462450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 msgr2=0x561eedd5d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af3e00700 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 0x561eedd5d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af3e00700 1 -- 172.21.15.67:0/4143462450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 msgr2=0x561eedd4f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af3e00700 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd4f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af3e00700 1 -- 172.21.15.67:0/4143462450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eedb233b0 con 0x561eedd4fef0 2024-03-30T02:46:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.896+0000 7f2af9e00700 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd4f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:01.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.897+0000 7f2af3e00700 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40ca0a0 tx=0x7f2ae4019ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f688f49f59b38d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:01.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.897+0000 7f2afa800700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ae00a1290 0x7f2ae00a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2af400b830 tx=0x7f2af4007b30 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:01.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.897+0000 7f2aeb000700 1 -- 172.21.15.67:0/4143462450 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae409d890 con 0x561eedd4fef0 2024-03-30T02:46:01.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.897+0000 7f2aeb000700 1 -- 172.21.15.67:0/4143462450 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae40a1ee0 con 0x561eedd4fef0 2024-03-30T02:46:01.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.897+0000 7f2aeb000700 1 -- 172.21.15.67:0/4143462450 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4099150 con 0x561eedd4fef0 2024-03-30T02:46:01.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 -- 172.21.15.67:0/4143462450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 msgr2=0x561eedd5a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:01.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40ca0a0 tx=0x7f2ae4019ee0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 -- 172.21.15.67:0/4143462450 shutdown_connections 2024-03-30T02:46:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd4f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 0x561eedd5d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/4143462450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 -- 172.21.15.67:0/4143462450 >> 172.21.15.67:0/4143462450 conn(0x561eedcaf6a0 msgr2=0x561eedd4ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 -- 172.21.15.67:0/4143462450 shutdown_connections 2024-03-30T02:46:01.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 -- 172.21.15.67:0/4143462450 wait complete. 2024-03-30T02:46:01.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.898+0000 7f2b146f6b80 1 -- start start 2024-03-30T02:46:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2b146f6b80 1 Processor -- start 2024-03-30T02:46:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd5a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 0x561eedd582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedc3dd80 con 0x561eedd4f5d0 2024-03-30T02:46:01.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedc3da20 con 0x561eedd5adf0 2024-03-30T02:46:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedb256a0 con 0x561eedd4fef0 2024-03-30T02:46:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2b146f6b80 10 client.0 _ll_register_callbacks cb 0x561eedc47470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2af9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd5a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2af3e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2af3e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd57b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56026/0 (socket says 172.21.15.67:56026) 2024-03-30T02:46:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2af3e00700 1 -- 172.21.15.67:0/3475166627 learned_addr learned my addr 172.21.15.67:0/3475166627 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.899+0000 7f2afa800700 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 0x561eedd582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2af3e00700 1 -- 172.21.15.67:0/3475166627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 msgr2=0x561eedd582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2af3e00700 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 0x561eedd582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2af3e00700 1 -- 172.21.15.67:0/3475166627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 msgr2=0x561eedd5a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2af3e00700 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd5a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2af3e00700 1 -- 172.21.15.67:0/3475166627 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec007d30 con 0x561eedd4fef0 2024-03-30T02:46:01.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2af3e00700 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd57b70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40a19b0 tx=0x7f2ae4095da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b79770e190a3a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2adf400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:46:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2adf400700 1 -- 172.21.15.67:0/3475166627 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4097ec0 con 0x561eedd4fef0 2024-03-30T02:46:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2adf400700 1 -- 172.21.15.67:0/3475166627 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2ae4098f50 con 0x561eedd4fef0 2024-03-30T02:46:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2b146f6b80 1 -- 172.21.15.67:0/3475166627 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561eedc62380 con 0x561eedd4fef0 2024-03-30T02:46:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2adf400700 1 -- 172.21.15.67:0/3475166627 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ae4099110 con 0x561eedd4fef0 2024-03-30T02:46:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2b146f6b80 10 client.27286 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.900+0000 7f2b146f6b80 1 -- 172.21.15.67:0/3475166627 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f2afc1a6620 con 0x561eedd4fef0 2024-03-30T02:46:01.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.901+0000 7f2adf400700 1 -- 172.21.15.67:0/3475166627 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561eedc62380 con 0x561eedd4fef0 2024-03-30T02:46:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.901+0000 7f2adf400700 1 -- 172.21.15.67:0/3475166627 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ae40989c0 con 0x561eedd4fef0 2024-03-30T02:46:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.901+0000 7f2b146f6b80 10 client.27286 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.901+0000 7f2b146f6b80 10 client.27286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.901+0000 7f2b146f6b80 20 client.27286 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.901+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561eedd72d00 0x561eedd750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.901+0000 7f2b146f6b80 4 client.27286 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.901+0000 7f2b146f6b80 1 -- 172.21.15.67:0/3475166627 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561eedc9a700 con 0x561eedd72d00 2024-03-30T02:46:01.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.901+0000 7f2af9e00700 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561eedd72d00 0x561eedd750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.902+0000 7f2af9e00700 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561eedd72d00 0x561eedd750f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4407cbe5a798ad38 server_cookie=888bba48f8b98bb5 in_seq=0 out_seq=0 2024-03-30T02:46:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.902+0000 7f2adf400700 10 client.27286 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2adf400700 1 -- 172.21.15.67:0/3475166627 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561eedc9a700 con 0x561eedd72d00 2024-03-30T02:46:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2adf400700 10 client.27286 handle_command_reply: tid=0 2024-03-30T02:46:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 client.27286 shutdown 2024-03-30T02:46:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 -- 172.21.15.67:0/3475166627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 msgr2=0x561eedd57b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:01.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd57b70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40a19b0 tx=0x7f2ae4095da0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 -- 172.21.15.67:0/3475166627 shutdown_connections 2024-03-30T02:46:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561eedd72d00 0x561eedd750f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4f5d0 0x561eedd5a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd5adf0 0x561eedd582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2adf400700 0 client.27286 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/3475166627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd57b70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 -- 172.21.15.67:0/3475166627 >> 172.21.15.67:0/3475166627 conn(0x561eedcaf6a0 msgr2=0x561eedd516c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:01.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 -- 172.21.15.67:0/3475166627 shutdown_connections 2024-03-30T02:46:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 1 -- 172.21.15.67:0/3475166627 wait complete. 2024-03-30T02:46:01.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.903+0000 7f2b146f6b80 20 client.27286 trim_cache size 0 max 16384 2024-03-30T02:46:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2b146f6b80 1 Processor -- start 2024-03-30T02:46:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2b146f6b80 1 -- start start 2024-03-30T02:46:01.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 0x561eedd5a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd5adf0 0x561eedd52e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedd59120 con 0x561eedd5adf0 2024-03-30T02:46:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedd58620 con 0x561eedd4f5d0 2024-03-30T02:46:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedd4bfa0 con 0x561eedd4fef0 2024-03-30T02:46:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2afa800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd5adf0 0x561eedd52e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2afa800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd5adf0 0x561eedd52e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37460/0 (socket says 172.21.15.67:37460) 2024-03-30T02:46:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2afa800700 1 -- 172.21.15.67:0/1736236226 learned_addr learned my addr 172.21.15.67:0/1736236226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:01.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.921+0000 7f2af3e00700 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2af9e00700 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 0x561eedd5a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2afa800700 1 -- 172.21.15.67:0/1736236226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 msgr2=0x561eedd5a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2afa800700 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2afa800700 1 -- 172.21.15.67:0/1736236226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 msgr2=0x561eedd5a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2afa800700 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 0x561eedd5a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2afa800700 1 -- 172.21.15.67:0/1736236226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eedacfb60 con 0x561eedd5adf0 2024-03-30T02:46:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2af9e00700 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 0x561eedd5a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:01.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2af3e00700 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2afa800700 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd5adf0 0x561eedd52e90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2af400b0a0 tx=0x7f2af400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e30d5c0f7815679 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2aeba00700 1 -- 172.21.15.67:0/1736236226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4025040 con 0x561eedd5adf0 2024-03-30T02:46:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2aeba00700 1 -- 172.21.15.67:0/1736236226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2af4007760 con 0x561eedd5adf0 2024-03-30T02:46:01.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.922+0000 7f2aeba00700 1 -- 172.21.15.67:0/1736236226 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af400e040 con 0x561eedd5adf0 2024-03-30T02:46:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 -- 172.21.15.67:0/1736236226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd5adf0 msgr2=0x561eedd52e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd5adf0 0x561eedd52e90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2af400b0a0 tx=0x7f2af400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 -- 172.21.15.67:0/1736236226 shutdown_connections 2024-03-30T02:46:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd5adf0 0x561eedd52e90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 0x561eedd5a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/1736236226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd4fef0 0x561eedd5a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 -- 172.21.15.67:0/1736236226 >> 172.21.15.67:0/1736236226 conn(0x561eedcaf6a0 msgr2=0x561eedd445a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 -- 172.21.15.67:0/1736236226 shutdown_connections 2024-03-30T02:46:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 -- 172.21.15.67:0/1736236226 wait complete. 2024-03-30T02:46:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.923+0000 7f2b146f6b80 1 -- start start 2024-03-30T02:46:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2b146f6b80 1 Processor -- start 2024-03-30T02:46:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 0x561eedd68c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4fef0 0x561eedd69160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2b146f6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd5adf0 0x561eedd696a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeddc6b10 con 0x561eedd4fef0 2024-03-30T02:46:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeddc6c50 con 0x561eedd4f5d0 2024-03-30T02:46:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2b146f6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeddc6e10 con 0x561eedd5adf0 2024-03-30T02:46:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2b146f6b80 10 client.0 _ll_register_callbacks cb 0x561eedc47470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2af3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4fef0 0x561eedd69160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2af9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 0x561eedd68c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2af3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4fef0 0x561eedd69160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37474/0 (socket says 172.21.15.67:37474) 2024-03-30T02:46:01.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2af3e00700 1 -- 172.21.15.67:0/2952669799 learned_addr learned my addr 172.21.15.67:0/2952669799 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.924+0000 7f2afa800700 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd5adf0 0x561eedd696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2af3e00700 1 -- 172.21.15.67:0/2952669799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd5adf0 msgr2=0x561eedd696a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2af3e00700 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd5adf0 0x561eedd696a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2af3e00700 1 -- 172.21.15.67:0/2952669799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 msgr2=0x561eedd68c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2af3e00700 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 0x561eedd68c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2af3e00700 1 -- 172.21.15.67:0/2952669799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc10e470 con 0x561eedd4fef0 2024-03-30T02:46:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2af3e00700 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4fef0 0x561eedd69160 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4099760 tx=0x7f2ae4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128f8150cca8291b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2adf400700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:01.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2adf400700 1 -- 172.21.15.67:0/2952669799 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40956a0 con 0x561eedd4fef0 2024-03-30T02:46:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2adf400700 1 -- 172.21.15.67:0/2952669799 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2ae4097ec0 con 0x561eedd4fef0 2024-03-30T02:46:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2b146f6b80 1 -- 172.21.15.67:0/2952669799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ae40a1ee0 con 0x561eedd4fef0 2024-03-30T02:46:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2adf400700 1 -- 172.21.15.67:0/2952669799 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ae40a1d10 con 0x561eedd4fef0 2024-03-30T02:46:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2b146f6b80 10 client.17217 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.925+0000 7f2b146f6b80 1 -- 172.21.15.67:0/2952669799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561eedc67ea0 con 0x561eedd4fef0 2024-03-30T02:46:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.926+0000 7f2adf400700 1 -- 172.21.15.67:0/2952669799 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae40a1ee0 con 0x561eedd4fef0 2024-03-30T02:46:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.926+0000 7f2adf400700 1 -- 172.21.15.67:0/2952669799 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2ae4098f50 con 0x561eedd4fef0 2024-03-30T02:46:01.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.926+0000 7f2b146f6b80 10 client.17217 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.926+0000 7f2b146f6b80 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.926+0000 7f2b146f6b80 20 client.17217 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.926+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561eeddc80d0 0x561eeddca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.926+0000 7f2b146f6b80 4 client.17217 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:46:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.926+0000 7f2b146f6b80 1 -- 172.21.15.67:0/2952669799 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561eedaf0cf0 con 0x561eeddc80d0 2024-03-30T02:46:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.926+0000 7f2af9e00700 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561eeddc80d0 0x561eeddca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2af9e00700 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561eeddc80d0 0x561eeddca4c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320cc91a829b67cc server_cookie=23788cb970dc0028 in_seq=0 out_seq=0 2024-03-30T02:46:01.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2adf400700 10 client.17217 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2adf400700 1 -- 172.21.15.67:0/2952669799 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561eedaf0cf0 con 0x561eeddc80d0 2024-03-30T02:46:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2adf400700 10 client.17217 handle_command_reply: tid=0 2024-03-30T02:46:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2b146f6b80 1 client.17217 shutdown 2024-03-30T02:46:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2b146f6b80 1 -- 172.21.15.67:0/2952669799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4fef0 msgr2=0x561eedd69160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4fef0 0x561eedd69160 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4099760 tx=0x7f2ae4004610 comp rx=0 tx=0).stop 2024-03-30T02:46:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2b146f6b80 1 -- 172.21.15.67:0/2952669799 shutdown_connections 2024-03-30T02:46:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561eedd5adf0 0x561eedd696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561eedd4f5d0 0x561eedd68c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x561eeddc80d0 0x561eeddca4c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2b146f6b80 1 --2- 172.21.15.67:0/2952669799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561eedd4fef0 0x561eedd69160 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2adf400700 0 client.17217 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.927+0000 7f2b146f6b80 1 -- 172.21.15.67:0/2952669799 >> 172.21.15.67:0/2952669799 conn(0x561eedcaf6a0 msgr2=0x561eedd52e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b146f6b80 1 -- 172.21.15.67:0/2952669799 shutdown_connections 2024-03-30T02:46:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b146f6b80 1 -- 172.21.15.67:0/2952669799 wait complete. 2024-03-30T02:46:01.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b146f6b80 20 client.17217 trim_cache size 0 max 16384 2024-03-30T02:46:01.937 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:46:01.937 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:46:01.937 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:46:01.937 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:46:01.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 -- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ae00a1290 msgr2=0x7f2ae00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:01.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ae00a1290 0x7f2ae00a3750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2af400b830 tx=0x7f2af4007b30 comp rx=0 tx=0).stop 2024-03-30T02:46:01.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 -- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 msgr2=0x7f2afc199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:01.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 0x7f2afc199b80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400d950 tx=0x7f2ae400dc60 comp rx=0 tx=0).stop 2024-03-30T02:46:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 -- 172.21.15.67:0/3819350747 shutdown_connections 2024-03-30T02:46:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ae00a1290 0x7f2ae00a3750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2afc19de30 0x7f2afc19e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2afc100f60 0x7f2afc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 --2- 172.21.15.67:0/3819350747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2afc106080 0x7f2afc199b80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.928+0000 7f2b00c00700 1 -- 172.21.15.67:0/3819350747 >> 172.21.15.67:0/3819350747 conn(0x7f2afc0fc7c0 msgr2=0x7f2afc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.929+0000 7f2b00c00700 1 -- 172.21.15.67:0/3819350747 shutdown_connections 2024-03-30T02:46:01.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:01.929+0000 7f2b00c00700 1 -- 172.21.15.67:0/3819350747 wait complete. 2024-03-30T02:46:01.947 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:46:01.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:46:02.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.147+0000 7f74ca000700 1 -- 172.21.15.67:0/1008656201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4100e60 msgr2=0x7f74c4101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.147+0000 7f74ca000700 1 --2- 172.21.15.67:0/1008656201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4100e60 0x7f74c4101240 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009990 tx=0x7f74b4009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.147+0000 7f74ca000700 1 -- 172.21.15.67:0/1008656201 shutdown_connections 2024-03-30T02:46:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.147+0000 7f74ca000700 1 --2- 172.21.15.67:0/1008656201 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4100e60 0x7f74c4101240 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.147+0000 7f74ca000700 1 --2- 172.21.15.67:0/1008656201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c4105f80 0x7f74c410db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.147+0000 7f74ca000700 1 --2- 172.21.15.67:0/1008656201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c4101810 0x7f74c4105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.147+0000 7f74ca000700 1 -- 172.21.15.67:0/1008656201 >> 172.21.15.67:0/1008656201 conn(0x7f74c40fc700 msgr2=0x7f74c40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.147+0000 7f74ca000700 1 -- 172.21.15.67:0/1008656201 shutdown_connections 2024-03-30T02:46:02.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.147+0000 7f74ca000700 1 -- 172.21.15.67:0/1008656201 wait complete. 2024-03-30T02:46:02.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.148+0000 7f74ca000700 1 Processor -- start 2024-03-30T02:46:02.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.148+0000 7f74ca000700 1 -- start start 2024-03-30T02:46:02.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74ca000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c4100e60 0x7f74c40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74ca000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4105f80 0x7f74c40735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74ca000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c4073b30 0x7f74c41a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74ca000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c41a5920 con 0x7f74c4105f80 2024-03-30T02:46:02.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74ca000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c41a5a90 con 0x7f74c4100e60 2024-03-30T02:46:02.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74ca000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c41a5c50 con 0x7f74c4073b30 2024-03-30T02:46:02.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4105f80 0x7f74c40735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4105f80 0x7f74c40735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37496/0 (socket says 172.21.15.67:37496) 2024-03-30T02:46:02.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c2a00700 1 -- 172.21.15.67:0/444510358 learned_addr learned my addr 172.21.15.67:0/444510358 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c3400700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c4100e60 0x7f74c40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c3e00700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c4073b30 0x7f74c41a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c2a00700 1 -- 172.21.15.67:0/444510358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c4073b30 msgr2=0x7f74c41a5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c2a00700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c4073b30 0x7f74c41a5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c2a00700 1 -- 172.21.15.67:0/444510358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c4100e60 msgr2=0x7f74c40730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c2a00700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c4100e60 0x7f74c40730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c2a00700 1 -- 172.21.15.67:0/444510358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc009c40 con 0x7f74c4105f80 2024-03-30T02:46:02.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.149+0000 7f74c3e00700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c4073b30 0x7f74c41a5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:02.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.150+0000 7f74c2a00700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4105f80 0x7f74c40735f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f74b000e970 tx=0x7f74b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.150+0000 7f74bbe00700 1 -- 172.21.15.67:0/444510358 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0009790 con 0x7f74c4105f80 2024-03-30T02:46:02.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.150+0000 7f74bbe00700 1 -- 172.21.15.67:0/444510358 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74b000eec0 con 0x7f74c4105f80 2024-03-30T02:46:02.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.150+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b40095f0 con 0x7f74c4105f80 2024-03-30T02:46:02.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.150+0000 7f74bbe00700 1 -- 172.21.15.67:0/444510358 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0005590 con 0x7f74c4105f80 2024-03-30T02:46:02.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.150+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c41a6250 con 0x7f74c4105f80 2024-03-30T02:46:02.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.151+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c41091a0 con 0x7f74c4105f80 2024-03-30T02:46:02.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.153+0000 7f74bbe00700 1 -- 172.21.15.67:0/444510358 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f74b000cbd0 con 0x7f74c4105f80 2024-03-30T02:46:02.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.154+0000 7f74bbe00700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74a80a1290 0x7f74a80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.155+0000 7f74bbe00700 1 -- 172.21.15.67:0/444510358 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f74b0014070 con 0x7f74c4105f80 2024-03-30T02:46:02.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.155+0000 7f74c3400700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74a80a1290 0x7f74a80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.155+0000 7f74c3400700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74a80a1290 0x7f74a80a3750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f74b400b2e0 tx=0x7f74b4005fd0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.159+0000 7f74bbe00700 1 -- 172.21.15.67:0/444510358 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f74b0096340 con 0x7f74c4105f80 2024-03-30T02:46:02.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.427+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74c4102330 con 0x7f74c4105f80 2024-03-30T02:46:02.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.428+0000 7f74bbe00700 1 -- 172.21.15.67:0/444510358 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f74b0004ef0 con 0x7f74c4105f80 2024-03-30T02:46:02.428 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:46:02.428 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:45:07.400846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:46:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.430+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74a80a1290 msgr2=0x7f74a80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.430+0000 7f74ca000700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74a80a1290 0x7f74a80a3750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f74b400b2e0 tx=0x7f74b4005fd0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.430+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4105f80 msgr2=0x7f74c40735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.430+0000 7f74ca000700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4105f80 0x7f74c40735f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f74b000e970 tx=0x7f74b000ec80 comp rx=0 tx=0).stop 2024-03-30T02:46:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.431+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 shutdown_connections 2024-03-30T02:46:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.431+0000 7f74ca000700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f74a80a1290 0x7f74a80a3750 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.431+0000 7f74ca000700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c4073b30 0x7f74c41a5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.431+0000 7f74ca000700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c4100e60 0x7f74c40730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.431+0000 7f74ca000700 1 --2- 172.21.15.67:0/444510358 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74c4105f80 0x7f74c40735f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.431+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 >> 172.21.15.67:0/444510358 conn(0x7f74c40fc700 msgr2=0x7f74c40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.431+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 shutdown_connections 2024-03-30T02:46:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.431+0000 7f74ca000700 1 -- 172.21.15.67:0/444510358 wait complete. 2024-03-30T02:46:02.432 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-03-30T02:46:02.451 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:46:02.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.653+0000 7f6151c00700 1 -- 172.21.15.67:0/1324270037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c106070 msgr2=0x7f614c10dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.653+0000 7f6151c00700 1 --2- 172.21.15.67:0/1324270037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c106070 0x7f614c10dbf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f614400b0a0 tx=0x7f614400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.654+0000 7f6151c00700 1 -- 172.21.15.67:0/1324270037 shutdown_connections 2024-03-30T02:46:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.654+0000 7f6151c00700 1 --2- 172.21.15.67:0/1324270037 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c106070 0x7f614c10dbf0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.654+0000 7f6151c00700 1 --2- 172.21.15.67:0/1324270037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f614c100f50 0x7f614c101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.654+0000 7f6151c00700 1 --2- 172.21.15.67:0/1324270037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f614c101900 0x7f614c105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.654+0000 7f6151c00700 1 -- 172.21.15.67:0/1324270037 >> 172.21.15.67:0/1324270037 conn(0x7f614c0fc7f0 msgr2=0x7f614c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.654+0000 7f6151c00700 1 -- 172.21.15.67:0/1324270037 shutdown_connections 2024-03-30T02:46:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.654+0000 7f6151c00700 1 -- 172.21.15.67:0/1324270037 wait complete. 2024-03-30T02:46:02.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f6151c00700 1 Processor -- start 2024-03-30T02:46:02.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f6151c00700 1 -- start start 2024-03-30T02:46:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f6151c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c100f50 0x7f614c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f6151c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f614c106070 0x7f614c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f6151c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f614c19ddc0 0x7f614c19e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f6151c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c19e840 con 0x7f614c100f50 2024-03-30T02:46:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f6151c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c19e9b0 con 0x7f614c106070 2024-03-30T02:46:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f6151c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c19eb70 con 0x7f614c19ddc0 2024-03-30T02:46:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f614aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c100f50 0x7f614c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f614b400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f614c19ddc0 0x7f614c19e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f614aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c100f50 0x7f614c1994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37512/0 (socket says 172.21.15.67:37512) 2024-03-30T02:46:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.655+0000 7f614aa00700 1 -- 172.21.15.67:0/1532868697 learned_addr learned my addr 172.21.15.67:0/1532868697 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f614a000700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f614c106070 0x7f614c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f614b400700 1 -- 172.21.15.67:0/1532868697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f614c106070 msgr2=0x7f614c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f614b400700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f614c106070 0x7f614c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f614b400700 1 -- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c100f50 msgr2=0x7f614c1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f614b400700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c100f50 0x7f614c1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f614b400700 1 -- 172.21.15.67:0/1532868697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144009d00 con 0x7f614c19ddc0 2024-03-30T02:46:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f614aa00700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c100f50 0x7f614c1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f614b400700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f614c19ddc0 0x7f614c19e240 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f61440062a0 tx=0x7f614400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f6143400700 1 -- 172.21.15.67:0/1532868697 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614400e040 con 0x7f614c19ddc0 2024-03-30T02:46:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f6151c00700 1 -- 172.21.15.67:0/1532868697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c1a6070 con 0x7f614c19ddc0 2024-03-30T02:46:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.656+0000 7f6143400700 1 -- 172.21.15.67:0/1532868697 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f614401d480 con 0x7f614c19ddc0 2024-03-30T02:46:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.657+0000 7f6143400700 1 -- 172.21.15.67:0/1532868697 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614401b6d0 con 0x7f614c19ddc0 2024-03-30T02:46:02.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.657+0000 7f6151c00700 1 -- 172.21.15.67:0/1532868697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c1a6540 con 0x7f614c19ddc0 2024-03-30T02:46:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.659+0000 7f6143400700 1 -- 172.21.15.67:0/1532868697 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f614401d5f0 con 0x7f614c19ddc0 2024-03-30T02:46:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f61657a9b80 1 Processor -- start 2024-03-30T02:46:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f61657a9b80 1 -- start start 2024-03-30T02:46:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f6143400700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f61300a1290 0x7f61300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f6143400700 1 -- 172.21.15.67:0/1532868697 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f61440ce680 con 0x7f614c19ddc0 2024-03-30T02:46:02.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b78015d0 0x5564b78019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b780c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 0x5564b780f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f61657a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b775a990 con 0x5564b78015d0 2024-03-30T02:46:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f61657a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b7758350 con 0x5564b7801ef0 2024-03-30T02:46:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f61657a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b7753c40 con 0x5564b780cdf0 2024-03-30T02:46:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f614a000700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f61300a1290 0x7f61300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.661+0000 7f614aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b78015d0 0x5564b78019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b780c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b78015d0 0x5564b78019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37528/0 (socket says 172.21.15.67:37528) 2024-03-30T02:46:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614aa00700 1 -- 172.21.15.67:0/1907579659 learned_addr learned my addr 172.21.15.67:0/1907579659 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614b400700 1 --2- 172.21.15.67:0/1907579659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 0x5564b780f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614aa00700 1 -- 172.21.15.67:0/1907579659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 msgr2=0x5564b780f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614aa00700 1 --2- 172.21.15.67:0/1907579659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 0x5564b780f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614aa00700 1 -- 172.21.15.67:0/1907579659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 msgr2=0x5564b780c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614aa00700 1 --2- 172.21.15.67:0/1907579659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b780c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614aa00700 1 -- 172.21.15.67:0/1907579659 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564b75d53b0 con 0x5564b78015d0 2024-03-30T02:46:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614b400700 1 --2- 172.21.15.67:0/1907579659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 0x5564b780f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614a000700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f61300a1290 0x7f61300a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f614c19aa30 tx=0x7f6138009ae0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.662+0000 7f614aa00700 1 --2- 172.21.15.67:0/1907579659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b78015d0 0x5564b78019b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f613c0117e0 tx=0x7f613c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7188f39f6642c74f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f6137e00700 1 -- 172.21.15.67:0/1907579659 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c00f480 con 0x5564b78015d0 2024-03-30T02:46:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f6137e00700 1 -- 172.21.15.67:0/1907579659 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613c00faa0 con 0x5564b78015d0 2024-03-30T02:46:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f6137e00700 1 -- 172.21.15.67:0/1907579659 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c019990 con 0x5564b78015d0 2024-03-30T02:46:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f61657a9b80 1 -- 172.21.15.67:0/1907579659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b78015d0 msgr2=0x5564b78019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1907579659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b78015d0 0x5564b78019b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f613c0117e0 tx=0x7f613c011af0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f61657a9b80 1 -- 172.21.15.67:0/1907579659 shutdown_connections 2024-03-30T02:46:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1907579659 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b78015d0 0x5564b78019b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1907579659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 0x5564b780f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1907579659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b780c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f61657a9b80 1 -- 172.21.15.67:0/1907579659 >> 172.21.15.67:0/1907579659 conn(0x5564b77616a0 msgr2=0x5564b77fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f61657a9b80 1 -- 172.21.15.67:0/1907579659 shutdown_connections 2024-03-30T02:46:02.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.663+0000 7f61657a9b80 1 -- 172.21.15.67:0/1907579659 wait complete. 2024-03-30T02:46:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.664+0000 7f61657a9b80 1 -- start start 2024-03-30T02:46:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.664+0000 7f61657a9b80 1 Processor -- start 2024-03-30T02:46:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 0x5564b780c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b7809a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b780a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f61657a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b76efd80 con 0x5564b780cdf0 2024-03-30T02:46:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f61657a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b76efa20 con 0x5564b7801ef0 2024-03-30T02:46:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f61657a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b75d76a0 con 0x5564b78015d0 2024-03-30T02:46:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f61657a9b80 10 client.0 _ll_register_callbacks cb 0x5564b76f9470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b780a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b780a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37534/0 (socket says 172.21.15.67:37534) 2024-03-30T02:46:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614b400700 1 -- 172.21.15.67:0/7609256 learned_addr learned my addr 172.21.15.67:0/7609256 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614aa00700 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 0x5564b780c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614a000700 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b7809a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614b400700 1 -- 172.21.15.67:0/7609256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 msgr2=0x5564b780c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614b400700 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 0x5564b780c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614b400700 1 -- 172.21.15.67:0/7609256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 msgr2=0x5564b7809a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614b400700 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b7809a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614b400700 1 -- 172.21.15.67:0/7609256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c1a6540 con 0x5564b780cdf0 2024-03-30T02:46:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.665+0000 7f614a000700 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b7809a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f614b400700 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b780a040 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6144009fd0 tx=0x7f614409afe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b375745d7a8240cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f6134c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f6134c00700 1 -- 172.21.15.67:0/7609256 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614401dad0 con 0x5564b780cdf0 2024-03-30T02:46:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f6134c00700 1 -- 172.21.15.67:0/7609256 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f614409b420 con 0x5564b780cdf0 2024-03-30T02:46:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f61657a9b80 1 -- 172.21.15.67:0/7609256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564b7714380 con 0x5564b780cdf0 2024-03-30T02:46:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f6134c00700 1 -- 172.21.15.67:0/7609256 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f614409f250 con 0x5564b780cdf0 2024-03-30T02:46:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f61657a9b80 10 client.17235 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f61657a9b80 1 -- 172.21.15.67:0/7609256 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f613c011440 con 0x5564b780cdf0 2024-03-30T02:46:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f6134c00700 1 -- 172.21.15.67:0/7609256 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564b7714380 con 0x5564b780cdf0 2024-03-30T02:46:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f6134c00700 1 -- 172.21.15.67:0/7609256 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f614409b780 con 0x5564b780cdf0 2024-03-30T02:46:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f61657a9b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.666+0000 7f61657a9b80 10 client.17235 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:46:02.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.667+0000 7f61657a9b80 20 client.17235 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.667+0000 7f61657a9b80 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5564b781c6a0 0x5564b781eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.667+0000 7f61657a9b80 4 client.17235 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.667+0000 7f61657a9b80 1 -- 172.21.15.67:0/7609256 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564b774c700 con 0x5564b781c6a0 2024-03-30T02:46:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.667+0000 7f614aa00700 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5564b781c6a0 0x5564b781eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.668+0000 7f614aa00700 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5564b781c6a0 0x5564b781eac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f6f984b313210d1 server_cookie=6ba1a4b6d404f88c in_seq=0 out_seq=0 2024-03-30T02:46:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.668+0000 7f6134c00700 10 client.17235 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:46:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.668+0000 7f6134c00700 1 -- 172.21.15.67:0/7609256 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5564b774c700 con 0x5564b781c6a0 2024-03-30T02:46:02.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.668+0000 7f6134c00700 10 client.17235 handle_command_reply: tid=0 2024-03-30T02:46:02.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.668+0000 7f61657a9b80 1 client.17235 shutdown 2024-03-30T02:46:02.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.668+0000 7f61657a9b80 1 -- 172.21.15.67:0/7609256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 msgr2=0x5564b780a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.668+0000 7f61657a9b80 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b780a040 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6144009fd0 tx=0x7f614409afe0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f61657a9b80 1 -- 172.21.15.67:0/7609256 shutdown_connections 2024-03-30T02:46:02.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f61657a9b80 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b780a040 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f61657a9b80 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5564b781c6a0 0x5564b781eac0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f61657a9b80 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 0x5564b780c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f61657a9b80 1 --2- 172.21.15.67:0/7609256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b7809a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f6134c00700 0 client.17235 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:46:02.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f61657a9b80 1 -- 172.21.15.67:0/7609256 >> 172.21.15.67:0/7609256 conn(0x5564b77616a0 msgr2=0x5564b7803670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f61657a9b80 1 -- 172.21.15.67:0/7609256 shutdown_connections 2024-03-30T02:46:02.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f61657a9b80 1 -- 172.21.15.67:0/7609256 wait complete. 2024-03-30T02:46:02.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.669+0000 7f61657a9b80 20 client.17235 trim_cache size 0 max 16384 2024-03-30T02:46:02.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.682+0000 7f61657a9b80 1 Processor -- start 2024-03-30T02:46:02.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.682+0000 7f61657a9b80 1 -- start start 2024-03-30T02:46:02.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 0x5564b77fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b7813a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b77f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f61657a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b7802850 con 0x5564b780cdf0 2024-03-30T02:46:02.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f61657a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b781ee50 con 0x5564b7801ef0 2024-03-30T02:46:02.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f61657a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b77fdfc0 con 0x5564b78015d0 2024-03-30T02:46:02.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b77f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614a000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b7813a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b77f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37546/0 (socket says 172.21.15.67:37546) 2024-03-30T02:46:02.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614b400700 1 -- 172.21.15.67:0/1592876662 learned_addr learned my addr 172.21.15.67:0/1592876662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614aa00700 1 --2- 172.21.15.67:0/1592876662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 0x5564b77fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614b400700 1 -- 172.21.15.67:0/1592876662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 msgr2=0x5564b77fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614b400700 1 --2- 172.21.15.67:0/1592876662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 0x5564b77fcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614b400700 1 -- 172.21.15.67:0/1592876662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 msgr2=0x5564b7813a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614b400700 1 --2- 172.21.15.67:0/1592876662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b7813a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.683+0000 7f614b400700 1 -- 172.21.15.67:0/1592876662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564b7581b60 con 0x5564b780cdf0 2024-03-30T02:46:02.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.684+0000 7f614b400700 1 --2- 172.21.15.67:0/1592876662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b77f65a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f614400d010 tx=0x7f614401dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847240db403923b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.684+0000 7f6140c00700 1 -- 172.21.15.67:0/1592876662 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614401dad0 con 0x5564b780cdf0 2024-03-30T02:46:02.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.684+0000 7f6140c00700 1 -- 172.21.15.67:0/1592876662 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f614409ab50 con 0x5564b780cdf0 2024-03-30T02:46:02.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.684+0000 7f6140c00700 1 -- 172.21.15.67:0/1592876662 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614409b8c0 con 0x5564b780cdf0 2024-03-30T02:46:02.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.684+0000 7f61657a9b80 1 -- 172.21.15.67:0/1592876662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 msgr2=0x5564b77f65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.684+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1592876662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b77f65a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f614400d010 tx=0x7f614401dc70 comp rx=0 tx=0).stop 2024-03-30T02:46:02.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.685+0000 7f61657a9b80 1 -- 172.21.15.67:0/1592876662 shutdown_connections 2024-03-30T02:46:02.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.685+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1592876662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b780cdf0 0x5564b77f65a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.685+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1592876662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b78015d0 0x5564b77fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.685+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1592876662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b7801ef0 0x5564b7813a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.685+0000 7f61657a9b80 1 -- 172.21.15.67:0/1592876662 >> 172.21.15.67:0/1592876662 conn(0x5564b77616a0 msgr2=0x5564b78092e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.685+0000 7f61657a9b80 1 -- 172.21.15.67:0/1592876662 shutdown_connections 2024-03-30T02:46:02.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.685+0000 7f61657a9b80 1 -- 172.21.15.67:0/1592876662 wait complete. 2024-03-30T02:46:02.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.685+0000 7f61657a9b80 1 -- start start 2024-03-30T02:46:02.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.685+0000 7f61657a9b80 1 Processor -- start 2024-03-30T02:46:02.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b78015d0 0x5564b7813770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b7801ef0 0x5564b7811bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f61657a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 0x5564b78120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f61657a9b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b7812630 con 0x5564b7801ef0 2024-03-30T02:46:02.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f61657a9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b78127a0 con 0x5564b78015d0 2024-03-30T02:46:02.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f61657a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564b7812960 con 0x5564b780cdf0 2024-03-30T02:46:02.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f61657a9b80 10 client.0 _ll_register_callbacks cb 0x5564b76f9470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:02.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b7801ef0 0x5564b7811bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614aa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b78015d0 0x5564b7813770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614a000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b7801ef0 0x5564b7811bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37552/0 (socket says 172.21.15.67:37552) 2024-03-30T02:46:02.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614a000700 1 -- 172.21.15.67:0/1876529424 learned_addr learned my addr 172.21.15.67:0/1876529424 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614b400700 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 0x5564b78120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614a000700 1 -- 172.21.15.67:0/1876529424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 msgr2=0x5564b78120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614a000700 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 0x5564b78120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614a000700 1 -- 172.21.15.67:0/1876529424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b78015d0 msgr2=0x5564b7813770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614a000700 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b78015d0 0x5564b7813770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614a000700 1 -- 172.21.15.67:0/1876529424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c011440 con 0x5564b7801ef0 2024-03-30T02:46:02.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.686+0000 7f614aa00700 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b78015d0 0x5564b7813770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.687+0000 7f614a000700 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b7801ef0 0x5564b7811bb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5564b780b090 tx=0x7f61380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b430e5bb1bb9836 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.687+0000 7f6134c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.687+0000 7f6134c00700 1 -- 172.21.15.67:0/1876529424 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138026070 con 0x5564b7801ef0 2024-03-30T02:46:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.687+0000 7f6134c00700 1 -- 172.21.15.67:0/1876529424 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f613800fbd0 con 0x5564b7801ef0 2024-03-30T02:46:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.687+0000 7f61657a9b80 1 -- 172.21.15.67:0/1876529424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f613c00faa0 con 0x5564b7801ef0 2024-03-30T02:46:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.687+0000 7f6134c00700 1 -- 172.21.15.67:0/1876529424 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f613800fd90 con 0x5564b7801ef0 2024-03-30T02:46:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.687+0000 7f61657a9b80 10 client.17241 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:02.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.687+0000 7f61657a9b80 1 -- 172.21.15.67:0/1876529424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5564b7719ea0 con 0x5564b7801ef0 2024-03-30T02:46:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.688+0000 7f6134c00700 1 -- 172.21.15.67:0/1876529424 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613c00faa0 con 0x5564b7801ef0 2024-03-30T02:46:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.688+0000 7f6134c00700 1 -- 172.21.15.67:0/1876529424 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f61380105c0 con 0x5564b7801ef0 2024-03-30T02:46:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.688+0000 7f61657a9b80 10 client.17241 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.688+0000 7f61657a9b80 10 client.17241 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:46:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.688+0000 7f61657a9b80 20 client.17241 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.688+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5564b7871930 0x5564b7873d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.688+0000 7f61657a9b80 4 client.17241 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:02.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.688+0000 7f61657a9b80 1 -- 172.21.15.67:0/1876529424 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5564b75a2cf0 con 0x5564b7871930 2024-03-30T02:46:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.688+0000 7f614aa00700 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5564b7871930 0x5564b7873d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f614aa00700 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5564b7871930 0x5564b7873d50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=928fe2307e589566 server_cookie=389f7138f507a2db in_seq=0 out_seq=0 2024-03-30T02:46:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f6134c00700 10 client.17241 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:46:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f6134c00700 1 -- 172.21.15.67:0/1876529424 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5564b75a2cf0 con 0x5564b7871930 2024-03-30T02:46:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f6134c00700 10 client.17241 handle_command_reply: tid=0 2024-03-30T02:46:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 client.17241 shutdown 2024-03-30T02:46:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 -- 172.21.15.67:0/1876529424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b7801ef0 msgr2=0x5564b7811bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b7801ef0 0x5564b7811bb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5564b780b090 tx=0x7f61380047c0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 -- 172.21.15.67:0/1876529424 shutdown_connections 2024-03-30T02:46:02.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5564b7871930 0x5564b7873d50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5564b780cdf0 0x5564b78120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564b78015d0 0x5564b7813770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 --2- 172.21.15.67:0/1876529424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5564b7801ef0 0x5564b7811bb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f6134c00700 0 client.17241 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:46:02.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 -- 172.21.15.67:0/1876529424 >> 172.21.15.67:0/1876529424 conn(0x5564b77616a0 msgr2=0x5564b7804d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 -- 172.21.15.67:0/1876529424 shutdown_connections 2024-03-30T02:46:02.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.689+0000 7f61657a9b80 1 -- 172.21.15.67:0/1876529424 wait complete. 2024-03-30T02:46:02.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f61657a9b80 20 client.17241 trim_cache size 0 max 16384 2024-03-30T02:46:02.698 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 -- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f61300a1290 msgr2=0x7f61300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f61300a1290 0x7f61300a3750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f614c19aa30 tx=0x7f6138009ae0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 -- 172.21.15.67:0/1532868697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f614c19ddc0 msgr2=0x7f614c19e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f614c19ddc0 0x7f614c19e240 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f61440062a0 tx=0x7f614400bd70 comp rx=0 tx=0).stop 2024-03-30T02:46:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 -- 172.21.15.67:0/1532868697 shutdown_connections 2024-03-30T02:46:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f61300a1290 0x7f61300a3750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f614c100f50 0x7f614c1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f614c19ddc0 0x7f614c19e240 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 --2- 172.21.15.67:0/1532868697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f614c106070 0x7f614c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 -- 172.21.15.67:0/1532868697 >> 172.21.15.67:0/1532868697 conn(0x7f614c0fc7f0 msgr2=0x7f614c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 -- 172.21.15.67:0/1532868697 shutdown_connections 2024-03-30T02:46:02.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.690+0000 7f6151c00700 1 -- 172.21.15.67:0/1532868697 wait complete. 2024-03-30T02:46:02.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:02 smithi067 ceph-mon[37808]: pgmap v276: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 15 op/s 2024-03-30T02:46:02.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:02 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/444510358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:46:02.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:46:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:02 smithi082 ceph-mon[79176]: pgmap v276: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 15 op/s 2024-03-30T02:46:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:02 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/444510358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:46:02.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.912+0000 7f6c69800700 1 -- 172.21.15.67:0/3411730616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64106080 msgr2=0x7f6c6410dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.912+0000 7f6c69800700 1 --2- 172.21.15.67:0/3411730616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64106080 0x7f6c6410dc00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b0a0 tx=0x7f6c5c00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.912+0000 7f6c69800700 1 -- 172.21.15.67:0/3411730616 shutdown_connections 2024-03-30T02:46:02.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.912+0000 7f6c69800700 1 --2- 172.21.15.67:0/3411730616 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64106080 0x7f6c6410dc00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.912+0000 7f6c69800700 1 --2- 172.21.15.67:0/3411730616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64100f60 0x7f6c64101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.912+0000 7f6c69800700 1 --2- 172.21.15.67:0/3411730616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c64101910 0x7f6c64105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.912+0000 7f6c69800700 1 -- 172.21.15.67:0/3411730616 >> 172.21.15.67:0/3411730616 conn(0x7f6c640fc7c0 msgr2=0x7f6c640febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.912+0000 7f6c69800700 1 -- 172.21.15.67:0/3411730616 shutdown_connections 2024-03-30T02:46:02.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.913+0000 7f6c69800700 1 -- 172.21.15.67:0/3411730616 wait complete. 2024-03-30T02:46:02.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.913+0000 7f6c69800700 1 Processor -- start 2024-03-30T02:46:02.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c69800700 1 -- start start 2024-03-30T02:46:02.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c69800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64100f60 0x7f6c64199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c69800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64106080 0x7f6c64199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c69800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c6419de30 0x7f6c6419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c69800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6419e8b0 con 0x7f6c64100f60 2024-03-30T02:46:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c69800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6419ea20 con 0x7f6c64106080 2024-03-30T02:46:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c69800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6419ebe0 con 0x7f6c6419de30 2024-03-30T02:46:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c62a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64100f60 0x7f6c64199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c62000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64106080 0x7f6c64199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c62a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64100f60 0x7f6c64199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37584/0 (socket says 172.21.15.67:37584) 2024-03-30T02:46:02.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c62a00700 1 -- 172.21.15.67:0/645311558 learned_addr learned my addr 172.21.15.67:0/645311558 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.914+0000 7f6c63400700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c6419de30 0x7f6c6419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c62a00700 1 -- 172.21.15.67:0/645311558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c6419de30 msgr2=0x7f6c6419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c62a00700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c6419de30 0x7f6c6419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c62a00700 1 -- 172.21.15.67:0/645311558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64106080 msgr2=0x7f6c64199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c62a00700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64106080 0x7f6c64199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c62a00700 1 -- 172.21.15.67:0/645311558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c009d00 con 0x7f6c64100f60 2024-03-30T02:46:02.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c62000700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64106080 0x7f6c64199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:02.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c62a00700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64100f60 0x7f6c64199550 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400e9a0 tx=0x7f6c5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c5b400700 1 -- 172.21.15.67:0/645311558 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54009790 con 0x7f6c64100f60 2024-03-30T02:46:02.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c5b400700 1 -- 172.21.15.67:0/645311558 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c54010430 con 0x7f6c64100f60 2024-03-30T02:46:02.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c69800700 1 -- 172.21.15.67:0/645311558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c641a6140 con 0x7f6c64100f60 2024-03-30T02:46:02.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c5b400700 1 -- 172.21.15.67:0/645311558 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54018680 con 0x7f6c64100f60 2024-03-30T02:46:02.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.915+0000 7f6c69800700 1 -- 172.21.15.67:0/645311558 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c641a6620 con 0x7f6c64100f60 2024-03-30T02:46:02.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.919+0000 7f6c5b400700 1 -- 172.21.15.67:0/645311558 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6c540105a0 con 0x7f6c64100f60 2024-03-30T02:46:02.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c7d282b80 1 Processor -- start 2024-03-30T02:46:02.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c7d282b80 1 -- start start 2024-03-30T02:46:02.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c5b400700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c480a1290 0x7f6c480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 0x55da5da7f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c5b400700 1 -- 172.21.15.67:0/645311558 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f6c54014070 con 0x7f6c64100f60 2024-03-30T02:46:02.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da8a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da8d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5d9d8990 con 0x55da5da7f5d0 2024-03-30T02:46:02.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5d9d6350 con 0x55da5da8adf0 2024-03-30T02:46:02.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5d9d1c40 con 0x55da5da7fef0 2024-03-30T02:46:02.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.920+0000 7f6c63400700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c480a1290 0x7f6c480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da8a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da8a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56140/0 (socket says 172.21.15.67:56140) 2024-03-30T02:46:02.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62000700 1 -- 172.21.15.67:0/2250162557 learned_addr learned my addr 172.21.15.67:0/2250162557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62a00700 1 --2- 172.21.15.67:0/2250162557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 0x55da5da7f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c63400700 1 --2- 172.21.15.67:0/2250162557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da8d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62000700 1 -- 172.21.15.67:0/2250162557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 msgr2=0x55da5da8d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62000700 1 --2- 172.21.15.67:0/2250162557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da8d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62000700 1 -- 172.21.15.67:0/2250162557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 msgr2=0x55da5da7f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62000700 1 --2- 172.21.15.67:0/2250162557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 0x55da5da7f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62000700 1 -- 172.21.15.67:0/2250162557 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da5d8533b0 con 0x55da5da7fef0 2024-03-30T02:46:02.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c63400700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c480a1290 0x7f6c480a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b780 tx=0x7f6c5c007d70 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.921+0000 7f6c62000700 1 --2- 172.21.15.67:0/2250162557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da8a8b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c500114f0 tx=0x7f6c50011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8c675c8d7a65fc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c4fe00700 1 -- 172.21.15.67:0/2250162557 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50004a30 con 0x55da5da7fef0 2024-03-30T02:46:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c4fe00700 1 -- 172.21.15.67:0/2250162557 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c500054e0 con 0x55da5da7fef0 2024-03-30T02:46:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c4fe00700 1 -- 172.21.15.67:0/2250162557 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000e8e0 con 0x55da5da7fef0 2024-03-30T02:46:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2250162557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 msgr2=0x55da5da8a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2250162557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da8a8b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c500114f0 tx=0x7f6c50011800 comp rx=0 tx=0).stop 2024-03-30T02:46:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2250162557 shutdown_connections 2024-03-30T02:46:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2250162557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 0x55da5da7f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2250162557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da8d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2250162557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da8a8b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2250162557 >> 172.21.15.67:0/2250162557 conn(0x55da5d9df6a0 msgr2=0x55da5da7ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.922+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2250162557 shutdown_connections 2024-03-30T02:46:02.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.923+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2250162557 wait complete. 2024-03-30T02:46:02.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.923+0000 7f6c7d282b80 1 -- start start 2024-03-30T02:46:02.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.923+0000 7f6c7d282b80 1 Processor -- start 2024-03-30T02:46:02.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.924+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7f5d0 0x55da5da8a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.924+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da87ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.924+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da880f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.924+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5d96dd80 con 0x55da5da7fef0 2024-03-30T02:46:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.924+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5d96da20 con 0x55da5da8adf0 2024-03-30T02:46:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.924+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5d8556a0 con 0x55da5da7f5d0 2024-03-30T02:46:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.924+0000 7f6c7d282b80 10 client.0 _ll_register_callbacks cb 0x55da5d977470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.924+0000 7f6c62000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da87ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.924+0000 7f6c62a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7f5d0 0x55da5da8a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c62000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da87ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37602/0 (socket says 172.21.15.67:37602) 2024-03-30T02:46:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c62000700 1 -- 172.21.15.67:0/2828518569 learned_addr learned my addr 172.21.15.67:0/2828518569 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c63400700 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da880f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c62a00700 1 -- 172.21.15.67:0/2828518569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 msgr2=0x55da5da880f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c62a00700 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da880f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c62a00700 1 -- 172.21.15.67:0/2828518569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 msgr2=0x55da5da87ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c62a00700 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da87ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c62a00700 1 -- 172.21.15.67:0/2828518569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c50009c90 con 0x55da5da7f5d0 2024-03-30T02:46:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c62000700 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da87ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:02.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c63400700 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da880f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:02.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c62a00700 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7f5d0 0x55da5da8a7f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6c540ca600 tx=0x7f6c54099fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fd748b89e737bdb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c4cc00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:46:02.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c4cc00700 1 -- 172.21.15.67:0/2828518569 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c540a2440 con 0x55da5da7f5d0 2024-03-30T02:46:02.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2828518569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da5d992380 con 0x55da5da7f5d0 2024-03-30T02:46:02.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.925+0000 7f6c4cc00700 1 -- 172.21.15.67:0/2828518569 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f6c540994b0 con 0x55da5da7f5d0 2024-03-30T02:46:02.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c4cc00700 1 -- 172.21.15.67:0/2828518569 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c54099670 con 0x55da5da7f5d0 2024-03-30T02:46:02.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c7d282b80 10 client.27328 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:02.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2828518569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55da5d7ffb60 con 0x55da5da7f5d0 2024-03-30T02:46:02.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c4cc00700 1 -- 172.21.15.67:0/2828518569 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c54063070 con 0x55da5da7f5d0 2024-03-30T02:46:02.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c4cc00700 1 -- 172.21.15.67:0/2828518569 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c540a20d0 con 0x55da5da7f5d0 2024-03-30T02:46:02.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c7d282b80 10 client.27328 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:02.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c7d282b80 10 client.27328 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:46:02.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c7d282b80 20 client.27328 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:02.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55da5da9a9a0 0x55da5da9cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c7d282b80 4 client.27328 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:02.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.926+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2828518569 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da5d9ca700 con 0x55da5da9a9a0 2024-03-30T02:46:02.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.927+0000 7f6c62000700 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55da5da9a9a0 0x55da5da9cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.927+0000 7f6c62000700 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55da5da9a9a0 0x55da5da9cdc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=211ba04f35056918 server_cookie=e45579a6bdf919fa in_seq=0 out_seq=0 2024-03-30T02:46:02.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.927+0000 7f6c4cc00700 10 client.27328 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:46:02.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c4cc00700 1 -- 172.21.15.67:0/2828518569 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55da5d9ca700 con 0x55da5da9a9a0 2024-03-30T02:46:02.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c4cc00700 10 client.27328 handle_command_reply: tid=0 2024-03-30T02:46:02.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 client.27328 shutdown 2024-03-30T02:46:02.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2828518569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7f5d0 msgr2=0x55da5da8a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7f5d0 0x55da5da8a7f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6c540ca600 tx=0x7f6c54099fc0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2828518569 shutdown_connections 2024-03-30T02:46:02.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55da5da9a9a0 0x55da5da9cdc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da880f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7f5d0 0x55da5da8a7f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/2828518569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da87ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c4cc00700 0 client.27328 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:46:02.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2828518569 >> 172.21.15.67:0/2828518569 conn(0x55da5d9df6a0 msgr2=0x55da5da816c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2828518569 shutdown_connections 2024-03-30T02:46:02.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 1 -- 172.21.15.67:0/2828518569 wait complete. 2024-03-30T02:46:02.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.928+0000 7f6c7d282b80 20 client.27328 trim_cache size 0 max 16384 2024-03-30T02:46:02.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:02 smithi170 ceph-mon[78798]: pgmap v276: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 83 KiB/s wr, 15 op/s 2024-03-30T02:46:02.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:02 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/444510358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:46:02.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.942+0000 7f6c7d282b80 1 Processor -- start 2024-03-30T02:46:02.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.942+0000 7f6c7d282b80 1 -- start start 2024-03-30T02:46:02.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.942+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 0x55da5da880f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.942+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da9cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.942+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da91a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.942+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5da886f0 con 0x55da5da7f5d0 2024-03-30T02:46:02.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.942+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5da91fb0 con 0x55da5da8adf0 2024-03-30T02:46:02.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.942+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5da7bf20 con 0x55da5da7fef0 2024-03-30T02:46:02.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.942+0000 7f6c62a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 0x55da5da880f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da9cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da9cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56160/0 (socket says 172.21.15.67:56160) 2024-03-30T02:46:02.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62000700 1 -- 172.21.15.67:0/852687077 learned_addr learned my addr 172.21.15.67:0/852687077 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c63400700 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da91a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62000700 1 -- 172.21.15.67:0/852687077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 msgr2=0x55da5da91a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62000700 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da91a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62000700 1 -- 172.21.15.67:0/852687077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 msgr2=0x55da5da880f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62000700 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 0x55da5da880f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62000700 1 -- 172.21.15.67:0/852687077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da5d997ea0 con 0x55da5da7fef0 2024-03-30T02:46:02.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c63400700 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da91a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:02.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62a00700 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 0x55da5da880f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:02.945 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c62000700 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da9cd90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009340 tx=0x7f6c500058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3015ad16fdd78245 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.943+0000 7f6c58c00700 1 -- 172.21.15.67:0/852687077 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5002a070 con 0x55da5da7fef0 2024-03-30T02:46:02.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c58c00700 1 -- 172.21.15.67:0/852687077 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c50004500 con 0x55da5da7fef0 2024-03-30T02:46:02.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c58c00700 1 -- 172.21.15.67:0/852687077 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50004c20 con 0x55da5da7fef0 2024-03-30T02:46:02.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c7d282b80 1 -- 172.21.15.67:0/852687077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 msgr2=0x55da5da9cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da9cd90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009340 tx=0x7f6c500058e0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c7d282b80 1 -- 172.21.15.67:0/852687077 shutdown_connections 2024-03-30T02:46:02.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7f5d0 0x55da5da880f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da8adf0 0x55da5da91a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/852687077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da7fef0 0x55da5da9cd90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c7d282b80 1 -- 172.21.15.67:0/852687077 >> 172.21.15.67:0/852687077 conn(0x55da5d9df6a0 msgr2=0x55da5da745a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c7d282b80 1 -- 172.21.15.67:0/852687077 shutdown_connections 2024-03-30T02:46:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.944+0000 7f6c7d282b80 1 -- 172.21.15.67:0/852687077 wait complete. 2024-03-30T02:46:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.945+0000 7f6c7d282b80 1 -- start start 2024-03-30T02:46:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.945+0000 7f6c7d282b80 1 Processor -- start 2024-03-30T02:46:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da7f5d0 0x55da5da911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da91730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c7d282b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da8adf0 0x55da5da8fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5da90020 con 0x55da5da7fef0 2024-03-30T02:46:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5da90160 con 0x55da5da7f5d0 2024-03-30T02:46:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c7d282b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5da90320 con 0x55da5da8adf0 2024-03-30T02:46:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c7d282b80 10 client.0 _ll_register_callbacks cb 0x55da5d977470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c62000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da91730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c62a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da7f5d0 0x55da5da911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c62a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da7f5d0 0x55da5da911f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49390/0 (socket says 172.21.15.67:49390) 2024-03-30T02:46:02.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c62a00700 1 -- 172.21.15.67:0/1886721808 learned_addr learned my addr 172.21.15.67:0/1886721808 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:02.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c63400700 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da8adf0 0x55da5da8fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c62a00700 1 -- 172.21.15.67:0/1886721808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da8adf0 msgr2=0x55da5da8fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c62a00700 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da8adf0 0x55da5da8fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c62a00700 1 -- 172.21.15.67:0/1886721808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 msgr2=0x55da5da91730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:02.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c62a00700 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da91730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.946+0000 7f6c62a00700 1 -- 172.21.15.67:0/1886721808 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da5d7ffb60 con 0x55da5da7f5d0 2024-03-30T02:46:02.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c62a00700 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da7f5d0 0x55da5da911f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6c54099cc0 tx=0x7f6c5409deb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa67e93b8e662d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:02.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c4cc00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:02.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c4cc00700 1 -- 172.21.15.67:0/1886721808 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c540a2440 con 0x55da5da7f5d0 2024-03-30T02:46:02.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c4cc00700 1 -- 172.21.15.67:0/1886721808 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f6c5407c060 con 0x55da5da7f5d0 2024-03-30T02:46:02.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c7d282b80 1 -- 172.21.15.67:0/1886721808 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c500054e0 con 0x55da5da7f5d0 2024-03-30T02:46:02.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c4cc00700 1 -- 172.21.15.67:0/1886721808 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c540b8250 con 0x55da5da7f5d0 2024-03-30T02:46:02.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c7d282b80 10 client.27398 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:02.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c7d282b80 1 -- 172.21.15.67:0/1886721808 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55da5d8533b0 con 0x55da5da7f5d0 2024-03-30T02:46:02.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c4cc00700 1 -- 172.21.15.67:0/1886721808 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c500054e0 con 0x55da5da7f5d0 2024-03-30T02:46:02.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.947+0000 7f6c4cc00700 1 -- 172.21.15.67:0/1886721808 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c540a1fb0 con 0x55da5da7f5d0 2024-03-30T02:46:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c7d282b80 10 client.27398 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c7d282b80 10 client.27398 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:46:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c7d282b80 20 client.27398 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55da5daef690 0x55da5daf1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c7d282b80 4 client.27398 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c7d282b80 1 -- 172.21.15.67:0/1886721808 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55da5d820cf0 con 0x55da5daef690 2024-03-30T02:46:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c62000700 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55da5daef690 0x55da5daf1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c62000700 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55da5daef690 0x55da5daf1ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3f6cfbcf9a5cfe7c server_cookie=a0af74933b619d81 in_seq=0 out_seq=0 2024-03-30T02:46:02.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c4cc00700 10 client.27398 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:46:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c4cc00700 1 -- 172.21.15.67:0/1886721808 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55da5d820cf0 con 0x55da5daef690 2024-03-30T02:46:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c4cc00700 10 client.27398 handle_command_reply: tid=0 2024-03-30T02:46:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.948+0000 7f6c7d282b80 1 client.27398 shutdown 2024-03-30T02:46:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 -- 172.21.15.67:0/1886721808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da7f5d0 msgr2=0x55da5da911f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da7f5d0 0x55da5da911f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6c54099cc0 tx=0x7f6c5409deb0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 -- 172.21.15.67:0/1886721808 shutdown_connections 2024-03-30T02:46:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55da5daef690 0x55da5daf1ab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da5da8adf0 0x55da5da8fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da5da7f5d0 0x55da5da911f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 --2- 172.21.15.67:0/1886721808 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55da5da7fef0 0x55da5da91730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c4cc00700 0 client.27398 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:46:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 -- 172.21.15.67:0/1886721808 >> 172.21.15.67:0/1886721808 conn(0x55da5d9df6a0 msgr2=0x55da5da86330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 -- 172.21.15.67:0/1886721808 shutdown_connections 2024-03-30T02:46:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 1 -- 172.21.15.67:0/1886721808 wait complete. 2024-03-30T02:46:02.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c7d282b80 20 client.27398 trim_cache size 0 max 16384 2024-03-30T02:46:02.957 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:02.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.949+0000 7f6c69800700 1 -- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c480a1290 msgr2=0x7f6c480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c480a1290 0x7f6c480a3750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b780 tx=0x7f6c5c007d70 comp rx=0 tx=0).stop 2024-03-30T02:46:02.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 -- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64100f60 msgr2=0x7f6c64199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:02.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64100f60 0x7f6c64199550 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400e9a0 tx=0x7f6c5400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 -- 172.21.15.67:0/645311558 shutdown_connections 2024-03-30T02:46:02.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f6c480a1290 0x7f6c480a3750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6c64100f60 0x7f6c64199550 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c6419de30 0x7f6c6419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 --2- 172.21.15.67:0/645311558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c64106080 0x7f6c64199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:02.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 -- 172.21.15.67:0/645311558 >> 172.21.15.67:0/645311558 conn(0x7f6c640fc7c0 msgr2=0x7f6c6410d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:02.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 -- 172.21.15.67:0/645311558 shutdown_connections 2024-03-30T02:46:02.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:02.950+0000 7f6c69800700 1 -- 172.21.15.67:0/645311558 wait complete. 2024-03-30T02:46:02.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:46:03.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.170+0000 7f2229e00700 1 -- 172.21.15.67:0/3395229316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224106080 msgr2=0x7f222410dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.170+0000 7f2229e00700 1 --2- 172.21.15.67:0/3395229316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224106080 0x7f222410dc00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f221c00b0a0 tx=0x7f221c00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.170+0000 7f2229e00700 1 -- 172.21.15.67:0/3395229316 shutdown_connections 2024-03-30T02:46:03.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.170+0000 7f2229e00700 1 --2- 172.21.15.67:0/3395229316 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224106080 0x7f222410dc00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.170+0000 7f2229e00700 1 --2- 172.21.15.67:0/3395229316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224100f60 0x7f2224101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.170+0000 7f2229e00700 1 --2- 172.21.15.67:0/3395229316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2224101910 0x7f2224105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.171+0000 7f2229e00700 1 -- 172.21.15.67:0/3395229316 >> 172.21.15.67:0/3395229316 conn(0x7f22240fc7c0 msgr2=0x7f22240febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.171+0000 7f2229e00700 1 -- 172.21.15.67:0/3395229316 shutdown_connections 2024-03-30T02:46:03.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.171+0000 7f2229e00700 1 -- 172.21.15.67:0/3395229316 wait complete. 2024-03-30T02:46:03.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.172+0000 7f2229e00700 1 Processor -- start 2024-03-30T02:46:03.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.172+0000 7f2229e00700 1 -- start start 2024-03-30T02:46:03.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.172+0000 7f2229e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224100f60 0x7f22240730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.172+0000 7f2229e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2224106080 0x7f22240735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.172+0000 7f2229e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224073b30 0x7f22241a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.172+0000 7f2229e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22241a5b40 con 0x7f2224100f60 2024-03-30T02:46:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.172+0000 7f2229e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22241a5c80 con 0x7f2224106080 2024-03-30T02:46:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.172+0000 7f2229e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22241a5e40 con 0x7f2224073b30 2024-03-30T02:46:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224100f60 0x7f22240730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224100f60 0x7f22240730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37634/0 (socket says 172.21.15.67:37634) 2024-03-30T02:46:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222a00700 1 -- 172.21.15.67:0/1015260725 learned_addr learned my addr 172.21.15.67:0/1015260725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222000700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2224106080 0x7f22240735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2223400700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224073b30 0x7f22241a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222a00700 1 -- 172.21.15.67:0/1015260725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224073b30 msgr2=0x7f22241a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222a00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224073b30 0x7f22241a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222a00700 1 -- 172.21.15.67:0/1015260725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2224106080 msgr2=0x7f22240735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222a00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2224106080 0x7f22240735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222a00700 1 -- 172.21.15.67:0/1015260725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c009d00 con 0x7f2224100f60 2024-03-30T02:46:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2223400700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224073b30 0x7f22241a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2222a00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224100f60 0x7f22240730b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9a0 tx=0x7f221400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2228a00700 1 -- 172.21.15.67:0/1015260725 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214009790 con 0x7f2224100f60 2024-03-30T02:46:03.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2229e00700 1 -- 172.21.15.67:0/1015260725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22241a6120 con 0x7f2224100f60 2024-03-30T02:46:03.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.173+0000 7f2228a00700 1 -- 172.21.15.67:0/1015260725 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221400cbd0 con 0x7f2224100f60 2024-03-30T02:46:03.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.174+0000 7f2228a00700 1 -- 172.21.15.67:0/1015260725 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214018680 con 0x7f2224100f60 2024-03-30T02:46:03.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.174+0000 7f2229e00700 1 -- 172.21.15.67:0/1015260725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22241a6630 con 0x7f2224100f60 2024-03-30T02:46:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.177+0000 7f2228a00700 1 -- 172.21.15.67:0/1015260725 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f221400cd40 con 0x7f2224100f60 2024-03-30T02:46:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f223d970b80 1 Processor -- start 2024-03-30T02:46:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f223d970b80 1 -- start start 2024-03-30T02:46:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f2228a00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f22080a1290 0x7f22080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f2228a00700 1 -- 172.21.15.67:0/1015260725 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2214014070 con 0x7f2224100f60 2024-03-30T02:46:03.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f223d970b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c23759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f223d970b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 0x55c6c23808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f223d970b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 0x55c6c23831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f2223400700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f22080a1290 0x7f22080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f223d970b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c22ce990 con 0x55c6c23755d0 2024-03-30T02:46:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f223d970b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c22cc350 con 0x55c6c2380df0 2024-03-30T02:46:03.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.178+0000 7f223d970b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c22c7c40 con 0x55c6c2375ef0 2024-03-30T02:46:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c23759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c23759b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37646/0 (socket says 172.21.15.67:37646) 2024-03-30T02:46:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222a00700 1 -- 172.21.15.67:0/951963922 learned_addr learned my addr 172.21.15.67:0/951963922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222000700 1 --2- 172.21.15.67:0/951963922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 0x55c6c23808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2223400700 1 --2- 172.21.15.67:0/951963922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 0x55c6c23831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222a00700 1 -- 172.21.15.67:0/951963922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 msgr2=0x55c6c23808b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222a00700 1 --2- 172.21.15.67:0/951963922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 0x55c6c23808b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222a00700 1 -- 172.21.15.67:0/951963922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 msgr2=0x55c6c23831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222a00700 1 --2- 172.21.15.67:0/951963922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 0x55c6c23831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222a00700 1 -- 172.21.15.67:0/951963922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c21493b0 con 0x55c6c23755d0 2024-03-30T02:46:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2223400700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f22080a1290 0x7f22080a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f221c0062a0 tx=0x7f221c0095c0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.179+0000 7f2222a00700 1 --2- 172.21.15.67:0/951963922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c23759b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2214096ad0 tx=0x7f2214099a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef31793b755a4530 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f220fe00700 1 -- 172.21.15.67:0/951963922 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22140cd160 con 0x55c6c23755d0 2024-03-30T02:46:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f220fe00700 1 -- 172.21.15.67:0/951963922 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221409a060 con 0x55c6c23755d0 2024-03-30T02:46:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f220fe00700 1 -- 172.21.15.67:0/951963922 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22140997b0 con 0x55c6c23755d0 2024-03-30T02:46:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f223d970b80 1 -- 172.21.15.67:0/951963922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 msgr2=0x55c6c23759b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f223d970b80 1 --2- 172.21.15.67:0/951963922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c23759b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2214096ad0 tx=0x7f2214099a40 comp rx=0 tx=0).stop 2024-03-30T02:46:03.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f223d970b80 1 -- 172.21.15.67:0/951963922 shutdown_connections 2024-03-30T02:46:03.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f223d970b80 1 --2- 172.21.15.67:0/951963922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c23759b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f223d970b80 1 --2- 172.21.15.67:0/951963922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 0x55c6c23831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f223d970b80 1 --2- 172.21.15.67:0/951963922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 0x55c6c23808b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f223d970b80 1 -- 172.21.15.67:0/951963922 >> 172.21.15.67:0/951963922 conn(0x55c6c22d56a0 msgr2=0x55c6c2371a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f223d970b80 1 -- 172.21.15.67:0/951963922 shutdown_connections 2024-03-30T02:46:03.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.180+0000 7f223d970b80 1 -- 172.21.15.67:0/951963922 wait complete. 2024-03-30T02:46:03.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.181+0000 7f223d970b80 1 -- start start 2024-03-30T02:46:03.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.181+0000 7f223d970b80 1 Processor -- start 2024-03-30T02:46:03.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f223d970b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 0x55c6c2380870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f223d970b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f223d970b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c237e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f223d970b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c2263d80 con 0x55c6c2380df0 2024-03-30T02:46:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f223d970b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c2263a20 con 0x55c6c2375ef0 2024-03-30T02:46:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f223d970b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c214b6a0 con 0x55c6c23755d0 2024-03-30T02:46:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f223d970b80 10 client.0 _ll_register_callbacks cb 0x55c6c226d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2223400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c237e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2222000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2222000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49424/0 (socket says 172.21.15.67:49424) 2024-03-30T02:46:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2223400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c237e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37652/0 (socket says 172.21.15.67:37652) 2024-03-30T02:46:03.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2222000700 1 -- 172.21.15.67:0/2529710607 learned_addr learned my addr 172.21.15.67:0/2529710607 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2222a00700 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 0x55c6c2380870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2222000700 1 -- 172.21.15.67:0/2529710607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 msgr2=0x55c6c2380870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2222000700 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 0x55c6c2380870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2222000700 1 -- 172.21.15.67:0/2529710607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 msgr2=0x55c6c237e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2222000700 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c237e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2222000700 1 -- 172.21.15.67:0/2529710607 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22241a6630 con 0x55c6c2375ef0 2024-03-30T02:46:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.182+0000 7f2223400700 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c237e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:03.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.183+0000 7f2222000700 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237db60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f22100126d0 tx=0x7f22100129e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b017e37f2057121 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.183+0000 7f220cc00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.183+0000 7f220cc00700 1 -- 172.21.15.67:0/2529710607 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210017070 con 0x55c6c2375ef0 2024-03-30T02:46:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.183+0000 7f223d970b80 1 -- 172.21.15.67:0/2529710607 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6c2288380 con 0x55c6c2375ef0 2024-03-30T02:46:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.183+0000 7f220cc00700 1 -- 172.21.15.67:0/2529710607 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2210004030 con 0x55c6c2375ef0 2024-03-30T02:46:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.183+0000 7f220cc00700 1 -- 172.21.15.67:0/2529710607 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22100041f0 con 0x55c6c2375ef0 2024-03-30T02:46:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.183+0000 7f223d970b80 10 client.27410 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.183+0000 7f223d970b80 1 -- 172.21.15.67:0/2529710607 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c6c20f5b60 con 0x55c6c2375ef0 2024-03-30T02:46:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.183+0000 7f220cc00700 1 -- 172.21.15.67:0/2529710607 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6c2288380 con 0x55c6c2375ef0 2024-03-30T02:46:03.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.184+0000 7f220cc00700 1 -- 172.21.15.67:0/2529710607 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f221001c4f0 con 0x55c6c2375ef0 2024-03-30T02:46:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.184+0000 7f223d970b80 10 client.27410 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.184+0000 7f223d970b80 10 client.27410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:46:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.184+0000 7f223d970b80 20 client.27410 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.184+0000 7f223d970b80 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c6c2390ad0 0x55c6c2392ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.184+0000 7f223d970b80 4 client.27410 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.184+0000 7f223d970b80 1 -- 172.21.15.67:0/2529710607 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6c22c0700 con 0x55c6c2390ad0 2024-03-30T02:46:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.184+0000 7f2222a00700 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c6c2390ad0 0x55c6c2392ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.185+0000 7f2222a00700 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c6c2390ad0 0x55c6c2392ef0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67cdb67345d548f6 server_cookie=62e5fd68c7296765 in_seq=0 out_seq=0 2024-03-30T02:46:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.185+0000 7f220cc00700 10 client.27410 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:46:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.185+0000 7f220cc00700 1 -- 172.21.15.67:0/2529710607 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6c22c0700 con 0x55c6c2390ad0 2024-03-30T02:46:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.185+0000 7f220cc00700 10 client.27410 handle_command_reply: tid=0 2024-03-30T02:46:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.185+0000 7f223d970b80 1 client.27410 shutdown 2024-03-30T02:46:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.185+0000 7f223d970b80 1 -- 172.21.15.67:0/2529710607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 msgr2=0x55c6c237db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.185+0000 7f223d970b80 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237db60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f22100126d0 tx=0x7f22100129e0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f223d970b80 1 -- 172.21.15.67:0/2529710607 shutdown_connections 2024-03-30T02:46:03.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f223d970b80 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c6c2390ad0 0x55c6c2392ef0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f223d970b80 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c237e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f223d970b80 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 0x55c6c2380870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f220cc00700 0 client.27410 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:46:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f223d970b80 1 --2- 172.21.15.67:0/2529710607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237db60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f223d970b80 1 -- 172.21.15.67:0/2529710607 >> 172.21.15.67:0/2529710607 conn(0x55c6c22d56a0 msgr2=0x55c6c23776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f223d970b80 1 -- 172.21.15.67:0/2529710607 shutdown_connections 2024-03-30T02:46:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f223d970b80 1 -- 172.21.15.67:0/2529710607 wait complete. 2024-03-30T02:46:03.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.186+0000 7f223d970b80 20 client.27410 trim_cache size 0 max 16384 2024-03-30T02:46:03.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.199+0000 7f223d970b80 1 Processor -- start 2024-03-30T02:46:03.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.199+0000 7f223d970b80 1 -- start start 2024-03-30T02:46:03.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f223d970b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 0x55c6c237e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f223d970b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f223d970b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c2387a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f223d970b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c2377130 con 0x55c6c2380df0 2024-03-30T02:46:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f223d970b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c2376f60 con 0x55c6c2375ef0 2024-03-30T02:46:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f223d970b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c237e850 con 0x55c6c23755d0 2024-03-30T02:46:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2223400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c2387a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2223400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c2387a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37668/0 (socket says 172.21.15.67:37668) 2024-03-30T02:46:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2223400700 1 -- 172.21.15.67:0/3333202033 learned_addr learned my addr 172.21.15.67:0/3333202033 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2222a00700 1 --2- 172.21.15.67:0/3333202033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 0x55c6c237e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2222000700 1 --2- 172.21.15.67:0/3333202033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2223400700 1 -- 172.21.15.67:0/3333202033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 msgr2=0x55c6c237e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2223400700 1 --2- 172.21.15.67:0/3333202033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 0x55c6c237e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2223400700 1 -- 172.21.15.67:0/3333202033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 msgr2=0x55c6c237d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2223400700 1 --2- 172.21.15.67:0/3333202033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.200+0000 7f2223400700 1 -- 172.21.15.67:0/3333202033 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c228dea0 con 0x55c6c2380df0 2024-03-30T02:46:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.201+0000 7f2223400700 1 --2- 172.21.15.67:0/3333202033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c2387a10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f221c015000 tx=0x7f221c01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e329d926143af222 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.201+0000 7f2219200700 1 -- 172.21.15.67:0/3333202033 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c003a00 con 0x55c6c2380df0 2024-03-30T02:46:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.201+0000 7f2219200700 1 -- 172.21.15.67:0/3333202033 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221c003b60 con 0x55c6c2380df0 2024-03-30T02:46:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.201+0000 7f2219200700 1 -- 172.21.15.67:0/3333202033 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c005040 con 0x55c6c2380df0 2024-03-30T02:46:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 -- 172.21.15.67:0/3333202033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 msgr2=0x55c6c2387a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 --2- 172.21.15.67:0/3333202033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c2387a10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f221c015000 tx=0x7f221c01f040 comp rx=0 tx=0).stop 2024-03-30T02:46:03.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 -- 172.21.15.67:0/3333202033 shutdown_connections 2024-03-30T02:46:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 --2- 172.21.15.67:0/3333202033 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c2380df0 0x55c6c2387a10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 --2- 172.21.15.67:0/3333202033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c23755d0 0x55c6c237e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 --2- 172.21.15.67:0/3333202033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2375ef0 0x55c6c237d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 -- 172.21.15.67:0/3333202033 >> 172.21.15.67:0/3333202033 conn(0x55c6c22d56a0 msgr2=0x55c6c236a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 -- 172.21.15.67:0/3333202033 shutdown_connections 2024-03-30T02:46:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 -- 172.21.15.67:0/3333202033 wait complete. 2024-03-30T02:46:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 -- start start 2024-03-30T02:46:03.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.202+0000 7f223d970b80 1 Processor -- start 2024-03-30T02:46:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f223d970b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c2385be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f223d970b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 0x55c6c2386120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f223d970b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 0x55c6c23ec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f223d970b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c23ec650 con 0x55c6c23755d0 2024-03-30T02:46:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f223d970b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c23ec790 con 0x55c6c2380df0 2024-03-30T02:46:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f223d970b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c23ec8d0 con 0x55c6c2375ef0 2024-03-30T02:46:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f223d970b80 10 client.0 _ll_register_callbacks cb 0x55c6c226d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2222a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c2385be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2222a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c2385be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37670/0 (socket says 172.21.15.67:37670) 2024-03-30T02:46:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2222a00700 1 -- 172.21.15.67:0/3858743623 learned_addr learned my addr 172.21.15.67:0/3858743623 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2222000700 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 0x55c6c2386120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2223400700 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 0x55c6c23ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2222a00700 1 -- 172.21.15.67:0/3858743623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 msgr2=0x55c6c2386120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2222a00700 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 0x55c6c2386120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2222a00700 1 -- 172.21.15.67:0/3858743623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 msgr2=0x55c6c23ec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2222a00700 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 0x55c6c23ec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.203+0000 7f2222a00700 1 -- 172.21.15.67:0/3858743623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222410e470 con 0x55c6c23755d0 2024-03-30T02:46:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.204+0000 7f2222a00700 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c2385be0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f22140cd2c0 tx=0x7f22140cac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32b0eb4d2fcaae3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.204+0000 7f220cc00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.204+0000 7f220cc00700 1 -- 172.21.15.67:0/3858743623 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22140b82c0 con 0x55c6c23755d0 2024-03-30T02:46:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.204+0000 7f220cc00700 1 -- 172.21.15.67:0/3858743623 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2214099550 con 0x55c6c23755d0 2024-03-30T02:46:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.204+0000 7f223d970b80 1 -- 172.21.15.67:0/3858743623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f221409a060 con 0x55c6c23755d0 2024-03-30T02:46:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.204+0000 7f220cc00700 1 -- 172.21.15.67:0/3858743623 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22140b8420 con 0x55c6c23755d0 2024-03-30T02:46:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.204+0000 7f223d970b80 10 client.17277 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.204+0000 7f223d970b80 1 -- 172.21.15.67:0/3858743623 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c6c214dcd0 con 0x55c6c23755d0 2024-03-30T02:46:03.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.204+0000 7f220cc00700 1 -- 172.21.15.67:0/3858743623 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221409a060 con 0x55c6c23755d0 2024-03-30T02:46:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f220cc00700 1 -- 172.21.15.67:0/3858743623 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f22140a20c0 con 0x55c6c23755d0 2024-03-30T02:46:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f223d970b80 10 client.17277 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f223d970b80 10 client.17277 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:46:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f223d970b80 20 client.17277 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f223d970b80 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c6c23edc90 0x55c6c23f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f223d970b80 4 client.17277 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f223d970b80 1 -- 172.21.15.67:0/3858743623 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6c2116cf0 con 0x55c6c23edc90 2024-03-30T02:46:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f2222000700 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c6c23edc90 0x55c6c23f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f2222000700 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c6c23edc90 0x55c6c23f00b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=839c2c3375c573ee server_cookie=2d6f1c1f7d82f182 in_seq=0 out_seq=0 2024-03-30T02:46:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.205+0000 7f220cc00700 10 client.17277 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:46:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f220cc00700 1 -- 172.21.15.67:0/3858743623 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6c2116cf0 con 0x55c6c23edc90 2024-03-30T02:46:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f220cc00700 10 client.17277 handle_command_reply: tid=0 2024-03-30T02:46:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 client.17277 shutdown 2024-03-30T02:46:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 -- 172.21.15.67:0/3858743623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 msgr2=0x55c6c2385be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c2385be0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f22140cd2c0 tx=0x7f22140cac20 comp rx=0 tx=0).stop 2024-03-30T02:46:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 -- 172.21.15.67:0/3858743623 shutdown_connections 2024-03-30T02:46:03.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55c6c23edc90 0x55c6c23f00b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c6c23755d0 0x55c6c2385be0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f220cc00700 0 client.17277 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:46:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6c2380df0 0x55c6c23ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 --2- 172.21.15.67:0/3858743623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6c2375ef0 0x55c6c2386120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 -- 172.21.15.67:0/3858743623 >> 172.21.15.67:0/3858743623 conn(0x55c6c22d56a0 msgr2=0x55c6c2378e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 -- 172.21.15.67:0/3858743623 shutdown_connections 2024-03-30T02:46:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 1 -- 172.21.15.67:0/3858743623 wait complete. 2024-03-30T02:46:03.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.206+0000 7f223d970b80 20 client.17277 trim_cache size 0 max 16384 2024-03-30T02:46:03.215 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 -- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f22080a1290 msgr2=0x7f22080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f22080a1290 0x7f22080a3750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f221c0062a0 tx=0x7f221c0095c0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 -- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224100f60 msgr2=0x7f22240730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224100f60 0x7f22240730b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9a0 tx=0x7f221400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 -- 172.21.15.67:0/1015260725 shutdown_connections 2024-03-30T02:46:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f22080a1290 0x7f22080a3750 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2224100f60 0x7f22240730b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224073b30 0x7f22241a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 --2- 172.21.15.67:0/1015260725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2224106080 0x7f22240735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 -- 172.21.15.67:0/1015260725 >> 172.21.15.67:0/1015260725 conn(0x7f22240fc7c0 msgr2=0x7f22240fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 -- 172.21.15.67:0/1015260725 shutdown_connections 2024-03-30T02:46:03.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.207+0000 7f2229e00700 1 -- 172.21.15.67:0/1015260725 wait complete. 2024-03-30T02:46:03.226 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:46:03.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.429+0000 7efebd000700 1 -- 172.21.15.67:0/414050140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeb8073180 msgr2=0x7efeb8073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.429+0000 7efebd000700 1 --2- 172.21.15.67:0/414050140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeb8073180 0x7efeb8073560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efea8009990 tx=0x7efea8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.429+0000 7efebd000700 1 -- 172.21.15.67:0/414050140 shutdown_connections 2024-03-30T02:46:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.429+0000 7efebd000700 1 --2- 172.21.15.67:0/414050140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb810d580 0x7efeb810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.429+0000 7efebd000700 1 --2- 172.21.15.67:0/414050140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeb8073180 0x7efeb8073560 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.429+0000 7efebd000700 1 --2- 172.21.15.67:0/414050140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb8073aa0 0x7efeb810d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.429+0000 7efebd000700 1 -- 172.21.15.67:0/414050140 >> 172.21.15.67:0/414050140 conn(0x7efeb80fc7f0 msgr2=0x7efeb80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.429+0000 7efebd000700 1 -- 172.21.15.67:0/414050140 shutdown_connections 2024-03-30T02:46:03.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.429+0000 7efebd000700 1 -- 172.21.15.67:0/414050140 wait complete. 2024-03-30T02:46:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.430+0000 7efebd000700 1 Processor -- start 2024-03-30T02:46:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efebd000700 1 -- start start 2024-03-30T02:46:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efebd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeb8073180 0x7efeb8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efebd000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb8073aa0 0x7efeb8199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efebd000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810d580 0x7efeb819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efebd000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb819e360 con 0x7efeb810d580 2024-03-30T02:46:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efebd000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb819e4d0 con 0x7efeb8073180 2024-03-30T02:46:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efebd000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb819e690 con 0x7efeb8073aa0 2024-03-30T02:46:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efeb6800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810d580 0x7efeb819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efeb5e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeb8073180 0x7efeb8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efeb6800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810d580 0x7efeb819dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37690/0 (socket says 172.21.15.67:37690) 2024-03-30T02:46:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efeb6800700 1 -- 172.21.15.67:0/3520049372 learned_addr learned my addr 172.21.15.67:0/3520049372 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.431+0000 7efeb5400700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb8073aa0 0x7efeb8199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeb6800700 1 -- 172.21.15.67:0/3520049372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb8073aa0 msgr2=0x7efeb8199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeb6800700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb8073aa0 0x7efeb8199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeb6800700 1 -- 172.21.15.67:0/3520049372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeb8073180 msgr2=0x7efeb8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeb6800700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeb8073180 0x7efeb8199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeb6800700 1 -- 172.21.15.67:0/3520049372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea80095f0 con 0x7efeb810d580 2024-03-30T02:46:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeb5e00700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeb8073180 0x7efeb8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeb6800700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810d580 0x7efeb819dd20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efeb000ef90 tx=0x7efeb000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeaea00700 1 -- 172.21.15.67:0/3520049372 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0018070 con 0x7efeb810d580 2024-03-30T02:46:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeaea00700 1 -- 172.21.15.67:0/3520049372 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb000f040 con 0x7efeb810d580 2024-03-30T02:46:03.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efebd000700 1 -- 172.21.15.67:0/3520049372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb819e910 con 0x7efeb810d580 2024-03-30T02:46:03.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efeaea00700 1 -- 172.21.15.67:0/3520049372 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb000cda0 con 0x7efeb810d580 2024-03-30T02:46:03.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.432+0000 7efebd000700 1 -- 172.21.15.67:0/3520049372 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb81a61b0 con 0x7efeb810d580 2024-03-30T02:46:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.436+0000 7efeaea00700 1 -- 172.21.15.67:0/3520049372 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7efeb0003a00 con 0x7efeb810d580 2024-03-30T02:46:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.437+0000 7efed0a90b80 1 Processor -- start 2024-03-30T02:46:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.437+0000 7efeaea00700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efe9c0a1290 0x7efe9c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.437+0000 7efeaea00700 1 -- 172.21.15.67:0/3520049372 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7efeb00ccbe0 con 0x7efeb810d580 2024-03-30T02:46:03.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.437+0000 7efed0a90b80 1 -- start start 2024-03-30T02:46:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.437+0000 7efeb5400700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efe9c0a1290 0x7efe9c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.437+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.437+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49b3ef0 0x55e0b49be8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.437+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49bedf0 0x55e0b49c11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efed0a90b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b490c990 con 0x55e0b49b35d0 2024-03-30T02:46:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efed0a90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b490a350 con 0x55e0b49b3ef0 2024-03-30T02:46:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efed0a90b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b4905c40 con 0x55e0b49bedf0 2024-03-30T02:46:03.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37700/0 (socket says 172.21.15.67:37700) 2024-03-30T02:46:03.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5e00700 1 -- 172.21.15.67:0/3003046540 learned_addr learned my addr 172.21.15.67:0/3003046540 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb6800700 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49bedf0 0x55e0b49c11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5400700 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49b3ef0 0x55e0b49be8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5400700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efe9c0a1290 0x7efe9c0a3750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efeb819a990 tx=0x7efea4009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5e00700 1 -- 172.21.15.67:0/3003046540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49bedf0 msgr2=0x55e0b49c11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5e00700 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49bedf0 0x55e0b49c11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5e00700 1 -- 172.21.15.67:0/3003046540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49b3ef0 msgr2=0x55e0b49be8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5e00700 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49b3ef0 0x55e0b49be8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5e00700 1 -- 172.21.15.67:0/3003046540 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b47873b0 con 0x55e0b49b35d0 2024-03-30T02:46:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb5400700 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49b3ef0 0x55e0b49be8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.438+0000 7efeb6800700 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49bedf0 0x55e0b49c11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.439+0000 7efeb5e00700 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49b39b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7efea8005f00 tx=0x7efea800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26de3f07f67ffb7b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.439+0000 7efea3400700 1 -- 172.21.15.67:0/3003046540 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea801d070 con 0x55e0b49b35d0 2024-03-30T02:46:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.439+0000 7efea3400700 1 -- 172.21.15.67:0/3003046540 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea801e040 con 0x55e0b49b35d0 2024-03-30T02:46:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.439+0000 7efea3400700 1 -- 172.21.15.67:0/3003046540 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8004c50 con 0x55e0b49b35d0 2024-03-30T02:46:03.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 -- 172.21.15.67:0/3003046540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 msgr2=0x55e0b49b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49b39b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7efea8005f00 tx=0x7efea800b610 comp rx=0 tx=0).stop 2024-03-30T02:46:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 -- 172.21.15.67:0/3003046540 shutdown_connections 2024-03-30T02:46:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49b39b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49bedf0 0x55e0b49c11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 --2- 172.21.15.67:0/3003046540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49b3ef0 0x55e0b49be8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 -- 172.21.15.67:0/3003046540 >> 172.21.15.67:0/3003046540 conn(0x55e0b49136a0 msgr2=0x55e0b49afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 -- 172.21.15.67:0/3003046540 shutdown_connections 2024-03-30T02:46:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 -- 172.21.15.67:0/3003046540 wait complete. 2024-03-30T02:46:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 -- start start 2024-03-30T02:46:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.440+0000 7efed0a90b80 1 Processor -- start 2024-03-30T02:46:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 0x55e0b49bba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efed0a90b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b48a1d80 con 0x55e0b49b35d0 2024-03-30T02:46:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efed0a90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b48a1a20 con 0x55e0b49bedf0 2024-03-30T02:46:03.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efed0a90b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b47896a0 con 0x55e0b49b3ef0 2024-03-30T02:46:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efed0a90b80 10 client.0 _ll_register_callbacks cb 0x55e0b48ab470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efeb5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efeb6800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efeb6800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49bbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49466/0 (socket says 172.21.15.67:49466) 2024-03-30T02:46:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efeb6800700 1 -- 172.21.15.67:0/764753020 learned_addr learned my addr 172.21.15.67:0/764753020 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.441+0000 7efeb5400700 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 0x55e0b49bba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efeb6800700 1 -- 172.21.15.67:0/764753020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 msgr2=0x55e0b49bba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efeb6800700 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 0x55e0b49bba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efeb6800700 1 -- 172.21.15.67:0/764753020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 msgr2=0x55e0b49be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efeb6800700 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49be750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efeb6800700 1 -- 172.21.15.67:0/764753020 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb81a61b0 con 0x55e0b49bedf0 2024-03-30T02:46:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efeb6800700 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49bbfc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c6f0 tx=0x7efeb0099540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a5b644922be5f80 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efe97e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efe97e00700 1 -- 172.21.15.67:0/764753020 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb009a080 con 0x55e0b49bedf0 2024-03-30T02:46:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efe97e00700 1 -- 172.21.15.67:0/764753020 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7efeb0099980 con 0x55e0b49bedf0 2024-03-30T02:46:03.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.442+0000 7efed0a90b80 1 -- 172.21.15.67:0/764753020 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0b48c6380 con 0x55e0b49bedf0 2024-03-30T02:46:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efe97e00700 1 -- 172.21.15.67:0/764753020 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeb009d7b0 con 0x55e0b49bedf0 2024-03-30T02:46:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efed0a90b80 10 client.27428 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efed0a90b80 1 -- 172.21.15.67:0/764753020 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7efeb81106c0 con 0x55e0b49bedf0 2024-03-30T02:46:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efe97e00700 1 -- 172.21.15.67:0/764753020 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e0b48c6380 con 0x55e0b49bedf0 2024-03-30T02:46:03.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efe97e00700 1 -- 172.21.15.67:0/764753020 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efeb0098f70 con 0x55e0b49bedf0 2024-03-30T02:46:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efed0a90b80 10 client.27428 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efed0a90b80 10 client.27428 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:46:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efed0a90b80 20 client.27428 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efed0a90b80 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e0b49ce850 0x55e0b49d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efed0a90b80 4 client.27428 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.446+0000 7efed0a90b80 1 -- 172.21.15.67:0/764753020 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0b48fe700 con 0x55e0b49ce850 2024-03-30T02:46:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.447+0000 7efeb5e00700 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e0b49ce850 0x55e0b49d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.447+0000 7efe97e00700 10 client.27428 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:46:03.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.447+0000 7efeb5e00700 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e0b49ce850 0x55e0b49d0c70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=deed8657fc6b5298 server_cookie=2fafef6d5060f25e in_seq=0 out_seq=0 2024-03-30T02:46:03.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efe97e00700 1 -- 172.21.15.67:0/764753020 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e0b48fe700 con 0x55e0b49ce850 2024-03-30T02:46:03.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efe97e00700 10 client.27428 handle_command_reply: tid=0 2024-03-30T02:46:03.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 client.27428 shutdown 2024-03-30T02:46:03.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 -- 172.21.15.67:0/764753020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 msgr2=0x55e0b49bbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49bbfc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c6f0 tx=0x7efeb0099540 comp rx=0 tx=0).stop 2024-03-30T02:46:03.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 -- 172.21.15.67:0/764753020 shutdown_connections 2024-03-30T02:46:03.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e0b49ce850 0x55e0b49d0c70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efe97e00700 0 client.27428 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:46:03.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49bbfc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 --2- 172.21.15.67:0/764753020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 0x55e0b49bba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 -- 172.21.15.67:0/764753020 >> 172.21.15.67:0/764753020 conn(0x55e0b49136a0 msgr2=0x55e0b49b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.448+0000 7efed0a90b80 1 -- 172.21.15.67:0/764753020 shutdown_connections 2024-03-30T02:46:03.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.449+0000 7efed0a90b80 1 -- 172.21.15.67:0/764753020 wait complete. 2024-03-30T02:46:03.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.449+0000 7efed0a90b80 20 client.27428 trim_cache size 0 max 16384 2024-03-30T02:46:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.462+0000 7efed0a90b80 1 Processor -- start 2024-03-30T02:46:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.462+0000 7efed0a90b80 1 -- start start 2024-03-30T02:46:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 0x55e0b49ba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efed0a90b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b49b4850 con 0x55e0b49b35d0 2024-03-30T02:46:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efed0a90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b49d1000 con 0x55e0b49bedf0 2024-03-30T02:46:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efed0a90b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b49bad90 con 0x55e0b49b3ef0 2024-03-30T02:46:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37722/0 (socket says 172.21.15.67:37722) 2024-03-30T02:46:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5e00700 1 -- 172.21.15.67:0/2740783546 learned_addr learned my addr 172.21.15.67:0/2740783546 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5400700 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 0x55e0b49ba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb6800700 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5e00700 1 -- 172.21.15.67:0/2740783546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 msgr2=0x55e0b49ba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5e00700 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 0x55e0b49ba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5e00700 1 -- 172.21.15.67:0/2740783546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 msgr2=0x55e0b49ba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5e00700 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49ba850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5e00700 1 -- 172.21.15.67:0/2740783546 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b4733b60 con 0x55e0b49b35d0 2024-03-30T02:46:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.463+0000 7efeb5400700 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 0x55e0b49ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.464+0000 7efeb5e00700 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5a10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efea800b6f0 tx=0x7efea800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fa10329e9548d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.464+0000 7efea3e00700 1 -- 172.21.15.67:0/2740783546 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea801d040 con 0x55e0b49b35d0 2024-03-30T02:46:03.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.464+0000 7efea3e00700 1 -- 172.21.15.67:0/2740783546 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea8034cd0 con 0x55e0b49b35d0 2024-03-30T02:46:03.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.464+0000 7efea3e00700 1 -- 172.21.15.67:0/2740783546 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80055a0 con 0x55e0b49b35d0 2024-03-30T02:46:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 -- 172.21.15.67:0/2740783546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 msgr2=0x55e0b49c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5a10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efea800b6f0 tx=0x7efea800b260 comp rx=0 tx=0).stop 2024-03-30T02:46:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 -- 172.21.15.67:0/2740783546 shutdown_connections 2024-03-30T02:46:03.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5a10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2740783546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b3ef0 0x55e0b49ba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 -- 172.21.15.67:0/2740783546 >> 172.21.15.67:0/2740783546 conn(0x55e0b49136a0 msgr2=0x55e0b49a85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 -- 172.21.15.67:0/2740783546 shutdown_connections 2024-03-30T02:46:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 -- 172.21.15.67:0/2740783546 wait complete. 2024-03-30T02:46:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 -- start start 2024-03-30T02:46:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.465+0000 7efed0a90b80 1 Processor -- start 2024-03-30T02:46:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b3ef0 0x55e0b49c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efed0a90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49c4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efed0a90b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b49c45a0 con 0x55e0b49b3ef0 2024-03-30T02:46:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efed0a90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b49c4710 con 0x55e0b49bedf0 2024-03-30T02:46:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efed0a90b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b49c48d0 con 0x55e0b49b35d0 2024-03-30T02:46:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efed0a90b80 10 client.0 _ll_register_callbacks cb 0x55e0b48ab470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efeb5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efeb5400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b3ef0 0x55e0b49c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efeb6800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efeb5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56270/0 (socket says 172.21.15.67:56270) 2024-03-30T02:46:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.466+0000 7efeb5e00700 1 -- 172.21.15.67:0/2213018694 learned_addr learned my addr 172.21.15.67:0/2213018694 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efeb5e00700 1 -- 172.21.15.67:0/2213018694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 msgr2=0x55e0b49c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efeb5e00700 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efeb5e00700 1 -- 172.21.15.67:0/2213018694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b3ef0 msgr2=0x55e0b49c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efeb5e00700 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b3ef0 0x55e0b49c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efeb5e00700 1 -- 172.21.15.67:0/2213018694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0b47873b0 con 0x55e0b49b35d0 2024-03-30T02:46:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efeb6800700 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:03.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efeb5e00700 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efea8009590 tx=0x7efea800bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7332415403f69cb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efe97e00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:46:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efe97e00700 1 -- 172.21.15.67:0/2213018694 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea801d070 con 0x55e0b49b35d0 2024-03-30T02:46:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efed0a90b80 1 -- 172.21.15.67:0/2213018694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efea801e040 con 0x55e0b49b35d0 2024-03-30T02:46:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efe97e00700 1 -- 172.21.15.67:0/2213018694 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7efea8005040 con 0x55e0b49b35d0 2024-03-30T02:46:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efe97e00700 1 -- 172.21.15.67:0/2213018694 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea8005200 con 0x55e0b49b35d0 2024-03-30T02:46:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efed0a90b80 10 client.27370 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.467+0000 7efed0a90b80 1 -- 172.21.15.67:0/2213018694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e0b48cbea0 con 0x55e0b49b35d0 2024-03-30T02:46:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.468+0000 7efe97e00700 1 -- 172.21.15.67:0/2213018694 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea801e040 con 0x55e0b49b35d0 2024-03-30T02:46:03.475 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.468+0000 7efe97e00700 1 -- 172.21.15.67:0/2213018694 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efea803b070 con 0x55e0b49b35d0 2024-03-30T02:46:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.468+0000 7efed0a90b80 10 client.27370 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.468+0000 7efed0a90b80 10 client.27370 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:46:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.468+0000 7efed0a90b80 20 client.27370 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.468+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e0b4a23670 0x55e0b4a25a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.468+0000 7efed0a90b80 4 client.27370 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.468+0000 7efed0a90b80 1 -- 172.21.15.67:0/2213018694 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e0b4754cf0 con 0x55e0b4a23670 2024-03-30T02:46:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.468+0000 7efeb5400700 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e0b4a23670 0x55e0b4a25a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efeb5400700 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e0b4a23670 0x55e0b4a25a90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=172594cdec02c6f1 server_cookie=5d0614288013f339 in_seq=0 out_seq=0 2024-03-30T02:46:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efe97e00700 10 client.27370 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:46:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efe97e00700 1 -- 172.21.15.67:0/2213018694 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e0b4754cf0 con 0x55e0b4a23670 2024-03-30T02:46:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efe97e00700 10 client.27370 handle_command_reply: tid=0 2024-03-30T02:46:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 client.27370 shutdown 2024-03-30T02:46:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 -- 172.21.15.67:0/2213018694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b35d0 msgr2=0x55e0b49c5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efea8009590 tx=0x7efea800bde0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 -- 172.21.15.67:0/2213018694 shutdown_connections 2024-03-30T02:46:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b49bedf0 0x55e0b49c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0b49b35d0 0x55e0b49c5730 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x55e0b4a23670 0x55e0b4a25a90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 --2- 172.21.15.67:0/2213018694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e0b49b3ef0 0x55e0b49c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 -- 172.21.15.67:0/2213018694 >> 172.21.15.67:0/2213018694 conn(0x55e0b49136a0 msgr2=0x55e0b49b6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efe97e00700 0 client.27370 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:46:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 -- 172.21.15.67:0/2213018694 shutdown_connections 2024-03-30T02:46:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.469+0000 7efed0a90b80 1 -- 172.21.15.67:0/2213018694 wait complete. 2024-03-30T02:46:03.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efed0a90b80 20 client.27370 trim_cache size 0 max 16384 2024-03-30T02:46:03.479 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:03.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 -- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efe9c0a1290 msgr2=0x7efe9c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efe9c0a1290 0x7efe9c0a3750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efeb819a990 tx=0x7efea4009260 comp rx=0 tx=0).stop 2024-03-30T02:46:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 -- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810d580 msgr2=0x7efeb819dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810d580 0x7efeb819dd20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efeb000ef90 tx=0x7efeb000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 -- 172.21.15.67:0/3520049372 shutdown_connections 2024-03-30T02:46:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7efe9c0a1290 0x7efe9c0a3750 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efeb810d580 0x7efeb819dd20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efeb8073180 0x7efeb8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 --2- 172.21.15.67:0/3520049372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb8073aa0 0x7efeb8199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 -- 172.21.15.67:0/3520049372 >> 172.21.15.67:0/3520049372 conn(0x7efeb80fc7f0 msgr2=0x7efeb80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 -- 172.21.15.67:0/3520049372 shutdown_connections 2024-03-30T02:46:03.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.470+0000 7efebd000700 1 -- 172.21.15.67:0/3520049372 wait complete. 2024-03-30T02:46:03.489 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:46:03.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.690+0000 7ffa01a00700 1 -- 172.21.15.67:0/949894908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9fc101920 msgr2=0x7ff9fc105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.690+0000 7ffa01a00700 1 --2- 172.21.15.67:0/949894908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9fc101920 0x7ff9fc105960 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009990 tx=0x7ff9e8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.690+0000 7ffa01a00700 1 -- 172.21.15.67:0/949894908 shutdown_connections 2024-03-30T02:46:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.690+0000 7ffa01a00700 1 --2- 172.21.15.67:0/949894908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9fc106090 0x7ff9fc10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.690+0000 7ffa01a00700 1 --2- 172.21.15.67:0/949894908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9fc100f70 0x7ff9fc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.690+0000 7ffa01a00700 1 --2- 172.21.15.67:0/949894908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9fc101920 0x7ff9fc105960 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.690+0000 7ffa01a00700 1 -- 172.21.15.67:0/949894908 >> 172.21.15.67:0/949894908 conn(0x7ff9fc0fc7f0 msgr2=0x7ff9fc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.690+0000 7ffa01a00700 1 -- 172.21.15.67:0/949894908 shutdown_connections 2024-03-30T02:46:03.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.690+0000 7ffa01a00700 1 -- 172.21.15.67:0/949894908 wait complete. 2024-03-30T02:46:03.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.691+0000 7ffa01a00700 1 Processor -- start 2024-03-30T02:46:03.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.691+0000 7ffa01a00700 1 -- start start 2024-03-30T02:46:03.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.691+0000 7ffa01a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9fc100f70 0x7ff9fc0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.691+0000 7ffa01a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9fc106090 0x7ff9fc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.691+0000 7ffa01a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9fc073b30 0x7ff9fc1a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.691+0000 7ffa01a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1a5990 con 0x7ff9fc106090 2024-03-30T02:46:03.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.691+0000 7ffa01a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1a5b00 con 0x7ff9fc073b30 2024-03-30T02:46:03.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.691+0000 7ffa01a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1a5cc0 con 0x7ff9fc100f70 2024-03-30T02:46:03.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fa000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9fc106090 0x7ff9fc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fb400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9fc073b30 0x7ff9fc1a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fa000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9fc106090 0x7ff9fc0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37760/0 (socket says 172.21.15.67:37760) 2024-03-30T02:46:03.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fa000700 1 -- 172.21.15.67:0/1501312059 learned_addr learned my addr 172.21.15.67:0/1501312059 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9faa00700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9fc100f70 0x7ff9fc0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fa000700 1 -- 172.21.15.67:0/1501312059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9fc100f70 msgr2=0x7ff9fc0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fa000700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9fc100f70 0x7ff9fc0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fa000700 1 -- 172.21.15.67:0/1501312059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9fc073b30 msgr2=0x7ff9fc1a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fa000700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9fc073b30 0x7ff9fc1a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fa000700 1 -- 172.21.15.67:0/1501312059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e80095f0 con 0x7ff9fc106090 2024-03-30T02:46:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9faa00700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9fc100f70 0x7ff9fc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.692+0000 7ff9fa000700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9fc106090 0x7ff9fc0735f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8000c00 tx=0x7ff9e8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.693+0000 7ff9f3400700 1 -- 172.21.15.67:0/1501312059 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e801d070 con 0x7ff9fc106090 2024-03-30T02:46:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.693+0000 7ff9f3400700 1 -- 172.21.15.67:0/1501312059 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9e8022470 con 0x7ff9fc106090 2024-03-30T02:46:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.693+0000 7ffa01a00700 1 -- 172.21.15.67:0/1501312059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc1a5f40 con 0x7ff9fc106090 2024-03-30T02:46:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.693+0000 7ff9f3400700 1 -- 172.21.15.67:0/1501312059 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8017650 con 0x7ff9fc106090 2024-03-30T02:46:03.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.693+0000 7ffa01a00700 1 -- 172.21.15.67:0/1501312059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc1a6370 con 0x7ff9fc106090 2024-03-30T02:46:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.696+0000 7ff9f3400700 1 -- 172.21.15.67:0/1501312059 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ff9e800fad0 con 0x7ff9fc106090 2024-03-30T02:46:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.697+0000 7ffa155a4b80 1 Processor -- start 2024-03-30T02:46:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.697+0000 7ff9f3400700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9e00a1310 0x7ff9e00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.697+0000 7ffa155a4b80 1 -- start start 2024-03-30T02:46:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.697+0000 7ff9f3400700 1 -- 172.21.15.67:0/1501312059 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7ff9e80ce270 con 0x7ff9fc106090 2024-03-30T02:46:03.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.697+0000 7ff9fb400700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9e00a1310 0x7ff9e00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcc29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfccd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfccddf0 0x55c4dfcd01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfc1b990 con 0x55c4dfccddf0 2024-03-30T02:46:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfc19350 con 0x55c4dfcc2ef0 2024-03-30T02:46:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfc14c40 con 0x55c4dfcc25d0 2024-03-30T02:46:03.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfccddf0 0x55c4dfcd01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfccddf0 0x55c4dfcd01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37762/0 (socket says 172.21.15.67:37762) 2024-03-30T02:46:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fa000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfccd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fb400700 1 -- 172.21.15.67:0/3841807046 learned_addr learned my addr 172.21.15.67:0/3841807046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9faa00700 1 --2- 172.21.15.67:0/3841807046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcc29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fb400700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9e00a1310 0x7ff9e00a37d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400f590 tx=0x7ff9f4009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fa000700 1 -- 172.21.15.67:0/3841807046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc25d0 msgr2=0x55c4dfcc29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fa000700 1 --2- 172.21.15.67:0/3841807046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcc29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fa000700 1 -- 172.21.15.67:0/3841807046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfccddf0 msgr2=0x55c4dfcd01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fa000700 1 --2- 172.21.15.67:0/3841807046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfccddf0 0x55c4dfcd01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fa000700 1 -- 172.21.15.67:0/3841807046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4dfa963b0 con 0x55c4dfcc2ef0 2024-03-30T02:46:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.698+0000 7ff9fb400700 1 --2- 172.21.15.67:0/3841807046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfccddf0 0x55c4dfcd01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.699+0000 7ff9fa000700 1 --2- 172.21.15.67:0/3841807046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfccd8b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80cb5c0 tx=0x7ff9e8022c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e8f920b0d3cdd4c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.699+0000 7ff9e7e00700 1 -- 172.21.15.67:0/3841807046 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e809aa90 con 0x55c4dfcc2ef0 2024-03-30T02:46:03.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.699+0000 7ff9e7e00700 1 -- 172.21.15.67:0/3841807046 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9e809d070 con 0x55c4dfcc2ef0 2024-03-30T02:46:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.699+0000 7ff9e7e00700 1 -- 172.21.15.67:0/3841807046 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80ce020 con 0x55c4dfcc2ef0 2024-03-30T02:46:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.699+0000 7ffa155a4b80 1 -- 172.21.15.67:0/3841807046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 msgr2=0x55c4dfccd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.699+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/3841807046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfccd8b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80cb5c0 tx=0x7ff9e8022c20 comp rx=0 tx=0).stop 2024-03-30T02:46:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.700+0000 7ffa155a4b80 1 -- 172.21.15.67:0/3841807046 shutdown_connections 2024-03-30T02:46:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.700+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/3841807046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfccddf0 0x55c4dfcd01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.700+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/3841807046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcc29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.700+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/3841807046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfccd8b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.700+0000 7ffa155a4b80 1 -- 172.21.15.67:0/3841807046 >> 172.21.15.67:0/3841807046 conn(0x55c4dfc226a0 msgr2=0x55c4dfcbea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.700+0000 7ffa155a4b80 1 -- 172.21.15.67:0/3841807046 shutdown_connections 2024-03-30T02:46:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.700+0000 7ffa155a4b80 1 -- 172.21.15.67:0/3841807046 wait complete. 2024-03-30T02:46:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.700+0000 7ffa155a4b80 1 -- start start 2024-03-30T02:46:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.700+0000 7ffa155a4b80 1 Processor -- start 2024-03-30T02:46:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfccab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfccddf0 0x55c4dfccb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfbb0d80 con 0x55c4dfcc25d0 2024-03-30T02:46:03.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfbb0a20 con 0x55c4dfccddf0 2024-03-30T02:46:03.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfa986a0 con 0x55c4dfcc2ef0 2024-03-30T02:46:03.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ffa155a4b80 10 client.0 _ll_register_callbacks cb 0x55c4dfbba470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:03.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ff9faa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ff9faa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37778/0 (socket says 172.21.15.67:37778) 2024-03-30T02:46:03.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ff9faa00700 1 -- 172.21.15.67:0/1630232721 learned_addr learned my addr 172.21.15.67:0/1630232721 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ff9fb400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfccddf0 0x55c4dfccb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.701+0000 7ff9fa000700 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfccab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9fb400700 1 -- 172.21.15.67:0/1630232721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc2ef0 msgr2=0x55c4dfccab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9fb400700 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfccab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9fb400700 1 -- 172.21.15.67:0/1630232721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 msgr2=0x55c4dfccd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9fb400700 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9fb400700 1 -- 172.21.15.67:0/1630232721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f4019460 con 0x55c4dfccddf0 2024-03-30T02:46:03.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9faa00700 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9fb400700 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfccddf0 0x55c4dfccb1c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40060a0 tx=0x7ff9f4004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8624d2039ecbac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9e4c00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1630232721 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400e040 con 0x55c4dfccddf0 2024-03-30T02:46:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1630232721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4dfbd5380 con 0x55c4dfccddf0 2024-03-30T02:46:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.702+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1630232721 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7ff9f4004bb0 con 0x55c4dfccddf0 2024-03-30T02:46:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1630232721 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9f4004d70 con 0x55c4dfccddf0 2024-03-30T02:46:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ffa155a4b80 10 client.27446 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:03.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1630232721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7ff9fc1a6370 con 0x55c4dfccddf0 2024-03-30T02:46:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1630232721 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c4dfbd5380 con 0x55c4dfccddf0 2024-03-30T02:46:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1630232721 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9f4022c20 con 0x55c4dfccddf0 2024-03-30T02:46:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ffa155a4b80 10 client.27446 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ffa155a4b80 10 client.27446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ffa155a4b80 20 client.27446 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c4dfcdd690 0x55c4dfcdfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ffa155a4b80 4 client.27446 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:03.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.703+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1630232721 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4dfc0d700 con 0x55c4dfcdd690 2024-03-30T02:46:03.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.704+0000 7ff9faa00700 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c4dfcdd690 0x55c4dfcdfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.704+0000 7ff9faa00700 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c4dfcdd690 0x55c4dfcdfab0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c18dfd579bc69c server_cookie=da503e302b083bf2 in_seq=0 out_seq=0 2024-03-30T02:46:03.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.704+0000 7ff9e4c00700 10 client.27446 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:03.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1630232721 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c4dfc0d700 con 0x55c4dfcdd690 2024-03-30T02:46:03.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ff9e4c00700 10 client.27446 handle_command_reply: tid=0 2024-03-30T02:46:03.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 client.27446 shutdown 2024-03-30T02:46:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1630232721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfccddf0 msgr2=0x55c4dfccb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfccddf0 0x55c4dfccb1c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40060a0 tx=0x7ff9f4004b40 comp rx=0 tx=0).stop 2024-03-30T02:46:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1630232721 shutdown_connections 2024-03-30T02:46:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c4dfcdd690 0x55c4dfcdfab0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfccddf0 0x55c4dfccb1c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ff9e4c00700 0 client.27446 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:03.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1630232721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfccab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1630232721 >> 172.21.15.67:0/1630232721 conn(0x55c4dfc226a0 msgr2=0x55c4dfcc4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1630232721 shutdown_connections 2024-03-30T02:46:03.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1630232721 wait complete. 2024-03-30T02:46:03.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.705+0000 7ffa155a4b80 20 client.27446 trim_cache size 0 max 16384 2024-03-30T02:46:03.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.719+0000 7ffa155a4b80 1 Processor -- start 2024-03-30T02:46:03.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.719+0000 7ffa155a4b80 1 -- start start 2024-03-30T02:46:03.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.719+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.719+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfcd47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.719+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 0x55c4dfcc94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfcc35b0 con 0x55c4dfcc25d0 2024-03-30T02:46:03.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfccbaa0 con 0x55c4dfcc2ef0 2024-03-30T02:46:03.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfcd4d20 con 0x55c4dfccddf0 2024-03-30T02:46:03.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9faa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9faa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37782/0 (socket says 172.21.15.67:37782) 2024-03-30T02:46:03.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9faa00700 1 -- 172.21.15.67:0/150347221 learned_addr learned my addr 172.21.15.67:0/150347221 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9fa000700 1 --2- 172.21.15.67:0/150347221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfcd47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9fb400700 1 --2- 172.21.15.67:0/150347221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 0x55c4dfcc94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9faa00700 1 -- 172.21.15.67:0/150347221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 msgr2=0x55c4dfcc94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9faa00700 1 --2- 172.21.15.67:0/150347221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 0x55c4dfcc94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9faa00700 1 -- 172.21.15.67:0/150347221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 msgr2=0x55c4dfcd47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9faa00700 1 --2- 172.21.15.67:0/150347221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfcd47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9faa00700 1 -- 172.21.15.67:0/150347221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4dfa42b60 con 0x55c4dfcc25d0 2024-03-30T02:46:03.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.720+0000 7ff9faa00700 1 --2- 172.21.15.67:0/150347221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccb120 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55c4dfce0390 tx=0x7ff9ec0043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ab44e305c5e3d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ff9f0c00700 1 -- 172.21.15.67:0/150347221 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec01d3c0 con 0x55c4dfcc25d0 2024-03-30T02:46:03.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ff9f0c00700 1 -- 172.21.15.67:0/150347221 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9ec018040 con 0x55c4dfcc25d0 2024-03-30T02:46:03.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ff9f0c00700 1 -- 172.21.15.67:0/150347221 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec004a90 con 0x55c4dfcc25d0 2024-03-30T02:46:03.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ffa155a4b80 1 -- 172.21.15.67:0/150347221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 msgr2=0x55c4dfccb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/150347221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccb120 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55c4dfce0390 tx=0x7ff9ec0043d0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ffa155a4b80 1 -- 172.21.15.67:0/150347221 shutdown_connections 2024-03-30T02:46:03.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/150347221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfccb120 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/150347221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 0x55c4dfcc94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/150347221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfcd47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ffa155a4b80 1 -- 172.21.15.67:0/150347221 >> 172.21.15.67:0/150347221 conn(0x55c4dfc226a0 msgr2=0x55c4dfcb75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ffa155a4b80 1 -- 172.21.15.67:0/150347221 shutdown_connections 2024-03-30T02:46:03.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.721+0000 7ffa155a4b80 1 -- 172.21.15.67:0/150347221 wait complete. 2024-03-30T02:46:03.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.722+0000 7ffa155a4b80 1 -- start start 2024-03-30T02:46:03.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.722+0000 7ffa155a4b80 1 Processor -- start 2024-03-30T02:46:03.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.722+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcd29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.722+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfcd2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ffa155a4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 0x55c4dfd304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfd30a00 con 0x55c4dfcc25d0 2024-03-30T02:46:03.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfd30b40 con 0x55c4dfcc2ef0 2024-03-30T02:46:03.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ffa155a4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4dfd30cd0 con 0x55c4dfccddf0 2024-03-30T02:46:03.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ffa155a4b80 10 client.0 _ll_register_callbacks cb 0x55c4dfbba470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:03.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9faa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcd29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9fa000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfcd2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9faa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcd29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37788/0 (socket says 172.21.15.67:37788) 2024-03-30T02:46:03.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9faa00700 1 -- 172.21.15.67:0/1784990420 learned_addr learned my addr 172.21.15.67:0/1784990420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:03.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9fb400700 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 0x55c4dfd304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.730 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9faa00700 1 -- 172.21.15.67:0/1784990420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 msgr2=0x55c4dfd304c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9faa00700 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 0x55c4dfd304c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9faa00700 1 -- 172.21.15.67:0/1784990420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 msgr2=0x55c4dfcd2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:03.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9faa00700 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfcd2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.723+0000 7ff9faa00700 1 -- 172.21.15.67:0/1784990420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec0121d0 con 0x55c4dfcc25d0 2024-03-30T02:46:03.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.724+0000 7ff9faa00700 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcd29b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec01d3c0 tx=0x7ff9ec01d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0619abfce6a7226 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:03.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.724+0000 7ff9e4c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:03.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.724+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1784990420 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec01db20 con 0x55c4dfcc25d0 2024-03-30T02:46:03.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.724+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1784990420 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7ff9ec01dc80 con 0x55c4dfcc25d0 2024-03-30T02:46:03.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.724+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1784990420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9e809d070 con 0x55c4dfcc25d0 2024-03-30T02:46:03.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.724+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1784990420 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9ec01de40 con 0x55c4dfcc25d0 2024-03-30T02:46:03.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.724+0000 7ffa155a4b80 10 client.17313 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:03.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.724+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1784990420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c4dfbdaea0 con 0x55c4dfcc25d0 2024-03-30T02:46:03.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.724+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1784990420 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9e809d070 con 0x55c4dfcc25d0 2024-03-30T02:46:03.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1784990420 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9ec004b40 con 0x55c4dfcc25d0 2024-03-30T02:46:03.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ffa155a4b80 10 client.17313 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:03.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ffa155a4b80 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:03.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ffa155a4b80 20 client.17313 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:03.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c4dfd323b0 0x55c4dfd347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:03.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ffa155a4b80 4 client.17313 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:03.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1784990420 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c4dfa63cf0 con 0x55c4dfd323b0 2024-03-30T02:46:03.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ff9fa000700 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c4dfd323b0 0x55c4dfd347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:03.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ff9fa000700 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c4dfd323b0 0x55c4dfd347d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3732c016ed29442 server_cookie=e504a18292003446 in_seq=0 out_seq=0 2024-03-30T02:46:03.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.725+0000 7ff9e4c00700 10 client.17313 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:03.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ff9e4c00700 1 -- 172.21.15.67:0/1784990420 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c4dfa63cf0 con 0x55c4dfd323b0 2024-03-30T02:46:03.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ff9e4c00700 10 client.17313 handle_command_reply: tid=0 2024-03-30T02:46:03.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 client.17313 shutdown 2024-03-30T02:46:03.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1784990420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 msgr2=0x55c4dfcd29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcd29b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec01d3c0 tx=0x7ff9ec01d3f0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1784990420 shutdown_connections 2024-03-30T02:46:03.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55c4dfd323b0 0x55c4dfd347d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c4dfcc25d0 0x55c4dfcd29b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c4dfccddf0 0x55c4dfd304c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 --2- 172.21.15.67:0/1784990420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4dfcc2ef0 0x55c4dfcd2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1784990420 >> 172.21.15.67:0/1784990420 conn(0x55c4dfc226a0 msgr2=0x55c4dfcc9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ff9e4c00700 0 client.17313 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:03.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1784990420 shutdown_connections 2024-03-30T02:46:03.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 1 -- 172.21.15.67:0/1784990420 wait complete. 2024-03-30T02:46:03.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.726+0000 7ffa155a4b80 20 client.17313 trim_cache size 0 max 16384 2024-03-30T02:46:03.736 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:03.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 -- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9e00a1310 msgr2=0x7ff9e00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9e00a1310 0x7ff9e00a37d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400f590 tx=0x7ff9f4009920 comp rx=0 tx=0).stop 2024-03-30T02:46:03.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 -- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9fc106090 msgr2=0x7ff9fc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:03.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9fc106090 0x7ff9fc0735f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8000c00 tx=0x7ff9e8005740 comp rx=0 tx=0).stop 2024-03-30T02:46:03.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 -- 172.21.15.67:0/1501312059 shutdown_connections 2024-03-30T02:46:03.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ff9e00a1310 0x7ff9e00a37d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9fc073b30 0x7ff9fc1a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9fc100f70 0x7ff9fc0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 --2- 172.21.15.67:0/1501312059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9fc106090 0x7ff9fc0735f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:03.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 -- 172.21.15.67:0/1501312059 >> 172.21.15.67:0/1501312059 conn(0x7ff9fc0fc7f0 msgr2=0x7ff9fc0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:03.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 -- 172.21.15.67:0/1501312059 shutdown_connections 2024-03-30T02:46:03.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:03.727+0000 7ffa01a00700 1 -- 172.21.15.67:0/1501312059 wait complete. 2024-03-30T02:46:04.746 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:46:04.746 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:46:04.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:04 smithi082 ceph-mon[79176]: pgmap v277: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 88 KiB/s wr, 15 op/s 2024-03-30T02:46:04.811 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:04 smithi067 ceph-mon[37808]: pgmap v277: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 88 KiB/s wr, 15 op/s 2024-03-30T02:46:04.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:04 smithi170 ceph-mon[78798]: pgmap v277: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 88 KiB/s wr, 15 op/s 2024-03-30T02:46:04.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.962+0000 7fe605800700 1 -- 172.21.15.67:0/2796887270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe600106c10 msgr2=0x7fe600106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:04.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.962+0000 7fe605800700 1 --2- 172.21.15.67:0/2796887270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe600106c10 0x7fe600106ff0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00099c0 tx=0x7fe5f0009cd0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.962+0000 7fe605800700 1 -- 172.21.15.67:0/2796887270 shutdown_connections 2024-03-30T02:46:04.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.962+0000 7fe605800700 1 --2- 172.21.15.67:0/2796887270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe600101510 0x7fe60010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.962+0000 7fe605800700 1 --2- 172.21.15.67:0/2796887270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe600106c10 0x7fe600106ff0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.962+0000 7fe605800700 1 --2- 172.21.15.67:0/2796887270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe600100b50 0x7fe600100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.962+0000 7fe605800700 1 -- 172.21.15.67:0/2796887270 >> 172.21.15.67:0/2796887270 conn(0x7fe6000fc620 msgr2=0x7fe6000fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:04.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.963+0000 7fe605800700 1 -- 172.21.15.67:0/2796887270 shutdown_connections 2024-03-30T02:46:04.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.963+0000 7fe605800700 1 -- 172.21.15.67:0/2796887270 wait complete. 2024-03-30T02:46:04.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.964+0000 7fe605800700 1 Processor -- start 2024-03-30T02:46:04.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.964+0000 7fe605800700 1 -- start start 2024-03-30T02:46:04.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.964+0000 7fe605800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe600100b50 0x7fe6001991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.964+0000 7fe605800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe600101510 0x7fe600199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.964+0000 7fe605800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe600106c10 0x7fe60019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.964+0000 7fe605800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60019e0e0 con 0x7fe600101510 2024-03-30T02:46:04.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.964+0000 7fe605800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60019e250 con 0x7fe600106c10 2024-03-30T02:46:04.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.964+0000 7fe605800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60019e410 con 0x7fe600100b50 2024-03-30T02:46:04.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.964+0000 7fe5ff400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe600101510 0x7fe600199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5ffe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe600100b50 0x7fe6001991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5ff400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe600101510 0x7fe600199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37806/0 (socket says 172.21.15.67:37806) 2024-03-30T02:46:04.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5ff400700 1 -- 172.21.15.67:0/2749479683 learned_addr learned my addr 172.21.15.67:0/2749479683 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:04.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe604e00700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe600106c10 0x7fe60019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5ff400700 1 -- 172.21.15.67:0/2749479683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe600100b50 msgr2=0x7fe6001991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:04.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5ff400700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe600100b50 0x7fe6001991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5ff400700 1 -- 172.21.15.67:0/2749479683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe600106c10 msgr2=0x7fe60019daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:04.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5ff400700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe600106c10 0x7fe60019daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5ff400700 1 -- 172.21.15.67:0/2749479683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f8009c40 con 0x7fe600101510 2024-03-30T02:46:04.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5ff400700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe600101510 0x7fe600199700 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00d950 tx=0x7fe5ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:04.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5fcc00700 1 -- 172.21.15.67:0/2749479683 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec009930 con 0x7fe600101510 2024-03-30T02:46:04.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe5fcc00700 1 -- 172.21.15.67:0/2749479683 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5ec010460 con 0x7fe600101510 2024-03-30T02:46:04.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.965+0000 7fe605800700 1 -- 172.21.15.67:0/2749479683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f00095f0 con 0x7fe600101510 2024-03-30T02:46:04.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.966+0000 7fe5fcc00700 1 -- 172.21.15.67:0/2749479683 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00f620 con 0x7fe600101510 2024-03-30T02:46:04.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.966+0000 7fe605800700 1 -- 172.21.15.67:0/2749479683 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6001a5d50 con 0x7fe600101510 2024-03-30T02:46:04.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.969+0000 7fe5fcc00700 1 -- 172.21.15.67:0/2749479683 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe5ec0108d0 con 0x7fe600101510 2024-03-30T02:46:04.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.970+0000 7fe61a866b80 1 Processor -- start 2024-03-30T02:46:04.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.970+0000 7fe61a866b80 1 -- start start 2024-03-30T02:46:04.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.970+0000 7fe5fcc00700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5e40a1290 0x7fe5e40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.970+0000 7fe5fcc00700 1 -- 172.21.15.67:0/2749479683 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fe5ec0cce80 con 0x7fe600101510 2024-03-30T02:46:04.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe604e00700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5e40a1290 0x7fe5e40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 0x558a010fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01105b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 0x558a01108430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe61a866b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a01053a10 con 0x558a01106040 2024-03-30T02:46:04.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe61a866b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a010513d0 con 0x558a010fb140 2024-03-30T02:46:04.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe61a866b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0104ccc0 con 0x558a010fa820 2024-03-30T02:46:04.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe5ff400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01105b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe5ff400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01105b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49570/0 (socket says 172.21.15.67:49570) 2024-03-30T02:46:04.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe5ff400700 1 -- 172.21.15.67:0/1891484552 learned_addr learned my addr 172.21.15.67:0/1891484552 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:04.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe604e00700 1 --2- 172.21.15.67:0/1891484552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 0x558a01108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe5ffe00700 1 --2- 172.21.15.67:0/1891484552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 0x558a010fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe5ff400700 1 -- 172.21.15.67:0/1891484552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 msgr2=0x558a010fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:04.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe5ff400700 1 --2- 172.21.15.67:0/1891484552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 0x558a010fac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe5ff400700 1 -- 172.21.15.67:0/1891484552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 msgr2=0x558a01108430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:04.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe5ff400700 1 --2- 172.21.15.67:0/1891484552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 0x558a01108430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.971+0000 7fe5ff400700 1 -- 172.21.15.67:0/1891484552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a00ece430 con 0x558a010fb140 2024-03-30T02:46:04.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.972+0000 7fe604e00700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5e40a1290 0x7fe5e40a3750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009fd0 tx=0x7fe5f800fdd0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:04.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.972+0000 7fe5ff400700 1 --2- 172.21.15.67:0/1891484552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01105b00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0a1b40 tx=0x7fe5ec099620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa495b5ae2f7cc2c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:04.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.972+0000 7fe5f5600700 1 -- 172.21.15.67:0/1891484552 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec09da20 con 0x558a010fb140 2024-03-30T02:46:04.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.972+0000 7fe5f5600700 1 -- 172.21.15.67:0/1891484552 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5ec0a2070 con 0x558a010fb140 2024-03-30T02:46:04.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.972+0000 7fe5f5600700 1 -- 172.21.15.67:0/1891484552 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec099e50 con 0x558a010fb140 2024-03-30T02:46:04.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 -- 172.21.15.67:0/1891484552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 msgr2=0x558a01105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:04.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1891484552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01105b00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0a1b40 tx=0x7fe5ec099620 comp rx=0 tx=0).stop 2024-03-30T02:46:04.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 -- 172.21.15.67:0/1891484552 shutdown_connections 2024-03-30T02:46:04.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1891484552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 0x558a01108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1891484552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 0x558a010fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1891484552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01105b00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 -- 172.21.15.67:0/1891484552 >> 172.21.15.67:0/1891484552 conn(0x558a0105a720 msgr2=0x558a010f6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 -- 172.21.15.67:0/1891484552 shutdown_connections 2024-03-30T02:46:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 -- 172.21.15.67:0/1891484552 wait complete. 2024-03-30T02:46:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.973+0000 7fe61a866b80 1 -- start start 2024-03-30T02:46:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe61a866b80 1 Processor -- start 2024-03-30T02:46:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 0x558a01105ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 0x558a01103510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe61a866b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a00fe8e00 con 0x558a01106040 2024-03-30T02:46:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe61a866b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a00fe8aa0 con 0x558a010fb140 2024-03-30T02:46:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe61a866b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a00ed0720 con 0x558a010fa820 2024-03-30T02:46:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe61a866b80 10 client.0 _ll_register_callbacks cb 0x558a00ff24f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe604e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 0x558a01103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe5ff400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe5ffe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 0x558a01105ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe5ff400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01102dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49578/0 (socket says 172.21.15.67:49578) 2024-03-30T02:46:04.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.974+0000 7fe5ff400700 1 -- 172.21.15.67:0/1685420038 learned_addr learned my addr 172.21.15.67:0/1685420038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:04.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ff400700 1 -- 172.21.15.67:0/1685420038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 msgr2=0x558a01105ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:04.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ff400700 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 0x558a01105ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ff400700 1 -- 172.21.15.67:0/1685420038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 msgr2=0x558a01103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:04.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ff400700 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 0x558a01103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ff400700 1 -- 172.21.15.67:0/1685420038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6001a5d50 con 0x558a010fb140 2024-03-30T02:46:04.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ff400700 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01102dd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009520 tx=0x7fe5ec000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea300396c3bacf5b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:04.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ea000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ea000700 1 -- 172.21.15.67:0/1685420038 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0b6070 con 0x558a010fb140 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ea000700 1 -- 172.21.15.67:0/1685420038 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fe5ec096300 con 0x558a010fb140 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe61a866b80 1 -- 172.21.15.67:0/1685420038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a0100d400 con 0x558a010fb140 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.975+0000 7fe5ea000700 1 -- 172.21.15.67:0/1685420038 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5ec09daa0 con 0x558a010fb140 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe61a866b80 10 client.27464 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe61a866b80 1 -- 172.21.15.67:0/1685420038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558a00e7abe0 con 0x558a010fb140 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe5ea000700 1 -- 172.21.15.67:0/1685420038 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a0100d400 con 0x558a010fb140 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe5ea000700 1 -- 172.21.15.67:0/1685420038 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe5ec099e50 con 0x558a010fb140 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe61a866b80 10 client.27464 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:04.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe61a866b80 10 client.27464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe61a866b80 20 client.27464 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558a01115790 0x558a01117b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe61a866b80 4 client.27464 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.976+0000 7fe61a866b80 1 -- 172.21.15.67:0/1685420038 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a01045780 con 0x558a01115790 2024-03-30T02:46:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.977+0000 7fe5ffe00700 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558a01115790 0x558a01117b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.977+0000 7fe5ffe00700 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558a01115790 0x558a01117b80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=507ce39f2c95821e server_cookie=22b8674fe62c3264 in_seq=0 out_seq=0 2024-03-30T02:46:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.977+0000 7fe5ea000700 10 client.27464 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:04.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe5ea000700 1 -- 172.21.15.67:0/1685420038 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558a01045780 con 0x558a01115790 2024-03-30T02:46:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe5ea000700 10 client.27464 handle_command_reply: tid=0 2024-03-30T02:46:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 client.27464 shutdown 2024-03-30T02:46:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 -- 172.21.15.67:0/1685420038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 msgr2=0x558a01102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01102dd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009520 tx=0x7fe5ec000bc0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 -- 172.21.15.67:0/1685420038 shutdown_connections 2024-03-30T02:46:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558a01115790 0x558a01117b80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a01106040 0x558a01103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fa820 0x558a01105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 --2- 172.21.15.67:0/1685420038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fb140 0x558a01102dd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe5ea000700 0 client.27464 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 -- 172.21.15.67:0/1685420038 >> 172.21.15.67:0/1685420038 conn(0x558a0105a720 msgr2=0x558a010fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 -- 172.21.15.67:0/1685420038 shutdown_connections 2024-03-30T02:46:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 1 -- 172.21.15.67:0/1685420038 wait complete. 2024-03-30T02:46:04.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.978+0000 7fe61a866b80 20 client.27464 trim_cache size 0 max 16384 2024-03-30T02:46:04.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.996+0000 7fe61a866b80 1 Processor -- start 2024-03-30T02:46:04.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.996+0000 7fe61a866b80 1 -- start start 2024-03-30T02:46:04.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.996+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a0105a720 0x558a0110cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.996+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fa820 0x558a01105610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.996+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fb140 0x558a01100050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:04.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.996+0000 7fe61a866b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a01103190 con 0x558a010fb140 2024-03-30T02:46:04.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.996+0000 7fe61a866b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a010fde80 con 0x558a010fa820 2024-03-30T02:46:04.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.996+0000 7fe61a866b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0110d1a0 con 0x558a0105a720 2024-03-30T02:46:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.996+0000 7fe604e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fb140 0x558a01100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ffe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a0105a720 0x558a0110cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ffe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a0105a720 0x558a0110cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56376/0 (socket says 172.21.15.67:56376) 2024-03-30T02:46:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ff400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fa820 0x558a01105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe604e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fb140 0x558a01100050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37836/0 (socket says 172.21.15.67:37836) 2024-03-30T02:46:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ffe00700 1 -- 172.21.15.67:0/576557785 learned_addr learned my addr 172.21.15.67:0/576557785 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ffe00700 1 -- 172.21.15.67:0/576557785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fa820 msgr2=0x558a01105610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ffe00700 1 --2- 172.21.15.67:0/576557785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fa820 0x558a01105610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ffe00700 1 -- 172.21.15.67:0/576557785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fb140 msgr2=0x558a01100050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ffe00700 1 --2- 172.21.15.67:0/576557785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fb140 0x558a01100050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ffe00700 1 -- 172.21.15.67:0/576557785 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a01012f20 con 0x558a0105a720 2024-03-30T02:46:05.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ff400700 1 --2- 172.21.15.67:0/576557785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fa820 0x558a01105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:05.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5ffe00700 1 --2- 172.21.15.67:0/576557785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a0105a720 0x558a0110cc60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009990 tx=0x7fe5f0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb93715f0ba6263d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:05.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.997+0000 7fe5f6000700 1 -- 172.21.15.67:0/576557785 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f001d040 con 0x558a0105a720 2024-03-30T02:46:05.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe5f6000700 1 -- 172.21.15.67:0/576557785 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5f001e040 con 0x558a0105a720 2024-03-30T02:46:05.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe5f6000700 1 -- 172.21.15.67:0/576557785 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000b9f0 con 0x558a0105a720 2024-03-30T02:46:05.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe61a866b80 1 -- 172.21.15.67:0/576557785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a0105a720 msgr2=0x558a0110cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe61a866b80 1 --2- 172.21.15.67:0/576557785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a0105a720 0x558a0110cc60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009990 tx=0x7fe5f0005fd0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe61a866b80 1 -- 172.21.15.67:0/576557785 shutdown_connections 2024-03-30T02:46:05.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe61a866b80 1 --2- 172.21.15.67:0/576557785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fb140 0x558a01100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe61a866b80 1 --2- 172.21.15.67:0/576557785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a0105a720 0x558a0110cc60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe61a866b80 1 --2- 172.21.15.67:0/576557785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a010fa820 0x558a01105610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe61a866b80 1 -- 172.21.15.67:0/576557785 >> 172.21.15.67:0/576557785 conn(0x558a010efbd0 msgr2=0x558a010f0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:05.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe61a866b80 1 -- 172.21.15.67:0/576557785 shutdown_connections 2024-03-30T02:46:05.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.998+0000 7fe61a866b80 1 -- 172.21.15.67:0/576557785 wait complete. 2024-03-30T02:46:05.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.999+0000 7fe61a866b80 1 -- start start 2024-03-30T02:46:05.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.999+0000 7fe61a866b80 1 Processor -- start 2024-03-30T02:46:05.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.999+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0105a720 0x558a0115f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.999+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fa820 0x558a0115fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.999+0000 7fe61a866b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fb140 0x558a0115ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.999+0000 7fe61a866b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0116a540 con 0x558a010fa820 2024-03-30T02:46:05.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.999+0000 7fe61a866b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0116a680 con 0x558a0105a720 2024-03-30T02:46:05.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.999+0000 7fe61a866b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0116a840 con 0x558a010fb140 2024-03-30T02:46:05.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:04.999+0000 7fe61a866b80 10 client.0 _ll_register_callbacks cb 0x558a00ff24f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:05.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe604e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fb140 0x558a0115ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ff400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fa820 0x558a0115fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe604e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fb140 0x558a0115ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:56392/0 (socket says 172.21.15.67:56392) 2024-03-30T02:46:05.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe604e00700 1 -- 172.21.15.67:0/2310929718 learned_addr learned my addr 172.21.15.67:0/2310929718 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:05.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ffe00700 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0105a720 0x558a0115f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ff400700 1 -- 172.21.15.67:0/2310929718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fb140 msgr2=0x558a0115ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ff400700 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fb140 0x558a0115ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ff400700 1 -- 172.21.15.67:0/2310929718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0105a720 msgr2=0x558a0115f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ff400700 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0105a720 0x558a0115f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ff400700 1 -- 172.21.15.67:0/2310929718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a00e7abe0 con 0x558a010fa820 2024-03-30T02:46:05.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe604e00700 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fb140 0x558a0115ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:05.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ff400700 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fa820 0x558a0115fa00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0960f0 tx=0x7fe5ec0965b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f18d808f3bcd837 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:05.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ea000700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:05.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.000+0000 7fe5ea000700 1 -- 172.21.15.67:0/2310929718 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0992e0 con 0x558a010fa820 2024-03-30T02:46:05.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe5ea000700 1 -- 172.21.15.67:0/2310929718 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fe5ec09da20 con 0x558a010fa820 2024-03-30T02:46:05.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe61a866b80 1 -- 172.21.15.67:0/2310929718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5ec0a2070 con 0x558a010fa820 2024-03-30T02:46:05.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe5ea000700 1 -- 172.21.15.67:0/2310929718 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5ec09a2f0 con 0x558a010fa820 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe61a866b80 10 client.17331 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe61a866b80 1 -- 172.21.15.67:0/2310929718 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fe60010e2d0 con 0x558a010fa820 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe5ea000700 1 -- 172.21.15.67:0/2310929718 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5ec0a2070 con 0x558a010fa820 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe5ea000700 1 -- 172.21.15.67:0/2310929718 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe5ec098b50 con 0x558a010fa820 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe61a866b80 10 client.17331 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe61a866b80 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe61a866b80 20 client.17331 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe61a866b80 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558a01106040 0x558a0116dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe61a866b80 4 client.17331 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:46:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.001+0000 7fe61a866b80 1 -- 172.21.15.67:0/2310929718 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558a00e9bd70 con 0x558a01106040 2024-03-30T02:46:05.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.002+0000 7fe5ffe00700 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558a01106040 0x558a0116dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.002+0000 7fe5ea000700 10 client.17331 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:05.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.002+0000 7fe5ffe00700 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558a01106040 0x558a0116dc30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b009db85f1f80c server_cookie=b8814939fddaa336 in_seq=0 out_seq=0 2024-03-30T02:46:05.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.003+0000 7fe5ea000700 1 -- 172.21.15.67:0/2310929718 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558a00e9bd70 con 0x558a01106040 2024-03-30T02:46:05.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.003+0000 7fe5ea000700 10 client.17331 handle_command_reply: tid=0 2024-03-30T02:46:05.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.003+0000 7fe61a866b80 1 client.17331 shutdown 2024-03-30T02:46:05.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.003+0000 7fe61a866b80 1 -- 172.21.15.67:0/2310929718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fa820 msgr2=0x558a0115fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fa820 0x558a0115fa00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0960f0 tx=0x7fe5ec0965b0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 1 -- 172.21.15.67:0/2310929718 shutdown_connections 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a010fb140 0x558a0115ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0105a720 0x558a0115f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x558a01106040 0x558a0116dc30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 1 --2- 172.21.15.67:0/2310929718 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558a010fa820 0x558a0115fa00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 1 -- 172.21.15.67:0/2310929718 >> 172.21.15.67:0/2310929718 conn(0x558a010efbd0 msgr2=0x558a010fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe5ea000700 0 client.17331 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 1 -- 172.21.15.67:0/2310929718 shutdown_connections 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 1 -- 172.21.15.67:0/2310929718 wait complete. 2024-03-30T02:46:05.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.004+0000 7fe61a866b80 20 client.17331 trim_cache size 0 max 16384 2024-03-30T02:46:05.012 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:46:05.012 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:46:05.012 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "82133bdd-8299-4c7a-b6f6-3088844c0659", 2024-03-30T02:46:05.012 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:46:05.012 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:46:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 -- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5e40a1290 msgr2=0x7fe5e40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5e40a1290 0x7fe5e40a3750 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009fd0 tx=0x7fe5f800fdd0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 -- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe600101510 msgr2=0x7fe600199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe600101510 0x7fe600199700 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00d950 tx=0x7fe5ec00dc60 comp rx=0 tx=0).stop 2024-03-30T02:46:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 -- 172.21.15.67:0/2749479683 shutdown_connections 2024-03-30T02:46:05.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe5e40a1290 0x7fe5e40a3750 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe600106c10 0x7fe60019daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe600100b50 0x7fe6001991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 --2- 172.21.15.67:0/2749479683 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe600101510 0x7fe600199700 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 -- 172.21.15.67:0/2749479683 >> 172.21.15.67:0/2749479683 conn(0x7fe6000fc620 msgr2=0x7fe6000fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 -- 172.21.15.67:0/2749479683 shutdown_connections 2024-03-30T02:46:05.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.005+0000 7fe605800700 1 -- 172.21.15.67:0/2749479683 wait complete. 2024-03-30T02:46:05.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:46:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.267+0000 7f532a800700 1 -- 172.21.15.67:0/729457667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069bf0 msgr2=0x7f5324105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.267+0000 7f532a800700 1 --2- 172.21.15.67:0/729457667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069bf0 0x7f5324105980 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f53100099c0 tx=0x7f5310009cd0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.267+0000 7f532a800700 1 -- 172.21.15.67:0/729457667 shutdown_connections 2024-03-30T02:46:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.267+0000 7f532a800700 1 --2- 172.21.15.67:0/729457667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53241060b0 0x7f532410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.267+0000 7f532a800700 1 --2- 172.21.15.67:0/729457667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5324069240 0x7f5324069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.267+0000 7f532a800700 1 --2- 172.21.15.67:0/729457667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069bf0 0x7f5324105980 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.267+0000 7f532a800700 1 -- 172.21.15.67:0/729457667 >> 172.21.15.67:0/729457667 conn(0x7f5324076d80 msgr2=0x7f5324077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:05.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.267+0000 7f532a800700 1 -- 172.21.15.67:0/729457667 shutdown_connections 2024-03-30T02:46:05.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.267+0000 7f532a800700 1 -- 172.21.15.67:0/729457667 wait complete. 2024-03-30T02:46:05.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.268+0000 7f532a800700 1 Processor -- start 2024-03-30T02:46:05.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.268+0000 7f532a800700 1 -- start start 2024-03-30T02:46:05.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f532a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069240 0x7f53241971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f532a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53241060b0 0x7f5324197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f532a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532419bac0 0x7f532419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f532a800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532419c540 con 0x7f5324069240 2024-03-30T02:46:05.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f532a800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532419c6b0 con 0x7f53241060b0 2024-03-30T02:46:05.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f532a800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532419c870 con 0x7f532419bac0 2024-03-30T02:46:05.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5323400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069240 0x7f53241971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5322a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53241060b0 0x7f5324197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5323400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069240 0x7f53241971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37854/0 (socket says 172.21.15.67:37854) 2024-03-30T02:46:05.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5323400700 1 -- 172.21.15.67:0/2602610604 learned_addr learned my addr 172.21.15.67:0/2602610604 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:05.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5322a00700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53241060b0 0x7f5324197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49616/0 (socket says 172.21.15.67:49616) 2024-03-30T02:46:05.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5323e00700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532419bac0 0x7f532419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5323400700 1 -- 172.21.15.67:0/2602610604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532419bac0 msgr2=0x7f532419bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5323400700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532419bac0 0x7f532419bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5323400700 1 -- 172.21.15.67:0/2602610604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53241060b0 msgr2=0x7f5324197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5323400700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53241060b0 0x7f5324197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5323400700 1 -- 172.21.15.67:0/2602610604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53100095f0 con 0x7f5324069240 2024-03-30T02:46:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.269+0000 7f5322a00700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53241060b0 0x7f5324197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.270+0000 7f5323400700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069240 0x7f53241971e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f531400e9a0 tx=0x7f531400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:05.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.270+0000 7f531be00700 1 -- 172.21.15.67:0/2602610604 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314009790 con 0x7f5324069240 2024-03-30T02:46:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.270+0000 7f532a800700 1 -- 172.21.15.67:0/2602610604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53241a3dd0 con 0x7f5324069240 2024-03-30T02:46:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.270+0000 7f531be00700 1 -- 172.21.15.67:0/2602610604 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5314010430 con 0x7f5324069240 2024-03-30T02:46:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.270+0000 7f531be00700 1 -- 172.21.15.67:0/2602610604 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314018680 con 0x7f5324069240 2024-03-30T02:46:05.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.270+0000 7f532a800700 1 -- 172.21.15.67:0/2602610604 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53241a41f0 con 0x7f5324069240 2024-03-30T02:46:05.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.273+0000 7f531be00700 1 -- 172.21.15.67:0/2602610604 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f53140105a0 con 0x7f5324069240 2024-03-30T02:46:05.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.275+0000 7f531be00700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53080a1290 0x7f53080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.275+0000 7f533e328b80 1 Processor -- start 2024-03-30T02:46:05.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.275+0000 7f531be00700 1 -- 172.21.15.67:0/2602610604 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f5314014070 con 0x7f5324069240 2024-03-30T02:46:05.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.275+0000 7f533e328b80 1 -- start start 2024-03-30T02:46:05.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.275+0000 7f5322a00700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53080a1290 0x7f53080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.276+0000 7f533e328b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a0da5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.276+0000 7f533e328b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.276+0000 7f533e328b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a0f0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.276+0000 7f533e328b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907a033990 con 0x55907a0dc1c0 2024-03-30T02:46:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.276+0000 7f533e328b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907a031350 con 0x55907a0dab10 2024-03-30T02:46:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.276+0000 7f533e328b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907a02cc40 con 0x55907a0ee490 2024-03-30T02:46:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.276+0000 7f5323400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a0da5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.276+0000 7f5323400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a0da5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37860/0 (socket says 172.21.15.67:37860) 2024-03-30T02:46:05.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.276+0000 7f5323400700 1 -- 172.21.15.67:0/1103555574 learned_addr learned my addr 172.21.15.67:0/1103555574 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:05.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5323e00700 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a0f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5322a00700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53080a1290 0x7f53080a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5310009fd0 tx=0x7f5310005fd0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:05.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5322a00700 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5323e00700 1 -- 172.21.15.67:0/1103555574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 msgr2=0x55907a0daf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5323e00700 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0daf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5323e00700 1 -- 172.21.15.67:0/1103555574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 msgr2=0x55907a0da5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5323e00700 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a0da5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5323e00700 1 -- 172.21.15.67:0/1103555574 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559079eae3b0 con 0x55907a0ee490 2024-03-30T02:46:05.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5323400700 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a0da5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:05.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.277+0000 7f5323e00700 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a0f0880 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f531c011580 tx=0x7f531c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17ed980aff7bee60 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:05.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.278+0000 7f5318c00700 1 -- 172.21.15.67:0/1103555574 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c00e040 con 0x55907a0ee490 2024-03-30T02:46:05.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.278+0000 7f5318c00700 1 -- 172.21.15.67:0/1103555574 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531c00f040 con 0x55907a0ee490 2024-03-30T02:46:05.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.278+0000 7f5318c00700 1 -- 172.21.15.67:0/1103555574 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0044f0 con 0x55907a0ee490 2024-03-30T02:46:05.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 -- 172.21.15.67:0/1103555574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 msgr2=0x55907a0f0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a0f0880 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f531c011580 tx=0x7f531c009920 comp rx=0 tx=0).stop 2024-03-30T02:46:05.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 -- 172.21.15.67:0/1103555574 shutdown_connections 2024-03-30T02:46:05.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a0da5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a0f0880 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 --2- 172.21.15.67:0/1103555574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 -- 172.21.15.67:0/1103555574 >> 172.21.15.67:0/1103555574 conn(0x55907a03a6a0 msgr2=0x55907a0d6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:05.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 -- 172.21.15.67:0/1103555574 shutdown_connections 2024-03-30T02:46:05.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 -- 172.21.15.67:0/1103555574 wait complete. 2024-03-30T02:46:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.279+0000 7f533e328b80 1 -- start start 2024-03-30T02:46:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f533e328b80 1 Processor -- start 2024-03-30T02:46:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f533e328b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f533e328b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 0x55907a0ded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f533e328b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 0x55907a0df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f533e328b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x559079fc8d80 con 0x55907a0ee490 2024-03-30T02:46:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f533e328b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559079fc8a20 con 0x55907a0dab10 2024-03-30T02:46:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f533e328b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559079eb06a0 con 0x55907a0dc1c0 2024-03-30T02:46:05.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f533e328b80 10 client.0 _ll_register_callbacks cb 0x559079fd2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:05.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f5323400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f5323400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0de7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49632/0 (socket says 172.21.15.67:49632) 2024-03-30T02:46:05.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f5323400700 1 -- 172.21.15.67:0/1635115565 learned_addr learned my addr 172.21.15.67:0/1635115565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:05.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.280+0000 7f5323e00700 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 0x55907a0df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f5322a00700 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 0x55907a0ded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f5323400700 1 -- 172.21.15.67:0/1635115565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 msgr2=0x55907a0ded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f5323400700 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 0x55907a0ded30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f5323400700 1 -- 172.21.15.67:0/1635115565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 msgr2=0x55907a0df270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f5323400700 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 0x55907a0df270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f5323400700 1 -- 172.21.15.67:0/1635115565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53241a41f0 con 0x55907a0dab10 2024-03-30T02:46:05.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f5322a00700 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 0x55907a0ded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:46:05.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f5323400700 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0de7f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f531400c510 tx=0x7f531409a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c49021c0930555a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:05.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f530d600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:05.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f530d600700 1 -- 172.21.15.67:0/1635115565 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53140996a0 con 0x55907a0dab10 2024-03-30T02:46:05.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f530d600700 1 -- 172.21.15.67:0/1635115565 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f531409dde0 con 0x55907a0dab10 2024-03-30T02:46:05.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f533e328b80 1 -- 172.21.15.67:0/1635115565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559079fed380 con 0x55907a0dab10 2024-03-30T02:46:05.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f530d600700 1 -- 172.21.15.67:0/1635115565 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53140a2430 con 0x55907a0dab10 2024-03-30T02:46:05.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f533e328b80 10 client.27482 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:05.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.281+0000 7f533e328b80 1 -- 172.21.15.67:0/1635115565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f531c011230 con 0x55907a0dab10 2024-03-30T02:46:05.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.282+0000 7f530d600700 1 -- 172.21.15.67:0/1635115565 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5314063020 con 0x55907a0dab10 2024-03-30T02:46:05.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.282+0000 7f530d600700 1 -- 172.21.15.67:0/1635115565 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5314094bc0 con 0x55907a0dab10 2024-03-30T02:46:05.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.282+0000 7f533e328b80 10 client.27482 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:05.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.282+0000 7f533e328b80 10 client.27482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:05.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.282+0000 7f533e328b80 20 client.27482 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:05.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.282+0000 7f533e328b80 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55907a0fe210 0x55907a100630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.282+0000 7f533e328b80 4 client.27482 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:05.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.282+0000 7f533e328b80 1 -- 172.21.15.67:0/1635115565 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55907a025700 con 0x55907a0fe210 2024-03-30T02:46:05.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.282+0000 7f5322a00700 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55907a0fe210 0x55907a100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.283+0000 7f5322a00700 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55907a0fe210 0x55907a100630 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af08b7dc0e9d7a53 server_cookie=e9887673d6131991 in_seq=0 out_seq=0 2024-03-30T02:46:05.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.283+0000 7f530d600700 10 client.27482 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:05.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f530d600700 1 -- 172.21.15.67:0/1635115565 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55907a025700 con 0x55907a0fe210 2024-03-30T02:46:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f530d600700 10 client.27482 handle_command_reply: tid=0 2024-03-30T02:46:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 client.27482 shutdown 2024-03-30T02:46:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 -- 172.21.15.67:0/1635115565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 msgr2=0x55907a0de7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0de7f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f531400c510 tx=0x7f531409a7a0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 -- 172.21.15.67:0/1635115565 shutdown_connections 2024-03-30T02:46:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55907a0fe210 0x55907a100630 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.298 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 0x55907a0df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0de7f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 --2- 172.21.15.67:0/1635115565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 0x55907a0ded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f530d600700 0 client.27482 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:05.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 -- 172.21.15.67:0/1635115565 >> 172.21.15.67:0/1635115565 conn(0x55907a03a6a0 msgr2=0x55907a0dc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:05.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 -- 172.21.15.67:0/1635115565 shutdown_connections 2024-03-30T02:46:05.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.297+0000 7f533e328b80 1 -- 172.21.15.67:0/1635115565 wait complete. 2024-03-30T02:46:05.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.298+0000 7f533e328b80 20 client.27482 trim_cache size 0 max 16384 2024-03-30T02:46:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.315+0000 7f533e328b80 1 Processor -- start 2024-03-30T02:46:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.315+0000 7f533e328b80 1 -- start start 2024-03-30T02:46:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.315+0000 7f533e328b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0f5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.315+0000 7f533e328b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 0x55907a0edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.315+0000 7f533e328b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 0x55907a0da310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.315+0000 7f533e328b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907a0fdfe0 con 0x55907a0ee490 2024-03-30T02:46:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.315+0000 7f533e328b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907a0f5740 con 0x55907a0dab10 2024-03-30T02:46:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.315+0000 7f533e328b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907a0ed960 con 0x55907a0dc1c0 2024-03-30T02:46:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49636/0 (socket says 172.21.15.67:49636) 2024-03-30T02:46:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323400700 1 -- 172.21.15.67:0/3306936249 learned_addr learned my addr 172.21.15.67:0/3306936249 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:05.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323e00700 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 0x55907a0da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5322a00700 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 0x55907a0edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323e00700 1 -- 172.21.15.67:0/3306936249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 msgr2=0x55907a0edce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323e00700 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 0x55907a0edce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323e00700 1 -- 172.21.15.67:0/3306936249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 msgr2=0x55907a0f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323e00700 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323e00700 1 -- 172.21.15.67:0/3306936249 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559079e5ab60 con 0x55907a0ee490 2024-03-30T02:46:05.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.316+0000 7f5323400700 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:05.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.317+0000 7f5323e00700 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 0x55907a0da310 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f531c009690 tx=0x7f531c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1daf9d4af24138c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:05.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.317+0000 7f5319600700 1 -- 172.21.15.67:0/3306936249 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c00e040 con 0x55907a0ee490 2024-03-30T02:46:05.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.317+0000 7f5319600700 1 -- 172.21.15.67:0/3306936249 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f531c005040 con 0x55907a0ee490 2024-03-30T02:46:05.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.317+0000 7f5319600700 1 -- 172.21.15.67:0/3306936249 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0058b0 con 0x55907a0ee490 2024-03-30T02:46:05.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.317+0000 7f533e328b80 1 -- 172.21.15.67:0/3306936249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 msgr2=0x55907a0da310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 0x55907a0da310 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f531c009690 tx=0x7f531c009b60 comp rx=0 tx=0).stop 2024-03-30T02:46:05.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 -- 172.21.15.67:0/3306936249 shutdown_connections 2024-03-30T02:46:05.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0ee490 0x55907a0da310 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a0f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 --2- 172.21.15.67:0/3306936249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0dc1c0 0x55907a0edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 -- 172.21.15.67:0/3306936249 >> 172.21.15.67:0/3306936249 conn(0x55907a03a6a0 msgr2=0x55907a0cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:05.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 -- 172.21.15.67:0/3306936249 shutdown_connections 2024-03-30T02:46:05.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 -- 172.21.15.67:0/3306936249 wait complete. 2024-03-30T02:46:05.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 -- start start 2024-03-30T02:46:05.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.318+0000 7f533e328b80 1 Processor -- start 2024-03-30T02:46:05.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f533e328b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a14b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f533e328b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a14b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f533e328b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a14a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f533e328b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907a14a590 con 0x55907a0dc1c0 2024-03-30T02:46:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f533e328b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907a14a700 con 0x55907a0dab10 2024-03-30T02:46:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f533e328b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55907a14a8c0 con 0x55907a0ee490 2024-03-30T02:46:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f533e328b80 10 client.0 _ll_register_callbacks cb 0x7f5314099af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a14b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a14b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:49638/0 (socket says 172.21.15.67:49638) 2024-03-30T02:46:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323400700 1 -- 172.21.15.67:0/3621479475 learned_addr learned my addr 172.21.15.67:0/3621479475 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:05.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5322a00700 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a14b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323e00700 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a14a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323400700 1 -- 172.21.15.67:0/3621479475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 msgr2=0x55907a14a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323400700 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a14a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323400700 1 -- 172.21.15.67:0/3621479475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 msgr2=0x55907a14b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:05.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323400700 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a14b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323400700 1 -- 172.21.15.67:0/3621479475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c011230 con 0x55907a0dab10 2024-03-30T02:46:05.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.319+0000 7f5323e00700 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a14a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:05.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f5323400700 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a14b330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f531400c2d0 tx=0x7f53140b8330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a971f0844aaf7e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:05.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f530d600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:05.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f530d600700 1 -- 172.21.15.67:0/3621479475 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531409dde0 con 0x55907a0dab10 2024-03-30T02:46:05.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f530d600700 1 -- 172.21.15.67:0/3621479475 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f5314064080 con 0x55907a0dab10 2024-03-30T02:46:05.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f533e328b80 1 -- 172.21.15.67:0/3621479475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f531c00f040 con 0x55907a0dab10 2024-03-30T02:46:05.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f530d600700 1 -- 172.21.15.67:0/3621479475 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53140a2430 con 0x55907a0dab10 2024-03-30T02:46:05.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f533e328b80 10 client.27488 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:05.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f533e328b80 1 -- 172.21.15.67:0/3621479475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559079eae3b0 con 0x55907a0dab10 2024-03-30T02:46:05.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f530d600700 1 -- 172.21.15.67:0/3621479475 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5314080020 con 0x55907a0dab10 2024-03-30T02:46:05.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f530d600700 1 -- 172.21.15.67:0/3621479475 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f53140b2cb0 con 0x55907a0dab10 2024-03-30T02:46:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f533e328b80 10 client.27488 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f533e328b80 10 client.27488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f533e328b80 20 client.27488 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f533e328b80 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55907a152fd0 0x55907a1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.320+0000 7f533e328b80 4 client.27488 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:46:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.321+0000 7f533e328b80 1 -- 172.21.15.67:0/3621479475 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559079e7bcf0 con 0x55907a152fd0 2024-03-30T02:46:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.321+0000 7f5322a00700 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55907a152fd0 0x55907a1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.321+0000 7f530d600700 10 client.27488 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:05.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.321+0000 7f5322a00700 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55907a152fd0 0x55907a1553f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e990a65208efcb6b server_cookie=7190dc6b74ac764f in_seq=0 out_seq=0 2024-03-30T02:46:05.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.329+0000 7f530d600700 1 -- 172.21.15.67:0/3621479475 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x559079e7bcf0 con 0x55907a152fd0 2024-03-30T02:46:05.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.329+0000 7f530d600700 10 client.27488 handle_command_reply: tid=0 2024-03-30T02:46:05.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.329+0000 7f533e328b80 1 client.27488 shutdown 2024-03-30T02:46:05.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 -- 172.21.15.67:0/3621479475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 msgr2=0x55907a14b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a14b330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f531400c2d0 tx=0x7f53140b8330 comp rx=0 tx=0).stop 2024-03-30T02:46:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 -- 172.21.15.67:0/3621479475 shutdown_connections 2024-03-30T02:46:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55907a0ee490 0x55907a14a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55907a0dab10 0x55907a14b330 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x55907a152fd0 0x55907a1553f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 --2- 172.21.15.67:0/3621479475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55907a0dc1c0 0x55907a14b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f530d600700 0 client.27488 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 -- 172.21.15.67:0/3621479475 >> 172.21.15.67:0/3621479475 conn(0x55907a03a6a0 msgr2=0x55907a0edce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 -- 172.21.15.67:0/3621479475 shutdown_connections 2024-03-30T02:46:05.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 1 -- 172.21.15.67:0/3621479475 wait complete. 2024-03-30T02:46:05.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.330+0000 7f533e328b80 20 client.27488 trim_cache size 0 max 16384 2024-03-30T02:46:05.332 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:46:05.332 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (7826 inodes in the stack)", 2024-03-30T02:46:05.332 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:46:05.333 INFO:teuthology.orchestra.run.smithi067.stdout: "82133bdd-8299-4c7a-b6f6-3088844c0659": { 2024-03-30T02:46:05.333 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:46:05.333 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "82133bdd-8299-4c7a-b6f6-3088844c0659", 2024-03-30T02:46:05.333 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:46:05.333 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:46:05.333 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:46:05.333 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:46:05.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 -- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53080a1290 msgr2=0x7f53080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53080a1290 0x7f53080a3750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5310009fd0 tx=0x7f5310005fd0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 -- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069240 msgr2=0x7f53241971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:05.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069240 0x7f53241971e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f531400e9a0 tx=0x7f531400ecb0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 -- 172.21.15.67:0/2602610604 shutdown_connections 2024-03-30T02:46:05.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f53080a1290 0x7f53080a3750 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5324069240 0x7f53241971e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532419bac0 0x7f532419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 --2- 172.21.15.67:0/2602610604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53241060b0 0x7f5324197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:05.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 -- 172.21.15.67:0/2602610604 >> 172.21.15.67:0/2602610604 conn(0x7f5324076d80 msgr2=0x7f53240fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:05.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 -- 172.21.15.67:0/2602610604 shutdown_connections 2024-03-30T02:46:05.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:05.331+0000 7f532a800700 1 -- 172.21.15.67:0/2602610604 wait complete. 2024-03-30T02:46:05.352 INFO:tasks.cephfs.filesystem:scrub status for tag:82133bdd-8299-4c7a-b6f6-3088844c0659 - {'path': '/', 'tag': '82133bdd-8299-4c7a-b6f6-3088844c0659', 'options': 'recursive,force'} 2024-03-30T02:46:05.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:05.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:05 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:05.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:05 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:05.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:05.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:05 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: pgmap v278: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 KiB/s wr, 15 op/s 2024-03-30T02:46:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:46:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:06.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:06 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: pgmap v278: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 KiB/s wr, 15 op/s 2024-03-30T02:46:06.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:46:06.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:06.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:06 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: pgmap v278: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 67 KiB/s wr, 15 op/s 2024-03-30T02:46:06.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:46:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:06.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:06.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:06 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:07 smithi082 ceph-mon[79176]: osdmap e93: 12 total, 12 up, 12 in 2024-03-30T02:46:07.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:07.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:07 smithi067 ceph-mon[37808]: osdmap e93: 12 total, 12 up, 12 in 2024-03-30T02:46:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:07.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:07.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:07 smithi170 ceph-mon[78798]: osdmap e93: 12 total, 12 up, 12 in 2024-03-30T02:46:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: pgmap v279: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 85 KiB/s wr, 35 op/s 2024-03-30T02:46:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:08.765 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:08 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: pgmap v279: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 85 KiB/s wr, 35 op/s 2024-03-30T02:46:08.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:08.930 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:08 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: pgmap v279: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 85 KiB/s wr, 35 op/s 2024-03-30T02:46:08.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:08.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:08.955 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:08 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:09 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:09.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:09 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:09.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:09.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:09.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:09 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: pgmap v281: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 81 KiB/s wr, 108 op/s 2024-03-30T02:46:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:10.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:10 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: pgmap v281: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 81 KiB/s wr, 108 op/s 2024-03-30T02:46:10.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:10.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:10 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: pgmap v281: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 81 KiB/s wr, 108 op/s 2024-03-30T02:46:10.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:10.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:10.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:10 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:11.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:11 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:11.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:11 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:11.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:11.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:11 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: pgmap v282: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 81 KiB/s wr, 1.05k op/s 2024-03-30T02:46:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:12.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:12 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: pgmap v282: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 81 KiB/s wr, 1.05k op/s 2024-03-30T02:46:12.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:12.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:12 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: pgmap v282: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 81 KiB/s wr, 1.05k op/s 2024-03-30T02:46:12.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:12.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:12.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:12 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:13.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:13 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:13.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:13 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:13.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:13.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:13 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: pgmap v283: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 46 KiB/s wr, 1.87k op/s 2024-03-30T02:46:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:14.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:14 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: pgmap v283: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 46 KiB/s wr, 1.87k op/s 2024-03-30T02:46:14.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:14.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:14 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: pgmap v283: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 46 KiB/s wr, 1.87k op/s 2024-03-30T02:46:14.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:14.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:14.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:14 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:15.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:15 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:15.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:15 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:15.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:15.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:15 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:16 smithi082 ceph-mon[79176]: pgmap v284: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 46 KiB/s wr, 2.57k op/s 2024-03-30T02:46:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:16 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:16.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:16 smithi082 ceph-mon[79176]: osdmap e94: 12 total, 12 up, 12 in 2024-03-30T02:46:16.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:16 smithi067 ceph-mon[37808]: pgmap v284: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 46 KiB/s wr, 2.57k op/s 2024-03-30T02:46:16.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:16 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:16.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:16 smithi067 ceph-mon[37808]: osdmap e94: 12 total, 12 up, 12 in 2024-03-30T02:46:16.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:16 smithi170 ceph-mon[78798]: pgmap v284: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 46 KiB/s wr, 2.57k op/s 2024-03-30T02:46:16.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:16 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:16.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:16 smithi170 ceph-mon[78798]: osdmap e94: 12 total, 12 up, 12 in 2024-03-30T02:46:17.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:17 smithi067 ceph-mon[37808]: pgmap v286: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 KiB/s wr, 4.95k op/s 2024-03-30T02:46:17.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:17 smithi170 ceph-mon[78798]: pgmap v286: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 KiB/s wr, 4.95k op/s 2024-03-30T02:46:17.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:17 smithi082 ceph-mon[79176]: pgmap v286: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 19 KiB/s wr, 4.95k op/s 2024-03-30T02:46:19.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:19 smithi067 ceph-mon[37808]: pgmap v287: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 KiB/s wr, 4.73k op/s 2024-03-30T02:46:19.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:19 smithi170 ceph-mon[78798]: pgmap v287: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 KiB/s wr, 4.73k op/s 2024-03-30T02:46:19.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:19 smithi082 ceph-mon[79176]: pgmap v287: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 18 KiB/s wr, 4.73k op/s 2024-03-30T02:46:22.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:22 smithi170 ceph-mon[78798]: pgmap v288: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 17 KiB/s wr, 3.91k op/s 2024-03-30T02:46:22.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:22 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:46:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:22 smithi082 ceph-mon[79176]: pgmap v288: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 17 KiB/s wr, 3.91k op/s 2024-03-30T02:46:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:22 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:46:22.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:22 smithi067 ceph-mon[37808]: pgmap v288: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 17 KiB/s wr, 3.91k op/s 2024-03-30T02:46:22.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:22 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:46:23.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:23 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:46:23.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:23 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:46:23.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:23 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:46:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:23 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:46:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:23 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:46:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:23 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:46:23.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:23 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:46:23.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:23 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:46:23.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:23 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:46:24.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:24 smithi170 ceph-mon[78798]: pgmap v289: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 KiB/s wr, 3.08k op/s 2024-03-30T02:46:24.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:24 smithi082 ceph-mon[79176]: pgmap v289: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 KiB/s wr, 3.08k op/s 2024-03-30T02:46:24.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:24 smithi067 ceph-mon[37808]: pgmap v289: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.5 KiB/s wr, 3.08k op/s 2024-03-30T02:46:26.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:26 smithi170 ceph-mon[78798]: pgmap v290: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 KiB/s wr, 2.38k op/s 2024-03-30T02:46:26.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:26 smithi082 ceph-mon[79176]: pgmap v290: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 KiB/s wr, 2.38k op/s 2024-03-30T02:46:26.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:26 smithi067 ceph-mon[37808]: pgmap v290: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 KiB/s wr, 2.38k op/s 2024-03-30T02:46:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:27 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:46:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:27 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:46:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:27 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:46:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:27 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:46:27.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:27 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:46:27.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:27 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:46:27.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:27 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:46:27.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:27 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:46:27.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:27 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:46:27.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:27 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:46:27.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:27 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:46:27.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:27 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:46:27.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:27 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:46:27.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:27 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:46:27.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:27 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:46:28.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:28 smithi082 ceph-mon[79176]: pgmap v291: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 7.3 KiB/s wr, 330 op/s 2024-03-30T02:46:28.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:28 smithi067 ceph-mon[37808]: pgmap v291: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 7.3 KiB/s wr, 330 op/s 2024-03-30T02:46:28.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:28 smithi170 ceph-mon[78798]: pgmap v291: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 7.3 KiB/s wr, 330 op/s 2024-03-30T02:46:30.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:30 smithi082 ceph-mon[79176]: pgmap v292: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 6.2 KiB/s wr, 281 op/s 2024-03-30T02:46:30.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:30 smithi067 ceph-mon[37808]: pgmap v292: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 6.2 KiB/s wr, 281 op/s 2024-03-30T02:46:30.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:30 smithi170 ceph-mon[78798]: pgmap v292: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 6.2 KiB/s wr, 281 op/s 2024-03-30T02:46:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:32 smithi082 ceph-mon[79176]: pgmap v293: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.2 KiB/s wr, 104 op/s 2024-03-30T02:46:32.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:32 smithi067 ceph-mon[37808]: pgmap v293: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.2 KiB/s wr, 104 op/s 2024-03-30T02:46:32.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:32 smithi170 ceph-mon[78798]: pgmap v293: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.2 KiB/s wr, 104 op/s 2024-03-30T02:46:34.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:34 smithi082 ceph-mon[79176]: pgmap v294: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-03-30T02:46:34.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:34 smithi067 ceph-mon[37808]: pgmap v294: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-03-30T02:46:34.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:34 smithi170 ceph-mon[78798]: pgmap v294: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-03-30T02:46:35.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:46:35.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.571+0000 7f1944000700 1 -- 172.21.15.67:0/3762788703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 msgr2=0x7f193c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.571+0000 7f1944000700 1 --2- 172.21.15.67:0/3762788703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 0x7f193c101340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1930009990 tx=0x7f1930009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.571+0000 7f1944000700 1 -- 172.21.15.67:0/3762788703 shutdown_connections 2024-03-30T02:46:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.571+0000 7f1944000700 1 --2- 172.21.15.67:0/3762788703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f193c106080 0x7f193c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.571+0000 7f1944000700 1 --2- 172.21.15.67:0/3762788703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 0x7f193c101340 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.571+0000 7f1944000700 1 --2- 172.21.15.67:0/3762788703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c101910 0x7f193c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.571+0000 7f1944000700 1 -- 172.21.15.67:0/3762788703 >> 172.21.15.67:0/3762788703 conn(0x7f193c0fc7c0 msgr2=0x7f193c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:35.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.571+0000 7f1944000700 1 -- 172.21.15.67:0/3762788703 shutdown_connections 2024-03-30T02:46:35.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.571+0000 7f1944000700 1 -- 172.21.15.67:0/3762788703 wait complete. 2024-03-30T02:46:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.572+0000 7f1944000700 1 Processor -- start 2024-03-30T02:46:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.572+0000 7f1944000700 1 -- start start 2024-03-30T02:46:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1944000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 0x7f193c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1944000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c106080 0x7f193c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1944000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f193c073b30 0x7f193c1a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1944000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1a5ab0 con 0x7f193c073b30 2024-03-30T02:46:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1944000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1a5bf0 con 0x7f193c106080 2024-03-30T02:46:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1944000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1a5db0 con 0x7f193c100f60 2024-03-30T02:46:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1942000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f193c073b30 0x7f193c1a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1941600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 0x7f193c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1941600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 0x7f193c0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:33894/0 (socket says 172.21.15.67:33894) 2024-03-30T02:46:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1941600700 1 -- 172.21.15.67:0/2395373530 learned_addr learned my addr 172.21.15.67:0/2395373530 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:35.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1940c00700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c106080 0x7f193c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1941600700 1 -- 172.21.15.67:0/2395373530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c106080 msgr2=0x7f193c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1941600700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c106080 0x7f193c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1941600700 1 -- 172.21.15.67:0/2395373530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f193c073b30 msgr2=0x7f193c1a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1941600700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f193c073b30 0x7f193c1a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.573+0000 7f1941600700 1 -- 172.21.15.67:0/2395373530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19300095f0 con 0x7f193c100f60 2024-03-30T02:46:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.574+0000 7f1941600700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 0x7f193c0730b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1930005740 tx=0x7f1930005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.574+0000 7f193a000700 1 -- 172.21.15.67:0/2395373530 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193001d070 con 0x7f193c100f60 2024-03-30T02:46:35.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.574+0000 7f193a000700 1 -- 172.21.15.67:0/2395373530 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1930022470 con 0x7f193c100f60 2024-03-30T02:46:35.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.574+0000 7f1944000700 1 -- 172.21.15.67:0/2395373530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c1a6030 con 0x7f193c100f60 2024-03-30T02:46:35.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.574+0000 7f193a000700 1 -- 172.21.15.67:0/2395373530 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930017650 con 0x7f193c100f60 2024-03-30T02:46:35.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.574+0000 7f1944000700 1 -- 172.21.15.67:0/2395373530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c1a6400 con 0x7f193c100f60 2024-03-30T02:46:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.577+0000 7f193a000700 1 -- 172.21.15.67:0/2395373530 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f19300225e0 con 0x7f193c100f60 2024-03-30T02:46:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.578+0000 7f1957a4fb80 1 Processor -- start 2024-03-30T02:46:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.578+0000 7f193a000700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19240a1290 0x7f19240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.578+0000 7f193a000700 1 -- 172.21.15.67:0/2395373530 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f19300ce3b0 con 0x7f193c100f60 2024-03-30T02:46:35.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.578+0000 7f1957a4fb80 1 -- start start 2024-03-30T02:46:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1942000700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19240a1290 0x7f19240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15aee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 0x563f15af98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15afc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f15a47990 con 0x563f15aee5d0 2024-03-30T02:46:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f15a45350 con 0x563f15aeeef0 2024-03-30T02:46:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f15a40c40 con 0x563f15af9df0 2024-03-30T02:46:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1941600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15aee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1941600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15aee9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43638/0 (socket says 172.21.15.67:43638) 2024-03-30T02:46:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1941600700 1 -- 172.21.15.67:0/4203190922 learned_addr learned my addr 172.21.15.67:0/4203190922 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1940c00700 1 --2- 172.21.15.67:0/4203190922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 0x563f15af98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1942000700 1 --2- 172.21.15.67:0/4203190922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15afc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1941600700 1 -- 172.21.15.67:0/4203190922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 msgr2=0x563f15afc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1941600700 1 --2- 172.21.15.67:0/4203190922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15afc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1941600700 1 -- 172.21.15.67:0/4203190922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 msgr2=0x563f15af98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1941600700 1 --2- 172.21.15.67:0/4203190922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 0x563f15af98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.579+0000 7f1941600700 1 -- 172.21.15.67:0/4203190922 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f158c23b0 con 0x563f15aee5d0 2024-03-30T02:46:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.580+0000 7f1942000700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19240a1290 0x7f19240a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f193400f540 tx=0x7f1934009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.580+0000 7f1941600700 1 --2- 172.21.15.67:0/4203190922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15aee9b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1930097650 tx=0x7f193009afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0e60c2e1ae8012 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.580+0000 7f192e600700 1 -- 172.21.15.67:0/4203190922 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193009fea0 con 0x563f15aee5d0 2024-03-30T02:46:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.580+0000 7f192e600700 1 -- 172.21.15.67:0/4203190922 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f193009b850 con 0x563f15aee5d0 2024-03-30T02:46:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.580+0000 7f192e600700 1 -- 172.21.15.67:0/4203190922 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19300a1070 con 0x563f15aee5d0 2024-03-30T02:46:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 -- 172.21.15.67:0/4203190922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 msgr2=0x563f15aee9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/4203190922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15aee9b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1930097650 tx=0x7f193009afa0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 -- 172.21.15.67:0/4203190922 shutdown_connections 2024-03-30T02:46:35.584 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/4203190922 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15aee9b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/4203190922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15afc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/4203190922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 0x563f15af98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 -- 172.21.15.67:0/4203190922 >> 172.21.15.67:0/4203190922 conn(0x563f15a4e6a0 msgr2=0x563f15aeaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 -- 172.21.15.67:0/4203190922 shutdown_connections 2024-03-30T02:46:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 -- 172.21.15.67:0/4203190922 wait complete. 2024-03-30T02:46:35.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.581+0000 7f1957a4fb80 1 -- start start 2024-03-30T02:46:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1957a4fb80 1 Processor -- start 2024-03-30T02:46:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aee5d0 0x563f15af97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aeeef0 0x563f15af6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15af7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f159dcd80 con 0x563f15aeeef0 2024-03-30T02:46:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f159dca20 con 0x563f15aee5d0 2024-03-30T02:46:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f158c46a0 con 0x563f15af9df0 2024-03-30T02:46:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1957a4fb80 10 client.0 _ll_register_callbacks cb 0x563f159e6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1941600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aee5d0 0x563f15af97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1941600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aee5d0 0x563f15af97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:33982/0 (socket says 172.21.15.67:33982) 2024-03-30T02:46:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1941600700 1 -- 172.21.15.67:0/1713230046 learned_addr learned my addr 172.21.15.67:0/1713230046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1942000700 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15af7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.582+0000 7f1940c00700 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aeeef0 0x563f15af6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1941600700 1 -- 172.21.15.67:0/1713230046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 msgr2=0x563f15af7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1941600700 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15af7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1941600700 1 -- 172.21.15.67:0/1713230046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aeeef0 msgr2=0x563f15af6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1941600700 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aeeef0 0x563f15af6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1941600700 1 -- 172.21.15.67:0/1713230046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c1a6400 con 0x563f15aee5d0 2024-03-30T02:46:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1941600700 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aee5d0 0x563f15af97d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f193000f740 tx=0x7f1930022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0337b1426e087e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1922a00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:35.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1922a00700 1 -- 172.21.15.67:0/1713230046 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193009fea0 con 0x563f15aee5d0 2024-03-30T02:46:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1922a00700 1 -- 172.21.15.67:0/1713230046 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f193009b150 con 0x563f15aee5d0 2024-03-30T02:46:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1713230046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f15a01380 con 0x563f15aee5d0 2024-03-30T02:46:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1922a00700 1 -- 172.21.15.67:0/1713230046 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f193009ef80 con 0x563f15aee5d0 2024-03-30T02:46:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1957a4fb80 10 client.27500 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.583+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1713230046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563f1586eb60 con 0x563f15aee5d0 2024-03-30T02:46:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.584+0000 7f1922a00700 1 -- 172.21.15.67:0/1713230046 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f15a01380 con 0x563f15aee5d0 2024-03-30T02:46:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.584+0000 7f1922a00700 1 -- 172.21.15.67:0/1713230046 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f193009a640 con 0x563f15aee5d0 2024-03-30T02:46:35.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.584+0000 7f1957a4fb80 10 client.27500 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.584+0000 7f1957a4fb80 10 client.27500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.584+0000 7f1957a4fb80 20 client.27500 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.584+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563f15b091f0 0x563f15b0b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.584+0000 7f1957a4fb80 4 client.27500 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.584+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1713230046 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f15a39700 con 0x563f15b091f0 2024-03-30T02:46:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.584+0000 7f1940c00700 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563f15b091f0 0x563f15b0b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1940c00700 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563f15b091f0 0x563f15b0b5e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17c7659de8cb534 server_cookie=c0565d014a91bb11 in_seq=0 out_seq=0 2024-03-30T02:46:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1922a00700 10 client.27500 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1922a00700 1 -- 172.21.15.67:0/1713230046 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563f15a39700 con 0x563f15b091f0 2024-03-30T02:46:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1922a00700 10 client.27500 handle_command_reply: tid=0 2024-03-30T02:46:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1957a4fb80 1 client.27500 shutdown 2024-03-30T02:46:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1713230046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aee5d0 msgr2=0x563f15af97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aee5d0 0x563f15af97d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f193000f740 tx=0x7f1930022c90 comp rx=0 tx=0).stop 2024-03-30T02:46:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1713230046 shutdown_connections 2024-03-30T02:46:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15af7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aee5d0 0x563f15af97d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563f15b091f0 0x563f15b0b5e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1713230046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aeeef0 0x563f15af6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1922a00700 0 client.27500 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:35.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.585+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1713230046 >> 172.21.15.67:0/1713230046 conn(0x563f15a4e6a0 msgr2=0x563f15af06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:35.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.586+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1713230046 shutdown_connections 2024-03-30T02:46:35.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.586+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1713230046 wait complete. 2024-03-30T02:46:35.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.586+0000 7f1957a4fb80 20 client.27500 trim_cache size 0 max 16384 2024-03-30T02:46:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.603+0000 7f1957a4fb80 1 Processor -- start 2024-03-30T02:46:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.603+0000 7f1957a4fb80 1 -- start start 2024-03-30T02:46:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15aeeef0 0x563f15aea530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15af9df0 0x563f15aeaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f15a01380 con 0x563f15aee5d0 2024-03-30T02:46:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f15aef640 con 0x563f15af9df0 2024-03-30T02:46:35.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f15af7a30 con 0x563f15aeeef0 2024-03-30T02:46:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1941600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1941600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b007e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43656/0 (socket says 172.21.15.67:43656) 2024-03-30T02:46:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1941600700 1 -- 172.21.15.67:0/1364136656 learned_addr learned my addr 172.21.15.67:0/1364136656 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1942000700 1 --2- 172.21.15.67:0/1364136656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15af9df0 0x563f15aeaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1940c00700 1 --2- 172.21.15.67:0/1364136656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15aeeef0 0x563f15aea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1941600700 1 -- 172.21.15.67:0/1364136656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15aeeef0 msgr2=0x563f15aea530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1941600700 1 --2- 172.21.15.67:0/1364136656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15aeeef0 0x563f15aea530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1941600700 1 -- 172.21.15.67:0/1364136656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15af9df0 msgr2=0x563f15aeaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:35.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1941600700 1 --2- 172.21.15.67:0/1364136656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15af9df0 0x563f15aeaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.604+0000 7f1941600700 1 -- 172.21.15.67:0/1364136656 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f15a06ea0 con 0x563f15aee5d0 2024-03-30T02:46:35.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.605+0000 7f1941600700 1 --2- 172.21.15.67:0/1364136656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b007e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1930097650 tx=0x7f1930022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b4d1da17aae57 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:35.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.605+0000 7f192f000700 1 -- 172.21.15.67:0/1364136656 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193009fea0 con 0x563f15aee5d0 2024-03-30T02:46:35.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.605+0000 7f192f000700 1 -- 172.21.15.67:0/1364136656 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f193009ef80 con 0x563f15aee5d0 2024-03-30T02:46:35.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.605+0000 7f192f000700 1 -- 172.21.15.67:0/1364136656 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193009b150 con 0x563f15aee5d0 2024-03-30T02:46:35.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1364136656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 msgr2=0x563f15b007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1364136656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b007e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1930097650 tx=0x7f1930022f90 comp rx=0 tx=0).stop 2024-03-30T02:46:35.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1364136656 shutdown_connections 2024-03-30T02:46:35.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1364136656 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b007e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1364136656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15af9df0 0x563f15aeaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/1364136656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15aeeef0 0x563f15aea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1364136656 >> 172.21.15.67:0/1364136656 conn(0x563f15a4e6a0 msgr2=0x563f15ae3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:35.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1364136656 shutdown_connections 2024-03-30T02:46:35.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 -- 172.21.15.67:0/1364136656 wait complete. 2024-03-30T02:46:35.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 -- start start 2024-03-30T02:46:35.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.606+0000 7f1957a4fb80 1 Processor -- start 2024-03-30T02:46:35.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 0x563f15afe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1957a4fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15afef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f15aff460 con 0x563f15aee5d0 2024-03-30T02:46:35.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f15aff5d0 con 0x563f15aeeef0 2024-03-30T02:46:35.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1957a4fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f15aff790 con 0x563f15af9df0 2024-03-30T02:46:35.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1957a4fb80 10 client.0 _ll_register_callbacks cb 0x563f159e6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:35.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1941600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1941600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43668/0 (socket says 172.21.15.67:43668) 2024-03-30T02:46:35.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1941600700 1 -- 172.21.15.67:0/446932678 learned_addr learned my addr 172.21.15.67:0/446932678 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:35.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1942000700 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15afef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.607+0000 7f1940c00700 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 0x563f15afe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1941600700 1 -- 172.21.15.67:0/446932678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 msgr2=0x563f15afef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:35.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1941600700 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15afef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1941600700 1 -- 172.21.15.67:0/446932678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 msgr2=0x563f15afe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:35.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1941600700 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 0x563f15afe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1941600700 1 -- 172.21.15.67:0/446932678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f1586eb60 con 0x563f15aee5d0 2024-03-30T02:46:35.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1941600700 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b005a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f19300cb840 tx=0x7f19300cbc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46784411e9d4a9f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:35.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1922a00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:35.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1922a00700 1 -- 172.21.15.67:0/446932678 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193009fea0 con 0x563f15aee5d0 2024-03-30T02:46:35.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1922a00700 1 -- 172.21.15.67:0/446932678 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f19300a3100 con 0x563f15aee5d0 2024-03-30T02:46:35.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1957a4fb80 1 -- 172.21.15.67:0/446932678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f193009b850 con 0x563f15aee5d0 2024-03-30T02:46:35.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1922a00700 1 -- 172.21.15.67:0/446932678 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19300a3340 con 0x563f15aee5d0 2024-03-30T02:46:35.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1957a4fb80 10 client.17367 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:35.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.608+0000 7f1957a4fb80 1 -- 172.21.15.67:0/446932678 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563f158c6cd0 con 0x563f15aee5d0 2024-03-30T02:46:35.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.609+0000 7f1922a00700 1 -- 172.21.15.67:0/446932678 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19300ac020 con 0x563f15aee5d0 2024-03-30T02:46:35.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.609+0000 7f1922a00700 1 -- 172.21.15.67:0/446932678 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1930096f40 con 0x563f15aee5d0 2024-03-30T02:46:35.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.609+0000 7f1957a4fb80 10 client.17367 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:35.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.609+0000 7f1957a4fb80 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:35.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.609+0000 7f1957a4fb80 20 client.17367 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:35.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.609+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563f15b5dfd0 0x563f15b603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.609+0000 7f1957a4fb80 4 client.17367 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:46:35.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.609+0000 7f1957a4fb80 1 -- 172.21.15.67:0/446932678 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563f1588fcf0 con 0x563f15b5dfd0 2024-03-30T02:46:35.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.609+0000 7f1940c00700 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563f15b5dfd0 0x563f15b603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1940c00700 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563f15b5dfd0 0x563f15b603c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec11266903b1c82 server_cookie=a053a709a6aad6e4 in_seq=0 out_seq=0 2024-03-30T02:46:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1922a00700 10 client.17367 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1922a00700 1 -- 172.21.15.67:0/446932678 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563f1588fcf0 con 0x563f15b5dfd0 2024-03-30T02:46:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1922a00700 10 client.17367 handle_command_reply: tid=0 2024-03-30T02:46:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 client.17367 shutdown 2024-03-30T02:46:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 -- 172.21.15.67:0/446932678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 msgr2=0x563f15b005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b005a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f19300cb840 tx=0x7f19300cbc60 comp rx=0 tx=0).stop 2024-03-30T02:46:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 -- 172.21.15.67:0/446932678 shutdown_connections 2024-03-30T02:46:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x563f15b5dfd0 0x563f15b603c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f15aee5d0 0x563f15b005a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f15af9df0 0x563f15afef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 --2- 172.21.15.67:0/446932678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f15aeeef0 0x563f15afe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 -- 172.21.15.67:0/446932678 >> 172.21.15.67:0/446932678 conn(0x563f15a4e6a0 msgr2=0x563f15aea510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1922a00700 0 client.17367 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:35.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 -- 172.21.15.67:0/446932678 shutdown_connections 2024-03-30T02:46:35.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 1 -- 172.21.15.67:0/446932678 wait complete. 2024-03-30T02:46:35.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.610+0000 7f1957a4fb80 20 client.17367 trim_cache size 0 max 16384 2024-03-30T02:46:35.619 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:46:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:46:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:46:35.620 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:46:35.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 -- 172.21.15.67:0/2395373530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19240a1290 msgr2=0x7f19240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19240a1290 0x7f19240a3750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f193400f540 tx=0x7f1934009920 comp rx=0 tx=0).stop 2024-03-30T02:46:35.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 -- 172.21.15.67:0/2395373530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 msgr2=0x7f193c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 0x7f193c0730b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1930005740 tx=0x7f1930005770 comp rx=0 tx=0).stop 2024-03-30T02:46:35.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 -- 172.21.15.67:0/2395373530 shutdown_connections 2024-03-30T02:46:35.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f19240a1290 0x7f19240a3750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f193c073b30 0x7f193c1a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f193c100f60 0x7f193c0730b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 --2- 172.21.15.67:0/2395373530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f193c106080 0x7f193c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 -- 172.21.15.67:0/2395373530 >> 172.21.15.67:0/2395373530 conn(0x7f193c0fc7c0 msgr2=0x7f193c0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:35.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 -- 172.21.15.67:0/2395373530 shutdown_connections 2024-03-30T02:46:35.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.611+0000 7f1944000700 1 -- 172.21.15.67:0/2395373530 wait complete. 2024-03-30T02:46:35.629 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:46:35.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:46:35.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.829+0000 7fd939600700 1 -- 172.21.15.67:0/3099429844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd934101810 msgr2=0x7fd934105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.829+0000 7fd939600700 1 --2- 172.21.15.67:0/3099429844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd934101810 0x7fd934105850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd9200098f0 tx=0x7fd920009c00 comp rx=0 tx=0).stop 2024-03-30T02:46:35.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.829+0000 7fd939600700 1 -- 172.21.15.67:0/3099429844 shutdown_connections 2024-03-30T02:46:35.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.829+0000 7fd939600700 1 --2- 172.21.15.67:0/3099429844 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd934105f80 0x7fd93410db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.829+0000 7fd939600700 1 --2- 172.21.15.67:0/3099429844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd934100e60 0x7fd934101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.829+0000 7fd939600700 1 --2- 172.21.15.67:0/3099429844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd934101810 0x7fd934105850 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.829+0000 7fd939600700 1 -- 172.21.15.67:0/3099429844 >> 172.21.15.67:0/3099429844 conn(0x7fd9340fc700 msgr2=0x7fd9340feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.830+0000 7fd939600700 1 -- 172.21.15.67:0/3099429844 shutdown_connections 2024-03-30T02:46:35.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.830+0000 7fd939600700 1 -- 172.21.15.67:0/3099429844 wait complete. 2024-03-30T02:46:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.831+0000 7fd939600700 1 Processor -- start 2024-03-30T02:46:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.831+0000 7fd939600700 1 -- start start 2024-03-30T02:46:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.831+0000 7fd939600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd934100e60 0x7fd9340730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.831+0000 7fd939600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd934105f80 0x7fd9340735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.831+0000 7fd939600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd934073b30 0x7fd9341a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.831+0000 7fd939600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341a5990 con 0x7fd934073b30 2024-03-30T02:46:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.831+0000 7fd939600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341a5ad0 con 0x7fd934100e60 2024-03-30T02:46:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.831+0000 7fd939600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341a5c90 con 0x7fd934105f80 2024-03-30T02:46:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd933400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd934073b30 0x7fd9341a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd934100e60 0x7fd9340730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd934105f80 0x7fd9340735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd934100e60 0x7fd9340730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40498/0 (socket says 172.21.15.67:40498) 2024-03-30T02:46:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd933400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd934073b30 0x7fd9341a53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39666/0 (socket says 172.21.15.67:39666) 2024-03-30T02:46:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932a00700 1 -- 172.21.15.67:0/1828966725 learned_addr learned my addr 172.21.15.67:0/1828966725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932000700 1 -- 172.21.15.67:0/1828966725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd934100e60 msgr2=0x7fd9340730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932000700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd934100e60 0x7fd9340730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932000700 1 -- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd934073b30 msgr2=0x7fd9341a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932000700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd934073b30 0x7fd9341a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:35.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932000700 1 -- 172.21.15.67:0/1828966725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c009d00 con 0x7fd934105f80 2024-03-30T02:46:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd933400700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd934073b30 0x7fd9341a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.832+0000 7fd932000700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd934105f80 0x7fd9340735f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd920000c00 tx=0x7fd92000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.833+0000 7fd92b400700 1 -- 172.21.15.67:0/1828966725 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92001c070 con 0x7fd934105f80 2024-03-30T02:46:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.833+0000 7fd92b400700 1 -- 172.21.15.67:0/1828966725 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd92000b9c0 con 0x7fd934105f80 2024-03-30T02:46:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.833+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd920009520 con 0x7fd934105f80 2024-03-30T02:46:35.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.833+0000 7fd92b400700 1 -- 172.21.15.67:0/1828966725 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9200038e0 con 0x7fd934105f80 2024-03-30T02:46:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.833+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9341a61a0 con 0x7fd934105f80 2024-03-30T02:46:35.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.834+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd934109250 con 0x7fd934105f80 2024-03-30T02:46:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.836+0000 7fd92b400700 1 -- 172.21.15.67:0/1828966725 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd920020030 con 0x7fd934105f80 2024-03-30T02:46:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.837+0000 7fd92b400700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd9180a1290 0x7fd9180a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.837+0000 7fd92b400700 1 -- 172.21.15.67:0/1828966725 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fd9200ce060 con 0x7fd934105f80 2024-03-30T02:46:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.837+0000 7fd932a00700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd9180a1290 0x7fd9180a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:35.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.838+0000 7fd932a00700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd9180a1290 0x7fd9180a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd924009d20 tx=0x7fd924009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:35.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:35.840+0000 7fd92b400700 1 -- 172.21.15.67:0/1828966725 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd920097140 con 0x7fd934105f80 2024-03-30T02:46:36.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.102+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd934102330 con 0x7fd934105f80 2024-03-30T02:46:36.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.103+0000 7fd92b400700 1 -- 172.21.15.67:0/1828966725 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10909 (secure 0 0 0) 0x7fd92009a760 con 0x7fd934105f80 2024-03-30T02:46:36.103 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:46:36.104 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:45:07.400846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:46:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd9180a1290 msgr2=0x7fd9180a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd9180a1290 0x7fd9180a3750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd924009d20 tx=0x7fd924009260 comp rx=0 tx=0).stop 2024-03-30T02:46:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd934105f80 msgr2=0x7fd9340735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd934105f80 0x7fd9340735f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd920000c00 tx=0x7fd92000fa00 comp rx=0 tx=0).stop 2024-03-30T02:46:36.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 shutdown_connections 2024-03-30T02:46:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd9180a1290 0x7fd9180a3750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd934073b30 0x7fd9341a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd934100e60 0x7fd9340730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 --2- 172.21.15.67:0/1828966725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd934105f80 0x7fd9340735f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 >> 172.21.15.67:0/1828966725 conn(0x7fd9340fc700 msgr2=0x7fd9340fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 shutdown_connections 2024-03-30T02:46:36.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.106+0000 7fd939600700 1 -- 172.21.15.67:0/1828966725 wait complete. 2024-03-30T02:46:36.108 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-03-30T02:46:36.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:46:36.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.326+0000 7f5eef800700 1 -- 172.21.15.67:0/3161175250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 msgr2=0x7f5ee81060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.326+0000 7f5eef800700 1 --2- 172.21.15.67:0/3161175250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 0x7f5ee81060c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009990 tx=0x7f5edc009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.326+0000 7f5eef800700 1 -- 172.21.15.67:0/3161175250 shutdown_connections 2024-03-30T02:46:36.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.326+0000 7f5eef800700 1 --2- 172.21.15.67:0/3161175250 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 0x7f5ee81060c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.326+0000 7f5eef800700 1 --2- 172.21.15.67:0/3161175250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee80687f0 0x7f5ee810db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.327 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.326+0000 7f5eef800700 1 --2- 172.21.15.67:0/3161175250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ee8067e30 0x7f5ee80682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.326+0000 7f5eef800700 1 -- 172.21.15.67:0/3161175250 >> 172.21.15.67:0/3161175250 conn(0x7f5ee8076d40 msgr2=0x7f5ee8077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.326+0000 7f5eef800700 1 -- 172.21.15.67:0/3161175250 shutdown_connections 2024-03-30T02:46:36.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.326+0000 7f5eef800700 1 -- 172.21.15.67:0/3161175250 wait complete. 2024-03-30T02:46:36.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.327+0000 7f5eef800700 1 Processor -- start 2024-03-30T02:46:36.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.327+0000 7f5eef800700 1 -- start start 2024-03-30T02:46:36.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eef800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ee8067e30 0x7f5ee80ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eef800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee80687f0 0x7f5ee81001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eef800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 0x7f5ee81006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eef800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee81a5600 con 0x7f5ee8105ce0 2024-03-30T02:46:36.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eef800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee81a5770 con 0x7f5ee8067e30 2024-03-30T02:46:36.330 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eef800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee81a5930 con 0x7f5ee80687f0 2024-03-30T02:46:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eed800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 0x7f5ee81006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eed800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 0x7f5ee81006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39686/0 (socket says 172.21.15.67:39686) 2024-03-30T02:46:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eed800700 1 -- 172.21.15.67:0/2199879562 learned_addr learned my addr 172.21.15.67:0/2199879562 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5eece00700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ee8067e30 0x7f5ee80ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.328+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee80687f0 0x7f5ee81001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5eed800700 1 -- 172.21.15.67:0/2199879562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee80687f0 msgr2=0x7f5ee81001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5eed800700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee80687f0 0x7f5ee81001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5eed800700 1 -- 172.21.15.67:0/2199879562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ee8067e30 msgr2=0x7f5ee80ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5eed800700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ee8067e30 0x7f5ee80ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5eed800700 1 -- 172.21.15.67:0/2199879562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc0095f0 con 0x7f5ee8105ce0 2024-03-30T02:46:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5eece00700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ee8067e30 0x7f5ee80ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5eed800700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 0x7f5ee81006f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000ef30 tx=0x7f5ee000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5ee5600700 1 -- 172.21.15.67:0/2199879562 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0018070 con 0x7f5ee8105ce0 2024-03-30T02:46:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5ee5600700 1 -- 172.21.15.67:0/2199879562 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee000f040 con 0x7f5ee8105ce0 2024-03-30T02:46:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5eef800700 1 -- 172.21.15.67:0/2199879562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee81a5bb0 con 0x7f5ee8105ce0 2024-03-30T02:46:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5ee5600700 1 -- 172.21.15.67:0/2199879562 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000cd00 con 0x7f5ee8105ce0 2024-03-30T02:46:36.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.329+0000 7f5eef800700 1 -- 172.21.15.67:0/2199879562 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee81a60c0 con 0x7f5ee8105ce0 2024-03-30T02:46:36.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.334+0000 7f5ee5600700 1 -- 172.21.15.67:0/2199879562 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5ee0003a00 con 0x7f5ee8105ce0 2024-03-30T02:46:36.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5f03273b80 1 Processor -- start 2024-03-30T02:46:36.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5f03273b80 1 -- start start 2024-03-30T02:46:36.335 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5ee5600700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5ed00a1290 0x7f5ed00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 0x55e9f28688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f286b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5f03273b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f27b6990 con 0x55e9f285d5d0 2024-03-30T02:46:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5f03273b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f27b4350 con 0x55e9f2868df0 2024-03-30T02:46:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5f03273b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f27afc40 con 0x55e9f285def0 2024-03-30T02:46:36.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5ee5600700 1 -- 172.21.15.67:0/2199879562 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f5ee00ccbe0 con 0x7f5ee8105ce0 2024-03-30T02:46:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5ed00a1290 0x7f5ed00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.335+0000 7f5eece00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eece00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39696/0 (socket says 172.21.15.67:39696) 2024-03-30T02:46:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eece00700 1 -- 172.21.15.67:0/263562446 learned_addr learned my addr 172.21.15.67:0/263562446 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eed800700 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f286b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 0x55e9f28688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eece00700 1 -- 172.21.15.67:0/263562446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 msgr2=0x55e9f28688b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eece00700 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 0x55e9f28688b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eece00700 1 -- 172.21.15.67:0/263562446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 msgr2=0x55e9f286b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eece00700 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f286b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eece00700 1 -- 172.21.15.67:0/263562446 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f26313b0 con 0x55e9f285d5d0 2024-03-30T02:46:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eed800700 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f286b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5ed00a1290 0x7f5ed00a3750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009fd0 tx=0x7f5ed4009b40 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.336+0000 7f5eece00700 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5edc005f00 tx=0x7f5edc00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd368d243c2d33a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5eda000700 1 -- 172.21.15.67:0/263562446 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc01d070 con 0x55e9f285d5d0 2024-03-30T02:46:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5eda000700 1 -- 172.21.15.67:0/263562446 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5edc01e040 con 0x55e9f285d5d0 2024-03-30T02:46:36.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5eda000700 1 -- 172.21.15.67:0/263562446 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc004b60 con 0x55e9f285d5d0 2024-03-30T02:46:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5f03273b80 1 -- 172.21.15.67:0/263562446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 msgr2=0x55e9f285d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5f03273b80 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5edc005f00 tx=0x7f5edc00b610 comp rx=0 tx=0).stop 2024-03-30T02:46:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5f03273b80 1 -- 172.21.15.67:0/263562446 shutdown_connections 2024-03-30T02:46:36.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5f03273b80 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5f03273b80 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f286b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5f03273b80 1 --2- 172.21.15.67:0/263562446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 0x55e9f28688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.337+0000 7f5f03273b80 1 -- 172.21.15.67:0/263562446 >> 172.21.15.67:0/263562446 conn(0x55e9f27bd6a0 msgr2=0x55e9f2859a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.338+0000 7f5f03273b80 1 -- 172.21.15.67:0/263562446 shutdown_connections 2024-03-30T02:46:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.338+0000 7f5f03273b80 1 -- 172.21.15.67:0/263562446 wait complete. 2024-03-30T02:46:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.338+0000 7f5f03273b80 1 -- start start 2024-03-30T02:46:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.338+0000 7f5f03273b80 1 Processor -- start 2024-03-30T02:46:36.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f2868680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 0x55e9f2865a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f2865fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5f03273b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f274bd80 con 0x55e9f285d5d0 2024-03-30T02:46:36.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5f03273b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f274ba20 con 0x55e9f2868df0 2024-03-30T02:46:36.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5f03273b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f26336a0 con 0x55e9f285def0 2024-03-30T02:46:36.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5f03273b80 10 client.0 _ll_register_callbacks cb 0x55e9f2755470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5eece00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f2868680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5eece00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f2868680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39710/0 (socket says 172.21.15.67:39710) 2024-03-30T02:46:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5eece00700 1 -- 172.21.15.67:0/1380214202 learned_addr learned my addr 172.21.15.67:0/1380214202 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5eed800700 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f2865fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 0x55e9f2865a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5eece00700 1 -- 172.21.15.67:0/1380214202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 msgr2=0x55e9f2865a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5eece00700 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 0x55e9f2865a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5eece00700 1 -- 172.21.15.67:0/1380214202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 msgr2=0x55e9f2865fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5eece00700 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f2865fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.339+0000 7f5eece00700 1 -- 172.21.15.67:0/1380214202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee81a60c0 con 0x55e9f285d5d0 2024-03-30T02:46:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.340+0000 7f5eece00700 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f2868680 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5edc004780 tx=0x7f5edc00fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cfec2461f46724 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.340+0000 7f5ecea00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.340+0000 7f5ecea00700 1 -- 172.21.15.67:0/1380214202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc01d040 con 0x55e9f285d5d0 2024-03-30T02:46:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.340+0000 7f5ecea00700 1 -- 172.21.15.67:0/1380214202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f5edc0058e0 con 0x55e9f285d5d0 2024-03-30T02:46:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.340+0000 7f5f03273b80 1 -- 172.21.15.67:0/1380214202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9f2770380 con 0x55e9f285d5d0 2024-03-30T02:46:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.340+0000 7f5ecea00700 1 -- 172.21.15.67:0/1380214202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5edc00fe80 con 0x55e9f285d5d0 2024-03-30T02:46:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.340+0000 7f5f03273b80 10 client.17385 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:36.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.340+0000 7f5f03273b80 1 -- 172.21.15.67:0/1380214202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f5ee810e550 con 0x55e9f285d5d0 2024-03-30T02:46:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.340+0000 7f5ecea00700 1 -- 172.21.15.67:0/1380214202 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e9f2770380 con 0x55e9f285d5d0 2024-03-30T02:46:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.341+0000 7f5ecea00700 1 -- 172.21.15.67:0/1380214202 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5edc0173f0 con 0x55e9f285d5d0 2024-03-30T02:46:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.341+0000 7f5f03273b80 10 client.17385 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.341+0000 7f5f03273b80 10 client.17385 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:46:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.341+0000 7f5f03273b80 20 client.17385 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.341+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e9f2878430 0x55e9f287a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.341+0000 7f5f03273b80 4 client.17385 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:36.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.341+0000 7f5f03273b80 1 -- 172.21.15.67:0/1380214202 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9f27a8700 con 0x55e9f2878430 2024-03-30T02:46:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.341+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e9f2878430 0x55e9f287a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.342+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e9f2878430 0x55e9f287a820 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7aa924160289505 server_cookie=6d2ca7ce68c8e56c in_seq=0 out_seq=0 2024-03-30T02:46:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.342+0000 7f5ecea00700 10 client.17385 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:46:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5ecea00700 1 -- 172.21.15.67:0/1380214202 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9f27a8700 con 0x55e9f2878430 2024-03-30T02:46:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5ecea00700 10 client.17385 handle_command_reply: tid=0 2024-03-30T02:46:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 client.17385 shutdown 2024-03-30T02:46:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 -- 172.21.15.67:0/1380214202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 msgr2=0x55e9f2868680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f2868680 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5edc004780 tx=0x7f5edc00fb60 comp rx=0 tx=0).stop 2024-03-30T02:46:36.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 -- 172.21.15.67:0/1380214202 shutdown_connections 2024-03-30T02:46:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f2868680 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e9f2878430 0x55e9f287a820 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f2865fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1380214202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285def0 0x55e9f2865a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5ecea00700 0 client.17385 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:46:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 -- 172.21.15.67:0/1380214202 >> 172.21.15.67:0/1380214202 conn(0x55e9f27bd6a0 msgr2=0x55e9f285f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 -- 172.21.15.67:0/1380214202 shutdown_connections 2024-03-30T02:46:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 1 -- 172.21.15.67:0/1380214202 wait complete. 2024-03-30T02:46:36.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.343+0000 7f5f03273b80 20 client.17385 trim_cache size 0 max 16384 2024-03-30T02:46:36.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.357+0000 7f5f03273b80 1 Processor -- start 2024-03-30T02:46:36.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.357+0000 7f5f03273b80 1 -- start start 2024-03-30T02:46:36.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.357+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.357+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f285def0 0x55e9f2864350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.357+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f2868df0 0x55e9f2864890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.357+0000 7f5f03273b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f28753c0 con 0x55e9f285d5d0 2024-03-30T02:46:36.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.357+0000 7f5f03273b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f285e850 con 0x55e9f285def0 2024-03-30T02:46:36.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.357+0000 7f5f03273b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f2864dd0 con 0x55e9f2868df0 2024-03-30T02:46:36.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eece00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eece00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39716/0 (socket says 172.21.15.67:39716) 2024-03-30T02:46:36.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eece00700 1 -- 172.21.15.67:0/2672151395 learned_addr learned my addr 172.21.15.67:0/2672151395 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f285def0 0x55e9f2864350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eed800700 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f2868df0 0x55e9f2864890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eece00700 1 -- 172.21.15.67:0/2672151395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f2868df0 msgr2=0x55e9f2864890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eece00700 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f2868df0 0x55e9f2864890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eece00700 1 -- 172.21.15.67:0/2672151395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f285def0 msgr2=0x55e9f2864350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eece00700 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f285def0 0x55e9f2864350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eece00700 1 -- 172.21.15.67:0/2672151395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f25ddb60 con 0x55e9f285d5d0 2024-03-30T02:46:36.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f285def0 0x55e9f2864350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.358+0000 7f5eece00700 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009ce0 tx=0x7f5edc017950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8573ede4e668009 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5edaa00700 1 -- 172.21.15.67:0/2672151395 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0045c0 con 0x55e9f285d5d0 2024-03-30T02:46:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5edaa00700 1 -- 172.21.15.67:0/2672151395 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5edc021820 con 0x55e9f285d5d0 2024-03-30T02:46:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5edaa00700 1 -- 172.21.15.67:0/2672151395 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc021c40 con 0x55e9f285d5d0 2024-03-30T02:46:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5f03273b80 1 -- 172.21.15.67:0/2672151395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 msgr2=0x55e9f285d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5f03273b80 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009ce0 tx=0x7f5edc017950 comp rx=0 tx=0).stop 2024-03-30T02:46:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5f03273b80 1 -- 172.21.15.67:0/2672151395 shutdown_connections 2024-03-30T02:46:36.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5f03273b80 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285d5d0 0x55e9f285d9b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5f03273b80 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f2868df0 0x55e9f2864890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5f03273b80 1 --2- 172.21.15.67:0/2672151395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f285def0 0x55e9f2864350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5f03273b80 1 -- 172.21.15.67:0/2672151395 >> 172.21.15.67:0/2672151395 conn(0x55e9f27bd6a0 msgr2=0x55e9f2852f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5f03273b80 1 -- 172.21.15.67:0/2672151395 shutdown_connections 2024-03-30T02:46:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.359+0000 7f5f03273b80 1 -- 172.21.15.67:0/2672151395 wait complete. 2024-03-30T02:46:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.360+0000 7f5f03273b80 1 -- start start 2024-03-30T02:46:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.360+0000 7f5f03273b80 1 Processor -- start 2024-03-30T02:46:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.360+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285d5d0 0x55e9f286d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.360+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285def0 0x55e9f286d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.360+0000 7f5f03273b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f286de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.360+0000 7f5f03273b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f28cb280 con 0x55e9f285def0 2024-03-30T02:46:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.360+0000 7f5f03273b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f28cb3f0 con 0x55e9f2868df0 2024-03-30T02:46:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.360+0000 7f5f03273b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f28cb5b0 con 0x55e9f285d5d0 2024-03-30T02:46:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5f03273b80 10 client.0 _ll_register_callbacks cb 0x55e9f2755470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5ee7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285def0 0x55e9f286d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5eece00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285d5d0 0x55e9f286d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5ee7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285def0 0x55e9f286d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39728/0 (socket says 172.21.15.67:39728) 2024-03-30T02:46:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5ee7e00700 1 -- 172.21.15.67:0/1438203331 learned_addr learned my addr 172.21.15.67:0/1438203331 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5eed800700 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f286de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5eece00700 1 -- 172.21.15.67:0/1438203331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 msgr2=0x55e9f286de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5eece00700 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f286de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5eece00700 1 -- 172.21.15.67:0/1438203331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285def0 msgr2=0x55e9f286d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5eece00700 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285def0 0x55e9f286d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5eece00700 1 -- 172.21.15.67:0/1438203331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed4013e70 con 0x55e9f285d5d0 2024-03-30T02:46:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285def0 0x55e9f286d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:36.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5eece00700 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285d5d0 0x55e9f286d3a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5edc032030 tx=0x7f5edc021760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25ddae236230e941 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.361+0000 7f5ecea00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:46:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.362+0000 7f5ecea00700 1 -- 172.21.15.67:0/1438203331 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc01d030 con 0x55e9f285d5d0 2024-03-30T02:46:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.362+0000 7f5f03273b80 1 -- 172.21.15.67:0/1438203331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5edc01e040 con 0x55e9f285d5d0 2024-03-30T02:46:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.362+0000 7f5ecea00700 1 -- 172.21.15.67:0/1438203331 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f5edc000e40 con 0x55e9f285d5d0 2024-03-30T02:46:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.362+0000 7f5ecea00700 1 -- 172.21.15.67:0/1438203331 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5edc017a20 con 0x55e9f285d5d0 2024-03-30T02:46:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.362+0000 7f5f03273b80 10 client.27463 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.362+0000 7f5f03273b80 1 -- 172.21.15.67:0/1438203331 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f5ee810e550 con 0x55e9f285d5d0 2024-03-30T02:46:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.362+0000 7f5ecea00700 1 -- 172.21.15.67:0/1438203331 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5edc01e040 con 0x55e9f285d5d0 2024-03-30T02:46:36.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.362+0000 7f5ecea00700 1 -- 172.21.15.67:0/1438203331 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5edc049080 con 0x55e9f285d5d0 2024-03-30T02:46:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.363+0000 7f5f03273b80 10 client.27463 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.363+0000 7f5f03273b80 10 client.27463 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:46:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.363+0000 7f5f03273b80 20 client.27463 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.363+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e9f28cc860 0x55e9f28cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.363+0000 7f5f03273b80 4 client.27463 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.363+0000 7f5f03273b80 1 -- 172.21.15.67:0/1438203331 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e9f25fecf0 con 0x55e9f28cc860 2024-03-30T02:46:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.363+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e9f28cc860 0x55e9f28cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.363+0000 7f5ee7e00700 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e9f28cc860 0x55e9f28cec80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d03b1693eda5a8dd server_cookie=347fd34a1612aa8a in_seq=0 out_seq=0 2024-03-30T02:46:36.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.363+0000 7f5ecea00700 10 client.27463 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:46:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5ecea00700 1 -- 172.21.15.67:0/1438203331 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e9f25fecf0 con 0x55e9f28cc860 2024-03-30T02:46:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5ecea00700 10 client.27463 handle_command_reply: tid=0 2024-03-30T02:46:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 client.27463 shutdown 2024-03-30T02:46:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 -- 172.21.15.67:0/1438203331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285d5d0 msgr2=0x55e9f286d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285d5d0 0x55e9f286d3a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5edc032030 tx=0x7f5edc021760 comp rx=0 tx=0).stop 2024-03-30T02:46:36.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 -- 172.21.15.67:0/1438203331 shutdown_connections 2024-03-30T02:46:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x55e9f28cc860 0x55e9f28cec80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f2868df0 0x55e9f286de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9f285d5d0 0x55e9f286d3a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 --2- 172.21.15.67:0/1438203331 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e9f285def0 0x55e9f286d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5ecea00700 0 client.27463 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:46:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 -- 172.21.15.67:0/1438203331 >> 172.21.15.67:0/1438203331 conn(0x55e9f27bd6a0 msgr2=0x55e9f28642d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 -- 172.21.15.67:0/1438203331 shutdown_connections 2024-03-30T02:46:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 1 -- 172.21.15.67:0/1438203331 wait complete. 2024-03-30T02:46:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.364+0000 7f5f03273b80 20 client.27463 trim_cache size 0 max 16384 2024-03-30T02:46:36.373 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 -- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5ed00a1290 msgr2=0x7f5ed00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5ed00a1290 0x7f5ed00a3750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009fd0 tx=0x7f5ed4009b40 comp rx=0 tx=0).stop 2024-03-30T02:46:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 -- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 msgr2=0x7f5ee81006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 0x7f5ee81006f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000ef30 tx=0x7f5ee000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 -- 172.21.15.67:0/2199879562 shutdown_connections 2024-03-30T02:46:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5ed00a1290 0x7f5ed00a3750 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5ee8105ce0 0x7f5ee81006f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ee8067e30 0x7f5ee80ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 --2- 172.21.15.67:0/2199879562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee80687f0 0x7f5ee81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 -- 172.21.15.67:0/2199879562 >> 172.21.15.67:0/2199879562 conn(0x7f5ee8076d40 msgr2=0x7f5ee80fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 -- 172.21.15.67:0/2199879562 shutdown_connections 2024-03-30T02:46:36.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.365+0000 7f5eef800700 1 -- 172.21.15.67:0/2199879562 wait complete. 2024-03-30T02:46:36.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:46:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:36 smithi067 ceph-mon[37808]: pgmap v295: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-30T02:46:36.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:36 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1828966725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:46:36.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.680+0000 7f66a7c00700 1 -- 172.21.15.67:0/3146590729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0106060 msgr2=0x7f66a010dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.680+0000 7f66a7c00700 1 --2- 172.21.15.67:0/3146590729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0106060 0x7f66a010dbe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6698009f90 tx=0x7f669800b330 comp rx=0 tx=0).stop 2024-03-30T02:46:36.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.680+0000 7f66a7c00700 1 -- 172.21.15.67:0/3146590729 shutdown_connections 2024-03-30T02:46:36.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.680+0000 7f66a7c00700 1 --2- 172.21.15.67:0/3146590729 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a0100f40 0x7f66a0101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.680+0000 7f66a7c00700 1 --2- 172.21.15.67:0/3146590729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0106060 0x7f66a010dbe0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.680+0000 7f66a7c00700 1 --2- 172.21.15.67:0/3146590729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a01018f0 0x7f66a0105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.680+0000 7f66a7c00700 1 -- 172.21.15.67:0/3146590729 >> 172.21.15.67:0/3146590729 conn(0x7f66a00fc7c0 msgr2=0x7f66a00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.681+0000 7f66a7c00700 1 -- 172.21.15.67:0/3146590729 shutdown_connections 2024-03-30T02:46:36.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.681+0000 7f66a7c00700 1 -- 172.21.15.67:0/3146590729 wait complete. 2024-03-30T02:46:36.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.682+0000 7f66a7c00700 1 Processor -- start 2024-03-30T02:46:36.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.683+0000 7f66a7c00700 1 -- start start 2024-03-30T02:46:36.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.683+0000 7f66a7c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100f40 0x7f66a00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.683+0000 7f66a7c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a0106060 0x7f66a00735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.683+0000 7f66a7c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a0073b30 0x7f66a01a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.683+0000 7f66a7c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a01a58a0 con 0x7f66a0106060 2024-03-30T02:46:36.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.683+0000 7f66a7c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a01a5a10 con 0x7f66a0100f40 2024-03-30T02:46:36.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.683+0000 7f66a7c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a01a5bd0 con 0x7f66a0073b30 2024-03-30T02:46:36.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.684+0000 7f66a5c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a0073b30 0x7f66a01a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.684+0000 7f669fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a0106060 0x7f66a00735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.684+0000 7f66a5c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a0073b30 0x7f66a01a52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44286/0 (socket says 172.21.15.67:44286) 2024-03-30T02:46:36.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.684+0000 7f66a5c00700 1 -- 172.21.15.67:0/50412131 learned_addr learned my addr 172.21.15.67:0/50412131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.686+0000 7f669fe00700 1 -- 172.21.15.67:0/50412131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a0073b30 msgr2=0x7f66a01a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.686+0000 7f669fe00700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a0073b30 0x7f66a01a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.686+0000 7f669fe00700 1 -- 172.21.15.67:0/50412131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100f40 msgr2=0x7f66a00730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.686+0000 7f669fe00700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100f40 0x7f66a00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.686+0000 7f669fe00700 1 -- 172.21.15.67:0/50412131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6698009c40 con 0x7f66a0106060 2024-03-30T02:46:36.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.688+0000 7f669fe00700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a0106060 0x7f66a00735f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f66a006a390 tx=0x7f669400dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.688+0000 7f669d600700 1 -- 172.21.15.67:0/50412131 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694009940 con 0x7f66a0106060 2024-03-30T02:46:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.688+0000 7f66a7c00700 1 -- 172.21.15.67:0/50412131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a01a5eb0 con 0x7f66a0106060 2024-03-30T02:46:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.688+0000 7f669d600700 1 -- 172.21.15.67:0/50412131 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6694010460 con 0x7f66a0106060 2024-03-30T02:46:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.688+0000 7f669d600700 1 -- 172.21.15.67:0/50412131 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669400b5d0 con 0x7f66a0106060 2024-03-30T02:46:36.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.689+0000 7f66a7c00700 1 -- 172.21.15.67:0/50412131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a01a6380 con 0x7f66a0106060 2024-03-30T02:46:36.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.698+0000 7f669d600700 1 -- 172.21.15.67:0/50412131 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f66940105d0 con 0x7f66a0106060 2024-03-30T02:46:36.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.699+0000 7f66bb6c5b80 1 Processor -- start 2024-03-30T02:46:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.699+0000 7f66bb6c5b80 1 -- start start 2024-03-30T02:46:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.700+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f31280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.700+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f317c0 0x55ebd6f2e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.700+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 0x55ebd6f47470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.700+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6e81990 con 0x55ebd6f317c0 2024-03-30T02:46:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.700+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6e7f350 con 0x55ebd6f45080 2024-03-30T02:46:36.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.700+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6e7ac40 con 0x55ebd6f30ea0 2024-03-30T02:46:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.700+0000 7f669fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 0x55ebd6f47470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.700+0000 7f669fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 0x55ebd6f47470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40574/0 (socket says 172.21.15.67:40574) 2024-03-30T02:46:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.700+0000 7f669fe00700 1 -- 172.21.15.67:0/891394295 learned_addr learned my addr 172.21.15.67:0/891394295 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.701+0000 7f66a5c00700 1 --2- 172.21.15.67:0/891394295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f31280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.702+0000 7f66a5200700 1 --2- 172.21.15.67:0/891394295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f317c0 0x55ebd6f2e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.702+0000 7f66a5c00700 1 -- 172.21.15.67:0/891394295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 msgr2=0x55ebd6f47470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.702+0000 7f66a5c00700 1 --2- 172.21.15.67:0/891394295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 0x55ebd6f47470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.702+0000 7f66a5c00700 1 -- 172.21.15.67:0/891394295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f317c0 msgr2=0x55ebd6f2e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.702+0000 7f66a5c00700 1 --2- 172.21.15.67:0/891394295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f317c0 0x55ebd6f2e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.702+0000 7f66a5c00700 1 -- 172.21.15.67:0/891394295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebd6cfc3b0 con 0x55ebd6f30ea0 2024-03-30T02:46:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.703+0000 7f66a5c00700 1 --2- 172.21.15.67:0/891394295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f31280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f669800b770 tx=0x7f6698007b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ce78ed87908c571 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.704+0000 7f6692000700 1 -- 172.21.15.67:0/891394295 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669800e030 con 0x55ebd6f30ea0 2024-03-30T02:46:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.704+0000 7f6692000700 1 -- 172.21.15.67:0/891394295 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6698003a00 con 0x55ebd6f30ea0 2024-03-30T02:46:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.704+0000 7f6692000700 1 -- 172.21.15.67:0/891394295 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698005040 con 0x55ebd6f30ea0 2024-03-30T02:46:36.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.704+0000 7f669d600700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f66880a1310 0x7f66880a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.704+0000 7f669d600700 1 -- 172.21.15.67:0/50412131 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f66940ccf50 con 0x7f66a0106060 2024-03-30T02:46:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.705+0000 7f66a5200700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f66880a1310 0x7f66880a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.705+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/891394295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f30ea0 msgr2=0x55ebd6f31280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.705+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/891394295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f31280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f669800b770 tx=0x7f6698007b20 comp rx=0 tx=0).stop 2024-03-30T02:46:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.706+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/891394295 shutdown_connections 2024-03-30T02:46:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.706+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/891394295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 0x55ebd6f47470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.706+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/891394295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f31280 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.706+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/891394295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f317c0 0x55ebd6f2e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.706+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/891394295 >> 172.21.15.67:0/891394295 conn(0x55ebd6e886a0 msgr2=0x55ebd6f24a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.707+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/891394295 shutdown_connections 2024-03-30T02:46:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.707+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/891394295 wait complete. 2024-03-30T02:46:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.708+0000 7f66bb6c5b80 1 -- start start 2024-03-30T02:46:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.708+0000 7f66bb6c5b80 1 Processor -- start 2024-03-30T02:46:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.708+0000 7f66a5200700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f66880a1310 0x7f66880a37d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f668c005d90 tx=0x7f668c00b2c0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.710+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f34260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.710+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.710+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 0x55ebd6f3f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.710+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6e16d80 con 0x55ebd6f30ea0 2024-03-30T02:46:36.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.710+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6e16a20 con 0x55ebd6f45080 2024-03-30T02:46:36.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.710+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6cfe6a0 con 0x55ebd6f317c0 2024-03-30T02:46:36.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.710+0000 7f66bb6c5b80 10 client.0 _ll_register_callbacks cb 0x55ebd6e20470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:36.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.710+0000 7f66a5200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f34260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.711+0000 7f66a5200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f34260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39760/0 (socket says 172.21.15.67:39760) 2024-03-30T02:46:36.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.712+0000 7f669fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.712+0000 7f669fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f347a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44310/0 (socket says 172.21.15.67:44310) 2024-03-30T02:46:36.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.712+0000 7f669fe00700 1 -- 172.21.15.67:0/4075526662 learned_addr learned my addr 172.21.15.67:0/4075526662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.712+0000 7f66a5200700 1 -- 172.21.15.67:0/4075526662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 msgr2=0x55ebd6f347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.713+0000 7f66a5200700 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.713+0000 7f66a5200700 1 -- 172.21.15.67:0/4075526662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 msgr2=0x55ebd6f3f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.713+0000 7f66a5200700 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 0x55ebd6f3f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.713+0000 7f66a5200700 1 -- 172.21.15.67:0/4075526662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a010e450 con 0x55ebd6f30ea0 2024-03-30T02:46:36.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.714+0000 7f66a5200700 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f34260 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f668c00fd60 tx=0x7f668c005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c48fe33473bec4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.714+0000 7f6686a00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:36.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.714+0000 7f6686a00700 1 -- 172.21.15.67:0/4075526662 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c011e70 con 0x55ebd6f30ea0 2024-03-30T02:46:36.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.714+0000 7f6686a00700 1 -- 172.21.15.67:0/4075526662 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f668c004050 con 0x55ebd6f30ea0 2024-03-30T02:46:36.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.715+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/4075526662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebd6e3b380 con 0x55ebd6f30ea0 2024-03-30T02:46:36.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.717+0000 7f6686a00700 1 -- 172.21.15.67:0/4075526662 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f668c004210 con 0x55ebd6f30ea0 2024-03-30T02:46:36.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.717+0000 7f66bb6c5b80 10 client.17403 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:36.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.717+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/4075526662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ebd6ca8b60 con 0x55ebd6f30ea0 2024-03-30T02:46:36.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.719+0000 7f6686a00700 1 -- 172.21.15.67:0/4075526662 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ebd6e3b380 con 0x55ebd6f30ea0 2024-03-30T02:46:36.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.720+0000 7f6686a00700 1 -- 172.21.15.67:0/4075526662 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f668c01e570 con 0x55ebd6f30ea0 2024-03-30T02:46:36.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.720+0000 7f66bb6c5b80 10 client.17403 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:36.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.720+0000 7f66bb6c5b80 10 client.17403 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:46:36.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.720+0000 7f66bb6c5b80 20 client.17403 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:36.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.720+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55ebd6f43660 0x55ebd6f43a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.720+0000 7f66bb6c5b80 4 client.17403 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:36.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.720+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/4075526662 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebd6e73700 con 0x55ebd6f43660 2024-03-30T02:46:36.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.721+0000 7f669fe00700 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55ebd6f43660 0x55ebd6f43a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.726+0000 7f669fe00700 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55ebd6f43660 0x55ebd6f43a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe6ca5c1243dc58c server_cookie=86a734567c22ce0 in_seq=0 out_seq=0 2024-03-30T02:46:36.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.726+0000 7f6686a00700 10 client.17403 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:46:36.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.733+0000 7f6686a00700 1 -- 172.21.15.67:0/4075526662 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ebd6e73700 con 0x55ebd6f43660 2024-03-30T02:46:36.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.733+0000 7f6686a00700 10 client.17403 handle_command_reply: tid=0 2024-03-30T02:46:36.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.733+0000 7f66bb6c5b80 1 client.17403 shutdown 2024-03-30T02:46:36.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/4075526662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f30ea0 msgr2=0x55ebd6f34260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f34260 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f668c00fd60 tx=0x7f668c005d20 comp rx=0 tx=0).stop 2024-03-30T02:46:36.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/4075526662 shutdown_connections 2024-03-30T02:46:36.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55ebd6f43660 0x55ebd6f43a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f34260 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f45080 0x55ebd6f3f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/4075526662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/4075526662 >> 172.21.15.67:0/4075526662 conn(0x55ebd6e886a0 msgr2=0x55ebd6f32f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f6686a00700 0 client.17403 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:46:36.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.734+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/4075526662 shutdown_connections 2024-03-30T02:46:36.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.735+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/4075526662 wait complete. 2024-03-30T02:46:36.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.735+0000 7f66bb6c5b80 20 client.17403 trim_cache size 0 max 16384 2024-03-30T02:46:36.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:36 smithi082 ceph-mon[79176]: pgmap v295: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-30T02:46:36.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:36 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1828966725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:46:36.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.768+0000 7f66bb6c5b80 1 Processor -- start 2024-03-30T02:46:36.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.768+0000 7f66bb6c5b80 1 -- start start 2024-03-30T02:46:36.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.768+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f2e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.768+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f4b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.768+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 0x55ebd6f2d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.768+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6f4b9f0 con 0x55ebd6f45080 2024-03-30T02:46:36.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.768+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6f2be70 con 0x55ebd6f30ea0 2024-03-30T02:46:36.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.768+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6f2d6d0 con 0x55ebd6f317c0 2024-03-30T02:46:36.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.769+0000 7f66a5200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f2e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.769+0000 7f66a5200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f2e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40596/0 (socket says 172.21.15.67:40596) 2024-03-30T02:46:36.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.769+0000 7f669fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f4b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.769+0000 7f66a5200700 1 -- 172.21.15.67:0/3146650091 learned_addr learned my addr 172.21.15.67:0/3146650091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.770+0000 7f66a5c00700 1 --2- 172.21.15.67:0/3146650091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 0x55ebd6f2d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.770+0000 7f66a5200700 1 -- 172.21.15.67:0/3146650091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 msgr2=0x55ebd6f4b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.770+0000 7f66a5200700 1 --2- 172.21.15.67:0/3146650091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f4b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.770+0000 7f66a5200700 1 -- 172.21.15.67:0/3146650091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 msgr2=0x55ebd6f2d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.770+0000 7f66a5200700 1 --2- 172.21.15.67:0/3146650091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 0x55ebd6f2d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.770+0000 7f66a5200700 1 -- 172.21.15.67:0/3146650091 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebd6e40ea0 con 0x55ebd6f30ea0 2024-03-30T02:46:36.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.771+0000 7f66a5200700 1 --2- 172.21.15.67:0/3146650091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f2e190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f668c01e8d0 tx=0x7f668c005940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dda9fb7f8ed8a6dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.772+0000 7f6692a00700 1 -- 172.21.15.67:0/3146650091 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c011ab0 con 0x55ebd6f30ea0 2024-03-30T02:46:36.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.772+0000 7f6692a00700 1 -- 172.21.15.67:0/3146650091 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f668c036070 con 0x55ebd6f30ea0 2024-03-30T02:46:36.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.772+0000 7f6692a00700 1 -- 172.21.15.67:0/3146650091 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c0038e0 con 0x55ebd6f30ea0 2024-03-30T02:46:36.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.773+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3146650091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 msgr2=0x55ebd6f2e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.773+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3146650091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f2e190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f668c01e8d0 tx=0x7f668c005940 comp rx=0 tx=0).stop 2024-03-30T02:46:36.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.777+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3146650091 shutdown_connections 2024-03-30T02:46:36.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.777+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3146650091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 0x55ebd6f2d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.777+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3146650091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f2e190 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.777+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3146650091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f4b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.777+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3146650091 >> 172.21.15.67:0/3146650091 conn(0x55ebd6e886a0 msgr2=0x55ebd6f28170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.777+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3146650091 shutdown_connections 2024-03-30T02:46:36.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.777+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3146650091 wait complete. 2024-03-30T02:46:36.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.778+0000 7f66bb6c5b80 1 -- start start 2024-03-30T02:46:36.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.778+0000 7f66bb6c5b80 1 Processor -- start 2024-03-30T02:46:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.782+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f93c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.782+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f91110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.782+0000 7f66bb6c5b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 0x55ebd6f91650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.782+0000 7f66a5200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f93c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.783+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6f91b90 con 0x55ebd6f45080 2024-03-30T02:46:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.783+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6f91d00 con 0x55ebd6f30ea0 2024-03-30T02:46:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.783+0000 7f66bb6c5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebd6f91ec0 con 0x55ebd6f317c0 2024-03-30T02:46:36.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.783+0000 7f66a5c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 0x55ebd6f91650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.783+0000 7f66bb6c5b80 10 client.0 _ll_register_callbacks cb 0x55ebd6e20470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:36.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.783+0000 7f66a5200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f93c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40602/0 (socket says 172.21.15.67:40602) 2024-03-30T02:46:36.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.783+0000 7f66a5200700 1 -- 172.21.15.67:0/3491979271 learned_addr learned my addr 172.21.15.67:0/3491979271 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.784+0000 7f66a5200700 1 -- 172.21.15.67:0/3491979271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 msgr2=0x55ebd6f91110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.784+0000 7f66a5200700 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f91110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.784+0000 7f66a5200700 1 -- 172.21.15.67:0/3491979271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 msgr2=0x55ebd6f91650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.784+0000 7f66a5200700 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 0x55ebd6f91650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.784+0000 7f66a5200700 1 -- 172.21.15.67:0/3491979271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebd6ca8b60 con 0x55ebd6f30ea0 2024-03-30T02:46:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.784+0000 7f66a5200700 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f93c80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f668c005d60 tx=0x7f668c004660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2cfc48b61ecf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.784+0000 7f6686a00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.785+0000 7f6686a00700 1 -- 172.21.15.67:0/3491979271 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c01e590 con 0x55ebd6f30ea0 2024-03-30T02:46:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.785+0000 7f6686a00700 1 -- 172.21.15.67:0/3491979271 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f668c003680 con 0x55ebd6f30ea0 2024-03-30T02:46:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.785+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3491979271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6698003a00 con 0x55ebd6f30ea0 2024-03-30T02:46:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.785+0000 7f6686a00700 1 -- 172.21.15.67:0/3491979271 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f668c01e6f0 con 0x55ebd6f30ea0 2024-03-30T02:46:36.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.786+0000 7f66bb6c5b80 10 client.27542 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:36.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.786+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3491979271 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ebd6d00cd0 con 0x55ebd6f30ea0 2024-03-30T02:46:36.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.789+0000 7f6686a00700 1 -- 172.21.15.67:0/3491979271 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6698003a00 con 0x55ebd6f30ea0 2024-03-30T02:46:36.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.790+0000 7f6686a00700 1 -- 172.21.15.67:0/3491979271 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f668c0327d0 con 0x55ebd6f30ea0 2024-03-30T02:46:36.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.791+0000 7f66bb6c5b80 10 client.27542 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:36.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.791+0000 7f66bb6c5b80 10 client.27542 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:46:36.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.791+0000 7f66bb6c5b80 20 client.27542 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:36.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.791+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55ebd6fa9540 0x55ebd6fab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:36.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.791+0000 7f66bb6c5b80 4 client.27542 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:36.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.791+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3491979271 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ebd6cc9cf0 con 0x55ebd6fa9540 2024-03-30T02:46:36.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.791+0000 7f669fe00700 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55ebd6fa9540 0x55ebd6fab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:36.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.792+0000 7f669fe00700 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55ebd6fa9540 0x55ebd6fab930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af9cceb80120b7c server_cookie=1514a8582a1e08b6 in_seq=0 out_seq=0 2024-03-30T02:46:36.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.792+0000 7f6686a00700 10 client.27542 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:46:36.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f6686a00700 1 -- 172.21.15.67:0/3491979271 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ebd6cc9cf0 con 0x55ebd6fa9540 2024-03-30T02:46:36.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f6686a00700 10 client.27542 handle_command_reply: tid=0 2024-03-30T02:46:36.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f66bb6c5b80 1 client.27542 shutdown 2024-03-30T02:46:36.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3491979271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 msgr2=0x55ebd6f93c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f93c80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f668c005d60 tx=0x7f668c004660 comp rx=0 tx=0).stop 2024-03-30T02:46:36.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3491979271 shutdown_connections 2024-03-30T02:46:36.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x55ebd6fa9540 0x55ebd6fab930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ebd6f45080 0x55ebd6f91650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ebd6f30ea0 0x55ebd6f93c80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f6686a00700 0 client.27542 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:46:36.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f66bb6c5b80 1 --2- 172.21.15.67:0/3491979271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ebd6f317c0 0x55ebd6f91110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.794+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3491979271 >> 172.21.15.67:0/3491979271 conn(0x55ebd6e886a0 msgr2=0x55ebd6f28c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.795+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3491979271 shutdown_connections 2024-03-30T02:46:36.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.795+0000 7f66bb6c5b80 1 -- 172.21.15.67:0/3491979271 wait complete. 2024-03-30T02:46:36.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.795+0000 7f66bb6c5b80 20 client.27542 trim_cache size 0 max 16384 2024-03-30T02:46:36.797 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:36.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 -- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f66880a1310 msgr2=0x7f66880a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f66880a1310 0x7f66880a37d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f668c005d90 tx=0x7f668c00b2c0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 -- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a0106060 msgr2=0x7f66a00735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:36.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a0106060 0x7f66a00735f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f66a006a390 tx=0x7f669400dce0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 -- 172.21.15.67:0/50412131 shutdown_connections 2024-03-30T02:46:36.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f66880a1310 0x7f66880a37d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66a0073b30 0x7f66a01a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a0100f40 0x7f66a00730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 --2- 172.21.15.67:0/50412131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66a0106060 0x7f66a00735f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:36.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.796+0000 7f66a7c00700 1 -- 172.21.15.67:0/50412131 >> 172.21.15.67:0/50412131 conn(0x7f66a00fc7c0 msgr2=0x7f66a00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:36.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.797+0000 7f66a7c00700 1 -- 172.21.15.67:0/50412131 shutdown_connections 2024-03-30T02:46:36.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:36.797+0000 7f66a7c00700 1 -- 172.21.15.67:0/50412131 wait complete. 2024-03-30T02:46:36.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:46:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:36 smithi170 ceph-mon[78798]: pgmap v295: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-30T02:46:36.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:36 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1828966725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:46:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.238+0000 7fcf17400700 1 -- 172.21.15.67:0/4230832257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073180 msgr2=0x7fcf10073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.238+0000 7fcf17400700 1 --2- 172.21.15.67:0/4230832257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073180 0x7fcf10073560 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009990 tx=0x7fcf04009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.238+0000 7fcf17400700 1 -- 172.21.15.67:0/4230832257 shutdown_connections 2024-03-30T02:46:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.238+0000 7fcf17400700 1 --2- 172.21.15.67:0/4230832257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf1010d580 0x7fcf1010f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.238+0000 7fcf17400700 1 --2- 172.21.15.67:0/4230832257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073180 0x7fcf10073560 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.238+0000 7fcf17400700 1 --2- 172.21.15.67:0/4230832257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf10073aa0 0x7fcf1010d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.238+0000 7fcf17400700 1 -- 172.21.15.67:0/4230832257 >> 172.21.15.67:0/4230832257 conn(0x7fcf100fc7f0 msgr2=0x7fcf100fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.239+0000 7fcf17400700 1 -- 172.21.15.67:0/4230832257 shutdown_connections 2024-03-30T02:46:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.239+0000 7fcf17400700 1 -- 172.21.15.67:0/4230832257 wait complete. 2024-03-30T02:46:37.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.240+0000 7fcf17400700 1 Processor -- start 2024-03-30T02:46:37.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.240+0000 7fcf17400700 1 -- start start 2024-03-30T02:46:37.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf17400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10073180 0x7fcf10100ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf17400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073aa0 0x7fcf101011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf17400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1010d580 0x7fcf10101720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf17400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf101a9b00 con 0x7fcf10073180 2024-03-30T02:46:37.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf17400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10199210 con 0x7fcf10073aa0 2024-03-30T02:46:37.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf17400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10199380 con 0x7fcf1010d580 2024-03-30T02:46:37.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf0fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073aa0 0x7fcf101011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf15400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1010d580 0x7fcf10101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf0fe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073aa0 0x7fcf101011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40610/0 (socket says 172.21.15.67:40610) 2024-03-30T02:46:37.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf15400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1010d580 0x7fcf10101720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44336/0 (socket says 172.21.15.67:44336) 2024-03-30T02:46:37.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.241+0000 7fcf0fe00700 1 -- 172.21.15.67:0/4118618166 learned_addr learned my addr 172.21.15.67:0/4118618166 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.242+0000 7fcf15400700 1 -- 172.21.15.67:0/4118618166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073aa0 msgr2=0x7fcf101011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.242+0000 7fcf15400700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073aa0 0x7fcf101011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.242+0000 7fcf15400700 1 -- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10073180 msgr2=0x7fcf10100ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.242+0000 7fcf14a00700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10073180 0x7fcf10100ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.242+0000 7fcf15400700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10073180 0x7fcf10100ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.242+0000 7fcf15400700 1 -- 172.21.15.67:0/4118618166 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf040095f0 con 0x7fcf1010d580 2024-03-30T02:46:37.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.242+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073aa0 0x7fcf101011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:37.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.242+0000 7fcf15400700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1010d580 0x7fcf10101720 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcf08007ee0 tx=0x7fcf0800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.243+0000 7fcf0d600700 1 -- 172.21.15.67:0/4118618166 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800ddd0 con 0x7fcf1010d580 2024-03-30T02:46:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.243+0000 7fcf0d600700 1 -- 172.21.15.67:0/4118618166 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf0800f040 con 0x7fcf1010d580 2024-03-30T02:46:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.243+0000 7fcf17400700 1 -- 172.21.15.67:0/4118618166 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf10199660 con 0x7fcf1010d580 2024-03-30T02:46:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.243+0000 7fcf0d600700 1 -- 172.21.15.67:0/4118618166 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08005a00 con 0x7fcf1010d580 2024-03-30T02:46:37.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.243+0000 7fcf17400700 1 -- 172.21.15.67:0/4118618166 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf10199bc0 con 0x7fcf1010d580 2024-03-30T02:46:37.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.249+0000 7fcf0d600700 1 -- 172.21.15.67:0/4118618166 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fcf08004ad0 con 0x7fcf1010d580 2024-03-30T02:46:37.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.251+0000 7fcf0d600700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcef80a1310 0x7fcef80a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.251+0000 7fcf2af0bb80 1 Processor -- start 2024-03-30T02:46:37.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.251+0000 7fcf2af0bb80 1 -- start start 2024-03-30T02:46:37.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf0d600700 1 -- 172.21.15.67:0/4118618166 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fcf080cc8d0 con 0x7fcf1010d580 2024-03-30T02:46:37.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf14a00700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcef80a1310 0x7fcef80a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 0x55557aae69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaf18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaf41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557aa3f990 con 0x55557aaf1df0 2024-03-30T02:46:37.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557aa3d350 con 0x55557aae65d0 2024-03-30T02:46:37.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557aa38c40 con 0x55557aae6ef0 2024-03-30T02:46:37.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf0fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaf18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf0fe00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaf18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44340/0 (socket says 172.21.15.67:44340) 2024-03-30T02:46:37.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.252+0000 7fcf0fe00700 1 -- 172.21.15.67:0/3720104870 learned_addr learned my addr 172.21.15.67:0/3720104870 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf15400700 1 --2- 172.21.15.67:0/3720104870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaf41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf0fe00700 1 -- 172.21.15.67:0/3720104870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 msgr2=0x55557aae69b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T02:46:37.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/3720104870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 0x55557aae69b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf0fe00700 1 -- 172.21.15.67:0/3720104870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 msgr2=0x55557aaf41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/3720104870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaf41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf0fe00700 1 -- 172.21.15.67:0/3720104870 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55557a8ba3b0 con 0x55557aae6ef0 2024-03-30T02:46:37.264 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf15400700 1 --2- 172.21.15.67:0/3720104870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaf41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:46:37.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/3720104870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaf18b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009f20 tx=0x7fcefc00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28fdb673adb4853b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf01c00700 1 -- 172.21.15.67:0/3720104870 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc009a20 con 0x55557aae6ef0 2024-03-30T02:46:37.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf01c00700 1 -- 172.21.15.67:0/3720104870 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcefc00ed40 con 0x55557aae6ef0 2024-03-30T02:46:37.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.253+0000 7fcf01c00700 1 -- 172.21.15.67:0/3720104870 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0127f0 con 0x55557aae6ef0 2024-03-30T02:46:37.265 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/3720104870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 msgr2=0x55557aaf18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/3720104870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaf18b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009f20 tx=0x7fcefc00e4b0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf14a00700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcef80a1310 0x7fcef80a37d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b5c0 tx=0x7fcf04005fd0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/3720104870 shutdown_connections 2024-03-30T02:46:37.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/3720104870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaf41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/3720104870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 0x55557aae69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/3720104870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaf18b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/3720104870 >> 172.21.15.67:0/3720104870 conn(0x55557aa466a0 msgr2=0x55557aae2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/3720104870 shutdown_connections 2024-03-30T02:46:37.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.254+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/3720104870 wait complete. 2024-03-30T02:46:37.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.255+0000 7fcf2af0bb80 1 -- start start 2024-03-30T02:46:37.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.255+0000 7fcf2af0bb80 1 Processor -- start 2024-03-30T02:46:37.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 0x55557aaf17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaeeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaef110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557a9d4d80 con 0x55557aaf1df0 2024-03-30T02:46:37.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557a9d4a20 con 0x55557aae65d0 2024-03-30T02:46:37.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557a8bc6a0 con 0x55557aae6ef0 2024-03-30T02:46:37.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf2af0bb80 10 client.0 _ll_register_callbacks cb 0x55557a9de470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:37.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf14a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 0x55557aaf17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf14a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 0x55557aaf17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40632/0 (socket says 172.21.15.67:40632) 2024-03-30T02:46:37.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf14a00700 1 -- 172.21.15.67:0/681155381 learned_addr learned my addr 172.21.15.67:0/681155381 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf15400700 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaeeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf14a00700 1 -- 172.21.15.67:0/681155381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 msgr2=0x55557aaeeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf14a00700 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaeeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf14a00700 1 -- 172.21.15.67:0/681155381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 msgr2=0x55557aaef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf14a00700 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf14a00700 1 -- 172.21.15.67:0/681155381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc009b80 con 0x55557aae65d0 2024-03-30T02:46:37.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.256+0000 7fcf15400700 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:37.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcf14a00700 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 0x55557aaf17c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcf04015000 tx=0x7fcf04009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a75d7e78b8c0f132 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcef6000700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcef6000700 1 -- 172.21.15.67:0/681155381 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04029070 con 0x55557aae65d0 2024-03-30T02:46:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcef6000700 1 -- 172.21.15.67:0/681155381 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fcf040053d0 con 0x55557aae65d0 2024-03-30T02:46:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/681155381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55557a9f9380 con 0x55557aae65d0 2024-03-30T02:46:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcef6000700 1 -- 172.21.15.67:0/681155381 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0401d070 con 0x55557aae65d0 2024-03-30T02:46:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcf2af0bb80 10 client.27551 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/681155381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55557a866b60 con 0x55557aae65d0 2024-03-30T02:46:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcef6000700 1 -- 172.21.15.67:0/681155381 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55557a9f9380 con 0x55557aae65d0 2024-03-30T02:46:37.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.257+0000 7fcef6000700 1 -- 172.21.15.67:0/681155381 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf0400f460 con 0x55557aae65d0 2024-03-30T02:46:37.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.258+0000 7fcf2af0bb80 10 client.27551 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:37.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.258+0000 7fcf2af0bb80 10 client.27551 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:46:37.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.258+0000 7fcf2af0bb80 20 client.27551 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:37.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.258+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55557ab01900 0x55557ab03d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.258+0000 7fcf2af0bb80 4 client.27551 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:37.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.258+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/681155381 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55557aa31700 con 0x55557ab01900 2024-03-30T02:46:37.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.258+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55557ab01900 0x55557ab03d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.259+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55557ab01900 0x55557ab03d20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1e689a7e517863c server_cookie=17a4b4d91e27aba1 in_seq=0 out_seq=0 2024-03-30T02:46:37.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.259+0000 7fcef6000700 10 client.27551 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:46:37.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.259+0000 7fcef6000700 1 -- 172.21.15.67:0/681155381 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55557aa31700 con 0x55557ab01900 2024-03-30T02:46:37.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.259+0000 7fcef6000700 10 client.27551 handle_command_reply: tid=0 2024-03-30T02:46:37.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 client.27551 shutdown 2024-03-30T02:46:37.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/681155381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 msgr2=0x55557aaf17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 0x55557aaf17c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcf04015000 tx=0x7fcf04009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/681155381 shutdown_connections 2024-03-30T02:46:37.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55557ab01900 0x55557ab03d20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aaf1df0 0x55557aaef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae65d0 0x55557aaf17c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/681155381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae6ef0 0x55557aaeeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcef6000700 0 client.27551 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:46:37.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/681155381 >> 172.21.15.67:0/681155381 conn(0x55557aa466a0 msgr2=0x55557aae86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/681155381 shutdown_connections 2024-03-30T02:46:37.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/681155381 wait complete. 2024-03-30T02:46:37.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.260+0000 7fcf2af0bb80 20 client.27551 trim_cache size 0 max 16384 2024-03-30T02:46:37.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.274+0000 7fcf2af0bb80 1 Processor -- start 2024-03-30T02:46:37.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.274+0000 7fcf2af0bb80 1 -- start start 2024-03-30T02:46:37.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.274+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae65d0 0x55557aae95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.274+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae6ef0 0x55557aaf8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.274+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aaf1df0 0x55557aadb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.274+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557aae2fa0 con 0x55557aae6ef0 2024-03-30T02:46:37.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.274+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557aaf8fb0 con 0x55557aaf1df0 2024-03-30T02:46:37.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.274+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557aaef9c0 con 0x55557aae65d0 2024-03-30T02:46:37.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf0fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae6ef0 0x55557aaf8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf14a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae65d0 0x55557aae95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf15400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aaf1df0 0x55557aadb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf14a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae65d0 0x55557aae95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44344/0 (socket says 172.21.15.67:44344) 2024-03-30T02:46:37.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf15400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aaf1df0 0x55557aadb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40646/0 (socket says 172.21.15.67:40646) 2024-03-30T02:46:37.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf0fe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae6ef0 0x55557aaf8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39824/0 (socket says 172.21.15.67:39824) 2024-03-30T02:46:37.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf14a00700 1 -- 172.21.15.67:0/697798442 learned_addr learned my addr 172.21.15.67:0/697798442 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf15400700 1 -- 172.21.15.67:0/697798442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae65d0 msgr2=0x55557aae95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf15400700 1 --2- 172.21.15.67:0/697798442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae65d0 0x55557aae95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf15400700 1 -- 172.21.15.67:0/697798442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae6ef0 msgr2=0x55557aaf8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf15400700 1 --2- 172.21.15.67:0/697798442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae6ef0 0x55557aaf8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.275+0000 7fcf15400700 1 -- 172.21.15.67:0/697798442 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55557a9feea0 con 0x55557aaf1df0 2024-03-30T02:46:37.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf15400700 1 --2- 172.21.15.67:0/697798442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aaf1df0 0x55557aadb5a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcf080c9c80 tx=0x7fcf080b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be6a2034d92b3ed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf02600700 1 -- 172.21.15.67:0/697798442 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08098fa0 con 0x55557aaf1df0 2024-03-30T02:46:37.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf02600700 1 -- 172.21.15.67:0/697798442 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf08099d40 con 0x55557aaf1df0 2024-03-30T02:46:37.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf02600700 1 -- 172.21.15.67:0/697798442 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf080a1b50 con 0x55557aaf1df0 2024-03-30T02:46:37.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/697798442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aaf1df0 msgr2=0x55557aadb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/697798442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aaf1df0 0x55557aadb5a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcf080c9c80 tx=0x7fcf080b9040 comp rx=0 tx=0).stop 2024-03-30T02:46:37.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/697798442 shutdown_connections 2024-03-30T02:46:37.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/697798442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aaf1df0 0x55557aadb5a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/697798442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aae65d0 0x55557aae95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/697798442 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae6ef0 0x55557aaf8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.276+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/697798442 >> 172.21.15.67:0/697798442 conn(0x55557aa466a0 msgr2=0x55557ab08680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.277+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/697798442 shutdown_connections 2024-03-30T02:46:37.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.277+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/697798442 wait complete. 2024-03-30T02:46:37.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.277+0000 7fcf2af0bb80 1 -- start start 2024-03-30T02:46:37.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.277+0000 7fcf2af0bb80 1 Processor -- start 2024-03-30T02:46:37.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae65d0 0x55557aaf88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae6ef0 0x55557aaf6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf2af0bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aaf1df0 0x55557aaf73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557aaf7a60 con 0x55557aae65d0 2024-03-30T02:46:37.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557aaf7bd0 con 0x55557aae6ef0 2024-03-30T02:46:37.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf2af0bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55557ab55740 con 0x55557aaf1df0 2024-03-30T02:46:37.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf2af0bb80 10 client.0 _ll_register_callbacks cb 0x55557a9de470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:37.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf14a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae65d0 0x55557aaf88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf14a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae65d0 0x55557aaf88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39832/0 (socket says 172.21.15.67:39832) 2024-03-30T02:46:37.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf14a00700 1 -- 172.21.15.67:0/2182117612 learned_addr learned my addr 172.21.15.67:0/2182117612 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae6ef0 0x55557aaf6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf15400700 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aaf1df0 0x55557aaf73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf14a00700 1 -- 172.21.15.67:0/2182117612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aaf1df0 msgr2=0x55557aaf73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf14a00700 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aaf1df0 0x55557aaf73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf14a00700 1 -- 172.21.15.67:0/2182117612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae6ef0 msgr2=0x55557aaf6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf14a00700 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae6ef0 0x55557aaf6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf14a00700 1 -- 172.21.15.67:0/2182117612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf10199bc0 con 0x55557aae65d0 2024-03-30T02:46:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf15400700 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aaf1df0 0x55557aaf73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:46:37.293 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.278+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae6ef0 0x55557aaf6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:37.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.279+0000 7fcf14a00700 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae65d0 0x55557aaf88b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400f890 tx=0x7fcf0400f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5538bfd0accab80 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.279+0000 7fcef6000700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:37.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.279+0000 7fcef6000700 1 -- 172.21.15.67:0/2182117612 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04027040 con 0x55557aae65d0 2024-03-30T02:46:37.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.279+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/2182117612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcefc00ed40 con 0x55557aae65d0 2024-03-30T02:46:37.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.279+0000 7fcef6000700 1 -- 172.21.15.67:0/2182117612 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fcf04005660 con 0x55557aae65d0 2024-03-30T02:46:37.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.279+0000 7fcef6000700 1 -- 172.21.15.67:0/2182117612 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf04029040 con 0x55557aae65d0 2024-03-30T02:46:37.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.279+0000 7fcf2af0bb80 10 client.17421 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:37.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.279+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/2182117612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55557a8becd0 con 0x55557aae65d0 2024-03-30T02:46:37.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.279+0000 7fcef6000700 1 -- 172.21.15.67:0/2182117612 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcefc00ed40 con 0x55557aae65d0 2024-03-30T02:46:37.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.280+0000 7fcef6000700 1 -- 172.21.15.67:0/2182117612 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf0402e5e0 con 0x55557aae65d0 2024-03-30T02:46:37.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.280+0000 7fcf2af0bb80 10 client.17421 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:37.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.280+0000 7fcf2af0bb80 10 client.17421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:46:37.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.280+0000 7fcf2af0bb80 20 client.17421 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:37.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.280+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55557ab569a0 0x55557ab58d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.280+0000 7fcf2af0bb80 4 client.17421 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:37.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.280+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/2182117612 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55557a887cf0 con 0x55557ab569a0 2024-03-30T02:46:37.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.280+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55557ab569a0 0x55557ab58d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf0fe00700 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55557ab569a0 0x55557ab58d90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11b97ddd0b06ccb7 server_cookie=9d7763639c6153f9 in_seq=0 out_seq=0 2024-03-30T02:46:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcef6000700 10 client.17421 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:46:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcef6000700 1 -- 172.21.15.67:0/2182117612 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55557a887cf0 con 0x55557ab569a0 2024-03-30T02:46:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcef6000700 10 client.17421 handle_command_reply: tid=0 2024-03-30T02:46:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 client.17421 shutdown 2024-03-30T02:46:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/2182117612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae65d0 msgr2=0x55557aaf88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae65d0 0x55557aaf88b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400f890 tx=0x7fcf0400f4e0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/2182117612 shutdown_connections 2024-03-30T02:46:37.296 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55557ab569a0 0x55557ab58d90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55557aae65d0 0x55557aaf88b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55557aaf1df0 0x55557aaf73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 --2- 172.21.15.67:0/2182117612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55557aae6ef0 0x55557aaf6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcef6000700 0 client.17421 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:46:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/2182117612 >> 172.21.15.67:0/2182117612 conn(0x55557aa466a0 msgr2=0x55557aadb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/2182117612 shutdown_connections 2024-03-30T02:46:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 1 -- 172.21.15.67:0/2182117612 wait complete. 2024-03-30T02:46:37.297 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.281+0000 7fcf2af0bb80 20 client.17421 trim_cache size 0 max 16384 2024-03-30T02:46:37.298 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:37.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 -- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcef80a1310 msgr2=0x7fcef80a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcef80a1310 0x7fcef80a37d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b5c0 tx=0x7fcf04005fd0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 -- 172.21.15.67:0/4118618166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1010d580 msgr2=0x7fcf10101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1010d580 0x7fcf10101720 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcf08007ee0 tx=0x7fcf0800d3b0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 -- 172.21.15.67:0/4118618166 shutdown_connections 2024-03-30T02:46:37.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fcef80a1310 0x7fcef80a37d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10073180 0x7fcf10100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf1010d580 0x7fcf10101720 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 --2- 172.21.15.67:0/4118618166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10073aa0 0x7fcf101011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 -- 172.21.15.67:0/4118618166 >> 172.21.15.67:0/4118618166 conn(0x7fcf100fc7f0 msgr2=0x7fcf100fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 -- 172.21.15.67:0/4118618166 shutdown_connections 2024-03-30T02:46:37.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.282+0000 7fcf17400700 1 -- 172.21.15.67:0/4118618166 wait complete. 2024-03-30T02:46:37.301 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:46:37.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.503+0000 7f674d400700 1 -- 172.21.15.67:0/2253791796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748101920 msgr2=0x7f6748105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.503+0000 7f674d400700 1 --2- 172.21.15.67:0/2253791796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748101920 0x7f6748105960 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f67340098f0 tx=0x7f6734009c00 comp rx=0 tx=0).stop 2024-03-30T02:46:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.503+0000 7f674d400700 1 -- 172.21.15.67:0/2253791796 shutdown_connections 2024-03-30T02:46:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.503+0000 7f674d400700 1 --2- 172.21.15.67:0/2253791796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6748106090 0x7f674810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.503+0000 7f674d400700 1 --2- 172.21.15.67:0/2253791796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6748100f70 0x7f6748101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.503+0000 7f674d400700 1 --2- 172.21.15.67:0/2253791796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748101920 0x7f6748105960 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.503+0000 7f674d400700 1 -- 172.21.15.67:0/2253791796 >> 172.21.15.67:0/2253791796 conn(0x7f67480fc7f0 msgr2=0x7f67480fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.503+0000 7f674d400700 1 -- 172.21.15.67:0/2253791796 shutdown_connections 2024-03-30T02:46:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.503+0000 7f674d400700 1 -- 172.21.15.67:0/2253791796 wait complete. 2024-03-30T02:46:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.504+0000 7f674d400700 1 Processor -- start 2024-03-30T02:46:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.504+0000 7f674d400700 1 -- start start 2024-03-30T02:46:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f674d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748100f70 0x7f6748197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f674d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6748106090 0x7f67481977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f674d400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f674819bb20 0x7f674819bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f674d400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674819c5a0 con 0x7f6748100f70 2024-03-30T02:46:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f674d400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674819c710 con 0x7f6748106090 2024-03-30T02:46:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f674d400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674819c8d0 con 0x7f674819bb20 2024-03-30T02:46:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6746800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748100f70 0x7f6748197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6746800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748100f70 0x7f6748197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39858/0 (socket says 172.21.15.67:39858) 2024-03-30T02:46:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6746800700 1 -- 172.21.15.67:0/1858910593 learned_addr learned my addr 172.21.15.67:0/1858910593 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6745e00700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6748106090 0x7f67481977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6747200700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f674819bb20 0x7f674819bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6746800700 1 -- 172.21.15.67:0/1858910593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f674819bb20 msgr2=0x7f674819bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6746800700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f674819bb20 0x7f674819bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6746800700 1 -- 172.21.15.67:0/1858910593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6748106090 msgr2=0x7f67481977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6746800700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6748106090 0x7f67481977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.505+0000 7f6746800700 1 -- 172.21.15.67:0/1858910593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67380095f0 con 0x7f6748100f70 2024-03-30T02:46:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.506+0000 7f6745e00700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6748106090 0x7f67481977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.506+0000 7f6746800700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748100f70 0x7f6748197290 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f673800ec10 tx=0x7f673800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.506+0000 7f673f400700 1 -- 172.21.15.67:0/1858910593 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673800cc90 con 0x7f6748100f70 2024-03-30T02:46:37.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.506+0000 7f673f400700 1 -- 172.21.15.67:0/1858910593 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67380044e0 con 0x7f6748100f70 2024-03-30T02:46:37.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.506+0000 7f674d400700 1 -- 172.21.15.67:0/1858910593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6734009520 con 0x7f6748100f70 2024-03-30T02:46:37.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.506+0000 7f673f400700 1 -- 172.21.15.67:0/1858910593 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738010430 con 0x7f6748100f70 2024-03-30T02:46:37.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.506+0000 7f674d400700 1 -- 172.21.15.67:0/1858910593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67481a4180 con 0x7f6748100f70 2024-03-30T02:46:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.509+0000 7f673f400700 1 -- 172.21.15.67:0/1858910593 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f6738003fb0 con 0x7f6748100f70 2024-03-30T02:46:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.510+0000 7f673f400700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f672c0a1290 0x7f672c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f6760e9cb80 1 Processor -- start 2024-03-30T02:46:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f673f400700 1 -- 172.21.15.67:0/1858910593 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f6738014070 con 0x7f6748100f70 2024-03-30T02:46:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f6760e9cb80 1 -- start start 2024-03-30T02:46:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f6745e00700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f672c0a1290 0x7f672c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c2799a40 0x5652c2799e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c27a5210 0x5652c27a7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c26ee990 con 0x5652c279a360 2024-03-30T02:46:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c26ec350 con 0x5652c27a5210 2024-03-30T02:46:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.511+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c26e7c40 con 0x5652c2799a40 2024-03-30T02:46:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6746800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c2799a40 0x5652c2799e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6745e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6746800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c2799a40 0x5652c2799e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44392/0 (socket says 172.21.15.67:44392) 2024-03-30T02:46:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6746800700 1 -- 172.21.15.67:0/2102186122 learned_addr learned my addr 172.21.15.67:0/2102186122 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6747200700 1 --2- 172.21.15.67:0/2102186122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c27a5210 0x5652c27a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6745e00700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f672c0a1290 0x7f672c0a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6734005e90 tx=0x7f6734005dc0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6746800700 1 -- 172.21.15.67:0/2102186122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c27a5210 msgr2=0x5652c27a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6746800700 1 --2- 172.21.15.67:0/2102186122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c27a5210 0x5652c27a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6746800700 1 -- 172.21.15.67:0/2102186122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 msgr2=0x5652c27a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6746800700 1 --2- 172.21.15.67:0/2102186122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6746800700 1 -- 172.21.15.67:0/2102186122 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652c25693b0 con 0x5652c2799a40 2024-03-30T02:46:37.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6746800700 1 --2- 172.21.15.67:0/2102186122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c2799a40 0x5652c2799e20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6738005d70 tx=0x7f673809a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef16ab993660ac9b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6733400700 1 -- 172.21.15.67:0/2102186122 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673809aa20 con 0x5652c2799a40 2024-03-30T02:46:37.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6733400700 1 -- 172.21.15.67:0/2102186122 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f673809a320 con 0x5652c2799a40 2024-03-30T02:46:37.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.512+0000 7f6733400700 1 -- 172.21.15.67:0/2102186122 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738096a30 con 0x5652c2799a40 2024-03-30T02:46:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.513+0000 7f6760e9cb80 1 -- 172.21.15.67:0/2102186122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c2799a40 msgr2=0x5652c2799e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.513+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/2102186122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c2799a40 0x5652c2799e20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6738005d70 tx=0x7f673809a1c0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.513+0000 7f6760e9cb80 1 -- 172.21.15.67:0/2102186122 shutdown_connections 2024-03-30T02:46:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.513+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/2102186122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c27a5210 0x5652c27a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.513+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/2102186122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c2799a40 0x5652c2799e20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.513+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/2102186122 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.513+0000 7f6760e9cb80 1 -- 172.21.15.67:0/2102186122 >> 172.21.15.67:0/2102186122 conn(0x5652c26f56a0 msgr2=0x5652c2791a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.513+0000 7f6760e9cb80 1 -- 172.21.15.67:0/2102186122 shutdown_connections 2024-03-30T02:46:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.513+0000 7f6760e9cb80 1 -- 172.21.15.67:0/2102186122 wait complete. 2024-03-30T02:46:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.514+0000 7f6760e9cb80 1 -- start start 2024-03-30T02:46:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.514+0000 7f6760e9cb80 1 Processor -- start 2024-03-30T02:46:37.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.514+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.514+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.514+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c27a5210 0x5652c27a2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c2683d80 con 0x5652c279a360 2024-03-30T02:46:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c2683a20 con 0x5652c2799a40 2024-03-30T02:46:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c256b6a0 con 0x5652c27a5210 2024-03-30T02:46:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6760e9cb80 10 client.0 _ll_register_callbacks cb 0x5652c268d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6745e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6745e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39868/0 (socket says 172.21.15.67:39868) 2024-03-30T02:46:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6745e00700 1 -- 172.21.15.67:0/3781550057 learned_addr learned my addr 172.21.15.67:0/3781550057 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6747200700 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c27a5210 0x5652c27a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6746800700 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6745e00700 1 -- 172.21.15.67:0/3781550057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c27a5210 msgr2=0x5652c27a2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6745e00700 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c27a5210 0x5652c27a2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6745e00700 1 -- 172.21.15.67:0/3781550057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 msgr2=0x5652c27a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6745e00700 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6745e00700 1 -- 172.21.15.67:0/3781550057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67481a4180 con 0x5652c279a360 2024-03-30T02:46:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6747200700 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c27a5210 0x5652c27a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6745e00700 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a1f80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6734009810 tx=0x7f6734009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22450561b9b51922 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.515+0000 7f6727e00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.516+0000 7f6727e00700 1 -- 172.21.15.67:0/3781550057 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400f900 con 0x5652c279a360 2024-03-30T02:46:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.516+0000 7f6727e00700 1 -- 172.21.15.67:0/3781550057 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f6734005040 con 0x5652c279a360 2024-03-30T02:46:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.516+0000 7f6760e9cb80 1 -- 172.21.15.67:0/3781550057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652c26a8380 con 0x5652c279a360 2024-03-30T02:46:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.516+0000 7f6727e00700 1 -- 172.21.15.67:0/3781550057 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6734005200 con 0x5652c279a360 2024-03-30T02:46:37.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.516+0000 7f6760e9cb80 10 client.17433 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.516+0000 7f6760e9cb80 1 -- 172.21.15.67:0/3781550057 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5652c2515b60 con 0x5652c279a360 2024-03-30T02:46:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.516+0000 7f6727e00700 1 -- 172.21.15.67:0/3781550057 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652c26a8380 con 0x5652c279a360 2024-03-30T02:46:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.516+0000 7f6727e00700 1 -- 172.21.15.67:0/3781550057 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f673402d460 con 0x5652c279a360 2024-03-30T02:46:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.516+0000 7f6760e9cb80 10 client.17433 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.517+0000 7f6760e9cb80 10 client.17433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:46:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.517+0000 7f6760e9cb80 20 client.17433 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.517+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5652c27b4f80 0x5652c27b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.517+0000 7f6760e9cb80 4 client.17433 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:37.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.517+0000 7f6760e9cb80 1 -- 172.21.15.67:0/3781550057 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652c26e0700 con 0x5652c27b4f80 2024-03-30T02:46:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.517+0000 7f6746800700 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5652c27b4f80 0x5652c27b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.517+0000 7f6727e00700 10 client.17433 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:46:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.517+0000 7f6746800700 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5652c27b4f80 0x5652c27b73a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=303894fae49a439 server_cookie=105f3174f1d37d44 in_seq=0 out_seq=0 2024-03-30T02:46:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6727e00700 1 -- 172.21.15.67:0/3781550057 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5652c26e0700 con 0x5652c27b4f80 2024-03-30T02:46:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6727e00700 10 client.17433 handle_command_reply: tid=0 2024-03-30T02:46:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 client.17433 shutdown 2024-03-30T02:46:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 -- 172.21.15.67:0/3781550057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 msgr2=0x5652c27a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a1f80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6734009810 tx=0x7f6734009c00 comp rx=0 tx=0).stop 2024-03-30T02:46:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 -- 172.21.15.67:0/3781550057 shutdown_connections 2024-03-30T02:46:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c27a5210 0x5652c27a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5652c27b4f80 0x5652c27b73a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/3781550057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c279a360 0x5652c27a1f80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 -- 172.21.15.67:0/3781550057 >> 172.21.15.67:0/3781550057 conn(0x5652c26f56a0 msgr2=0x5652c279b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6727e00700 0 client.17433 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:46:37.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 -- 172.21.15.67:0/3781550057 shutdown_connections 2024-03-30T02:46:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.518+0000 7f6760e9cb80 1 -- 172.21.15.67:0/3781550057 wait complete. 2024-03-30T02:46:37.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.519+0000 7f6760e9cb80 20 client.17433 trim_cache size 0 max 16384 2024-03-30T02:46:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6760e9cb80 1 Processor -- start 2024-03-30T02:46:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6760e9cb80 1 -- start start 2024-03-30T02:46:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 0x5652c27abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 0x5652c279d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c27b7730 con 0x5652c27a5210 2024-03-30T02:46:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c279a740 con 0x5652c2799a40 2024-03-30T02:46:37.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c27ac420 con 0x5652c279a360 2024-03-30T02:46:37.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6745e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 0x5652c27abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6746800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6745e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 0x5652c27abee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44408/0 (socket says 172.21.15.67:44408) 2024-03-30T02:46:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6745e00700 1 -- 172.21.15.67:0/271540955 learned_addr learned my addr 172.21.15.67:0/271540955 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.533+0000 7f6747200700 1 --2- 172.21.15.67:0/271540955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 0x5652c279d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6746800700 1 -- 172.21.15.67:0/271540955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 msgr2=0x5652c27abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6746800700 1 --2- 172.21.15.67:0/271540955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 0x5652c27abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6746800700 1 -- 172.21.15.67:0/271540955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 msgr2=0x5652c279d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6746800700 1 --2- 172.21.15.67:0/271540955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 0x5652c279d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6746800700 1 -- 172.21.15.67:0/271540955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652c26adea0 con 0x5652c2799a40 2024-03-30T02:46:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6747200700 1 --2- 172.21.15.67:0/271540955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 0x5652c279d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6746800700 1 --2- 172.21.15.67:0/271540955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27a25f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f673809a2b0 tx=0x7f673800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=563a55d7c6cbb77c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6733e00700 1 -- 172.21.15.67:0/271540955 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673807d070 con 0x5652c2799a40 2024-03-30T02:46:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6733e00700 1 -- 172.21.15.67:0/271540955 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67380a27a0 con 0x5652c2799a40 2024-03-30T02:46:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.534+0000 7f6733e00700 1 -- 172.21.15.67:0/271540955 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380986c0 con 0x5652c2799a40 2024-03-30T02:46:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 -- 172.21.15.67:0/271540955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 msgr2=0x5652c27a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/271540955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27a25f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f673809a2b0 tx=0x7f673800ef90 comp rx=0 tx=0).stop 2024-03-30T02:46:37.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 -- 172.21.15.67:0/271540955 shutdown_connections 2024-03-30T02:46:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/271540955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 0x5652c279d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/271540955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27a25f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/271540955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 0x5652c27abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 -- 172.21.15.67:0/271540955 >> 172.21.15.67:0/271540955 conn(0x5652c26f56a0 msgr2=0x5652c278a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 -- 172.21.15.67:0/271540955 shutdown_connections 2024-03-30T02:46:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 -- 172.21.15.67:0/271540955 wait complete. 2024-03-30T02:46:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 -- start start 2024-03-30T02:46:37.540 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.535+0000 7f6760e9cb80 1 Processor -- start 2024-03-30T02:46:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 0x5652c27aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6760e9cb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 0x5652c2807d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c27aad30 con 0x5652c27a5210 2024-03-30T02:46:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c28082b0 con 0x5652c2799a40 2024-03-30T02:46:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6760e9cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c2808440 con 0x5652c279a360 2024-03-30T02:46:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6760e9cb80 10 client.0 _ll_register_callbacks cb 0x7f673400f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6746800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6746800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27aa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40722/0 (socket says 172.21.15.67:40722) 2024-03-30T02:46:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6747200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 0x5652c2807d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6746800700 1 -- 172.21.15.67:0/77989514 learned_addr learned my addr 172.21.15.67:0/77989514 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6745e00700 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 0x5652c27aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6746800700 1 -- 172.21.15.67:0/77989514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 msgr2=0x5652c27aa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6746800700 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 0x5652c27aa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6746800700 1 -- 172.21.15.67:0/77989514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 msgr2=0x5652c2807d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.536+0000 7f6746800700 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 0x5652c2807d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6746800700 1 -- 172.21.15.67:0/77989514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652c25693b0 con 0x5652c2799a40 2024-03-30T02:46:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6746800700 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27aa0b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6738096d90 tx=0x7f673809ab20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38ac1734719f501b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6727e00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6727e00700 1 -- 172.21.15.67:0/77989514 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67380a6070 con 0x5652c2799a40 2024-03-30T02:46:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6727e00700 1 -- 172.21.15.67:0/77989514 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f673809e150 con 0x5652c2799a40 2024-03-30T02:46:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6760e9cb80 1 -- 172.21.15.67:0/77989514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f673809a320 con 0x5652c2799a40 2024-03-30T02:46:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6727e00700 1 -- 172.21.15.67:0/77989514 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f673807d070 con 0x5652c2799a40 2024-03-30T02:46:37.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6760e9cb80 10 client.27575 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6760e9cb80 1 -- 172.21.15.67:0/77989514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5652c256dcd0 con 0x5652c2799a40 2024-03-30T02:46:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6727e00700 1 -- 172.21.15.67:0/77989514 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f673809a320 con 0x5652c2799a40 2024-03-30T02:46:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6727e00700 1 -- 172.21.15.67:0/77989514 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67380a9030 con 0x5652c2799a40 2024-03-30T02:46:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6760e9cb80 10 client.27575 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6760e9cb80 10 client.27575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:46:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.537+0000 7f6760e9cb80 20 client.27575 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:37.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.538+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5652c2809710 0x5652c280bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.538+0000 7f6760e9cb80 4 client.27575 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:37.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.538+0000 7f6760e9cb80 1 -- 172.21.15.67:0/77989514 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5652c2536cf0 con 0x5652c2809710 2024-03-30T02:46:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.538+0000 7f6745e00700 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5652c2809710 0x5652c280bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.538+0000 7f6745e00700 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5652c2809710 0x5652c280bb00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eed4798e43a29429 server_cookie=cd4941df53e5cc2b in_seq=0 out_seq=0 2024-03-30T02:46:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.538+0000 7f6727e00700 10 client.27575 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:46:37.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.538+0000 7f6727e00700 1 -- 172.21.15.67:0/77989514 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5652c2536cf0 con 0x5652c2809710 2024-03-30T02:46:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.538+0000 7f6727e00700 10 client.27575 handle_command_reply: tid=0 2024-03-30T02:46:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.538+0000 7f6760e9cb80 1 client.27575 shutdown 2024-03-30T02:46:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 -- 172.21.15.67:0/77989514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 msgr2=0x5652c27aa0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27aa0b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6738096d90 tx=0x7f673809ab20 comp rx=0 tx=0).stop 2024-03-30T02:46:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 -- 172.21.15.67:0/77989514 shutdown_connections 2024-03-30T02:46:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x5652c2809710 0x5652c280bb00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5652c27a5210 0x5652c2807d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652c2799a40 0x5652c27aa0b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 --2- 172.21.15.67:0/77989514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652c279a360 0x5652c27aa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6727e00700 0 client.27575 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:46:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 -- 172.21.15.67:0/77989514 >> 172.21.15.67:0/77989514 conn(0x5652c26f56a0 msgr2=0x5652c279d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 -- 172.21.15.67:0/77989514 shutdown_connections 2024-03-30T02:46:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 1 -- 172.21.15.67:0/77989514 wait complete. 2024-03-30T02:46:37.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.539+0000 7f6760e9cb80 20 client.27575 trim_cache size 0 max 16384 2024-03-30T02:46:37.548 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 -- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f672c0a1290 msgr2=0x7f672c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f672c0a1290 0x7f672c0a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6734005e90 tx=0x7f6734005dc0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 -- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748100f70 msgr2=0x7f6748197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748100f70 0x7f6748197290 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f673800ec10 tx=0x7f673800ef20 comp rx=0 tx=0).stop 2024-03-30T02:46:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 -- 172.21.15.67:0/1858910593 shutdown_connections 2024-03-30T02:46:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f672c0a1290 0x7f672c0a3750 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6748100f70 0x7f6748197290 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f674819bb20 0x7f674819bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 --2- 172.21.15.67:0/1858910593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6748106090 0x7f67481977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 -- 172.21.15.67:0/1858910593 >> 172.21.15.67:0/1858910593 conn(0x7f67480fc7f0 msgr2=0x7f67480fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 -- 172.21.15.67:0/1858910593 shutdown_connections 2024-03-30T02:46:37.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.540+0000 7f674d400700 1 -- 172.21.15.67:0/1858910593 wait complete. 2024-03-30T02:46:37.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:46:37.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.759+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1507128407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b54106080 msgr2=0x7f5b5410dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.759+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1507128407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b54106080 0x7f5b5410dc00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00b0a0 tx=0x7f5b4c00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.759+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1507128407 shutdown_connections 2024-03-30T02:46:37.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.759+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1507128407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b54106080 0x7f5b5410dc00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.759+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1507128407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b54100f60 0x7f5b54101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.759+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1507128407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b54101910 0x7f5b54105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.759+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1507128407 >> 172.21.15.67:0/1507128407 conn(0x7f5b540fc7c0 msgr2=0x7f5b540febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.759+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1507128407 shutdown_connections 2024-03-30T02:46:37.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.759+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1507128407 wait complete. 2024-03-30T02:46:37.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.760+0000 7f5b5bc00700 1 Processor -- start 2024-03-30T02:46:37.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b5bc00700 1 -- start start 2024-03-30T02:46:37.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b5bc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b54100f60 0x7f5b54199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b5bc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b54106080 0x7f5b54199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b5bc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5419de10 0x7f5b5419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b5bc00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419e890 con 0x7f5b54100f60 2024-03-30T02:46:37.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b5bc00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419ea00 con 0x7f5b54106080 2024-03-30T02:46:37.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b5bc00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5419ebc0 con 0x7f5b5419de10 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b59200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b54100f60 0x7f5b54199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b59200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b54100f60 0x7f5b54199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39912/0 (socket says 172.21.15.67:39912) 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b53e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b54106080 0x7f5b54199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b59200700 1 -- 172.21.15.67:0/1344732738 learned_addr learned my addr 172.21.15.67:0/1344732738 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.761+0000 7f5b59c00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5419de10 0x7f5b5419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b59200700 1 -- 172.21.15.67:0/1344732738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5419de10 msgr2=0x7f5b5419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b59200700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5419de10 0x7f5b5419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b59200700 1 -- 172.21.15.67:0/1344732738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b54106080 msgr2=0x7f5b54199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b59200700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b54106080 0x7f5b54199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b59200700 1 -- 172.21.15.67:0/1344732738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c009d00 con 0x7f5b54100f60 2024-03-30T02:46:37.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b53e00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b54106080 0x7f5b54199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:37.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b59c00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5419de10 0x7f5b5419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:37.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b59200700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b54100f60 0x7f5b54199580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9a0 tx=0x7f5b4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b51600700 1 -- 172.21.15.67:0/1344732738 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48009790 con 0x7f5b54100f60 2024-03-30T02:46:37.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b51600700 1 -- 172.21.15.67:0/1344732738 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b48010430 con 0x7f5b54100f60 2024-03-30T02:46:37.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1344732738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b541a6120 con 0x7f5b54100f60 2024-03-30T02:46:37.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b51600700 1 -- 172.21.15.67:0/1344732738 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48018680 con 0x7f5b54100f60 2024-03-30T02:46:37.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.762+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1344732738 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541a6650 con 0x7f5b54100f60 2024-03-30T02:46:37.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.765+0000 7f5b51600700 1 -- 172.21.15.67:0/1344732738 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5b480105a0 con 0x7f5b54100f60 2024-03-30T02:46:37.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.767+0000 7f5b6f654b80 1 Processor -- start 2024-03-30T02:46:37.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.767+0000 7f5b6f654b80 1 -- start start 2024-03-30T02:46:37.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b51600700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5b3c0a1310 0x7f5b3c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b51600700 1 -- 172.21.15.67:0/1344732738 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f5b48014070 con 0x7f5b54100f60 2024-03-30T02:46:37.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 0x562c2e3279b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 0x562c2e3351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e280990 con 0x562c2e327ef0 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e27e350 con 0x562c2e332df0 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e279c40 con 0x562c2e3275d0 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b59c00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5b3c0a1310 0x7f5b3c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b53e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b53e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3328b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39928/0 (socket says 172.21.15.67:39928) 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b53e00700 1 -- 172.21.15.67:0/1860202940 learned_addr learned my addr 172.21.15.67:0/1860202940 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.768+0000 7f5b59200700 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 0x562c2e3279b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b59c00700 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 0x562c2e3351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b53e00700 1 -- 172.21.15.67:0/1860202940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 msgr2=0x562c2e3279b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b53e00700 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 0x562c2e3279b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b53e00700 1 -- 172.21.15.67:0/1860202940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 msgr2=0x562c2e3351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b53e00700 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 0x562c2e3351e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b53e00700 1 -- 172.21.15.67:0/1860202940 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c2e0fb3b0 con 0x562c2e327ef0 2024-03-30T02:46:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b59c00700 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 0x562c2e3351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:46:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b59c00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5b3c0a1310 0x7f5b3c0a37d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00b780 tx=0x7f5b4c0093c0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b53e00700 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3328b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5b400114f0 tx=0x7f5b40011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f95f215dc462f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b46000700 1 -- 172.21.15.67:0/1860202940 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40004a30 con 0x562c2e327ef0 2024-03-30T02:46:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b46000700 1 -- 172.21.15.67:0/1860202940 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b400054e0 con 0x562c2e327ef0 2024-03-30T02:46:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.769+0000 7f5b46000700 1 -- 172.21.15.67:0/1860202940 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000e8f0 con 0x562c2e327ef0 2024-03-30T02:46:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.770+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1860202940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 msgr2=0x562c2e3328b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.770+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3328b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5b400114f0 tx=0x7f5b40011800 comp rx=0 tx=0).stop 2024-03-30T02:46:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.770+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1860202940 shutdown_connections 2024-03-30T02:46:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.770+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 0x562c2e3351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.770+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 0x562c2e3279b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.770+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1860202940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3328b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.770+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1860202940 >> 172.21.15.67:0/1860202940 conn(0x562c2e2876a0 msgr2=0x562c2e323a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.770+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1860202940 shutdown_connections 2024-03-30T02:46:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.770+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1860202940 wait complete. 2024-03-30T02:46:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.771+0000 7f5b6f654b80 1 -- start start 2024-03-30T02:46:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.771+0000 7f5b6f654b80 1 Processor -- start 2024-03-30T02:46:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.771+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e3328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.771+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e32fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.771+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 0x562c2e330260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.771+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e215d80 con 0x562c2e327ef0 2024-03-30T02:46:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.771+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e215a20 con 0x562c2e3275d0 2024-03-30T02:46:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.771+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e0fd6a0 con 0x562c2e332df0 2024-03-30T02:46:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b6f654b80 10 client.0 _ll_register_callbacks cb 0x562c2e21f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b53e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e32fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b53e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e32fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39936/0 (socket says 172.21.15.67:39936) 2024-03-30T02:46:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b59c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 0x562c2e330260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b53e00700 1 -- 172.21.15.67:0/1710339020 learned_addr learned my addr 172.21.15.67:0/1710339020 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b59200700 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e3328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b53e00700 1 -- 172.21.15.67:0/1710339020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 msgr2=0x562c2e330260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b53e00700 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 0x562c2e330260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b53e00700 1 -- 172.21.15.67:0/1710339020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 msgr2=0x562c2e3328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b53e00700 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e3328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b53e00700 1 -- 172.21.15.67:0/1710339020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b40009c90 con 0x562c2e327ef0 2024-03-30T02:46:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b59200700 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e3328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b59c00700 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 0x562c2e330260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b53e00700 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e32fb20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5b400114c0 tx=0x7f5b400053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3565b8407cecbe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.772+0000 7f5b3aa00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b3aa00700 1 -- 172.21.15.67:0/1710339020 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40004960 con 0x562c2e327ef0 2024-03-30T02:46:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b3aa00700 1 -- 172.21.15.67:0/1710339020 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f5b40004ac0 con 0x562c2e327ef0 2024-03-30T02:46:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1710339020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c2e23a380 con 0x562c2e327ef0 2024-03-30T02:46:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b3aa00700 1 -- 172.21.15.67:0/1710339020 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b40004c80 con 0x562c2e327ef0 2024-03-30T02:46:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b6f654b80 10 client.17451 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1710339020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562c2e0a7b60 con 0x562c2e327ef0 2024-03-30T02:46:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b3aa00700 1 -- 172.21.15.67:0/1710339020 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c2e23a380 con 0x562c2e327ef0 2024-03-30T02:46:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b3aa00700 1 -- 172.21.15.67:0/1710339020 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b40021040 con 0x562c2e327ef0 2024-03-30T02:46:37.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b6f654b80 10 client.17451 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:37.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.773+0000 7f5b6f654b80 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:37.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.774+0000 7f5b6f654b80 20 client.17451 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:37.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.774+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562c2e342920 0x562c2e344d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.774+0000 7f5b6f654b80 4 client.17451 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:37.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.774+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1710339020 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c2e272700 con 0x562c2e342920 2024-03-30T02:46:37.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.774+0000 7f5b59200700 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562c2e342920 0x562c2e344d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.774+0000 7f5b59200700 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562c2e342920 0x562c2e344d40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30518240ad375d01 server_cookie=dbe5b1f41a79003a in_seq=0 out_seq=0 2024-03-30T02:46:37.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.774+0000 7f5b3aa00700 10 client.17451 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:37.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b3aa00700 1 -- 172.21.15.67:0/1710339020 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562c2e272700 con 0x562c2e342920 2024-03-30T02:46:37.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b3aa00700 10 client.17451 handle_command_reply: tid=0 2024-03-30T02:46:37.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 client.17451 shutdown 2024-03-30T02:46:37.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1710339020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 msgr2=0x562c2e32fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e32fb20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5b400114c0 tx=0x7f5b400053d0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1710339020 shutdown_connections 2024-03-30T02:46:37.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 0x562c2e330260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.782 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e3328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562c2e342920 0x562c2e344d40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/1710339020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e32fb20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b3aa00700 0 client.17451 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:37.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1710339020 >> 172.21.15.67:0/1710339020 conn(0x562c2e2876a0 msgr2=0x562c2e3296c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.775+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1710339020 shutdown_connections 2024-03-30T02:46:37.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.776+0000 7f5b6f654b80 1 -- 172.21.15.67:0/1710339020 wait complete. 2024-03-30T02:46:37.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.776+0000 7f5b6f654b80 20 client.17451 trim_cache size 0 max 16384 2024-03-30T02:46:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.789+0000 7f5b6f654b80 1 Processor -- start 2024-03-30T02:46:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.789+0000 7f5b6f654b80 1 -- start start 2024-03-30T02:46:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.789+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e330220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.789+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e339a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.789+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 0x562c2e3323d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.789+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e330a70 con 0x562c2e327ef0 2024-03-30T02:46:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.789+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e330860 con 0x562c2e3275d0 2024-03-30T02:46:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.789+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e339fb0 con 0x562c2e332df0 2024-03-30T02:46:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b53e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e339a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b53e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e339a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39952/0 (socket says 172.21.15.67:39952) 2024-03-30T02:46:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b53e00700 1 -- 172.21.15.67:0/2716342452 learned_addr learned my addr 172.21.15.67:0/2716342452 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b59200700 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e330220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b59c00700 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 0x562c2e3323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b53e00700 1 -- 172.21.15.67:0/2716342452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 msgr2=0x562c2e3323d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b53e00700 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 0x562c2e3323d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b53e00700 1 -- 172.21.15.67:0/2716342452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 msgr2=0x562c2e330220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b53e00700 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e330220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b53e00700 1 -- 172.21.15.67:0/2716342452 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c2e23fea0 con 0x562c2e327ef0 2024-03-30T02:46:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b59200700 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e330220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.790+0000 7f5b53e00700 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e339a70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5b400114f0 tx=0x7f5b40005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89eb866563dd22db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b46a00700 1 -- 172.21.15.67:0/2716342452 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000e5e0 con 0x562c2e327ef0 2024-03-30T02:46:37.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b46a00700 1 -- 172.21.15.67:0/2716342452 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b40003680 con 0x562c2e327ef0 2024-03-30T02:46:37.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b46a00700 1 -- 172.21.15.67:0/2716342452 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b400041c0 con 0x562c2e327ef0 2024-03-30T02:46:37.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b6f654b80 1 -- 172.21.15.67:0/2716342452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 msgr2=0x562c2e339a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e339a70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5b400114f0 tx=0x7f5b40005a00 comp rx=0 tx=0).stop 2024-03-30T02:46:37.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b6f654b80 1 -- 172.21.15.67:0/2716342452 shutdown_connections 2024-03-30T02:46:37.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e332df0 0x562c2e3323d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e3275d0 0x562c2e330220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/2716342452 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e339a70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b6f654b80 1 -- 172.21.15.67:0/2716342452 >> 172.21.15.67:0/2716342452 conn(0x562c2e2876a0 msgr2=0x562c2e31c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b6f654b80 1 -- 172.21.15.67:0/2716342452 shutdown_connections 2024-03-30T02:46:37.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.791+0000 7f5b6f654b80 1 -- 172.21.15.67:0/2716342452 wait complete. 2024-03-30T02:46:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.792+0000 7f5b6f654b80 1 -- start start 2024-03-30T02:46:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.792+0000 7f5b6f654b80 1 Processor -- start 2024-03-30T02:46:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.792+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 0x562c2e3371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.792+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.792+0000 7f5b6f654b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 0x562c2e337c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.792+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e395750 con 0x562c2e327ef0 2024-03-30T02:46:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.792+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e3958c0 con 0x562c2e332df0 2024-03-30T02:46:37.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.792+0000 7f5b6f654b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c2e395a80 con 0x562c2e3275d0 2024-03-30T02:46:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.792+0000 7f5b6f654b80 10 client.0 _ll_register_callbacks cb 0x562c2e21f470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b53e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b53e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3376f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39962/0 (socket says 172.21.15.67:39962) 2024-03-30T02:46:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b53e00700 1 -- 172.21.15.67:0/122475439 learned_addr learned my addr 172.21.15.67:0/122475439 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b59c00700 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 0x562c2e337c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b59200700 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 0x562c2e3371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b53e00700 1 -- 172.21.15.67:0/122475439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 msgr2=0x562c2e3371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b53e00700 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 0x562c2e3371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b53e00700 1 -- 172.21.15.67:0/122475439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 msgr2=0x562c2e337c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b53e00700 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 0x562c2e337c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b53e00700 1 -- 172.21.15.67:0/122475439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541a6650 con 0x562c2e327ef0 2024-03-30T02:46:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b53e00700 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3376f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5b40005fe0 tx=0x7f5b40004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511f54337b260b32 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b3aa00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.793+0000 7f5b3aa00700 1 -- 172.21.15.67:0/122475439 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40005340 con 0x562c2e327ef0 2024-03-30T02:46:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b3aa00700 1 -- 172.21.15.67:0/122475439 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f5b40004d80 con 0x562c2e327ef0 2024-03-30T02:46:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b6f654b80 1 -- 172.21.15.67:0/122475439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b400054e0 con 0x562c2e327ef0 2024-03-30T02:46:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b3aa00700 1 -- 172.21.15.67:0/122475439 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b4000e5e0 con 0x562c2e327ef0 2024-03-30T02:46:37.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b6f654b80 10 client.17457 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b6f654b80 1 -- 172.21.15.67:0/122475439 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562c2e0a7b60 con 0x562c2e327ef0 2024-03-30T02:46:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b3aa00700 1 -- 172.21.15.67:0/122475439 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b40048020 con 0x562c2e327ef0 2024-03-30T02:46:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b3aa00700 1 -- 172.21.15.67:0/122475439 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5b40021040 con 0x562c2e327ef0 2024-03-30T02:46:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b6f654b80 10 client.17457 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b6f654b80 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b6f654b80 20 client.17457 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562c2e396d50 0x562c2e399170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b6f654b80 4 client.17457 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:46:37.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.794+0000 7f5b6f654b80 1 -- 172.21.15.67:0/122475439 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c2e0c8cf0 con 0x562c2e396d50 2024-03-30T02:46:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b59200700 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562c2e396d50 0x562c2e399170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b59200700 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562c2e396d50 0x562c2e399170 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a364f658e54133df server_cookie=28c9cf4e8f6bc554 in_seq=0 out_seq=0 2024-03-30T02:46:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b3aa00700 10 client.17457 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b3aa00700 1 -- 172.21.15.67:0/122475439 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c2e0c8cf0 con 0x562c2e396d50 2024-03-30T02:46:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b3aa00700 10 client.17457 handle_command_reply: tid=0 2024-03-30T02:46:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b6f654b80 1 client.17457 shutdown 2024-03-30T02:46:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b6f654b80 1 -- 172.21.15.67:0/122475439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 msgr2=0x562c2e3376f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3376f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5b40005fe0 tx=0x7f5b40004b90 comp rx=0 tx=0).stop 2024-03-30T02:46:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b6f654b80 1 -- 172.21.15.67:0/122475439 shutdown_connections 2024-03-30T02:46:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562c2e332df0 0x562c2e337c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c2e3275d0 0x562c2e3371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x562c2e396d50 0x562c2e399170 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b6f654b80 1 --2- 172.21.15.67:0/122475439 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562c2e327ef0 0x562c2e3376f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b3aa00700 0 client.17457 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.795+0000 7f5b6f654b80 1 -- 172.21.15.67:0/122475439 >> 172.21.15.67:0/122475439 conn(0x562c2e2876a0 msgr2=0x562c2e32adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.796+0000 7f5b6f654b80 1 -- 172.21.15.67:0/122475439 shutdown_connections 2024-03-30T02:46:37.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.796+0000 7f5b6f654b80 1 -- 172.21.15.67:0/122475439 wait complete. 2024-03-30T02:46:37.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.796+0000 7f5b6f654b80 20 client.17457 trim_cache size 0 max 16384 2024-03-30T02:46:37.804 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:46:37.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.796+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5b3c0a1310 msgr2=0x7f5b3c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.796+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5b3c0a1310 0x7f5b3c0a37d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00b780 tx=0x7f5b4c0093c0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.796+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b54100f60 msgr2=0x7f5b54199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:37.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.796+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b54100f60 0x7f5b54199580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9a0 tx=0x7f5b4800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.797+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1344732738 shutdown_connections 2024-03-30T02:46:37.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.797+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5b3c0a1310 0x7f5b3c0a37d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.797+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5b54100f60 0x7f5b54199580 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.797+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b5419de10 0x7f5b5419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.797+0000 7f5b5bc00700 1 --2- 172.21.15.67:0/1344732738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b54106080 0x7f5b54199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:37.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.797+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1344732738 >> 172.21.15.67:0/1344732738 conn(0x7f5b540fc7c0 msgr2=0x7f5b540feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:37.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.797+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1344732738 shutdown_connections 2024-03-30T02:46:37.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:37.797+0000 7f5b5bc00700 1 -- 172.21.15.67:0/1344732738 wait complete. 2024-03-30T02:46:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:38 smithi082 ceph-mon[79176]: pgmap v296: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 822 KiB/s wr, 11 op/s 2024-03-30T02:46:38.816 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-30T02:46:38.816 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-30T02:46:38.883 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:38 smithi067 ceph-mon[37808]: pgmap v296: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 822 KiB/s wr, 11 op/s 2024-03-30T02:46:38.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:38 smithi170 ceph-mon[78798]: pgmap v296: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 822 KiB/s wr, 11 op/s 2024-03-30T02:46:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.034+0000 7f38bd400700 1 -- 172.21.15.67:0/3620794661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8101910 msgr2=0x7f38b8105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.034+0000 7f38bd400700 1 --2- 172.21.15.67:0/3620794661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8101910 0x7f38b8105950 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009990 tx=0x7f38a0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.034+0000 7f38bd400700 1 -- 172.21.15.67:0/3620794661 shutdown_connections 2024-03-30T02:46:39.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.034+0000 7f38bd400700 1 --2- 172.21.15.67:0/3620794661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b8106080 0x7f38b810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.034+0000 7f38bd400700 1 --2- 172.21.15.67:0/3620794661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b8100f60 0x7f38b8101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.034+0000 7f38bd400700 1 --2- 172.21.15.67:0/3620794661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8101910 0x7f38b8105950 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.034+0000 7f38bd400700 1 -- 172.21.15.67:0/3620794661 >> 172.21.15.67:0/3620794661 conn(0x7f38b80fc7c0 msgr2=0x7f38b80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.034+0000 7f38bd400700 1 -- 172.21.15.67:0/3620794661 shutdown_connections 2024-03-30T02:46:39.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.034+0000 7f38bd400700 1 -- 172.21.15.67:0/3620794661 wait complete. 2024-03-30T02:46:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.035+0000 7f38bd400700 1 Processor -- start 2024-03-30T02:46:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.035+0000 7f38bd400700 1 -- start start 2024-03-30T02:46:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.035+0000 7f38bd400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b8100f60 0x7f38b80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.035+0000 7f38bd400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8106080 0x7f38b80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.035+0000 7f38bd400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b8074d70 0x7f38b8073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.035+0000 7f38bd400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81a9e90 con 0x7f38b8106080 2024-03-30T02:46:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.035+0000 7f38bd400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81aa000 con 0x7f38b8100f60 2024-03-30T02:46:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.035+0000 7f38bd400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81aa1c0 con 0x7f38b8074d70 2024-03-30T02:46:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38afe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8106080 0x7f38b80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38afe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8106080 0x7f38b80735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39994/0 (socket says 172.21.15.67:39994) 2024-03-30T02:46:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38afe00700 1 -- 172.21.15.67:0/199461024 learned_addr learned my addr 172.21.15.67:0/199461024 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38b6800700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b8100f60 0x7f38b80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38b7200700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b8074d70 0x7f38b8073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38afe00700 1 -- 172.21.15.67:0/199461024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b8074d70 msgr2=0x7f38b8073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38afe00700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b8074d70 0x7f38b8073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38afe00700 1 -- 172.21.15.67:0/199461024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b8100f60 msgr2=0x7f38b80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38afe00700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b8100f60 0x7f38b80730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38afe00700 1 -- 172.21.15.67:0/199461024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00095f0 con 0x7f38b8106080 2024-03-30T02:46:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38b7200700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b8074d70 0x7f38b8073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38afe00700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8106080 0x7f38b80735f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f38a0000c00 tx=0x7f38a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.036+0000 7f38af400700 1 -- 172.21.15.67:0/199461024 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a001d070 con 0x7f38b8106080 2024-03-30T02:46:39.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.037+0000 7f38af400700 1 -- 172.21.15.67:0/199461024 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a0021410 con 0x7f38b8106080 2024-03-30T02:46:39.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.037+0000 7f38af400700 1 -- 172.21.15.67:0/199461024 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0017650 con 0x7f38b8106080 2024-03-30T02:46:39.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.037+0000 7f38bd400700 1 -- 172.21.15.67:0/199461024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b81aa440 con 0x7f38b8106080 2024-03-30T02:46:39.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.037+0000 7f38bd400700 1 -- 172.21.15.67:0/199461024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b81aa860 con 0x7f38b8106080 2024-03-30T02:46:39.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.040+0000 7f38af400700 1 -- 172.21.15.67:0/199461024 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f38a00217f0 con 0x7f38b8106080 2024-03-30T02:46:39.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38d0ee1b80 1 Processor -- start 2024-03-30T02:46:39.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38d0ee1b80 1 -- start start 2024-03-30T02:46:39.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38af400700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f389c0a1310 0x7f389c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d87aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87af70 0x560d1d885930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38af400700 1 -- 172.21.15.67:0/199461024 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f38a00ce2d0 con 0x7f38b8106080 2024-03-30T02:46:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d888260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d7d3a10 con 0x560d1d885e70 2024-03-30T02:46:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d7d13d0 con 0x560d1d87a650 2024-03-30T02:46:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d7cccc0 con 0x560d1d87af70 2024-03-30T02:46:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.041+0000 7f38b7200700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f389c0a1310 0x7f389c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b6800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d87aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b6800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d87aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40790/0 (socket says 172.21.15.67:40790) 2024-03-30T02:46:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b6800700 1 -- 172.21.15.67:0/1277869848 learned_addr learned my addr 172.21.15.67:0/1277869848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38afe00700 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87af70 0x560d1d885930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b7200700 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d888260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b6800700 1 -- 172.21.15.67:0/1277869848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87af70 msgr2=0x560d1d885930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b6800700 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87af70 0x560d1d885930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b6800700 1 -- 172.21.15.67:0/1277869848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 msgr2=0x560d1d888260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b6800700 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d888260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b6800700 1 -- 172.21.15.67:0/1277869848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d1d64e430 con 0x560d1d87a650 2024-03-30T02:46:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b7200700 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d888260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:46:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b6800700 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d87aa30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009d80 tx=0x7f38a8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=252c1a25d02168c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.042+0000 7f38b7200700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f389c0a1310 0x7f389c0a37d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38b80751c0 tx=0x7f38b0009750 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38a7400700 1 -- 172.21.15.67:0/1277869848 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8004230 con 0x560d1d87a650 2024-03-30T02:46:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38a7400700 1 -- 172.21.15.67:0/1277869848 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a8004390 con 0x560d1d87a650 2024-03-30T02:46:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38a7400700 1 -- 172.21.15.67:0/1277869848 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8019820 con 0x560d1d87a650 2024-03-30T02:46:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/1277869848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 msgr2=0x560d1d87aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d87aa30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009d80 tx=0x7f38a8011720 comp rx=0 tx=0).stop 2024-03-30T02:46:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/1277869848 shutdown_connections 2024-03-30T02:46:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d888260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d87aa30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/1277869848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87af70 0x560d1d885930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.043+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/1277869848 >> 172.21.15.67:0/1277869848 conn(0x560d1d7da720 msgr2=0x560d1d876b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.044+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/1277869848 shutdown_connections 2024-03-30T02:46:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.044+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/1277869848 wait complete. 2024-03-30T02:46:39.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.044+0000 7f38d0ee1b80 1 -- start start 2024-03-30T02:46:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.044+0000 7f38d0ee1b80 1 Processor -- start 2024-03-30T02:46:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 0x560d1d8857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87af70 0x560d1d882a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d883010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d768e00 con 0x560d1d885e70 2024-03-30T02:46:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d768aa0 con 0x560d1d87af70 2024-03-30T02:46:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d650720 con 0x560d1d87a650 2024-03-30T02:46:39.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38d0ee1b80 10 client.0 _ll_register_callbacks cb 0x560d1d7724f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38b7200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d883010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38b7200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d883010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40006/0 (socket says 172.21.15.67:40006) 2024-03-30T02:46:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38b7200700 1 -- 172.21.15.67:0/3576215083 learned_addr learned my addr 172.21.15.67:0/3576215083 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38afe00700 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87af70 0x560d1d882a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38b6800700 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 0x560d1d8857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38b7200700 1 -- 172.21.15.67:0/3576215083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 msgr2=0x560d1d8857c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38b7200700 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 0x560d1d8857c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38b7200700 1 -- 172.21.15.67:0/3576215083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87af70 msgr2=0x560d1d882a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38b7200700 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87af70 0x560d1d882a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38b7200700 1 -- 172.21.15.67:0/3576215083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a80099e0 con 0x560d1d885e70 2024-03-30T02:46:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.045+0000 7f38afe00700 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87af70 0x560d1d882a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.046+0000 7f38b7200700 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d883010 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f38b00042e0 tx=0x7f38b0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97381545694bd68 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.046+0000 7f389be00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.046+0000 7f389be00700 1 -- 172.21.15.67:0/3576215083 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b001f070 con 0x560d1d885e70 2024-03-30T02:46:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.046+0000 7f389be00700 1 -- 172.21.15.67:0/3576215083 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f38b0019610 con 0x560d1d885e70 2024-03-30T02:46:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.046+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3576215083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d1d78d400 con 0x560d1d885e70 2024-03-30T02:46:39.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.046+0000 7f389be00700 1 -- 172.21.15.67:0/3576215083 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38b00197d0 con 0x560d1d885e70 2024-03-30T02:46:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.046+0000 7f38d0ee1b80 10 client.17469 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.046+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3576215083 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560d1d5fabe0 con 0x560d1d885e70 2024-03-30T02:46:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.047+0000 7f389be00700 1 -- 172.21.15.67:0/3576215083 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d1d78d400 con 0x560d1d885e70 2024-03-30T02:46:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.047+0000 7f389be00700 1 -- 172.21.15.67:0/3576215083 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f38b00249a0 con 0x560d1d885e70 2024-03-30T02:46:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.047+0000 7f38d0ee1b80 10 client.17469 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.047+0000 7f38d0ee1b80 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.047+0000 7f38d0ee1b80 20 client.17469 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.047+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560d1d895a10 0x560d1d897e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.047+0000 7f38d0ee1b80 4 client.17469 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.047+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3576215083 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d1d7c5780 con 0x560d1d895a10 2024-03-30T02:46:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.047+0000 7f38b6800700 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560d1d895a10 0x560d1d897e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.048+0000 7f38b6800700 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560d1d895a10 0x560d1d897e30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e795199ebd3942ff server_cookie=cb008773737aa29b in_seq=0 out_seq=0 2024-03-30T02:46:39.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.048+0000 7f389be00700 10 client.17469 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f389be00700 1 -- 172.21.15.67:0/3576215083 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560d1d7c5780 con 0x560d1d895a10 2024-03-30T02:46:39.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f389be00700 10 client.17469 handle_command_reply: tid=0 2024-03-30T02:46:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 client.17469 shutdown 2024-03-30T02:46:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3576215083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 msgr2=0x560d1d883010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d883010 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f38b00042e0 tx=0x7f38b0009040 comp rx=0 tx=0).stop 2024-03-30T02:46:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3576215083 shutdown_connections 2024-03-30T02:46:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560d1d895a10 0x560d1d897e30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d885e70 0x560d1d883010 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 0x560d1d8857c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3576215083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87af70 0x560d1d882a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3576215083 >> 172.21.15.67:0/3576215083 conn(0x560d1d7da720 msgr2=0x560d1d87c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f389be00700 0 client.17469 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3576215083 shutdown_connections 2024-03-30T02:46:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3576215083 wait complete. 2024-03-30T02:46:39.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.049+0000 7f38d0ee1b80 20 client.17469 trim_cache size 0 max 16384 2024-03-30T02:46:39.067 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.067+0000 7f38d0ee1b80 1 Processor -- start 2024-03-30T02:46:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.067+0000 7f38d0ee1b80 1 -- start start 2024-03-30T02:46:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.067+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 0x560d1d88caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.067+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d891240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.067+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d885e70 0x560d1d891780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.067+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d88d060 con 0x560d1d87af70 2024-03-30T02:46:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.067+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d883610 con 0x560d1d885e70 2024-03-30T02:46:39.068 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.067+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d881000 con 0x560d1d87a650 2024-03-30T02:46:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38afe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d891240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38afe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d891240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40022/0 (socket says 172.21.15.67:40022) 2024-03-30T02:46:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38afe00700 1 -- 172.21.15.67:0/2065210613 learned_addr learned my addr 172.21.15.67:0/2065210613 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38b7200700 1 --2- 172.21.15.67:0/2065210613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d885e70 0x560d1d891780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38b6800700 1 --2- 172.21.15.67:0/2065210613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 0x560d1d88caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38afe00700 1 -- 172.21.15.67:0/2065210613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 msgr2=0x560d1d88caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38afe00700 1 --2- 172.21.15.67:0/2065210613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 0x560d1d88caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38afe00700 1 -- 172.21.15.67:0/2065210613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d885e70 msgr2=0x560d1d891780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38afe00700 1 --2- 172.21.15.67:0/2065210613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d885e70 0x560d1d891780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.068+0000 7f38afe00700 1 -- 172.21.15.67:0/2065210613 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d1d792f20 con 0x560d1d87af70 2024-03-30T02:46:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38afe00700 1 --2- 172.21.15.67:0/2065210613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d891240 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f38a009ae80 tx=0x7f38a00a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d922c80f93ce94 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38a7e00700 1 -- 172.21.15.67:0/2065210613 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a009aae0 con 0x560d1d87af70 2024-03-30T02:46:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38a7e00700 1 -- 172.21.15.67:0/2065210613 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a009b880 con 0x560d1d87af70 2024-03-30T02:46:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38a7e00700 1 -- 172.21.15.67:0/2065210613 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a000fd30 con 0x560d1d87af70 2024-03-30T02:46:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/2065210613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 msgr2=0x560d1d891240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/2065210613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d891240 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f38a009ae80 tx=0x7f38a00a1040 comp rx=0 tx=0).stop 2024-03-30T02:46:39.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/2065210613 shutdown_connections 2024-03-30T02:46:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/2065210613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d885e70 0x560d1d891780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/2065210613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d87a650 0x560d1d88caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/2065210613 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d891240 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.069+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/2065210613 >> 172.21.15.67:0/2065210613 conn(0x560d1d7da720 msgr2=0x560d1d881380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.070+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/2065210613 shutdown_connections 2024-03-30T02:46:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.070+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/2065210613 wait complete. 2024-03-30T02:46:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.070+0000 7f38d0ee1b80 1 -- start start 2024-03-30T02:46:39.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.070+0000 7f38d0ee1b80 1 Processor -- start 2024-03-30T02:46:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d88b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d88b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38d0ee1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d885e70 0x560d1d88bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d88cff0 con 0x560d1d87af70 2024-03-30T02:46:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d8e9850 con 0x560d1d87a650 2024-03-30T02:46:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38d0ee1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d1d8e9990 con 0x560d1d885e70 2024-03-30T02:46:39.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38d0ee1b80 10 client.0 _ll_register_callbacks cb 0x560d1d7724f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38afe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d88b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38afe00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d88b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40032/0 (socket says 172.21.15.67:40032) 2024-03-30T02:46:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38afe00700 1 -- 172.21.15.67:0/3510341544 learned_addr learned my addr 172.21.15.67:0/3510341544 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38b6800700 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d88b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38b7200700 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d885e70 0x560d1d88bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38afe00700 1 -- 172.21.15.67:0/3510341544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d885e70 msgr2=0x560d1d88bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38afe00700 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d885e70 0x560d1d88bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38afe00700 1 -- 172.21.15.67:0/3510341544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 msgr2=0x560d1d88b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38afe00700 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d88b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38afe00700 1 -- 172.21.15.67:0/3510341544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b81aa860 con 0x560d1d87af70 2024-03-30T02:46:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.071+0000 7f38b6800700 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d88b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f38afe00700 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d88b6f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f38a000b2e0 tx=0x7f38a0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42de8b1735acd68 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f389be00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:46:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f389be00700 1 -- 172.21.15.67:0/3510341544 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a009aae0 con 0x560d1d87af70 2024-03-30T02:46:39.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3510341544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38a8004390 con 0x560d1d87af70 2024-03-30T02:46:39.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f389be00700 1 -- 172.21.15.67:0/3510341544 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f38a009d070 con 0x560d1d87af70 2024-03-30T02:46:39.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f389be00700 1 -- 172.21.15.67:0/3510341544 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a009b180 con 0x560d1d87af70 2024-03-30T02:46:39.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f38d0ee1b80 10 client.17475 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:39.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3510341544 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560d1d652d50 con 0x560d1d87af70 2024-03-30T02:46:39.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f389be00700 1 -- 172.21.15.67:0/3510341544 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a00a7080 con 0x560d1d87af70 2024-03-30T02:46:39.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f389be00700 1 -- 172.21.15.67:0/3510341544 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f38a009a670 con 0x560d1d87af70 2024-03-30T02:46:39.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f38d0ee1b80 10 client.17475 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:39.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f38d0ee1b80 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:39.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.072+0000 7f38d0ee1b80 20 client.17475 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:39.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.073+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560d1d8eaca0 0x560d1d8ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.073+0000 7f38d0ee1b80 4 client.17475 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-30T02:46:39.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.073+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3510341544 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560d1d61bd70 con 0x560d1d8eaca0 2024-03-30T02:46:39.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.073+0000 7f38b6800700 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560d1d8eaca0 0x560d1d8ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.081 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.073+0000 7f38b6800700 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560d1d8eaca0 0x560d1d8ed090 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b6f9b1dcad5f27 server_cookie=8efefd9dd17f87e5 in_seq=0 out_seq=0 2024-03-30T02:46:39.082 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.073+0000 7f389be00700 10 client.17475 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:39.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.074+0000 7f389be00700 1 -- 172.21.15.67:0/3510341544 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560d1d61bd70 con 0x560d1d8eaca0 2024-03-30T02:46:39.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.074+0000 7f389be00700 10 client.17475 handle_command_reply: tid=0 2024-03-30T02:46:39.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.074+0000 7f38d0ee1b80 1 client.17475 shutdown 2024-03-30T02:46:39.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3510341544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 msgr2=0x560d1d88b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d88b6f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f38a000b2e0 tx=0x7f38a0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3510341544 shutdown_connections 2024-03-30T02:46:39.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d1d885e70 0x560d1d88bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d1d87a650 0x560d1d88b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x560d1d8eaca0 0x560d1d8ed090 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 --2- 172.21.15.67:0/3510341544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x560d1d87af70 0x560d1d88b6f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3510341544 >> 172.21.15.67:0/3510341544 conn(0x560d1d7da720 msgr2=0x560d1d87c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f389be00700 0 client.17475 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:39.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3510341544 shutdown_connections 2024-03-30T02:46:39.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 1 -- 172.21.15.67:0/3510341544 wait complete. 2024-03-30T02:46:39.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.075+0000 7f38d0ee1b80 20 client.17475 trim_cache size 0 max 16384 2024-03-30T02:46:39.085 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:46:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-03-30T02:46:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "4c12ba12-215d-4f72-80db-5ca40aaeb957", 2024-03-30T02:46:39.085 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-03-30T02:46:39.085 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:46:39.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 -- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f389c0a1310 msgr2=0x7f389c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f389c0a1310 0x7f389c0a37d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38b80751c0 tx=0x7f38b0009750 comp rx=0 tx=0).stop 2024-03-30T02:46:39.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 -- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8106080 msgr2=0x7f38b80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8106080 0x7f38b80735f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f38a0000c00 tx=0x7f38a0004730 comp rx=0 tx=0).stop 2024-03-30T02:46:39.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 -- 172.21.15.67:0/199461024 shutdown_connections 2024-03-30T02:46:39.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f389c0a1310 0x7f389c0a37d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38b8074d70 0x7f38b8073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b8100f60 0x7f38b80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 --2- 172.21.15.67:0/199461024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38b8106080 0x7f38b80735f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 -- 172.21.15.67:0/199461024 >> 172.21.15.67:0/199461024 conn(0x7f38b80fc7c0 msgr2=0x7f38b80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 -- 172.21.15.67:0/199461024 shutdown_connections 2024-03-30T02:46:39.088 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.076+0000 7f38bd400700 1 -- 172.21.15.67:0/199461024 wait complete. 2024-03-30T02:46:39.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:46:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.339+0000 7f43da000700 1 -- 172.21.15.67:0/2427737792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d4101900 msgr2=0x7f43d4105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.339+0000 7f43da000700 1 --2- 172.21.15.67:0/2427737792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d4101900 0x7f43d4105940 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009990 tx=0x7f43c0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.339+0000 7f43da000700 1 -- 172.21.15.67:0/2427737792 shutdown_connections 2024-03-30T02:46:39.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.339+0000 7f43da000700 1 --2- 172.21.15.67:0/2427737792 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d4100f50 0x7f43d4101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.339+0000 7f43da000700 1 --2- 172.21.15.67:0/2427737792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d4106070 0x7f43d410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.339+0000 7f43da000700 1 --2- 172.21.15.67:0/2427737792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d4101900 0x7f43d4105940 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.339+0000 7f43da000700 1 -- 172.21.15.67:0/2427737792 >> 172.21.15.67:0/2427737792 conn(0x7f43d40fc7f0 msgr2=0x7f43d40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.339+0000 7f43da000700 1 -- 172.21.15.67:0/2427737792 shutdown_connections 2024-03-30T02:46:39.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.339+0000 7f43da000700 1 -- 172.21.15.67:0/2427737792 wait complete. 2024-03-30T02:46:39.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.341+0000 7f43da000700 1 Processor -- start 2024-03-30T02:46:39.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.341+0000 7f43da000700 1 -- start start 2024-03-30T02:46:39.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.341+0000 7f43da000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d4100f50 0x7f43d4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43da000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d4106070 0x7f43d4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43da000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d419baa0 0x7f43d419bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43da000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d419c520 con 0x7f43d4106070 2024-03-30T02:46:39.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43da000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d419c690 con 0x7f43d419baa0 2024-03-30T02:46:39.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43da000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d419c850 con 0x7f43d4100f50 2024-03-30T02:46:39.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d4100f50 0x7f43d4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d4106070 0x7f43d4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d419baa0 0x7f43d419bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d419baa0 0x7f43d419bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40842/0 (socket says 172.21.15.67:40842) 2024-03-30T02:46:39.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d2a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d4106070 0x7f43d4197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40048/0 (socket says 172.21.15.67:40048) 2024-03-30T02:46:39.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d4100f50 0x7f43d4197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:44536/0 (socket says 172.21.15.67:44536) 2024-03-30T02:46:39.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3e00700 1 -- 172.21.15.67:0/4189190962 learned_addr learned my addr 172.21.15.67:0/4189190962 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3400700 1 -- 172.21.15.67:0/4189190962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d419baa0 msgr2=0x7f43d419bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3400700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d419baa0 0x7f43d419bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3400700 1 -- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d4106070 msgr2=0x7f43d4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3400700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d4106070 0x7f43d4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d3400700 1 -- 172.21.15.67:0/4189190962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c00095f0 con 0x7f43d4100f50 2024-03-30T02:46:39.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.342+0000 7f43d2a00700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d4106070 0x7f43d4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:39.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.343+0000 7f43d3400700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d4100f50 0x7f43d4197210 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9f0 tx=0x7f43c400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.343+0000 7f43cbe00700 1 -- 172.21.15.67:0/4189190962 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400c9c0 con 0x7f43d4100f50 2024-03-30T02:46:39.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.343+0000 7f43cbe00700 1 -- 172.21.15.67:0/4189190962 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c4010430 con 0x7f43d4100f50 2024-03-30T02:46:39.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.343+0000 7f43da000700 1 -- 172.21.15.67:0/4189190962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d41a3db0 con 0x7f43d4100f50 2024-03-30T02:46:39.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.343+0000 7f43cbe00700 1 -- 172.21.15.67:0/4189190962 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4018680 con 0x7f43d4100f50 2024-03-30T02:46:39.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.343+0000 7f43da000700 1 -- 172.21.15.67:0/4189190962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d41a4230 con 0x7f43d4100f50 2024-03-30T02:46:39.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.346+0000 7f43cbe00700 1 -- 172.21.15.67:0/4189190962 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f43c4010860 con 0x7f43d4100f50 2024-03-30T02:46:39.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.347+0000 7f43edc0db80 1 Processor -- start 2024-03-30T02:46:39.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43cbe00700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f43b80a1290 0x7f43b80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43edc0db80 1 -- start start 2024-03-30T02:46:39.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43cbe00700 1 -- 172.21.15.67:0/4189190962 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f43c4014070 con 0x7f43d4100f50 2024-03-30T02:46:39.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43d3e00700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f43b80a1290 0x7f43b80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec75a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec7658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43edc0db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec6b3990 con 0x556aec765df0 2024-03-30T02:46:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43edc0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec6b1350 con 0x556aec75a5d0 2024-03-30T02:46:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43edc0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec6acc40 con 0x556aec75aef0 2024-03-30T02:46:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43d3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec75a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43d3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec75a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40850/0 (socket says 172.21.15.67:40850) 2024-03-30T02:46:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43d3400700 1 -- 172.21.15.67:0/2680860292 learned_addr learned my addr 172.21.15.67:0/2680860292 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.348+0000 7f43d2a00700 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec7658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43d3e00700 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43d3400700 1 -- 172.21.15.67:0/2680860292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 msgr2=0x556aec7658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43d3400700 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec7658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43d3400700 1 -- 172.21.15.67:0/2680860292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 msgr2=0x556aec7681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43d3400700 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43d3400700 1 -- 172.21.15.67:0/2680860292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aec52e3b0 con 0x556aec75a5d0 2024-03-30T02:46:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43d3e00700 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:46:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43d3e00700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f43b80a1290 0x7f43b80a3750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009c10 tx=0x7f43cc009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43d3400700 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec75a9b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f43c4096970 tx=0x7f43c4099960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c51bac322327620 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43bfe00700 1 -- 172.21.15.67:0/2680860292 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c409dd30 con 0x556aec75a5d0 2024-03-30T02:46:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43bfe00700 1 -- 172.21.15.67:0/2680860292 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c40a2380 con 0x556aec75a5d0 2024-03-30T02:46:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.349+0000 7f43bfe00700 1 -- 172.21.15.67:0/2680860292 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c40995f0 con 0x556aec75a5d0 2024-03-30T02:46:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.350+0000 7f43edc0db80 1 -- 172.21.15.67:0/2680860292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 msgr2=0x556aec75a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.350+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec75a9b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f43c4096970 tx=0x7f43c4099960 comp rx=0 tx=0).stop 2024-03-30T02:46:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.350+0000 7f43edc0db80 1 -- 172.21.15.67:0/2680860292 shutdown_connections 2024-03-30T02:46:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.350+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.350+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec75a9b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.350+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2680860292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec7658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.350+0000 7f43edc0db80 1 -- 172.21.15.67:0/2680860292 >> 172.21.15.67:0/2680860292 conn(0x556aec6ba6a0 msgr2=0x556aec756a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.350+0000 7f43edc0db80 1 -- 172.21.15.67:0/2680860292 shutdown_connections 2024-03-30T02:46:39.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.350+0000 7f43edc0db80 1 -- 172.21.15.67:0/2680860292 wait complete. 2024-03-30T02:46:39.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.351+0000 7f43edc0db80 1 -- start start 2024-03-30T02:46:39.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.351+0000 7f43edc0db80 1 Processor -- start 2024-03-30T02:46:39.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.351+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec7657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.351+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec762b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.351+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.351+0000 7f43edc0db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec648d80 con 0x556aec765df0 2024-03-30T02:46:39.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.351+0000 7f43edc0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec648a20 con 0x556aec75a5d0 2024-03-30T02:46:39.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.351+0000 7f43edc0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec5306a0 con 0x556aec75aef0 2024-03-30T02:46:39.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.351+0000 7f43edc0db80 10 client.0 _ll_register_callbacks cb 0x556aec652470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:39.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec7657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d2a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec762b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec7657b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40860/0 (socket says 172.21.15.67:40860) 2024-03-30T02:46:39.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3400700 1 -- 172.21.15.67:0/349822780 learned_addr learned my addr 172.21.15.67:0/349822780 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3e00700 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3400700 1 -- 172.21.15.67:0/349822780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 msgr2=0x556aec762b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3400700 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec762b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3400700 1 -- 172.21.15.67:0/349822780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 msgr2=0x556aec7631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3400700 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3400700 1 -- 172.21.15.67:0/349822780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d41a4230 con 0x556aec75a5d0 2024-03-30T02:46:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43d3400700 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec7657b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f43c40caac0 tx=0x7f43c409a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5331118da82bd755 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43bcc00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43bcc00700 1 -- 172.21.15.67:0/349822780 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c409dd30 con 0x556aec75a5d0 2024-03-30T02:46:39.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43bcc00700 1 -- 172.21.15.67:0/349822780 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f43c40a2100 con 0x556aec75a5d0 2024-03-30T02:46:39.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.352+0000 7f43edc0db80 1 -- 172.21.15.67:0/349822780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556aec66d380 con 0x556aec75a5d0 2024-03-30T02:46:39.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43bcc00700 1 -- 172.21.15.67:0/349822780 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43c400cb20 con 0x556aec75a5d0 2024-03-30T02:46:39.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43edc0db80 10 client.27620 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:39.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43edc0db80 1 -- 172.21.15.67:0/349822780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556aec4dab60 con 0x556aec75a5d0 2024-03-30T02:46:39.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43bcc00700 1 -- 172.21.15.67:0/349822780 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c407c020 con 0x556aec75a5d0 2024-03-30T02:46:39.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43bcc00700 1 -- 172.21.15.67:0/349822780 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43c4095d80 con 0x556aec75a5d0 2024-03-30T02:46:39.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43edc0db80 10 client.27620 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:39.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43edc0db80 10 client.27620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:39.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43edc0db80 20 client.27620 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:39.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43edc0db80 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556aec7799a0 0x556aec77bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43edc0db80 4 client.27620 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:46:39.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.353+0000 7f43edc0db80 1 -- 172.21.15.67:0/349822780 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556aec6a5700 con 0x556aec7799a0 2024-03-30T02:46:39.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.354+0000 7f43d2a00700 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556aec7799a0 0x556aec77bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.354+0000 7f43d2a00700 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556aec7799a0 0x556aec77bd90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a8a1c1038644b2 server_cookie=530c6fc45ecb3816 in_seq=0 out_seq=0 2024-03-30T02:46:39.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.354+0000 7f43bcc00700 10 client.27620 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.374+0000 7f43bcc00700 1 -- 172.21.15.67:0/349822780 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556aec6a5700 con 0x556aec7799a0 2024-03-30T02:46:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.374+0000 7f43bcc00700 10 client.27620 handle_command_reply: tid=0 2024-03-30T02:46:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.374+0000 7f43edc0db80 1 client.27620 shutdown 2024-03-30T02:46:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 -- 172.21.15.67:0/349822780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 msgr2=0x556aec7657b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec7657b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f43c40caac0 tx=0x7f43c409a080 comp rx=0 tx=0).stop 2024-03-30T02:46:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 -- 172.21.15.67:0/349822780 shutdown_connections 2024-03-30T02:46:39.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556aec7799a0 0x556aec77bd90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec7657b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 --2- 172.21.15.67:0/349822780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec762b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43bcc00700 0 client.27620 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 -- 172.21.15.67:0/349822780 >> 172.21.15.67:0/349822780 conn(0x556aec6ba6a0 msgr2=0x556aec75c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 -- 172.21.15.67:0/349822780 shutdown_connections 2024-03-30T02:46:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 1 -- 172.21.15.67:0/349822780 wait complete. 2024-03-30T02:46:39.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.375+0000 7f43edc0db80 20 client.27620 trim_cache size 0 max 16384 2024-03-30T02:46:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.393+0000 7f43edc0db80 1 Processor -- start 2024-03-30T02:46:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.393+0000 7f43edc0db80 1 -- start start 2024-03-30T02:46:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.393+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75a5d0 0x556aec75d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.393+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75aef0 0x556aec770f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.393+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec765450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.393+0000 7f43edc0db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec771440 con 0x556aec765df0 2024-03-30T02:46:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.393+0000 7f43edc0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec758160 con 0x556aec75aef0 2024-03-30T02:46:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.393+0000 7f43edc0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec7637c0 con 0x556aec75a5d0 2024-03-30T02:46:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.393+0000 7f43d3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec765450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d2a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75aef0 0x556aec770f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d2a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75aef0 0x556aec770f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:40864/0 (socket says 172.21.15.67:40864) 2024-03-30T02:46:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d2a00700 1 -- 172.21.15.67:0/2188093789 learned_addr learned my addr 172.21.15.67:0/2188093789 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec765450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40072/0 (socket says 172.21.15.67:40072) 2024-03-30T02:46:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d3400700 1 --2- 172.21.15.67:0/2188093789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75a5d0 0x556aec75d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d2a00700 1 -- 172.21.15.67:0/2188093789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75a5d0 msgr2=0x556aec75d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d2a00700 1 --2- 172.21.15.67:0/2188093789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75a5d0 0x556aec75d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d2a00700 1 -- 172.21.15.67:0/2188093789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 msgr2=0x556aec765450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d2a00700 1 --2- 172.21.15.67:0/2188093789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec765450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d2a00700 1 -- 172.21.15.67:0/2188093789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aec672ea0 con 0x556aec75aef0 2024-03-30T02:46:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d3400700 1 --2- 172.21.15.67:0/2188093789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75a5d0 0x556aec75d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43d2a00700 1 --2- 172.21.15.67:0/2188093789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75aef0 0x556aec770f00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f43c000b6f0 tx=0x7f43c0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39efe65e826d2fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43c9200700 1 -- 172.21.15.67:0/2188093789 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c001d040 con 0x556aec75aef0 2024-03-30T02:46:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.394+0000 7f43c9200700 1 -- 172.21.15.67:0/2188093789 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c001e040 con 0x556aec75aef0 2024-03-30T02:46:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43c9200700 1 -- 172.21.15.67:0/2188093789 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0005550 con 0x556aec75aef0 2024-03-30T02:46:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43edc0db80 1 -- 172.21.15.67:0/2188093789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75aef0 msgr2=0x556aec770f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2188093789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75aef0 0x556aec770f00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f43c000b6f0 tx=0x7f43c0005fd0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43edc0db80 1 -- 172.21.15.67:0/2188093789 shutdown_connections 2024-03-30T02:46:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2188093789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec765450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2188093789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75a5d0 0x556aec75d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2188093789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75aef0 0x556aec770f00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43edc0db80 1 -- 172.21.15.67:0/2188093789 >> 172.21.15.67:0/2188093789 conn(0x556aec6ba6a0 msgr2=0x556aec74f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43edc0db80 1 -- 172.21.15.67:0/2188093789 shutdown_connections 2024-03-30T02:46:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.395+0000 7f43edc0db80 1 -- 172.21.15.67:0/2188093789 wait complete. 2024-03-30T02:46:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.396+0000 7f43edc0db80 1 -- start start 2024-03-30T02:46:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.396+0000 7f43edc0db80 1 Processor -- start 2024-03-30T02:46:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.396+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec76f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.396+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec76fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.396+0000 7f43edc0db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.396+0000 7f43edc0db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec7703d0 con 0x556aec765df0 2024-03-30T02:46:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.396+0000 7f43edc0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec770540 con 0x556aec75a5d0 2024-03-30T02:46:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.396+0000 7f43edc0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aec7cdce0 con 0x556aec75aef0 2024-03-30T02:46:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.396+0000 7f43edc0db80 10 client.0 _ll_register_callbacks cb 0x556aec652470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:46:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec76f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40088/0 (socket says 172.21.15.67:40088) 2024-03-30T02:46:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3e00700 1 -- 172.21.15.67:0/2450405119 learned_addr learned my addr 172.21.15.67:0/2450405119 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:46:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d2a00700 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec76fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3400700 1 -- 172.21.15.67:0/2450405119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 msgr2=0x556aec76fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3400700 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec76fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3400700 1 -- 172.21.15.67:0/2450405119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 msgr2=0x556aec7cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:46:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3400700 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3400700 1 -- 172.21.15.67:0/2450405119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556aec4dab60 con 0x556aec75a5d0 2024-03-30T02:46:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3e00700 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:46:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43d3400700 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec76f6b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009800 tx=0x7f43c4099cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=214c198811109ec4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:46:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.397+0000 7f43bcc00700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:46:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43bcc00700 1 -- 172.21.15.67:0/2450405119 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4098cd0 con 0x556aec75a5d0 2024-03-30T02:46:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43bcc00700 1 -- 172.21.15.67:0/2450405119 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f43c407d080 con 0x556aec75a5d0 2024-03-30T02:46:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43edc0db80 1 -- 172.21.15.67:0/2450405119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43c40a2380 con 0x556aec75a5d0 2024-03-30T02:46:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43bcc00700 1 -- 172.21.15.67:0/2450405119 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43c409dd30 con 0x556aec75a5d0 2024-03-30T02:46:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43edc0db80 10 client.27626 fetch_fsmap learned FSMap version 41 2024-03-30T02:46:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43edc0db80 1 -- 172.21.15.67:0/2450405119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f43d410e460 con 0x556aec75a5d0 2024-03-30T02:46:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43bcc00700 1 -- 172.21.15.67:0/2450405119 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c40a2380 con 0x556aec75a5d0 2024-03-30T02:46:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43bcc00700 1 -- 172.21.15.67:0/2450405119 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43c4094b10 con 0x556aec75a5d0 2024-03-30T02:46:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43edc0db80 10 client.27626 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:46:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43edc0db80 10 client.27626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:46:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43edc0db80 20 client.27626 populate_metadata read hostname 'smithi067' 2024-03-30T02:46:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556aec7cefe0 0x556aec7d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:46:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43edc0db80 4 client.27626 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:46:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43edc0db80 1 -- 172.21.15.67:0/2450405119 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556aec4fbcf0 con 0x556aec7cefe0 2024-03-30T02:46:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.398+0000 7f43d2a00700 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556aec7cefe0 0x556aec7d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:46:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.399+0000 7f43bcc00700 10 client.27626 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.399+0000 7f43d2a00700 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556aec7cefe0 0x556aec7d13d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8031b5a7fe5587f4 server_cookie=964a1d005e460959 in_seq=0 out_seq=0 2024-03-30T02:46:39.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.406+0000 7f43bcc00700 1 -- 172.21.15.67:0/2450405119 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x556aec4fbcf0 con 0x556aec7cefe0 2024-03-30T02:46:39.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.406+0000 7f43bcc00700 10 client.27626 handle_command_reply: tid=0 2024-03-30T02:46:39.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.406+0000 7f43edc0db80 1 client.27626 shutdown 2024-03-30T02:46:39.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 -- 172.21.15.67:0/2450405119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 msgr2=0x556aec76f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec76f6b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009800 tx=0x7f43c4099cd0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 -- 172.21.15.67:0/2450405119 shutdown_connections 2024-03-30T02:46:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556aec7cefe0 0x556aec7d13d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556aec765df0 0x556aec7cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556aec75a5d0 0x556aec76f6b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43bcc00700 0 client.27626 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:46:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 --2- 172.21.15.67:0/2450405119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556aec75aef0 0x556aec76fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 -- 172.21.15.67:0/2450405119 >> 172.21.15.67:0/2450405119 conn(0x556aec6ba6a0 msgr2=0x556aec765450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 -- 172.21.15.67:0/2450405119 shutdown_connections 2024-03-30T02:46:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 1 -- 172.21.15.67:0/2450405119 wait complete. 2024-03-30T02:46:39.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43edc0db80 20 client.27626 trim_cache size 0 max 16384 2024-03-30T02:46:39.409 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:46:39.409 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "scrub active (7826 inodes in the stack)", 2024-03-30T02:46:39.409 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-03-30T02:46:39.409 INFO:teuthology.orchestra.run.smithi067.stdout: "4c12ba12-215d-4f72-80db-5ca40aaeb957": { 2024-03-30T02:46:39.409 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-03-30T02:46:39.409 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "4c12ba12-215d-4f72-80db-5ca40aaeb957", 2024-03-30T02:46:39.410 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-03-30T02:46:39.410 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:46:39.410 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:46:39.410 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:46:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43da000700 1 -- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f43b80a1290 msgr2=0x7f43b80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43da000700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f43b80a1290 0x7f43b80a3750 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009c10 tx=0x7f43cc009920 comp rx=0 tx=0).stop 2024-03-30T02:46:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43da000700 1 -- 172.21.15.67:0/4189190962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d4100f50 msgr2=0x7f43d4197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:46:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.407+0000 7f43da000700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d4100f50 0x7f43d4197210 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9f0 tx=0x7f43c400ed00 comp rx=0 tx=0).stop 2024-03-30T02:46:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.408+0000 7f43da000700 1 -- 172.21.15.67:0/4189190962 shutdown_connections 2024-03-30T02:46:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.408+0000 7f43da000700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f43b80a1290 0x7f43b80a3750 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.408+0000 7f43da000700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43d419baa0 0x7f43d419bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.408+0000 7f43da000700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f43d4100f50 0x7f43d4197210 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.408+0000 7f43da000700 1 --2- 172.21.15.67:0/4189190962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43d4106070 0x7f43d4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:46:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.408+0000 7f43da000700 1 -- 172.21.15.67:0/4189190962 >> 172.21.15.67:0/4189190962 conn(0x7f43d40fc7f0 msgr2=0x7f43d40fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:46:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.408+0000 7f43da000700 1 -- 172.21.15.67:0/4189190962 shutdown_connections 2024-03-30T02:46:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:46:39.408+0000 7f43da000700 1 -- 172.21.15.67:0/4189190962 wait complete. 2024-03-30T02:46:39.427 INFO:tasks.cephfs.filesystem:scrub status for tag:4c12ba12-215d-4f72-80db-5ca40aaeb957 - {'path': '/', 'tag': '4c12ba12-215d-4f72-80db-5ca40aaeb957', 'options': 'recursive,force'} 2024-03-30T02:46:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:39.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:39 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:39.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:39 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:39.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:39.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:39 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: pgmap v297: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.2 MiB/s wr, 42 op/s 2024-03-30T02:46:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub queued for path: / 2024-03-30T02:46:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:40.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:40 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: pgmap v297: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.2 MiB/s wr, 42 op/s 2024-03-30T02:46:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub queued for path: / 2024-03-30T02:46:40.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.928 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:40.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:40 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: pgmap v297: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.2 MiB/s wr, 42 op/s 2024-03-30T02:46:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub queued for path: / 2024-03-30T02:46:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.953 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:40.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:40.954 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:40 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:41.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:41 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:41.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:41 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:41.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:41.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:41 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: pgmap v298: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-03-30T02:46:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.762 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.763 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:42.764 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:42 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: pgmap v298: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-03-30T02:46:42.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:42.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:42 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: pgmap v298: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-03-30T02:46:42.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.951 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:42.952 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:42 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:43 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:43.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:43 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:43.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:43.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:43.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:43 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: pgmap v299: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 367 op/s 2024-03-30T02:46:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:44.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:44 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: pgmap v299: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 367 op/s 2024-03-30T02:46:44.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:44.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:44 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: pgmap v299: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 367 op/s 2024-03-30T02:46:44.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:44.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:44.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:44 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.759 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.760 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.761 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:45 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:45.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.922 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.923 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.924 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.925 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:45 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.949 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:45.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:45.950 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:45 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: pgmap v300: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 704 op/s 2024-03-30T02:46:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.753 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.754 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.755 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.756 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.757 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:46.758 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:46 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: pgmap v300: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 704 op/s 2024-03-30T02:46:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:46.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:46 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: pgmap v300: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 704 op/s 2024-03-30T02:46:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.945 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.946 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.947 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:46.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:46.948 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:46 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.513 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.513 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.514 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.515 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: pgmap v301: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 2.12k op/s 2024-03-30T02:46:47.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:47 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: pgmap v301: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 2.12k op/s 2024-03-30T02:46:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:47.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:47 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:47.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:47.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: pgmap v301: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 2.12k op/s 2024-03-30T02:46:48.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:47 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:48.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:48 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:48.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:48 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:48.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:48.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:48 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 671 KiB/s wr, 2.47k op/s 2024-03-30T02:46:49.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: active paths [/] 2024-03-30T02:46:49.918 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:49 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 671 KiB/s wr, 2.47k op/s 2024-03-30T02:46:49.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.943 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: active paths [/] 2024-03-30T02:46:49.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:49 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:49.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:49.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: pgmap v302: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 671 KiB/s wr, 2.47k op/s 2024-03-30T02:46:50.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.003 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: active paths [/] 2024-03-30T02:46:50.004 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:49 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:50 smithi067 ceph-mon[37808]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:50 smithi170 ceph-mon[78798]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:50.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:50 smithi082 ceph-mon[79176]: scrub summary: idle+waiting paths [/] 2024-03-30T02:46:52.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:52 smithi082 ceph-mon[79176]: pgmap v303: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 321 KiB/s wr, 3.93k op/s 2024-03-30T02:46:52.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:52 smithi067 ceph-mon[37808]: pgmap v303: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 321 KiB/s wr, 3.93k op/s 2024-03-30T02:46:52.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:52 smithi170 ceph-mon[78798]: pgmap v303: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 321 KiB/s wr, 3.93k op/s 2024-03-30T02:46:54.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:54 smithi082 ceph-mon[79176]: pgmap v304: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 45 KiB/s wr, 4.19k op/s 2024-03-30T02:46:54.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:54 smithi067 ceph-mon[37808]: pgmap v304: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 45 KiB/s wr, 4.19k op/s 2024-03-30T02:46:54.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:54 smithi170 ceph-mon[78798]: pgmap v304: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 45 KiB/s wr, 4.19k op/s 2024-03-30T02:46:56.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:56 smithi082 ceph-mon[79176]: pgmap v305: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 56 KiB/s wr, 4.05k op/s 2024-03-30T02:46:56.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:56 smithi067 ceph-mon[37808]: pgmap v305: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 56 KiB/s wr, 4.05k op/s 2024-03-30T02:46:56.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:56 smithi170 ceph-mon[78798]: pgmap v305: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 56 KiB/s wr, 4.05k op/s 2024-03-30T02:46:58.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:46:58 smithi082 ceph-mon[79176]: pgmap v306: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 24 KiB/s wr, 3.71k op/s 2024-03-30T02:46:58.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:46:58 smithi067 ceph-mon[37808]: pgmap v306: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 24 KiB/s wr, 3.71k op/s 2024-03-30T02:46:58.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:46:58 smithi170 ceph-mon[78798]: pgmap v306: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 24 KiB/s wr, 3.71k op/s 2024-03-30T02:47:00.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:00 smithi082 ceph-mon[79176]: pgmap v307: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 24 KiB/s wr, 2.30k op/s 2024-03-30T02:47:00.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:00 smithi067 ceph-mon[37808]: pgmap v307: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 24 KiB/s wr, 2.30k op/s 2024-03-30T02:47:00.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:00 smithi170 ceph-mon[78798]: pgmap v307: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 24 KiB/s wr, 2.30k op/s 2024-03-30T02:47:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:02 smithi082 ceph-mon[79176]: pgmap v308: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 24 KiB/s wr, 1.95k op/s 2024-03-30T02:47:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:47:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:47:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:47:02.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:47:02.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:02 smithi082 ceph-mon[79176]: scrub summary: idle 2024-03-30T02:47:02.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:02 smithi067 ceph-mon[37808]: pgmap v308: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 24 KiB/s wr, 1.95k op/s 2024-03-30T02:47:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:47:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:47:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:47:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:47:02.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:02 smithi067 ceph-mon[37808]: scrub summary: idle 2024-03-30T02:47:02.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:02 smithi170 ceph-mon[78798]: pgmap v308: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 24 KiB/s wr, 1.95k op/s 2024-03-30T02:47:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:47:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:47:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:47:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:47:02.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:02 smithi170 ceph-mon[78798]: scrub summary: idle 2024-03-30T02:47:04.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:04 smithi082 ceph-mon[79176]: pgmap v309: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 38 KiB/s wr, 458 op/s 2024-03-30T02:47:04.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:04 smithi067 ceph-mon[37808]: pgmap v309: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 38 KiB/s wr, 458 op/s 2024-03-30T02:47:04.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:04 smithi170 ceph-mon[78798]: pgmap v309: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 38 KiB/s wr, 458 op/s 2024-03-30T02:47:05.561 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-30T02:47:05.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.769+0000 7ffbb3000700 1 -- 172.21.15.67:0/2987349863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbac10d560 msgr2=0x7ffbac10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.769+0000 7ffbb3000700 1 --2- 172.21.15.67:0/2987349863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbac10d560 0x7ffbac10f950 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ffba400b0a0 tx=0x7ffba400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.769+0000 7ffbb3000700 1 -- 172.21.15.67:0/2987349863 shutdown_connections 2024-03-30T02:47:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.769+0000 7ffbb3000700 1 --2- 172.21.15.67:0/2987349863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbac10d560 0x7ffbac10f950 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.769+0000 7ffbb3000700 1 --2- 172.21.15.67:0/2987349863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac073160 0x7ffbac073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:05.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.769+0000 7ffbb3000700 1 --2- 172.21.15.67:0/2987349863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbac073a80 0x7ffbac10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.769+0000 7ffbb3000700 1 -- 172.21.15.67:0/2987349863 >> 172.21.15.67:0/2987349863 conn(0x7ffbac0fc7b0 msgr2=0x7ffbac0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.769+0000 7ffbb3000700 1 -- 172.21.15.67:0/2987349863 shutdown_connections 2024-03-30T02:47:05.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.769+0000 7ffbb3000700 1 -- 172.21.15.67:0/2987349863 wait complete. 2024-03-30T02:47:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.770+0000 7ffbb3000700 1 Processor -- start 2024-03-30T02:47:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbb3000700 1 -- start start 2024-03-30T02:47:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbb3000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac073160 0x7ffbac1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbb3000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbac073a80 0x7ffbac1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbb3000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbac10d560 0x7ffbac19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbb3000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac19e3c0 con 0x7ffbac10d560 2024-03-30T02:47:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbb3000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac19e530 con 0x7ffbac073160 2024-03-30T02:47:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbb3000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac19e6f0 con 0x7ffbac073a80 2024-03-30T02:47:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbab400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbac073a80 0x7ffbac1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbabe00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac073160 0x7ffbac1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbab400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbac073a80 0x7ffbac1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:54606/0 (socket says 172.21.15.67:54606) 2024-03-30T02:47:05.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbab400700 1 -- 172.21.15.67:0/1019168690 learned_addr learned my addr 172.21.15.67:0/1019168690 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.771+0000 7ffbb1000700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbac10d560 0x7ffbac19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffbabe00700 1 -- 172.21.15.67:0/1019168690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbac073a80 msgr2=0x7ffbac1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffbabe00700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbac073a80 0x7ffbac1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffbabe00700 1 -- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbac10d560 msgr2=0x7ffbac19dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffbabe00700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbac10d560 0x7ffbac19dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffbabe00700 1 -- 172.21.15.67:0/1019168690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4009d00 con 0x7ffbac073160 2024-03-30T02:47:05.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffbabe00700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac073160 0x7ffbac1994a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00e9f0 tx=0x7ffb9c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:05.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffba8c00700 1 -- 172.21.15.67:0/1019168690 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00c9c0 con 0x7ffbac073160 2024-03-30T02:47:05.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac19e9d0 con 0x7ffbac073160 2024-03-30T02:47:05.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffba8c00700 1 -- 172.21.15.67:0/1019168690 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb9c010430 con 0x7ffbac073160 2024-03-30T02:47:05.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffba8c00700 1 -- 172.21.15.67:0/1019168690 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c018680 con 0x7ffbac073160 2024-03-30T02:47:05.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.772+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac1a6210 con 0x7ffbac073160 2024-03-30T02:47:05.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.773+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac108770 con 0x7ffbac073160 2024-03-30T02:47:05.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.775+0000 7ffba8c00700 1 -- 172.21.15.67:0/1019168690 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7ffb9c010860 con 0x7ffbac073160 2024-03-30T02:47:05.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.777+0000 7ffba8c00700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ffb900a1290 0x7ffb900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:05.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.777+0000 7ffba8c00700 1 -- 172.21.15.67:0/1019168690 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7ffb9c014070 con 0x7ffbac073160 2024-03-30T02:47:05.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.777+0000 7ffbab400700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ffb900a1290 0x7ffb900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:05.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.777+0000 7ffbab400700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ffb900a1290 0x7ffb900a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbac19a9f0 tx=0x7ffb98009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:05.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:05.779+0000 7ffba8c00700 1 -- 172.21.15.67:0/1019168690 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7ffb9c0963f0 con 0x7ffbac073160 2024-03-30T02:47:06.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.024+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 --> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbac19a810 con 0x7ffb900a1290 2024-03-30T02:47:06.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.033+0000 7ffba8c00700 1 -- 172.21.15.67:0/1019168690 <== mgr.25634 v2:172.21.15.67:6800/1710497985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7ffbac19a810 con 0x7ffb900a1290 2024-03-30T02:47:06.032 INFO:teuthology.orchestra.run.smithi067.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-30T00:00:00", "created": "2024-03-30T02:37:03", "first": "2024-03-30T02:38:00", "last": "2024-03-30T02:47:00", "last_pruned": "2024-03-30T02:47:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-30T02:47:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ffb900a1290 msgr2=0x7ffb900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:06.034 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ffb900a1290 0x7ffb900a3750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbac19a9f0 tx=0x7ffb98009260 comp rx=0 tx=0).stop 2024-03-30T02:47:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac073160 msgr2=0x7ffbac1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac073160 0x7ffbac1994a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00e9f0 tx=0x7ffb9c00ed00 comp rx=0 tx=0).stop 2024-03-30T02:47:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 shutdown_connections 2024-03-30T02:47:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7ffb900a1290 0x7ffb900a3750 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffbac10d560 0x7ffbac19dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbac073160 0x7ffbac1994a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 --2- 172.21.15.67:0/1019168690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbac073a80 0x7ffbac1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 >> 172.21.15.67:0/1019168690 conn(0x7ffbac0fc7b0 msgr2=0x7ffbac0fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:06.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 shutdown_connections 2024-03-30T02:47:06.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.035+0000 7ffbb3000700 1 -- 172.21.15.67:0/1019168690 wait complete. 2024-03-30T02:47:06.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-30T02:47:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.252+0000 7fe77a600700 1 -- 172.21.15.67:0/3326412710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe774106030 msgr2=0x7fe77410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.252+0000 7fe77a600700 1 --2- 172.21.15.67:0/3326412710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe774106030 0x7fe77410dbb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00b0a0 tx=0x7fe76c00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.252+0000 7fe77a600700 1 -- 172.21.15.67:0/3326412710 shutdown_connections 2024-03-30T02:47:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.252+0000 7fe77a600700 1 --2- 172.21.15.67:0/3326412710 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe774106030 0x7fe77410dbb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.252+0000 7fe77a600700 1 --2- 172.21.15.67:0/3326412710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe774100f10 0x7fe7741012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.252+0000 7fe77a600700 1 --2- 172.21.15.67:0/3326412710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7741018c0 0x7fe774105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.252+0000 7fe77a600700 1 -- 172.21.15.67:0/3326412710 >> 172.21.15.67:0/3326412710 conn(0x7fe7740fc7b0 msgr2=0x7fe7740febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.252+0000 7fe77a600700 1 -- 172.21.15.67:0/3326412710 shutdown_connections 2024-03-30T02:47:06.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.252+0000 7fe77a600700 1 -- 172.21.15.67:0/3326412710 wait complete. 2024-03-30T02:47:06.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.253+0000 7fe77a600700 1 Processor -- start 2024-03-30T02:47:06.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.253+0000 7fe77a600700 1 -- start start 2024-03-30T02:47:06.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.253+0000 7fe77a600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe774100f10 0x7fe774199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:06.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.253+0000 7fe77a600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe774106030 0x7fe774199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:06.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.253+0000 7fe77a600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe77419dda0 0x7fe77419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.253+0000 7fe77a600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77419e820 con 0x7fe77419dda0 2024-03-30T02:47:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.253+0000 7fe77a600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77419e990 con 0x7fe774106030 2024-03-30T02:47:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.253+0000 7fe77a600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77419eb50 con 0x7fe774100f10 2024-03-30T02:47:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe77419dda0 0x7fe77419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe77419dda0 0x7fe77419e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51428/0 (socket says 172.21.15.67:51428) 2024-03-30T02:47:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773e00700 1 -- 172.21.15.67:0/2060664524 learned_addr learned my addr 172.21.15.67:0/2060664524 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773400700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe774100f10 0x7fe774199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:06.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe772a00700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe774106030 0x7fe774199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773e00700 1 -- 172.21.15.67:0/2060664524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe774100f10 msgr2=0x7fe774199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773e00700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe774100f10 0x7fe774199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773e00700 1 -- 172.21.15.67:0/2060664524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe774106030 msgr2=0x7fe774199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773e00700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe774106030 0x7fe774199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773e00700 1 -- 172.21.15.67:0/2060664524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c009d00 con 0x7fe77419dda0 2024-03-30T02:47:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.254+0000 7fe773e00700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe77419dda0 0x7fe77419e220 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00b070 tx=0x7fe76c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.255+0000 7fe76be00700 1 -- 172.21.15.67:0/2060664524 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00e040 con 0x7fe77419dda0 2024-03-30T02:47:06.256 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.255+0000 7fe76be00700 1 -- 172.21.15.67:0/2060664524 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe76c01d480 con 0x7fe77419dda0 2024-03-30T02:47:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.255+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7741a6050 con 0x7fe77419dda0 2024-03-30T02:47:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.255+0000 7fe76be00700 1 -- 172.21.15.67:0/2060664524 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c0052a0 con 0x7fe77419dda0 2024-03-30T02:47:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.255+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7741a6420 con 0x7fe77419dda0 2024-03-30T02:47:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.256+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7741023e0 con 0x7fe77419dda0 2024-03-30T02:47:06.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.258+0000 7fe76be00700 1 -- 172.21.15.67:0/2060664524 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe76c01d5f0 con 0x7fe77419dda0 2024-03-30T02:47:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.259+0000 7fe76be00700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe7580a1290 0x7fe7580a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.259+0000 7fe76be00700 1 -- 172.21.15.67:0/2060664524 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fe76c0ce0f0 con 0x7fe77419dda0 2024-03-30T02:47:06.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.259+0000 7fe773400700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe7580a1290 0x7fe7580a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:06.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.260+0000 7fe773400700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe7580a1290 0x7fe7580a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe764009c60 tx=0x7fe764009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:06.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.264+0000 7fe76be00700 1 -- 172.21.15.67:0/2060664524 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe76c0971d0 con 0x7fe77419dda0 2024-03-30T02:47:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.519+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 --> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe7741018c0 con 0x7fe7580a1290 2024-03-30T02:47:06.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.540+0000 7fe76be00700 1 -- 172.21.15.67:0/2060664524 <== mgr.25634 v2:172.21.15.67:6800/1710497985 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fe7741018c0 con 0x7fe7580a1290 2024-03-30T02:47:06.539 INFO:teuthology.orchestra.run.smithi067.stdout:/ 1m 6m3h 2024-03-30T02:47:06.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe7580a1290 msgr2=0x7fe7580a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:06.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe7580a1290 0x7fe7580a3750 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe764009c60 tx=0x7fe764009260 comp rx=0 tx=0).stop 2024-03-30T02:47:06.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe77419dda0 msgr2=0x7fe77419e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:06.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe77419dda0 0x7fe77419e220 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00b070 tx=0x7fe76c00bda0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 shutdown_connections 2024-03-30T02:47:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe7580a1290 0x7fe7580a3750 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe77419dda0 0x7fe77419e220 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe774100f10 0x7fe774199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 --2- 172.21.15.67:0/2060664524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe774106030 0x7fe774199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 >> 172.21.15.67:0/2060664524 conn(0x7fe7740fc7b0 msgr2=0x7fe7740fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 shutdown_connections 2024-03-30T02:47:06.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:06.543+0000 7fe77a600700 1 -- 172.21.15.67:0/2060664524 wait complete. 2024-03-30T02:47:06.560 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-30T02:47:06.570 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-30T02:47:06.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:06 smithi082 ceph-mon[79176]: pgmap v310: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-03-30T02:47:06.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:06 smithi067 ceph-mon[37808]: pgmap v310: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-03-30T02:47:06.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:06 smithi170 ceph-mon[78798]: pgmap v310: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 52 KiB/s wr, 17 op/s 2024-03-30T02:47:07.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:07 smithi082 ceph-mon[79176]: from='client.27632 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:47:07.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:07 smithi082 ceph-mon[79176]: osdmap e95: 12 total, 12 up, 12 in 2024-03-30T02:47:07.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:07 smithi067 ceph-mon[37808]: from='client.27632 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:47:07.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:07 smithi067 ceph-mon[37808]: osdmap e95: 12 total, 12 up, 12 in 2024-03-30T02:47:07.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:07 smithi170 ceph-mon[78798]: from='client.27632 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:47:07.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:07 smithi170 ceph-mon[78798]: osdmap e95: 12 total, 12 up, 12 in 2024-03-30T02:47:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:08 smithi082 ceph-mon[79176]: from='client.17502 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:47:08.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:08 smithi082 ceph-mon[79176]: pgmap v312: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 69 KiB/s wr, 31 op/s 2024-03-30T02:47:08.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:08 smithi067 ceph-mon[37808]: from='client.17502 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:47:08.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:08 smithi067 ceph-mon[37808]: pgmap v312: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 69 KiB/s wr, 31 op/s 2024-03-30T02:47:08.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:08 smithi170 ceph-mon[78798]: from='client.17502 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-30T02:47:08.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:08 smithi170 ceph-mon[78798]: pgmap v312: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 69 KiB/s wr, 31 op/s 2024-03-30T02:47:09.428 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-30T02:47:09.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.644+0000 7f4a42c00700 1 -- 172.21.15.67:0/2729286127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 msgr2=0x7f4a3c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.644+0000 7f4a42c00700 1 --2- 172.21.15.67:0/2729286127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 0x7f4a3c101340 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009990 tx=0x7f4a2c009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.644+0000 7f4a42c00700 1 -- 172.21.15.67:0/2729286127 shutdown_connections 2024-03-30T02:47:09.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.644+0000 7f4a42c00700 1 --2- 172.21.15.67:0/2729286127 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 0x7f4a3c101340 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.644+0000 7f4a42c00700 1 --2- 172.21.15.67:0/2729286127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c106080 0x7f4a3c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.644+0000 7f4a42c00700 1 --2- 172.21.15.67:0/2729286127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a3c101910 0x7f4a3c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.644+0000 7f4a42c00700 1 -- 172.21.15.67:0/2729286127 >> 172.21.15.67:0/2729286127 conn(0x7f4a3c0fc7c0 msgr2=0x7f4a3c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:09.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.644+0000 7f4a42c00700 1 -- 172.21.15.67:0/2729286127 shutdown_connections 2024-03-30T02:47:09.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.644+0000 7f4a42c00700 1 -- 172.21.15.67:0/2729286127 wait complete. 2024-03-30T02:47:09.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.646+0000 7f4a42c00700 1 Processor -- start 2024-03-30T02:47:09.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.646+0000 7f4a42c00700 1 -- start start 2024-03-30T02:47:09.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.646+0000 7f4a42c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 0x7f4a3c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.646+0000 7f4a42c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c106080 0x7f4a3c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.646+0000 7f4a42c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a3c19dcc0 0x7f4a3c19e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.646+0000 7f4a42c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c19e740 con 0x7f4a3c100f60 2024-03-30T02:47:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.646+0000 7f4a42c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c19e8b0 con 0x7f4a3c19dcc0 2024-03-30T02:47:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.646+0000 7f4a42c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c19ea70 con 0x7f4a3c106080 2024-03-30T02:47:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 0x7f4a3c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 0x7f4a3c199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51458/0 (socket says 172.21.15.67:51458) 2024-03-30T02:47:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3be00700 1 -- 172.21.15.67:0/3861639791 learned_addr learned my addr 172.21.15.67:0/3861639791 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a40c00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a3c19dcc0 0x7f4a3c19e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3b400700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c106080 0x7f4a3c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3be00700 1 -- 172.21.15.67:0/3861639791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c106080 msgr2=0x7f4a3c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3be00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c106080 0x7f4a3c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3be00700 1 -- 172.21.15.67:0/3861639791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a3c19dcc0 msgr2=0x7f4a3c19e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3be00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a3c19dcc0 0x7f4a3c19e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3be00700 1 -- 172.21.15.67:0/3861639791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c0095f0 con 0x7f4a3c100f60 2024-03-30T02:47:09.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3b400700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c106080 0x7f4a3c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:47:09.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.647+0000 7f4a3be00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 0x7f4a3c199480 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c000c00 tx=0x7f4a2c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:09.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.648+0000 7f4a38c00700 1 -- 172.21.15.67:0/3861639791 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c01d070 con 0x7f4a3c100f60 2024-03-30T02:47:09.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.648+0000 7f4a38c00700 1 -- 172.21.15.67:0/3861639791 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a2c00fb30 con 0x7f4a3c100f60 2024-03-30T02:47:09.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.648+0000 7f4a38c00700 1 -- 172.21.15.67:0/3861639791 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c022c60 con 0x7f4a3c100f60 2024-03-30T02:47:09.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.648+0000 7f4a42c00700 1 -- 172.21.15.67:0/3861639791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c1a5f70 con 0x7f4a3c100f60 2024-03-30T02:47:09.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.648+0000 7f4a42c00700 1 -- 172.21.15.67:0/3861639791 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c1a62e0 con 0x7f4a3c100f60 2024-03-30T02:47:09.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.651+0000 7f4a38c00700 1 -- 172.21.15.67:0/3861639791 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f4a2c00fca0 con 0x7f4a3c100f60 2024-03-30T02:47:09.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.653+0000 7f4a38c00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4a200a1290 0x7f4a200a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.653+0000 7f4a567ffb80 1 Processor -- start 2024-03-30T02:47:09.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.653+0000 7f4a38c00700 1 -- 172.21.15.67:0/3861639791 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f4a2c0ce4f0 con 0x7f4a3c100f60 2024-03-30T02:47:09.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a567ffb80 1 -- start start 2024-03-30T02:47:09.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a3b400700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4a200a1290 0x7f4a200a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22ef29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22efd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22efddf0 0x556d22f001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22e4b990 con 0x556d22efddf0 2024-03-30T02:47:09.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22e49350 con 0x556d22ef25d0 2024-03-30T02:47:09.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22e44c40 con 0x556d22ef2ef0 2024-03-30T02:47:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a3b400700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4a200a1290 0x7f4a200a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c19a9d0 tx=0x7f4a2800b5f0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a40c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22efddf0 0x556d22f001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a3be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22ef29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a40c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22efddf0 0x556d22f001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51462/0 (socket says 172.21.15.67:51462) 2024-03-30T02:47:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a3be00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22ef29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:44466/0 (socket says 172.21.15.67:44466) 2024-03-30T02:47:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.654+0000 7f4a40c00700 1 -- 172.21.15.67:0/2703040061 learned_addr learned my addr 172.21.15.67:0/2703040061 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a3b400700 1 --2- 172.21.15.67:0/2703040061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22efd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a40c00700 1 -- 172.21.15.67:0/2703040061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 msgr2=0x556d22efd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a40c00700 1 --2- 172.21.15.67:0/2703040061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22efd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a40c00700 1 -- 172.21.15.67:0/2703040061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 msgr2=0x556d22ef29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a40c00700 1 --2- 172.21.15.67:0/2703040061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22ef29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a40c00700 1 -- 172.21.15.67:0/2703040061 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d22cc63b0 con 0x556d22efddf0 2024-03-30T02:47:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a3be00700 1 --2- 172.21.15.67:0/2703040061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22ef29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a40c00700 1 --2- 172.21.15.67:0/2703040061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22efddf0 0x556d22f001e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4a34003fa0 tx=0x7f4a34009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4894c9931e29ce8a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a31200700 1 -- 172.21.15.67:0/2703040061 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34019070 con 0x556d22efddf0 2024-03-30T02:47:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a31200700 1 -- 172.21.15.67:0/2703040061 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3400f040 con 0x556d22efddf0 2024-03-30T02:47:09.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.655+0000 7f4a31200700 1 -- 172.21.15.67:0/2703040061 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a340079f0 con 0x556d22efddf0 2024-03-30T02:47:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.656+0000 7f4a567ffb80 1 -- 172.21.15.67:0/2703040061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22efddf0 msgr2=0x556d22f001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.656+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/2703040061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22efddf0 0x556d22f001e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4a34003fa0 tx=0x7f4a34009ba0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.656+0000 7f4a567ffb80 1 -- 172.21.15.67:0/2703040061 shutdown_connections 2024-03-30T02:47:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.656+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/2703040061 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22efddf0 0x556d22f001e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.656+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/2703040061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22ef29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.656+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/2703040061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22efd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.656+0000 7f4a567ffb80 1 -- 172.21.15.67:0/2703040061 >> 172.21.15.67:0/2703040061 conn(0x556d22e526a0 msgr2=0x556d22eeea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.656+0000 7f4a567ffb80 1 -- 172.21.15.67:0/2703040061 shutdown_connections 2024-03-30T02:47:09.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.656+0000 7f4a567ffb80 1 -- 172.21.15.67:0/2703040061 wait complete. 2024-03-30T02:47:09.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.657+0000 7f4a567ffb80 1 -- start start 2024-03-30T02:47:09.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.657+0000 7f4a567ffb80 1 Processor -- start 2024-03-30T02:47:09.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.657+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22efd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.657+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef2ef0 0x556d22efab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22efddf0 0x556d22efb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22de0d80 con 0x556d22ef2ef0 2024-03-30T02:47:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22de0a20 con 0x556d22ef25d0 2024-03-30T02:47:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22cc86a0 con 0x556d22efddf0 2024-03-30T02:47:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a567ffb80 10 client.0 _ll_register_callbacks cb 0x556d22dea470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef2ef0 0x556d22efab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3b400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef2ef0 0x556d22efab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51466/0 (socket says 172.21.15.67:51466) 2024-03-30T02:47:09.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3b400700 1 -- 172.21.15.67:0/1951525794 learned_addr learned my addr 172.21.15.67:0/1951525794 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:09.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22efd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a40c00700 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22efddf0 0x556d22efb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3be00700 1 -- 172.21.15.67:0/1951525794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22efddf0 msgr2=0x556d22efb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:09.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22efddf0 0x556d22efb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3be00700 1 -- 172.21.15.67:0/1951525794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef2ef0 msgr2=0x556d22efab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:09.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef2ef0 0x556d22efab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3be00700 1 -- 172.21.15.67:0/1951525794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c1a62e0 con 0x556d22ef25d0 2024-03-30T02:47:09.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a40c00700 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22efddf0 0x556d22efb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:47:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.658+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22efd770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0cb7e0 tx=0x7f4a2c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71fad0bf4b541304 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a25600700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-30T02:47:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a25600700 1 -- 172.21.15.67:0/1951525794 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c09f060 con 0x556d22ef25d0 2024-03-30T02:47:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a25600700 1 -- 172.21.15.67:0/1951525794 <== mon.2 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f4a2c095e40 con 0x556d22ef25d0 2024-03-30T02:47:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1951525794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d22e05380 con 0x556d22ef25d0 2024-03-30T02:47:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a25600700 1 -- 172.21.15.67:0/1951525794 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a2c0a3620 con 0x556d22ef25d0 2024-03-30T02:47:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a567ffb80 10 client.27650 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1951525794 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f4a34003bd0 con 0x556d22ef25d0 2024-03-30T02:47:09.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a25600700 1 -- 172.21.15.67:0/1951525794 <== mon.2 v2:172.21.15.82:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d22e05380 con 0x556d22ef25d0 2024-03-30T02:47:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a25600700 1 -- 172.21.15.67:0/1951525794 <== mon.2 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a2c0a3190 con 0x556d22ef25d0 2024-03-30T02:47:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a567ffb80 10 client.27650 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a567ffb80 10 client.27650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:47:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.659+0000 7f4a567ffb80 20 client.27650 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.660+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556d22f0d5e0 0x556d22f0f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.660+0000 7f4a567ffb80 4 client.27650 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:47:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.660+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1951525794 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d22e3d700 con 0x556d22f0d5e0 2024-03-30T02:47:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.660+0000 7f4a3b400700 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556d22f0d5e0 0x556d22f0f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.660+0000 7f4a3b400700 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556d22f0d5e0 0x556d22f0f9d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a4215e588507a14 server_cookie=9fbc870aa59c1993 in_seq=0 out_seq=0 2024-03-30T02:47:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.660+0000 7f4a25600700 10 client.27650 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:47:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a25600700 1 -- 172.21.15.67:0/1951525794 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556d22e3d700 con 0x556d22f0d5e0 2024-03-30T02:47:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a25600700 10 client.27650 handle_command_reply: tid=0 2024-03-30T02:47:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a567ffb80 1 client.27650 shutdown 2024-03-30T02:47:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1951525794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 msgr2=0x556d22efd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22efd770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0cb7e0 tx=0x7f4a2c009310 comp rx=0 tx=0).stop 2024-03-30T02:47:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1951525794 shutdown_connections 2024-03-30T02:47:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22efddf0 0x556d22efb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22ef25d0 0x556d22efd770 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556d22f0d5e0 0x556d22f0f9d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.661+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1951525794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef2ef0 0x556d22efab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.662+0000 7f4a25600700 0 client.27650 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:47:09.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.662+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1951525794 >> 172.21.15.67:0/1951525794 conn(0x556d22e526a0 msgr2=0x556d22ef46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:09.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.662+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1951525794 shutdown_connections 2024-03-30T02:47:09.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.662+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1951525794 wait complete. 2024-03-30T02:47:09.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.662+0000 7f4a567ffb80 20 client.27650 trim_cache size 0 max 16384 2024-03-30T02:47:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.679+0000 7f4a567ffb80 1 Processor -- start 2024-03-30T02:47:09.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.679+0000 7f4a567ffb80 1 -- start start 2024-03-30T02:47:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22efb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22f04a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 0x556d22ef9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22ef0240 con 0x556d22ef25d0 2024-03-30T02:47:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22ef3850 con 0x556d22efddf0 2024-03-30T02:47:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22ef59a0 con 0x556d22ef2ef0 2024-03-30T02:47:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22efb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22efb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51476/0 (socket says 172.21.15.67:51476) 2024-03-30T02:47:09.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3be00700 1 -- 172.21.15.67:0/1795462547 learned_addr learned my addr 172.21.15.67:0/1795462547 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:09.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3b400700 1 --2- 172.21.15.67:0/1795462547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22f04a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a40c00700 1 --2- 172.21.15.67:0/1795462547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 0x556d22ef9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3be00700 1 -- 172.21.15.67:0/1795462547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 msgr2=0x556d22f04a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:09.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1795462547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22f04a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3be00700 1 -- 172.21.15.67:0/1795462547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 msgr2=0x556d22ef9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:09.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1795462547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 0x556d22ef9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3be00700 1 -- 172.21.15.67:0/1795462547 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d22c72b60 con 0x556d22ef25d0 2024-03-30T02:47:09.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.680+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1795462547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22efb0b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0976e0 tx=0x7f4a2c09acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73943c7043fd9c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:09.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a31c00700 1 -- 172.21.15.67:0/1795462547 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c063080 con 0x556d22ef25d0 2024-03-30T02:47:09.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a31c00700 1 -- 172.21.15.67:0/1795462547 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a2c09f060 con 0x556d22ef25d0 2024-03-30T02:47:09.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a31c00700 1 -- 172.21.15.67:0/1795462547 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0ce170 con 0x556d22ef25d0 2024-03-30T02:47:09.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1795462547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 msgr2=0x556d22efb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1795462547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22efb0b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0976e0 tx=0x7f4a2c09acc0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1795462547 shutdown_connections 2024-03-30T02:47:09.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1795462547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22efb0b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1795462547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 0x556d22ef9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1795462547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22f04a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1795462547 >> 172.21.15.67:0/1795462547 conn(0x556d22e526a0 msgr2=0x556d22ee7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:09.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1795462547 shutdown_connections 2024-03-30T02:47:09.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.681+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1795462547 wait complete. 2024-03-30T02:47:09.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.682+0000 7f4a567ffb80 1 -- start start 2024-03-30T02:47:09.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.682+0000 7f4a567ffb80 1 Processor -- start 2024-03-30T02:47:09.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22f048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22f02e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a567ffb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 0x556d22f033c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22f03900 con 0x556d22ef25d0 2024-03-30T02:47:09.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22f03a70 con 0x556d22efddf0 2024-03-30T02:47:09.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a567ffb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d22f03c30 con 0x556d22ef2ef0 2024-03-30T02:47:09.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a567ffb80 10 client.0 _ll_register_callbacks cb 0x556d22dea470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:09.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a3be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22f048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a3be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22f048c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51492/0 (socket says 172.21.15.67:51492) 2024-03-30T02:47:09.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a3be00700 1 -- 172.21.15.67:0/1105099585 learned_addr learned my addr 172.21.15.67:0/1105099585 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:09.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a40c00700 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 0x556d22f033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a3b400700 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22f02e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a3be00700 1 -- 172.21.15.67:0/1105099585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 msgr2=0x556d22f02e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:09.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22f02e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a3be00700 1 -- 172.21.15.67:0/1105099585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 msgr2=0x556d22f033c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:09.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 0x556d22f033c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.683+0000 7f4a3be00700 1 -- 172.21.15.67:0/1105099585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a34003bd0 con 0x556d22ef25d0 2024-03-30T02:47:09.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a3be00700 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22f048c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c097ca0 tx=0x7f4a2c0cbd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e60bcbe088dd472 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:09.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a25600700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:47:09.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a25600700 1 -- 172.21.15.67:0/1105099585 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c063070 con 0x556d22ef25d0 2024-03-30T02:47:09.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a25600700 1 -- 172.21.15.67:0/1105099585 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f4a2c095e40 con 0x556d22ef25d0 2024-03-30T02:47:09.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1105099585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a3400f040 con 0x556d22ef25d0 2024-03-30T02:47:09.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a25600700 1 -- 172.21.15.67:0/1105099585 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a2c0ce170 con 0x556d22ef25d0 2024-03-30T02:47:09.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a567ffb80 10 client.17520 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:09.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1105099585 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556d22e0aea0 con 0x556d22ef25d0 2024-03-30T02:47:09.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a25600700 1 -- 172.21.15.67:0/1105099585 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3400f040 con 0x556d22ef25d0 2024-03-30T02:47:09.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.684+0000 7f4a25600700 1 -- 172.21.15.67:0/1105099585 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4a2c097020 con 0x556d22ef25d0 2024-03-30T02:47:09.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.685+0000 7f4a567ffb80 10 client.17520 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:09.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.685+0000 7f4a567ffb80 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:47:09.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.685+0000 7f4a567ffb80 20 client.17520 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:09.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.685+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556d22f62420 0x556d22f64810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.685+0000 7f4a567ffb80 4 client.17520 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T02:47:09.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.685+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1105099585 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556d22c93cf0 con 0x556d22f62420 2024-03-30T02:47:09.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.685+0000 7f4a3b400700 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556d22f62420 0x556d22f64810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.685+0000 7f4a3b400700 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556d22f62420 0x556d22f64810 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1016d68a5698095 server_cookie=ab28a5c5c696b858 in_seq=0 out_seq=0 2024-03-30T02:47:09.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.685+0000 7f4a25600700 10 client.17520 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:47:09.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a25600700 1 -- 172.21.15.67:0/1105099585 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556d22c93cf0 con 0x556d22f62420 2024-03-30T02:47:09.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a25600700 10 client.17520 handle_command_reply: tid=0 2024-03-30T02:47:09.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 client.17520 shutdown 2024-03-30T02:47:09.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1105099585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 msgr2=0x556d22f048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22f048c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c097ca0 tx=0x7f4a2c0cbd70 comp rx=0 tx=0).stop 2024-03-30T02:47:09.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1105099585 shutdown_connections 2024-03-30T02:47:09.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x556d22f62420 0x556d22f64810 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556d22ef25d0 0x556d22f048c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d22efddf0 0x556d22f033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 --2- 172.21.15.67:0/1105099585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556d22ef2ef0 0x556d22f02e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a25600700 0 client.17520 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:47:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1105099585 >> 172.21.15.67:0/1105099585 conn(0x556d22e526a0 msgr2=0x556d22ef9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1105099585 shutdown_connections 2024-03-30T02:47:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 1 -- 172.21.15.67:0/1105099585 wait complete. 2024-03-30T02:47:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.686+0000 7f4a567ffb80 20 client.17520 trim_cache size 0 max 16384 2024-03-30T02:47:09.694 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:47:09.694 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-03-30T02:47:09.694 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-03-30T02:47:09.694 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:47:09.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 -- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4a200a1290 msgr2=0x7f4a200a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4a200a1290 0x7f4a200a3750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c19a9d0 tx=0x7f4a2800b5f0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 -- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 msgr2=0x7f4a3c199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 0x7f4a3c199480 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c000c00 tx=0x7f4a2c009580 comp rx=0 tx=0).stop 2024-03-30T02:47:09.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 -- 172.21.15.67:0/3861639791 shutdown_connections 2024-03-30T02:47:09.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f4a200a1290 0x7f4a200a3750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4a3c100f60 0x7f4a3c199480 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a3c19dcc0 0x7f4a3c19e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 --2- 172.21.15.67:0/3861639791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a3c106080 0x7f4a3c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 -- 172.21.15.67:0/3861639791 >> 172.21.15.67:0/3861639791 conn(0x7f4a3c0fc7c0 msgr2=0x7f4a3c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 -- 172.21.15.67:0/3861639791 shutdown_connections 2024-03-30T02:47:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.687+0000 7f4a42c00700 1 -- 172.21.15.67:0/3861639791 wait complete. 2024-03-30T02:47:09.704 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T02:47:09.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:47:09.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.904+0000 7f46dcc00700 1 -- 172.21.15.67:0/2850415972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8100e60 msgr2=0x7f46d8101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.904+0000 7f46dcc00700 1 --2- 172.21.15.67:0/2850415972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8100e60 0x7f46d8101240 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009990 tx=0x7f46c8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.905+0000 7f46dcc00700 1 -- 172.21.15.67:0/2850415972 shutdown_connections 2024-03-30T02:47:09.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.905+0000 7f46dcc00700 1 --2- 172.21.15.67:0/2850415972 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8100e60 0x7f46d8101240 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.905+0000 7f46dcc00700 1 --2- 172.21.15.67:0/2850415972 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8105f80 0x7f46d810db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.905+0000 7f46dcc00700 1 --2- 172.21.15.67:0/2850415972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d8101810 0x7f46d8105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.905+0000 7f46dcc00700 1 -- 172.21.15.67:0/2850415972 >> 172.21.15.67:0/2850415972 conn(0x7f46d80fc700 msgr2=0x7f46d80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:09.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.905+0000 7f46dcc00700 1 -- 172.21.15.67:0/2850415972 shutdown_connections 2024-03-30T02:47:09.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.905+0000 7f46dcc00700 1 -- 172.21.15.67:0/2850415972 wait complete. 2024-03-30T02:47:09.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.906+0000 7f46dcc00700 1 Processor -- start 2024-03-30T02:47:09.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.906+0000 7f46dcc00700 1 -- start start 2024-03-30T02:47:09.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.906+0000 7f46dcc00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100e60 0x7f46d80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.906+0000 7f46dcc00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8105f80 0x7f46d80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.906+0000 7f46dcc00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d8073b30 0x7f46d81a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.906+0000 7f46dcc00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d81a5960 con 0x7f46d8105f80 2024-03-30T02:47:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.906+0000 7f46dcc00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d81a5ad0 con 0x7f46d8073b30 2024-03-30T02:47:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.906+0000 7f46dcc00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d81a5c90 con 0x7f46d8100e60 2024-03-30T02:47:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8105f80 0x7f46d80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8105f80 0x7f46d80735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51512/0 (socket says 172.21.15.67:51512) 2024-03-30T02:47:09.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100e60 0x7f46d80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5400700 1 -- 172.21.15.67:0/1291218066 learned_addr learned my addr 172.21.15.67:0/1291218066 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d6800700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d8073b30 0x7f46d81a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5400700 1 -- 172.21.15.67:0/1291218066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100e60 msgr2=0x7f46d80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5400700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100e60 0x7f46d80730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5400700 1 -- 172.21.15.67:0/1291218066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d8073b30 msgr2=0x7f46d81a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5400700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d8073b30 0x7f46d81a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5400700 1 -- 172.21.15.67:0/1291218066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c80095f0 con 0x7f46d8105f80 2024-03-30T02:47:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.907+0000 7f46d5400700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8105f80 0x7f46d80735f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f46c400e970 tx=0x7f46c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:09.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.908+0000 7f46cea00700 1 -- 172.21.15.67:0/1291218066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4009790 con 0x7f46d8105f80 2024-03-30T02:47:09.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.908+0000 7f46cea00700 1 -- 172.21.15.67:0/1291218066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46c400c7e0 con 0x7f46d8105f80 2024-03-30T02:47:09.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.908+0000 7f46cea00700 1 -- 172.21.15.67:0/1291218066 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c40054e0 con 0x7f46d8105f80 2024-03-30T02:47:09.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.908+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d81a5f70 con 0x7f46d8105f80 2024-03-30T02:47:09.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.908+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d81a63a0 con 0x7f46d8105f80 2024-03-30T02:47:09.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.909+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d81091f0 con 0x7f46d8105f80 2024-03-30T02:47:09.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.911+0000 7f46cea00700 1 -- 172.21.15.67:0/1291218066 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f46c400c950 con 0x7f46d8105f80 2024-03-30T02:47:09.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.912+0000 7f46cea00700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f46bc0a1290 0x7f46bc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:09.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.912+0000 7f46cea00700 1 -- 172.21.15.67:0/1291218066 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f46c4014070 con 0x7f46d8105f80 2024-03-30T02:47:09.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.912+0000 7f46d5e00700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f46bc0a1290 0x7f46bc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:09.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.913+0000 7f46d5e00700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f46bc0a1290 0x7f46bc0a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46c800b2e0 tx=0x7f46c8005fd0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:09.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:09.916+0000 7f46cea00700 1 -- 172.21.15.67:0/1291218066 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f46c4096360 con 0x7f46d8105f80 2024-03-30T02:47:10.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.184+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46d8102330 con 0x7f46d8105f80 2024-03-30T02:47:10.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.184+0000 7f46cea00700 1 -- 172.21.15.67:0/1291218066 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f46c4099980 con 0x7f46d8105f80 2024-03-30T02:47:10.184 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:47:10.184 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:45:07.400846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:47:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f46bc0a1290 msgr2=0x7f46bc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f46bc0a1290 0x7f46bc0a3750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46c800b2e0 tx=0x7f46c8005fd0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8105f80 msgr2=0x7f46d80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8105f80 0x7f46d80735f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f46c400e970 tx=0x7f46c400ec80 comp rx=0 tx=0).stop 2024-03-30T02:47:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 shutdown_connections 2024-03-30T02:47:10.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f46bc0a1290 0x7f46bc0a3750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d8073b30 0x7f46d81a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100e60 0x7f46d80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 --2- 172.21.15.67:0/1291218066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46d8105f80 0x7f46d80735f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 >> 172.21.15.67:0/1291218066 conn(0x7f46d80fc700 msgr2=0x7f46d80fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 shutdown_connections 2024-03-30T02:47:10.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.187+0000 7f46dcc00700 1 -- 172.21.15.67:0/1291218066 wait complete. 2024-03-30T02:47:10.187 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-03-30T02:47:10.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-30T02:47:10.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.407+0000 7f85ae200700 1 -- 172.21.15.67:0/194054400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a8106080 msgr2=0x7f85a810dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.407+0000 7f85ae200700 1 --2- 172.21.15.67:0/194054400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a8106080 0x7f85a810dc00 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f85a000b0a0 tx=0x7f85a000b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.407+0000 7f85ae200700 1 -- 172.21.15.67:0/194054400 shutdown_connections 2024-03-30T02:47:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.407+0000 7f85ae200700 1 --2- 172.21.15.67:0/194054400 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a8106080 0x7f85a810dc00 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.407+0000 7f85ae200700 1 --2- 172.21.15.67:0/194054400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85a8100f60 0x7f85a8101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.407+0000 7f85ae200700 1 --2- 172.21.15.67:0/194054400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a8101910 0x7f85a8105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.407+0000 7f85ae200700 1 -- 172.21.15.67:0/194054400 >> 172.21.15.67:0/194054400 conn(0x7f85a80fc7c0 msgr2=0x7f85a80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.408+0000 7f85ae200700 1 -- 172.21.15.67:0/194054400 shutdown_connections 2024-03-30T02:47:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.408+0000 7f85ae200700 1 -- 172.21.15.67:0/194054400 wait complete. 2024-03-30T02:47:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85ae200700 1 Processor -- start 2024-03-30T02:47:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85ae200700 1 -- start start 2024-03-30T02:47:10.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85ae200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85a8100f60 0x7f85a8199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85ae200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a8106080 0x7f85a8199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85ae200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a819de40 0x7f85a819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85ae200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a819e8c0 con 0x7f85a819de40 2024-03-30T02:47:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85ae200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a819ea30 con 0x7f85a8100f60 2024-03-30T02:47:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85ae200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a819ebf0 con 0x7f85a8106080 2024-03-30T02:47:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85a7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85a8100f60 0x7f85a8199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85a7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a819de40 0x7f85a819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85a7400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85a8100f60 0x7f85a8199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:44538/0 (socket says 172.21.15.67:44538) 2024-03-30T02:47:10.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85a7400700 1 -- 172.21.15.67:0/3510955291 learned_addr learned my addr 172.21.15.67:0/3510955291 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.409+0000 7f85a6a00700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a8106080 0x7f85a8199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f85a7400700 1 -- 172.21.15.67:0/3510955291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a8106080 msgr2=0x7f85a8199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f85a7400700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a8106080 0x7f85a8199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f85a7400700 1 -- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a819de40 msgr2=0x7f85a819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f85a7400700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a819de40 0x7f85a819e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f85a7400700 1 -- 172.21.15.67:0/3510955291 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0009d00 con 0x7f85a8100f60 2024-03-30T02:47:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f85a7e00700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a819de40 0x7f85a819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:10.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f85a7400700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85a8100f60 0x7f85a8199560 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f859800e9a0 tx=0x7f859800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f859fe00700 1 -- 172.21.15.67:0/3510955291 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598009790 con 0x7f85a8100f60 2024-03-30T02:47:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f859fe00700 1 -- 172.21.15.67:0/3510955291 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8598010430 con 0x7f85a8100f60 2024-03-30T02:47:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f85ae200700 1 -- 172.21.15.67:0/3510955291 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a81a6150 con 0x7f85a8100f60 2024-03-30T02:47:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.410+0000 7f859fe00700 1 -- 172.21.15.67:0/3510955291 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598018680 con 0x7f85a8100f60 2024-03-30T02:47:10.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.411+0000 7f85ae200700 1 -- 172.21.15.67:0/3510955291 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a81a6630 con 0x7f85a8100f60 2024-03-30T02:47:10.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.414+0000 7f859fe00700 1 -- 172.21.15.67:0/3510955291 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f85980105a0 con 0x7f85a8100f60 2024-03-30T02:47:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f85c1d89b80 1 Processor -- start 2024-03-30T02:47:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f85c1d89b80 1 -- start start 2024-03-30T02:47:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d55d0 0x5613cf9d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f859fe00700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f858c0a1290 0x7f858c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 0x5613cf9e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf92e990 con 0x5613cf9d5ef0 2024-03-30T02:47:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf92c350 con 0x5613cf9e0df0 2024-03-30T02:47:10.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf927c40 con 0x5613cf9d55d0 2024-03-30T02:47:10.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.415+0000 7f859fe00700 1 -- 172.21.15.67:0/3510955291 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f8598014070 con 0x7f85a8100f60 2024-03-30T02:47:10.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7400700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f858c0a1290 0x7f858c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d55d0 0x5613cf9d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d55d0 0x5613cf9d59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43190/0 (socket says 172.21.15.67:43190) 2024-03-30T02:47:10.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 -- 172.21.15.67:0/4284262095 learned_addr learned my addr 172.21.15.67:0/4284262095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 --2- 172.21.15.67:0/4284262095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 0x5613cf9e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 -- 172.21.15.67:0/4284262095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 msgr2=0x5613cf9e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 --2- 172.21.15.67:0/4284262095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 0x5613cf9e31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 -- 172.21.15.67:0/4284262095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 msgr2=0x5613cf9e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 --2- 172.21.15.67:0/4284262095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 -- 172.21.15.67:0/4284262095 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613cf7a93b0 con 0x5613cf9d55d0 2024-03-30T02:47:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a6a00700 1 --2- 172.21.15.67:0/4284262095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:10.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7400700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f858c0a1290 0x7f858c0a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8598096a30 tx=0x7f8598010b80 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.416+0000 7f85a7e00700 1 --2- 172.21.15.67:0/4284262095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d55d0 0x5613cf9d59b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f85a00060b0 tx=0x7f85a0006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7810fb31e574f5db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f8593e00700 1 -- 172.21.15.67:0/4284262095 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a000e030 con 0x5613cf9d55d0 2024-03-30T02:47:10.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f8593e00700 1 -- 172.21.15.67:0/4284262095 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a0005040 con 0x5613cf9d55d0 2024-03-30T02:47:10.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f8593e00700 1 -- 172.21.15.67:0/4284262095 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0012650 con 0x5613cf9d55d0 2024-03-30T02:47:10.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f85c1d89b80 1 -- 172.21.15.67:0/4284262095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d55d0 msgr2=0x5613cf9d59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/4284262095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d55d0 0x5613cf9d59b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f85a00060b0 tx=0x7f85a0006000 comp rx=0 tx=0).stop 2024-03-30T02:47:10.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f85c1d89b80 1 -- 172.21.15.67:0/4284262095 shutdown_connections 2024-03-30T02:47:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/4284262095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 0x5613cf9e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/4284262095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d55d0 0x5613cf9d59b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/4284262095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f85c1d89b80 1 -- 172.21.15.67:0/4284262095 >> 172.21.15.67:0/4284262095 conn(0x5613cf9356a0 msgr2=0x5613cf9d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f85c1d89b80 1 -- 172.21.15.67:0/4284262095 shutdown_connections 2024-03-30T02:47:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.417+0000 7f85c1d89b80 1 -- 172.21.15.67:0/4284262095 wait complete. 2024-03-30T02:47:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.418+0000 7f85c1d89b80 1 -- start start 2024-03-30T02:47:10.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.418+0000 7f85c1d89b80 1 Processor -- start 2024-03-30T02:47:10.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9e0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d5ef0 0x5613cf9ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 0x5613cf9de2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf8c3d80 con 0x5613cf9d55d0 2024-03-30T02:47:10.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf8c3a20 con 0x5613cf9e0df0 2024-03-30T02:47:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf7ab6a0 con 0x5613cf9d5ef0 2024-03-30T02:47:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85c1d89b80 10 client.0 _ll_register_callbacks cb 0x5613cf8cd470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9e0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9e0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51548/0 (socket says 172.21.15.67:51548) 2024-03-30T02:47:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a7e00700 1 -- 172.21.15.67:0/845307740 learned_addr learned my addr 172.21.15.67:0/845307740 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a6a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d5ef0 0x5613cf9ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a7e00700 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 0x5613cf9de2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a6a00700 1 -- 172.21.15.67:0/845307740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 msgr2=0x5613cf9de2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a6a00700 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 0x5613cf9de2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a6a00700 1 -- 172.21.15.67:0/845307740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 msgr2=0x5613cf9e0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a6a00700 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9e0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a6a00700 1 -- 172.21.15.67:0/845307740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85940098e0 con 0x5613cf9d5ef0 2024-03-30T02:47:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.419+0000 7f85a7e00700 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 0x5613cf9de2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:47:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.420+0000 7f85a6a00700 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d5ef0 0x5613cf9ddb80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f85a806a380 tx=0x7f8594011a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52cdca0cbde26a4b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.420+0000 7f8590c00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:47:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.420+0000 7f8590c00700 1 -- 172.21.15.67:0/845307740 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594016070 con 0x5613cf9d5ef0 2024-03-30T02:47:10.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.420+0000 7f85c1d89b80 1 -- 172.21.15.67:0/845307740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613cf8e8380 con 0x5613cf9d5ef0 2024-03-30T02:47:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.420+0000 7f8590c00700 1 -- 172.21.15.67:0/845307740 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f8594004300 con 0x5613cf9d5ef0 2024-03-30T02:47:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.420+0000 7f8590c00700 1 -- 172.21.15.67:0/845307740 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8594004500 con 0x5613cf9d5ef0 2024-03-30T02:47:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.420+0000 7f85c1d89b80 10 client.27607 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.420+0000 7f85c1d89b80 1 -- 172.21.15.67:0/845307740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f85a810e470 con 0x5613cf9d5ef0 2024-03-30T02:47:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.421+0000 7f8590c00700 1 -- 172.21.15.67:0/845307740 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613cf8e8380 con 0x5613cf9d5ef0 2024-03-30T02:47:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.421+0000 7f8590c00700 1 -- 172.21.15.67:0/845307740 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f859400f820 con 0x5613cf9d5ef0 2024-03-30T02:47:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.421+0000 7f85c1d89b80 10 client.27607 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.421+0000 7f85c1d89b80 10 client.27607 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:47:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.421+0000 7f85c1d89b80 20 client.27607 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.421+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5613cf9f09b0 0x5613cf9f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.421+0000 7f85c1d89b80 4 client.27607 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:47:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.421+0000 7f85c1d89b80 1 -- 172.21.15.67:0/845307740 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613cf920700 con 0x5613cf9f09b0 2024-03-30T02:47:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.421+0000 7f85a7e00700 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5613cf9f09b0 0x5613cf9f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.422+0000 7f85a7e00700 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5613cf9f09b0 0x5613cf9f2dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd4f00fac659dafe server_cookie=4247358698fa96f6 in_seq=0 out_seq=0 2024-03-30T02:47:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.422+0000 7f8590c00700 10 client.27607 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:47:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.422+0000 7f8590c00700 1 -- 172.21.15.67:0/845307740 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5613cf920700 con 0x5613cf9f09b0 2024-03-30T02:47:10.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.422+0000 7f8590c00700 10 client.27607 handle_command_reply: tid=0 2024-03-30T02:47:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.422+0000 7f85c1d89b80 1 client.27607 shutdown 2024-03-30T02:47:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 -- 172.21.15.67:0/845307740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d5ef0 msgr2=0x5613cf9ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d5ef0 0x5613cf9ddb80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f85a806a380 tx=0x7f8594011a70 comp rx=0 tx=0).stop 2024-03-30T02:47:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 -- 172.21.15.67:0/845307740 shutdown_connections 2024-03-30T02:47:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9e0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5613cf9f09b0 0x5613cf9f2dd0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9e0df0 0x5613cf9de2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/845307740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9d5ef0 0x5613cf9ddb80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f8590c00700 0 client.27607 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:47:10.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 -- 172.21.15.67:0/845307740 >> 172.21.15.67:0/845307740 conn(0x5613cf9356a0 msgr2=0x5613cf9d76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 -- 172.21.15.67:0/845307740 shutdown_connections 2024-03-30T02:47:10.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 1 -- 172.21.15.67:0/845307740 wait complete. 2024-03-30T02:47:10.431 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.423+0000 7f85c1d89b80 20 client.27607 trim_cache size 0 max 16384 2024-03-30T02:47:10.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.436+0000 7f85c1d89b80 1 Processor -- start 2024-03-30T02:47:10.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85c1d89b80 1 -- start start 2024-03-30T02:47:10.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf9deed0 con 0x5613cf9d55d0 2024-03-30T02:47:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf9f3160 con 0x5613cf9d5ef0 2024-03-30T02:47:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf9ee130 con 0x5613cf9e0df0 2024-03-30T02:47:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85a7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85a7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9ddb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51558/0 (socket says 172.21.15.67:51558) 2024-03-30T02:47:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85a6a00700 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85a7e00700 1 -- 172.21.15.67:0/3609065651 learned_addr learned my addr 172.21.15.67:0/3609065651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.437+0000 7f85a7e00700 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f85a6a00700 1 -- 172.21.15.67:0/3609065651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 msgr2=0x5613cf9ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f85a6a00700 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f85a6a00700 1 -- 172.21.15.67:0/3609065651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 msgr2=0x5613cf9ddb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f85a6a00700 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9ddb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f85a6a00700 1 -- 172.21.15.67:0/3609065651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613cf755b60 con 0x5613cf9d5ef0 2024-03-30T02:47:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f85a7e00700 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:47:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f85a6a00700 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e7a10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8594000c00 tx=0x7f859400fd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65ea70436d38aecc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f859d200700 1 -- 172.21.15.67:0/3609065651 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594016070 con 0x5613cf9d5ef0 2024-03-30T02:47:10.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f859d200700 1 -- 172.21.15.67:0/3609065651 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f859400fea0 con 0x5613cf9d5ef0 2024-03-30T02:47:10.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.438+0000 7f859d200700 1 -- 172.21.15.67:0/3609065651 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85940038c0 con 0x5613cf9d5ef0 2024-03-30T02:47:10.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3609065651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d5ef0 msgr2=0x5613cf9e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e7a10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8594000c00 tx=0x7f859400fd50 comp rx=0 tx=0).stop 2024-03-30T02:47:10.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3609065651 shutdown_connections 2024-03-30T02:47:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d55d0 0x5613cf9ddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3609065651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e7a10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3609065651 >> 172.21.15.67:0/3609065651 conn(0x5613cf9356a0 msgr2=0x5613cf9dd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3609065651 shutdown_connections 2024-03-30T02:47:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3609065651 wait complete. 2024-03-30T02:47:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 -- start start 2024-03-30T02:47:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.439+0000 7f85c1d89b80 1 Processor -- start 2024-03-30T02:47:10.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d55d0 0x5613cf9e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85c1d89b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9e60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf9e6610 con 0x5613cf9d5ef0 2024-03-30T02:47:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf9e6750 con 0x5613cf9d55d0 2024-03-30T02:47:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85c1d89b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613cf9e6910 con 0x5613cf9e0df0 2024-03-30T02:47:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85c1d89b80 10 client.0 _ll_register_callbacks cb 0x5613cf8cd470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:10.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a6a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51560/0 (socket says 172.21.15.67:51560) 2024-03-30T02:47:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a6a00700 1 -- 172.21.15.67:0/3774514098 learned_addr learned my addr 172.21.15.67:0/3774514098 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a7e00700 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d55d0 0x5613cf9e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a7e00700 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9e60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a6a00700 1 -- 172.21.15.67:0/3774514098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 msgr2=0x5613cf9e60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a6a00700 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9e60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a6a00700 1 -- 172.21.15.67:0/3774514098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d55d0 msgr2=0x5613cf9e76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a6a00700 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d55d0 0x5613cf9e76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a6a00700 1 -- 172.21.15.67:0/3774514098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a810e470 con 0x5613cf9d5ef0 2024-03-30T02:47:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.440+0000 7f85a7e00700 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9e60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:47:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.441+0000 7f85a6a00700 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e5b90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8594005a80 tx=0x7f859400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b8106a399db0bd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.441+0000 7f8590c00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:47:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.441+0000 7f8590c00700 1 -- 172.21.15.67:0/3774514098 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594016070 con 0x5613cf9d5ef0 2024-03-30T02:47:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.441+0000 7f8590c00700 1 -- 172.21.15.67:0/3774514098 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f8594000e40 con 0x5613cf9d5ef0 2024-03-30T02:47:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.441+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3774514098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85a0005040 con 0x5613cf9d5ef0 2024-03-30T02:47:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.441+0000 7f8590c00700 1 -- 172.21.15.67:0/3774514098 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8594005790 con 0x5613cf9d5ef0 2024-03-30T02:47:10.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.441+0000 7f85c1d89b80 10 client.17544 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.441+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3774514098 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5613cf8edea0 con 0x5613cf9d5ef0 2024-03-30T02:47:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.442+0000 7f8590c00700 1 -- 172.21.15.67:0/3774514098 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a0005040 con 0x5613cf9d5ef0 2024-03-30T02:47:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.442+0000 7f8590c00700 1 -- 172.21.15.67:0/3774514098 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8594011790 con 0x5613cf9d5ef0 2024-03-30T02:47:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.442+0000 7f85c1d89b80 10 client.17544 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.442+0000 7f85c1d89b80 10 client.17544 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-03-30T02:47:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.442+0000 7f85c1d89b80 20 client.17544 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.442+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5613cfa45800 0x5613cfa47c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.442+0000 7f85c1d89b80 4 client.17544 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:47:10.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.442+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3774514098 --> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5613cf776cf0 con 0x5613cfa45800 2024-03-30T02:47:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.442+0000 7f85a7e00700 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5613cfa45800 0x5613cfa47c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85a7e00700 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5613cfa45800 0x5613cfa47c20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9bb1c8677af34faa server_cookie=6cfc1a268242faaa in_seq=0 out_seq=0 2024-03-30T02:47:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f8590c00700 10 client.17544 ms_handle_connect on v2:172.21.15.170:6836/500476759 2024-03-30T02:47:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f8590c00700 1 -- 172.21.15.67:0/3774514098 <== mds.4 v2:172.21.15.170:6836/500476759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5613cf776cf0 con 0x5613cfa45800 2024-03-30T02:47:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f8590c00700 10 client.17544 handle_command_reply: tid=0 2024-03-30T02:47:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 client.17544 shutdown 2024-03-30T02:47:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3774514098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 msgr2=0x5613cf9e5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e5b90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8594005a80 tx=0x7f859400f790 comp rx=0 tx=0).stop 2024-03-30T02:47:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3774514098 shutdown_connections 2024-03-30T02:47:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.170:6836/500476759,v1:172.21.15.170:6837/500476759] conn(0x5613cfa45800 0x5613cfa47c20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613cf9e0df0 0x5613cf9e60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5613cf9d55d0 0x5613cf9e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f8590c00700 0 client.17544 ms_handle_reset on v2:172.21.15.170:6836/500476759 2024-03-30T02:47:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 --2- 172.21.15.67:0/3774514098 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613cf9d5ef0 0x5613cf9e5b90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3774514098 >> 172.21.15.67:0/3774514098 conn(0x5613cf9356a0 msgr2=0x5613cf9caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3774514098 shutdown_connections 2024-03-30T02:47:10.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.443+0000 7f85c1d89b80 1 -- 172.21.15.67:0/3774514098 wait complete. 2024-03-30T02:47:10.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85c1d89b80 20 client.17544 trim_cache size 0 max 16384 2024-03-30T02:47:10.451 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:47:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 -- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f858c0a1290 msgr2=0x7f858c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f858c0a1290 0x7f858c0a3750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8598096a30 tx=0x7f8598010b80 comp rx=0 tx=0).stop 2024-03-30T02:47:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 -- 172.21.15.67:0/3510955291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85a8100f60 msgr2=0x7f85a8199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85a8100f60 0x7f85a8199560 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f859800e9a0 tx=0x7f859800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 -- 172.21.15.67:0/3510955291 shutdown_connections 2024-03-30T02:47:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f858c0a1290 0x7f858c0a3750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85a819de40 0x7f85a819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85a8100f60 0x7f85a8199560 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 --2- 172.21.15.67:0/3510955291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85a8106080 0x7f85a8199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 -- 172.21.15.67:0/3510955291 >> 172.21.15.67:0/3510955291 conn(0x7f85a80fc7c0 msgr2=0x7f85a80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 -- 172.21.15.67:0/3510955291 shutdown_connections 2024-03-30T02:47:10.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.444+0000 7f85ae200700 1 -- 172.21.15.67:0/3510955291 wait complete. 2024-03-30T02:47:10.462 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-30T02:47:10.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.664+0000 7f3092000700 1 -- 172.21.15.67:0/4056537040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308c1026b0 msgr2=0x7f308c10ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.664+0000 7f3092000700 1 --2- 172.21.15.67:0/4056537040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308c1026b0 0x7f308c10ab90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3078009990 tx=0x7f3078009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.664+0000 7f3092000700 1 -- 172.21.15.67:0/4056537040 shutdown_connections 2024-03-30T02:47:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.664+0000 7f3092000700 1 --2- 172.21.15.67:0/4056537040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f308c10b160 0x7f308c10d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.664+0000 7f3092000700 1 --2- 172.21.15.67:0/4056537040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c101d90 0x7f308c102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.664+0000 7f3092000700 1 --2- 172.21.15.67:0/4056537040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308c1026b0 0x7f308c10ab90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.664+0000 7f3092000700 1 -- 172.21.15.67:0/4056537040 >> 172.21.15.67:0/4056537040 conn(0x7f308c0fb510 msgr2=0x7f308c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.664+0000 7f3092000700 1 -- 172.21.15.67:0/4056537040 shutdown_connections 2024-03-30T02:47:10.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.664+0000 7f3092000700 1 -- 172.21.15.67:0/4056537040 wait complete. 2024-03-30T02:47:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.665+0000 7f3092000700 1 Processor -- start 2024-03-30T02:47:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.665+0000 7f3092000700 1 -- start start 2024-03-30T02:47:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3092000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f308c101d90 0x7f308c0ff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3092000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308c1026b0 0x7f308c0fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3092000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c10b160 0x7f308c100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3092000700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c1a53e0 con 0x7f308c1026b0 2024-03-30T02:47:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3092000700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c1a5550 con 0x7f308c101d90 2024-03-30T02:47:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3092000700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c1a5710 con 0x7f308c10b160 2024-03-30T02:47:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f308be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308c1026b0 0x7f308c0fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3090c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f308c101d90 0x7f308c0ff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3090c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f308c101d90 0x7f308c0ff9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:44594/0 (socket says 172.21.15.67:44594) 2024-03-30T02:47:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3090c00700 1 -- 172.21.15.67:0/1704302824 learned_addr learned my addr 172.21.15.67:0/1704302824 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.666+0000 7f3091600700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c10b160 0x7f308c100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3090c00700 1 -- 172.21.15.67:0/1704302824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c10b160 msgr2=0x7f308c100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3090c00700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c10b160 0x7f308c100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3090c00700 1 -- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308c1026b0 msgr2=0x7f308c0fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3090c00700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308c1026b0 0x7f308c0fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3090c00700 1 -- 172.21.15.67:0/1704302824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30780095f0 con 0x7f308c101d90 2024-03-30T02:47:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f308be00700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308c1026b0 0x7f308c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3090c00700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f308c101d90 0x7f308c0ff9e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f308000e9a0 tx=0x7f308000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3089600700 1 -- 172.21.15.67:0/1704302824 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080009790 con 0x7f308c101d90 2024-03-30T02:47:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3089600700 1 -- 172.21.15.67:0/1704302824 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3080010430 con 0x7f308c101d90 2024-03-30T02:47:10.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3092000700 1 -- 172.21.15.67:0/1704302824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c1a59f0 con 0x7f308c101d90 2024-03-30T02:47:10.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3089600700 1 -- 172.21.15.67:0/1704302824 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080018680 con 0x7f308c101d90 2024-03-30T02:47:10.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.667+0000 7f3092000700 1 -- 172.21.15.67:0/1704302824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c1a5f00 con 0x7f308c101d90 2024-03-30T02:47:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.670+0000 7f3089600700 1 -- 172.21.15.67:0/1704302824 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f30800105a0 con 0x7f308c101d90 2024-03-30T02:47:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.672+0000 7f3089600700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f30740a1290 0x7f30740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.672+0000 7f30a7193b80 1 Processor -- start 2024-03-30T02:47:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.672+0000 7f3089600700 1 -- 172.21.15.67:0/1704302824 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f3080014070 con 0x7f308c101d90 2024-03-30T02:47:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.672+0000 7f308be00700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f30740a1290 0x7f30740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.672+0000 7f30a7193b80 1 -- start start 2024-03-30T02:47:10.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9bf1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf20c0 0x5629c9bfca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 0x5629c9bff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f30a7193b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9b4a990 con 0x5629c9bf20c0 2024-03-30T02:47:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f30a7193b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9b48350 con 0x5629c9bf17a0 2024-03-30T02:47:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f30a7193b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9b43c40 con 0x5629c9bfcfc0 2024-03-30T02:47:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f308be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf20c0 0x5629c9bfca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f308be00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf20c0 0x5629c9bfca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51580/0 (socket says 172.21.15.67:51580) 2024-03-30T02:47:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f308be00700 1 -- 172.21.15.67:0/217468020 learned_addr learned my addr 172.21.15.67:0/217468020 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f3090c00700 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9bf1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f308be00700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f30740a1290 0x7f30740a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f308c101110 tx=0x7f307800b260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f3091600700 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 0x5629c9bff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f308be00700 1 -- 172.21.15.67:0/217468020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 msgr2=0x5629c9bff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f308be00700 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 0x5629c9bff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f308be00700 1 -- 172.21.15.67:0/217468020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 msgr2=0x5629c9bf1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f308be00700 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9bf1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f308be00700 1 -- 172.21.15.67:0/217468020 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629c99c53b0 con 0x5629c9bf20c0 2024-03-30T02:47:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f3091600700 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 0x5629c9bff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:47:10.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.673+0000 7f3090c00700 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9bf1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.674+0000 7f308be00700 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf20c0 0x5629c9bfca80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f307800b6c0 tx=0x7f3078020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a359d84fb1abea3a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.674+0000 7f307dc00700 1 -- 172.21.15.67:0/217468020 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078004800 con 0x5629c9bf20c0 2024-03-30T02:47:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.674+0000 7f307dc00700 1 -- 172.21.15.67:0/217468020 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f307801e040 con 0x5629c9bf20c0 2024-03-30T02:47:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.674+0000 7f307dc00700 1 -- 172.21.15.67:0/217468020 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307800fc00 con 0x5629c9bf20c0 2024-03-30T02:47:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.674+0000 7f30a7193b80 1 -- 172.21.15.67:0/217468020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf20c0 msgr2=0x5629c9bfca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.674+0000 7f30a7193b80 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf20c0 0x5629c9bfca80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f307800b6c0 tx=0x7f3078020000 comp rx=0 tx=0).stop 2024-03-30T02:47:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.675+0000 7f30a7193b80 1 -- 172.21.15.67:0/217468020 shutdown_connections 2024-03-30T02:47:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.675+0000 7f30a7193b80 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 0x5629c9bff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.675+0000 7f30a7193b80 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9bf1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.675+0000 7f30a7193b80 1 --2- 172.21.15.67:0/217468020 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf20c0 0x5629c9bfca80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.675+0000 7f30a7193b80 1 -- 172.21.15.67:0/217468020 >> 172.21.15.67:0/217468020 conn(0x5629c9b516a0 msgr2=0x5629c9bedc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.675+0000 7f30a7193b80 1 -- 172.21.15.67:0/217468020 shutdown_connections 2024-03-30T02:47:10.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.675+0000 7f30a7193b80 1 -- 172.21.15.67:0/217468020 wait complete. 2024-03-30T02:47:10.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.675+0000 7f30a7193b80 1 -- start start 2024-03-30T02:47:10.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.675+0000 7f30a7193b80 1 Processor -- start 2024-03-30T02:47:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf17a0 0x5629c9bfc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf20c0 0x5629c9bf9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 0x5629c9bfa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f30a7193b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9adfd80 con 0x5629c9bf17a0 2024-03-30T02:47:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f30a7193b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9adfa20 con 0x5629c9bf20c0 2024-03-30T02:47:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f30a7193b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c99c76a0 con 0x5629c9bfcfc0 2024-03-30T02:47:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f30a7193b80 10 client.0 _ll_register_callbacks cb 0x5629c9ae9470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:10.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f3090c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf17a0 0x5629c9bfc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f3090c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf17a0 0x5629c9bfc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51590/0 (socket says 172.21.15.67:51590) 2024-03-30T02:47:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f3090c00700 1 -- 172.21.15.67:0/1114045870 learned_addr learned my addr 172.21.15.67:0/1114045870 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f308be00700 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf20c0 0x5629c9bf9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.676+0000 7f3091600700 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 0x5629c9bfa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3090c00700 1 -- 172.21.15.67:0/1114045870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 msgr2=0x5629c9bfa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3090c00700 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 0x5629c9bfa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3090c00700 1 -- 172.21.15.67:0/1114045870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf20c0 msgr2=0x5629c9bf9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3090c00700 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf20c0 0x5629c9bf9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3090c00700 1 -- 172.21.15.67:0/1114045870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c1a5f00 con 0x5629c9bf17a0 2024-03-30T02:47:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f308be00700 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf20c0 0x5629c9bf9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3090c00700 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf17a0 0x5629c9bfc940 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3080096a30 tx=0x7f308009a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa167fa5ce0c44 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3072000700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:47:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3072000700 1 -- 172.21.15.67:0/1114045870 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308009ddf0 con 0x5629c9bf17a0 2024-03-30T02:47:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3072000700 1 -- 172.21.15.67:0/1114045870 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f30800a2440 con 0x5629c9bf17a0 2024-03-30T02:47:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f30a7193b80 1 -- 172.21.15.67:0/1114045870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629c9b04380 con 0x5629c9bf17a0 2024-03-30T02:47:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f3072000700 1 -- 172.21.15.67:0/1114045870 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30800cd0b0 con 0x5629c9bf17a0 2024-03-30T02:47:10.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.677+0000 7f30a7193b80 10 client.17556 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:10.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f30a7193b80 1 -- 172.21.15.67:0/1114045870 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f308c10e290 con 0x5629c9bf17a0 2024-03-30T02:47:10.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f3072000700 1 -- 172.21.15.67:0/1114045870 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629c9b04380 con 0x5629c9bf17a0 2024-03-30T02:47:10.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f3072000700 1 -- 172.21.15.67:0/1114045870 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f30800a1fb0 con 0x5629c9bf17a0 2024-03-30T02:47:10.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f30a7193b80 10 client.17556 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:10.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f30a7193b80 10 client.17556 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:47:10.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f30a7193b80 20 client.17556 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:10.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5629c9c0cb90 0x5629c9c0ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f30a7193b80 4 client.17556 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:47:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f30a7193b80 1 -- 172.21.15.67:0/1114045870 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629c9b3c700 con 0x5629c9c0cb90 2024-03-30T02:47:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.678+0000 7f308be00700 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5629c9c0cb90 0x5629c9c0ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.679+0000 7f308be00700 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5629c9c0cb90 0x5629c9c0ef80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b82182f35fb53ef3 server_cookie=16cf674639cd46ee in_seq=0 out_seq=0 2024-03-30T02:47:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.679+0000 7f3072000700 10 client.17556 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:47:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.679+0000 7f3072000700 1 -- 172.21.15.67:0/1114045870 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5629c9b3c700 con 0x5629c9c0cb90 2024-03-30T02:47:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.679+0000 7f3072000700 10 client.17556 handle_command_reply: tid=0 2024-03-30T02:47:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 client.17556 shutdown 2024-03-30T02:47:10.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 -- 172.21.15.67:0/1114045870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf17a0 msgr2=0x5629c9bfc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf17a0 0x5629c9bfc940 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3080096a30 tx=0x7f308009a740 comp rx=0 tx=0).stop 2024-03-30T02:47:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 -- 172.21.15.67:0/1114045870 shutdown_connections 2024-03-30T02:47:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5629c9c0cb90 0x5629c9c0ef80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bf17a0 0x5629c9bfc940 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f3072000700 0 client.17556 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:47:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bfcfc0 0x5629c9bfa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1114045870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf20c0 0x5629c9bf9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 -- 172.21.15.67:0/1114045870 >> 172.21.15.67:0/1114045870 conn(0x5629c9b516a0 msgr2=0x5629c9bf3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 -- 172.21.15.67:0/1114045870 shutdown_connections 2024-03-30T02:47:10.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 1 -- 172.21.15.67:0/1114045870 wait complete. 2024-03-30T02:47:10.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.680+0000 7f30a7193b80 20 client.17556 trim_cache size 0 max 16384 2024-03-30T02:47:10.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.693+0000 7f30a7193b80 1 Processor -- start 2024-03-30T02:47:10.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f30a7193b80 1 -- start start 2024-03-30T02:47:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9c03c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 0x5629c9bed700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9bedc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f30a7193b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9bf36a0 con 0x5629c9bfcfc0 2024-03-30T02:47:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f30a7193b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9bee180 con 0x5629c9bf17a0 2024-03-30T02:47:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f30a7193b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9bee830 con 0x5629c9bf20c0 2024-03-30T02:47:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f3091600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9bedc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f3091600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9bedc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51592/0 (socket says 172.21.15.67:51592) 2024-03-30T02:47:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f3091600700 1 -- 172.21.15.67:0/1591174149 learned_addr learned my addr 172.21.15.67:0/1591174149 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f3090c00700 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9c03c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.694+0000 7f308be00700 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 0x5629c9bed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f3091600700 1 -- 172.21.15.67:0/1591174149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 msgr2=0x5629c9bed700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f3091600700 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 0x5629c9bed700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f3091600700 1 -- 172.21.15.67:0/1591174149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 msgr2=0x5629c9c03c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f3091600700 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9c03c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f3091600700 1 -- 172.21.15.67:0/1591174149 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629c9971b60 con 0x5629c9bfcfc0 2024-03-30T02:47:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f3090c00700 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9c03c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f3091600700 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9bedc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3084007710 tx=0x7f308400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b523bdaba671e74 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f307e600700 1 -- 172.21.15.67:0/1591174149 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308400e040 con 0x5629c9bfcfc0 2024-03-30T02:47:10.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f307e600700 1 -- 172.21.15.67:0/1591174149 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f308400f040 con 0x5629c9bfcfc0 2024-03-30T02:47:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.695+0000 7f307e600700 1 -- 172.21.15.67:0/1591174149 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084016a40 con 0x5629c9bfcfc0 2024-03-30T02:47:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 -- 172.21.15.67:0/1591174149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 msgr2=0x5629c9bedc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9bedc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3084007710 tx=0x7f308400b040 comp rx=0 tx=0).stop 2024-03-30T02:47:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 -- 172.21.15.67:0/1591174149 shutdown_connections 2024-03-30T02:47:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9bedc40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9c03c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 --2- 172.21.15.67:0/1591174149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 0x5629c9bed700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 -- 172.21.15.67:0/1591174149 >> 172.21.15.67:0/1591174149 conn(0x5629c9b516a0 msgr2=0x5629c9bf8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 -- 172.21.15.67:0/1591174149 shutdown_connections 2024-03-30T02:47:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 -- 172.21.15.67:0/1591174149 wait complete. 2024-03-30T02:47:10.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.696+0000 7f30a7193b80 1 -- start start 2024-03-30T02:47:10.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.697+0000 7f30a7193b80 1 Processor -- start 2024-03-30T02:47:10.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.697+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9c038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.697+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 0x5629c9c01da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.697+0000 7f30a7193b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9c022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.697+0000 7f30a7193b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9c02860 con 0x5629c9bfcfc0 2024-03-30T02:47:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.697+0000 7f30a7193b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9c029a0 con 0x5629c9bf17a0 2024-03-30T02:47:10.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.697+0000 7f30a7193b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c9c02b60 con 0x5629c9bf20c0 2024-03-30T02:47:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f30a7193b80 10 client.0 _ll_register_callbacks cb 0x5629c9ae9470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3091600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9c022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3091600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9c022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51604/0 (socket says 172.21.15.67:51604) 2024-03-30T02:47:10.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3091600700 1 -- 172.21.15.67:0/551338177 learned_addr learned my addr 172.21.15.67:0/551338177 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3090c00700 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9c038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f308be00700 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 0x5629c9c01da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3091600700 1 -- 172.21.15.67:0/551338177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 msgr2=0x5629c9c01da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3091600700 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 0x5629c9c01da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3091600700 1 -- 172.21.15.67:0/551338177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 msgr2=0x5629c9c038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3091600700 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9c038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3091600700 1 -- 172.21.15.67:0/551338177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c10e290 con 0x5629c9bfcfc0 2024-03-30T02:47:10.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3091600700 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9c022e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3084016620 tx=0x7f30840168b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35d463073e4d6e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.698+0000 7f3072000700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:47:10.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f3072000700 1 -- 172.21.15.67:0/551338177 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308400e070 con 0x5629c9bfcfc0 2024-03-30T02:47:10.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f30a7193b80 1 -- 172.21.15.67:0/551338177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f307801e040 con 0x5629c9bfcfc0 2024-03-30T02:47:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f3072000700 1 -- 172.21.15.67:0/551338177 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f3084016aa0 con 0x5629c9bfcfc0 2024-03-30T02:47:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f3072000700 1 -- 172.21.15.67:0/551338177 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3084016c60 con 0x5629c9bfcfc0 2024-03-30T02:47:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f30a7193b80 10 client.17562 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f30a7193b80 1 -- 172.21.15.67:0/551338177 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f30840092c0 con 0x5629c9bfcfc0 2024-03-30T02:47:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f3072000700 1 -- 172.21.15.67:0/551338177 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f307801e040 con 0x5629c9bfcfc0 2024-03-30T02:47:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f3072000700 1 -- 172.21.15.67:0/551338177 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3084029480 con 0x5629c9bfcfc0 2024-03-30T02:47:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f30a7193b80 10 client.17562 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:10.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f30a7193b80 10 client.17562 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-03-30T02:47:10.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f30a7193b80 20 client.17562 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:10.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f30a7193b80 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5629c9c61df0 0x5629c9c64210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f30a7193b80 4 client.17562 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:47:10.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.699+0000 7f30a7193b80 1 -- 172.21.15.67:0/551338177 --> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5629c9992cf0 con 0x5629c9c61df0 2024-03-30T02:47:10.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.700+0000 7f3090c00700 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5629c9c61df0 0x5629c9c64210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.700+0000 7f3090c00700 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5629c9c61df0 0x5629c9c64210 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c8a456f218196aa server_cookie=84e43e00b5b68180 in_seq=0 out_seq=0 2024-03-30T02:47:10.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.700+0000 7f3072000700 10 client.17562 ms_handle_connect on v2:172.21.15.82:6836/1183051781 2024-03-30T02:47:10.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.700+0000 7f3072000700 1 -- 172.21.15.67:0/551338177 <== mds.3 v2:172.21.15.82:6836/1183051781 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5629c9992cf0 con 0x5629c9c61df0 2024-03-30T02:47:10.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.700+0000 7f3072000700 10 client.17562 handle_command_reply: tid=0 2024-03-30T02:47:10.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.700+0000 7f30a7193b80 1 client.17562 shutdown 2024-03-30T02:47:10.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 -- 172.21.15.67:0/551338177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 msgr2=0x5629c9c022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9c022e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3084016620 tx=0x7f30840168b0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 -- 172.21.15.67:0/551338177 shutdown_connections 2024-03-30T02:47:10.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.82:6836/1183051781,v1:172.21.15.82:6837/1183051781] conn(0x5629c9c61df0 0x5629c9c64210 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5629c9bfcfc0 0x5629c9c022e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629c9bf17a0 0x5629c9c038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 --2- 172.21.15.67:0/551338177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5629c9bf20c0 0x5629c9c01da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 -- 172.21.15.67:0/551338177 >> 172.21.15.67:0/551338177 conn(0x5629c9b516a0 msgr2=0x5629c9bf3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f3072000700 0 client.17562 ms_handle_reset on v2:172.21.15.82:6836/1183051781 2024-03-30T02:47:10.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 -- 172.21.15.67:0/551338177 shutdown_connections 2024-03-30T02:47:10.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 1 -- 172.21.15.67:0/551338177 wait complete. 2024-03-30T02:47:10.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f30a7193b80 20 client.17562 trim_cache size 0 max 16384 2024-03-30T02:47:10.718 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:47:10.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f3092000700 1 -- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f30740a1290 msgr2=0x7f30740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f3092000700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f30740a1290 0x7f30740a3750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f308c101110 tx=0x7f307800b260 comp rx=0 tx=0).stop 2024-03-30T02:47:10.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f3092000700 1 -- 172.21.15.67:0/1704302824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f308c101d90 msgr2=0x7f308c0ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.701+0000 7f3092000700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f308c101d90 0x7f308c0ff9e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f308000e9a0 tx=0x7f308000ecb0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.702+0000 7f3092000700 1 -- 172.21.15.67:0/1704302824 shutdown_connections 2024-03-30T02:47:10.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.702+0000 7f3092000700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f30740a1290 0x7f30740a3750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.702+0000 7f3092000700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f308c10b160 0x7f308c100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.702+0000 7f3092000700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f308c101d90 0x7f308c0ff9e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.702+0000 7f3092000700 1 --2- 172.21.15.67:0/1704302824 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f308c1026b0 0x7f308c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.702+0000 7f3092000700 1 -- 172.21.15.67:0/1704302824 >> 172.21.15.67:0/1704302824 conn(0x7f308c0fb510 msgr2=0x7f308c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.702+0000 7f3092000700 1 -- 172.21.15.67:0/1704302824 shutdown_connections 2024-03-30T02:47:10.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.702+0000 7f3092000700 1 -- 172.21.15.67:0/1704302824 wait complete. 2024-03-30T02:47:10.723 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:10 smithi067 ceph-mon[37808]: pgmap v313: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 69 KiB/s wr, 31 op/s 2024-03-30T02:47:10.723 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:10 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1291218066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:47:10.724 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-30T02:47:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:10 smithi082 ceph-mon[79176]: pgmap v313: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 69 KiB/s wr, 31 op/s 2024-03-30T02:47:10.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:10 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1291218066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:47:10.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.929+0000 7fe2f1600700 1 -- 172.21.15.67:0/396208917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec101910 msgr2=0x7fe2ec105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.929+0000 7fe2f1600700 1 --2- 172.21.15.67:0/396208917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec101910 0x7fe2ec105950 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009990 tx=0x7fe2d8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.929+0000 7fe2f1600700 1 -- 172.21.15.67:0/396208917 shutdown_connections 2024-03-30T02:47:10.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.930+0000 7fe2f1600700 1 --2- 172.21.15.67:0/396208917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2ec106080 0x7fe2ec10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.930+0000 7fe2f1600700 1 --2- 172.21.15.67:0/396208917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2ec100f60 0x7fe2ec101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.930+0000 7fe2f1600700 1 --2- 172.21.15.67:0/396208917 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec101910 0x7fe2ec105950 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.930+0000 7fe2f1600700 1 -- 172.21.15.67:0/396208917 >> 172.21.15.67:0/396208917 conn(0x7fe2ec0fc7c0 msgr2=0x7fe2ec0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.930+0000 7fe2f1600700 1 -- 172.21.15.67:0/396208917 shutdown_connections 2024-03-30T02:47:10.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.930+0000 7fe2f1600700 1 -- 172.21.15.67:0/396208917 wait complete. 2024-03-30T02:47:10.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2f1600700 1 Processor -- start 2024-03-30T02:47:10.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2f1600700 1 -- start start 2024-03-30T02:47:10.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2f1600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2ec100f60 0x7fe2ec199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2f1600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec106080 0x7fe2ec199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2f1600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2ec19dd30 0x7fe2ec19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2f1600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec19e7b0 con 0x7fe2ec106080 2024-03-30T02:47:10.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2f1600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec19e920 con 0x7fe2ec100f60 2024-03-30T02:47:10.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2f1600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec19eae0 con 0x7fe2ec19dd30 2024-03-30T02:47:10.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2ea000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec106080 0x7fe2ec199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.931+0000 7fe2eb400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2ec19dd30 0x7fe2ec19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2ea000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec106080 0x7fe2ec199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51620/0 (socket says 172.21.15.67:51620) 2024-03-30T02:47:10.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2ea000700 1 -- 172.21.15.67:0/2906087227 learned_addr learned my addr 172.21.15.67:0/2906087227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2ec100f60 0x7fe2ec199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2ea000700 1 -- 172.21.15.67:0/2906087227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2ec19dd30 msgr2=0x7fe2ec19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2ea000700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2ec19dd30 0x7fe2ec19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2ea000700 1 -- 172.21.15.67:0/2906087227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2ec100f60 msgr2=0x7fe2ec199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2ea000700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2ec100f60 0x7fe2ec199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2ea000700 1 -- 172.21.15.67:0/2906087227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d80095f0 con 0x7fe2ec106080 2024-03-30T02:47:10.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2ea000700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec106080 0x7fe2ec199990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8004760 tx=0x7fe2d8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2e3400700 1 -- 172.21.15.67:0/2906087227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d801d070 con 0x7fe2ec106080 2024-03-30T02:47:10.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2e3400700 1 -- 172.21.15.67:0/2906087227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2d8022470 con 0x7fe2ec106080 2024-03-30T02:47:10.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2e3400700 1 -- 172.21.15.67:0/2906087227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8017650 con 0x7fe2ec106080 2024-03-30T02:47:10.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.932+0000 7fe2f1600700 1 -- 172.21.15.67:0/2906087227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec1a5fe0 con 0x7fe2ec106080 2024-03-30T02:47:10.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.933+0000 7fe2f1600700 1 -- 172.21.15.67:0/2906087227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec1a6350 con 0x7fe2ec106080 2024-03-30T02:47:10.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.936+0000 7fe2e3400700 1 -- 172.21.15.67:0/2906087227 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe2d80225e0 con 0x7fe2ec106080 2024-03-30T02:47:10.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.937+0000 7fe30507db80 1 Processor -- start 2024-03-30T02:47:10.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe30507db80 1 -- start start 2024-03-30T02:47:10.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe2e3400700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe2d00a1310 0x7fe2d00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe30507db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe30507db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a87738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.939 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe30507db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a87761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe30507db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a86c1990 con 0x55d1a8773df0 2024-03-30T02:47:10.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe30507db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a86bf350 con 0x55d1a87685d0 2024-03-30T02:47:10.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe30507db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a86bac40 con 0x55d1a8768ef0 2024-03-30T02:47:10.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe2e3400700 1 -- 172.21.15.67:0/2906087227 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7fe2d80ce2d0 con 0x7fe2ec106080 2024-03-30T02:47:10.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe2eaa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe2ea000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a87738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe2ea000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a87738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43270/0 (socket says 172.21.15.67:43270) 2024-03-30T02:47:10.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe2eaa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:44656/0 (socket says 172.21.15.67:44656) 2024-03-30T02:47:10.940 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.938+0000 7fe2ea000700 1 -- 172.21.15.67:0/1816263529 learned_addr learned my addr 172.21.15.67:0/1816263529 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2eb400700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe2d00a1310 0x7fe2d00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2eb400700 1 --2- 172.21.15.67:0/1816263529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a87761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2ea000700 1 -- 172.21.15.67:0/1816263529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 msgr2=0x55d1a87689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2ea000700 1 --2- 172.21.15.67:0/1816263529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2ea000700 1 -- 172.21.15.67:0/1816263529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 msgr2=0x55d1a87761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2ea000700 1 --2- 172.21.15.67:0/1816263529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a87761e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2ea000700 1 -- 172.21.15.67:0/1816263529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1a853c3b0 con 0x55d1a8768ef0 2024-03-30T02:47:10.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2eb400700 1 --2- 172.21.15.67:0/1816263529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a87761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:47:10.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2ea000700 1 --2- 172.21.15.67:0/1816263529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a87738b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80a2ed0 tx=0x7fe2d8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9472d89fae0ea06b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2eb400700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe2d00a1310 0x7fe2d00a37d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009ec0 tx=0x7fe2e4009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.939+0000 7fe2d7400700 1 -- 172.21.15.67:0/1816263529 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d809ab50 con 0x55d1a8768ef0 2024-03-30T02:47:10.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe2d7400700 1 -- 172.21.15.67:0/1816263529 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2d809a970 con 0x55d1a8768ef0 2024-03-30T02:47:10.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe2d7400700 1 -- 172.21.15.67:0/1816263529 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8085f90 con 0x55d1a8768ef0 2024-03-30T02:47:10.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe30507db80 1 -- 172.21.15.67:0/1816263529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 msgr2=0x55d1a87738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe30507db80 1 --2- 172.21.15.67:0/1816263529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a87738b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80a2ed0 tx=0x7fe2d8009310 comp rx=0 tx=0).stop 2024-03-30T02:47:10.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe30507db80 1 -- 172.21.15.67:0/1816263529 shutdown_connections 2024-03-30T02:47:10.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe30507db80 1 --2- 172.21.15.67:0/1816263529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a87761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe30507db80 1 --2- 172.21.15.67:0/1816263529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe30507db80 1 --2- 172.21.15.67:0/1816263529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a87738b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe30507db80 1 -- 172.21.15.67:0/1816263529 >> 172.21.15.67:0/1816263529 conn(0x55d1a86c86a0 msgr2=0x55d1a8764a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe30507db80 1 -- 172.21.15.67:0/1816263529 shutdown_connections 2024-03-30T02:47:10.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.940+0000 7fe30507db80 1 -- 172.21.15.67:0/1816263529 wait complete. 2024-03-30T02:47:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:10 smithi170 ceph-mon[78798]: pgmap v313: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 69 KiB/s wr, 31 op/s 2024-03-30T02:47:10.944 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:10 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1291218066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:47:10.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.941+0000 7fe30507db80 1 -- start start 2024-03-30T02:47:10.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.941+0000 7fe30507db80 1 Processor -- start 2024-03-30T02:47:10.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe30507db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe30507db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a8770a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe30507db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a8770f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe30507db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a8656d80 con 0x55d1a8773df0 2024-03-30T02:47:10.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe30507db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a8656a20 con 0x55d1a87685d0 2024-03-30T02:47:10.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe30507db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a853e6a0 con 0x55d1a8768ef0 2024-03-30T02:47:10.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe30507db80 10 client.0 _ll_register_callbacks cb 0x55d1a8660470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:10.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a8770f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eb400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a8770f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51650/0 (socket says 172.21.15.67:51650) 2024-03-30T02:47:10.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eb400700 1 -- 172.21.15.67:0/3098085687 learned_addr learned my addr 172.21.15.67:0/3098085687 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2ea000700 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a8770a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eb400700 1 -- 172.21.15.67:0/3098085687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 msgr2=0x55d1a8770a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eb400700 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a8770a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eb400700 1 -- 172.21.15.67:0/3098085687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 msgr2=0x55d1a87736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eb400700 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eb400700 1 -- 172.21.15.67:0/3098085687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec1a6350 con 0x55d1a8773df0 2024-03-30T02:47:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.942+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:10.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.943+0000 7fe2eb400700 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a8770f60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4017e60 tx=0x7fe2e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e099730972427a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.943+0000 7fe2cbe00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:47:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.943+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3098085687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e400e040 con 0x55d1a8773df0 2024-03-30T02:47:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.943+0000 7fe30507db80 1 -- 172.21.15.67:0/3098085687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1a867b380 con 0x55d1a8773df0 2024-03-30T02:47:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.943+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3098085687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7fe2e4018b80 con 0x55d1a8773df0 2024-03-30T02:47:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.943+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3098085687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2e4018d40 con 0x55d1a8773df0 2024-03-30T02:47:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.943+0000 7fe30507db80 10 client.17571 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.943+0000 7fe30507db80 1 -- 172.21.15.67:0/3098085687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d1a84e8b60 con 0x55d1a8773df0 2024-03-30T02:47:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.944+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3098085687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d1a867b380 con 0x55d1a8773df0 2024-03-30T02:47:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.944+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3098085687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe2e4004680 con 0x55d1a8773df0 2024-03-30T02:47:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.944+0000 7fe30507db80 10 client.17571 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.944+0000 7fe30507db80 10 client.17571 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:47:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.944+0000 7fe30507db80 20 client.17571 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.944+0000 7fe30507db80 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55d1a8783590 0x55d1a87859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.944+0000 7fe30507db80 4 client.17571 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:47:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.944+0000 7fe30507db80 1 -- 172.21.15.67:0/3098085687 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1a86b3700 con 0x55d1a8783590 2024-03-30T02:47:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.944+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55d1a8783590 0x55d1a87859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.945+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55d1a8783590 0x55d1a87859b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f784ecfe11b64b73 server_cookie=c6f4cc974777c537 in_seq=0 out_seq=0 2024-03-30T02:47:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.945+0000 7fe2cbe00700 10 client.17571 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:47:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3098085687 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d1a86b3700 con 0x55d1a8783590 2024-03-30T02:47:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe2cbe00700 10 client.17571 handle_command_reply: tid=0 2024-03-30T02:47:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 client.17571 shutdown 2024-03-30T02:47:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 -- 172.21.15.67:0/3098085687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 msgr2=0x55d1a8770f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a8770f60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4017e60 tx=0x7fe2e4004ad0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 -- 172.21.15.67:0/3098085687 shutdown_connections 2024-03-30T02:47:10.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55d1a8783590 0x55d1a87859b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a8770f60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a87685d0 0x55d1a87736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 --2- 172.21.15.67:0/3098085687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a8768ef0 0x55d1a8770a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 -- 172.21.15.67:0/3098085687 >> 172.21.15.67:0/3098085687 conn(0x55d1a86c86a0 msgr2=0x55d1a876a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe2cbe00700 0 client.17571 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:47:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 -- 172.21.15.67:0/3098085687 shutdown_connections 2024-03-30T02:47:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 1 -- 172.21.15.67:0/3098085687 wait complete. 2024-03-30T02:47:10.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.946+0000 7fe30507db80 20 client.17571 trim_cache size 0 max 16384 2024-03-30T02:47:10.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.960+0000 7fe30507db80 1 Processor -- start 2024-03-30T02:47:10.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.960+0000 7fe30507db80 1 -- start start 2024-03-30T02:47:10.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.960+0000 7fe30507db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.960+0000 7fe30507db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8768ef0 0x55d1a877a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.960+0000 7fe30507db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8773df0 0x55d1a875df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.960+0000 7fe30507db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a8780120 con 0x55d1a8768ef0 2024-03-30T02:47:10.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.960+0000 7fe30507db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a876b830 con 0x55d1a8773df0 2024-03-30T02:47:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.960+0000 7fe30507db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a877ad20 con 0x55d1a87685d0 2024-03-30T02:47:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2ea000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8768ef0 0x55d1a877a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2eaa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2ea000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8768ef0 0x55d1a877a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51652/0 (socket says 172.21.15.67:51652) 2024-03-30T02:47:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2ea000700 1 -- 172.21.15.67:0/862853183 learned_addr learned my addr 172.21.15.67:0/862853183 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2eb400700 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8773df0 0x55d1a875df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2eaa00700 1 -- 172.21.15.67:0/862853183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8773df0 msgr2=0x55d1a875df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8773df0 0x55d1a875df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2eaa00700 1 -- 172.21.15.67:0/862853183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8768ef0 msgr2=0x55d1a877a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8768ef0 0x55d1a877a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2eaa00700 1 -- 172.21.15.67:0/862853183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1a8680ea0 con 0x55d1a87685d0 2024-03-30T02:47:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2eb400700 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8773df0 0x55d1a875df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:47:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2ea000700 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8768ef0 0x55d1a877a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d1a8785f90 tx=0x7fe2dc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f57adfcad6e173 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.961+0000 7fe2d7e00700 1 -- 172.21.15.67:0/862853183 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc017040 con 0x55d1a87685d0 2024-03-30T02:47:10.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe2d7e00700 1 -- 172.21.15.67:0/862853183 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2dc018040 con 0x55d1a87685d0 2024-03-30T02:47:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe2d7e00700 1 -- 172.21.15.67:0/862853183 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0043a0 con 0x55d1a87685d0 2024-03-30T02:47:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe30507db80 1 -- 172.21.15.67:0/862853183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 msgr2=0x55d1a87689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe30507db80 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d1a8785f90 tx=0x7fe2dc009260 comp rx=0 tx=0).stop 2024-03-30T02:47:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe30507db80 1 -- 172.21.15.67:0/862853183 shutdown_connections 2024-03-30T02:47:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe30507db80 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8773df0 0x55d1a875df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe30507db80 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87689b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe30507db80 1 --2- 172.21.15.67:0/862853183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8768ef0 0x55d1a877a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe30507db80 1 -- 172.21.15.67:0/862853183 >> 172.21.15.67:0/862853183 conn(0x55d1a86c86a0 msgr2=0x55d1a875d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe30507db80 1 -- 172.21.15.67:0/862853183 shutdown_connections 2024-03-30T02:47:10.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.962+0000 7fe30507db80 1 -- 172.21.15.67:0/862853183 wait complete. 2024-03-30T02:47:10.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.963+0000 7fe30507db80 1 -- start start 2024-03-30T02:47:10.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.963+0000 7fe30507db80 1 Processor -- start 2024-03-30T02:47:10.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.963+0000 7fe30507db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.963+0000 7fe30507db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8768ef0 0x55d1a8778ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.963+0000 7fe30507db80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a87d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.963+0000 7fe30507db80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a87d68c0 con 0x55d1a8773df0 2024-03-30T02:47:10.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.963+0000 7fe30507db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a87d6a00 con 0x55d1a8768ef0 2024-03-30T02:47:10.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.963+0000 7fe30507db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1a87d6b40 con 0x55d1a87685d0 2024-03-30T02:47:10.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.963+0000 7fe30507db80 10 client.0 _ll_register_callbacks cb 0x55d1a8660470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:10.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eaa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2ea000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8768ef0 0x55d1a8778ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eaa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87789b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43304/0 (socket says 172.21.15.67:43304) 2024-03-30T02:47:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eaa00700 1 -- 172.21.15.67:0/3847585777 learned_addr learned my addr 172.21.15.67:0/3847585777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eb400700 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a87d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eaa00700 1 -- 172.21.15.67:0/3847585777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8768ef0 msgr2=0x55d1a8778ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8768ef0 0x55d1a8778ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eaa00700 1 -- 172.21.15.67:0/3847585777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 msgr2=0x55d1a87d6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a87d6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eaa00700 1 -- 172.21.15.67:0/3847585777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1a853c3b0 con 0x55d1a87685d0 2024-03-30T02:47:10.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2eaa00700 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87789b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc012340 tx=0x7fe2dc012370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf24bcff0d5a808 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.964+0000 7fe2cbe00700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:47:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3847585777 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0213f0 con 0x55d1a87685d0 2024-03-30T02:47:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe30507db80 1 -- 172.21.15.67:0/3847585777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2d809a970 con 0x55d1a87685d0 2024-03-30T02:47:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3847585777 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7fe2dc012a20 con 0x55d1a87685d0 2024-03-30T02:47:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3847585777 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2dc012be0 con 0x55d1a87685d0 2024-03-30T02:47:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe30507db80 10 client.27643 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe30507db80 1 -- 172.21.15.67:0/3847585777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fe2dc021a00 con 0x55d1a87685d0 2024-03-30T02:47:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3847585777 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2d809a970 con 0x55d1a87685d0 2024-03-30T02:47:10.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3847585777 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe2dc030020 con 0x55d1a87685d0 2024-03-30T02:47:10.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe30507db80 10 client.27643 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:10.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe30507db80 10 client.27643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-03-30T02:47:10.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe30507db80 20 client.27643 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:10.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe30507db80 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55d1a87d82e0 0x55d1a87da700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:10.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe30507db80 4 client.27643 mds_command: new command op to 24541 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:47:10.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.965+0000 7fe30507db80 1 -- 172.21.15.67:0/3847585777 --> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d1a8509cf0 con 0x55d1a87d82e0 2024-03-30T02:47:10.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.966+0000 7fe2ea000700 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55d1a87d82e0 0x55d1a87da700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:10.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.966+0000 7fe2ea000700 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55d1a87d82e0 0x55d1a87da700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6f76f08815a3ad3 server_cookie=f8d35851934d0d6d in_seq=0 out_seq=0 2024-03-30T02:47:10.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.966+0000 7fe2cbe00700 10 client.27643 ms_handle_connect on v2:172.21.15.170:6838/2570108618 2024-03-30T02:47:10.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.966+0000 7fe2cbe00700 1 -- 172.21.15.67:0/3847585777 <== mds.2 v2:172.21.15.170:6838/2570108618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d1a8509cf0 con 0x55d1a87d82e0 2024-03-30T02:47:10.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.966+0000 7fe2cbe00700 10 client.27643 handle_command_reply: tid=0 2024-03-30T02:47:10.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.966+0000 7fe30507db80 1 client.27643 shutdown 2024-03-30T02:47:10.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.966+0000 7fe30507db80 1 -- 172.21.15.67:0/3847585777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 msgr2=0x55d1a87789b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.966+0000 7fe30507db80 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87789b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc012340 tx=0x7fe2dc012370 comp rx=0 tx=0).stop 2024-03-30T02:47:10.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe30507db80 1 -- 172.21.15.67:0/3847585777 shutdown_connections 2024-03-30T02:47:10.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe30507db80 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.170:6838/2570108618,v1:172.21.15.170:6839/2570108618] conn(0x55d1a87d82e0 0x55d1a87da700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe30507db80 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d1a8773df0 0x55d1a87d6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe30507db80 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d1a87685d0 0x55d1a87789b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe30507db80 1 --2- 172.21.15.67:0/3847585777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1a8768ef0 0x55d1a8778ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe2cbe00700 0 client.27643 ms_handle_reset on v2:172.21.15.170:6838/2570108618 2024-03-30T02:47:10.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe30507db80 1 -- 172.21.15.67:0/3847585777 >> 172.21.15.67:0/3847585777 conn(0x55d1a86c86a0 msgr2=0x55d1a876f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe30507db80 1 -- 172.21.15.67:0/3847585777 shutdown_connections 2024-03-30T02:47:10.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe30507db80 1 -- 172.21.15.67:0/3847585777 wait complete. 2024-03-30T02:47:10.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe30507db80 20 client.27643 trim_cache size 0 max 16384 2024-03-30T02:47:10.975 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:47:10.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe2f1600700 1 -- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe2d00a1310 msgr2=0x7fe2d00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe2f1600700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe2d00a1310 0x7fe2d00a37d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009ec0 tx=0x7fe2e4009920 comp rx=0 tx=0).stop 2024-03-30T02:47:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe2f1600700 1 -- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec106080 msgr2=0x7fe2ec199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.967+0000 7fe2f1600700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec106080 0x7fe2ec199990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8004760 tx=0x7fe2d8004790 comp rx=0 tx=0).stop 2024-03-30T02:47:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.968+0000 7fe2f1600700 1 -- 172.21.15.67:0/2906087227 shutdown_connections 2024-03-30T02:47:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.968+0000 7fe2f1600700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe2d00a1310 0x7fe2d00a37d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.968+0000 7fe2f1600700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2ec19dd30 0x7fe2ec19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.968+0000 7fe2f1600700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2ec100f60 0x7fe2ec199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.968+0000 7fe2f1600700 1 --2- 172.21.15.67:0/2906087227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe2ec106080 0x7fe2ec199990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:10.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.968+0000 7fe2f1600700 1 -- 172.21.15.67:0/2906087227 >> 172.21.15.67:0/2906087227 conn(0x7fe2ec0fc7c0 msgr2=0x7fe2ec0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:10.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.968+0000 7fe2f1600700 1 -- 172.21.15.67:0/2906087227 shutdown_connections 2024-03-30T02:47:10.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:10.968+0000 7fe2f1600700 1 -- 172.21.15.67:0/2906087227 wait complete. 2024-03-30T02:47:10.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-30T02:47:11.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.187+0000 7faa13e00700 1 -- 172.21.15.67:0/4145445299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 msgr2=0x7faa0c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.187+0000 7faa13e00700 1 --2- 172.21.15.67:0/4145445299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 0x7faa0c101340 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7faa00009990 tx=0x7faa00009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.187+0000 7faa13e00700 1 -- 172.21.15.67:0/4145445299 shutdown_connections 2024-03-30T02:47:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.187+0000 7faa13e00700 1 --2- 172.21.15.67:0/4145445299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 0x7faa0c101340 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.187+0000 7faa13e00700 1 --2- 172.21.15.67:0/4145445299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c106080 0x7faa0c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.187+0000 7faa13e00700 1 --2- 172.21.15.67:0/4145445299 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c101910 0x7faa0c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.187+0000 7faa13e00700 1 -- 172.21.15.67:0/4145445299 >> 172.21.15.67:0/4145445299 conn(0x7faa0c0fc7c0 msgr2=0x7faa0c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.187+0000 7faa13e00700 1 -- 172.21.15.67:0/4145445299 shutdown_connections 2024-03-30T02:47:11.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.187+0000 7faa13e00700 1 -- 172.21.15.67:0/4145445299 wait complete. 2024-03-30T02:47:11.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.188+0000 7faa13e00700 1 Processor -- start 2024-03-30T02:47:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.188+0000 7faa13e00700 1 -- start start 2024-03-30T02:47:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa13e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 0x7faa0c074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa13e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c106080 0x7faa0c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa13e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c073640 0x7faa0c073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa13e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c1a9df0 con 0x7faa0c100f60 2024-03-30T02:47:11.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa13e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c1a9f60 con 0x7faa0c073640 2024-03-30T02:47:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa13e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c1aa120 con 0x7faa0c106080 2024-03-30T02:47:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa11400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 0x7faa0c074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa11400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 0x7faa0c074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51690/0 (socket says 172.21.15.67:51690) 2024-03-30T02:47:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa11400700 1 -- 172.21.15.67:0/2956841956 learned_addr learned my addr 172.21.15.67:0/2956841956 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa10a00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c106080 0x7faa0c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa11e00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c073640 0x7faa0c073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa11400700 1 -- 172.21.15.67:0/2956841956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c106080 msgr2=0x7faa0c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa11400700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c106080 0x7faa0c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa11400700 1 -- 172.21.15.67:0/2956841956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c073640 msgr2=0x7faa0c073af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa11400700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c073640 0x7faa0c073af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.189+0000 7faa11400700 1 -- 172.21.15.67:0/2956841956 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f8009520 con 0x7faa0c100f60 2024-03-30T02:47:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.190+0000 7faa11400700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 0x7faa0c074cb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7faa00000c00 tx=0x7faa00005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.190+0000 7faa0a000700 1 -- 172.21.15.67:0/2956841956 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0001d070 con 0x7faa0c100f60 2024-03-30T02:47:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.190+0000 7faa0a000700 1 -- 172.21.15.67:0/2956841956 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa00022470 con 0x7faa0c100f60 2024-03-30T02:47:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.190+0000 7faa0a000700 1 -- 172.21.15.67:0/2956841956 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00017650 con 0x7faa0c100f60 2024-03-30T02:47:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.190+0000 7faa13e00700 1 -- 172.21.15.67:0/2956841956 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa000095f0 con 0x7faa0c100f60 2024-03-30T02:47:11.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.190+0000 7faa13e00700 1 -- 172.21.15.67:0/2956841956 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c1aa630 con 0x7faa0c100f60 2024-03-30T02:47:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.193+0000 7faa0a000700 1 -- 172.21.15.67:0/2956841956 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7faa000225e0 con 0x7faa0c100f60 2024-03-30T02:47:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.194+0000 7faa279efb80 1 Processor -- start 2024-03-30T02:47:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.194+0000 7faa279efb80 1 -- start start 2024-03-30T02:47:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa0a000700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9f40a1310 0x7fa9f40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa279efb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d57e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa279efb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57eef0 0x563f6d5898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa279efb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d58c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa279efb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d4d7990 con 0x563f6d57e5d0 2024-03-30T02:47:11.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa279efb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d4d5350 con 0x563f6d57eef0 2024-03-30T02:47:11.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa279efb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d4d0c40 con 0x563f6d589df0 2024-03-30T02:47:11.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa0a000700 1 -- 172.21.15.67:0/2956841956 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7faa000ce430 con 0x7faa0c100f60 2024-03-30T02:47:11.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa11400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d57e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa11e00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9f40a1310 0x7fa9f40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa11400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d57e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51694/0 (socket says 172.21.15.67:51694) 2024-03-30T02:47:11.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa11400700 1 -- 172.21.15.67:0/4064428010 learned_addr learned my addr 172.21.15.67:0/4064428010 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa10a00700 1 --2- 172.21.15.67:0/4064428010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57eef0 0x563f6d5898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa11e00700 1 --2- 172.21.15.67:0/4064428010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d58c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa11400700 1 -- 172.21.15.67:0/4064428010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 msgr2=0x563f6d58c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.195+0000 7faa11400700 1 --2- 172.21.15.67:0/4064428010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d58c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.196+0000 7faa11400700 1 -- 172.21.15.67:0/4064428010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57eef0 msgr2=0x563f6d5898b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.196+0000 7faa11400700 1 --2- 172.21.15.67:0/4064428010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57eef0 0x563f6d5898b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.196+0000 7faa11400700 1 -- 172.21.15.67:0/4064428010 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f6d3523b0 con 0x563f6d57e5d0 2024-03-30T02:47:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.196+0000 7faa11e00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9f40a1310 0x7fa9f40a37d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faa04009ec0 tx=0x7faa0400fc40 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.196+0000 7faa11400700 1 --2- 172.21.15.67:0/4064428010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d57e9b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faa00097790 tx=0x7faa000976a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d34587e3a81c41 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.196+0000 7fa9fe600700 1 -- 172.21.15.67:0/4064428010 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00095ea0 con 0x563f6d57e5d0 2024-03-30T02:47:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.196+0000 7fa9fe600700 1 -- 172.21.15.67:0/4064428010 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0009abf0 con 0x563f6d57e5d0 2024-03-30T02:47:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.196+0000 7fa9fe600700 1 -- 172.21.15.67:0/4064428010 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa000a3680 con 0x563f6d57e5d0 2024-03-30T02:47:11.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 -- 172.21.15.67:0/4064428010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 msgr2=0x563f6d57e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 --2- 172.21.15.67:0/4064428010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d57e9b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faa00097790 tx=0x7faa000976a0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 -- 172.21.15.67:0/4064428010 shutdown_connections 2024-03-30T02:47:11.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 --2- 172.21.15.67:0/4064428010 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d57e9b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 --2- 172.21.15.67:0/4064428010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d58c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 --2- 172.21.15.67:0/4064428010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57eef0 0x563f6d5898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 -- 172.21.15.67:0/4064428010 >> 172.21.15.67:0/4064428010 conn(0x563f6d4de6a0 msgr2=0x563f6d57aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 -- 172.21.15.67:0/4064428010 shutdown_connections 2024-03-30T02:47:11.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 -- 172.21.15.67:0/4064428010 wait complete. 2024-03-30T02:47:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.197+0000 7faa279efb80 1 -- start start 2024-03-30T02:47:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa279efb80 1 Processor -- start 2024-03-30T02:47:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa279efb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 0x563f6d58a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa279efb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d58a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa279efb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d595250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa279efb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d46cd80 con 0x563f6d57eef0 2024-03-30T02:47:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa279efb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d46ca20 con 0x563f6d57e5d0 2024-03-30T02:47:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa279efb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d3546a0 con 0x563f6d589df0 2024-03-30T02:47:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa279efb80 10 client.0 _ll_register_callbacks cb 0x563f6d476470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa10a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d58a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa10a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d58a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51708/0 (socket says 172.21.15.67:51708) 2024-03-30T02:47:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa10a00700 1 -- 172.21.15.67:0/1600585014 learned_addr learned my addr 172.21.15.67:0/1600585014 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.198+0000 7faa11400700 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 0x563f6d58a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7faa11e00700 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d595250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7faa10a00700 1 -- 172.21.15.67:0/1600585014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 msgr2=0x563f6d595250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7faa10a00700 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d595250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7faa10a00700 1 -- 172.21.15.67:0/1600585014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 msgr2=0x563f6d58a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7faa10a00700 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 0x563f6d58a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7faa10a00700 1 -- 172.21.15.67:0/1600585014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c1aa630 con 0x563f6d57eef0 2024-03-30T02:47:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7faa10a00700 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d58a710 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8012980 tx=0x7fa9f8012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a48db866165a4fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7fa9f2a00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:47:11.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7fa9f2a00700 1 -- 172.21.15.67:0/1600585014 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8017070 con 0x563f6d57eef0 2024-03-30T02:47:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7faa279efb80 1 -- 172.21.15.67:0/1600585014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f6d491380 con 0x563f6d57eef0 2024-03-30T02:47:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.199+0000 7fa9f2a00700 1 -- 172.21.15.67:0/1600585014 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7fa9f800f5b0 con 0x563f6d57eef0 2024-03-30T02:47:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7fa9f2a00700 1 -- 172.21.15.67:0/1600585014 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9f800f770 con 0x563f6d57eef0 2024-03-30T02:47:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7faa279efb80 10 client.17589 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7faa279efb80 1 -- 172.21.15.67:0/1600585014 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563f6d2feb60 con 0x563f6d57eef0 2024-03-30T02:47:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7fa9f2a00700 1 -- 172.21.15.67:0/1600585014 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f6d491380 con 0x563f6d57eef0 2024-03-30T02:47:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7fa9f2a00700 1 -- 172.21.15.67:0/1600585014 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa9f8010990 con 0x563f6d57eef0 2024-03-30T02:47:11.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7faa279efb80 10 client.17589 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7faa279efb80 10 client.17589 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:47:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7faa279efb80 20 client.17589 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7faa279efb80 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x563f6d599700 0x563f6d59bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7faa279efb80 4 client.17589 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:47:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.200+0000 7faa279efb80 1 -- 172.21.15.67:0/1600585014 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f6d4c9700 con 0x563f6d599700 2024-03-30T02:47:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.201+0000 7faa11400700 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x563f6d599700 0x563f6d59bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.201+0000 7faa11400700 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x563f6d599700 0x563f6d59bb20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4ed4c1d1a8d8dbf server_cookie=60d3660caa5590ad in_seq=0 out_seq=0 2024-03-30T02:47:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.201+0000 7fa9f2a00700 10 client.17589 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:47:11.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.201+0000 7fa9f2a00700 1 -- 172.21.15.67:0/1600585014 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563f6d4c9700 con 0x563f6d599700 2024-03-30T02:47:11.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.201+0000 7fa9f2a00700 10 client.17589 handle_command_reply: tid=0 2024-03-30T02:47:11.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 client.17589 shutdown 2024-03-30T02:47:11.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 -- 172.21.15.67:0/1600585014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 msgr2=0x563f6d58a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d58a710 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8012980 tx=0x7fa9f8012c90 comp rx=0 tx=0).stop 2024-03-30T02:47:11.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 -- 172.21.15.67:0/1600585014 shutdown_connections 2024-03-30T02:47:11.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d595250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 0x563f6d58a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x563f6d599700 0x563f6d59bb20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 --2- 172.21.15.67:0/1600585014 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d58a710 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 -- 172.21.15.67:0/1600585014 >> 172.21.15.67:0/1600585014 conn(0x563f6d4de6a0 msgr2=0x563f6d5806c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7fa9f2a00700 0 client.17589 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:47:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 -- 172.21.15.67:0/1600585014 shutdown_connections 2024-03-30T02:47:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 1 -- 172.21.15.67:0/1600585014 wait complete. 2024-03-30T02:47:11.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.202+0000 7faa279efb80 20 client.17589 trim_cache size 0 max 16384 2024-03-30T02:47:11.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.216+0000 7faa279efb80 1 Processor -- start 2024-03-30T02:47:11.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.216+0000 7faa279efb80 1 -- start start 2024-03-30T02:47:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.216+0000 7faa279efb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d58f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.216+0000 7faa279efb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d57eef0 0x563f6d58a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.216+0000 7faa279efb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d589df0 0x563f6d58a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.216+0000 7faa279efb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d57f640 con 0x563f6d57e5d0 2024-03-30T02:47:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.216+0000 7faa279efb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d595ab0 con 0x563f6d589df0 2024-03-30T02:47:11.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.216+0000 7faa279efb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d59beb0 con 0x563f6d57eef0 2024-03-30T02:47:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d58f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d58f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51712/0 (socket says 172.21.15.67:51712) 2024-03-30T02:47:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11400700 1 -- 172.21.15.67:0/3062279817 learned_addr learned my addr 172.21.15.67:0/3062279817 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11e00700 1 --2- 172.21.15.67:0/3062279817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d589df0 0x563f6d58a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa10a00700 1 --2- 172.21.15.67:0/3062279817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d57eef0 0x563f6d58a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11400700 1 -- 172.21.15.67:0/3062279817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d57eef0 msgr2=0x563f6d58a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11400700 1 --2- 172.21.15.67:0/3062279817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d57eef0 0x563f6d58a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11400700 1 -- 172.21.15.67:0/3062279817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d589df0 msgr2=0x563f6d58a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11400700 1 --2- 172.21.15.67:0/3062279817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d589df0 0x563f6d58a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11400700 1 -- 172.21.15.67:0/3062279817 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f6d496ea0 con 0x563f6d57e5d0 2024-03-30T02:47:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7faa11400700 1 --2- 172.21.15.67:0/3062279817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d58f100 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faa00097d00 tx=0x7faa0001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751a5171514a5fac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7fa9ff000700 1 -- 172.21.15.67:0/3062279817 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00095ea0 con 0x563f6d57e5d0 2024-03-30T02:47:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.217+0000 7fa9ff000700 1 -- 172.21.15.67:0/3062279817 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa000a3680 con 0x563f6d57e5d0 2024-03-30T02:47:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7fa9ff000700 1 -- 172.21.15.67:0/3062279817 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0007f870 con 0x563f6d57e5d0 2024-03-30T02:47:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7faa279efb80 1 -- 172.21.15.67:0/3062279817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 msgr2=0x563f6d58f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7faa279efb80 1 --2- 172.21.15.67:0/3062279817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d58f100 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faa00097d00 tx=0x7faa0001a040 comp rx=0 tx=0).stop 2024-03-30T02:47:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7faa279efb80 1 -- 172.21.15.67:0/3062279817 shutdown_connections 2024-03-30T02:47:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7faa279efb80 1 --2- 172.21.15.67:0/3062279817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57e5d0 0x563f6d58f100 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7faa279efb80 1 --2- 172.21.15.67:0/3062279817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d589df0 0x563f6d58a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7faa279efb80 1 --2- 172.21.15.67:0/3062279817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d57eef0 0x563f6d58a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7faa279efb80 1 -- 172.21.15.67:0/3062279817 >> 172.21.15.67:0/3062279817 conn(0x563f6d4de6a0 msgr2=0x563f6d5735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7faa279efb80 1 -- 172.21.15.67:0/3062279817 shutdown_connections 2024-03-30T02:47:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.218+0000 7faa279efb80 1 -- 172.21.15.67:0/3062279817 wait complete. 2024-03-30T02:47:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa279efb80 1 -- start start 2024-03-30T02:47:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa279efb80 1 Processor -- start 2024-03-30T02:47:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa279efb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 0x563f6d5e14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa279efb80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d5e1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa279efb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d5e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa279efb80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d5ec710 con 0x563f6d57eef0 2024-03-30T02:47:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa279efb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d5ec880 con 0x563f6d57e5d0 2024-03-30T02:47:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa279efb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f6d5eca40 con 0x563f6d589df0 2024-03-30T02:47:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa279efb80 10 client.0 _ll_register_callbacks cb 0x563f6d476470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.219+0000 7faa10a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d5e1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa11400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 0x563f6d5e14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa11e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d5e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa10a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d5e1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51720/0 (socket says 172.21.15.67:51720) 2024-03-30T02:47:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa10a00700 1 -- 172.21.15.67:0/2619026328 learned_addr learned my addr 172.21.15.67:0/2619026328 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa10a00700 1 -- 172.21.15.67:0/2619026328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 msgr2=0x563f6d5e1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa10a00700 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d5e1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa10a00700 1 -- 172.21.15.67:0/2619026328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 msgr2=0x563f6d5e14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa10a00700 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 0x563f6d5e14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa10a00700 1 -- 172.21.15.67:0/2619026328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f6d3523b0 con 0x563f6d57eef0 2024-03-30T02:47:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa10a00700 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d5e1a00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x563f6d5894a0 tx=0x7fa9f8010ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9c8ad55e4c165a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7fa9f2a00700 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-03-30T02:47:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7fa9f2a00700 1 -- 172.21.15.67:0/2619026328 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8017070 con 0x563f6d57eef0 2024-03-30T02:47:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7faa279efb80 1 -- 172.21.15.67:0/2619026328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa0009abf0 con 0x563f6d57eef0 2024-03-30T02:47:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.220+0000 7fa9f2a00700 1 -- 172.21.15.67:0/2619026328 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7fa9f8005450 con 0x563f6d57eef0 2024-03-30T02:47:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7fa9f2a00700 1 -- 172.21.15.67:0/2619026328 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9f8005610 con 0x563f6d57eef0 2024-03-30T02:47:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7faa279efb80 10 client.17595 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7faa279efb80 1 -- 172.21.15.67:0/2619026328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563f6d356cd0 con 0x563f6d57eef0 2024-03-30T02:47:11.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7fa9f2a00700 1 -- 172.21.15.67:0/2619026328 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9f8034080 con 0x563f6d57eef0 2024-03-30T02:47:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7fa9f2a00700 1 -- 172.21.15.67:0/2619026328 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa9f800f5e0 con 0x563f6d57eef0 2024-03-30T02:47:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7faa279efb80 10 client.17595 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7faa279efb80 10 client.17595 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2024-03-30T02:47:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7faa279efb80 20 client.17595 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7faa279efb80 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x563f6d5ee250 0x563f6d5f0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7faa279efb80 4 client.17595 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:47:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7faa279efb80 1 -- 172.21.15.67:0/2619026328 --> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f6d31fcf0 con 0x563f6d5ee250 2024-03-30T02:47:11.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.221+0000 7faa11400700 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x563f6d5ee250 0x563f6d5f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7fa9f2a00700 10 client.17595 ms_handle_connect on v2:172.21.15.67:6838/2974132199 2024-03-30T02:47:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa11400700 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x563f6d5ee250 0x563f6d5f0670 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d84871ce9730ff88 server_cookie=79a6143b265e10ba in_seq=0 out_seq=0 2024-03-30T02:47:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7fa9f2a00700 1 -- 172.21.15.67:0/2619026328 <== mds.1 v2:172.21.15.67:6838/2974132199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f6d31fcf0 con 0x563f6d5ee250 2024-03-30T02:47:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7fa9f2a00700 10 client.17595 handle_command_reply: tid=0 2024-03-30T02:47:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa279efb80 1 client.17595 shutdown 2024-03-30T02:47:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa279efb80 1 -- 172.21.15.67:0/2619026328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 msgr2=0x563f6d5e1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa279efb80 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d5e1a00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x563f6d5894a0 tx=0x7fa9f8010ac0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa279efb80 1 -- 172.21.15.67:0/2619026328 shutdown_connections 2024-03-30T02:47:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa279efb80 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f6d589df0 0x563f6d5e1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa279efb80 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f6d57e5d0 0x563f6d5e14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa279efb80 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.67:6838/2974132199,v1:172.21.15.67:6839/2974132199] conn(0x563f6d5ee250 0x563f6d5f0670 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa279efb80 1 --2- 172.21.15.67:0/2619026328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563f6d57eef0 0x563f6d5e1a00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7faa279efb80 1 -- 172.21.15.67:0/2619026328 >> 172.21.15.67:0/2619026328 conn(0x563f6d4de6a0 msgr2=0x563f6d58a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.222+0000 7fa9f2a00700 0 client.17595 ms_handle_reset on v2:172.21.15.67:6838/2974132199 2024-03-30T02:47:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa279efb80 1 -- 172.21.15.67:0/2619026328 shutdown_connections 2024-03-30T02:47:11.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa279efb80 1 -- 172.21.15.67:0/2619026328 wait complete. 2024-03-30T02:47:11.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa279efb80 20 client.17595 trim_cache size 0 max 16384 2024-03-30T02:47:11.231 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:47:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 -- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9f40a1310 msgr2=0x7fa9f40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9f40a1310 0x7fa9f40a37d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faa04009ec0 tx=0x7faa0400fc40 comp rx=0 tx=0).stop 2024-03-30T02:47:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 -- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 msgr2=0x7faa0c074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 0x7faa0c074cb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7faa00000c00 tx=0x7faa00005740 comp rx=0 tx=0).stop 2024-03-30T02:47:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 -- 172.21.15.67:0/2956841956 shutdown_connections 2024-03-30T02:47:11.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fa9f40a1310 0x7fa9f40a37d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faa0c100f60 0x7faa0c074cb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa0c073640 0x7faa0c073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 --2- 172.21.15.67:0/2956841956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa0c106080 0x7faa0c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.223+0000 7faa13e00700 1 -- 172.21.15.67:0/2956841956 >> 172.21.15.67:0/2956841956 conn(0x7faa0c0fc7c0 msgr2=0x7faa0c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.224+0000 7faa13e00700 1 -- 172.21.15.67:0/2956841956 shutdown_connections 2024-03-30T02:47:11.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.224+0000 7faa13e00700 1 -- 172.21.15.67:0/2956841956 wait complete. 2024-03-30T02:47:11.240 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-30T02:47:11.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.447+0000 7f6584400700 1 -- 172.21.15.67:0/1303428455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f657c0731e0 msgr2=0x7f657c0735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.447+0000 7f6584400700 1 --2- 172.21.15.67:0/1303428455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f657c0731e0 0x7f657c0735c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6570009990 tx=0x7f6570009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.447+0000 7f6584400700 1 -- 172.21.15.67:0/1303428455 shutdown_connections 2024-03-30T02:47:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.447+0000 7f6584400700 1 --2- 172.21.15.67:0/1303428455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f657c10d530 0x7f657c10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.447+0000 7f6584400700 1 --2- 172.21.15.67:0/1303428455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f657c0731e0 0x7f657c0735c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.447+0000 7f6584400700 1 --2- 172.21.15.67:0/1303428455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657c073b00 0x7f657c10cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.447+0000 7f6584400700 1 -- 172.21.15.67:0/1303428455 >> 172.21.15.67:0/1303428455 conn(0x7f657c0fc7c0 msgr2=0x7f657c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.447+0000 7f6584400700 1 -- 172.21.15.67:0/1303428455 shutdown_connections 2024-03-30T02:47:11.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.447+0000 7f6584400700 1 -- 172.21.15.67:0/1303428455 wait complete. 2024-03-30T02:47:11.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.449+0000 7f6584400700 1 Processor -- start 2024-03-30T02:47:11.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.449+0000 7f6584400700 1 -- start start 2024-03-30T02:47:11.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.449+0000 7f6584400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f657c0731e0 0x7f657c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.449+0000 7f6584400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657c073b00 0x7f657c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.449+0000 7f6584400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f657c10d530 0x7f657c19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.449+0000 7f6584400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c19e310 con 0x7f657c073b00 2024-03-30T02:47:11.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.449+0000 7f6584400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c19e480 con 0x7f657c10d530 2024-03-30T02:47:11.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.449+0000 7f6584400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c19e640 con 0x7f657c0731e0 2024-03-30T02:47:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657c073b00 0x7f657c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f657c0731e0 0x7f657c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657c073b00 0x7f657c199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51738/0 (socket says 172.21.15.67:51738) 2024-03-30T02:47:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581000700 1 -- 172.21.15.67:0/2929961280 learned_addr learned my addr 172.21.15.67:0/2929961280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6582400700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f657c10d530 0x7f657c19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581000700 1 -- 172.21.15.67:0/2929961280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f657c0731e0 msgr2=0x7f657c1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581000700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f657c0731e0 0x7f657c1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581000700 1 -- 172.21.15.67:0/2929961280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f657c10d530 msgr2=0x7f657c19dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581000700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f657c10d530 0x7f657c19dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581000700 1 -- 172.21.15.67:0/2929961280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65700095f0 con 0x7f657c073b00 2024-03-30T02:47:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581a00700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f657c0731e0 0x7f657c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.450+0000 7f6581000700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657c073b00 0x7f657c199930 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f656800e9a0 tx=0x7f656800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.451+0000 7f657a000700 1 -- 172.21.15.67:0/2929961280 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568009790 con 0x7f657c073b00 2024-03-30T02:47:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.451+0000 7f657a000700 1 -- 172.21.15.67:0/2929961280 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6568010430 con 0x7f657c073b00 2024-03-30T02:47:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.451+0000 7f657a000700 1 -- 172.21.15.67:0/2929961280 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568018680 con 0x7f657c073b00 2024-03-30T02:47:11.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.451+0000 7f6584400700 1 -- 172.21.15.67:0/2929961280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c19e920 con 0x7f657c073b00 2024-03-30T02:47:11.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.451+0000 7f6584400700 1 -- 172.21.15.67:0/2929961280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c1a6160 con 0x7f657c073b00 2024-03-30T02:47:11.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.454+0000 7f657a000700 1 -- 172.21.15.67:0/2929961280 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f65680105a0 con 0x7f657c073b00 2024-03-30T02:47:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.455+0000 7f6598017b80 1 Processor -- start 2024-03-30T02:47:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.455+0000 7f657a000700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f65640a1310 0x7f65640a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.455+0000 7f6598017b80 1 -- start start 2024-03-30T02:47:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f657a000700 1 -- 172.21.15.67:0/2929961280 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f6568014070 con 0x7f657c073b00 2024-03-30T02:47:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6582400700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f65640a1310 0x7f65640a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6598017b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6598017b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6598017b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 0x5570e7a331e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6598017b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e797e990 con 0x5570e7a25ef0 2024-03-30T02:47:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6598017b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e797c350 con 0x5570e7a30df0 2024-03-30T02:47:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6598017b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e7977c40 con 0x5570e7a255d0 2024-03-30T02:47:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6581a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6581000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6581a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a259b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43406/0 (socket says 172.21.15.67:43406) 2024-03-30T02:47:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6581a00700 1 -- 172.21.15.67:0/3011721481 learned_addr learned my addr 172.21.15.67:0/3011721481 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.456+0000 7f6582400700 1 --2- 172.21.15.67:0/3011721481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 0x5570e7a331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f6581a00700 1 -- 172.21.15.67:0/3011721481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 msgr2=0x5570e7a331e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f6581a00700 1 --2- 172.21.15.67:0/3011721481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 0x5570e7a331e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f6581a00700 1 -- 172.21.15.67:0/3011721481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 msgr2=0x5570e7a308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f6581a00700 1 --2- 172.21.15.67:0/3011721481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a308b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f6581a00700 1 -- 172.21.15.67:0/3011721481 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570e77f93b0 con 0x5570e7a255d0 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f6581000700 1 --2- 172.21.15.67:0/3011721481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f6582400700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f65640a1310 0x7f65640a37d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f657400f590 tx=0x7f6574009920 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f6581a00700 1 --2- 172.21.15.67:0/3011721481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a259b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6570005ec0 tx=0x7f6570005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49626a60930e469 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f656ea00700 1 -- 172.21.15.67:0/3011721481 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570027070 con 0x5570e7a255d0 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f656ea00700 1 -- 172.21.15.67:0/3011721481 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f657001e040 con 0x5570e7a255d0 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.457+0000 7f656ea00700 1 -- 172.21.15.67:0/3011721481 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657001d050 con 0x5570e7a255d0 2024-03-30T02:47:11.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 -- 172.21.15.67:0/3011721481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 msgr2=0x5570e7a259b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 --2- 172.21.15.67:0/3011721481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a259b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6570005ec0 tx=0x7f6570005ef0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 -- 172.21.15.67:0/3011721481 shutdown_connections 2024-03-30T02:47:11.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 --2- 172.21.15.67:0/3011721481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 0x5570e7a331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 --2- 172.21.15.67:0/3011721481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a259b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 --2- 172.21.15.67:0/3011721481 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 -- 172.21.15.67:0/3011721481 >> 172.21.15.67:0/3011721481 conn(0x5570e79856a0 msgr2=0x5570e7a21a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 -- 172.21.15.67:0/3011721481 shutdown_connections 2024-03-30T02:47:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 -- 172.21.15.67:0/3011721481 wait complete. 2024-03-30T02:47:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.458+0000 7f6598017b80 1 -- start start 2024-03-30T02:47:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.459+0000 7f6598017b80 1 Processor -- start 2024-03-30T02:47:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.459+0000 7f6598017b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a255d0 0x5570e7a30800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.459+0000 7f6598017b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a25ef0 0x5570e7a2da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.459+0000 7f6598017b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a2e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.459+0000 7f6598017b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e7913d80 con 0x5570e7a30df0 2024-03-30T02:47:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.459+0000 7f6598017b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e7913a20 con 0x5570e7a255d0 2024-03-30T02:47:11.466 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.459+0000 7f6598017b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e77fb6a0 con 0x5570e7a25ef0 2024-03-30T02:47:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.459+0000 7f6598017b80 10 client.0 _ll_register_callbacks cb 0x5570e791d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.459+0000 7f6582400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a2e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6581000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a25ef0 0x5570e7a2da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6582400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a2e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51748/0 (socket says 172.21.15.67:51748) 2024-03-30T02:47:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6582400700 1 -- 172.21.15.67:0/3701124474 learned_addr learned my addr 172.21.15.67:0/3701124474 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6581a00700 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a255d0 0x5570e7a30800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6581000700 1 -- 172.21.15.67:0/3701124474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a255d0 msgr2=0x5570e7a30800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6581000700 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a255d0 0x5570e7a30800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6581000700 1 -- 172.21.15.67:0/3701124474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 msgr2=0x5570e7a2e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6581000700 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a2e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6581000700 1 -- 172.21.15.67:0/3701124474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c1a6160 con 0x5570e7a25ef0 2024-03-30T02:47:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6582400700 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a2e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6581000700 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a25ef0 0x5570e7a2da80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f65680ca570 tx=0x7f656809a770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97be9b57669914fd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6563400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:47:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6563400700 1 -- 172.21.15.67:0/3701124474 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656809dd60 con 0x5570e7a25ef0 2024-03-30T02:47:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6563400700 1 -- 172.21.15.67:0/3701124474 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f65680a23b0 con 0x5570e7a25ef0 2024-03-30T02:47:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.460+0000 7f6598017b80 1 -- 172.21.15.67:0/3701124474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570e7938380 con 0x5570e7a25ef0 2024-03-30T02:47:11.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6563400700 1 -- 172.21.15.67:0/3701124474 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6568099420 con 0x5570e7a25ef0 2024-03-30T02:47:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6598017b80 10 client.27673 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6598017b80 1 -- 172.21.15.67:0/3701124474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f657c110660 con 0x5570e7a25ef0 2024-03-30T02:47:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6563400700 1 -- 172.21.15.67:0/3701124474 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570e7938380 con 0x5570e7a25ef0 2024-03-30T02:47:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6563400700 1 -- 172.21.15.67:0/3701124474 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6568095db0 con 0x5570e7a25ef0 2024-03-30T02:47:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6598017b80 10 client.27673 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6598017b80 10 client.27673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:47:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6598017b80 20 client.27673 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6598017b80 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570e7a40990 0x5570e7a42db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6598017b80 4 client.27673 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T02:47:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6598017b80 1 -- 172.21.15.67:0/3701124474 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570e7970700 con 0x5570e7a40990 2024-03-30T02:47:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.461+0000 7f6581a00700 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570e7a40990 0x5570e7a42db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.462+0000 7f6581a00700 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570e7a40990 0x5570e7a42db0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1005b5a38dd2ef2 server_cookie=b936815dedf74a58 in_seq=0 out_seq=0 2024-03-30T02:47:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.462+0000 7f6563400700 10 client.27673 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:47:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6563400700 1 -- 172.21.15.67:0/3701124474 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5570e7970700 con 0x5570e7a40990 2024-03-30T02:47:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6563400700 10 client.27673 handle_command_reply: tid=0 2024-03-30T02:47:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 client.27673 shutdown 2024-03-30T02:47:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 -- 172.21.15.67:0/3701124474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a25ef0 msgr2=0x5570e7a2da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a25ef0 0x5570e7a2da80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f65680ca570 tx=0x7f656809a770 comp rx=0 tx=0).stop 2024-03-30T02:47:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 -- 172.21.15.67:0/3701124474 shutdown_connections 2024-03-30T02:47:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570e7a40990 0x5570e7a42db0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a2e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6563400700 0 client.27673 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:47:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a255d0 0x5570e7a30800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 --2- 172.21.15.67:0/3701124474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a25ef0 0x5570e7a2da80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 -- 172.21.15.67:0/3701124474 >> 172.21.15.67:0/3701124474 conn(0x5570e79856a0 msgr2=0x5570e7a27670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 -- 172.21.15.67:0/3701124474 shutdown_connections 2024-03-30T02:47:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 1 -- 172.21.15.67:0/3701124474 wait complete. 2024-03-30T02:47:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.463+0000 7f6598017b80 20 client.27673 trim_cache size 0 max 16384 2024-03-30T02:47:11.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.477+0000 7f6598017b80 1 Processor -- start 2024-03-30T02:47:11.476 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.477+0000 7f6598017b80 1 -- start start 2024-03-30T02:47:11.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.477+0000 7f6598017b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a37a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.478 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.477+0000 7f6598017b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a30330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.477+0000 7f6598017b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 0x5570e7a2c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.477+0000 7f6598017b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e7a37f50 con 0x5570e7a25ef0 2024-03-30T02:47:11.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.477+0000 7f6598017b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e7a2c010 con 0x5570e7a30df0 2024-03-30T02:47:11.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.477+0000 7f6598017b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e7a253c0 con 0x5570e7a255d0 2024-03-30T02:47:11.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a30330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581000700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a30330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51750/0 (socket says 172.21.15.67:51750) 2024-03-30T02:47:11.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581000700 1 -- 172.21.15.67:0/845844230 learned_addr learned my addr 172.21.15.67:0/845844230 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.479 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6582400700 1 --2- 172.21.15.67:0/845844230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 0x5570e7a2c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581a00700 1 --2- 172.21.15.67:0/845844230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a37a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581000700 1 -- 172.21.15.67:0/845844230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 msgr2=0x5570e7a37a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581000700 1 --2- 172.21.15.67:0/845844230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a37a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581000700 1 -- 172.21.15.67:0/845844230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 msgr2=0x5570e7a2c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581000700 1 --2- 172.21.15.67:0/845844230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 0x5570e7a2c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581000700 1 -- 172.21.15.67:0/845844230 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570e77a5b60 con 0x5570e7a25ef0 2024-03-30T02:47:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.478+0000 7f6581000700 1 --2- 172.21.15.67:0/845844230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a30330 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f656800c510 tx=0x7f6568010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1690af608f916c31 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f656f400700 1 -- 172.21.15.67:0/845844230 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65680a23b0 con 0x5570e7a25ef0 2024-03-30T02:47:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f656f400700 1 -- 172.21.15.67:0/845844230 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f656809dd60 con 0x5570e7a25ef0 2024-03-30T02:47:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f656f400700 1 -- 172.21.15.67:0/845844230 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65680a2160 con 0x5570e7a25ef0 2024-03-30T02:47:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f6598017b80 1 -- 172.21.15.67:0/845844230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 msgr2=0x5570e7a30330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f6598017b80 1 --2- 172.21.15.67:0/845844230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a30330 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f656800c510 tx=0x7f6568010a90 comp rx=0 tx=0).stop 2024-03-30T02:47:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f6598017b80 1 -- 172.21.15.67:0/845844230 shutdown_connections 2024-03-30T02:47:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f6598017b80 1 --2- 172.21.15.67:0/845844230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a30df0 0x5570e7a2c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f6598017b80 1 --2- 172.21.15.67:0/845844230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a37a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f6598017b80 1 --2- 172.21.15.67:0/845844230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a25ef0 0x5570e7a30330 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f6598017b80 1 -- 172.21.15.67:0/845844230 >> 172.21.15.67:0/845844230 conn(0x5570e79856a0 msgr2=0x5570e7a1af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f6598017b80 1 -- 172.21.15.67:0/845844230 shutdown_connections 2024-03-30T02:47:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.479+0000 7f6598017b80 1 -- 172.21.15.67:0/845844230 wait complete. 2024-03-30T02:47:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.480+0000 7f6598017b80 1 -- start start 2024-03-30T02:47:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.480+0000 7f6598017b80 1 Processor -- start 2024-03-30T02:47:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.480+0000 7f6598017b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a35c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.480+0000 7f6598017b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a25ef0 0x5570e7a36140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.480+0000 7f6598017b80 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a8d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.480+0000 7f6598017b80 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e7a8dcd0 con 0x5570e7a30df0 2024-03-30T02:47:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.480+0000 7f6598017b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e7a8de40 con 0x5570e7a25ef0 2024-03-30T02:47:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.480+0000 7f6598017b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570e7a8e000 con 0x5570e7a255d0 2024-03-30T02:47:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.480+0000 7f6598017b80 10 client.0 _ll_register_callbacks cb 0x5570e791d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T02:47:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a35c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6582400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a8d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a35c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43432/0 (socket says 172.21.15.67:43432) 2024-03-30T02:47:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581a00700 1 -- 172.21.15.67:0/3584459077 learned_addr learned my addr 172.21.15.67:0/3584459077 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581000700 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a25ef0 0x5570e7a36140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581a00700 1 -- 172.21.15.67:0/3584459077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a25ef0 msgr2=0x5570e7a36140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581a00700 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a25ef0 0x5570e7a36140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581a00700 1 -- 172.21.15.67:0/3584459077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 msgr2=0x5570e7a8d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T02:47:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581a00700 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a8d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581a00700 1 -- 172.21.15.67:0/3584459077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c110660 con 0x5570e7a255d0 2024-03-30T02:47:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6582400700 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a8d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6581a00700 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a35c00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6570005d10 tx=0x7f6570009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5ca99e679896de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.481+0000 7f6563400700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-03-30T02:47:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6563400700 1 -- 172.21.15.67:0/3584459077 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570031040 con 0x5570e7a255d0 2024-03-30T02:47:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6598017b80 1 -- 172.21.15.67:0/3584459077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f657001e040 con 0x5570e7a255d0 2024-03-30T02:47:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6563400700 1 -- 172.21.15.67:0/3584459077 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f6570032410 con 0x5570e7a255d0 2024-03-30T02:47:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6563400700 1 -- 172.21.15.67:0/3584459077 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f657001d040 con 0x5570e7a255d0 2024-03-30T02:47:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6598017b80 10 client.27679 fetch_fsmap learned FSMap version 41 2024-03-30T02:47:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6598017b80 1 -- 172.21.15.67:0/3584459077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5570e793dea0 con 0x5570e7a255d0 2024-03-30T02:47:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6563400700 1 -- 172.21.15.67:0/3584459077 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f657001e040 con 0x5570e7a255d0 2024-03-30T02:47:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6563400700 1 -- 172.21.15.67:0/3584459077 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 41) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6570038020 con 0x5570e7a255d0 2024-03-30T02:47:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6598017b80 10 client.27679 fetch_fsmap finished waiting for FSMap version 41 2024-03-30T02:47:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6598017b80 10 client.27679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-30T02:47:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6598017b80 20 client.27679 populate_metadata read hostname 'smithi067' 2024-03-30T02:47:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.482+0000 7f6598017b80 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570e7a954c0 0x5570e7a978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.483+0000 7f6598017b80 4 client.27679 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-30T02:47:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.483+0000 7f6598017b80 1 -- 172.21.15.67:0/3584459077 --> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5570e77c6cf0 con 0x5570e7a954c0 2024-03-30T02:47:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.483+0000 7f6581000700 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570e7a954c0 0x5570e7a978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.483+0000 7f6581000700 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570e7a954c0 0x5570e7a978e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=527db159aa176b95 server_cookie=67222b21cbf44e37 in_seq=0 out_seq=0 2024-03-30T02:47:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.483+0000 7f6563400700 10 client.27679 ms_handle_connect on v2:172.21.15.82:6838/604601206 2024-03-30T02:47:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.483+0000 7f6563400700 1 -- 172.21.15.67:0/3584459077 <== mds.0 v2:172.21.15.82:6838/604601206 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5570e77c6cf0 con 0x5570e7a954c0 2024-03-30T02:47:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6563400700 10 client.27679 handle_command_reply: tid=0 2024-03-30T02:47:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 client.27679 shutdown 2024-03-30T02:47:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 -- 172.21.15.67:0/3584459077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 msgr2=0x5570e7a35c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a35c00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6570005d10 tx=0x7f6570009380 comp rx=0 tx=0).stop 2024-03-30T02:47:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 -- 172.21.15.67:0/3584459077 shutdown_connections 2024-03-30T02:47:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.82:6838/604601206,v1:172.21.15.82:6839/604601206] conn(0x5570e7a954c0 0x5570e7a978e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5570e7a30df0 0x5570e7a8d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5570e7a255d0 0x5570e7a35c00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6563400700 0 client.27679 ms_handle_reset on v2:172.21.15.82:6838/604601206 2024-03-30T02:47:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 --2- 172.21.15.67:0/3584459077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5570e7a25ef0 0x5570e7a36140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 -- 172.21.15.67:0/3584459077 >> 172.21.15.67:0/3584459077 conn(0x5570e79856a0 msgr2=0x5570e7a2c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 -- 172.21.15.67:0/3584459077 shutdown_connections 2024-03-30T02:47:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 1 -- 172.21.15.67:0/3584459077 wait complete. 2024-03-30T02:47:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.484+0000 7f6598017b80 20 client.27679 trim_cache size 0 max 16384 2024-03-30T02:47:11.494 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-03-30T02:47:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 -- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f65640a1310 msgr2=0x7f65640a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f65640a1310 0x7f65640a37d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f657400f590 tx=0x7f6574009920 comp rx=0 tx=0).stop 2024-03-30T02:47:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 -- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657c073b00 msgr2=0x7f657c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657c073b00 0x7f657c199930 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f656800e9a0 tx=0x7f656800ecb0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 -- 172.21.15.67:0/2929961280 shutdown_connections 2024-03-30T02:47:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f65640a1310 0x7f65640a37d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f657c10d530 0x7f657c19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f657c0731e0 0x7f657c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 --2- 172.21.15.67:0/2929961280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f657c073b00 0x7f657c199930 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:11.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 -- 172.21.15.67:0/2929961280 >> 172.21.15.67:0/2929961280 conn(0x7f657c0fc7c0 msgr2=0x7f657c0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:11.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 -- 172.21.15.67:0/2929961280 shutdown_connections 2024-03-30T02:47:11.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:11.485+0000 7f6584400700 1 -- 172.21.15.67:0/2929961280 wait complete. 2024-03-30T02:47:12.503 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-30T02:47:12.503 INFO:tasks.fwd_scrub:done joining 2024-03-30T02:47:12.504 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-30T02:47:12.516 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:47:12.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.728+0000 7fd6a0c00700 1 -- 172.21.15.67:0/3775481557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c105f80 msgr2=0x7fd69c10db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:12.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.728+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/3775481557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c105f80 0x7fd69c10db00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd69400b0a0 tx=0x7fd69400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:12.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.729+0000 7fd6a0c00700 1 -- 172.21.15.67:0/3775481557 shutdown_connections 2024-03-30T02:47:12.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.729+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/3775481557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c105f80 0x7fd69c10db00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.729+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/3775481557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd69c100e60 0x7fd69c101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.729+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/3775481557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd69c101810 0x7fd69c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.729+0000 7fd6a0c00700 1 -- 172.21.15.67:0/3775481557 >> 172.21.15.67:0/3775481557 conn(0x7fd69c0fc700 msgr2=0x7fd69c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.729+0000 7fd6a0c00700 1 -- 172.21.15.67:0/3775481557 shutdown_connections 2024-03-30T02:47:12.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.729+0000 7fd6a0c00700 1 -- 172.21.15.67:0/3775481557 wait complete. 2024-03-30T02:47:12.731 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.730+0000 7fd6a0c00700 1 Processor -- start 2024-03-30T02:47:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.730+0000 7fd6a0c00700 1 -- start start 2024-03-30T02:47:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.730+0000 7fd6a0c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c100e60 0x7fd69c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.730+0000 7fd6a0c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd69c105f80 0x7fd69c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.730+0000 7fd6a0c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd69c073b30 0x7fd69c1a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.730+0000 7fd6a0c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1a5a50 con 0x7fd69c100e60 2024-03-30T02:47:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.730+0000 7fd6a0c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1a5bc0 con 0x7fd69c105f80 2024-03-30T02:47:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.730+0000 7fd6a0c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1a5d80 con 0x7fd69c073b30 2024-03-30T02:47:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c100e60 0x7fd69c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:12.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c100e60 0x7fd69c0730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51782/0 (socket says 172.21.15.67:51782) 2024-03-30T02:47:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699e00700 1 -- 172.21.15.67:0/2686774440 learned_addr learned my addr 172.21.15.67:0/2686774440 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699400700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd69c105f80 0x7fd69c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd69a800700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd69c073b30 0x7fd69c1a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699e00700 1 -- 172.21.15.67:0/2686774440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd69c073b30 msgr2=0x7fd69c1a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699e00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd69c073b30 0x7fd69c1a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699e00700 1 -- 172.21.15.67:0/2686774440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd69c105f80 msgr2=0x7fd69c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699e00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd69c105f80 0x7fd69c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:12.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699e00700 1 -- 172.21.15.67:0/2686774440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd694009d00 con 0x7fd69c100e60 2024-03-30T02:47:12.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.731+0000 7fd699e00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c100e60 0x7fd69c0730b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00e9a0 tx=0x7fd68c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:12.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.732+0000 7fd692a00700 1 -- 172.21.15.67:0/2686774440 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c009790 con 0x7fd69c100e60 2024-03-30T02:47:12.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.732+0000 7fd692a00700 1 -- 172.21.15.67:0/2686774440 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd68c010430 con 0x7fd69c100e60 2024-03-30T02:47:12.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.732+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c1a6060 con 0x7fd69c100e60 2024-03-30T02:47:12.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.732+0000 7fd692a00700 1 -- 172.21.15.67:0/2686774440 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c018680 con 0x7fd69c100e60 2024-03-30T02:47:12.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.732+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c1a6570 con 0x7fd69c100e60 2024-03-30T02:47:12.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.733+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c109250 con 0x7fd69c100e60 2024-03-30T02:47:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.735+0000 7fd692a00700 1 -- 172.21.15.67:0/2686774440 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fd68c0105a0 con 0x7fd69c100e60 2024-03-30T02:47:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.736+0000 7fd692a00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd6800a1290 0x7fd6800a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:12.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.736+0000 7fd692a00700 1 -- 172.21.15.67:0/2686774440 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7fd68c014070 con 0x7fd69c100e60 2024-03-30T02:47:12.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.736+0000 7fd699400700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd6800a1290 0x7fd6800a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:12.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.737+0000 7fd699400700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd6800a1290 0x7fd6800a3750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd69c074790 tx=0x7fd6880073d0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:12.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.739+0000 7fd692a00700 1 -- 172.21.15.67:0/2686774440 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fd68c0964c0 con 0x7fd69c100e60 2024-03-30T02:47:12.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:12 smithi082 ceph-mon[79176]: pgmap v314: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-30T02:47:12.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:12 smithi067 ceph-mon[37808]: pgmap v314: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-30T02:47:12.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:12 smithi170 ceph-mon[78798]: pgmap v314: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-03-30T02:47:12.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.997+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd69c102330 con 0x7fd69c100e60 2024-03-30T02:47:12.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:12.999+0000 7fd692a00700 1 -- 172.21.15.67:0/2686774440 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+25234 (secure 0 0 0) 0x7fd68c026090 con 0x7fd69c100e60 2024-03-30T02:47:12.998 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:47:12.998 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":95,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:47:06.468354+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/759150839":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/3018284010":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/3801648476":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/4226270132":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1220244664":"2024-03-31T02:37:02.362975+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/3801648476":"2024-03-31T02:37:02.362975+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2449045224":"2024-03-31T02:37:02.362975+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:47:13.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd6800a1290 msgr2=0x7fd6800a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd6800a1290 0x7fd6800a3750 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd69c074790 tx=0x7fd6880073d0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c100e60 msgr2=0x7fd69c0730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c100e60 0x7fd69c0730b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00e9a0 tx=0x7fd68c00ecb0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 shutdown_connections 2024-03-30T02:47:13.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fd6800a1290 0x7fd6800a3750 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd69c100e60 0x7fd69c0730b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd69c073b30 0x7fd69c1a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 --2- 172.21.15.67:0/2686774440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd69c105f80 0x7fd69c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 >> 172.21.15.67:0/2686774440 conn(0x7fd69c0fc700 msgr2=0x7fd69c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:13.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 shutdown_connections 2024-03-30T02:47:13.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.001+0000 7fd6a0c00700 1 -- 172.21.15.67:0/2686774440 wait complete. 2024-03-30T02:47:13.019 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:47:13.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:47:13.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.222+0000 7f8efee00700 1 -- 172.21.15.67:0/3805160382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef8106090 msgr2=0x7f8ef810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.222+0000 7f8efee00700 1 --2- 172.21.15.67:0/3805160382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef8106090 0x7f8ef810dc10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000b0a0 tx=0x7f8ef000b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.222+0000 7f8efee00700 1 -- 172.21.15.67:0/3805160382 shutdown_connections 2024-03-30T02:47:13.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.222+0000 7f8efee00700 1 --2- 172.21.15.67:0/3805160382 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef8100f70 0x7f8ef8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.222+0000 7f8efee00700 1 --2- 172.21.15.67:0/3805160382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef8106090 0x7f8ef810dc10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.222+0000 7f8efee00700 1 --2- 172.21.15.67:0/3805160382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef8101920 0x7f8ef8105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.223+0000 7f8efee00700 1 -- 172.21.15.67:0/3805160382 >> 172.21.15.67:0/3805160382 conn(0x7f8ef80fc7f0 msgr2=0x7f8ef80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:13.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.223+0000 7f8efee00700 1 -- 172.21.15.67:0/3805160382 shutdown_connections 2024-03-30T02:47:13.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.223+0000 7f8efee00700 1 -- 172.21.15.67:0/3805160382 wait complete. 2024-03-30T02:47:13.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8efee00700 1 Processor -- start 2024-03-30T02:47:13.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8efee00700 1 -- start start 2024-03-30T02:47:13.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8efee00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef8100f70 0x7f8ef80730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:13.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8efee00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef8106090 0x7f8ef80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:13.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8efee00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef8073b30 0x7f8ef81a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:13.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8efee00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef81a5870 con 0x7f8ef8106090 2024-03-30T02:47:13.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8efee00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef81a59e0 con 0x7f8ef8073b30 2024-03-30T02:47:13.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8efee00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef81a5ba0 con 0x7f8ef8100f70 2024-03-30T02:47:13.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8ef7e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef8100f70 0x7f8ef80730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:13.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8ef7400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef8106090 0x7f8ef80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:13.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8ef7e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef8100f70 0x7f8ef80730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:43466/0 (socket says 172.21.15.67:43466) 2024-03-30T02:47:13.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.224+0000 7f8ef7e00700 1 -- 172.21.15.67:0/2385437473 learned_addr learned my addr 172.21.15.67:0/2385437473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:13.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8efce00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef8073b30 0x7f8ef81a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:13.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef7400700 1 -- 172.21.15.67:0/2385437473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef8100f70 msgr2=0x7f8ef80730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef7400700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef8100f70 0x7f8ef80730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef7400700 1 -- 172.21.15.67:0/2385437473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef8073b30 msgr2=0x7f8ef81a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef7400700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef8073b30 0x7f8ef81a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef7400700 1 -- 172.21.15.67:0/2385437473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef0009d00 con 0x7f8ef8106090 2024-03-30T02:47:13.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef7e00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef8100f70 0x7f8ef80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:13.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef7400700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef8106090 0x7f8ef80735f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400e9c0 tx=0x7f8ee400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:13.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef4c00700 1 -- 172.21.15.67:0/2385437473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400c9c0 con 0x7f8ef8106090 2024-03-30T02:47:13.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef4c00700 1 -- 172.21.15.67:0/2385437473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ee4010430 con 0x7f8ef8106090 2024-03-30T02:47:13.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef81a5e80 con 0x7f8ef8106090 2024-03-30T02:47:13.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8ef4c00700 1 -- 172.21.15.67:0/2385437473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4018680 con 0x7f8ef8106090 2024-03-30T02:47:13.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.225+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef81a6300 con 0x7f8ef8106090 2024-03-30T02:47:13.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.227+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef8109360 con 0x7f8ef8106090 2024-03-30T02:47:13.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.229+0000 7f8ef4c00700 1 -- 172.21.15.67:0/2385437473 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f8ee400cb20 con 0x7f8ef8106090 2024-03-30T02:47:13.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.231+0000 7f8ef4c00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8edc0a1290 0x7f8edc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:13.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.231+0000 7f8ef4c00700 1 -- 172.21.15.67:0/2385437473 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f8ee4014070 con 0x7f8ef8106090 2024-03-30T02:47:13.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.231+0000 7f8ef7e00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8edc0a1290 0x7f8edc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:13.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.232+0000 7f8ef7e00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8edc0a1290 0x7f8edc0a3750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8011550 tx=0x7f8ee8009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:13.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.234+0000 7f8ef4c00700 1 -- 172.21.15.67:0/2385437473 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f8ee40963f0 con 0x7f8ef8106090 2024-03-30T02:47:13.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.457+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ef8074590 con 0x7f8ef8106090 2024-03-30T02:47:13.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.457+0000 7f8ef4c00700 1 -- 172.21.15.67:0/2385437473 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ee4099a10 con 0x7f8ef8106090 2024-03-30T02:47:13.456 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:47:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8edc0a1290 msgr2=0x7f8edc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8edc0a1290 0x7f8edc0a3750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8011550 tx=0x7f8ee8009260 comp rx=0 tx=0).stop 2024-03-30T02:47:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef8106090 msgr2=0x7f8ef80735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef8106090 0x7f8ef80735f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400e9c0 tx=0x7f8ee400ecd0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 shutdown_connections 2024-03-30T02:47:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f8edc0a1290 0x7f8edc0a3750 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef8073b30 0x7f8ef81a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef8100f70 0x7f8ef80730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 --2- 172.21.15.67:0/2385437473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ef8106090 0x7f8ef80735f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 >> 172.21.15.67:0/2385437473 conn(0x7f8ef80fc7f0 msgr2=0x7f8ef80feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:13.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 shutdown_connections 2024-03-30T02:47:13.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.460+0000 7f8efee00700 1 -- 172.21.15.67:0/2385437473 wait complete. 2024-03-30T02:47:13.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:47:13.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.683+0000 7f06cb600700 1 -- 172.21.15.67:0/983547199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4106090 msgr2=0x7f06c410dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.683+0000 7f06cb600700 1 --2- 172.21.15.67:0/983547199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4106090 0x7f06c410dc10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00b0a0 tx=0x7f06bc00b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.683+0000 7f06cb600700 1 -- 172.21.15.67:0/983547199 shutdown_connections 2024-03-30T02:47:13.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.683+0000 7f06cb600700 1 --2- 172.21.15.67:0/983547199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4106090 0x7f06c410dc10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.683+0000 7f06cb600700 1 --2- 172.21.15.67:0/983547199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06c4100f70 0x7f06c4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.683+0000 7f06cb600700 1 --2- 172.21.15.67:0/983547199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06c4101920 0x7f06c4105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.683+0000 7f06cb600700 1 -- 172.21.15.67:0/983547199 >> 172.21.15.67:0/983547199 conn(0x7f06c40fc7f0 msgr2=0x7f06c40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:13.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.683+0000 7f06cb600700 1 -- 172.21.15.67:0/983547199 shutdown_connections 2024-03-30T02:47:13.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.683+0000 7f06cb600700 1 -- 172.21.15.67:0/983547199 wait complete. 2024-03-30T02:47:13.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.684+0000 7f06cb600700 1 Processor -- start 2024-03-30T02:47:13.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.684+0000 7f06cb600700 1 -- start start 2024-03-30T02:47:13.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06cb600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4100f70 0x7f06c4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:13.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06cb600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06c4106090 0x7f06c4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:13.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06cb600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06c419de00 0x7f06c419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:13.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06cb600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c419e880 con 0x7f06c4100f70 2024-03-30T02:47:13.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06cb600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c419e9f0 con 0x7f06c419de00 2024-03-30T02:47:13.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06cb600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c419ebb0 con 0x7f06c4106090 2024-03-30T02:47:13.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c8c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4100f70 0x7f06c4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:13.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c8c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4100f70 0x7f06c4199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51818/0 (socket says 172.21.15.67:51818) 2024-03-30T02:47:13.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c8c00700 1 -- 172.21.15.67:0/3614659118 learned_addr learned my addr 172.21.15.67:0/3614659118 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:13.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c3e00700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06c4106090 0x7f06c4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:13.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c9600700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06c419de00 0x7f06c419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:13.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c8c00700 1 -- 172.21.15.67:0/3614659118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06c4106090 msgr2=0x7f06c4199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c8c00700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06c4106090 0x7f06c4199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c8c00700 1 -- 172.21.15.67:0/3614659118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06c419de00 msgr2=0x7f06c419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c8c00700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06c419de00 0x7f06c419e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.685+0000 7f06c8c00700 1 -- 172.21.15.67:0/3614659118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc009d00 con 0x7f06c4100f70 2024-03-30T02:47:13.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.686+0000 7f06c8c00700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4100f70 0x7f06c4199520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f06b800d950 tx=0x7f06b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:13.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.686+0000 7f06c1600700 1 -- 172.21.15.67:0/3614659118 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800f890 con 0x7f06c4100f70 2024-03-30T02:47:13.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.686+0000 7f06c1600700 1 -- 172.21.15.67:0/3614659118 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06b8010460 con 0x7f06c4100f70 2024-03-30T02:47:13.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.686+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c41a6110 con 0x7f06c4100f70 2024-03-30T02:47:13.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.686+0000 7f06c1600700 1 -- 172.21.15.67:0/3614659118 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800e610 con 0x7f06c4100f70 2024-03-30T02:47:13.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.686+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c41a6640 con 0x7f06c4100f70 2024-03-30T02:47:13.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.687+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c4102440 con 0x7f06c4100f70 2024-03-30T02:47:13.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.689+0000 7f06c1600700 1 -- 172.21.15.67:0/3614659118 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f06b80108d0 con 0x7f06c4100f70 2024-03-30T02:47:13.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.691+0000 7f06c1600700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f06ac0a1310 0x7f06ac0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:13.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.691+0000 7f06c1600700 1 -- 172.21.15.67:0/3614659118 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f06b80cce70 con 0x7f06c4100f70 2024-03-30T02:47:13.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.691+0000 7f06c3e00700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f06ac0a1310 0x7f06ac0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:13.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.692+0000 7f06c3e00700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f06ac0a1310 0x7f06ac0a37d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f06c419aa70 tx=0x7f06b00073d0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:13.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.696+0000 7f06c1600700 1 -- 172.21.15.67:0/3614659118 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f06b8095e60 con 0x7f06c4100f70 2024-03-30T02:47:13.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:13 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2686774440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:47:13.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:13 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2385437473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:47:13.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:13 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2686774440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:47:13.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:13 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2385437473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:47:13.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.914+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06c419a820 con 0x7f06c4100f70 2024-03-30T02:47:13.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.914+0000 7f06c1600700 1 -- 172.21.15.67:0/3614659118 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06b8016020 con 0x7f06c4100f70 2024-03-30T02:47:13.913 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:47:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f06ac0a1310 msgr2=0x7f06ac0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f06ac0a1310 0x7f06ac0a37d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f06c419aa70 tx=0x7f06b00073d0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4100f70 msgr2=0x7f06c4199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4100f70 0x7f06c4199520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f06b800d950 tx=0x7f06b800dc60 comp rx=0 tx=0).stop 2024-03-30T02:47:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 shutdown_connections 2024-03-30T02:47:13.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f06ac0a1310 0x7f06ac0a37d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f06c4100f70 0x7f06c4199520 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06c419de00 0x7f06c419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 --2- 172.21.15.67:0/3614659118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06c4106090 0x7f06c4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 >> 172.21.15.67:0/3614659118 conn(0x7f06c40fc7f0 msgr2=0x7f06c40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 shutdown_connections 2024-03-30T02:47:13.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:13.917+0000 7f06cb600700 1 -- 172.21.15.67:0/3614659118 wait complete. 2024-03-30T02:47:13.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:47:13.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:13 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2686774440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:47:13.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:13 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2385437473' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:47:14.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.138+0000 7f57bfa00700 1 -- 172.21.15.67:0/3302502163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 msgr2=0x7f57b8101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.138+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3302502163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 0x7f57b8101340 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0098f0 tx=0x7f57ac009c00 comp rx=0 tx=0).stop 2024-03-30T02:47:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.138+0000 7f57bfa00700 1 -- 172.21.15.67:0/3302502163 shutdown_connections 2024-03-30T02:47:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.138+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3302502163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57b8106080 0x7f57b810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.138+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3302502163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 0x7f57b8101340 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.138+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3302502163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57b8101910 0x7f57b8105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.138+0000 7f57bfa00700 1 -- 172.21.15.67:0/3302502163 >> 172.21.15.67:0/3302502163 conn(0x7f57b80fc7c0 msgr2=0x7f57b80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.138+0000 7f57bfa00700 1 -- 172.21.15.67:0/3302502163 shutdown_connections 2024-03-30T02:47:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.138+0000 7f57bfa00700 1 -- 172.21.15.67:0/3302502163 wait complete. 2024-03-30T02:47:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.139+0000 7f57bfa00700 1 Processor -- start 2024-03-30T02:47:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.140+0000 7f57bfa00700 1 -- start start 2024-03-30T02:47:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.140+0000 7f57bfa00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 0x7f57b8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.140+0000 7f57bfa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57b8106080 0x7f57b81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.140+0000 7f57bfa00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57b819dcd0 0x7f57b819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.140+0000 7f57bfa00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b819e750 con 0x7f57b8106080 2024-03-30T02:47:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.140+0000 7f57bfa00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b819e8c0 con 0x7f57b8100f60 2024-03-30T02:47:14.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.140+0000 7f57bfa00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b819ea80 con 0x7f57b819dcd0 2024-03-30T02:47:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.140+0000 7f57bd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 0x7f57b8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57bd000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 0x7f57b8199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:44870/0 (socket says 172.21.15.67:44870) 2024-03-30T02:47:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57bd000700 1 -- 172.21.15.67:0/3720773015 learned_addr learned my addr 172.21.15.67:0/3720773015 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57b7e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57b8106080 0x7f57b81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57bda00700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57b819dcd0 0x7f57b819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57bd000700 1 -- 172.21.15.67:0/3720773015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57b819dcd0 msgr2=0x7f57b819e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57bd000700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57b819dcd0 0x7f57b819e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57bd000700 1 -- 172.21.15.67:0/3720773015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57b8106080 msgr2=0x7f57b81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57bd000700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57b8106080 0x7f57b81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57bd000700 1 -- 172.21.15.67:0/3720773015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a40095f0 con 0x7f57b8100f60 2024-03-30T02:47:14.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57bd000700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 0x7f57b8199490 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00fa00 tx=0x7f57ac00fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:14.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.141+0000 7f57b5600700 1 -- 172.21.15.67:0/3720773015 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac01c070 con 0x7f57b8100f60 2024-03-30T02:47:14.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.142+0000 7f57b5600700 1 -- 172.21.15.67:0/3720773015 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57ac005040 con 0x7f57b8100f60 2024-03-30T02:47:14.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.142+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac009520 con 0x7f57b8100f60 2024-03-30T02:47:14.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.142+0000 7f57b5600700 1 -- 172.21.15.67:0/3720773015 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac004a90 con 0x7f57b8100f60 2024-03-30T02:47:14.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.142+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b81a61c0 con 0x7f57b8100f60 2024-03-30T02:47:14.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.143+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b8102430 con 0x7f57b8100f60 2024-03-30T02:47:14.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.145+0000 7f57b5600700 1 -- 172.21.15.67:0/3720773015 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f57ac0051b0 con 0x7f57b8100f60 2024-03-30T02:47:14.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.146+0000 7f57b5600700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f57a00a1290 0x7f57a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:14.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.146+0000 7f57b5600700 1 -- 172.21.15.67:0/3720773015 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f57ac0ce910 con 0x7f57b8100f60 2024-03-30T02:47:14.145 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.146+0000 7f57b7e00700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f57a00a1290 0x7f57a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.147+0000 7f57b7e00700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f57a00a1290 0x7f57a00a3750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57b819a9e0 tx=0x7f57a4009310 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.149+0000 7f57b5600700 1 -- 172.21.15.67:0/3720773015 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f57ac097990 con 0x7f57b8100f60 2024-03-30T02:47:14.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.364+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57b819a800 con 0x7f57b8100f60 2024-03-30T02:47:14.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.365+0000 7f57b5600700 1 -- 172.21.15.67:0/3720773015 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57ac09afb0 con 0x7f57b8100f60 2024-03-30T02:47:14.364 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:47:14.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f57a00a1290 msgr2=0x7f57a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f57a00a1290 0x7f57a00a3750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57b819a9e0 tx=0x7f57a4009310 comp rx=0 tx=0).stop 2024-03-30T02:47:14.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 msgr2=0x7f57b8199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 0x7f57b8199490 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00fa00 tx=0x7f57ac00fa30 comp rx=0 tx=0).stop 2024-03-30T02:47:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 shutdown_connections 2024-03-30T02:47:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f57a00a1290 0x7f57a00a3750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57b819dcd0 0x7f57b819e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57b8100f60 0x7f57b8199490 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 --2- 172.21.15.67:0/3720773015 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57b8106080 0x7f57b81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 >> 172.21.15.67:0/3720773015 conn(0x7f57b80fc7c0 msgr2=0x7f57b80fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:14.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 shutdown_connections 2024-03-30T02:47:14.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.367+0000 7f57bfa00700 1 -- 172.21.15.67:0/3720773015 wait complete. 2024-03-30T02:47:14.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:47:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.590+0000 7f5e65200700 1 -- 172.21.15.67:0/2250865314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e60106080 msgr2=0x7f5e6010dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.590+0000 7f5e65200700 1 --2- 172.21.15.67:0/2250865314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e60106080 0x7f5e6010dc00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009f90 tx=0x7f5e5800b330 comp rx=0 tx=0).stop 2024-03-30T02:47:14.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.591+0000 7f5e65200700 1 -- 172.21.15.67:0/2250865314 shutdown_connections 2024-03-30T02:47:14.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.591+0000 7f5e65200700 1 --2- 172.21.15.67:0/2250865314 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60100f60 0x7f5e60101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.591+0000 7f5e65200700 1 --2- 172.21.15.67:0/2250865314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e60106080 0x7f5e6010dc00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.591+0000 7f5e65200700 1 --2- 172.21.15.67:0/2250865314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e60101910 0x7f5e60105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.591+0000 7f5e65200700 1 -- 172.21.15.67:0/2250865314 >> 172.21.15.67:0/2250865314 conn(0x7f5e600fc7c0 msgr2=0x7f5e600febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:14.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.591+0000 7f5e65200700 1 -- 172.21.15.67:0/2250865314 shutdown_connections 2024-03-30T02:47:14.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.591+0000 7f5e65200700 1 -- 172.21.15.67:0/2250865314 wait complete. 2024-03-30T02:47:14.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e65200700 1 Processor -- start 2024-03-30T02:47:14.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e65200700 1 -- start start 2024-03-30T02:47:14.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e65200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e60100f60 0x7f5e601973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:14.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e65200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60106080 0x7f5e60197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:14.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e65200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6019bcc0 0x7f5e6019c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e65200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6019c740 con 0x7f5e60106080 2024-03-30T02:47:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e65200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6019c8b0 con 0x7f5e6019bcc0 2024-03-30T02:47:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e65200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6019ca70 con 0x7f5e60100f60 2024-03-30T02:47:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e5d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60106080 0x7f5e60197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e5d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60106080 0x7f5e60197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51844/0 (socket says 172.21.15.67:51844) 2024-03-30T02:47:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e5d400700 1 -- 172.21.15.67:0/3298059590 learned_addr learned my addr 172.21.15.67:0/3298059590 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.592+0000 7f5e5de00700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e60100f60 0x7f5e601973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e5e800700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6019bcc0 0x7f5e6019c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:14.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e5d400700 1 -- 172.21.15.67:0/3298059590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e60100f60 msgr2=0x7f5e601973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e5d400700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e60100f60 0x7f5e601973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e5d400700 1 -- 172.21.15.67:0/3298059590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6019bcc0 msgr2=0x7f5e6019c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e5d400700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6019bcc0 0x7f5e6019c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e5d400700 1 -- 172.21.15.67:0/3298059590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e500095f0 con 0x7f5e60106080 2024-03-30T02:47:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e5d400700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60106080 0x7f5e60197920 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00ea30 tx=0x7f5e4c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e56a00700 1 -- 172.21.15.67:0/3298059590 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00cbd0 con 0x7f5e60106080 2024-03-30T02:47:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e56a00700 1 -- 172.21.15.67:0/3298059590 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e4c00eed0 con 0x7f5e60106080 2024-03-30T02:47:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c009520 con 0x7f5e60106080 2024-03-30T02:47:14.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.593+0000 7f5e56a00700 1 -- 172.21.15.67:0/3298059590 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c003d20 con 0x7f5e60106080 2024-03-30T02:47:14.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.594+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e58009c40 con 0x7f5e60106080 2024-03-30T02:47:14.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.595+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e60109350 con 0x7f5e60106080 2024-03-30T02:47:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.596+0000 7f5e56a00700 1 -- 172.21.15.67:0/3298059590 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5e4c003fa0 con 0x7f5e60106080 2024-03-30T02:47:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.597+0000 7f5e56a00700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e440a1290 0x7f5e440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.598+0000 7f5e56a00700 1 -- 172.21.15.67:0/3298059590 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f5e4c014070 con 0x7f5e60106080 2024-03-30T02:47:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.598+0000 7f5e5de00700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e440a1290 0x7f5e440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:14.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.598+0000 7f5e5de00700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e440a1290 0x7f5e440a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5e50005fd0 tx=0x7f5e50009310 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:14.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.601+0000 7f5e56a00700 1 -- 172.21.15.67:0/3298059590 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5e4c0963a0 con 0x7f5e60106080 2024-03-30T02:47:14.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:14 smithi067 ceph-mon[37808]: pgmap v315: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-03-30T02:47:14.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:14 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3614659118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:47:14.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:14 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3720773015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:47:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:14 smithi082 ceph-mon[79176]: pgmap v315: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-03-30T02:47:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:14 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3614659118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:47:14.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:14 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3720773015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:47:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.819+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5e601986e0 con 0x7f5e60106080 2024-03-30T02:47:14.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.819+0000 7f5e56a00700 1 -- 172.21.15.67:0/3298059590 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5e4c0999c0 con 0x7f5e60106080 2024-03-30T02:47:14.819 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:47:14.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e440a1290 msgr2=0x7f5e440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e440a1290 0x7f5e440a3750 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5e50005fd0 tx=0x7f5e50009310 comp rx=0 tx=0).stop 2024-03-30T02:47:14.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60106080 msgr2=0x7f5e60197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:14.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60106080 0x7f5e60197920 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00ea30 tx=0x7f5e4c00ed40 comp rx=0 tx=0).stop 2024-03-30T02:47:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 shutdown_connections 2024-03-30T02:47:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e440a1290 0x7f5e440a3750 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6019bcc0 0x7f5e6019c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e60100f60 0x7f5e601973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 --2- 172.21.15.67:0/3298059590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e60106080 0x7f5e60197920 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 >> 172.21.15.67:0/3298059590 conn(0x7f5e600fc7c0 msgr2=0x7f5e600feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 shutdown_connections 2024-03-30T02:47:14.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:14.822+0000 7f5e65200700 1 -- 172.21.15.67:0/3298059590 wait complete. 2024-03-30T02:47:14.839 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-30T02:47:14.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:14 smithi170 ceph-mon[78798]: pgmap v315: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-03-30T02:47:14.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:14 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3614659118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:47:14.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:14 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3720773015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:47:15.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.042+0000 7f52c0e00700 1 -- 172.21.15.67:0/2233973543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc073b00 msgr2=0x7f52bc10cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.042+0000 7f52c0e00700 1 --2- 172.21.15.67:0/2233973543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc073b00 0x7f52bc10cef0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009990 tx=0x7f52a8009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.042+0000 7f52c0e00700 1 -- 172.21.15.67:0/2233973543 shutdown_connections 2024-03-30T02:47:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.042+0000 7f52c0e00700 1 --2- 172.21.15.67:0/2233973543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc10d430 0x7f52bc10f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.042+0000 7f52c0e00700 1 --2- 172.21.15.67:0/2233973543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52bc0731e0 0x7f52bc0735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.042+0000 7f52c0e00700 1 --2- 172.21.15.67:0/2233973543 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc073b00 0x7f52bc10cef0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:15.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.042+0000 7f52c0e00700 1 -- 172.21.15.67:0/2233973543 >> 172.21.15.67:0/2233973543 conn(0x7f52bc0fc700 msgr2=0x7f52bc0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.042+0000 7f52c0e00700 1 -- 172.21.15.67:0/2233973543 shutdown_connections 2024-03-30T02:47:15.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.042+0000 7f52c0e00700 1 -- 172.21.15.67:0/2233973543 wait complete. 2024-03-30T02:47:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.043+0000 7f52c0e00700 1 Processor -- start 2024-03-30T02:47:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52c0e00700 1 -- start start 2024-03-30T02:47:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52c0e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52bc0731e0 0x7f52bc1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52c0e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc073b00 0x7f52bc199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52c0e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc10d430 0x7f52bc19dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:15.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52c0e00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc19e300 con 0x7f52bc10d430 2024-03-30T02:47:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52c0e00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc19e470 con 0x7f52bc0731e0 2024-03-30T02:47:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52c0e00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc19e630 con 0x7f52bc073b00 2024-03-30T02:47:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52ba800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc10d430 0x7f52bc19dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52b9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52bc0731e0 0x7f52bc1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52ba800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc10d430 0x7f52bc19dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51868/0 (socket says 172.21.15.67:51868) 2024-03-30T02:47:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52b9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52bc0731e0 0x7f52bc1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:44908/0 (socket says 172.21.15.67:44908) 2024-03-30T02:47:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52ba800700 1 -- 172.21.15.67:0/4218682815 learned_addr learned my addr 172.21.15.67:0/4218682815 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.044+0000 7f52b9400700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc073b00 0x7f52bc199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52ba800700 1 -- 172.21.15.67:0/4218682815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc073b00 msgr2=0x7f52bc199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52ba800700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc073b00 0x7f52bc199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52ba800700 1 -- 172.21.15.67:0/4218682815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52bc0731e0 msgr2=0x7f52bc1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52ba800700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52bc0731e0 0x7f52bc1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52ba800700 1 -- 172.21.15.67:0/4218682815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a80095f0 con 0x7f52bc10d430 2024-03-30T02:47:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52b9400700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc073b00 0x7f52bc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:47:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52b9e00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52bc0731e0 0x7f52bc1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:15.047 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52ba800700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc10d430 0x7f52bc19dcc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f52b400ef30 tx=0x7f52b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52b2a00700 1 -- 172.21.15.67:0/4218682815 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4018070 con 0x7f52bc10d430 2024-03-30T02:47:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52b2a00700 1 -- 172.21.15.67:0/4218682815 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52b400f040 con 0x7f52bc10d430 2024-03-30T02:47:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc19e8b0 con 0x7f52bc10d430 2024-03-30T02:47:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52b2a00700 1 -- 172.21.15.67:0/4218682815 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b400cdb0 con 0x7f52bc10d430 2024-03-30T02:47:15.048 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.045+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc1a6150 con 0x7f52bc10d430 2024-03-30T02:47:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.047+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52bc108640 con 0x7f52bc10d430 2024-03-30T02:47:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.048+0000 7f52b2a00700 1 -- 172.21.15.67:0/4218682815 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f52b4003a00 con 0x7f52bc10d430 2024-03-30T02:47:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.050+0000 7f52b2a00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f52a00a1310 0x7f52a00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.050+0000 7f52b2a00700 1 -- 172.21.15.67:0/4218682815 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9542+0+0 (secure 0 0 0) 0x7f52b40ccc10 con 0x7f52bc10d430 2024-03-30T02:47:15.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.050+0000 7f52b9e00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f52a00a1310 0x7f52a00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:15.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.051+0000 7f52b9e00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f52a00a1310 0x7f52a00a37d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52ac011490 tx=0x7f52ac009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:15.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.054+0000 7f52b2a00700 1 -- 172.21.15.67:0/4218682815 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f52b4095c90 con 0x7f52bc10d430 2024-03-30T02:47:15.331 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.333+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f52bc074890 con 0x7f52bc10d430 2024-03-30T02:47:15.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.344+0000 7f52b2a00700 1 -- 172.21.15.67:0/4218682815 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v28) v1 ==== 81+0+733923 (secure 0 0 0) 0x7f52b40992b0 con 0x7f52bc10d430 2024-03-30T02:47:15.354 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:47:15.354 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:47:15.354 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 28, 2024-03-30T02:47:15.355 INFO:teuthology.orchestra.run.smithi067.stdout: "active_gid": 25634, 2024-03-30T02:47:15.355 INFO:teuthology.orchestra.run.smithi067.stdout: "active_name": "x", 2024-03-30T02:47:15.355 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addrs": { 2024-03-30T02:47:15.355 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-30T02:47:15.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.355 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-30T02:47:15.355 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6800", 2024-03-30T02:47:15.355 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1710497985 2024-03-30T02:47:15.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.356 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-03-30T02:47:15.356 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6801", 2024-03-30T02:47:15.356 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1710497985 2024-03-30T02:47:15.356 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.356 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:15.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.356 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addr": "172.21.15.67:6801/1710497985", 2024-03-30T02:47:15.357 INFO:teuthology.orchestra.run.smithi067.stdout: "active_change": "2024-03-30T02:37:02.363197+0000", 2024-03-30T02:47:15.357 INFO:teuthology.orchestra.run.smithi067.stdout: "active_mgr_features": 4540138322906710015, 2024-03-30T02:47:15.357 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-03-30T02:47:15.357 INFO:teuthology.orchestra.run.smithi067.stdout: "standbys": [ 2024-03-30T02:47:15.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.357 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 25618, 2024-03-30T02:47:15.357 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "z", 2024-03-30T02:47:15.357 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540138322906710015, 2024-03-30T02:47:15.358 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-03-30T02:47:15.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-03-30T02:47:15.358 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.358 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.358 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.358 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:15.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:15.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.359 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.359 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.359 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:15.359 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.359 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.359 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.359 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-03-30T02:47:15.359 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.360 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.360 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.360 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.360 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.360 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.360 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.361 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.361 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.361 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.361 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.361 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.361 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.361 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.361 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.361 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.362 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.362 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.362 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.362 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.362 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.362 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.363 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.363 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.363 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.363 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.363 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.363 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.363 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.363 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.364 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.364 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.364 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.364 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.364 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.364 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.364 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.365 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.366 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.366 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.366 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.366 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.366 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.366 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.367 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.367 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.367 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.367 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.367 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.367 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.367 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.367 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.368 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.368 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.368 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-03-30T02:47:15.368 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-03-30T02:47:15.368 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.368 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.368 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.369 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.369 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.369 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.369 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.369 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to", 2024-03-30T02:47:15.369 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.369 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.369 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.370 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-03-30T02:47:15.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-03-30T02:47:15.370 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.370 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.370 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.370 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-03-30T02:47:15.370 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.370 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.371 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.371 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-03-30T02:47:15.371 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.371 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.371 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.371 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-03-30T02:47:15.371 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-03-30T02:47:15.371 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.372 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.372 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.372 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.372 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.372 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.372 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.372 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-03-30T02:47:15.372 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.372 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.373 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.373 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-03-30T02:47:15.373 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-03-30T02:47:15.373 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.373 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.373 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.373 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.373 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.374 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.374 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.374 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-03-30T02:47:15.374 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.374 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.374 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.374 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-03-30T02:47:15.375 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-03-30T02:47:15.375 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.375 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.375 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.375 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-03-30T02:47:15.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.375 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.375 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.375 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-03-30T02:47:15.376 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.376 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.376 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.376 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-03-30T02:47:15.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-03-30T02:47:15.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.376 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.376 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.377 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.377 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.377 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.377 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-03-30T02:47:15.377 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.377 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.377 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.378 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-03-30T02:47:15.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-03-30T02:47:15.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.378 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.378 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.378 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.378 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.379 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.379 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.379 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-30T02:47:15.379 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.379 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.379 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.379 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-03-30T02:47:15.379 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-03-30T02:47:15.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.380 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.380 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.380 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.380 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.380 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.380 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.380 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-03-30T02:47:15.380 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.381 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.381 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.381 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-03-30T02:47:15.381 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.381 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.382 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.382 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-03-30T02:47:15.382 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-03-30T02:47:15.382 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.382 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.382 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.382 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.382 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.383 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.383 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.383 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-03-30T02:47:15.383 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.383 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.383 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.383 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-03-30T02:47:15.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-03-30T02:47:15.384 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.384 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.384 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.384 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-03-30T02:47:15.384 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.384 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.384 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.384 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-03-30T02:47:15.385 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T02:47:15.385 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.385 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.385 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-03-30T02:47:15.385 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-03-30T02:47:15.385 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:15.385 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.385 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.386 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:15.386 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-03-30T02:47:15.386 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-03-30T02:47:15.386 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.386 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-30T02:47:15.386 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T02:47:15.386 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.386 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.387 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-03-30T02:47:15.387 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-03-30T02:47:15.387 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:15.387 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.387 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.387 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-03-30T02:47:15.387 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:15.387 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-03-30T02:47:15.387 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-30T02:47:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-03-30T02:47:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-03-30T02:47:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.388 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.389 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.389 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-03-30T02:47:15.389 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.389 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.389 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.389 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-30T02:47:15.389 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-30T02:47:15.389 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.389 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.390 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-03-30T02:47:15.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-03-30T02:47:15.390 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:15.390 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.390 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.390 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-03-30T02:47:15.390 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-03-30T02:47:15.391 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-03-30T02:47:15.391 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.391 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-03-30T02:47:15.391 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-30T02:47:15.391 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.391 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.391 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-03-30T02:47:15.391 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-03-30T02:47:15.392 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.392 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.392 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.392 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-03-30T02:47:15.392 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.392 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.392 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.392 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-03-30T02:47:15.392 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T02:47:15.393 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.393 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.393 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-03-30T02:47:15.393 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-03-30T02:47:15.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:15.393 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.393 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.393 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:15.394 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-03-30T02:47:15.394 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-03-30T02:47:15.394 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.394 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-30T02:47:15.394 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T02:47:15.394 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.394 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.395 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.395 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.395 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.395 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.395 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.395 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.396 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.396 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.396 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.396 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.396 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.396 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.396 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.396 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.396 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.397 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.397 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.397 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.398 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.398 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.398 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.398 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.398 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.398 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.398 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.398 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.399 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.399 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.399 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.399 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.399 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.399 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.399 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.399 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.400 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.401 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.402 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.402 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.402 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.402 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.402 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.402 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.402 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.402 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.403 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.403 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-03-30T02:47:15.403 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-03-30T02:47:15.403 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:15.403 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.403 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.403 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:15.404 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.404 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.404 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.404 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-30T02:47:15.404 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.404 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.404 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.404 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-03-30T02:47:15.405 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-03-30T02:47:15.405 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.405 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.405 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.405 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-03-30T02:47:15.405 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.405 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.405 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.405 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-03-30T02:47:15.406 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-03-30T02:47:15.406 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap" 2024-03-30T02:47:15.406 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.406 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-03-30T02:47:15.406 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.406 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.406 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.407 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-03-30T02:47:15.407 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-03-30T02:47:15.407 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.407 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.407 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.407 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.407 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.407 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.407 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.408 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-30T02:47:15.408 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.408 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.408 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.408 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:15.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:15.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.408 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.409 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.409 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:15.409 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.409 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.409 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.409 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-30T02:47:15.409 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.409 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.410 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.410 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-03-30T02:47:15.410 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-03-30T02:47:15.410 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.410 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.410 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.410 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-30T02:47:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-30T02:47:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.411 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-03-30T02:47:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-03-30T02:47:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.412 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-30T02:47:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-03-30T02:47:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-03-30T02:47:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-03-30T02:47:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:15.414 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-03-30T02:47:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-30T02:47:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.415 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-03-30T02:47:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-03-30T02:47:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-03-30T02:47:15.416 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-30T02:47:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-03-30T02:47:15.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-03-30T02:47:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-03-30T02:47:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.418 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-30T02:47:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-03-30T02:47:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-03-30T02:47:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.419 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.420 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.420 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.420 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.420 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.420 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.420 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-30T02:47:15.420 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-30T02:47:15.420 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.420 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.421 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-03-30T02:47:15.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-03-30T02:47:15.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.421 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.421 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.421 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:15.421 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.422 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.422 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.422 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-30T02:47:15.422 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.422 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.422 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.422 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-03-30T02:47:15.422 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-03-30T02:47:15.423 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:15.423 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.423 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.423 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-03-30T02:47:15.423 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.423 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.423 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.423 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-30T02:47:15.424 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.424 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-03-30T02:47:15.424 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-03-30T02:47:15.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.424 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.424 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.425 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.425 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.425 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.425 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.425 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-30T02:47:15.425 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.425 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.425 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.426 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-03-30T02:47:15.426 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-03-30T02:47:15.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.426 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.426 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.426 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.426 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.426 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.427 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.427 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-30T02:47:15.427 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.427 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.427 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.427 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-03-30T02:47:15.427 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-03-30T02:47:15.427 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.428 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.428 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.428 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.428 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.428 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.428 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.428 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-30T02:47:15.428 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.428 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.429 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.429 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-03-30T02:47:15.429 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-03-30T02:47:15.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.429 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.429 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.429 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-30T02:47:15.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.430 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.430 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.430 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:15.430 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.430 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.430 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.430 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-03-30T02:47:15.431 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-03-30T02:47:15.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.431 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.431 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.431 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-30T02:47:15.431 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.431 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.431 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.431 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-03-30T02:47:15.432 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.432 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.432 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.432 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-03-30T02:47:15.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-03-30T02:47:15.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.432 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.432 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.433 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-30T02:47:15.433 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.433 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.433 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.433 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-03-30T02:47:15.433 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.433 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.433 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.434 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-03-30T02:47:15.434 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-03-30T02:47:15.434 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.434 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.434 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.434 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-30T02:47:15.434 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.434 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.435 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.435 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:15.435 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.435 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.435 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.435 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-03-30T02:47:15.435 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-03-30T02:47:15.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-30T02:47:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-03-30T02:47:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.437 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.437 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-03-30T02:47:15.437 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-03-30T02:47:15.437 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.437 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.437 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.437 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-30T02:47:15.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.438 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.438 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.438 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-03-30T02:47:15.438 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.438 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.438 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.439 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-03-30T02:47:15.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-03-30T02:47:15.439 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.439 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.439 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.439 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-30T02:47:15.439 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.439 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.439 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.440 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.441 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-30T02:47:15.441 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.441 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.441 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.441 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-03-30T02:47:15.441 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.441 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.441 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.442 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-03-30T02:47:15.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-03-30T02:47:15.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.442 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.442 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.442 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-30T02:47:15.442 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.442 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.443 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.443 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-03-30T02:47:15.443 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.443 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.443 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.443 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-03-30T02:47:15.443 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-03-30T02:47:15.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.444 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.444 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.444 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-30T02:47:15.444 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.444 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.444 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.444 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-03-30T02:47:15.444 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.444 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.445 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.445 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-03-30T02:47:15.445 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-03-30T02:47:15.445 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.445 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.445 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.445 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-30T02:47:15.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.446 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.446 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.446 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:15.446 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.446 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.446 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.446 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-03-30T02:47:15.447 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-03-30T02:47:15.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.447 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.447 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-30T02:47:15.447 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.447 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.447 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.447 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-03-30T02:47:15.448 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.448 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.448 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.448 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-03-30T02:47:15.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-03-30T02:47:15.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.448 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.449 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.449 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-30T02:47:15.449 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.449 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.449 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.449 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:15.449 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.449 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.449 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.450 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-03-30T02:47:15.450 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-03-30T02:47:15.450 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.450 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.450 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.450 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-30T02:47:15.450 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.450 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.451 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.451 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-03-30T02:47:15.451 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.451 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.451 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.451 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-03-30T02:47:15.451 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-03-30T02:47:15.451 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.452 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.452 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.452 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-30T02:47:15.452 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.452 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.452 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.452 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-03-30T02:47:15.452 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.453 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.453 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.453 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-03-30T02:47:15.453 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-03-30T02:47:15.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.453 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.453 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.453 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.454 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.454 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.454 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.454 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-03-30T02:47:15.454 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.454 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.455 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-03-30T02:47:15.455 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-03-30T02:47:15.455 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.455 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.455 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.455 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:15.455 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.455 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.455 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.456 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-30T02:47:15.456 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.456 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.456 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.456 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-03-30T02:47:15.456 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-03-30T02:47:15.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.457 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.457 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.457 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-03-30T02:47:15.457 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.457 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.457 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.457 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-30T02:47:15.457 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.458 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.458 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.458 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-03-30T02:47:15.458 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-03-30T02:47:15.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.458 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.458 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.459 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-03-30T02:47:15.459 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.459 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.459 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.459 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-30T02:47:15.459 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.459 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.459 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.460 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-03-30T02:47:15.460 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-03-30T02:47:15.460 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.460 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.460 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.460 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-03-30T02:47:15.460 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.460 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.460 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.461 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-03-30T02:47:15.461 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.461 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.461 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.461 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-03-30T02:47:15.461 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-03-30T02:47:15.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.461 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.462 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.462 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.462 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.462 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.462 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.462 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-30T02:47:15.462 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.462 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.463 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.463 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-03-30T02:47:15.463 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-03-30T02:47:15.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.463 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.463 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.463 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:15.463 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.464 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.464 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.464 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-03-30T02:47:15.464 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.464 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.464 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-03-30T02:47:15.465 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-03-30T02:47:15.465 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.465 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.465 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:15.465 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.465 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.465 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.465 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-03-30T02:47:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-03-30T02:47:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-03-30T02:47:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.466 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.467 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.467 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.467 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.467 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-30T02:47:15.467 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.467 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.467 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.468 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-03-30T02:47:15.468 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-03-30T02:47:15.468 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.468 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.468 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.468 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.468 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.468 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.469 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.469 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-30T02:47:15.469 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.469 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.469 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.469 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.469 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.469 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.470 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.470 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.470 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.470 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.470 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.470 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.470 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.470 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.471 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.471 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.471 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.471 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.471 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.471 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.471 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.472 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-03-30T02:47:15.472 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-03-30T02:47:15.472 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.472 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.472 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.472 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.472 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.473 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.473 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-30T02:47:15.473 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.473 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.473 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.473 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.473 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.473 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.474 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.474 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.474 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.474 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.474 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.474 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.474 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-30T02:47:15.474 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.474 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.475 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.475 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.475 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.475 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.475 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.476 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.476 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.476 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.476 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.476 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.476 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.476 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.476 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.476 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.477 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.477 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.477 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.477 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.477 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.477 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.478 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.478 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.478 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.478 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.478 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.478 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.478 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.478 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.479 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.479 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.479 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-30T02:47:15.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-30T02:47:15.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.479 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.479 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.480 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.480 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.480 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.480 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.480 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-30T02:47:15.480 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.480 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.480 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.481 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-30T02:47:15.481 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-30T02:47:15.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.481 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.481 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.481 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-03-30T02:47:15.481 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.481 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.481 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.482 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-30T02:47:15.482 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.482 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.482 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.482 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-03-30T02:47:15.482 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-03-30T02:47:15.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.483 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.483 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.483 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:15.483 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.483 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.483 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.483 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-03-30T02:47:15.484 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.484 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.484 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.484 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-03-30T02:47:15.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-03-30T02:47:15.484 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.484 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.484 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.485 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:15.485 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.485 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.485 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.485 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-30T02:47:15.485 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.485 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.485 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.486 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-03-30T02:47:15.486 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-03-30T02:47:15.486 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.486 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.486 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.486 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.486 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.487 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.487 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.487 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-03-30T02:47:15.487 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.487 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.487 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.487 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-03-30T02:47:15.487 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-03-30T02:47:15.488 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.488 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.488 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.488 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-03-30T02:47:15.488 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.488 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.488 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.488 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-03-30T02:47:15.489 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-03-30T02:47:15.489 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.489 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-03-30T02:47:15.489 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.489 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.489 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.489 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-03-30T02:47:15.489 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-03-30T02:47:15.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.490 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.490 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.490 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-03-30T02:47:15.490 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.490 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.490 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.490 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-30T02:47:15.490 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.491 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.491 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.491 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-03-30T02:47:15.491 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-03-30T02:47:15.491 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.491 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.491 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.492 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-30T02:47:15.492 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.492 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.492 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.492 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-30T02:47:15.492 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.492 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.492 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.493 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-03-30T02:47:15.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-03-30T02:47:15.493 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.493 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.493 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.493 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.493 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.493 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.494 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-30T02:47:15.494 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.494 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.494 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.494 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-03-30T02:47:15.494 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-03-30T02:47:15.494 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.495 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.495 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.495 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.495 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.495 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.495 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.495 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-30T02:47:15.495 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.495 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.496 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.496 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-03-30T02:47:15.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-03-30T02:47:15.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.496 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.496 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.496 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.496 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.497 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.497 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.497 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-30T02:47:15.497 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.497 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.497 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-03-30T02:47:15.497 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-03-30T02:47:15.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.498 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.498 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.498 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.498 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.498 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.498 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.499 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-30T02:47:15.499 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.499 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.499 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.499 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-03-30T02:47:15.499 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-03-30T02:47:15.499 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.499 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.500 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.500 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.500 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.500 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.500 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.500 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-30T02:47:15.500 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.500 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.500 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.501 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-03-30T02:47:15.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-03-30T02:47:15.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.501 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.501 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.501 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-03-30T02:47:15.501 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.502 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.502 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.502 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-03-30T02:47:15.502 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.502 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.502 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.502 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-03-30T02:47:15.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-03-30T02:47:15.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.503 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.503 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.503 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.503 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.503 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.503 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.503 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-30T02:47:15.504 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.504 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.504 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.504 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-03-30T02:47:15.504 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-03-30T02:47:15.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.504 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.504 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.505 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.505 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.505 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.505 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.505 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-30T02:47:15.505 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.505 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.505 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.506 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-03-30T02:47:15.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-03-30T02:47:15.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.506 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.506 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.506 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.506 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.506 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.507 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.507 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-30T02:47:15.507 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.507 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.507 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.507 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-03-30T02:47:15.507 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-03-30T02:47:15.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.508 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.508 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.508 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.508 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.508 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.508 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.508 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-03-30T02:47:15.508 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.509 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.509 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-03-30T02:47:15.509 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-03-30T02:47:15.509 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.509 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.509 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.509 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.510 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.510 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.510 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.510 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-30T02:47:15.510 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.510 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.510 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.511 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-03-30T02:47:15.511 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-03-30T02:47:15.511 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.511 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.511 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.511 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.511 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.511 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.511 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.512 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-30T02:47:15.512 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.512 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.512 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-03-30T02:47:15.513 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.513 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.513 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.513 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.513 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.513 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.513 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.513 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.514 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.514 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.514 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.514 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.514 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.514 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.514 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.514 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.514 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.515 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.515 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.515 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.515 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.515 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.515 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.515 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.515 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.516 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.516 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.516 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.516 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.516 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.516 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.516 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.516 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.517 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.517 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.517 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.517 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.517 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.517 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.517 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.517 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.518 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.518 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.518 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.518 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.518 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.518 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.518 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.518 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.519 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.519 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.519 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.519 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.519 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.519 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.519 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.520 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.520 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.520 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.520 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.520 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.520 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.520 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.520 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-03-30T02:47:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-03-30T02:47:15.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.522 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.522 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.522 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-03-30T02:47:15.522 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.522 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.522 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.522 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-30T02:47:15.522 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.523 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.523 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.523 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-03-30T02:47:15.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-03-30T02:47:15.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.523 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.523 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.523 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-03-30T02:47:15.524 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.524 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.524 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.524 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-30T02:47:15.524 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.524 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.524 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.524 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-03-30T02:47:15.525 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.525 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.525 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.525 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-30T02:47:15.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-30T02:47:15.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.526 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.526 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.526 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:15.526 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.526 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.526 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.526 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.527 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.527 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.527 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.527 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-03-30T02:47:15.527 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-30T02:47:15.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.527 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.528 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.528 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.528 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.528 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.528 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.528 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.528 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.528 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.529 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-30T02:47:15.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-30T02:47:15.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.529 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.529 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.529 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.529 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.529 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.530 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.530 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.530 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.530 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.530 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.530 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-03-30T02:47:15.530 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-03-30T02:47:15.530 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.531 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.531 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.531 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.531 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.531 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.532 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.532 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-30T02:47:15.532 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-30T02:47:15.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.532 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.532 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.532 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.533 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.533 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.533 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.533 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.533 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.533 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-03-30T02:47:15.534 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-30T02:47:15.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.534 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.534 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.534 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.534 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.534 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.535 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.535 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.535 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.535 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-30T02:47:15.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-30T02:47:15.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.535 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.536 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.536 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.536 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.536 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.536 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.536 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.536 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.536 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.537 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-30T02:47:15.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-30T02:47:15.537 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.537 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.537 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.537 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.537 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.537 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.538 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.538 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.538 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.538 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.538 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.538 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-30T02:47:15.539 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-30T02:47:15.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.539 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.539 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.539 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.539 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.540 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.540 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.540 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.540 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-30T02:47:15.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-30T02:47:15.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.541 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.541 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.541 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.541 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.542 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.542 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-30T02:47:15.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-30T02:47:15.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.542 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.542 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.542 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.542 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.543 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.543 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.543 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.543 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.543 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.543 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-30T02:47:15.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-30T02:47:15.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.544 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.544 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.544 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.544 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.544 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.544 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.545 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.545 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.545 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.545 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-30T02:47:15.545 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-30T02:47:15.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.545 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.546 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.546 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.546 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.546 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.546 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.546 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.546 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.547 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.547 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-30T02:47:15.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-30T02:47:15.547 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.547 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.547 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.547 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.547 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.548 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.548 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.548 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.548 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.548 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.548 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.548 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-03-30T02:47:15.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-30T02:47:15.549 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.549 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.549 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.549 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-03-30T02:47:15.549 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.549 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.549 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.549 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.549 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.550 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.550 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.550 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-30T02:47:15.550 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-30T02:47:15.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.550 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.550 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.551 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.551 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.551 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.551 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.551 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.551 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.551 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.551 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-03-30T02:47:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-03-30T02:47:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.553 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.553 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.553 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.553 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.553 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.553 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-03-30T02:47:15.553 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-30T02:47:15.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.554 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.554 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.554 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-03-30T02:47:15.554 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.554 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.554 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.554 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.554 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.555 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.555 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.555 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-30T02:47:15.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-30T02:47:15.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.555 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.555 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.555 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.556 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.556 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.556 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.556 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.556 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.556 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.556 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-30T02:47:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-30T02:47:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.558 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.558 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.558 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.558 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-30T02:47:15.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-30T02:47:15.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.558 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.559 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.559 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.559 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.559 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.559 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.559 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.559 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.560 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-30T02:47:15.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-30T02:47:15.560 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.560 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.560 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.560 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.560 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.560 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.561 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.561 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.561 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.561 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.561 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-03-30T02:47:15.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-30T02:47:15.562 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.562 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.562 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.562 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.562 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.562 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.562 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.563 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.563 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.563 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-30T02:47:15.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-30T02:47:15.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.564 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.564 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.564 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.564 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.564 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.565 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.565 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-30T02:47:15.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-30T02:47:15.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.565 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.565 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.565 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.566 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.566 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.566 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.566 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.566 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.566 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.566 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-30T02:47:15.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-30T02:47:15.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.567 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.567 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.567 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.567 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.567 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.567 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.568 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.568 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.568 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.568 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-30T02:47:15.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-30T02:47:15.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.569 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.569 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.569 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.569 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.569 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.569 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.569 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.570 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-30T02:47:15.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-30T02:47:15.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.570 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.570 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.570 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.570 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.570 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.571 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.571 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.571 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.571 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.571 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-30T02:47:15.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-30T02:47:15.572 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.572 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.572 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.572 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.572 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.572 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.572 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.572 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.573 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.573 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.573 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.573 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-30T02:47:15.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-30T02:47:15.573 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.573 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.573 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.574 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.574 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.574 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.574 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.574 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.574 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.574 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.575 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-30T02:47:15.575 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-30T02:47:15.575 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.575 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.575 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.575 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.575 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.576 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.576 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.576 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.576 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.576 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-03-30T02:47:15.576 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.577 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.578 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.578 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-30T02:47:15.578 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-30T02:47:15.578 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.578 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.578 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.578 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-30T02:47:15.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.579 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.579 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.579 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-30T02:47:15.579 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-30T02:47:15.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.580 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.580 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.580 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:15.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.580 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.580 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.580 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.580 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.581 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.581 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-30T02:47:15.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-30T02:47:15.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.581 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.581 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.581 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-03-30T02:47:15.582 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.582 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.582 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.582 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.582 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.582 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.583 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-30T02:47:15.583 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-30T02:47:15.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.583 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.583 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.583 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-03-30T02:47:15.583 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.583 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.584 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.584 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.584 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.584 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.584 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.584 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-03-30T02:47:15.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-30T02:47:15.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.585 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.585 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.585 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.585 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.585 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.585 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.585 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.585 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.585 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.586 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.586 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-30T02:47:15.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-30T02:47:15.586 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.586 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.586 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.586 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-03-30T02:47:15.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.587 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.587 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.587 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.587 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.587 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-03-30T02:47:15.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-30T02:47:15.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.588 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.588 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.588 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.588 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.588 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.589 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.589 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.589 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-03-30T02:47:15.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-30T02:47:15.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.590 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.590 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.590 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.590 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.591 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.591 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-03-30T02:47:15.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-30T02:47:15.591 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.591 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.591 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.592 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.592 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.592 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.592 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.592 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.592 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.592 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-30T02:47:15.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-30T02:47:15.593 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.593 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.593 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.593 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:15.593 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.593 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.593 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.594 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.594 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.594 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.594 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-30T02:47:15.594 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-30T02:47:15.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.595 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.595 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.595 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:15.595 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.595 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.595 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.595 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.595 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.596 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.596 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-30T02:47:15.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-30T02:47:15.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.596 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.596 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:15.597 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.597 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.597 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.597 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.597 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.597 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.597 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.597 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-03-30T02:47:15.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-03-30T02:47:15.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.598 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.598 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.598 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.598 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.598 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.598 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.599 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.599 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.599 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.599 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.599 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-03-30T02:47:15.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-03-30T02:47:15.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.599 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.600 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.600 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.600 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.600 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.600 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.600 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.600 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.600 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.600 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.601 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-03-30T02:47:15.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-03-30T02:47:15.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.601 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.601 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.601 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.602 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.602 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-03-30T02:47:15.602 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.602 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.602 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-03-30T02:47:15.602 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-03-30T02:47:15.603 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.603 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.603 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.603 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-03-30T02:47:15.603 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.603 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.603 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.603 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.604 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.604 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.604 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-03-30T02:47:15.604 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-03-30T02:47:15.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.604 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.604 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.605 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.605 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.605 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.605 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.605 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.605 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.605 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.605 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.606 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.606 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.606 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.606 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.606 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.606 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.607 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.607 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.607 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.607 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.607 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.607 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.607 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.607 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.607 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.608 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.608 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.608 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.608 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.608 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.608 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.608 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.609 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.609 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.609 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.609 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.609 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.609 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.609 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.609 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.610 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.610 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.610 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.610 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.610 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.610 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.611 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.611 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.611 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.611 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.611 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.611 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.611 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.612 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.612 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.612 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.612 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.612 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.612 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.612 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.613 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.613 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.613 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.613 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.613 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.613 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.613 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.614 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.614 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.614 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.614 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-03-30T02:47:15.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-03-30T02:47:15.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.614 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.615 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.615 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.615 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.615 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.615 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-03-30T02:47:15.615 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.615 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.616 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-03-30T02:47:15.616 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-03-30T02:47:15.616 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.616 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.616 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.616 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.617 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.617 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.617 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.617 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.617 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.617 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.617 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-03-30T02:47:15.618 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-03-30T02:47:15.618 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.618 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.618 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-03-30T02:47:15.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.618 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.618 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.619 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.619 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.619 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.619 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.619 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-03-30T02:47:15.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-03-30T02:47:15.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.620 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.620 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-03-30T02:47:15.620 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.621 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.621 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-03-30T02:47:15.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-03-30T02:47:15.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.621 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.621 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.621 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.621 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.622 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.622 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.622 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.622 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.622 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.622 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.622 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-03-30T02:47:15.623 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-03-30T02:47:15.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.623 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.623 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.623 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-03-30T02:47:15.623 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.623 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.623 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.623 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.624 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.624 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.624 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.624 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-03-30T02:47:15.624 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-03-30T02:47:15.624 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.624 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.625 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.625 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-03-30T02:47:15.625 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.625 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.625 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.625 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.625 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-03-30T02:47:15.625 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.625 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.626 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.626 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.626 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.626 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-03-30T02:47:15.626 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-03-30T02:47:15.626 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.626 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.627 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.627 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-03-30T02:47:15.627 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-03-30T02:47:15.627 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-03-30T02:47:15.627 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.627 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.627 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.628 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-03-30T02:47:15.628 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-03-30T02:47:15.628 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.628 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.628 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.628 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.628 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.629 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.629 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.629 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.629 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.629 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.629 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.629 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.629 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.630 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.630 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-03-30T02:47:15.630 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.630 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.630 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.630 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-03-30T02:47:15.630 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-03-30T02:47:15.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.631 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.631 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.631 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.631 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.631 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.631 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.631 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-03-30T02:47:15.631 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.632 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.632 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.632 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.632 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.632 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.632 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.633 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.633 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.633 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.633 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.633 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.633 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.633 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.633 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.633 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.634 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.634 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.634 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.634 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.634 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.634 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.634 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.635 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.635 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.635 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.635 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.635 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.635 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.635 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.635 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.635 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.636 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.636 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.636 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.636 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.637 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.637 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.637 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.637 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.637 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.637 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.637 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.637 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.637 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.638 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.638 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.638 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.638 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.638 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.638 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.638 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.638 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.639 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-03-30T02:47:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-03-30T02:47:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.640 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.641 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-03-30T02:47:15.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.641 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.641 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.641 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-30T02:47:15.641 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.641 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.642 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-03-30T02:47:15.642 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-03-30T02:47:15.642 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.642 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.642 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.642 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-03-30T02:47:15.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.642 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.643 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.643 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-30T02:47:15.643 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.643 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.643 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.643 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-03-30T02:47:15.643 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-03-30T02:47:15.643 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.644 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.644 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.644 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-03-30T02:47:15.644 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.644 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.644 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.644 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-03-30T02:47:15.644 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.645 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.645 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-03-30T02:47:15.645 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-03-30T02:47:15.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.645 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-03-30T02:47:15.646 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.646 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.646 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.646 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-30T02:47:15.646 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.646 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.646 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-03-30T02:47:15.647 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-03-30T02:47:15.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.647 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.647 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.647 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.647 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.647 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.647 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.647 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-30T02:47:15.648 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.648 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.648 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.648 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:15.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:15.648 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.648 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-03-30T02:47:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.649 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.650 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-03-30T02:47:15.650 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-03-30T02:47:15.650 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.650 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.650 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.650 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-03-30T02:47:15.650 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.651 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.651 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.651 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-30T02:47:15.651 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.651 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.651 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.652 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-03-30T02:47:15.652 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.652 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.652 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.652 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.653 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.653 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.653 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.653 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.653 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.653 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.653 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.653 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.653 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.654 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.654 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.654 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.654 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.654 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.654 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.654 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.655 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.656 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.657 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.657 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.657 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.657 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.657 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.657 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.657 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.657 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.658 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.658 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.658 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.658 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.658 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.658 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.658 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.658 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.659 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.659 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.659 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.659 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.659 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.660 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.660 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.660 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.660 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.660 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.660 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.660 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.661 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-03-30T02:47:15.661 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-03-30T02:47:15.661 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.661 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.661 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.661 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-03-30T02:47:15.661 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.661 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.662 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.662 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.662 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.662 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.662 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.662 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-03-30T02:47:15.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-03-30T02:47:15.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.663 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.663 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.663 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-03-30T02:47:15.663 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.663 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.663 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.663 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.663 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.664 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.664 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.664 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:15.664 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:15.664 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.664 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.664 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.665 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:15.665 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.665 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.665 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.665 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.665 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.665 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.665 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.665 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.666 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.666 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.666 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-03-30T02:47:15.666 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-03-30T02:47:15.666 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-03-30T02:47:15.666 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.666 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-03-30T02:47:15.667 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-03-30T02:47:15.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.667 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.667 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.667 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-03-30T02:47:15.667 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.667 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.667 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.667 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-30T02:47:15.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.668 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.668 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-03-30T02:47:15.668 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-03-30T02:47:15.668 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.668 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-03-30T02:47:15.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.669 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.669 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.669 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-30T02:47:15.669 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.669 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.669 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.670 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-03-30T02:47:15.670 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-03-30T02:47:15.670 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.670 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.670 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.670 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.670 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.671 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.671 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-03-30T02:47:15.671 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.671 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.671 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.671 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:15.671 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:15.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.672 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.672 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.672 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-03-30T02:47:15.672 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-03-30T02:47:15.672 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.672 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.672 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-30T02:47:15.673 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.673 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.673 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.673 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.673 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.673 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.673 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.673 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.673 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.674 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.674 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.674 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.674 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.674 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.674 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.674 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.674 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.675 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.675 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.675 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.675 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.675 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.675 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.675 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.675 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.676 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.676 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.676 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.676 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.676 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.676 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.676 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.677 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.677 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.677 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.677 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.677 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.677 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.677 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.678 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.678 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.678 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.678 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.678 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.678 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.678 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.680 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.680 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.680 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.680 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.680 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.680 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.680 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.681 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.681 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.681 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-03-30T02:47:15.681 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-03-30T02:47:15.681 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.681 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.681 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.682 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.682 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.682 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-03-30T02:47:15.682 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.682 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.682 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-03-30T02:47:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-03-30T02:47:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-03-30T02:47:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.683 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.684 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.684 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-03-30T02:47:15.684 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.684 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.684 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.684 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-03-30T02:47:15.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-03-30T02:47:15.684 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.685 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.685 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.685 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-03-30T02:47:15.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.685 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.685 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.685 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T02:47:15.685 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.686 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.686 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.686 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-03-30T02:47:15.686 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-03-30T02:47:15.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.686 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.686 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.687 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:15.687 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:15.687 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-03-30T02:47:15.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.687 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-30T02:47:15.687 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.687 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.687 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.688 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-03-30T02:47:15.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-03-30T02:47:15.688 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.688 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.688 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.688 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.688 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.688 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.689 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.689 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-03-30T02:47:15.689 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.689 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.689 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.689 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-03-30T02:47:15.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-03-30T02:47:15.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.690 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.690 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.690 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-03-30T02:47:15.690 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.690 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.690 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.690 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T02:47:15.691 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.691 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.691 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.691 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.691 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.691 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-03-30T02:47:15.692 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.692 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.692 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.692 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.692 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.692 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.692 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.693 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.694 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.694 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.694 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.694 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.694 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.694 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.694 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.695 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.696 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.696 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.696 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.696 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.696 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.696 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.696 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.696 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.697 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.697 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.697 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.697 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.697 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.697 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.697 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.697 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.697 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.698 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.698 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.698 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.698 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.698 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.698 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.699 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.700 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-03-30T02:47:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.702 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.703 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.704 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.705 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.706 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.707 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.707 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.707 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.707 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.707 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.707 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.707 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.707 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.708 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.708 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.708 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.708 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.708 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.708 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.708 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.708 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.709 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.709 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.709 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.709 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.709 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.709 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.709 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.709 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.709 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.710 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.710 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-03-30T02:47:15.710 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.710 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.710 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.710 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-03-30T02:47:15.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-03-30T02:47:15.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.711 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.711 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-03-30T02:47:15.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.711 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.711 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.712 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-30T02:47:15.712 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.712 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.712 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.712 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-03-30T02:47:15.712 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-03-30T02:47:15.712 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.712 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.713 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.713 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:15.713 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-03-30T02:47:15.713 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.713 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.713 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-30T02:47:15.713 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.713 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.713 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.714 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.714 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.714 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.714 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.714 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.714 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.715 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.715 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.715 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.715 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.715 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.715 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.715 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.715 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.716 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.716 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.716 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.716 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.716 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.716 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.716 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.716 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.717 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.717 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.717 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.717 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.717 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.717 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.717 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.717 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.717 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.718 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.718 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.718 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.718 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.718 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.718 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.719 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.719 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.719 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.719 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.719 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.719 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.719 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.720 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.720 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.720 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.720 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.720 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.720 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.720 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.720 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.721 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.721 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.721 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.721 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.721 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.721 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.721 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.721 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.722 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.722 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.722 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.722 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.722 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.722 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-03-30T02:47:15.723 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.723 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.723 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.723 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-03-30T02:47:15.723 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-03-30T02:47:15.723 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.723 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.723 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.724 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-03-30T02:47:15.724 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.724 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.724 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.724 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-03-30T02:47:15.724 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-30T02:47:15.724 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.724 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.725 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.725 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.725 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.725 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.725 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.725 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.725 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.726 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.726 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.726 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.726 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.726 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.726 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.726 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.726 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.727 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.727 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.727 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.727 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.727 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.727 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.728 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.728 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.728 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.728 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.728 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.728 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.728 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.728 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.729 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.729 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.729 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.729 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.729 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.729 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.730 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.730 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.730 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.730 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.730 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.730 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.730 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.730 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.730 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.731 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.731 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.731 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.731 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.731 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.732 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.732 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.732 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.732 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.732 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.733 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.733 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-03-30T02:47:15.733 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-03-30T02:47:15.733 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.733 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.733 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.733 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.734 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.734 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.734 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.734 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-03-30T02:47:15.734 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-30T02:47:15.734 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.734 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.734 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-03-30T02:47:15.735 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-03-30T02:47:15.735 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.735 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.735 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.735 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-03-30T02:47:15.735 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.735 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.735 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.735 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-03-30T02:47:15.736 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.736 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.736 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-03-30T02:47:15.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-03-30T02:47:15.736 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.736 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-03-30T02:47:15.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.737 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.737 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.737 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-03-30T02:47:15.737 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.737 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.737 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.738 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-03-30T02:47:15.738 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-03-30T02:47:15.738 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.738 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.738 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.738 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.738 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.738 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.739 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-03-30T02:47:15.739 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.739 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.739 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.739 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-03-30T02:47:15.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-03-30T02:47:15.740 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.740 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.740 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.740 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-03-30T02:47:15.740 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.740 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.740 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.740 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-30T02:47:15.740 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-30T02:47:15.741 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.741 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.741 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.741 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-03-30T02:47:15.741 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.741 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.742 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.742 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.742 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.742 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.742 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.742 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.742 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.742 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.742 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.743 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.743 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.743 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.743 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.743 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.743 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.743 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.743 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.743 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.744 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.744 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.744 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.744 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.744 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.744 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.745 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.745 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.745 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.745 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.745 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.745 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.745 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.745 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.746 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.746 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.746 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.746 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.746 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.746 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.747 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.747 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.747 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.747 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.747 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.747 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.747 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.747 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.748 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.748 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.748 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.748 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.748 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.749 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.749 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.749 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.749 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.749 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.750 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.750 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.750 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-03-30T02:47:15.751 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.751 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.751 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.751 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.751 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.751 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.751 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.752 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.752 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.752 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.752 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.752 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.752 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.752 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.752 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.754 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.754 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.754 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.754 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.754 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.754 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.755 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.755 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.755 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.755 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.755 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.756 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.756 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.756 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.756 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.756 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.756 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.756 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.756 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.756 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.757 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.757 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.757 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.757 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.757 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.757 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.757 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.758 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.758 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.758 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.758 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.758 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.758 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.758 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.758 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.758 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.759 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.759 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.759 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.759 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.759 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.759 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.759 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.760 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-03-30T02:47:15.760 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.760 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.760 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.760 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.760 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.760 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.760 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.760 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.761 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.761 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.761 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.761 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.761 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.761 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.761 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.761 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.761 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.762 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.762 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.762 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.762 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.762 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.762 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.762 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.763 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.763 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.763 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.763 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.763 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.764 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.764 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.764 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.764 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.764 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.764 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.764 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.765 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.765 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.765 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.765 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.765 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.765 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.765 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.765 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.766 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.766 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.766 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.766 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.766 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.766 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.766 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.766 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.767 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.767 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.767 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.767 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.767 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.767 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.767 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.767 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.768 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.768 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.768 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.768 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.768 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.768 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-03-30T02:47:15.769 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.769 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.769 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.769 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-03-30T02:47:15.769 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-03-30T02:47:15.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.769 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.769 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.770 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.770 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.770 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.770 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-30T02:47:15.770 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.770 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.770 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.771 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.771 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.771 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.771 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.771 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.771 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.772 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.772 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.772 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.772 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.772 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.772 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.772 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.772 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.772 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.773 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.773 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.773 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.773 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.773 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.773 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.774 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.774 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.774 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.774 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.774 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.774 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.775 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.776 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.776 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.776 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.776 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.776 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.776 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.776 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.776 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.776 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.777 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.777 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.777 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.777 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.777 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.777 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.778 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.778 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.778 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.778 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.778 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.778 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.778 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.778 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.779 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-03-30T02:47:15.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-03-30T02:47:15.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.779 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.779 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.779 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.779 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.780 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.782 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.782 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.782 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.782 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.782 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.782 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.782 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.783 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.783 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.783 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.783 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.783 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.783 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.783 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.784 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.784 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.784 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.784 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.784 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.785 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.785 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.785 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.785 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.785 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.786 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.786 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.786 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.786 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.786 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.787 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.787 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.787 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.787 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.787 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.787 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.787 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.787 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.788 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.788 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.788 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.788 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.788 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.788 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.788 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.789 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.789 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.789 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.789 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.789 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.789 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.789 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.789 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.789 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.790 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.790 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.790 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.790 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-03-30T02:47:15.790 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.790 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.791 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.791 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.791 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.791 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.791 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.791 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.791 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.792 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.792 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.792 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.792 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.792 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.792 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.792 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.792 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.793 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.793 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.793 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.793 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.793 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.794 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.794 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.794 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.794 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.794 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.794 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.794 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.795 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.795 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.795 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.795 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.795 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.796 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.796 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.796 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.796 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.796 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.796 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.796 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.796 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.797 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.797 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.797 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.797 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.797 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.797 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.798 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.798 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.798 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.798 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.798 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.798 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.799 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.799 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.799 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.799 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.799 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.799 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.799 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-03-30T02:47:15.800 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.800 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.800 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.800 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.800 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.800 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.800 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.801 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.801 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.801 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.801 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.801 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.801 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.801 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.801 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.802 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.802 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.802 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.802 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.802 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.802 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.802 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.803 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.803 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.803 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.803 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.803 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.803 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.803 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.803 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.803 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.805 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.806 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.806 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.806 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.806 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.806 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.806 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.806 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.807 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.807 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.807 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.807 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.807 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.807 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.807 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.807 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.808 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.808 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.808 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.808 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:15.808 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:15.808 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.808 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.808 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.809 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:15.809 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.809 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.809 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.809 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.809 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.809 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.809 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.810 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-03-30T02:47:15.810 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-03-30T02:47:15.810 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:15.810 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.810 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.810 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-03-30T02:47:15.810 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-03-30T02:47:15.810 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.811 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.811 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-03-30T02:47:15.811 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-30T02:47:15.811 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.811 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.811 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.811 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-03-30T02:47:15.812 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.812 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.812 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.812 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-03-30T02:47:15.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-03-30T02:47:15.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.812 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.813 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.813 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.813 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.813 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.813 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.813 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-30T02:47:15.813 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.813 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.814 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-03-30T02:47:15.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-03-30T02:47:15.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.814 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.814 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.814 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.815 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.815 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.815 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.815 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.815 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.815 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.815 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.815 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.816 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.816 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.816 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.816 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.816 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.816 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.816 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.816 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.816 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.817 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.817 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.817 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.817 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.817 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.817 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.817 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.817 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.817 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.818 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.818 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.818 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.818 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.818 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.819 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.819 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.819 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.819 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.819 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.819 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.819 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.820 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.820 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.820 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.820 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.820 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.820 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.820 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.820 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.821 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.821 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.821 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.821 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.821 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.821 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.821 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.821 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.821 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.822 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.822 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.822 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.822 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.822 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.822 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.823 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.823 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.823 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.823 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.823 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.823 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.823 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.823 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-03-30T02:47:15.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-03-30T02:47:15.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.824 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.824 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.824 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-03-30T02:47:15.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.824 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.825 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-03-30T02:47:15.825 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.825 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.825 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.825 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:15.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:15.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.825 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.826 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.826 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:15.826 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.826 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.826 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.826 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-03-30T02:47:15.826 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.826 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.827 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.827 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.827 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-03-30T02:47:15.827 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.827 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.827 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.828 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-03-30T02:47:15.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-03-30T02:47:15.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.828 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.828 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.828 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.828 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.828 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.828 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.829 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.829 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.829 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.829 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.829 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-03-30T02:47:15.829 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-03-30T02:47:15.829 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.829 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.830 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.830 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.830 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.830 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.830 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.830 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-30T02:47:15.830 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-30T02:47:15.831 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.831 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.831 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.831 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.831 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.831 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.831 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.832 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.832 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.832 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.832 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.832 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.832 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.832 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.832 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.832 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.833 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.833 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.833 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.833 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.833 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.833 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.833 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.834 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.834 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.834 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.834 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.834 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.834 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.834 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.834 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.834 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.835 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.835 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.835 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.835 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.835 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.835 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.836 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.836 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.836 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.836 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.836 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.836 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.836 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.836 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.837 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.837 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.837 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.837 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.837 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.837 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.837 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.837 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.838 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.838 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.838 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.838 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.838 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.838 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.838 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.838 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.839 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.839 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.839 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.839 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-03-30T02:47:15.839 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-03-30T02:47:15.839 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.839 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.840 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.840 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.840 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.840 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.840 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.840 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.840 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.841 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-30T02:47:15.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-30T02:47:15.841 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.841 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.841 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.841 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-03-30T02:47:15.841 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.841 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.842 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.842 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.842 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.842 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.842 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.842 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-03-30T02:47:15.842 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-03-30T02:47:15.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:15.843 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.843 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.843 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-03-30T02:47:15.843 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.843 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.843 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.843 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.843 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-03-30T02:47:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-03-30T02:47:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.844 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-03-30T02:47:15.845 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.845 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.845 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.845 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-30T02:47:15.845 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.845 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.845 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.845 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-03-30T02:47:15.846 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-03-30T02:47:15.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.846 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.846 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.846 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-03-30T02:47:15.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.846 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.846 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.847 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-30T02:47:15.847 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.847 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.847 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.847 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.847 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-03-30T02:47:15.847 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-03-30T02:47:15.847 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.848 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.848 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.848 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-03-30T02:47:15.848 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.848 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.848 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.848 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.848 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.848 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.849 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.849 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-03-30T02:47:15.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-03-30T02:47:15.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.849 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.849 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.849 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-03-30T02:47:15.850 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.850 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.850 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.850 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-03-30T02:47:15.850 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-03-30T02:47:15.850 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.850 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.851 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.851 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.851 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.851 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-03-30T02:47:15.851 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-03-30T02:47:15.851 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.851 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.851 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.852 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-03-30T02:47:15.852 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-03-30T02:47:15.852 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-03-30T02:47:15.852 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.852 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.852 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.852 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.852 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.853 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.853 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-03-30T02:47:15.853 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.853 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.853 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.853 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.854 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.854 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.854 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.854 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.854 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.854 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.854 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.855 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.855 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.855 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.855 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.855 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.855 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.855 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.855 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.855 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.856 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.856 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.856 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.856 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.856 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.856 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.857 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.857 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.857 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.857 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.857 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.857 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.857 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.857 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.858 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.858 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.858 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.858 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.858 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.858 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.858 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.858 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.859 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.859 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.859 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.859 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.859 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.859 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.859 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.859 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.860 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.860 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.860 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.860 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.860 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.860 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.861 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.862 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-03-30T02:47:15.862 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-03-30T02:47:15.862 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.862 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.862 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.862 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:15.862 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.863 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.863 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.863 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.863 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.863 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.863 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.863 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-03-30T02:47:15.864 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-03-30T02:47:15.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.864 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.864 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.864 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.864 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.864 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.864 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.864 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.865 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.865 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.865 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.865 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-03-30T02:47:15.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-03-30T02:47:15.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.865 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.866 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.866 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.866 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.866 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.866 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.866 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.866 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.866 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.867 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.867 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-03-30T02:47:15.867 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.867 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.867 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.868 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-03-30T02:47:15.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-03-30T02:47:15.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.868 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.868 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.868 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.868 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.868 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.869 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.869 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.869 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.869 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.869 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.869 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-03-30T02:47:15.869 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-03-30T02:47:15.869 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.870 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.870 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.870 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.870 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.870 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.870 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.871 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.871 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.871 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.871 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.871 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.871 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.871 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.871 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.872 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.872 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.872 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.872 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.872 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.872 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.872 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.872 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.873 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.873 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.873 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.873 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.873 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.873 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.873 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.873 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.874 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.874 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.874 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.874 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.874 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.874 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.874 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.874 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.874 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.875 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.875 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.875 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.875 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.876 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.876 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.876 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.876 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.876 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.876 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.876 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.876 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.877 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.877 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.877 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.877 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.877 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.877 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.877 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.877 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.877 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.878 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.878 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.878 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.878 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.878 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.878 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.878 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.879 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.879 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.879 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.879 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.879 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.879 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.879 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-03-30T02:47:15.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-03-30T02:47:15.880 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.880 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.880 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.880 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.880 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.880 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.880 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.880 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.880 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.882 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.882 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.882 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.882 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.882 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.882 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.882 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.882 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-03-30T02:47:15.883 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.883 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.883 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.883 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.884 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.884 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.884 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.884 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.884 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.884 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.884 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.884 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.884 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.885 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.885 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.885 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.885 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.885 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.885 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.885 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.885 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.885 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.886 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.886 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.886 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.886 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.886 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.886 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.887 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.888 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.889 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.890 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.890 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.890 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.890 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.890 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.890 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.890 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.890 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.890 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.891 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.891 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.891 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.891 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.891 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.891 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-03-30T02:47:15.891 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.891 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.892 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.892 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.892 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.892 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.892 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.892 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.893 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.893 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.893 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.893 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.893 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.893 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.893 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.893 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.894 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.895 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.895 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.895 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.895 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.895 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.895 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.895 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.896 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.896 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.896 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.896 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.896 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.896 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.896 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.896 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.897 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.897 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.897 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.897 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.897 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.897 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.897 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.897 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.897 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.898 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.898 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.898 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.898 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.898 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.898 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.899 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.899 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.899 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.899 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.899 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.899 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.899 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.899 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.899 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.900 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.900 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.900 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-03-30T02:47:15.900 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-03-30T02:47:15.900 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.900 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.900 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.901 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-03-30T02:47:15.901 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.901 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.901 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.901 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-30T02:47:15.901 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.901 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.901 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.901 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.902 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.902 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-03-30T02:47:15.902 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.902 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.902 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.902 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.903 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.903 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.903 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.903 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.903 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.903 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.903 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.903 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.903 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.904 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.904 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.904 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.904 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.904 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.904 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.904 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.904 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.905 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.905 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.905 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.905 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.905 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.905 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.905 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.905 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.906 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.906 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.906 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.906 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.906 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.906 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.906 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.907 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.907 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.907 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.907 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.907 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.907 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.907 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.907 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.908 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.908 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.908 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.908 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.908 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.908 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.908 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.909 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.909 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.909 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.909 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.909 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.910 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.910 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.910 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.910 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.910 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.910 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.910 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.910 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.910 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.911 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-03-30T02:47:15.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-03-30T02:47:15.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.911 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.911 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.911 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.911 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.911 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.912 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.912 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.912 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.912 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.912 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.912 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-03-30T02:47:15.912 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-03-30T02:47:15.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.913 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.913 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.913 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-03-30T02:47:15.913 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.913 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.913 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.913 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.914 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.914 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.914 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.914 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-03-30T02:47:15.914 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-03-30T02:47:15.914 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.914 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.914 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.915 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.915 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.915 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.915 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.915 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.915 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.915 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.915 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-03-30T02:47:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-03-30T02:47:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.916 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.917 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.918 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.918 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.918 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.918 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.918 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.918 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.918 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.919 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-03-30T02:47:15.919 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-03-30T02:47:15.919 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.919 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.919 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.919 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.919 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-03-30T02:47:15.920 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.921 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.921 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.921 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.921 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.921 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.921 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.922 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-03-30T02:47:15.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-03-30T02:47:15.922 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.922 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.922 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.923 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.923 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.923 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.923 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.923 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.923 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.923 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.923 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.924 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-03-30T02:47:15.924 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-03-30T02:47:15.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.924 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.924 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.924 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.924 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:15.924 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-03-30T02:47:15.924 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.925 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.925 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.925 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.925 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.925 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-03-30T02:47:15.925 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-03-30T02:47:15.925 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.926 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.926 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.926 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.926 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.926 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.926 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.926 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.926 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.928 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.928 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.928 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.928 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.928 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.928 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.928 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-03-30T02:47:15.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-03-30T02:47:15.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.929 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.929 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.929 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.929 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.929 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.929 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.929 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.929 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.930 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.930 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.930 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.930 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-03-30T02:47:15.930 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.931 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.931 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.931 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-03-30T02:47:15.931 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-03-30T02:47:15.931 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.931 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.931 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.931 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.931 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-03-30T02:47:15.932 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.933 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.933 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.933 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.933 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.933 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.933 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.933 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-03-30T02:47:15.933 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.934 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.934 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.934 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.934 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.934 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.934 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.934 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.935 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.935 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.935 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.935 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.935 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.935 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.935 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.935 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.935 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.936 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.936 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.936 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.936 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.936 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.936 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.936 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.937 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.938 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.938 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.938 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.938 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.938 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.938 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.939 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.939 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.939 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.939 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.939 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.939 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.939 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.939 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.940 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.940 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.940 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.940 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.940 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.940 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.940 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.941 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.941 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.941 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.941 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.941 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.941 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.941 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.941 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.941 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-03-30T02:47:15.942 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.943 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.943 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.943 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.943 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.943 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.943 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.943 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.943 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.944 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.945 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.946 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.946 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.946 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.946 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.946 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.946 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.946 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.946 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.946 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.947 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.947 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.947 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.947 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.947 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.947 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.947 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.948 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.948 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.948 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.948 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.948 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.948 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.948 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.948 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.948 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.949 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.950 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.950 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.950 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.950 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.950 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.950 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.950 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.950 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.951 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.951 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.951 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.951 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.951 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.951 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-03-30T02:47:15.951 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.952 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.952 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.952 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.952 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.952 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.952 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.952 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.952 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.953 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.953 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.953 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.953 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.953 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.953 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.953 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.953 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.953 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.954 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.954 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.954 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.954 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.954 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.954 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.954 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.955 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.955 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.955 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.955 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.955 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.955 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.955 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.955 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-03-30T02:47:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-03-30T02:47:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-03-30T02:47:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-30T02:47:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-03-30T02:47:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-03-30T02:47:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-03-30T02:47:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-03-30T02:47:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-30T02:47:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-03-30T02:47:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-03-30T02:47:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-30T02:47:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-03-30T02:47:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-03-30T02:47:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-30T02:47:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-03-30T02:47:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-03-30T02:47:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-30T02:47:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-03-30T02:47:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-03-30T02:47:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-30T02:47:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-03-30T02:47:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-03-30T02:47:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-03-30T02:47:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-03-30T02:47:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-03-30T02:47:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-03-30T02:47:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-30T02:47:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-03-30T02:47:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-03-30T02:47:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-03-30T02:47:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-03-30T02:47:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-03-30T02:47:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-03-30T02:47:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-03-30T02:47:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-03-30T02:47:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-03-30T02:47:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-03-30T02:47:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-03-30T02:47:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-03-30T02:47:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-03-30T02:47:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-03-30T02:47:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-03-30T02:47:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-03-30T02:47:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-03-30T02:47:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-30T02:47:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-03-30T02:47:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-03-30T02:47:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-30T02:47:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-03-30T02:47:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-03-30T02:47:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-30T02:47:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-03-30T02:47:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-03-30T02:47:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-30T02:47:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-03-30T02:47:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-03-30T02:47:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-03-30T02:47:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-03-30T02:47:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-03-30T02:47:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-03-30T02:47:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-03-30T02:47:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-03-30T02:47:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-03-30T02:47:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-03-30T02:47:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-03-30T02:47:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-03-30T02:47:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-03-30T02:47:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-30T02:47:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 25631, 2024-03-30T02:47:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "y", 2024-03-30T02:47:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540138322906710015, 2024-03-30T02:47:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-03-30T02:47:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-03-30T02:47:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-03-30T02:47:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-03-30T02:47:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-03-30T02:47:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to", 2024-03-30T02:47:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-03-30T02:47:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-03-30T02:47:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-03-30T02:47:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-03-30T02:47:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-03-30T02:47:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-03-30T02:47:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-03-30T02:47:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-03-30T02:47:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-03-30T02:47:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-03-30T02:47:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-03-30T02:47:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-03-30T02:47:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-03-30T02:47:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-03-30T02:47:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-03-30T02:47:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-03-30T02:47:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-03-30T02:47:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-30T02:47:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-03-30T02:47:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-03-30T02:47:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-03-30T02:47:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-03-30T02:47:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-03-30T02:47:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-03-30T02:47:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-03-30T02:47:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-03-30T02:47:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-03-30T02:47:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-03-30T02:47:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-03-30T02:47:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T02:47:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-03-30T02:47:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-03-30T02:47:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-03-30T02:47:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-03-30T02:47:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-30T02:47:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T02:47:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-03-30T02:47:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-03-30T02:47:16.082 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.082 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.082 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.082 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-03-30T02:47:16.082 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:16.082 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-03-30T02:47:16.082 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.082 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-30T02:47:16.083 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.083 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.083 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.083 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-03-30T02:47:16.083 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-03-30T02:47:16.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.083 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.084 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.084 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-03-30T02:47:16.084 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.084 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.084 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.084 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-30T02:47:16.084 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-30T02:47:16.084 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.084 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.085 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-03-30T02:47:16.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-03-30T02:47:16.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.085 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.085 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.085 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-03-30T02:47:16.085 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-03-30T02:47:16.086 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-03-30T02:47:16.086 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.086 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-03-30T02:47:16.086 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-30T02:47:16.086 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.086 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.086 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-03-30T02:47:16.087 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-03-30T02:47:16.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.087 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.087 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.087 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-03-30T02:47:16.087 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.087 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.087 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.087 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-03-30T02:47:16.088 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T02:47:16.088 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.088 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.088 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-03-30T02:47:16.088 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-03-30T02:47:16.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.088 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.089 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.089 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.089 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-03-30T02:47:16.089 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-03-30T02:47:16.089 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.089 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-30T02:47:16.089 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T02:47:16.089 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.089 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.090 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.090 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.090 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.090 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.090 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.090 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.091 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.091 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.091 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.091 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.091 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.091 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.091 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.091 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.093 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.093 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.093 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.093 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.093 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.093 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.093 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.093 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.093 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.094 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.094 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.094 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.094 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.094 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.094 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.095 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.095 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.095 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.095 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.095 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.095 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.095 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.095 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.095 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.096 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.096 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.096 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.096 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.096 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.096 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.096 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.097 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.098 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.098 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.098 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.098 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-03-30T02:47:16.098 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-03-30T02:47:16.098 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.098 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-30T02:47:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.099 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.100 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-03-30T02:47:16.100 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-03-30T02:47:16.100 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.100 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.100 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.100 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-03-30T02:47:16.100 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.100 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.101 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.101 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-03-30T02:47:16.101 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-03-30T02:47:16.101 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap" 2024-03-30T02:47:16.101 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.101 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-03-30T02:47:16.101 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.101 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.102 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.102 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-03-30T02:47:16.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-03-30T02:47:16.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.102 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.102 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.102 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.103 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.103 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.103 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.103 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-30T02:47:16.103 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.103 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.103 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.103 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:16.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:16.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.104 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.104 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.104 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:16.104 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.104 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.104 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.104 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-30T02:47:16.105 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.105 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.105 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.105 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-03-30T02:47:16.105 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-03-30T02:47:16.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.105 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.105 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.106 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:16.106 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:16.106 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.106 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.106 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-30T02:47:16.106 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-30T02:47:16.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.106 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.107 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-03-30T02:47:16.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-03-30T02:47:16.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.107 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.107 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.107 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.107 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.108 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.108 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.108 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-30T02:47:16.108 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.108 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.108 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.109 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-03-30T02:47:16.109 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.109 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.109 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.109 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-03-30T02:47:16.109 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-03-30T02:47:16.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.110 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.110 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.110 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-03-30T02:47:16.110 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.110 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.110 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.110 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-30T02:47:16.110 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.111 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.111 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.111 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-03-30T02:47:16.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-03-30T02:47:16.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.111 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.111 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.112 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-03-30T02:47:16.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.112 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.112 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.112 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-30T02:47:16.112 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.112 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.112 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.113 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-03-30T02:47:16.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-03-30T02:47:16.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.113 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.113 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.113 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-03-30T02:47:16.113 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.113 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.114 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.114 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-30T02:47:16.114 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.114 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.114 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.114 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-03-30T02:47:16.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-03-30T02:47:16.115 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.115 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.115 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.115 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.115 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.115 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.115 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.115 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-30T02:47:16.116 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-30T02:47:16.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.116 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.116 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-03-30T02:47:16.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-03-30T02:47:16.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.116 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.117 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:16.117 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.117 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.117 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.117 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-30T02:47:16.117 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.117 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.117 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.118 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-03-30T02:47:16.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-03-30T02:47:16.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.118 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.118 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.118 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-03-30T02:47:16.118 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.119 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.119 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.119 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-30T02:47:16.119 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.119 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.119 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.119 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-03-30T02:47:16.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-03-30T02:47:16.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.120 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.120 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.120 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.120 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.120 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.121 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-30T02:47:16.121 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.121 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.121 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.121 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-03-30T02:47:16.121 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-03-30T02:47:16.121 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.122 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.122 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.122 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.122 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.122 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.122 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.122 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-30T02:47:16.122 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.123 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.123 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-03-30T02:47:16.123 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-03-30T02:47:16.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.123 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.123 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.123 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.124 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.124 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.124 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.124 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-30T02:47:16.124 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.124 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.124 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.125 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-03-30T02:47:16.125 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-03-30T02:47:16.125 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.125 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.125 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.125 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-30T02:47:16.125 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.125 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.125 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.126 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:16.126 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.126 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.126 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.126 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-03-30T02:47:16.126 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-03-30T02:47:16.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.127 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.127 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.127 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-30T02:47:16.127 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.127 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.127 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.127 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-03-30T02:47:16.127 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.127 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.128 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.128 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-03-30T02:47:16.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-03-30T02:47:16.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.128 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.128 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.128 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-30T02:47:16.128 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.129 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.129 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.129 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-03-30T02:47:16.129 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.129 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.129 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.129 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-03-30T02:47:16.130 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-03-30T02:47:16.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.130 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.130 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-30T02:47:16.130 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.130 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.130 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.130 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:16.131 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.131 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.131 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.131 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-03-30T02:47:16.131 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-03-30T02:47:16.131 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.131 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.132 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.132 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-30T02:47:16.132 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.132 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.132 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.132 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-03-30T02:47:16.132 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.132 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.132 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.133 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-03-30T02:47:16.133 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-03-30T02:47:16.133 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.133 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.133 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-30T02:47:16.133 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.133 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.134 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.134 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-03-30T02:47:16.134 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.134 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.134 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-03-30T02:47:16.134 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-03-30T02:47:16.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.135 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.135 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.135 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-30T02:47:16.135 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.135 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.135 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.135 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-03-30T02:47:16.135 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.136 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.136 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.136 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-03-30T02:47:16.136 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-03-30T02:47:16.136 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.136 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.136 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.136 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-30T02:47:16.137 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.137 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.137 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.137 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-03-30T02:47:16.137 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.137 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.137 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-03-30T02:47:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-03-30T02:47:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-30T02:47:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.138 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-03-30T02:47:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-03-30T02:47:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-03-30T02:47:16.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-30T02:47:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-03-30T02:47:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.141 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.141 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-03-30T02:47:16.141 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-03-30T02:47:16.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.141 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.141 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.141 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-30T02:47:16.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.142 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.142 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.142 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:16.142 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.142 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.142 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.143 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-03-30T02:47:16.143 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-03-30T02:47:16.143 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.143 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.143 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.143 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-30T02:47:16.143 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.143 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.143 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.144 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-03-30T02:47:16.144 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.144 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.144 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.144 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-03-30T02:47:16.144 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-03-30T02:47:16.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.145 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.145 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.145 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-30T02:47:16.145 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.145 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.145 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.145 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:16.145 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.145 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.146 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.146 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-03-30T02:47:16.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-03-30T02:47:16.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.146 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.146 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.146 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-30T02:47:16.147 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.147 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.147 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.147 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-03-30T02:47:16.147 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.147 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.147 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.147 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-03-30T02:47:16.148 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-03-30T02:47:16.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.148 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.148 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-30T02:47:16.148 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.148 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.148 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.148 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-03-30T02:47:16.149 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.149 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.149 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.149 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-03-30T02:47:16.149 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-03-30T02:47:16.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.149 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.149 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.150 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.150 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.150 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.150 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.150 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-03-30T02:47:16.150 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.150 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.150 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.151 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-03-30T02:47:16.151 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-03-30T02:47:16.151 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.151 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.151 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:16.151 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.151 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.152 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-03-30T02:47:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-30T02:47:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.153 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-03-30T02:47:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-03-30T02:47:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-03-30T02:47:16.154 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-03-30T02:47:16.155 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-03-30T02:47:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-03-30T02:47:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.156 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-03-30T02:47:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-03-30T02:47:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.157 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-03-30T02:47:16.158 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-03-30T02:47:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-03-30T02:47:16.159 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-03-30T02:47:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-03-30T02:47:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.160 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-03-30T02:47:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-03-30T02:47:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-03-30T02:47:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.162 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.163 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.163 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-30T02:47:16.163 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.163 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.163 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.163 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-03-30T02:47:16.163 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-03-30T02:47:16.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.164 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.164 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.164 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.164 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.164 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.164 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.164 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-30T02:47:16.164 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.164 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.165 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.165 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.165 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.165 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.165 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.166 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.166 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.166 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.166 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.166 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.166 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.166 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.166 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.167 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.167 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.167 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.167 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.167 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-03-30T02:47:16.167 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-03-30T02:47:16.167 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.167 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.168 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.168 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.168 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.168 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.168 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.168 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-30T02:47:16.168 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.168 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.168 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.169 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.169 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.169 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.169 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.169 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.169 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.169 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.170 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.170 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-30T02:47:16.170 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.170 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.170 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.170 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.170 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.170 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.171 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.171 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.171 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.171 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.171 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.171 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.171 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.171 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.171 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.172 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.172 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.172 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.172 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.172 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.172 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.172 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.172 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.173 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.173 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.173 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.173 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.173 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.173 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.173 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.173 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.174 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.174 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.174 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.174 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.174 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.174 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-30T02:47:16.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-30T02:47:16.175 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.175 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.175 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.175 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.175 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.175 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.175 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.175 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-30T02:47:16.175 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-30T02:47:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-30T02:47:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.176 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-03-30T02:47:16.177 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.177 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.177 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.177 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-30T02:47:16.177 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.177 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.177 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.177 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-03-30T02:47:16.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-03-30T02:47:16.178 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.178 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.178 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.178 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.178 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.178 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.178 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.179 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-03-30T02:47:16.179 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.179 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.179 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.179 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-03-30T02:47:16.179 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-03-30T02:47:16.179 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.179 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.180 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.180 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.180 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.180 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.180 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.180 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-30T02:47:16.180 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.180 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.181 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.181 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-03-30T02:47:16.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-03-30T02:47:16.181 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.181 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.181 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.181 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.182 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.182 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.182 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.182 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-03-30T02:47:16.182 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.182 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.182 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.183 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-03-30T02:47:16.183 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-03-30T02:47:16.183 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.183 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.183 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.183 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-03-30T02:47:16.183 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.183 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.184 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.184 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-03-30T02:47:16.184 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-03-30T02:47:16.184 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.184 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-03-30T02:47:16.184 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.184 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.184 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.185 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-03-30T02:47:16.185 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-03-30T02:47:16.185 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.185 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.185 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.185 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-03-30T02:47:16.185 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.185 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.185 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.186 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-30T02:47:16.186 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.186 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.186 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.186 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-03-30T02:47:16.186 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-03-30T02:47:16.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.187 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.187 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.187 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-30T02:47:16.187 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.187 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.187 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.187 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-30T02:47:16.187 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.187 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.188 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.189 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.189 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-30T02:47:16.189 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.189 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.189 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.189 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-03-30T02:47:16.189 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-03-30T02:47:16.189 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.190 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.190 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.190 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.190 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.190 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.190 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.190 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-30T02:47:16.190 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.190 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.191 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.191 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-03-30T02:47:16.191 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-03-30T02:47:16.191 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.191 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.191 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.191 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.191 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.192 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.192 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.192 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-30T02:47:16.192 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.192 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.192 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.192 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-03-30T02:47:16.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-03-30T02:47:16.193 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.193 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.193 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.193 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.193 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.193 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.193 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.193 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-30T02:47:16.194 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.194 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.194 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.194 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-03-30T02:47:16.194 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-03-30T02:47:16.194 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.194 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.194 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.195 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.195 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.195 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.195 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.195 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-30T02:47:16.195 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.195 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.195 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.196 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-03-30T02:47:16.196 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-03-30T02:47:16.196 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.196 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.196 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.196 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-03-30T02:47:16.196 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.196 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.196 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.197 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-03-30T02:47:16.197 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.197 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.197 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.197 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-03-30T02:47:16.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-03-30T02:47:16.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.198 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.198 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.198 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.198 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.198 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.198 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.198 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-30T02:47:16.198 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.199 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.200 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.200 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.200 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-30T02:47:16.200 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.200 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.200 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.200 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-03-30T02:47:16.200 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-03-30T02:47:16.201 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.201 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.201 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.201 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.201 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.201 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.201 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.201 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-30T02:47:16.202 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.202 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.202 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.202 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-03-30T02:47:16.202 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-03-30T02:47:16.202 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.202 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.202 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.203 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.203 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.203 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.203 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.203 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-03-30T02:47:16.203 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.203 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.203 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.204 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-03-30T02:47:16.204 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-03-30T02:47:16.204 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.204 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.204 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.204 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.204 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.205 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.205 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-30T02:47:16.205 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.205 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.205 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.205 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-03-30T02:47:16.205 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-03-30T02:47:16.205 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.206 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.206 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.206 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.206 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.206 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.206 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.206 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-30T02:47:16.206 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-03-30T02:47:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.207 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.208 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.208 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.208 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.208 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.208 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.208 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.208 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.208 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.208 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.209 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.209 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.209 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.209 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.209 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.209 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.209 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.209 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.209 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.210 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.210 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.210 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.210 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.210 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.210 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.210 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.211 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.211 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.211 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.211 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.211 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.211 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.211 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.211 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.211 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.212 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.212 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.212 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.212 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.212 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.212 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.212 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.213 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.214 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.214 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.214 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.214 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.214 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.214 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.214 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.215 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.215 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.215 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.215 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.215 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.215 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.215 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.215 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.216 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.217 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.217 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-30T02:47:16.217 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.217 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.217 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.217 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-03-30T02:47:16.217 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-03-30T02:47:16.217 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-03-30T02:47:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-30T02:47:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.218 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-03-30T02:47:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.219 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.220 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-30T02:47:16.220 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-30T02:47:16.220 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.220 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.220 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.220 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.220 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.220 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.220 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.221 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.221 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.221 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.221 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.221 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-03-30T02:47:16.221 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-30T02:47:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.222 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.223 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.223 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.223 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-30T02:47:16.223 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-30T02:47:16.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.223 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.223 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.224 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.224 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.224 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.224 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.224 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.224 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.224 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.224 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.225 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-03-30T02:47:16.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-03-30T02:47:16.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.225 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.225 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.225 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.225 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.225 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.226 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.226 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.226 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.226 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.226 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.226 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-30T02:47:16.226 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-30T02:47:16.226 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.227 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.227 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.227 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.227 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.227 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.227 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.227 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.227 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.227 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.228 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.228 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-03-30T02:47:16.228 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-30T02:47:16.228 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.228 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.228 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.228 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-30T02:47:16.229 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.230 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.230 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.230 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.230 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.230 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.230 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.230 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.230 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.231 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.231 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.231 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-30T02:47:16.231 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-30T02:47:16.231 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.231 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.231 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.231 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.232 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.232 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.232 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.232 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.232 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.232 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.232 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.233 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-30T02:47:16.233 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-30T02:47:16.233 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.233 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.233 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.233 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.233 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.233 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.233 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.234 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.234 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.234 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.234 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.234 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-30T02:47:16.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-30T02:47:16.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.235 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.236 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-30T02:47:16.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-30T02:47:16.236 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.236 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.236 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.236 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.236 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.237 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.237 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.237 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.237 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.237 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.237 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.237 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-30T02:47:16.237 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-30T02:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.238 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.239 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.239 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.239 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-30T02:47:16.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-30T02:47:16.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.239 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.239 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.239 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.240 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.240 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.240 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.240 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.240 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.240 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.240 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.240 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-30T02:47:16.241 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-30T02:47:16.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.241 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.241 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.241 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.241 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.241 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.241 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.241 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.242 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.242 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.242 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.242 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-03-30T02:47:16.242 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-30T02:47:16.242 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.242 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.243 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.243 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-03-30T02:47:16.243 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.243 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.243 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.243 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.243 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.243 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.243 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.244 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-30T02:47:16.244 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-30T02:47:16.244 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.244 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.244 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.244 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.244 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.244 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.245 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.245 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.245 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.245 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.245 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.245 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-03-30T02:47:16.245 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-03-30T02:47:16.246 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.246 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.246 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.246 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.246 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.246 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.246 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.246 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.246 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.247 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.247 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.247 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-03-30T02:47:16.247 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-30T02:47:16.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.247 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.247 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.248 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-03-30T02:47:16.248 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.248 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.248 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.248 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.248 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.248 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.248 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.249 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-30T02:47:16.249 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-30T02:47:16.249 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.249 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.249 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.249 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.249 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.249 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.250 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.250 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.250 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.250 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.250 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-30T02:47:16.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-30T02:47:16.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.251 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.251 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.251 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.251 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.251 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.251 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.251 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.251 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.251 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-30T02:47:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-30T02:47:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.252 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.253 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.253 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.253 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.253 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.253 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.253 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.253 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-30T02:47:16.253 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-30T02:47:16.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.254 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.254 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.254 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.254 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.254 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.254 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.254 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.255 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.255 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.255 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.255 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-03-30T02:47:16.255 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-30T02:47:16.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.255 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.255 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.256 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.256 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.256 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.256 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.256 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.256 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.256 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.256 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.257 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-30T02:47:16.257 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-30T02:47:16.257 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.257 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.257 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.257 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.257 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.257 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.258 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.258 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.258 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.258 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.258 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.258 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-30T02:47:16.258 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-30T02:47:16.259 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.259 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.259 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.259 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.259 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.259 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.259 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.259 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.259 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.260 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.260 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.260 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-30T02:47:16.260 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-30T02:47:16.260 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.260 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.260 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.260 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.261 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.261 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.261 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.261 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.261 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.261 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.261 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.261 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-30T02:47:16.262 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-30T02:47:16.262 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.262 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.262 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.262 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.262 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.262 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.262 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.263 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.263 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.263 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.263 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.263 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-30T02:47:16.263 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-30T02:47:16.263 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.264 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.264 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.264 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.264 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.264 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.264 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.264 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.264 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.265 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.265 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-30T02:47:16.265 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-30T02:47:16.265 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.265 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.265 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.265 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.265 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.266 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.266 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.266 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.266 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.266 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.266 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.266 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-30T02:47:16.266 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-30T02:47:16.267 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.267 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.267 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.267 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.267 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.267 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.267 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.267 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.267 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.268 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.268 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.268 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-30T02:47:16.268 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-30T02:47:16.268 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.268 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.268 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.269 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.269 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.269 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.269 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.269 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.269 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.269 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.269 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.270 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-03-30T02:47:16.270 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-30T02:47:16.270 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.270 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.270 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.270 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.270 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.270 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.271 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.271 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.271 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.271 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.271 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.271 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-30T02:47:16.271 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-30T02:47:16.271 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.272 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.272 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.272 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-30T02:47:16.272 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.272 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.272 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.272 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.272 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.272 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.273 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.273 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-30T02:47:16.273 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-30T02:47:16.273 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.273 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.273 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.273 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.273 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.274 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.274 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.274 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.274 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.274 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.274 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.274 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-30T02:47:16.275 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-30T02:47:16.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.275 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.275 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.275 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-03-30T02:47:16.275 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.275 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.275 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.275 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.276 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-03-30T02:47:16.277 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.277 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.277 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.277 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.277 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.277 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.277 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.277 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-03-30T02:47:16.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-30T02:47:16.278 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.278 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.278 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.278 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.278 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.278 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.278 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.279 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.279 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.279 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.279 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.279 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-30T02:47:16.279 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-30T02:47:16.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.280 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.280 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.280 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-03-30T02:47:16.280 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.280 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.280 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.280 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.280 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.280 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.281 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.281 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-03-30T02:47:16.281 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-30T02:47:16.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.281 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.281 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.281 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.281 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.282 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.282 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.282 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.282 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.282 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.282 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.282 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-03-30T02:47:16.283 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-30T02:47:16.283 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.283 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.283 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.283 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.283 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.283 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.283 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.283 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.284 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.284 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.284 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.284 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-03-30T02:47:16.284 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-30T02:47:16.284 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.284 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.285 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.285 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.285 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.285 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.285 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.285 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.285 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.285 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.285 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.286 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-30T02:47:16.286 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-30T02:47:16.286 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.286 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.286 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.286 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.286 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.287 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.287 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.287 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.287 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.287 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.287 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.287 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-30T02:47:16.287 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-30T02:47:16.288 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.288 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.288 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.288 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.288 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.288 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.288 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.288 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.288 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.289 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.289 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.289 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-30T02:47:16.289 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-30T02:47:16.289 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.289 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.289 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.290 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:16.290 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.290 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.290 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.290 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.290 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.290 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.291 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.291 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-03-30T02:47:16.291 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-03-30T02:47:16.291 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.291 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.291 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.291 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.292 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.292 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.292 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.292 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.292 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.292 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.292 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.292 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-03-30T02:47:16.293 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-03-30T02:47:16.293 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.293 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.293 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.293 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.293 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.293 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.293 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.294 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.294 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.294 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.294 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.294 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-03-30T02:47:16.294 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-03-30T02:47:16.294 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.295 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.295 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.295 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.295 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.295 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.295 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.295 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-03-30T02:47:16.295 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.296 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.296 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.296 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-03-30T02:47:16.296 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-03-30T02:47:16.296 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.296 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.296 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.296 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-03-30T02:47:16.297 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.297 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.297 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.297 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.297 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.297 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.297 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.298 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-03-30T02:47:16.298 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-03-30T02:47:16.298 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.298 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.298 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.298 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.298 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.298 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.299 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.299 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.299 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.299 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.299 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.299 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.300 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.300 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.300 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.300 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.300 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.300 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.300 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.301 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.302 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.302 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.302 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.302 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.302 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.302 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.302 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.303 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.303 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.303 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.303 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.303 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.303 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.303 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.304 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.304 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.304 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.304 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.304 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.304 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.304 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.304 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.305 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.306 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.306 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.306 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.306 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.306 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.306 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.306 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.306 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.307 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.307 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.307 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.307 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.307 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.307 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.307 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.307 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.307 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.308 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.308 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-03-30T02:47:16.308 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-03-30T02:47:16.308 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.308 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.308 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.309 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.309 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.309 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.309 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.309 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-03-30T02:47:16.309 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.309 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.309 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.310 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-03-30T02:47:16.310 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-03-30T02:47:16.310 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.310 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.310 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.310 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.310 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.310 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.311 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.311 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.311 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.311 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.311 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.311 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-03-30T02:47:16.311 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-03-30T02:47:16.312 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.312 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.312 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.312 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-03-30T02:47:16.312 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.312 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.312 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.312 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.313 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.313 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.313 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.313 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-03-30T02:47:16.313 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-03-30T02:47:16.313 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.313 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.313 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.314 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-03-30T02:47:16.314 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.314 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.314 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.314 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.314 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.314 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.314 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-03-30T02:47:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-03-30T02:47:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.315 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.316 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.316 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.316 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.316 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.316 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.316 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-03-30T02:47:16.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-03-30T02:47:16.316 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.317 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.317 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.317 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-03-30T02:47:16.317 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.317 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.317 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.317 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.317 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.318 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.318 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.318 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-03-30T02:47:16.318 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-03-30T02:47:16.318 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.318 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.318 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.318 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-03-30T02:47:16.319 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.319 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.319 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.319 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.319 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-03-30T02:47:16.319 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.319 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.319 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.320 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.320 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.320 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-03-30T02:47:16.320 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-03-30T02:47:16.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.320 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.320 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.320 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-03-30T02:47:16.321 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-03-30T02:47:16.321 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-03-30T02:47:16.321 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.321 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.321 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.321 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.321 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.322 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-03-30T02:47:16.322 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-03-30T02:47:16.322 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.322 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.322 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.322 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.322 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.322 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.322 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.323 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.323 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.323 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.323 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.323 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.323 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.324 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-03-30T02:47:16.324 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.324 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.324 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.324 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-03-30T02:47:16.324 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-03-30T02:47:16.324 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.324 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.324 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.325 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.325 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.325 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.325 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.325 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-03-30T02:47:16.325 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.325 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.325 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.326 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.326 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.326 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.326 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.326 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.326 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.326 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.326 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.327 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.327 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.327 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.327 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.327 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.327 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.327 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.327 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.327 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.328 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.328 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.328 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.328 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.328 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.328 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.328 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.328 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.329 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.329 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.329 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.329 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.329 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.329 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.329 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.330 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.330 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.330 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.330 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.330 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.330 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.330 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.331 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.331 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.331 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.331 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.331 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.331 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.331 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.331 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.332 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.332 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.332 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.332 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.332 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.332 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.332 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.333 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.333 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.333 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.333 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.333 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.333 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.333 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.333 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.334 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.334 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-03-30T02:47:16.334 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-03-30T02:47:16.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.334 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.334 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.334 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-03-30T02:47:16.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-03-30T02:47:16.336 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.336 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.336 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.336 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-03-30T02:47:16.336 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.336 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.336 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.336 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-30T02:47:16.336 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.337 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.337 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.337 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-03-30T02:47:16.337 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-03-30T02:47:16.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.337 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.337 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.337 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-03-30T02:47:16.338 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.338 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.338 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.338 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-03-30T02:47:16.338 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.338 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.338 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.338 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-03-30T02:47:16.339 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-03-30T02:47:16.339 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.339 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.339 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.339 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-03-30T02:47:16.339 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.339 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.339 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.340 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-30T02:47:16.340 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.340 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.340 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.340 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-03-30T02:47:16.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-03-30T02:47:16.340 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.340 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.341 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.341 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.341 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.341 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.341 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-30T02:47:16.341 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.341 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.341 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.342 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:16.342 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:16.342 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.342 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.342 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.342 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:16.342 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.342 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.343 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.343 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-03-30T02:47:16.343 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.343 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.343 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.343 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-03-30T02:47:16.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.344 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.345 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.345 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.345 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.345 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-03-30T02:47:16.345 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.345 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.345 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.346 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.346 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.346 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.346 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.346 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.346 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.347 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.347 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.347 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.347 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.347 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.347 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.347 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.347 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.348 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.348 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.348 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.348 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.348 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.348 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.348 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.348 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.349 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.350 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.351 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.351 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.351 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.351 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.351 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.351 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.351 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.351 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.352 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.352 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.352 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.352 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.352 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.352 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.353 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.353 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.353 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.353 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.353 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.353 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.353 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.354 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.354 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.354 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.354 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.354 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-03-30T02:47:16.354 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-03-30T02:47:16.354 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.355 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.355 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.355 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-03-30T02:47:16.355 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.355 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.355 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.355 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.355 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.356 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.356 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.356 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-03-30T02:47:16.356 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-03-30T02:47:16.356 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.356 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.356 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.356 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-03-30T02:47:16.357 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.357 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.357 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.357 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.357 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.357 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.357 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.358 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:16.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:16.358 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.358 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.358 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.358 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:16.358 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.358 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.359 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.359 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.359 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.359 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.359 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-03-30T02:47:16.360 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-03-30T02:47:16.360 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-03-30T02:47:16.360 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.360 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-03-30T02:47:16.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-03-30T02:47:16.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.360 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.361 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.361 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-03-30T02:47:16.361 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.361 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.361 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.361 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-30T02:47:16.361 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.361 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.362 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.362 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-03-30T02:47:16.362 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-03-30T02:47:16.362 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.362 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.362 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.362 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-03-30T02:47:16.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.363 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.363 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.363 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-30T02:47:16.363 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.363 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.363 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.363 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-03-30T02:47:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-03-30T02:47:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-03-30T02:47:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.366 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-03-30T02:47:16.366 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-03-30T02:47:16.366 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.366 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.366 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-30T02:47:16.366 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.366 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.366 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.367 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.367 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.367 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.367 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.367 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.367 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.368 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.368 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.368 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.368 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.368 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.368 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.368 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.368 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.369 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.369 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.369 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.369 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.369 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.369 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.369 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.369 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.370 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.370 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.370 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.370 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.370 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.370 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.370 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.370 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.371 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.371 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.371 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.371 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.371 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.371 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.371 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.371 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.372 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.373 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.373 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.373 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.373 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.373 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.373 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.373 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.373 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.374 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.374 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.374 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.374 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.374 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.374 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.376 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.376 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.376 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-03-30T02:47:16.376 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.376 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.376 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.376 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-03-30T02:47:16.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.377 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.378 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.378 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-03-30T02:47:16.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-03-30T02:47:16.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.378 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.378 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.378 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-03-30T02:47:16.379 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.379 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.379 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.379 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T02:47:16.379 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.379 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.379 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.379 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-03-30T02:47:16.380 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-03-30T02:47:16.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.380 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.380 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.380 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:16.380 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:16.380 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-03-30T02:47:16.380 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.380 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-30T02:47:16.381 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.381 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.381 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.381 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-03-30T02:47:16.381 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-03-30T02:47:16.381 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.381 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.381 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.382 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.382 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.382 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.382 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.382 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-03-30T02:47:16.382 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.382 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.382 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.383 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T02:47:16.384 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.384 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.384 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.384 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.384 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.384 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-03-30T02:47:16.384 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.385 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.386 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.386 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.386 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.386 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.386 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.386 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.386 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.386 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.386 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.387 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.388 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.388 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.388 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.388 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.388 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.388 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.388 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.388 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.389 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.389 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.389 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.389 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.389 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.389 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.389 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.389 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.389 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.390 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.391 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.391 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.391 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.391 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.391 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.391 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.391 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.391 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.392 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.392 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.392 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.392 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.392 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.392 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.392 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.392 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.393 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-03-30T02:47:16.393 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.393 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.393 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.393 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.393 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.394 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.394 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.394 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.394 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.394 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.394 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.394 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.394 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.394 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.395 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.396 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.396 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.396 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.396 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.396 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.396 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.396 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.396 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.397 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.397 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.397 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.397 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.397 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.397 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.398 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.398 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.398 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.398 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.398 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.398 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.398 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.398 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.399 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.399 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.399 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.399 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.399 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.399 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.399 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.399 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.400 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.400 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.400 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.400 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.400 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.400 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.400 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.400 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.401 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.401 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.401 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.401 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.401 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.401 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.401 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.401 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-03-30T02:47:16.402 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.402 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.402 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.402 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-03-30T02:47:16.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-03-30T02:47:16.402 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.403 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.403 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.403 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-03-30T02:47:16.403 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.403 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.403 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.403 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-30T02:47:16.403 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.404 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.404 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.404 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-03-30T02:47:16.404 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-03-30T02:47:16.404 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.404 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.404 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.404 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.405 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.406 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.406 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.406 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.406 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.406 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.406 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.406 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.407 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.407 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.407 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.407 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.407 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.407 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.407 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.407 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.407 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.408 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.408 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.408 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.408 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.408 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.409 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.409 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.409 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.409 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.409 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.409 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.409 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.410 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.410 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.410 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.410 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.410 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.410 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.410 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.410 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.410 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.411 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.411 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.411 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.411 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.411 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.411 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.411 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.411 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.412 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.412 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.412 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.412 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.412 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.412 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.412 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.412 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.413 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.413 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.413 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.413 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.413 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.413 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.413 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.413 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.414 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.414 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.414 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-03-30T02:47:16.414 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.414 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.414 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.414 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-03-30T02:47:16.415 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-03-30T02:47:16.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.415 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.415 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-03-30T02:47:16.415 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.415 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.415 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-03-30T02:47:16.416 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-30T02:47:16.416 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.416 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.416 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.416 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.416 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.417 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.417 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.417 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.417 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.417 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.417 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.417 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.417 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.417 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.418 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.418 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.418 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.418 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.418 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.418 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.418 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.419 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.419 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.419 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.419 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.419 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.419 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.420 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.420 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.420 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.420 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.420 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.420 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.420 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.421 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.421 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.421 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.421 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.421 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.421 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.421 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.421 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.422 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.422 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.422 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.422 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.422 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.422 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.422 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.422 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.423 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.423 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.423 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.423 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.423 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.423 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.423 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.424 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.424 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.424 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.424 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.424 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.425 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-03-30T02:47:16.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-03-30T02:47:16.425 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.425 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.425 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.425 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.425 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.425 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.425 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.426 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-03-30T02:47:16.426 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-30T02:47:16.426 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.426 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.426 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-03-30T02:47:16.426 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-03-30T02:47:16.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.427 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.427 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.427 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-03-30T02:47:16.427 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.427 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.427 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.427 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-03-30T02:47:16.427 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.428 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.428 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.428 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-03-30T02:47:16.428 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-03-30T02:47:16.428 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.428 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.428 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.428 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-03-30T02:47:16.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.429 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.429 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.429 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-03-30T02:47:16.429 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.429 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.429 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.430 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-03-30T02:47:16.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-03-30T02:47:16.430 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.430 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.430 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.430 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.430 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.430 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.431 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.432 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.432 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-03-30T02:47:16.432 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.432 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.432 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.432 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-30T02:47:16.432 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-30T02:47:16.432 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.432 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.433 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.433 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.433 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-03-30T02:47:16.433 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.433 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.433 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.434 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.434 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.434 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.434 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.434 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.434 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.434 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.434 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.435 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.435 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.435 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.435 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.435 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.435 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.435 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.435 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.435 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.436 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.436 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.436 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.436 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.436 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.436 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.437 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.437 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.437 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.437 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.437 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.437 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.437 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.437 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.438 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.438 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.438 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.438 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.438 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.438 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.438 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.438 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.439 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.439 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.439 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.439 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.439 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.439 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.439 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.440 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.440 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.440 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.440 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.440 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.440 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.440 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.441 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.441 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.441 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.441 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.441 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.441 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.441 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.441 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.442 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.442 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.442 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.442 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.442 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-03-30T02:47:16.443 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.443 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.443 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.443 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.443 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.443 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.443 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.444 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.444 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.444 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.444 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.444 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.444 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.444 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.444 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.444 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.445 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.446 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.446 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.446 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.446 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.446 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.446 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.446 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.447 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.447 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.447 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.447 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.447 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.447 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.448 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.448 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.448 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.448 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.448 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.448 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.448 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.448 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.448 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.449 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.450 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.450 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.450 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.450 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.450 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.450 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.450 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.450 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.451 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.451 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.451 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.451 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-03-30T02:47:16.451 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.451 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.451 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.452 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.452 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.452 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.452 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.452 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.452 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.452 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.452 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.452 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.453 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.453 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.453 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.453 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.453 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.453 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.453 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.453 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.453 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.454 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.454 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.454 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.454 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.454 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.455 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.456 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.457 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.457 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.457 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.457 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.457 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.457 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.457 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.457 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.457 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.458 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.458 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.458 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.458 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.458 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.458 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.458 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-03-30T02:47:16.460 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.460 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.460 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.460 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-03-30T02:47:16.460 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-03-30T02:47:16.460 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.460 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.460 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.460 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.461 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.461 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.461 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.461 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-30T02:47:16.461 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.461 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.461 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.461 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.462 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.462 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.462 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.462 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.462 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.462 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.462 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.463 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.463 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.463 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.463 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.463 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.463 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.463 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.463 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.463 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.464 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.464 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.464 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.464 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.464 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.464 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.465 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.465 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.465 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.465 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.465 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.465 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.465 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.465 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.466 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.466 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.466 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.466 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.466 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.466 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.467 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.468 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.468 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.468 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.468 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.468 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.468 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.468 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.469 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.469 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.469 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.469 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.469 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.469 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.469 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.469 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.469 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.470 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-03-30T02:47:16.470 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-03-30T02:47:16.470 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.470 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.470 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.470 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.470 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-03-30T02:47:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-03-30T02:47:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-03-30T02:47:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-03-30T02:47:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.472 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.472 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.472 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.472 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-03-30T02:47:16.472 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.472 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.472 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.473 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.474 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.474 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.474 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.474 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.474 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.474 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.474 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.474 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.475 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.475 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.475 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.475 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.475 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.475 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.476 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.476 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.476 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.476 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.476 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.476 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.476 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.476 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.477 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.477 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.477 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.477 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.477 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.478 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.478 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.478 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.478 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.478 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.478 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.478 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.478 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.479 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.479 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.479 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.479 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.479 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.479 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.480 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.480 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.480 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.480 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.480 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.480 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.480 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.480 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.481 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.481 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.481 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-03-30T02:47:16.481 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.481 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.482 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.482 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.482 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.482 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.482 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.482 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.482 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.483 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.483 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.483 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.483 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.483 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.483 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.483 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.483 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.484 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.484 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.484 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.484 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.484 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.484 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.484 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.485 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.485 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.485 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.485 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.485 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.485 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.485 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.485 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.485 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.486 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.486 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.486 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.486 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.486 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.486 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.486 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.487 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.488 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.488 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.488 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.488 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.488 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.488 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.488 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.489 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.489 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.489 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.489 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.489 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.489 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.489 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.489 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.489 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.490 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.490 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.490 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.490 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.490 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-03-30T02:47:16.490 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.491 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.491 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.491 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.491 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.491 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.491 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.491 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.491 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.491 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.492 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.492 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.492 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.492 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.492 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.492 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.492 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.492 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.492 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.493 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.493 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.493 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.493 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.493 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.493 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.493 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.494 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.494 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.494 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.494 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.494 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.494 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.494 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.494 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.495 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.495 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.495 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.495 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.495 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.495 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.495 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.495 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.496 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.496 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.496 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.496 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.496 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.496 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.496 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.496 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.496 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.497 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.497 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.497 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.497 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.497 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.497 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.497 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.498 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.499 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:16.499 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:16.499 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.499 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.499 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.499 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:16.499 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.499 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.500 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.500 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.500 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.500 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.500 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.500 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-03-30T02:47:16.500 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-03-30T02:47:16.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.501 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.501 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.501 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-03-30T02:47:16.501 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-03-30T02:47:16.501 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.501 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.501 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-03-30T02:47:16.501 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-30T02:47:16.501 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.502 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.502 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.502 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-03-30T02:47:16.502 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.502 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.503 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.503 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-03-30T02:47:16.503 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-03-30T02:47:16.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.503 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.503 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.503 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.503 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.503 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.504 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.504 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-30T02:47:16.504 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.504 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.504 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.504 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-03-30T02:47:16.504 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-03-30T02:47:16.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.505 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.505 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.505 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.505 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.505 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.505 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.505 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.505 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.505 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.506 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.506 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.506 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.506 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.506 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.506 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.507 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.507 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.507 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.507 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.507 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.507 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.507 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.507 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.508 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.508 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.508 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.508 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.508 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.508 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.508 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.509 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.509 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.509 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.509 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.509 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.509 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.509 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.509 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.510 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.510 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.510 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.510 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.510 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.510 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.510 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.510 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.511 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.511 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.511 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.511 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.511 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.511 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.511 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.511 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.513 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-03-30T02:47:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-03-30T02:47:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.514 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-03-30T02:47:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-03-30T02:47:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.515 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.516 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-03-30T02:47:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-03-30T02:47:16.517 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.518 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.518 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.518 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-03-30T02:47:16.518 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-03-30T02:47:16.518 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.518 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.518 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.518 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.519 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.519 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.519 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.519 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.519 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.519 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.519 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.519 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-03-30T02:47:16.520 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-03-30T02:47:16.520 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.520 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.520 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.520 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.520 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.520 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.520 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.521 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-30T02:47:16.521 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-30T02:47:16.521 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.521 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.521 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.521 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.521 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.521 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.522 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.522 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.522 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.522 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.522 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.522 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.522 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.522 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.523 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.523 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.523 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.523 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.523 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.523 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.523 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.523 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.524 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.524 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.524 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.524 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.524 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.524 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.524 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.525 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.525 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.525 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.525 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.525 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.525 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.525 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.526 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.526 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.526 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.526 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.526 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.526 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.526 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.526 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.527 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.527 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.527 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.527 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.527 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.527 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.527 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.527 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.527 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.528 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.528 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.528 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.528 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.528 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.528 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.528 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.528 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.529 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.529 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.529 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.529 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.529 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.529 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-03-30T02:47:16.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-03-30T02:47:16.530 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.530 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.530 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.530 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.530 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.530 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.530 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.530 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.531 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.531 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.531 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.531 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-30T02:47:16.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-30T02:47:16.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.531 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.532 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-03-30T02:47:16.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.532 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.532 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.532 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.532 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.532 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.532 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.533 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-03-30T02:47:16.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-03-30T02:47:16.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.533 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.533 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-03-30T02:47:16.533 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.533 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.534 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.534 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.534 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.534 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-03-30T02:47:16.534 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-03-30T02:47:16.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.535 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.535 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.535 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-03-30T02:47:16.535 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.535 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.535 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.535 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-30T02:47:16.535 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.536 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.536 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-03-30T02:47:16.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-03-30T02:47:16.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.536 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.536 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.537 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-03-30T02:47:16.537 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.537 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.537 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.537 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-30T02:47:16.537 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.537 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.537 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.538 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-03-30T02:47:16.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-03-30T02:47:16.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.538 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.538 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.538 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-03-30T02:47:16.538 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-03-30T02:47:16.539 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-03-30T02:47:16.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.540 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-03-30T02:47:16.540 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.540 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.540 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.540 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-03-30T02:47:16.541 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-03-30T02:47:16.541 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.541 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.541 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.541 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.541 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-03-30T02:47:16.541 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-03-30T02:47:16.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.542 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.542 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.542 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-03-30T02:47:16.542 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-03-30T02:47:16.542 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-03-30T02:47:16.542 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.542 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-03-30T02:47:16.543 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.544 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.544 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.544 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.544 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.544 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.544 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.545 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.545 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.545 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.545 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.545 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.545 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.545 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.546 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.546 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.546 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.546 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.546 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.547 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.547 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.547 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.547 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.547 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.547 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.547 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.547 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.547 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.548 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.548 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.548 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.548 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.548 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.548 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.549 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.549 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.549 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.549 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.549 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.549 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.549 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.549 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.549 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.550 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.550 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.550 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.550 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.550 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.550 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.551 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.551 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.551 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.551 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.551 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.551 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.551 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.551 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.552 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-03-30T02:47:16.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-03-30T02:47:16.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.552 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.552 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.552 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.553 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.553 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.553 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.553 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.553 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.553 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.553 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.553 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-03-30T02:47:16.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-03-30T02:47:16.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.554 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.554 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.554 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.554 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.554 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.554 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.555 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.555 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.555 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.555 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.555 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-03-30T02:47:16.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-03-30T02:47:16.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.555 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.556 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.556 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.556 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.556 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.556 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.556 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.556 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.556 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.557 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.557 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-03-30T02:47:16.557 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.557 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.557 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-03-30T02:47:16.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-03-30T02:47:16.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.558 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.558 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.558 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.558 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.558 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.558 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.559 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.559 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.559 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.559 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-03-30T02:47:16.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-03-30T02:47:16.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.559 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.560 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.560 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.560 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.560 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.560 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.560 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.560 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.561 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.561 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.561 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.561 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.561 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.562 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.562 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.562 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.562 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.562 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.562 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.562 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.562 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.563 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.563 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.563 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.563 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.563 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.564 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.564 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.564 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.564 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.564 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.565 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.565 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.565 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.565 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.565 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.566 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.566 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.566 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.566 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.566 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.566 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.566 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.566 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.568 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.568 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.568 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.568 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.568 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.568 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.568 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-03-30T02:47:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-03-30T02:47:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.569 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.570 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.570 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.570 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.570 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.570 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.570 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.570 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-03-30T02:47:16.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-03-30T02:47:16.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.571 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.571 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.571 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.571 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.571 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.571 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.572 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.572 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.572 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-03-30T02:47:16.573 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.573 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.573 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.573 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.573 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.573 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.573 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.574 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.574 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.574 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.574 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.574 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.574 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.574 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.574 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.575 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.575 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.575 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.575 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.575 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.575 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.575 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.575 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.576 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.576 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.576 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.576 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.576 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.576 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.576 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.576 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.578 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.578 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.578 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.578 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.578 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.578 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.578 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.578 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.579 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.579 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.579 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.579 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.580 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.580 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.580 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.580 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.580 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-03-30T02:47:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.582 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.583 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.584 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.585 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.586 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.587 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.589 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-03-30T02:47:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-03-30T02:47:16.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-03-30T02:47:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-30T02:47:16.591 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-03-30T02:47:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.592 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.593 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.594 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.595 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.596 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.597 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.598 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.598 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.598 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.598 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.598 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.598 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.598 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.598 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.599 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.599 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.599 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.599 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.599 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.599 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.600 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.600 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.600 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.600 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.600 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.600 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.600 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.600 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.601 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.601 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.601 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-03-30T02:47:16.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-03-30T02:47:16.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.601 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.601 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.602 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.602 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.602 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.602 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.602 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.603 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-03-30T02:47:16.603 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-03-30T02:47:16.603 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.603 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.603 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.603 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-03-30T02:47:16.603 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.603 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-03-30T02:47:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-03-30T02:47:16.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.605 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.605 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.605 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.605 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.605 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.605 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.605 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.605 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.605 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.606 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.606 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-03-30T02:47:16.606 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-03-30T02:47:16.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.606 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.606 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.606 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.607 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.607 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.607 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.607 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.607 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.607 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.607 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.607 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.608 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.609 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.609 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.609 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-03-30T02:47:16.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-03-30T02:47:16.609 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.609 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.609 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.610 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.610 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.610 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.610 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.610 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.610 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.610 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.610 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.611 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-03-30T02:47:16.611 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-03-30T02:47:16.611 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.611 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.611 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.611 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.611 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.611 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.612 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.612 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.612 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.612 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.612 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-03-30T02:47:16.612 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-03-30T02:47:16.612 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.613 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.613 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.613 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.613 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.613 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.613 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.613 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.614 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.614 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.614 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-03-30T02:47:16.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-03-30T02:47:16.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.614 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.614 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.615 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:16.615 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-03-30T02:47:16.615 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.615 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.615 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.615 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.616 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-03-30T02:47:16.616 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-03-30T02:47:16.616 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.616 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.616 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.616 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.616 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.616 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.616 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.617 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.617 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.617 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.617 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.617 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-03-30T02:47:16.617 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-03-30T02:47:16.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.617 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.618 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.618 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.618 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.618 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.618 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.618 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.619 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.619 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-03-30T02:47:16.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-03-30T02:47:16.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.619 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.620 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.620 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.620 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.620 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-03-30T02:47:16.621 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.621 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.621 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.621 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-03-30T02:47:16.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-03-30T02:47:16.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.622 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.622 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.622 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.622 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.622 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.622 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.622 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-03-30T02:47:16.622 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.623 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.623 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.623 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-03-30T02:47:16.623 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-03-30T02:47:16.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.623 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.623 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.623 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.624 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.624 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.624 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.624 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-03-30T02:47:16.624 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.624 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.624 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.624 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.625 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.626 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.626 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.626 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.626 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.626 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.626 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.626 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.626 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.627 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.627 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.627 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.627 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.627 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.627 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.628 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.628 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.628 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.628 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.628 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.628 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.628 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.629 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.630 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.630 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.630 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.630 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.630 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.630 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.630 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.630 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.631 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.631 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.631 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.631 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.631 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.631 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.631 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.631 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.632 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.632 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.632 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.632 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.632 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.632 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.632 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.633 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.633 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-03-30T02:47:16.633 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.633 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.633 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.633 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.633 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.633 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.634 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.634 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.634 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.634 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.634 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.634 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.634 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.634 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.634 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.635 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.635 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.635 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.635 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.635 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.635 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.635 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.635 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.636 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.636 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.636 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.636 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.637 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.638 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.638 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.638 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.638 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.638 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.638 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.638 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.638 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.638 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.639 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.639 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.639 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.639 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.639 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.639 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.639 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.639 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.640 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.640 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.640 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.640 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.640 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.640 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.640 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.640 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.640 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.641 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.641 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.641 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.641 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.641 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.641 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.642 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-03-30T02:47:16.642 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.642 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.642 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.642 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.642 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.642 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.643 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.643 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.643 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.643 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.643 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.643 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.643 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.643 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.643 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.645 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.645 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.645 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.645 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.645 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.646 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.646 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.646 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.646 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.646 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.646 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.647 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.647 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.647 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.647 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.647 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.647 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.647 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.648 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.649 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.649 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.649 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.649 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.649 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.650 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.650 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.650 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.650 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.650 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.650 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.650 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.651 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-03-30T02:47:16.651 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.651 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.651 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.651 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-03-30T02:47:16.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-03-30T02:47:16.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.652 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.652 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.652 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-30T02:47:16.652 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.652 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.652 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.652 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.653 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.653 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.653 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.653 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:16.653 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:16.653 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.653 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.653 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.654 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-03-30T02:47:16.654 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.654 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.654 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.654 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.654 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.654 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.655 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.655 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.655 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.655 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.655 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.655 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.655 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.656 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.656 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.656 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.656 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.656 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.656 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.656 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.656 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.656 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.657 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.657 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.657 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.657 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.657 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.657 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.657 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.658 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.658 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.658 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.658 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.658 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.658 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.658 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.658 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.658 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.659 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.659 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.659 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.659 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.659 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.660 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.660 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.660 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.660 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.660 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.660 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.660 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.660 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.661 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.661 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.661 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.661 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.661 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.661 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.662 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.662 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.662 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.662 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.662 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.662 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.662 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.662 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.662 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.663 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.663 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.663 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.663 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-03-30T02:47:16.663 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.664 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.664 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.664 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-03-30T02:47:16.664 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-03-30T02:47:16.664 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.664 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.664 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.664 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.664 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.665 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.665 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.665 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-30T02:47:16.665 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.665 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.665 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.665 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-03-30T02:47:16.666 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-03-30T02:47:16.666 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.666 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.666 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.666 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.666 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.666 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.666 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.667 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-30T02:47:16.667 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.667 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.667 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.667 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-03-30T02:47:16.667 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-03-30T02:47:16.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-30T02:47:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.669 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.669 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-03-30T02:47:16.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-03-30T02:47:16.669 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.669 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.670 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.670 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.670 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-30T02:47:16.670 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.670 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.670 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.670 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-03-30T02:47:16.671 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-03-30T02:47:16.671 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.671 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.671 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.671 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.671 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.671 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.671 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-30T02:47:16.672 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.672 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.672 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.672 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-03-30T02:47:16.672 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-03-30T02:47:16.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.672 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.673 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.673 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.673 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.673 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.673 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.673 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.673 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.673 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.673 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.674 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-03-30T02:47:16.674 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-03-30T02:47:16.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.674 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.674 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.674 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.674 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.674 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.675 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.675 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.675 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.675 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.675 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.675 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-03-30T02:47:16.675 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-03-30T02:47:16.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.676 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.676 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-30T02:47:16.676 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.676 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.676 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.676 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.676 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-03-30T02:47:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-03-30T02:47:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.677 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.678 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.678 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.678 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.678 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.678 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.678 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.679 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.680 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.680 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.680 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.680 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-03-30T02:47:16.680 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-03-30T02:47:16.680 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.680 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.680 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.681 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-03-30T02:47:16.682 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-03-30T02:47:16.682 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.682 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.682 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.682 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.683 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.683 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.683 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.683 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.683 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-03-30T02:47:16.683 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-03-30T02:47:16.683 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.683 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.684 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.684 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.684 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.684 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.684 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.684 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.684 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.684 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.685 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.685 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.685 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.685 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.685 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.685 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.685 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.686 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.686 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.686 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.686 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.686 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.686 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.686 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.686 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.687 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.687 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.687 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.687 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.687 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.687 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.687 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.687 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.688 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.688 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.688 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.688 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.688 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.688 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.688 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.688 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.688 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.689 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.689 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.689 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.689 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.689 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.689 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.690 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.690 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.690 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.690 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.690 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.690 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.690 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.690 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.691 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.691 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.691 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.691 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.691 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.691 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.691 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.691 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.692 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.692 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.692 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.692 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.692 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.692 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.692 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.692 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.692 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.693 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.693 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-03-30T02:47:16.693 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-03-30T02:47:16.693 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.693 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.693 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.693 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.694 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.694 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.694 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.694 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.694 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.694 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.694 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.694 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-03-30T02:47:16.695 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-03-30T02:47:16.695 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.695 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.695 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.695 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.695 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.695 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.695 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.695 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.696 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.696 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.696 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.696 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-03-30T02:47:16.696 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-03-30T02:47:16.696 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.696 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.696 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.697 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-30T02:47:16.697 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.697 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.697 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.697 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.697 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.697 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.697 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.698 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.698 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.698 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.698 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-03-30T02:47:16.698 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.698 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.698 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.698 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.699 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.699 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.699 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.699 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.699 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.699 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.700 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.700 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.700 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.700 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.700 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.700 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.700 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.700 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.701 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.701 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.701 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.701 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.701 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.701 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.702 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.702 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.702 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.702 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.702 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.702 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.702 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.703 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.703 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.703 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.703 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.703 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.703 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.703 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.704 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.704 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.704 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.704 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.704 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.704 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.704 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.704 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.704 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.705 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.705 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.705 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.705 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.705 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.705 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.706 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.706 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.706 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.706 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.706 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.706 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.706 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.706 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.707 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.707 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.707 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.707 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.707 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-03-30T02:47:16.707 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.707 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.707 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.708 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.708 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.708 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.708 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.708 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.708 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.708 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.708 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.708 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.709 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.709 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.709 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.709 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.709 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.709 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.709 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.709 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.710 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.711 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.711 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.711 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.711 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.711 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.711 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.712 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.712 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.712 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.712 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.712 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.712 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.712 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.713 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.713 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.713 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.713 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.713 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.713 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.713 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.713 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.714 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.714 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.714 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.714 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.714 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.714 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.714 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.715 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.715 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.715 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.715 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.715 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.715 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.715 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.715 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.716 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.716 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.716 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.716 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-03-30T02:47:16.716 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-03-30T02:47:16.716 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.716 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.716 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.717 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-03-30T02:47:16.717 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.717 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.717 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.717 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-30T02:47:16.717 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.717 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.717 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.718 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-03-30T02:47:16.718 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-03-30T02:47:16.718 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.718 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.718 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.718 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.718 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.719 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.719 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-30T02:47:16.719 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.719 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.719 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.719 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-03-30T02:47:16.719 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-03-30T02:47:16.720 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.720 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.720 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.720 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.720 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.720 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.720 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-30T02:47:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-03-30T02:47:16.721 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.722 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.722 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.722 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-03-30T02:47:16.722 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-03-30T02:47:16.722 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.722 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.722 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.722 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-03-30T02:47:16.723 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.723 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.723 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.723 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.723 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.723 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.723 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.724 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-03-30T02:47:16.724 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-03-30T02:47:16.724 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.724 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.724 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.724 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.724 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.724 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.724 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.725 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.725 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.725 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.725 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.725 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:16.725 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:16.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.726 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.726 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.726 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:16.726 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.726 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.726 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.726 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.726 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.727 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.727 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.727 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.727 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.727 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.728 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.728 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.728 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.728 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.728 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.728 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.728 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.728 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.729 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.730 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.730 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.730 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.730 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.730 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.730 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.730 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.731 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.731 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.731 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.731 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.731 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.731 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.732 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.732 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.732 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.732 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.732 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.732 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.733 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.733 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.733 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.733 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.733 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.733 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.733 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.734 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.734 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.734 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.734 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.734 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.734 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.734 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.734 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.734 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.735 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.735 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.735 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.735 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.735 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-03-30T02:47:16.735 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-03-30T02:47:16.735 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.735 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.736 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.736 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.736 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.736 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.736 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.736 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.736 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.737 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.737 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-03-30T02:47:16.737 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-03-30T02:47:16.737 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.737 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-03-30T02:47:16.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.738 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.738 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.738 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.738 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.738 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.738 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-03-30T02:47:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-03-30T02:47:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-30T02:47:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.739 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.740 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.740 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.740 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.740 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:16.740 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.741 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.741 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-03-30T02:47:16.741 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-03-30T02:47:16.741 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard", 2024-03-30T02:47:16.741 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-03-30T02:47:16.741 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-03-30T02:47:16.741 INFO:teuthology.orchestra.run.smithi067.stdout: "restful", 2024-03-30T02:47:16.741 INFO:teuthology.orchestra.run.smithi067.stdout: "snap_schedule" 2024-03-30T02:47:16.741 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.742 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-03-30T02:47:16.742 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.742 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-03-30T02:47:16.742 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.742 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.742 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.742 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:16.742 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:16.743 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.743 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.743 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.743 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:16.743 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.743 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.743 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.743 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-03-30T02:47:16.743 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.744 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.744 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.744 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.744 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.744 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.744 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.744 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.745 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.745 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.745 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.745 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.745 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.745 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.745 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.745 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.745 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.746 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.746 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.746 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.746 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.746 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.746 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.746 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.747 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.747 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.747 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.747 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.747 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.747 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.747 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.747 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.747 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.748 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.748 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.748 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.748 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.748 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.749 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.749 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.749 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.749 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.749 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.749 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.749 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.750 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.750 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.750 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.750 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.750 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.750 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.751 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.752 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.752 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-03-30T02:47:16.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-03-30T02:47:16.752 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.752 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.752 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.752 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.752 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.753 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.753 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.753 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to", 2024-03-30T02:47:16.753 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.753 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.753 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.753 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-03-30T02:47:16.754 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-03-30T02:47:16.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.754 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.754 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.754 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-03-30T02:47:16.754 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.754 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.755 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-03-30T02:47:16.755 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.755 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.755 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.755 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-03-30T02:47:16.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-03-30T02:47:16.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.756 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.756 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.756 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.756 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.756 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.756 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.756 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-03-30T02:47:16.756 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.756 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.757 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.757 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-03-30T02:47:16.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-03-30T02:47:16.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.757 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.757 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.757 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.757 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.758 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.758 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.758 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-03-30T02:47:16.758 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.758 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.758 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.758 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-03-30T02:47:16.759 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-03-30T02:47:16.759 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.759 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.759 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.759 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-03-30T02:47:16.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.759 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.759 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.759 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-03-30T02:47:16.760 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.760 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.760 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.760 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-03-30T02:47:16.760 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-03-30T02:47:16.760 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.760 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.761 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.761 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.761 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.761 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.761 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.761 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-03-30T02:47:16.761 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.761 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.762 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-03-30T02:47:16.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-03-30T02:47:16.762 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.762 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.762 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.762 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.763 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.763 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.763 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-30T02:47:16.763 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.763 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.763 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.763 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-03-30T02:47:16.763 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-03-30T02:47:16.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.764 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.764 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.764 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.764 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.764 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.764 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.764 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-03-30T02:47:16.765 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.765 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.765 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.765 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.765 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-03-30T02:47:16.765 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.766 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.766 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.766 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-03-30T02:47:16.766 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-03-30T02:47:16.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.766 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.766 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.766 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.767 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.767 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.767 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.767 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-03-30T02:47:16.767 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.767 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.767 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.767 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-03-30T02:47:16.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-03-30T02:47:16.768 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.768 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.768 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.768 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-03-30T02:47:16.768 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.768 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.768 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.769 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-03-30T02:47:16.769 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T02:47:16.769 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.769 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.769 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-03-30T02:47:16.769 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-03-30T02:47:16.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.769 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.770 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.770 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-03-30T02:47:16.770 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-03-30T02:47:16.770 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.770 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-30T02:47:16.770 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T02:47:16.770 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.770 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.771 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-03-30T02:47:16.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-03-30T02:47:16.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.771 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.771 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.771 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-03-30T02:47:16.771 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:16.772 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-03-30T02:47:16.772 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.772 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-30T02:47:16.772 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.772 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.772 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.772 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-03-30T02:47:16.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-03-30T02:47:16.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.773 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.773 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.773 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-03-30T02:47:16.773 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.773 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.773 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.773 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-30T02:47:16.774 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-30T02:47:16.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.774 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.774 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-03-30T02:47:16.774 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-03-30T02:47:16.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.774 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.774 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.775 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-03-30T02:47:16.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-03-30T02:47:16.775 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-03-30T02:47:16.775 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.775 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-03-30T02:47:16.775 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-30T02:47:16.775 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.775 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.776 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-03-30T02:47:16.776 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-03-30T02:47:16.776 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.776 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.776 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.776 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-03-30T02:47:16.776 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.776 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.777 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.777 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-03-30T02:47:16.777 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-30T02:47:16.777 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.777 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.777 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-03-30T02:47:16.777 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-03-30T02:47:16.777 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.778 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.778 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.778 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.778 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-03-30T02:47:16.778 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-03-30T02:47:16.778 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.778 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-30T02:47:16.778 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-30T02:47:16.779 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.779 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.779 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.779 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.779 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.779 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.780 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.780 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.780 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.780 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.780 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.780 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.780 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.780 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.781 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.781 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.781 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.781 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.781 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.781 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.781 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.782 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.782 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.782 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.782 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.782 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.782 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.782 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.782 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.783 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.783 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.783 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.783 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.783 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.783 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.784 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.784 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.784 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.784 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.784 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.785 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.785 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.785 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.785 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.785 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.785 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.785 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.785 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.786 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.786 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.786 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.786 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.786 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.787 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.787 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.787 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.787 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.787 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.787 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-03-30T02:47:16.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-03-30T02:47:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-30T02:47:16.788 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.789 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.789 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.789 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-03-30T02:47:16.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-03-30T02:47:16.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.789 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.789 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.790 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-03-30T02:47:16.790 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.790 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.790 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.790 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-03-30T02:47:16.790 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-03-30T02:47:16.790 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap" 2024-03-30T02:47:16.790 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.791 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-03-30T02:47:16.791 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.791 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.791 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-03-30T02:47:16.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-03-30T02:47:16.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.791 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.792 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.792 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.792 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.792 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.792 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.792 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-30T02:47:16.792 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.792 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.792 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.793 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:16.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:16.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.793 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.793 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.793 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:16.793 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.793 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.794 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-30T02:47:16.794 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.794 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.794 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.794 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-03-30T02:47:16.794 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-03-30T02:47:16.795 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.795 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.795 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.795 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:16.795 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:16.795 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.795 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.795 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-30T02:47:16.795 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-30T02:47:16.796 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.796 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.796 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-03-30T02:47:16.796 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-03-30T02:47:16.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:16.796 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.796 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.797 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.797 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.797 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.797 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.797 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-30T02:47:16.797 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.797 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.797 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.798 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-03-30T02:47:16.798 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.798 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.798 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.798 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-03-30T02:47:16.799 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-03-30T02:47:16.799 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.799 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.799 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.799 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-03-30T02:47:16.799 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.799 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.799 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.799 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-30T02:47:16.800 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.800 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.800 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.800 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-03-30T02:47:16.800 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-03-30T02:47:16.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.800 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.800 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.801 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-03-30T02:47:16.801 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.801 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.801 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.801 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-30T02:47:16.801 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.801 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.801 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.802 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-03-30T02:47:16.802 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-03-30T02:47:16.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.802 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.802 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.802 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-03-30T02:47:16.802 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.802 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.803 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.803 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-30T02:47:16.803 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.803 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.803 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.803 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-03-30T02:47:16.803 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-03-30T02:47:16.803 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.804 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.804 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.804 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.804 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.804 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.804 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.804 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-30T02:47:16.804 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-30T02:47:16.805 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.805 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.805 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-03-30T02:47:16.805 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-03-30T02:47:16.805 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.805 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.805 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.805 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:16.806 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.806 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.806 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.806 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-30T02:47:16.806 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.806 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.806 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.806 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-03-30T02:47:16.807 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-03-30T02:47:16.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:16.807 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.807 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.807 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-03-30T02:47:16.807 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.807 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.807 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.807 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-30T02:47:16.808 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.808 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.808 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.808 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-03-30T02:47:16.808 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-03-30T02:47:16.808 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.809 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.809 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.809 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.809 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.809 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.809 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.809 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-30T02:47:16.809 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.809 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.810 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.810 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-03-30T02:47:16.810 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-03-30T02:47:16.810 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.810 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.810 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.810 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.810 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.811 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.811 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.811 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-30T02:47:16.811 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.811 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.811 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.811 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-03-30T02:47:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-03-30T02:47:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.812 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-30T02:47:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-03-30T02:47:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-03-30T02:47:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.813 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-30T02:47:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.814 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-03-30T02:47:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-03-30T02:47:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.815 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-30T02:47:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-03-30T02:47:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.816 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-03-30T02:47:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-03-30T02:47:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-30T02:47:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.817 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-03-30T02:47:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-03-30T02:47:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-03-30T02:47:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.818 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-03-30T02:47:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-03-30T02:47:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-03-30T02:47:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-30T02:47:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.820 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-03-30T02:47:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-03-30T02:47:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-03-30T02:47:16.821 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-30T02:47:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-03-30T02:47:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.822 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-03-30T02:47:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-03-30T02:47:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.823 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-30T02:47:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-03-30T02:47:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.824 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-03-30T02:47:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-03-30T02:47:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-30T02:47:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.825 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-03-30T02:47:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-03-30T02:47:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-03-30T02:47:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.826 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-30T02:47:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-03-30T02:47:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.827 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.828 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-03-30T02:47:16.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-03-30T02:47:16.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.828 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.828 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.828 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-30T02:47:16.828 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.829 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.829 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-03-30T02:47:16.829 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.829 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.829 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.829 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-03-30T02:47:16.829 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-03-30T02:47:16.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.830 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.830 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.830 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-30T02:47:16.830 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.830 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.830 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.830 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:16.831 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.831 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.831 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.831 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-03-30T02:47:16.831 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-03-30T02:47:16.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.831 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.832 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.832 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-30T02:47:16.832 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.832 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.832 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.832 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-03-30T02:47:16.832 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.832 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.833 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.833 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-03-30T02:47:16.833 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-03-30T02:47:16.833 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.833 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.833 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.833 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-30T02:47:16.833 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.834 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.834 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.834 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-03-30T02:47:16.834 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.834 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.834 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.834 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-03-30T02:47:16.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-03-30T02:47:16.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.835 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.835 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.835 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-30T02:47:16.835 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.835 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.835 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.836 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-03-30T02:47:16.836 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.836 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.836 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.836 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-03-30T02:47:16.836 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-03-30T02:47:16.836 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.837 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.837 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.837 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-30T02:47:16.837 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.837 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.837 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.837 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-03-30T02:47:16.838 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.838 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.838 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.838 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-03-30T02:47:16.838 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-03-30T02:47:16.838 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.838 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.839 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.839 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.839 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.839 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.839 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-03-30T02:47:16.839 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.839 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.839 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.840 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-03-30T02:47:16.840 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-03-30T02:47:16.840 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.840 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.840 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.840 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:16.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.841 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.841 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.841 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-30T02:47:16.841 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.841 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.841 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.841 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-03-30T02:47:16.842 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-03-30T02:47:16.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.842 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.842 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.842 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-03-30T02:47:16.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.842 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.842 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.842 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-30T02:47:16.843 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.843 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.843 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.843 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-03-30T02:47:16.843 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-03-30T02:47:16.843 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.843 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.844 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.844 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-03-30T02:47:16.844 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.844 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.844 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.844 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-30T02:47:16.844 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.844 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.844 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.845 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-03-30T02:47:16.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-03-30T02:47:16.845 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.845 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.845 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.845 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-03-30T02:47:16.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.846 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.846 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.846 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-03-30T02:47:16.846 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.846 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.846 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.846 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-03-30T02:47:16.847 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-03-30T02:47:16.847 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.847 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.847 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.847 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.847 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.847 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.847 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.848 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-30T02:47:16.848 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.848 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.848 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.848 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-03-30T02:47:16.848 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-03-30T02:47:16.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.849 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.849 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.849 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:16.849 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.849 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.849 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.849 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-03-30T02:47:16.849 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.849 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.850 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.850 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-03-30T02:47:16.850 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-03-30T02:47:16.850 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.850 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.850 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.850 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:16.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.851 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.851 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.851 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-03-30T02:47:16.851 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.851 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.851 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.851 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-03-30T02:47:16.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-03-30T02:47:16.852 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.852 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.852 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.852 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.852 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.852 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.852 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.853 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-30T02:47:16.853 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.853 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.853 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.853 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-03-30T02:47:16.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-03-30T02:47:16.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.853 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.854 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.854 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.854 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.854 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.854 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.854 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-30T02:47:16.854 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.854 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.855 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.855 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.855 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.855 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.855 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.855 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.855 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.856 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.856 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.856 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.856 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.856 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.856 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.856 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.856 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.856 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.857 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.857 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.857 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.857 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.857 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.857 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-03-30T02:47:16.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-03-30T02:47:16.857 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.858 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.858 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.858 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.858 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.858 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.858 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.858 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-30T02:47:16.859 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.859 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.859 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.859 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.859 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.859 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.859 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.859 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.860 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.860 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.860 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.860 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.860 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-30T02:47:16.860 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.860 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.860 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.861 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.861 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.861 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.861 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.861 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.861 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.861 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.862 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.862 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.862 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.862 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.862 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.862 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.862 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.862 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.863 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.863 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.863 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.863 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.863 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.863 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.863 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.863 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.864 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.864 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.864 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.864 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.864 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.864 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.864 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.865 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.865 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-30T02:47:16.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-30T02:47:16.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.865 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.865 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.865 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.865 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.866 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.866 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.866 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-30T02:47:16.866 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.866 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.866 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.866 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-30T02:47:16.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-30T02:47:16.867 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.867 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.867 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.867 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-03-30T02:47:16.867 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.867 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.867 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.868 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-30T02:47:16.868 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.868 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.868 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.868 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-03-30T02:47:16.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-03-30T02:47:16.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.868 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.869 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.869 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.869 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.869 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.869 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.869 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-03-30T02:47:16.869 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.869 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.869 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-03-30T02:47:16.870 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-03-30T02:47:16.870 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.870 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.870 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.870 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.870 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.871 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.871 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.871 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-30T02:47:16.871 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.871 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.871 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.871 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-03-30T02:47:16.871 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-03-30T02:47:16.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.872 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.872 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.872 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.872 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.872 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.872 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.872 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-03-30T02:47:16.873 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.873 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.873 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.873 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-03-30T02:47:16.873 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-03-30T02:47:16.873 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.873 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.873 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.874 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-03-30T02:47:16.874 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.874 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.874 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.874 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-03-30T02:47:16.874 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-03-30T02:47:16.874 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.874 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-03-30T02:47:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-03-30T02:47:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-03-30T02:47:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.876 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-03-30T02:47:16.876 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.876 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.876 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.876 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-30T02:47:16.876 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.876 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.876 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.877 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-03-30T02:47:16.877 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-03-30T02:47:16.877 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.877 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.877 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.877 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-30T02:47:16.877 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.877 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.877 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.878 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-30T02:47:16.878 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.878 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.878 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.878 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-03-30T02:47:16.878 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-03-30T02:47:16.878 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.879 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.879 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.879 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.879 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.879 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.879 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.879 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-30T02:47:16.879 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.879 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-03-30T02:47:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-03-30T02:47:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.880 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.881 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.881 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.881 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-30T02:47:16.881 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.881 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.881 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.881 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-03-30T02:47:16.882 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-03-30T02:47:16.882 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.882 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.882 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.882 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.882 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.882 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.882 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.882 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-30T02:47:16.883 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.883 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.883 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-03-30T02:47:16.883 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-03-30T02:47:16.883 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.883 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.884 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.884 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.884 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.884 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.884 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.884 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-30T02:47:16.884 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.884 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.884 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.885 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-03-30T02:47:16.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-03-30T02:47:16.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.885 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.885 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.885 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.885 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.886 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.886 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.886 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-30T02:47:16.886 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.886 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.886 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.886 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-03-30T02:47:16.887 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-03-30T02:47:16.887 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.887 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.887 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.887 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-03-30T02:47:16.887 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.887 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.887 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.887 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.888 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.889 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.889 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.889 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.889 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-30T02:47:16.889 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.889 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.889 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.889 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-03-30T02:47:16.890 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-03-30T02:47:16.890 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.890 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.890 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.890 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.890 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.890 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.890 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.891 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-30T02:47:16.891 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.891 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.891 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.891 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-03-30T02:47:16.891 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-03-30T02:47:16.891 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.891 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.892 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.892 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.892 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.892 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.892 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-30T02:47:16.892 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.892 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.892 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.893 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-03-30T02:47:16.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-03-30T02:47:16.893 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.893 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.893 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.893 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.893 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.893 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.894 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.894 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-03-30T02:47:16.894 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.894 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.894 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.894 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-03-30T02:47:16.894 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-03-30T02:47:16.895 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.895 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.895 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.895 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.895 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.895 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.895 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.895 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-30T02:47:16.896 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.896 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.896 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.896 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-03-30T02:47:16.896 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-03-30T02:47:16.896 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.896 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.897 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.897 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.897 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.897 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.897 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.897 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-30T02:47:16.897 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.897 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.897 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.898 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.898 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.898 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-03-30T02:47:16.898 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.898 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.898 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.899 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.899 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.899 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.899 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.899 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.899 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.899 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.899 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.899 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.900 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.900 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.900 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.900 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.900 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.900 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.900 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.900 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.901 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.901 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.901 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.901 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.901 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.901 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.901 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.901 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.902 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.902 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.902 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.902 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.902 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.902 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.902 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.902 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.903 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.903 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.903 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.903 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.903 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.903 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.903 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.903 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.904 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.904 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.904 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.904 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.904 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.904 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.904 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.904 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.904 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.905 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.905 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.905 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.905 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.905 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.905 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.905 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.905 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.906 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.906 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.906 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.906 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.906 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.906 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.906 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.906 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.907 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-03-30T02:47:16.907 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-03-30T02:47:16.907 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.907 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.907 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.907 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-03-30T02:47:16.907 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.908 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-30T02:47:16.908 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.908 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.908 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.908 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-03-30T02:47:16.908 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-03-30T02:47:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-03-30T02:47:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-30T02:47:16.909 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-03-30T02:47:16.910 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-30T02:47:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-30T02:47:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.911 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.912 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.912 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.912 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.912 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.912 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.912 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.912 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-03-30T02:47:16.913 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-30T02:47:16.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.913 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.913 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.913 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.913 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.913 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.913 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.913 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.914 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.914 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.914 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.914 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-30T02:47:16.914 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-30T02:47:16.914 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.914 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.915 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.915 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.915 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.915 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.915 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.915 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.915 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.915 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.916 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.916 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-03-30T02:47:16.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-03-30T02:47:16.916 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.916 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.916 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.916 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.917 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.917 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.917 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.917 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.917 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.917 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.917 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.917 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-30T02:47:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-30T02:47:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.918 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.919 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.919 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.919 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.919 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-03-30T02:47:16.919 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-30T02:47:16.919 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.919 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.919 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.920 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.920 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.920 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.920 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.920 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.920 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.920 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.920 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.921 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-30T02:47:16.921 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-30T02:47:16.921 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.921 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.921 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.921 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.922 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.922 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.922 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.922 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-30T02:47:16.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-30T02:47:16.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.923 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.923 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.923 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.923 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.923 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.923 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.923 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.923 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.924 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.924 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.924 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-30T02:47:16.924 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-30T02:47:16.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.924 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.924 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.924 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.925 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.925 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.925 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.925 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.925 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.925 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.925 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.925 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-30T02:47:16.926 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-30T02:47:16.926 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.926 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.926 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.926 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.926 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.926 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.926 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.926 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.927 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.927 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.927 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-30T02:47:16.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-30T02:47:16.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.927 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.928 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.928 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.928 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.928 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.928 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.928 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.928 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.928 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.929 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.930 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.930 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.930 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.930 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-30T02:47:16.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-30T02:47:16.930 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.930 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.931 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.931 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.931 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.931 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.931 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.931 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.931 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.931 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.931 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.932 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-30T02:47:16.932 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-30T02:47:16.932 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.932 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.932 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.932 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.932 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.933 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.933 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.933 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.933 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.933 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.933 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.933 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-03-30T02:47:16.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-30T02:47:16.934 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.934 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.934 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.934 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-03-30T02:47:16.934 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.934 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.934 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.934 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.935 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.935 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.935 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.935 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-30T02:47:16.935 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-30T02:47:16.935 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.935 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.936 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.936 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.936 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.936 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.936 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.936 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.936 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.936 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.936 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.937 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-03-30T02:47:16.937 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-03-30T02:47:16.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.937 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.937 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.937 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.937 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.937 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.938 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.938 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.938 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.938 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.938 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.938 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-03-30T02:47:16.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-30T02:47:16.939 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.939 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.939 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.939 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-03-30T02:47:16.939 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.939 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.939 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.939 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.939 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.940 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.940 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.940 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-30T02:47:16.940 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-30T02:47:16.940 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.940 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.940 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.940 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.941 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.941 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.941 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.941 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.941 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.941 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.941 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.942 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-30T02:47:16.942 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-30T02:47:16.942 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.942 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.942 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.942 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.942 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.942 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.943 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.943 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.943 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.943 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.943 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.943 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-30T02:47:16.943 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-30T02:47:16.943 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.944 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.944 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.944 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.944 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.944 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.944 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.944 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.944 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.944 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.945 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.945 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-30T02:47:16.945 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-30T02:47:16.945 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.945 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.945 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.945 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.946 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.946 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.946 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.946 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.946 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.946 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.946 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.947 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-03-30T02:47:16.947 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-30T02:47:16.947 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.947 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.947 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.947 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.947 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.947 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.948 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.948 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.948 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.948 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.948 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-30T02:47:16.948 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-30T02:47:16.948 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.948 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.949 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.949 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.949 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.949 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.949 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.949 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.949 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.949 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-30T02:47:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-30T02:47:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.950 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.951 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.951 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.951 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.951 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.951 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.951 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.951 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-30T02:47:16.951 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-30T02:47:16.952 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.952 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.952 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.952 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.952 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.952 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.952 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.952 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.952 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.953 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.953 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.953 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-30T02:47:16.953 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-30T02:47:16.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.953 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.953 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.953 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.954 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.954 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.954 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.954 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.954 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.954 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.954 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.955 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-30T02:47:16.955 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-30T02:47:16.955 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.955 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.955 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.955 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.955 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.955 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.956 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.956 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.956 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.956 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.956 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.956 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-30T02:47:16.956 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-30T02:47:16.956 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.957 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.957 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.957 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.957 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.957 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.957 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.957 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.957 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.958 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.958 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.958 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-30T02:47:16.958 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-30T02:47:16.958 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.958 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.958 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.959 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.959 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.959 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.959 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.959 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.959 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.959 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.959 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.960 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-30T02:47:16.960 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-30T02:47:16.960 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.960 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.960 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.960 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.960 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.960 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.960 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.961 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.961 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.961 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.961 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.961 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-03-30T02:47:16.961 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-30T02:47:16.961 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.962 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.962 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.962 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.962 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.962 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.962 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.962 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.962 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.962 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.963 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.963 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-30T02:47:16.963 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-30T02:47:16.963 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.963 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.963 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.963 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-30T02:47:16.964 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.964 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.964 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.964 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.964 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.964 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.964 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.965 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-30T02:47:16.965 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-30T02:47:16.965 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.965 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.965 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.965 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.965 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.965 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.965 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.966 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.966 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.966 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.966 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.966 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-30T02:47:16.966 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-30T02:47:16.966 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.966 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.967 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.967 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-03-30T02:47:16.967 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.967 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.967 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.967 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.967 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.967 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.968 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.968 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-30T02:47:16.968 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-30T02:47:16.968 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.968 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.968 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.968 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-03-30T02:47:16.968 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.969 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.969 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.969 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.969 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.969 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.969 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.969 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-03-30T02:47:16.970 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-30T02:47:16.970 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.970 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.970 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.970 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.970 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.970 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.970 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.971 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.971 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.971 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.971 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.971 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-30T02:47:16.971 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-30T02:47:16.971 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.971 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.972 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.972 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-03-30T02:47:16.972 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.972 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.972 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.972 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.972 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.972 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.972 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.973 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-03-30T02:47:16.973 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-30T02:47:16.973 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.973 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.973 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.973 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.973 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.973 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.974 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.974 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.974 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.974 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.974 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.974 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-03-30T02:47:16.974 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-30T02:47:16.975 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.975 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.975 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.975 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:16.975 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.975 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.975 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.975 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.975 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.976 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.976 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.976 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-03-30T02:47:16.976 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-30T02:47:16.976 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.976 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.976 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.977 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.977 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.977 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.977 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.977 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.977 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.977 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.977 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.978 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-30T02:47:16.978 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-30T02:47:16.978 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.978 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.978 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.978 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:16.978 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.978 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.978 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.979 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.979 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.979 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.979 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.979 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-30T02:47:16.979 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-30T02:47:16.979 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.980 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.980 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.980 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:16.980 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.980 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.980 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.980 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.980 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.980 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.981 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.981 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-30T02:47:16.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-30T02:47:16.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.981 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.981 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.981 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.982 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-03-30T02:47:16.983 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-03-30T02:47:16.983 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.983 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.983 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.983 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.983 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.983 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.983 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.984 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.984 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.984 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.984 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.984 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-03-30T02:47:16.984 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-03-30T02:47:16.984 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.984 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.985 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.985 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.985 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.985 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.985 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.985 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.985 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.985 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.986 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.986 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-03-30T02:47:16.986 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-03-30T02:47:16.986 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.986 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.986 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.986 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.987 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.987 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.987 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.987 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-03-30T02:47:16.987 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.987 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.987 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.988 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-03-30T02:47:16.988 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-03-30T02:47:16.988 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:16.988 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.988 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.988 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-03-30T02:47:16.988 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.988 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.988 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.989 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.989 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.989 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.989 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.989 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-03-30T02:47:16.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-03-30T02:47:16.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.990 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.990 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:16.990 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.990 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.990 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.990 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.990 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.990 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.991 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.991 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.991 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:16.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:16.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.991 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.991 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.991 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:16.992 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.992 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.992 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.992 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.992 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.992 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.992 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.992 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.992 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.993 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.993 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.993 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.993 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.993 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.993 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:16.993 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:16.993 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.994 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.994 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.994 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.994 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.994 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.994 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.994 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.994 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.995 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.995 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.995 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:16.995 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:16.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.995 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.995 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.995 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:16.996 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.996 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.996 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:16.996 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:16.996 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:16.996 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:16.996 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:16.996 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:16.997 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:16.997 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:16.997 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.997 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.997 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.997 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.997 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:16.997 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:16.998 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:16.998 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.998 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:16.998 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:16.998 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:16.998 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:16.998 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:16.998 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:16.999 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:16.999 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:16.999 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:16.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:16.999 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-03-30T02:47:16.999 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-03-30T02:47:16.999 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:16.999 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:16.999 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.000 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.000 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.000 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.000 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.000 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-03-30T02:47:17.000 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.000 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.000 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.001 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-03-30T02:47:17.001 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-03-30T02:47:17.001 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.001 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.001 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.001 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.001 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.001 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.002 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.002 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.002 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.002 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.002 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.002 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-03-30T02:47:17.002 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-03-30T02:47:17.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.003 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.003 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.003 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-03-30T02:47:17.003 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.003 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.003 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.003 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.003 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.004 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.004 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.004 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-03-30T02:47:17.004 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-03-30T02:47:17.004 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.004 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.004 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.005 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-03-30T02:47:17.005 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.005 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.005 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.005 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.005 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.005 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.005 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.006 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-03-30T02:47:17.006 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-03-30T02:47:17.006 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.006 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.006 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.006 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.006 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.006 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.007 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.007 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.007 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.007 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.007 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.007 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-03-30T02:47:17.007 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-03-30T02:47:17.007 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.008 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.008 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.008 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-03-30T02:47:17.008 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.008 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.008 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.008 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.008 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.009 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.009 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.009 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-03-30T02:47:17.009 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-03-30T02:47:17.009 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.009 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.009 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.009 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-03-30T02:47:17.010 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.010 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.010 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.010 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.010 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-03-30T02:47:17.010 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.010 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.010 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-03-30T02:47:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-03-30T02:47:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.011 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-03-30T02:47:17.012 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-03-30T02:47:17.012 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-03-30T02:47:17.012 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.012 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.012 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.012 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.012 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-03-30T02:47:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-03-30T02:47:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.013 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.015 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-03-30T02:47:17.015 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.015 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.015 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.015 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-03-30T02:47:17.015 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-03-30T02:47:17.015 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.015 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.015 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.016 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.016 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.016 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.016 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.016 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-03-30T02:47:17.016 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.016 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.016 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.017 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.017 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.017 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.017 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.017 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.017 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.017 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.017 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.018 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.018 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.018 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.018 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.018 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.018 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.018 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.018 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.019 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.019 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.019 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.019 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.019 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.019 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.019 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.019 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.020 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.020 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.020 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.020 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.020 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.020 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.020 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.020 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.020 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.021 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.021 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.021 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.021 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.021 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.021 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.021 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.021 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.022 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.022 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.022 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.022 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.022 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.022 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.022 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.022 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.023 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.023 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.023 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.023 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.023 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.023 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.023 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.023 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.024 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.024 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.024 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.024 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.024 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.024 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.024 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.024 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.024 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.025 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-03-30T02:47:17.025 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-03-30T02:47:17.025 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.025 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.025 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.025 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-03-30T02:47:17.025 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.025 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.026 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.026 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-30T02:47:17.026 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.026 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.026 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.026 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-03-30T02:47:17.026 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-03-30T02:47:17.027 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.027 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.027 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.027 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-03-30T02:47:17.027 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.027 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.027 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.027 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-30T02:47:17.027 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.028 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.028 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.028 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-03-30T02:47:17.028 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-03-30T02:47:17.028 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.028 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.028 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.029 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-03-30T02:47:17.029 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.029 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.029 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.029 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-03-30T02:47:17.029 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.029 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.029 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.030 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-03-30T02:47:17.030 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-03-30T02:47:17.030 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.030 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.030 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.030 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-03-30T02:47:17.030 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.030 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.031 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.031 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-30T02:47:17.031 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.031 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.031 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.031 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-03-30T02:47:17.031 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-03-30T02:47:17.031 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.032 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.032 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.032 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.032 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.032 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.032 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.032 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-30T02:47:17.032 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.033 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.033 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.033 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:17.033 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:17.033 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.033 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.033 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:17.034 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.034 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.034 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.034 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-03-30T02:47:17.034 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.034 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.034 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.034 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-03-30T02:47:17.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-03-30T02:47:17.035 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.035 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.035 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.035 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-03-30T02:47:17.035 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.035 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.035 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.035 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-30T02:47:17.036 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.036 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.036 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.036 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.036 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.036 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.036 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-03-30T02:47:17.036 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.037 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.037 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.037 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.037 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.037 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.037 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.037 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.037 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.037 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.038 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.038 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.038 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.038 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.038 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.038 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.038 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.039 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.039 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.039 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.039 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.039 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.039 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.039 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.040 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.040 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.040 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.040 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.040 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.040 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.040 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.040 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.040 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.041 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.041 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.041 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.041 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.041 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.041 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.041 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.042 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.042 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.042 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.042 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.042 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.042 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.042 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.042 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.042 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.043 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.043 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.043 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.043 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.043 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.043 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.043 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.044 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.044 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.044 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.044 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.044 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.044 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.044 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.044 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.045 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.045 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.045 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.045 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.045 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.045 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-03-30T02:47:17.045 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-03-30T02:47:17.045 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.046 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.046 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.046 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-03-30T02:47:17.046 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.046 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.046 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.046 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.046 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.047 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.047 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.047 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-03-30T02:47:17.047 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-03-30T02:47:17.047 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.047 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.047 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.047 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-03-30T02:47:17.048 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.048 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.048 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.048 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.048 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.048 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.048 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.048 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:17.049 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:17.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.049 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.049 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.049 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-03-30T02:47:17.049 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.049 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.049 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.050 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.050 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.050 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.050 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.050 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.050 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-03-30T02:47:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-03-30T02:47:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-03-30T02:47:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-03-30T02:47:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-03-30T02:47:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.051 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-03-30T02:47:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-30T02:47:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-03-30T02:47:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-03-30T02:47:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-03-30T02:47:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.053 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-30T02:47:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-03-30T02:47:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-03-30T02:47:17.054 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-03-30T02:47:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.055 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.056 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-03-30T02:47:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-03-30T02:47:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-30T02:47:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.057 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.058 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.059 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.060 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.062 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.063 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.064 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-03-30T02:47:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-03-30T02:47:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.066 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-03-30T02:47:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-03-30T02:47:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-03-30T02:47:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-03-30T02:47:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.068 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-03-30T02:47:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-03-30T02:47:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-03-30T02:47:17.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.070 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.070 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.070 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-03-30T02:47:17.070 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.070 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.070 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.070 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T02:47:17.070 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.071 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.071 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.071 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-03-30T02:47:17.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-03-30T02:47:17.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.071 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.071 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.072 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:17.072 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:17.072 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-03-30T02:47:17.072 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.072 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-30T02:47:17.072 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.072 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.073 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-03-30T02:47:17.073 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-03-30T02:47:17.073 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.073 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.073 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.073 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.073 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.073 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.074 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.074 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-03-30T02:47:17.074 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.074 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.074 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.074 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-03-30T02:47:17.074 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-03-30T02:47:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-03-30T02:47:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-30T02:47:17.075 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.076 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.076 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.076 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.076 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-03-30T02:47:17.076 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.076 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.077 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.078 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.078 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.078 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.078 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.078 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.078 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.078 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.078 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.079 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.079 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.079 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.079 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.079 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.079 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.079 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.079 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.080 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.080 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.080 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.080 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.080 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.080 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.080 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.080 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.080 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.081 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.081 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.081 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.081 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.081 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.081 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.082 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.082 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.082 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.082 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.082 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.082 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.082 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.082 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.082 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.083 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.083 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.083 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.083 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.083 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.083 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.083 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.084 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.084 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.084 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.084 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.084 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.084 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.084 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.084 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.085 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.085 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.085 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-03-30T02:47:17.085 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.085 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.085 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.086 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.087 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.088 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.090 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.091 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.092 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.093 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-03-30T02:47:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.094 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-03-30T02:47:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-03-30T02:47:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-03-30T02:47:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.095 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-30T02:47:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-03-30T02:47:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-03-30T02:47:17.096 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-03-30T02:47:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-30T02:47:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.097 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.098 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.099 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.099 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.099 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.099 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.099 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.099 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.099 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.099 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.099 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.100 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.100 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.100 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.100 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.100 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.100 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.100 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.101 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.101 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.101 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.101 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.101 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.101 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.101 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.101 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.102 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.102 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.102 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.102 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.102 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.103 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.103 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.103 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.103 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.103 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.103 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.103 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.103 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.103 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.105 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.105 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.105 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.105 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.105 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.105 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.105 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.106 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.106 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-03-30T02:47:17.106 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.107 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.107 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.107 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-03-30T02:47:17.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-03-30T02:47:17.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.107 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.107 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.107 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-03-30T02:47:17.108 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.108 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.108 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.108 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-03-30T02:47:17.108 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-30T02:47:17.108 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.108 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.108 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.109 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.109 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.109 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.109 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.109 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.109 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.109 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.110 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.110 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.110 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.110 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.110 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.110 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.110 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.110 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.111 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.111 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.111 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.111 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.111 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.112 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.112 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.112 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.112 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.112 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.112 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.112 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.112 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.112 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.113 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.113 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.113 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.113 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.113 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.113 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.114 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.114 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.114 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.114 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.114 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.114 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.114 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.114 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.115 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.115 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.115 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.115 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.115 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.115 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.115 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.116 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.116 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.116 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.116 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.116 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.116 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.116 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.117 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-03-30T02:47:17.117 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-03-30T02:47:17.117 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.117 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.117 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.117 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.117 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.117 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.118 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.118 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-03-30T02:47:17.118 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-30T02:47:17.118 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.118 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.118 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-03-30T02:47:17.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-03-30T02:47:17.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.119 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.119 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.119 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-03-30T02:47:17.119 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.119 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.119 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.119 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-03-30T02:47:17.119 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.120 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.120 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.120 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-03-30T02:47:17.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-03-30T02:47:17.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.120 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.120 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-03-30T02:47:17.121 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.121 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.121 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.121 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-03-30T02:47:17.121 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.121 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.121 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.122 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-03-30T02:47:17.122 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-03-30T02:47:17.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.122 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.122 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.122 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.122 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.122 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.122 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.123 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-03-30T02:47:17.123 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.123 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.123 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-03-30T02:47:17.123 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-03-30T02:47:17.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-03-30T02:47:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-30T02:47:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-30T02:47:17.124 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.125 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.125 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-03-30T02:47:17.125 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.125 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.126 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.126 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.126 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.126 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.126 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.126 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.126 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.126 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.127 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.127 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.127 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.127 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.127 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.127 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.127 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.127 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.128 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.128 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.128 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.128 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.128 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.128 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.129 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.129 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.129 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.129 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.129 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.129 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.129 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.129 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.129 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.130 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.130 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.130 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.130 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.130 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.131 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.131 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.131 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.131 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.131 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.131 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.131 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.131 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.131 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.132 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.132 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.132 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.132 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.132 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.132 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.132 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.133 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.133 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.133 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.133 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.133 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.133 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.133 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.133 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.134 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.134 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.134 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.134 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-03-30T02:47:17.134 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.134 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.134 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.135 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.135 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.135 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.135 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.135 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.135 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.135 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.136 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.137 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.138 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.139 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.140 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.141 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.142 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-03-30T02:47:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.143 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.144 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.145 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.146 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.148 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.149 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.150 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.151 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.151 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.151 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.151 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.151 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.151 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.151 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.151 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.152 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.152 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.152 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-03-30T02:47:17.152 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.152 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.152 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.152 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-03-30T02:47:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-03-30T02:47:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.154 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-30T02:47:17.154 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.154 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.154 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.154 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.154 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.154 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.155 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.155 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.155 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.155 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.155 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.155 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.155 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.155 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.155 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.156 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.156 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.156 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.156 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.156 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.156 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.156 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.157 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.158 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.159 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.159 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.159 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.159 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.159 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.159 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.159 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.159 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.159 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.160 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.160 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.160 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.160 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.160 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.160 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.160 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.161 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.161 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.161 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.161 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.161 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.161 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.161 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.161 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-03-30T02:47:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-03-30T02:47:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.162 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.163 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.163 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.163 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.163 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.163 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.163 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-03-30T02:47:17.163 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-03-30T02:47:17.163 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-03-30T02:47:17.164 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.164 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-03-30T02:47:17.164 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.164 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.164 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.164 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.164 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-03-30T02:47:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.166 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.166 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.166 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.166 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.166 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.166 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.166 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.167 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.167 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.167 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.167 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.167 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.167 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.167 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.167 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.168 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.168 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.168 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.168 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.168 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.168 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.168 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.168 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.168 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.169 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.169 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.169 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.169 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.169 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.169 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.170 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.170 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.170 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.170 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.170 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.170 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.170 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.170 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.170 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.171 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.172 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.173 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.173 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.173 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.173 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.173 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.173 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.173 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-03-30T02:47:17.173 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.174 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.174 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.174 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.174 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.174 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.174 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.175 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.176 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.176 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.176 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.176 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.176 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.176 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.176 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.177 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.177 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.177 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.177 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.177 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.177 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.177 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.177 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.177 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.178 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.178 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.178 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.178 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.178 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.179 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.179 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.179 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.179 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.179 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.179 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.179 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.179 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.179 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.180 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.180 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.180 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.180 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.180 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.180 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.180 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.180 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.181 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.181 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.181 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.181 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.181 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.181 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.181 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.181 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.182 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.182 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.182 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.182 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.182 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.182 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.182 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-03-30T02:47:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.183 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.184 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.184 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.184 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.184 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.184 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.184 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.184 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.184 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.185 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.185 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.185 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.185 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.185 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.185 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.185 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.185 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.186 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.186 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.186 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.186 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.186 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.186 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.186 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.187 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.187 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.187 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.187 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.187 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.187 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.187 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.187 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.188 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.188 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.188 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.188 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.188 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.188 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.188 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.188 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.189 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.189 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.189 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.189 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.189 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.189 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.189 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.189 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.190 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.190 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.190 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.190 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.190 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.190 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.190 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.190 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.190 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.191 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.192 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.192 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:17.192 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.192 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.192 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.192 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.192 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.192 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.192 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.193 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-03-30T02:47:17.193 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-03-30T02:47:17.193 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:17.193 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.193 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.193 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-03-30T02:47:17.193 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-03-30T02:47:17.193 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.194 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.194 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-03-30T02:47:17.194 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-30T02:47:17.194 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.194 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.194 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.194 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.195 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.195 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-03-30T02:47:17.195 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.195 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.195 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.195 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-03-30T02:47:17.195 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-03-30T02:47:17.195 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.195 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.196 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.196 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.196 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.196 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.196 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.196 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-30T02:47:17.196 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.196 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.197 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.197 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-03-30T02:47:17.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-03-30T02:47:17.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.197 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.197 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.197 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.197 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.198 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.198 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.198 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.198 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.198 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.198 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.198 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.199 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.199 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.199 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.199 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.199 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.199 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.199 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.199 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.200 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.200 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.200 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.200 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.200 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.200 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.200 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.200 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.200 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.201 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.201 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.201 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.201 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.201 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.201 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.201 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.201 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.202 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.202 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.202 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.202 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.202 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.202 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.202 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.203 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.203 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.203 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.203 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.203 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.203 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.203 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.203 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.204 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.204 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.204 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.204 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.204 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.204 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.204 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.204 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.205 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.205 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.205 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.205 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.205 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.205 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.205 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.206 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.206 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.206 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.206 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.206 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.206 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.206 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.206 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.207 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.207 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-03-30T02:47:17.207 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-03-30T02:47:17.207 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.207 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.207 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.207 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-03-30T02:47:17.208 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.208 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.208 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.208 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-03-30T02:47:17.208 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.208 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.208 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.209 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-03-30T02:47:17.209 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-03-30T02:47:17.209 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.209 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.209 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.209 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-03-30T02:47:17.209 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.209 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.209 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.210 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-03-30T02:47:17.210 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.210 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.210 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.210 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.210 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.210 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.211 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-03-30T02:47:17.212 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-03-30T02:47:17.213 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.213 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.213 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.213 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.213 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.213 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.213 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.213 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-30T02:47:17.214 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-30T02:47:17.214 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.214 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.214 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.214 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.214 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.215 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.215 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.215 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.215 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.215 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.215 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.215 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.215 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.215 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.216 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.217 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.217 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.217 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.217 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.217 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.217 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.217 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.217 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.217 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.218 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.218 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.218 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.218 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.218 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.218 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.218 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.219 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.219 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.219 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.219 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.219 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.219 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.219 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.219 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.219 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.220 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.220 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.220 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.220 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.220 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.220 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.220 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.220 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.221 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.221 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.221 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.221 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.221 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.221 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.221 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.221 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.222 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.222 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.222 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-03-30T02:47:17.222 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-03-30T02:47:17.222 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.222 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.222 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.223 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.223 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.223 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.223 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.223 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.223 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.223 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.223 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.224 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-30T02:47:17.224 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-30T02:47:17.224 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.224 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.224 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.224 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-03-30T02:47:17.224 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.224 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.225 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.225 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.225 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.225 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.225 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.225 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-03-30T02:47:17.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-03-30T02:47:17.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:17.226 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.226 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.226 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-03-30T02:47:17.226 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.226 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.226 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.226 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.226 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.227 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.227 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.227 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-03-30T02:47:17.227 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-03-30T02:47:17.227 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.227 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.227 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.227 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-03-30T02:47:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-30T02:47:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.228 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-03-30T02:47:17.229 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-03-30T02:47:17.229 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.229 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.229 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.229 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-03-30T02:47:17.229 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.229 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.229 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.229 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-30T02:47:17.230 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.230 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.230 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.230 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-03-30T02:47:17.230 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-03-30T02:47:17.230 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.230 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.231 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.231 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-03-30T02:47:17.231 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.231 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.231 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.231 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.231 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.231 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.232 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.232 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-03-30T02:47:17.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-03-30T02:47:17.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.232 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.232 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.232 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-03-30T02:47:17.232 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.233 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.233 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.233 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-03-30T02:47:17.233 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-03-30T02:47:17.233 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.233 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.233 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.233 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.233 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.234 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-03-30T02:47:17.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-03-30T02:47:17.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.234 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.234 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.234 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-03-30T02:47:17.234 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-03-30T02:47:17.235 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-03-30T02:47:17.235 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.235 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.235 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.235 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.235 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.235 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-03-30T02:47:17.236 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.236 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.236 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.236 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.236 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.236 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.237 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.237 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.237 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.237 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.237 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.237 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.237 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.237 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.238 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.238 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.238 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.238 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.238 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.238 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.238 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.238 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.239 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.239 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.239 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.239 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.239 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.239 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.240 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.240 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.240 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.240 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.240 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.240 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.240 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.241 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.241 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.241 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.241 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.241 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.241 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.241 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.241 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.242 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.242 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.242 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.242 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.242 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.242 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.242 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.242 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.242 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.243 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.243 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.243 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.243 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.243 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.243 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.243 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.244 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.244 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.244 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.244 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.244 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.244 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.244 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-03-30T02:47:17.244 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-03-30T02:47:17.245 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.245 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.245 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.245 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-03-30T02:47:17.245 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.245 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.245 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.245 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.246 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.246 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.246 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.246 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-03-30T02:47:17.246 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-03-30T02:47:17.246 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.246 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.247 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.247 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.247 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.247 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.247 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.247 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.247 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.247 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.248 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-03-30T02:47:17.248 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-03-30T02:47:17.248 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.248 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.248 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.248 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.248 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.249 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.249 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.249 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.249 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.249 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.249 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.249 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.249 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-03-30T02:47:17.250 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.250 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.250 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.250 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-03-30T02:47:17.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-03-30T02:47:17.251 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.251 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.251 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.251 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.251 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.251 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.251 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.251 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-03-30T02:47:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-03-30T02:47:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.252 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.253 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.253 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.253 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.253 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.253 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.253 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.253 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.253 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.254 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.254 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.254 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.254 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.254 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.254 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.254 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.255 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.255 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.255 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.255 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.255 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.255 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.255 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.255 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.255 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.256 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.256 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.256 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.256 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.256 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.256 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.257 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.257 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.257 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.257 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.257 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.257 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.257 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.257 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.257 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.258 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.258 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.258 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.258 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.258 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.258 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.258 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.259 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.259 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.259 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.259 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.259 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.259 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.259 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.259 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.259 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.260 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.261 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.261 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.261 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.261 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.261 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.261 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.261 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.261 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.261 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.262 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.262 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-03-30T02:47:17.262 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-03-30T02:47:17.262 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.262 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.262 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.262 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.263 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.263 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.263 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.263 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.263 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.263 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.263 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.263 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-03-30T02:47:17.264 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-03-30T02:47:17.264 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.264 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.264 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.264 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.264 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.264 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.264 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.265 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.265 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.265 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.265 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.265 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.265 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-03-30T02:47:17.266 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.266 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.266 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.266 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.266 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.266 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.266 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.266 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.267 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.267 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.267 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.267 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.267 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.267 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.267 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.267 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.267 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.268 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.268 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.268 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.268 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.268 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.268 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.268 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.269 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.269 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.269 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.269 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.269 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.269 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.269 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.269 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.270 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.270 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.270 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.270 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.270 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.270 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.270 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.271 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.271 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.271 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.271 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.271 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.271 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.271 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.271 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.271 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.272 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.272 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.272 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.272 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.272 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.272 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.272 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.272 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.273 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.273 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.273 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.273 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.273 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.273 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.273 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.273 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.274 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.274 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.274 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.274 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.274 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.274 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.274 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.275 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-03-30T02:47:17.275 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.275 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.275 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.275 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.275 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.275 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.276 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.277 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.277 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.277 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.277 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.277 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.277 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.277 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.277 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.278 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.278 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.278 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.278 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.278 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.278 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.278 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.279 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.279 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.279 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.279 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.279 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.279 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.280 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.280 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.280 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.280 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.280 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.280 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.280 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.280 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.281 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.281 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.281 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.281 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.281 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.281 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.281 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.281 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.281 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.282 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.282 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.282 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.282 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.282 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.282 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.282 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.283 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.283 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.283 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.283 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.283 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.283 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.283 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-03-30T02:47:17.283 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-03-30T02:47:17.284 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.284 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.284 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.284 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-03-30T02:47:17.284 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.284 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.284 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.284 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-30T02:47:17.284 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.285 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.285 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.285 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.285 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.285 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.285 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-03-30T02:47:17.285 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.285 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.286 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.286 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.286 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.286 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.286 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.286 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.286 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.286 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.286 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.287 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.287 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.287 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.287 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.287 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.287 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.287 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.288 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.288 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.288 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.288 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.288 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.288 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.288 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.288 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.289 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.289 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.289 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.289 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.289 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.289 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.289 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.289 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.289 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.290 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.290 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.290 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.290 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.290 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.290 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.290 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.291 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.291 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.291 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.291 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.291 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.291 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.291 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.291 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.291 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.292 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.292 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.292 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.292 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.292 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.292 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.293 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.293 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.293 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.293 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.293 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.293 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.293 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.293 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.294 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.294 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.294 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.294 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.294 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-03-30T02:47:17.294 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-03-30T02:47:17.294 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.295 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.295 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.295 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.295 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.295 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.295 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.295 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.295 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.296 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.296 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.296 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-03-30T02:47:17.296 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-03-30T02:47:17.296 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.296 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.296 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.296 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-03-30T02:47:17.297 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.297 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.297 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.297 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.297 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.297 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.297 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.298 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-03-30T02:47:17.298 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-03-30T02:47:17.298 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.298 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.298 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.298 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.298 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.298 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.298 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.299 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.299 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.299 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.299 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.299 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-03-30T02:47:17.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-03-30T02:47:17.299 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.299 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.300 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.300 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.300 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.300 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.300 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.300 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.300 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.300 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.300 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.301 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-03-30T02:47:17.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-03-30T02:47:17.301 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-03-30T02:47:17.301 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.301 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.301 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.301 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.302 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.302 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.302 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.302 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.302 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.302 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.302 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-03-30T02:47:17.302 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-03-30T02:47:17.303 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.303 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.303 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.303 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.303 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.303 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.303 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.303 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.304 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.304 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.304 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.304 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-03-30T02:47:17.304 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-03-30T02:47:17.304 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.304 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.304 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.305 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.305 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.305 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.305 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.305 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.305 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.305 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.305 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.306 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-03-30T02:47:17.306 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-03-30T02:47:17.306 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.306 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.306 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.306 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.306 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.306 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.307 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.307 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.307 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.307 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.307 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.307 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-03-30T02:47:17.307 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-03-30T02:47:17.308 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.308 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.308 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.308 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.308 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-03-30T02:47:17.308 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-03-30T02:47:17.308 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.308 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.308 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.309 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.309 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.309 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-03-30T02:47:17.309 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-03-30T02:47:17.309 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.309 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.309 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.309 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.310 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.310 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.310 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.310 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.310 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.310 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.310 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.311 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-03-30T02:47:17.311 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-03-30T02:47:17.311 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.311 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.311 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.311 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.311 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.311 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.312 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.312 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.312 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.312 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.312 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.312 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-03-30T02:47:17.312 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-03-30T02:47:17.312 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.313 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.313 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.313 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.313 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.313 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.313 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.313 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.313 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.313 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.314 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.314 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.314 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.314 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-03-30T02:47:17.314 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.314 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.315 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.315 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-03-30T02:47:17.315 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-03-30T02:47:17.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.315 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.315 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.315 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.315 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.316 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.316 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.316 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-03-30T02:47:17.316 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.316 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.316 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.316 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-03-30T02:47:17.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-03-30T02:47:17.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.317 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.317 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.317 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.317 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.317 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.317 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.317 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-03-30T02:47:17.318 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.318 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.318 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.318 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.318 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.318 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.318 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.319 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.319 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.319 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.319 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.319 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.319 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.319 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.319 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.319 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.320 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.320 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.320 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.320 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.320 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.320 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.320 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.320 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.321 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.321 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.321 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.321 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.321 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.321 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.321 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.321 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.322 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.322 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.322 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.322 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.322 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.322 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.322 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.323 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.323 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.323 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.323 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.323 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.323 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.323 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.323 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.323 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.324 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.324 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.324 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.324 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.324 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.324 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.324 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.324 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.325 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.326 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.326 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.326 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.326 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.326 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.326 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.327 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-03-30T02:47:17.327 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.327 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.327 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.327 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.327 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.327 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.327 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.328 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.328 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.328 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.328 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.328 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.328 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.328 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.328 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.328 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.329 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.329 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.329 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.329 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.329 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.329 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.329 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.330 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.330 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.330 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.330 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.330 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.330 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.330 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.331 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.331 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.331 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.331 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.331 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.331 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.331 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.332 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.332 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.332 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.332 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.332 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.332 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.332 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.332 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.333 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.333 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.333 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.333 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.333 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.333 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.333 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.333 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.334 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.334 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.334 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.334 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.334 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.334 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.335 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.335 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.335 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.335 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.335 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.335 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.335 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-03-30T02:47:17.336 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.336 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.336 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.336 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.337 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.337 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.337 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.337 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.337 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.337 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.337 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.337 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.338 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.338 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.338 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.338 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.338 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.338 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.338 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.338 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.339 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.339 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.339 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.339 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.339 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.339 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.339 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.341 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.341 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.341 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.341 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.342 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.342 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.342 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.342 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.342 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.342 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.342 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.343 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.343 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.343 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.343 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.343 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.343 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.343 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.343 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.344 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.344 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.344 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.344 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.344 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.344 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.344 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.344 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.345 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.345 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.345 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.345 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.345 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.345 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.345 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.346 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.346 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.346 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.346 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.346 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.346 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.347 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-03-30T02:47:17.347 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.347 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.347 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.347 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-03-30T02:47:17.347 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-03-30T02:47:17.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.347 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.348 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.348 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-30T02:47:17.348 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.348 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.348 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.348 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.348 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.348 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.348 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.349 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:17.349 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:17.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.349 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.349 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.349 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-03-30T02:47:17.350 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.350 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.350 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.350 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.350 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.350 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.350 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.351 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.351 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.351 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.351 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.351 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.351 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.351 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.351 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.351 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.352 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.352 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.352 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.352 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.352 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.352 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.352 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.352 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.353 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.353 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.353 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.353 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.353 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.353 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.354 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.354 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.354 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.354 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.354 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.354 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.354 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.354 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.355 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.355 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.355 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.355 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.355 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.355 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.355 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.355 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.356 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.356 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.356 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.356 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.356 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.356 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.356 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.356 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.356 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.357 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.357 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.357 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.357 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.357 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.357 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.357 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.358 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.358 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.358 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.358 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.358 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.358 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.358 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.358 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.359 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.359 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.359 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-03-30T02:47:17.359 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.359 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-03-30T02:47:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-03-30T02:47:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.360 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.361 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.361 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-30T02:47:17.361 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.361 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.361 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.361 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-03-30T02:47:17.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-03-30T02:47:17.362 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.362 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.362 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.362 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.362 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.362 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.362 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.362 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-30T02:47:17.363 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.363 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.363 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.363 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-03-30T02:47:17.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-03-30T02:47:17.363 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.363 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.363 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.364 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.364 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.364 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.364 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.364 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-30T02:47:17.364 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.364 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.364 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.365 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-03-30T02:47:17.365 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-03-30T02:47:17.365 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.365 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.365 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.365 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.365 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.365 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.366 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.366 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-30T02:47:17.366 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.366 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.366 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.366 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-03-30T02:47:17.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-03-30T02:47:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-30T02:47:17.367 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.368 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.368 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.368 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-03-30T02:47:17.368 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-03-30T02:47:17.368 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.368 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.368 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.369 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.369 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.369 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.369 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.369 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.369 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.369 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.369 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.370 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-03-30T02:47:17.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-03-30T02:47:17.370 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.370 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.370 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.370 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.370 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.370 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.370 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-03-30T02:47:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-03-30T02:47:17.371 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-30T02:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.372 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-03-30T02:47:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-03-30T02:47:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.373 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.374 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-03-30T02:47:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-03-30T02:47:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.375 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-03-30T02:47:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-03-30T02:47:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.376 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-03-30T02:47:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.377 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-03-30T02:47:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-03-30T02:47:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.378 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-03-30T02:47:17.379 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-03-30T02:47:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.380 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.381 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.382 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.384 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.385 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.385 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.385 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.385 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.385 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.385 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.385 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.385 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.386 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.386 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.386 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.386 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.386 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.386 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.386 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.386 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.387 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.387 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.387 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.387 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.387 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.387 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.387 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.388 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.388 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.388 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.388 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.388 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.388 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.388 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.388 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.389 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.389 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.389 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.389 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-03-30T02:47:17.389 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-03-30T02:47:17.389 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.389 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.389 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.390 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.390 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.390 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.390 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.390 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.390 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.390 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.390 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.391 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-03-30T02:47:17.391 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-03-30T02:47:17.391 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.391 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.391 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.391 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.391 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.392 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.392 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.392 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.392 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.392 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.392 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.392 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-03-30T02:47:17.392 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-03-30T02:47:17.393 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.393 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.393 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.393 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-30T02:47:17.393 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.393 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.393 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.393 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.393 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.394 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.394 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.394 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.394 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.394 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-03-30T02:47:17.394 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.395 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.396 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.397 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.397 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.397 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.397 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.397 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.397 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.398 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.398 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.398 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.398 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.398 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.398 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.398 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.399 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.400 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.400 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.400 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.400 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.400 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.400 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.400 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.400 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.400 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.401 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.401 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.401 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.401 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.401 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.401 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.401 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.401 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.402 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.402 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.402 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.402 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.402 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.402 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.402 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.402 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.403 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.403 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-03-30T02:47:17.403 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.403 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.403 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.403 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.403 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.403 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.403 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.404 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.404 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.404 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.404 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.404 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.404 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.404 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.404 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.404 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.405 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.405 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.405 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.405 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.405 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.405 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.406 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.406 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.406 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.406 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.406 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.406 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.406 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.406 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.407 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.408 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.409 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.409 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.409 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.409 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.409 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.409 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.409 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.410 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.410 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.410 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.410 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.410 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.410 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.410 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.410 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.411 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.411 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.411 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.411 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.411 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.411 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-03-30T02:47:17.411 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-03-30T02:47:17.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.412 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.412 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.412 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-03-30T02:47:17.412 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.412 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.412 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.412 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-30T02:47:17.412 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.412 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.413 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.413 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-03-30T02:47:17.413 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-03-30T02:47:17.413 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.413 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.413 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.413 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-03-30T02:47:17.414 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.414 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.414 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.414 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-30T02:47:17.414 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.414 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.414 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.414 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-03-30T02:47:17.415 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-03-30T02:47:17.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.415 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.415 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-03-30T02:47:17.415 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.415 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.415 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-30T02:47:17.416 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.416 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.416 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.416 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-03-30T02:47:17.416 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-03-30T02:47:17.417 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-03-30T02:47:17.417 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-03-30T02:47:17.417 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-03-30T02:47:17.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-03-30T02:47:17.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-03-30T02:47:17.417 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.417 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.417 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-03-30T02:47:17.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.418 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.418 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.418 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.418 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.418 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.418 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.418 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-03-30T02:47:17.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-03-30T02:47:17.419 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.419 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.419 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.419 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.419 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.419 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.419 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.419 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.420 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.420 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.420 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.420 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-03-30T02:47:17.420 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-03-30T02:47:17.420 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-03-30T02:47:17.420 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.421 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.421 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-03-30T02:47:17.421 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.421 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.421 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.421 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.421 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.421 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.421 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.422 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-03-30T02:47:17.422 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-03-30T02:47:17.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.422 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.422 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.422 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.422 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.423 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.423 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.423 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.423 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.423 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.423 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.423 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.423 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.423 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.424 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.424 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.424 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-03-30T02:47:17.424 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-03-30T02:47:17.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.424 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.426 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-03-30T02:47:17.426 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-03-30T02:47:17.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.426 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.426 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.426 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-03-30T02:47:17.426 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.426 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.427 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-03-30T02:47:17.427 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-03-30T02:47:17.427 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-03-30T02:47:17.427 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-03-30T02:47:17.427 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-03-30T02:47:17.427 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-03-30T02:47:17.427 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-03-30T02:47:17.427 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.427 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.428 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.428 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.428 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.428 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-03-30T02:47:17.428 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-03-30T02:47:17.428 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-03-30T02:47:17.428 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.428 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-03-30T02:47:17.429 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-03-30T02:47:17.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.429 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.429 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.429 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.429 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.429 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.429 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-03-30T02:47:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-03-30T02:47:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-03-30T02:47:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.430 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.431 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.431 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.431 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.431 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.431 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-03-30T02:47:17.431 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-03-30T02:47:17.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-03-30T02:47:17.431 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.432 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.432 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-03-30T02:47:17.432 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.432 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.432 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.432 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.432 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.432 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.432 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-03-30T02:47:17.433 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-03-30T02:47:17.434 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-03-30T02:47:17.434 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-03-30T02:47:17.434 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-03-30T02:47:17.434 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-03-30T02:47:17.434 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.434 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.434 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.435 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.435 INFO:teuthology.orchestra.run.smithi067.stdout: "services": { 2024-03-30T02:47:17.435 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard": "https://172.21.15.67:8443/" 2024-03-30T02:47:17.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.435 INFO:teuthology.orchestra.run.smithi067.stdout: "always_on_modules": { 2024-03-30T02:47:17.435 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus": [ 2024-03-30T02:47:17.435 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-03-30T02:47:17.435 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-03-30T02:47:17.436 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-03-30T02:47:17.436 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-03-30T02:47:17.436 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-03-30T02:47:17.436 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-03-30T02:47:17.436 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-03-30T02:47:17.436 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-03-30T02:47:17.436 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-03-30T02:47:17.436 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-03-30T02:47:17.436 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.437 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific": [ 2024-03-30T02:47:17.437 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-03-30T02:47:17.437 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-03-30T02:47:17.437 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-03-30T02:47:17.437 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-03-30T02:47:17.437 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-03-30T02:47:17.437 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-03-30T02:47:17.437 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-03-30T02:47:17.438 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-03-30T02:47:17.438 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-03-30T02:47:17.438 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-03-30T02:47:17.438 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.438 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy": [ 2024-03-30T02:47:17.438 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-03-30T02:47:17.438 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-03-30T02:47:17.438 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-03-30T02:47:17.439 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-03-30T02:47:17.439 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-03-30T02:47:17.439 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-03-30T02:47:17.439 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-03-30T02:47:17.439 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-03-30T02:47:17.439 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-03-30T02:47:17.439 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-03-30T02:47:17.439 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "reef": [ 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-03-30T02:47:17.440 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-03-30T02:47:17.441 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-03-30T02:47:17.441 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:17.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.441 INFO:teuthology.orchestra.run.smithi067.stdout: "last_failure_osd_epoch": 88, 2024-03-30T02:47:17.441 INFO:teuthology.orchestra.run.smithi067.stdout: "active_clients": [ 2024-03-30T02:47:17.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.441 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-03-30T02:47:17.441 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-30T02:47:17.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-30T02:47:17.442 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-03-30T02:47:17.442 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 3998085424 2024-03-30T02:47:17.442 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.442 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:17.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "libcephsqlite", 2024-03-30T02:47:17.443 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-30T02:47:17.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-30T02:47:17.443 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-03-30T02:47:17.443 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1498262787 2024-03-30T02:47:17.443 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.443 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:17.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.444 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-03-30T02:47:17.444 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-30T02:47:17.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.444 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-30T02:47:17.444 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-03-30T02:47:17.444 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 2549036100 2024-03-30T02:47:17.444 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.444 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:17.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.445 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-03-30T02:47:17.445 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-30T02:47:17.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.445 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-30T02:47:17.445 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-03-30T02:47:17.445 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 402302894 2024-03-30T02:47:17.445 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.445 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:17.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.446 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-03-30T02:47:17.446 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-30T02:47:17.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.446 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-30T02:47:17.446 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-03-30T02:47:17.446 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 310797802 2024-03-30T02:47:17.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.447 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:17.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:17.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.447 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-03-30T02:47:17.447 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-03-30T02:47:17.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-03-30T02:47:17.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-03-30T02:47:17.447 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-03-30T02:47:17.447 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 296533390 2024-03-30T02:47:17.448 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.448 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:17.448 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:17.448 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-03-30T02:47:17.448 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:47:17.450 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:15 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3298059590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:47:17.450 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:15 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4218682815' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-30T02:47:17.450 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:16 smithi082 ceph-mon[79176]: pgmap v316: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-03-30T02:47:17.450 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:16 smithi082 ceph-mon[79176]: osdmap e96: 12 total, 12 up, 12 in 2024-03-30T02:47:17.450 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:15 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3298059590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:47:17.450 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:15 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4218682815' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-30T02:47:17.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:16 smithi170 ceph-mon[78798]: pgmap v316: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-03-30T02:47:17.451 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:16 smithi170 ceph-mon[78798]: osdmap e96: 12 total, 12 up, 12 in 2024-03-30T02:47:17.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f52a00a1310 msgr2=0x7f52a00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:17.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f52a00a1310 0x7f52a00a37d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52ac011490 tx=0x7f52ac009260 comp rx=0 tx=0).stop 2024-03-30T02:47:17.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc10d430 msgr2=0x7f52bc19dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:17.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc10d430 0x7f52bc19dcc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f52b400ef30 tx=0x7f52b400c3a0 comp rx=0 tx=0).stop 2024-03-30T02:47:17.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 shutdown_connections 2024-03-30T02:47:17.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f52a00a1310 0x7f52a00a37d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:17.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52bc10d430 0x7f52bc19dcc0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:17.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52bc0731e0 0x7f52bc1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:17.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 --2- 172.21.15.67:0/4218682815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc073b00 0x7f52bc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:17.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 >> 172.21.15.67:0/4218682815 conn(0x7f52bc0fc700 msgr2=0x7f52bc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:17.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 shutdown_connections 2024-03-30T02:47:17.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:15.351+0000 7f52c0e00700 1 -- 172.21.15.67:0/4218682815 wait complete. 2024-03-30T02:47:17.461 DEBUG:tasks.check_counter:Getting stats from a 2024-03-30T02:47:17.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.a perf dump 2024-03-30T02:47:17.461 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:15 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3298059590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:47:17.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:15 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4218682815' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-30T02:47:17.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:16 smithi067 ceph-mon[37808]: pgmap v316: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-03-30T02:47:17.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:16 smithi067 ceph-mon[37808]: osdmap e96: 12 total, 12 up, 12 in 2024-03-30T02:47:17.840 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:47:17.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:17 smithi067 ceph-mon[37808]: pgmap v318: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-03-30T02:47:17.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:17 smithi170 ceph-mon[78798]: pgmap v318: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-03-30T02:47:17.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:17 smithi082 ceph-mon[79176]: pgmap v318: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-03-30T02:47:19.391 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:47:19.391 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:19.391 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 806, 2024-03-30T02:47:19.391 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 806, 2024-03-30T02:47:19.391 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 4459750, 2024-03-30T02:47:19.392 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 249871, 2024-03-30T02:47:19.392 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 6, 2024-03-30T02:47:19.392 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 2, 2024-03-30T02:47:19.392 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.127504845, 2024-03-30T02:47:19.392 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.049277187, 2024-03-30T02:47:19.392 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.032863989, 2024-03-30T02:47:19.392 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.031486036, 2024-03-30T02:47:19.392 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:19.393 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 805, 2024-03-30T02:47:19.393 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.075892293, 2024-03-30T02:47:19.393 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000094276 2024-03-30T02:47:19.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.393 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:19.393 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.393 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.393 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.394 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-30T02:47:19.394 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 448 2024-03-30T02:47:19.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.394 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:19.394 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 1242, 2024-03-30T02:47:19.394 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 1216, 2024-03-30T02:47:19.394 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 18965208, 2024-03-30T02:47:19.395 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 428519, 2024-03-30T02:47:19.395 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-03-30T02:47:19.395 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 3, 2024-03-30T02:47:19.395 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.222944824, 2024-03-30T02:47:19.395 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.050928891, 2024-03-30T02:47:19.395 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.127708275, 2024-03-30T02:47:19.395 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.047646571, 2024-03-30T02:47:19.395 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:19.396 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 1215, 2024-03-30T02:47:19.396 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.096746568, 2024-03-30T02:47:19.396 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000079626 2024-03-30T02:47:19.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.396 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:19.396 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.396 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.396 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.397 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1933520, 2024-03-30T02:47:19.397 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 119328 2024-03-30T02:47:19.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.397 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:19.397 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 375, 2024-03-30T02:47:19.397 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 529, 2024-03-30T02:47:19.397 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 745015, 2024-03-30T02:47:19.397 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 536222, 2024-03-30T02:47:19.398 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 7, 2024-03-30T02:47:19.398 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 5, 2024-03-30T02:47:19.398 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.069661184, 2024-03-30T02:47:19.398 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.026133912, 2024-03-30T02:47:19.398 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.016383120, 2024-03-30T02:47:19.398 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.015629122, 2024-03-30T02:47:19.398 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:19.398 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 526, 2024-03-30T02:47:19.398 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.061333176, 2024-03-30T02:47:19.399 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000116602 2024-03-30T02:47:19.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.399 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:19.399 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.399 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.399 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.400 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-30T02:47:19.400 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 420208 2024-03-30T02:47:19.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.400 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-03-30T02:47:19.400 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 1, 2024-03-30T02:47:19.400 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-03-30T02:47:19.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.401 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-MDSRank": { 2024-03-30T02:47:19.401 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-03-30T02:47:19.401 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-03-30T02:47:19.401 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 3599, 2024-03-30T02:47:19.401 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.147191628, 2024-03-30T02:47:19.401 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000040897 2024-03-30T02:47:19.401 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:19.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.401 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-03-30T02:47:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-03-30T02:47:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.402 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": { 2024-03-30T02:47:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "request": 0, 2024-03-30T02:47:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "reply": 0, 2024-03-30T02:47:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "reply_latency": { 2024-03-30T02:47:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_reply": 0, 2024-03-30T02:47:19.403 INFO:teuthology.orchestra.run.smithi067.stdout: "forward": 0, 2024-03-30T02:47:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_complete": 0, 2024-03-30T02:47:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_commit": 0, 2024-03-30T02:47:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_split": 0, 2024-03-30T02:47:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_merge": 0, 2024-03-30T02:47:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes": 6099, 2024-03-30T02:47:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_top": 0, 2024-03-30T02:47:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_bottom": 6099, 2024-03-30T02:47:19.404 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pin_tail": 0, 2024-03-30T02:47:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pinned": 6099, 2024-03-30T02:47:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_expired": 0, 2024-03-30T02:47:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_with_caps": 0, 2024-03-30T02:47:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": 0, 2024-03-30T02:47:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "subtrees": 176, 2024-03-30T02:47:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse": 0, 2024-03-30T02:47:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_hit": 0, 2024-03-30T02:47:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_forward": 0, 2024-03-30T02:47:19.405 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_discover": 0, 2024-03-30T02:47:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_lock": 0, 2024-03-30T02:47:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "load_cent": 0, 2024-03-30T02:47:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "q": 0, 2024-03-30T02:47:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "exported": 0, 2024-03-30T02:47:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "exported_inodes": 0, 2024-03-30T02:47:19.406 INFO:teuthology.orchestra.run.smithi067.stdout: "imported": 0, 2024-03-30T02:47:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "imported_inodes": 0, 2024-03-30T02:47:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_backtrace_fetch": 0, 2024-03-30T02:47:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rfiles": 0, 2024-03-30T02:47:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rbytes": 0, 2024-03-30T02:47:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rsnaps": 0, 2024-03-30T02:47:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_fetch": 0, 2024-03-30T02:47:19.407 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_inodes": 0, 2024-03-30T02:47:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T02:47:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T02:47:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_file_inodes": 0, 2024-03-30T02:47:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_inode_file_caps": 0, 2024-03-30T02:47:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_revoke": 0, 2024-03-30T02:47:19.408 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_grant": 0, 2024-03-30T02:47:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T02:47:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps": 0, 2024-03-30T02:47:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps_dirty": 0, 2024-03-30T02:47:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_cap_release": 0, 2024-03-30T02:47:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: "process_request_cap_release": 0 2024-03-30T02:47:19.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache": { 2024-03-30T02:47:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update": 0, 2024-03-30T02:47:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_try_discover": 0, 2024-03-30T02:47:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_send_discover": 0, 2024-03-30T02:47:19.410 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_handle_discover": 0, 2024-03-30T02:47:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays": 0, 2024-03-30T02:47:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_created": 0, 2024-03-30T02:47:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_enqueued": 0, 2024-03-30T02:47:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_migrated": 0, 2024-03-30T02:47:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:19.411 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_started": 0, 2024-03-30T02:47:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_completed": 0, 2024-03-30T02:47:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T02:47:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_exportdir": 0, 2024-03-30T02:47:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_flush": 0, 2024-03-30T02:47:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragmentdir": 0, 2024-03-30T02:47:19.412 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_inodestats": 0 2024-03-30T02:47:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log": { 2024-03-30T02:47:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "evadd": 0, 2024-03-30T02:47:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "evex": 0, 2024-03-30T02:47:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "evtrm": 0, 2024-03-30T02:47:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "ev": 7279, 2024-03-30T02:47:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "evexg": 0, 2024-03-30T02:47:19.413 INFO:teuthology.orchestra.run.smithi067.stdout: "evexd": 0, 2024-03-30T02:47:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "segadd": 0, 2024-03-30T02:47:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "segex": 0, 2024-03-30T02:47:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "segtrm": 0, 2024-03-30T02:47:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "seg": 11, 2024-03-30T02:47:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "segexg": 0, 2024-03-30T02:47:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "segexd": 0, 2024-03-30T02:47:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "expos": 4194304, 2024-03-30T02:47:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "wrpos": 25814519, 2024-03-30T02:47:19.414 INFO:teuthology.orchestra.run.smithi067.stdout: "rdpos": 25813945, 2024-03-30T02:47:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "jlat": { 2024-03-30T02:47:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "replayed": 7279 2024-03-30T02:47:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.415 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_mem": { 2024-03-30T02:47:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "ino": 6068, 2024-03-30T02:47:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "ino+": 6085, 2024-03-30T02:47:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "ino-": 17, 2024-03-30T02:47:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "dir": 529, 2024-03-30T02:47:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "dir+": 554, 2024-03-30T02:47:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "dir-": 25, 2024-03-30T02:47:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "dn": 6099, 2024-03-30T02:47:19.416 INFO:teuthology.orchestra.run.smithi067.stdout: "dn+": 6099, 2024-03-30T02:47:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "dn-": 0, 2024-03-30T02:47:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "cap": 0, 2024-03-30T02:47:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "cap+": 0, 2024-03-30T02:47:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "cap-": 0, 2024-03-30T02:47:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "rss": 72644, 2024-03-30T02:47:19.417 INFO:teuthology.orchestra.run.smithi067.stdout: "heap": 207156 2024-03-30T02:47:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_server": { 2024-03-30T02:47:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_client_request": 0, 2024-03-30T02:47:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_server_request": 0, 2024-03-30T02:47:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_request": 0, 2024-03-30T02:47:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_session": 0, 2024-03-30T02:47:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_peer_request": 0, 2024-03-30T02:47:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "req_create_latency": { 2024-03-30T02:47:19.418 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getattr_latency": { 2024-03-30T02:47:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "req_link_latency": { 2024-03-30T02:47:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.420 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookup_latency": { 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.421 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupino_latency": { 2024-03-30T02:47:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupname_latency": { 2024-03-30T02:47:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.422 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:19.423 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lssnap_latency": { 2024-03-30T02:47:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.424 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mkdir_latency": { 2024-03-30T02:47:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mknod_latency": { 2024-03-30T02:47:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.425 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mksnap_latency": { 2024-03-30T02:47:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "req_open_latency": { 2024-03-30T02:47:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.426 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "req_readdir_latency": { 2024-03-30T02:47:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rename_latency": { 2024-03-30T02:47:19.427 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmdir_latency": { 2024-03-30T02:47:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.429 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:19.430 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setattr_latency": { 2024-03-30T02:47:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.431 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.432 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setlayout_latency": { 2024-03-30T02:47:19.433 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.433 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.433 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.433 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setxattr_latency": { 2024-03-30T02:47:19.433 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.433 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.433 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.434 INFO:teuthology.orchestra.run.smithi067.stdout: "req_symlink_latency": { 2024-03-30T02:47:19.434 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.434 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.434 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.434 INFO:teuthology.orchestra.run.smithi067.stdout: "req_unlink_latency": { 2024-03-30T02:47:19.434 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.434 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.435 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.435 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:19.435 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:19.435 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:19.435 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.435 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.435 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:19.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.436 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sessions": { 2024-03-30T02:47:19.436 INFO:teuthology.orchestra.run.smithi067.stdout: "session_count": 1, 2024-03-30T02:47:19.436 INFO:teuthology.orchestra.run.smithi067.stdout: "session_add": 2, 2024-03-30T02:47:19.436 INFO:teuthology.orchestra.run.smithi067.stdout: "session_remove": 1, 2024-03-30T02:47:19.436 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_open": 1, 2024-03-30T02:47:19.436 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_stale": 0, 2024-03-30T02:47:19.436 INFO:teuthology.orchestra.run.smithi067.stdout: "total_load": 0, 2024-03-30T02:47:19.436 INFO:teuthology.orchestra.run.smithi067.stdout: "average_load": 0, 2024-03-30T02:47:19.437 INFO:teuthology.orchestra.run.smithi067.stdout: "avg_session_uptime": 2131, 2024-03-30T02:47:19.437 INFO:teuthology.orchestra.run.smithi067.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:19.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.437 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-03-30T02:47:19.437 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:19.437 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:19.437 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:19.437 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:19.437 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:19.438 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:19.438 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:19.438 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:19.438 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:19.438 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:19.438 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:19.438 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:19.438 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:19.438 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:19.439 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:19.439 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:19.439 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:19.439 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:19.439 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:19.439 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:19.439 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:19.439 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:19.439 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:19.440 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:19.440 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:19.440 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:19.440 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:19.440 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:19.440 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:19.440 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:19.440 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:19.440 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-03-30T02:47:19.441 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:19.441 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:19.441 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:19.441 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:19.441 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 417691, 2024-03-30T02:47:19.441 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 6532, 2024-03-30T02:47:19.441 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-03-30T02:47:19.441 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-03-30T02:47:19.441 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-03-30T02:47:19.442 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-03-30T02:47:19.442 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:19.442 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:19.442 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:19.442 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:19.442 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 9896, 2024-03-30T02:47:19.442 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 209, 2024-03-30T02:47:19.442 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:19.442 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:19.443 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:19.443 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-03-30T02:47:19.443 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 19134844, 2024-03-30T02:47:19.443 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 219095, 2024-03-30T02:47:19.443 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:19.443 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-03-30T02:47:19.443 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:19.443 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-03-30T02:47:19.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.444 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter": { 2024-03-30T02:47:19.444 INFO:teuthology.orchestra.run.smithi067.stdout: "op_active": 0, 2024-03-30T02:47:19.444 INFO:teuthology.orchestra.run.smithi067.stdout: "op_laggy": 0, 2024-03-30T02:47:19.444 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send": 2169, 2024-03-30T02:47:19.444 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send_bytes": 21, 2024-03-30T02:47:19.444 INFO:teuthology.orchestra.run.smithi067.stdout: "op_resend": 3, 2024-03-30T02:47:19.444 INFO:teuthology.orchestra.run.smithi067.stdout: "op_reply": 2169, 2024-03-30T02:47:19.444 INFO:teuthology.orchestra.run.smithi067.stdout: "op_latency": { 2024-03-30T02:47:19.444 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 2169, 2024-03-30T02:47:19.445 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 15.366664499, 2024-03-30T02:47:19.445 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.007084677 2024-03-30T02:47:19.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.445 INFO:teuthology.orchestra.run.smithi067.stdout: "op_inflight": 0, 2024-03-30T02:47:19.445 INFO:teuthology.orchestra.run.smithi067.stdout: "oplen_avg": { 2024-03-30T02:47:19.445 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 2169, 2024-03-30T02:47:19.445 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 2170 2024-03-30T02:47:19.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.445 INFO:teuthology.orchestra.run.smithi067.stdout: "op": 2169, 2024-03-30T02:47:19.446 INFO:teuthology.orchestra.run.smithi067.stdout: "op_r": 2169, 2024-03-30T02:47:19.446 INFO:teuthology.orchestra.run.smithi067.stdout: "op_w": 0, 2024-03-30T02:47:19.446 INFO:teuthology.orchestra.run.smithi067.stdout: "op_rmw": 0, 2024-03-30T02:47:19.446 INFO:teuthology.orchestra.run.smithi067.stdout: "op_pg": 0, 2024-03-30T02:47:19.446 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_stat": 1314, 2024-03-30T02:47:19.446 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_create": 0, 2024-03-30T02:47:19.446 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_read": 853, 2024-03-30T02:47:19.446 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_write": 0, 2024-03-30T02:47:19.446 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writefull": 0, 2024-03-30T02:47:19.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writesame": 0, 2024-03-30T02:47:19.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_append": 0, 2024-03-30T02:47:19.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_zero": 0, 2024-03-30T02:47:19.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_truncate": 0, 2024-03-30T02:47:19.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_delete": 0, 2024-03-30T02:47:19.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_mapext": 0, 2024-03-30T02:47:19.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:19.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:19.447 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_getxattr": 1, 2024-03-30T02:47:19.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_setxattr": 0, 2024-03-30T02:47:19.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:19.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:19.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:19.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_call": 0, 2024-03-30T02:47:19.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_watch": 0, 2024-03-30T02:47:19.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_notify": 0, 2024-03-30T02:47:19.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:19.448 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_other": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_active": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_send": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_resend": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_ping": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_active": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_send": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_resend": 0, 2024-03-30T02:47:19.449 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_active": 0, 2024-03-30T02:47:19.450 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_send": 0, 2024-03-30T02:47:19.450 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_resend": 0, 2024-03-30T02:47:19.450 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_active": 0, 2024-03-30T02:47:19.450 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_send": 0, 2024-03-30T02:47:19.450 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_resend": 0, 2024-03-30T02:47:19.450 INFO:teuthology.orchestra.run.smithi067.stdout: "command_active": 0, 2024-03-30T02:47:19.450 INFO:teuthology.orchestra.run.smithi067.stdout: "command_send": 0, 2024-03-30T02:47:19.450 INFO:teuthology.orchestra.run.smithi067.stdout: "command_resend": 0, 2024-03-30T02:47:19.451 INFO:teuthology.orchestra.run.smithi067.stdout: "map_epoch": 96, 2024-03-30T02:47:19.451 INFO:teuthology.orchestra.run.smithi067.stdout: "map_full": 0, 2024-03-30T02:47:19.451 INFO:teuthology.orchestra.run.smithi067.stdout: "map_inc": 13, 2024-03-30T02:47:19.451 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_sessions": 6, 2024-03-30T02:47:19.451 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_open": 6, 2024-03-30T02:47:19.451 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_close": 0, 2024-03-30T02:47:19.451 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_laggy": 0, 2024-03-30T02:47:19.451 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_wr": 0, 2024-03-30T02:47:19.451 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_rd": 2, 2024-03-30T02:47:19.452 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_del": 0 2024-03-30T02:47:19.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.452 INFO:teuthology.orchestra.run.smithi067.stdout: "oft": { 2024-03-30T02:47:19.452 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_objs": 0, 2024-03-30T02:47:19.452 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:19.452 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_updates": 0, 2024-03-30T02:47:19.452 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_removes": 0 2024-03-30T02:47:19.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.452 INFO:teuthology.orchestra.run.smithi067.stdout: "osd-slow-ops": { 2024-03-30T02:47:19.453 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_ops_count": 0 2024-03-30T02:47:19.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.453 INFO:teuthology.orchestra.run.smithi067.stdout: "purge_queue": { 2024-03-30T02:47:19.453 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:19.453 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops_high_water": 0, 2024-03-30T02:47:19.453 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing": 0, 2024-03-30T02:47:19.453 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_high_water": 0, 2024-03-30T02:47:19.453 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed": 0, 2024-03-30T02:47:19.453 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:19.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.454 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:19.454 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:19.454 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-03-30T02:47:19.454 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:19.454 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 2419, 2024-03-30T02:47:19.454 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 23936685, 2024-03-30T02:47:19.454 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:19.454 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 2419, 2024-03-30T02:47:19.455 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-03-30T02:47:19.455 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-03-30T02:47:19.455 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 2419, 2024-03-30T02:47:19.455 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 23936685, 2024-03-30T02:47:19.455 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:19.455 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.455 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.455 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.455 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:19.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.456 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:19.456 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:19.456 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-03-30T02:47:19.456 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:19.456 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-03-30T02:47:19.456 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-03-30T02:47:19.456 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 2169, 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 21623535, 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 2169, 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 21623535, 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:19.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.458 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:19.458 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:19.458 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1024, 2024-03-30T02:47:19.458 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:19.458 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-03-30T02:47:19.458 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-03-30T02:47:19.458 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:19.459 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:19.459 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 2169, 2024-03-30T02:47:19.459 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 2169, 2024-03-30T02:47:19.459 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 2169, 2024-03-30T02:47:19.459 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 2169, 2024-03-30T02:47:19.459 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:19.459 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.459 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.459 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.460 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:19.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.460 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:19.460 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:19.460 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-03-30T02:47:19.460 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:19.460 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-03-30T02:47:19.460 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-03-30T02:47:19.460 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.461 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:19.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:19.462 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle-0x55ba231e8640": { 2024-03-30T02:47:19.462 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:19.462 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-03-30T02:47:19.462 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:19.462 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-03-30T02:47:19.462 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-03-30T02:47:19.462 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:19.463 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:19.464 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:19.464 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:47:19.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:19 smithi067 ceph-mon[37808]: pgmap v319: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-03-30T02:47:19.690 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-30T02:47:19.690 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-30T02:47:19.690 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-30T02:47:19.690 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-30T02:47:19.690 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-30T02:47:19.690 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-03-30T02:47:19.691 DEBUG:tasks.check_counter:Getting stats from d 2024-03-30T02:47:19.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.d perf dump 2024-03-30T02:47:19.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:19 smithi170 ceph-mon[78798]: pgmap v319: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-03-30T02:47:19.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:19 smithi082 ceph-mon[79176]: pgmap v319: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2024-03-30T02:47:20.063 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:47:21.296 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:47:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 0, 2024-03-30T02:47:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2, 2024-03-30T02:47:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 0, 2024-03-30T02:47:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 0, 2024-03-30T02:47:21.297 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-03-30T02:47:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-03-30T02:47:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.003605113, 2024-03-30T02:47:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000000000, 2024-03-30T02:47:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.000804510, 2024-03-30T02:47:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T02:47:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:21.298 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T02:47:21.299 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T02:47:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 2, 2024-03-30T02:47:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 159, 2024-03-30T02:47:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 320, 2024-03-30T02:47:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 180192, 2024-03-30T02:47:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-03-30T02:47:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 3, 2024-03-30T02:47:21.300 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.019959130, 2024-03-30T02:47:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.013670015, 2024-03-30T02:47:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.002718584, 2024-03-30T02:47:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T02:47:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 156, 2024-03-30T02:47:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.024372629, 2024-03-30T02:47:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000156234 2024-03-30T02:47:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.301 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-30T02:47:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 180192 2024-03-30T02:47:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:21.302 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 225, 2024-03-30T02:47:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 210, 2024-03-30T02:47:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 1833408, 2024-03-30T02:47:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 119024, 2024-03-30T02:47:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 2, 2024-03-30T02:47:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-03-30T02:47:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.059120940, 2024-03-30T02:47:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.019126319, 2024-03-30T02:47:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.052248549, 2024-03-30T02:47:21.303 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.004959214, 2024-03-30T02:47:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 210, 2024-03-30T02:47:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.019847563, 2024-03-30T02:47:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000094512 2024-03-30T02:47:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:21.304 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1833408, 2024-03-30T02:47:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 119024 2024-03-30T02:47:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-03-30T02:47:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-03-30T02:47:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-03-30T02:47:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.305 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-03-30T02:47:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:21.306 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:21.307 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:21.308 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-03-30T02:47:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:21.309 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148977, 2024-03-30T02:47:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 30, 2024-03-30T02:47:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-03-30T02:47:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-03-30T02:47:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-03-30T02:47:21.310 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:21.311 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-03-30T02:47:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 928, 2024-03-30T02:47:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-03-30T02:47:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-03-30T02:47:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-03-30T02:47:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:21.312 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 223, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 1799297, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 223, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 223, 2024-03-30T02:47:21.313 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 1799297, 2024-03-30T02:47:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:21.314 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:21.314 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:47:21.591 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-30T02:47:21.591 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-30T02:47:21.591 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-30T02:47:21.591 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-30T02:47:21.592 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-30T02:47:21.592 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-03-30T02:47:21.592 DEBUG:tasks.check_counter:Getting stats from g 2024-03-30T02:47:21.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.g perf dump 2024-03-30T02:47:21.963 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:47:22.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:22 smithi082 ceph-mon[79176]: pgmap v320: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-03-30T02:47:22.836 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:22 smithi067 ceph-mon[37808]: pgmap v320: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-03-30T02:47:22.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:22 smithi170 ceph-mon[78798]: pgmap v320: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-03-30T02:47:23.123 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:47:23.123 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:23.123 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 6272, 2024-03-30T02:47:23.123 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 6079, 2024-03-30T02:47:23.124 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 1956404, 2024-03-30T02:47:23.124 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 13887404, 2024-03-30T02:47:23.124 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 21, 2024-03-30T02:47:23.124 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 6, 2024-03-30T02:47:23.124 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.862051871, 2024-03-30T02:47:23.124 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.343231828, 2024-03-30T02:47:23.124 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.205879959, 2024-03-30T02:47:23.124 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.208964480, 2024-03-30T02:47:23.124 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:23.125 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 6074, 2024-03-30T02:47:23.125 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 1.375296074, 2024-03-30T02:47:23.125 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000226423 2024-03-30T02:47:23.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.125 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:23.125 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 1188, 2024-03-30T02:47:23.125 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000219864, 2024-03-30T02:47:23.126 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000185 2024-03-30T02:47:23.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.126 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1072, 2024-03-30T02:47:23.126 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 304880 2024-03-30T02:47:23.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.127 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:23.127 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 18494, 2024-03-30T02:47:23.127 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 16907, 2024-03-30T02:47:23.127 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 8496855, 2024-03-30T02:47:23.127 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 7366233, 2024-03-30T02:47:23.128 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 18, 2024-03-30T02:47:23.128 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 6, 2024-03-30T02:47:23.128 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 2.541691877, 2024-03-30T02:47:23.128 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 1.352111593, 2024-03-30T02:47:23.128 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 1.499639048, 2024-03-30T02:47:23.128 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.163096695, 2024-03-30T02:47:23.128 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:23.128 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 16908, 2024-03-30T02:47:23.128 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 4.957906837, 2024-03-30T02:47:23.129 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000293228 2024-03-30T02:47:23.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.129 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:23.129 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 23429, 2024-03-30T02:47:23.129 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.006274903, 2024-03-30T02:47:23.129 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000267 2024-03-30T02:47:23.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.129 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-03-30T02:47:23.130 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 122224 2024-03-30T02:47:23.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.130 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:23.130 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 11719, 2024-03-30T02:47:23.130 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 11545, 2024-03-30T02:47:23.130 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 4918542, 2024-03-30T02:47:23.130 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 4655714, 2024-03-30T02:47:23.131 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 25, 2024-03-30T02:47:23.131 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 10, 2024-03-30T02:47:23.131 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 1.285766207, 2024-03-30T02:47:23.131 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.442794268, 2024-03-30T02:47:23.131 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.336301312, 2024-03-30T02:47:23.132 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.381120078, 2024-03-30T02:47:23.132 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:23.132 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 11544, 2024-03-30T02:47:23.132 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 2.469138551, 2024-03-30T02:47:23.132 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000213889 2024-03-30T02:47:23.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.133 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:23.133 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 885, 2024-03-30T02:47:23.133 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000205551, 2024-03-30T02:47:23.133 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000232 2024-03-30T02:47:23.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.133 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1982128, 2024-03-30T02:47:23.133 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 365488 2024-03-30T02:47:23.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.134 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-03-30T02:47:23.134 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 1, 2024-03-30T02:47:23.134 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-03-30T02:47:23.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.134 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-MDSRank": { 2024-03-30T02:47:23.134 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-03-30T02:47:23.134 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-03-30T02:47:23.135 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 13288, 2024-03-30T02:47:23.135 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 6.006400006, 2024-03-30T02:47:23.135 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000452016 2024-03-30T02:47:23.135 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:23.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.135 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:23.135 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-03-30T02:47:23.135 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-03-30T02:47:23.135 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 126, 2024-03-30T02:47:23.136 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.035935230, 2024-03-30T02:47:23.136 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000285200 2024-03-30T02:47:23.136 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:23.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.136 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": { 2024-03-30T02:47:23.136 INFO:teuthology.orchestra.run.smithi067.stdout: "request": 577, 2024-03-30T02:47:23.136 INFO:teuthology.orchestra.run.smithi067.stdout: "reply": 515, 2024-03-30T02:47:23.136 INFO:teuthology.orchestra.run.smithi067.stdout: "reply_latency": { 2024-03-30T02:47:23.137 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 515, 2024-03-30T02:47:23.137 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 1.459891342, 2024-03-30T02:47:23.138 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.002834740 2024-03-30T02:47:23.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.138 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_reply": 0, 2024-03-30T02:47:23.138 INFO:teuthology.orchestra.run.smithi067.stdout: "forward": 62, 2024-03-30T02:47:23.139 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_complete": 0, 2024-03-30T02:47:23.139 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:23.139 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_commit": 47, 2024-03-30T02:47:23.139 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_split": 4, 2024-03-30T02:47:23.139 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_merge": 0, 2024-03-30T02:47:23.139 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes": 3586, 2024-03-30T02:47:23.139 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_top": 229, 2024-03-30T02:47:23.140 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_bottom": 3357, 2024-03-30T02:47:23.140 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pin_tail": 0, 2024-03-30T02:47:23.140 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pinned": 3258, 2024-03-30T02:47:23.140 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_expired": 22, 2024-03-30T02:47:23.140 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_with_caps": 2416, 2024-03-30T02:47:23.140 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": 2416, 2024-03-30T02:47:23.140 INFO:teuthology.orchestra.run.smithi067.stdout: "subtrees": 119, 2024-03-30T02:47:23.140 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse": 819, 2024-03-30T02:47:23.141 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_hit": 514, 2024-03-30T02:47:23.141 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_forward": 58, 2024-03-30T02:47:23.141 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_discover": 127, 2024-03-30T02:47:23.141 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:23.141 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:23.142 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_lock": 0, 2024-03-30T02:47:23.142 INFO:teuthology.orchestra.run.smithi067.stdout: "load_cent": 0, 2024-03-30T02:47:23.142 INFO:teuthology.orchestra.run.smithi067.stdout: "q": 0, 2024-03-30T02:47:23.142 INFO:teuthology.orchestra.run.smithi067.stdout: "exported": 1, 2024-03-30T02:47:23.142 INFO:teuthology.orchestra.run.smithi067.stdout: "exported_inodes": 22, 2024-03-30T02:47:23.143 INFO:teuthology.orchestra.run.smithi067.stdout: "imported": 108, 2024-03-30T02:47:23.143 INFO:teuthology.orchestra.run.smithi067.stdout: "imported_inodes": 2302, 2024-03-30T02:47:23.143 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:23.143 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_backtrace_fetch": 20097, 2024-03-30T02:47:23.143 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:23.144 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rfiles": 85384, 2024-03-30T02:47:23.144 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rbytes": 1566089959, 2024-03-30T02:47:23.144 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rsnaps": 6, 2024-03-30T02:47:23.144 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_fetch": 20097, 2024-03-30T02:47:23.144 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:23.144 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:23.144 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:23.145 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_inodes": 591, 2024-03-30T02:47:23.145 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T02:47:23.145 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dirfrag_rstats": 591, 2024-03-30T02:47:23.145 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_file_inodes": 19506, 2024-03-30T02:47:23.145 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_inode_file_caps": 4, 2024-03-30T02:47:23.145 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_revoke": 161, 2024-03-30T02:47:23.145 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_grant": 352, 2024-03-30T02:47:23.145 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_trunc": 5, 2024-03-30T02:47:23.145 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:23.146 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:23.146 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps": 3847, 2024-03-30T02:47:23.146 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps_dirty": 3434, 2024-03-30T02:47:23.146 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_cap_release": 65, 2024-03-30T02:47:23.146 INFO:teuthology.orchestra.run.smithi067.stdout: "process_request_cap_release": 214 2024-03-30T02:47:23.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.146 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache": { 2024-03-30T02:47:23.147 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update": 0, 2024-03-30T02:47:23.147 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:23.147 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_try_discover": 0, 2024-03-30T02:47:23.147 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_send_discover": 112, 2024-03-30T02:47:23.147 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_handle_discover": 14, 2024-03-30T02:47:23.147 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays": 0, 2024-03-30T02:47:23.147 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:23.147 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:23.148 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_created": 39, 2024-03-30T02:47:23.148 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_enqueued": 39, 2024-03-30T02:47:23.148 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:23.148 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_migrated": 0, 2024-03-30T02:47:23.148 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:23.148 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:23.148 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:23.149 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_started": 0, 2024-03-30T02:47:23.149 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_completed": 0, 2024-03-30T02:47:23.149 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T02:47:23.149 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_exportdir": 1, 2024-03-30T02:47:23.149 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_flush": 0, 2024-03-30T02:47:23.149 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragmentdir": 5, 2024-03-30T02:47:23.149 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:23.149 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_inodestats": 0 2024-03-30T02:47:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log": { 2024-03-30T02:47:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "evadd": 4488, 2024-03-30T02:47:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "evex": 0, 2024-03-30T02:47:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "evtrm": 0, 2024-03-30T02:47:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "ev": 4488, 2024-03-30T02:47:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "evexg": 0, 2024-03-30T02:47:23.150 INFO:teuthology.orchestra.run.smithi067.stdout: "evexd": 0, 2024-03-30T02:47:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "segadd": 6, 2024-03-30T02:47:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "segex": 0, 2024-03-30T02:47:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "segtrm": 0, 2024-03-30T02:47:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "seg": 6, 2024-03-30T02:47:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "segexg": 0, 2024-03-30T02:47:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "segexd": 0, 2024-03-30T02:47:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "expos": 4194304, 2024-03-30T02:47:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "wrpos": 16422237, 2024-03-30T02:47:23.151 INFO:teuthology.orchestra.run.smithi067.stdout: "rdpos": 0, 2024-03-30T02:47:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "jlat": { 2024-03-30T02:47:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 302, 2024-03-30T02:47:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 8.524004234, 2024-03-30T02:47:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.028225179 2024-03-30T02:47:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: "replayed": 0 2024-03-30T02:47:23.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_mem": { 2024-03-30T02:47:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "ino": 3524, 2024-03-30T02:47:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "ino+": 3585, 2024-03-30T02:47:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "ino-": 61, 2024-03-30T02:47:23.153 INFO:teuthology.orchestra.run.smithi067.stdout: "dir": 448, 2024-03-30T02:47:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "dir+": 469, 2024-03-30T02:47:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "dir-": 21, 2024-03-30T02:47:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "dn": 3586, 2024-03-30T02:47:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "dn+": 3660, 2024-03-30T02:47:23.154 INFO:teuthology.orchestra.run.smithi067.stdout: "dn-": 74, 2024-03-30T02:47:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: "cap": 2416, 2024-03-30T02:47:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: "cap+": 3665, 2024-03-30T02:47:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: "cap-": 1249, 2024-03-30T02:47:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: "rss": 75700, 2024-03-30T02:47:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: "heap": 198964 2024-03-30T02:47:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_server": { 2024-03-30T02:47:23.155 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_client_request": 705, 2024-03-30T02:47:23.156 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_server_request": 10, 2024-03-30T02:47:23.156 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_request": 577, 2024-03-30T02:47:23.156 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_session": 13, 2024-03-30T02:47:23.156 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_peer_request": 32, 2024-03-30T02:47:23.156 INFO:teuthology.orchestra.run.smithi067.stdout: "req_create_latency": { 2024-03-30T02:47:23.156 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 130, 2024-03-30T02:47:23.156 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.256178657, 2024-03-30T02:47:23.156 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.001970605 2024-03-30T02:47:23.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.157 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getattr_latency": { 2024-03-30T02:47:23.157 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 11, 2024-03-30T02:47:23.157 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.636090979, 2024-03-30T02:47:23.157 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.057826452 2024-03-30T02:47:23.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "req_link_latency": { 2024-03-30T02:47:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookup_latency": { 2024-03-30T02:47:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 71, 2024-03-30T02:47:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.046863863, 2024-03-30T02:47:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000660054 2024-03-30T02:47:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.160 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupino_latency": { 2024-03-30T02:47:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupname_latency": { 2024-03-30T02:47:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:23.162 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.163 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lssnap_latency": { 2024-03-30T02:47:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mkdir_latency": { 2024-03-30T02:47:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mknod_latency": { 2024-03-30T02:47:23.165 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mksnap_latency": { 2024-03-30T02:47:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.166 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: "req_open_latency": { 2024-03-30T02:47:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 62, 2024-03-30T02:47:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.031148822, 2024-03-30T02:47:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000502400 2024-03-30T02:47:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: "req_readdir_latency": { 2024-03-30T02:47:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 197, 2024-03-30T02:47:23.167 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.272577321, 2024-03-30T02:47:23.168 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.001383641 2024-03-30T02:47:23.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.168 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rename_latency": { 2024-03-30T02:47:23.168 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.168 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.168 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.169 INFO:teuthology.orchestra.run.smithi067.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:23.169 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.169 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.169 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.169 INFO:teuthology.orchestra.run.smithi067.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:23.170 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.170 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.170 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.170 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmdir_latency": { 2024-03-30T02:47:23.170 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.171 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.171 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.171 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:23.171 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.171 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.171 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.171 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:23.172 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.172 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.172 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.172 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setattr_latency": { 2024-03-30T02:47:23.172 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 5, 2024-03-30T02:47:23.172 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.092138622, 2024-03-30T02:47:23.172 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.018427724 2024-03-30T02:47:23.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.173 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:23.173 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.173 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.173 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.173 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:23.173 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.173 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.174 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.174 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setlayout_latency": { 2024-03-30T02:47:23.174 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.174 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.175 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.175 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setxattr_latency": { 2024-03-30T02:47:23.175 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.175 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.175 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.175 INFO:teuthology.orchestra.run.smithi067.stdout: "req_symlink_latency": { 2024-03-30T02:47:23.176 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.176 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.176 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.176 INFO:teuthology.orchestra.run.smithi067.stdout: "req_unlink_latency": { 2024-03-30T02:47:23.176 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 39, 2024-03-30T02:47:23.176 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.124893078, 2024-03-30T02:47:23.176 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.003202386 2024-03-30T02:47:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:23.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.178 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sessions": { 2024-03-30T02:47:23.178 INFO:teuthology.orchestra.run.smithi067.stdout: "session_count": 1, 2024-03-30T02:47:23.178 INFO:teuthology.orchestra.run.smithi067.stdout: "session_add": 2, 2024-03-30T02:47:23.178 INFO:teuthology.orchestra.run.smithi067.stdout: "session_remove": 1, 2024-03-30T02:47:23.178 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_open": 1, 2024-03-30T02:47:23.178 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_stale": 0, 2024-03-30T02:47:23.178 INFO:teuthology.orchestra.run.smithi067.stdout: "total_load": 106, 2024-03-30T02:47:23.179 INFO:teuthology.orchestra.run.smithi067.stdout: "average_load": 106, 2024-03-30T02:47:23.179 INFO:teuthology.orchestra.run.smithi067.stdout: "avg_session_uptime": 2139, 2024-03-30T02:47:23.179 INFO:teuthology.orchestra.run.smithi067.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:23.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.179 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-03-30T02:47:23.179 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:23.179 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:23.180 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:23.180 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:23.180 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:23.180 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:23.180 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:23.180 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:23.181 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:23.181 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:23.181 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:23.181 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:23.181 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:23.181 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:23.181 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:23.182 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:23.182 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:23.182 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:23.182 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:23.182 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:23.182 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:23.183 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:23.183 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:23.183 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:23.183 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:23.183 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:23.183 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:23.184 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:23.184 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:23.184 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:23.184 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:23.184 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-03-30T02:47:23.184 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:23.185 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:23.185 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:23.185 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:23.185 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 390247, 2024-03-30T02:47:23.185 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 2752, 2024-03-30T02:47:23.186 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 1232, 2024-03-30T02:47:23.186 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 14, 2024-03-30T02:47:23.186 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-03-30T02:47:23.186 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-03-30T02:47:23.186 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:23.186 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:23.187 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:23.187 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:23.187 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 9656, 2024-03-30T02:47:23.187 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 203, 2024-03-30T02:47:23.187 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:23.187 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:23.187 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:23.188 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-03-30T02:47:23.188 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 12157979, 2024-03-30T02:47:23.188 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 127899, 2024-03-30T02:47:23.188 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:23.188 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-03-30T02:47:23.188 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:23.188 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-03-30T02:47:23.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.189 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter": { 2024-03-30T02:47:23.189 INFO:teuthology.orchestra.run.smithi067.stdout: "op_active": 0, 2024-03-30T02:47:23.189 INFO:teuthology.orchestra.run.smithi067.stdout: "op_laggy": 0, 2024-03-30T02:47:23.189 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send": 20688, 2024-03-30T02:47:23.189 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send_bytes": 13331435, 2024-03-30T02:47:23.189 INFO:teuthology.orchestra.run.smithi067.stdout: "op_resend": 27, 2024-03-30T02:47:23.189 INFO:teuthology.orchestra.run.smithi067.stdout: "op_reply": 20688, 2024-03-30T02:47:23.189 INFO:teuthology.orchestra.run.smithi067.stdout: "op_latency": { 2024-03-30T02:47:23.190 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 20688, 2024-03-30T02:47:23.190 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 37.709184313, 2024-03-30T02:47:23.190 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.001822756 2024-03-30T02:47:23.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.190 INFO:teuthology.orchestra.run.smithi067.stdout: "op_inflight": 0, 2024-03-30T02:47:23.190 INFO:teuthology.orchestra.run.smithi067.stdout: "oplen_avg": { 2024-03-30T02:47:23.190 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 20688, 2024-03-30T02:47:23.191 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 20845 2024-03-30T02:47:23.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.191 INFO:teuthology.orchestra.run.smithi067.stdout: "op": 20688, 2024-03-30T02:47:23.191 INFO:teuthology.orchestra.run.smithi067.stdout: "op_r": 20110, 2024-03-30T02:47:23.191 INFO:teuthology.orchestra.run.smithi067.stdout: "op_w": 578, 2024-03-30T02:47:23.191 INFO:teuthology.orchestra.run.smithi067.stdout: "op_rmw": 0, 2024-03-30T02:47:23.192 INFO:teuthology.orchestra.run.smithi067.stdout: "op_pg": 0, 2024-03-30T02:47:23.192 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_stat": 0, 2024-03-30T02:47:23.192 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_create": 10, 2024-03-30T02:47:23.192 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_read": 13, 2024-03-30T02:47:23.192 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_write": 315, 2024-03-30T02:47:23.192 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writefull": 38, 2024-03-30T02:47:23.192 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writesame": 0, 2024-03-30T02:47:23.192 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_append": 0, 2024-03-30T02:47:23.192 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_zero": 0, 2024-03-30T02:47:23.193 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_truncate": 4, 2024-03-30T02:47:23.193 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_delete": 92, 2024-03-30T02:47:23.193 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_mapext": 0, 2024-03-30T02:47:23.193 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:23.193 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:23.193 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_getxattr": 20097, 2024-03-30T02:47:23.193 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_setxattr": 20, 2024-03-30T02:47:23.193 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:23.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:23.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:23.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_call": 0, 2024-03-30T02:47:23.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_watch": 0, 2024-03-30T02:47:23.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_notify": 0, 2024-03-30T02:47:23.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:23.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls": 0, 2024-03-30T02:47:23.194 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:23.195 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_other": 5, 2024-03-30T02:47:23.195 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_active": 0, 2024-03-30T02:47:23.195 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_send": 0, 2024-03-30T02:47:23.195 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_resend": 0, 2024-03-30T02:47:23.195 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_ping": 0, 2024-03-30T02:47:23.195 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_active": 0, 2024-03-30T02:47:23.195 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_send": 0, 2024-03-30T02:47:23.196 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_resend": 0, 2024-03-30T02:47:23.196 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_active": 0, 2024-03-30T02:47:23.196 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_send": 0, 2024-03-30T02:47:23.196 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_resend": 0, 2024-03-30T02:47:23.196 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_active": 0, 2024-03-30T02:47:23.196 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_send": 0, 2024-03-30T02:47:23.196 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_resend": 0, 2024-03-30T02:47:23.197 INFO:teuthology.orchestra.run.smithi067.stdout: "command_active": 0, 2024-03-30T02:47:23.197 INFO:teuthology.orchestra.run.smithi067.stdout: "command_send": 0, 2024-03-30T02:47:23.197 INFO:teuthology.orchestra.run.smithi067.stdout: "command_resend": 0, 2024-03-30T02:47:23.197 INFO:teuthology.orchestra.run.smithi067.stdout: "map_epoch": 96, 2024-03-30T02:47:23.197 INFO:teuthology.orchestra.run.smithi067.stdout: "map_full": 0, 2024-03-30T02:47:23.198 INFO:teuthology.orchestra.run.smithi067.stdout: "map_inc": 13, 2024-03-30T02:47:23.198 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_sessions": 12, 2024-03-30T02:47:23.198 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_open": 12, 2024-03-30T02:47:23.198 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_close": 0, 2024-03-30T02:47:23.198 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_laggy": 0, 2024-03-30T02:47:23.198 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_wr": 202, 2024-03-30T02:47:23.198 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_rd": 0, 2024-03-30T02:47:23.199 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_del": 49 2024-03-30T02:47:23.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.199 INFO:teuthology.orchestra.run.smithi067.stdout: "oft": { 2024-03-30T02:47:23.199 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_objs": 1, 2024-03-30T02:47:23.199 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:23.199 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_updates": 0, 2024-03-30T02:47:23.199 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_removes": 0 2024-03-30T02:47:23.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.199 INFO:teuthology.orchestra.run.smithi067.stdout: "osd-slow-ops": { 2024-03-30T02:47:23.200 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_ops_count": 621 2024-03-30T02:47:23.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.200 INFO:teuthology.orchestra.run.smithi067.stdout: "purge_queue": { 2024-03-30T02:47:23.200 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:23.200 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops_high_water": 10, 2024-03-30T02:47:23.200 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing": 0, 2024-03-30T02:47:23.200 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_high_water": 5, 2024-03-30T02:47:23.200 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed": 39, 2024-03-30T02:47:23.201 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:23.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.201 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:23.201 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:23.201 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-03-30T02:47:23.201 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:23.201 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 36481, 2024-03-30T02:47:23.202 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 12065047, 2024-03-30T02:47:23.202 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:23.202 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 36481, 2024-03-30T02:47:23.202 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-03-30T02:47:23.202 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-03-30T02:47:23.202 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 36481, 2024-03-30T02:47:23.202 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 12065047, 2024-03-30T02:47:23.203 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:23.203 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.203 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.203 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.203 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:23.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.203 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:23.203 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:23.203 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-03-30T02:47:23.204 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:23.204 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-03-30T02:47:23.204 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-03-30T02:47:23.204 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:23.204 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:23.204 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 20688, 2024-03-30T02:47:23.204 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 13335686, 2024-03-30T02:47:23.204 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 20688, 2024-03-30T02:47:23.205 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 13335686, 2024-03-30T02:47:23.205 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:23.205 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.205 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.205 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.205 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:23.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.205 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:23.205 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:23.206 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1024, 2024-03-30T02:47:23.206 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:23.206 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-03-30T02:47:23.206 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-03-30T02:47:23.206 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:23.207 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:23.207 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 20688, 2024-03-30T02:47:23.207 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 20688, 2024-03-30T02:47:23.207 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 20688, 2024-03-30T02:47:23.207 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 20688, 2024-03-30T02:47:23.207 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:23.207 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.207 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.208 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.208 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:23.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.208 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:23.208 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:23.209 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-03-30T02:47:23.209 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:23.209 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 39, 2024-03-30T02:47:23.209 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 4251, 2024-03-30T02:47:23.209 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:23.209 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 39, 2024-03-30T02:47:23.209 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-03-30T02:47:23.209 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-03-30T02:47:23.209 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 13, 2024-03-30T02:47:23.210 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 4251, 2024-03-30T02:47:23.210 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:23.210 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.210 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.210 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.211 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:23.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:23.211 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle-0x55d6a265c0a0": { 2024-03-30T02:47:23.211 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:23.211 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-03-30T02:47:23.211 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:23.211 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 4488, 2024-03-30T02:47:23.211 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 12227933, 2024-03-30T02:47:23.211 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:23.212 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 4488, 2024-03-30T02:47:23.212 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-03-30T02:47:23.212 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-03-30T02:47:23.212 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 302, 2024-03-30T02:47:23.212 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 12227933, 2024-03-30T02:47:23.213 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:23.213 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:23.213 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:23.213 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:23.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:23.213 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:23.213 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:47:23.427 INFO:tasks.check_counter:Daemon mds.g mds.exported=1 2024-03-30T02:47:23.428 INFO:tasks.check_counter:Daemon mds.g mds.imported=108 2024-03-30T02:47:23.428 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-03-30T02:47:23.428 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-30T02:47:23.428 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-30T02:47:23.428 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=4 2024-03-30T02:47:23.428 DEBUG:tasks.check_counter:Getting stats from j 2024-03-30T02:47:23.429 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.j perf dump 2024-03-30T02:47:23.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:23 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:47:23.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:23 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:47:23.809 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.a/config 2024-03-30T02:47:23.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:23 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-30T02:47:24.662 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:24 smithi067 ceph-mon[37808]: pgmap v321: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-03-30T02:47:24.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:24 smithi082 ceph-mon[79176]: pgmap v321: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-03-30T02:47:24.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:24 smithi170 ceph-mon[78798]: pgmap v321: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-03-30T02:47:25.197 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-03-30T02:47:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 4, 2024-03-30T02:47:25.197 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 4, 2024-03-30T02:47:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 2992, 2024-03-30T02:47:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 448, 2024-03-30T02:47:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-03-30T02:47:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-03-30T02:47:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.003755343, 2024-03-30T02:47:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000062885, 2024-03-30T02:47:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.001105646, 2024-03-30T02:47:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T02:47:25.198 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 3, 2024-03-30T02:47:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.001484913, 2024-03-30T02:47:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000494971 2024-03-30T02:47:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:25.199 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 448 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 223, 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 210, 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 1830960, 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 119760, 2024-03-30T02:47:25.200 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 2, 2024-03-30T02:47:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-03-30T02:47:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.059358306, 2024-03-30T02:47:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.022380517, 2024-03-30T02:47:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.044552863, 2024-03-30T02:47:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.005168704, 2024-03-30T02:47:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 209, 2024-03-30T02:47:25.201 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.023065433, 2024-03-30T02:47:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000110360 2024-03-30T02:47:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 1830960, 2024-03-30T02:47:25.202 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 119760 2024-03-30T02:47:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 2, 2024-03-30T02:47:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 160, 2024-03-30T02:47:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 320, 2024-03-30T02:47:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 180368, 2024-03-30T02:47:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-03-30T02:47:25.203 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 3, 2024-03-30T02:47:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.019395856, 2024-03-30T02:47:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.013083001, 2024-03-30T02:47:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.003152191, 2024-03-30T02:47:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-30T02:47:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 157, 2024-03-30T02:47:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.024443931, 2024-03-30T02:47:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000155693 2024-03-30T02:47:25.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-30T02:47:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 180368 2024-03-30T02:47:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.205 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-03-30T02:47:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-03-30T02:47:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-03-30T02:47:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-03-30T02:47:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:25.206 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:25.207 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:25.208 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:25.209 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-03-30T02:47:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148977, 2024-03-30T02:47:25.210 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 30, 2024-03-30T02:47:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-03-30T02:47:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-03-30T02:47:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-03-30T02:47:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-03-30T02:47:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:25.211 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-03-30T02:47:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-03-30T02:47:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-03-30T02:47:25.212 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 928, 2024-03-30T02:47:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-03-30T02:47:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-03-30T02:47:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-03-30T02:47:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-03-30T02:47:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-03-30T02:47:25.213 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-03-30T02:47:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-03-30T02:47:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 225, 2024-03-30T02:47:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 1799557, 2024-03-30T02:47:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 225, 2024-03-30T02:47:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-03-30T02:47:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-03-30T02:47:25.214 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 225, 2024-03-30T02:47:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 1799557, 2024-03-30T02:47:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-03-30T02:47:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-03-30T02:47:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-03-30T02:47:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-03-30T02:47:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:25.215 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-03-30T02:47:25.215 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-03-30T02:47:25.602 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-30T02:47:25.603 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-30T02:47:25.603 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-30T02:47:25.603 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-30T02:47:25.603 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-30T02:47:25.603 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-03-30T02:47:25.603 DEBUG:tasks.check_counter:Getting stats from b 2024-03-30T02:47:25.604 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.b perf dump 2024-03-30T02:47:25.905 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:47:25.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:25.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: pgmap v322: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-30T02:47:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi170 to 182.5M 2024-03-30T02:47:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: Unable to set osd_memory_target on smithi170 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-03-30T02:47:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi082 to 182.4M 2024-03-30T02:47:25.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:25 smithi067 ceph-mon[37808]: Unable to set osd_memory_target on smithi082 to 191351193: error parsing value: Value '191351193' is below minimum 939524096 2024-03-30T02:47:25.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: pgmap v322: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-30T02:47:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi170 to 182.5M 2024-03-30T02:47:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: Unable to set osd_memory_target on smithi170 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-03-30T02:47:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:25.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi082 to 182.4M 2024-03-30T02:47:25.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:25 smithi170 ceph-mon[78798]: Unable to set osd_memory_target on smithi082 to 191351193: error parsing value: Value '191351193' is below minimum 939524096 2024-03-30T02:47:25.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:25.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:26.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: pgmap v322: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-03-30T02:47:26.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:26.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:26.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:26.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:26.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:26.000 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:26.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi170 to 182.5M 2024-03-30T02:47:26.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: Unable to set osd_memory_target on smithi170 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2024-03-30T02:47:26.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:26.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:26.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:26.001 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:26.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi082 to 182.4M 2024-03-30T02:47:26.002 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:25 smithi082 ceph-mon[79176]: Unable to set osd_memory_target on smithi082 to 191351193: error parsing value: Value '191351193' is below minimum 939524096 2024-03-30T02:47:26.903 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-30T02:47:26.904 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:26.904 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 244399, 2024-03-30T02:47:26.904 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 252494, 2024-03-30T02:47:26.904 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 123777489, 2024-03-30T02:47:26.904 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 300974150, 2024-03-30T02:47:26.904 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 27, 2024-03-30T02:47:26.904 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-03-30T02:47:26.905 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 10.914592442, 2024-03-30T02:47:26.905 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 3.369852891, 2024-03-30T02:47:26.905 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 2.781136106, 2024-03-30T02:47:26.905 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 4.105191273, 2024-03-30T02:47:26.905 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:26.905 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 252494, 2024-03-30T02:47:26.905 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 122.296212020, 2024-03-30T02:47:26.905 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000484352 2024-03-30T02:47:26.905 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.906 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:26.906 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 35806, 2024-03-30T02:47:26.906 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005952961, 2024-03-30T02:47:26.906 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000166 2024-03-30T02:47:26.906 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.906 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2281088, 2024-03-30T02:47:26.906 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 237008 2024-03-30T02:47:26.907 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.907 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:26.907 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 489767, 2024-03-30T02:47:26.907 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 670930, 2024-03-30T02:47:26.907 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 214553221, 2024-03-30T02:47:26.907 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 618954956, 2024-03-30T02:47:26.907 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 128, 2024-03-30T02:47:26.907 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 8, 2024-03-30T02:47:26.907 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 23.320777485, 2024-03-30T02:47:26.908 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 8.648086035, 2024-03-30T02:47:26.908 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 33.726582929, 2024-03-30T02:47:26.908 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 5.299208027, 2024-03-30T02:47:26.908 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:26.908 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 670925, 2024-03-30T02:47:26.908 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 226.239566328, 2024-03-30T02:47:26.908 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000337205 2024-03-30T02:47:26.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.909 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:26.909 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 550392, 2024-03-30T02:47:26.909 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.086906391, 2024-03-30T02:47:26.909 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000157 2024-03-30T02:47:26.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.909 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 90304, 2024-03-30T02:47:26.909 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 162432 2024-03-30T02:47:26.909 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.910 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:26.910 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 269399, 2024-03-30T02:47:26.910 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 278673, 2024-03-30T02:47:26.910 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 135035461, 2024-03-30T02:47:26.910 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 437035405, 2024-03-30T02:47:26.910 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 13, 2024-03-30T02:47:26.910 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 6, 2024-03-30T02:47:26.910 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 13.879684008, 2024-03-30T02:47:26.911 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 4.940206628, 2024-03-30T02:47:26.911 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 3.266902678, 2024-03-30T02:47:26.911 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 4.878626269, 2024-03-30T02:47:26.911 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:26.911 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 278673, 2024-03-30T02:47:26.911 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 132.351362739, 2024-03-30T02:47:26.911 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000474934 2024-03-30T02:47:26.911 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.912 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:26.912 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 29826, 2024-03-30T02:47:26.912 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005318357, 2024-03-30T02:47:26.912 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000178 2024-03-30T02:47:26.912 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.912 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 5888, 2024-03-30T02:47:26.912 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 310528 2024-03-30T02:47:26.912 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.913 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-03-30T02:47:26.913 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-30T02:47:26.913 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-30T02:47:26.913 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.913 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-03-30T02:47:26.913 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-30T02:47:26.913 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-30T02:47:26.913 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 432017, 2024-03-30T02:47:26.913 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 227.822672783, 2024-03-30T02:47:26.914 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000527346 2024-03-30T02:47:26.914 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:26.914 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.914 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:26.914 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-30T02:47:26.914 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-30T02:47:26.914 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2569, 2024-03-30T02:47:26.914 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.492801307, 2024-03-30T02:47:26.915 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000191826 2024-03-30T02:47:26.915 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:26.915 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.915 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-03-30T02:47:26.915 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 95982, 2024-03-30T02:47:26.915 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 95332, 2024-03-30T02:47:26.915 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-30T02:47:26.915 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 95332, 2024-03-30T02:47:26.915 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 216.737259292, 2024-03-30T02:47:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002273499 2024-03-30T02:47:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-30T02:47:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 650, 2024-03-30T02:47:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 55, 2024-03-30T02:47:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 5306, 2024-03-30T02:47:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 217, 2024-03-30T02:47:26.916 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-30T02:47:26.917 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 98964, 2024-03-30T02:47:26.917 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 16258, 2024-03-30T02:47:26.917 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 82666, 2024-03-30T02:47:26.917 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 40, 2024-03-30T02:47:26.917 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 75607, 2024-03-30T02:47:26.917 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 11507, 2024-03-30T02:47:26.917 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 74213, 2024-03-30T02:47:26.917 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 74213, 2024-03-30T02:47:26.918 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 573, 2024-03-30T02:47:26.918 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 96672, 2024-03-30T02:47:26.918 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 95410, 2024-03-30T02:47:26.918 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 314, 2024-03-30T02:47:26.918 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 5, 2024-03-30T02:47:26.918 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:26.918 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:26.918 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-30T02:47:26.918 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-30T02:47:26.919 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-30T02:47:26.919 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 591, 2024-03-30T02:47:26.919 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 11509, 2024-03-30T02:47:26.919 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 3, 2024-03-30T02:47:26.919 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 42, 2024-03-30T02:47:26.919 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:26.919 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 642299, 2024-03-30T02:47:26.919 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:26.920 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 85384, 2024-03-30T02:47:26.920 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 1566089959, 2024-03-30T02:47:26.920 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 6, 2024-03-30T02:47:26.920 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 642299, 2024-03-30T02:47:26.920 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:26.920 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:26.920 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:26.920 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 39080, 2024-03-30T02:47:26.920 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 15, 2024-03-30T02:47:26.921 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 39065, 2024-03-30T02:47:26.921 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 603219, 2024-03-30T02:47:26.921 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 343, 2024-03-30T02:47:26.921 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 5591, 2024-03-30T02:47:26.921 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 91437, 2024-03-30T02:47:26.921 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 70, 2024-03-30T02:47:26.921 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:26.921 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:26.922 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 120257, 2024-03-30T02:47:26.922 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 106946, 2024-03-30T02:47:26.922 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 825, 2024-03-30T02:47:26.922 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 12657 2024-03-30T02:47:26.922 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.922 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-03-30T02:47:26.922 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-30T02:47:26.922 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:26.922 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-30T02:47:26.923 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 5, 2024-03-30T02:47:26.923 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 619, 2024-03-30T02:47:26.923 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-30T02:47:26.923 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:26.923 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:26.923 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 1262, 2024-03-30T02:47:26.923 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 1262, 2024-03-30T02:47:26.923 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:26.924 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-30T02:47:26.924 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:26.924 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:26.924 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:26.924 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-30T02:47:26.924 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-30T02:47:26.924 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 15, 2024-03-30T02:47:26.924 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 591, 2024-03-30T02:47:26.925 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-30T02:47:26.925 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 220, 2024-03-30T02:47:26.925 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:26.925 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-30T02:47:26.925 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.925 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-03-30T02:47:26.925 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 211876, 2024-03-30T02:47:26.925 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 136554, 2024-03-30T02:47:26.925 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 136554, 2024-03-30T02:47:26.926 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 75322, 2024-03-30T02:47:26.926 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-30T02:47:26.926 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 1079, 2024-03-30T02:47:26.926 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 333, 2024-03-30T02:47:26.926 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 205, 2024-03-30T02:47:26.926 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 205, 2024-03-30T02:47:26.926 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 130, 2024-03-30T02:47:26.926 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-30T02:47:26.927 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 2, 2024-03-30T02:47:26.927 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 440411099, 2024-03-30T02:47:26.927 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 800756914, 2024-03-30T02:47:26.927 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-30T02:47:26.927 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-30T02:47:26.927 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10764, 2024-03-30T02:47:26.927 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1795.943580253, 2024-03-30T02:47:26.927 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.166847229 2024-03-30T02:47:26.927 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.928 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-30T02:47:26.928 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.928 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-03-30T02:47:26.928 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 97465, 2024-03-30T02:47:26.928 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 110246, 2024-03-30T02:47:26.928 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 12781, 2024-03-30T02:47:26.929 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 6294, 2024-03-30T02:47:26.929 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 7014, 2024-03-30T02:47:26.929 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 720, 2024-03-30T02:47:26.929 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 98964, 2024-03-30T02:47:26.929 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 112114, 2024-03-30T02:47:26.929 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 13150, 2024-03-30T02:47:26.929 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 74213, 2024-03-30T02:47:26.929 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 87320, 2024-03-30T02:47:26.929 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 13107, 2024-03-30T02:47:26.930 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 526868, 2024-03-30T02:47:26.930 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 207156 2024-03-30T02:47:26.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.930 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-03-30T02:47:26.930 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 101578, 2024-03-30T02:47:26.930 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 108, 2024-03-30T02:47:26.930 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 95982, 2024-03-30T02:47:26.930 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 625, 2024-03-30T02:47:26.930 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 222, 2024-03-30T02:47:26.931 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-30T02:47:26.931 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 81772, 2024-03-30T02:47:26.931 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 166.736822206, 2024-03-30T02:47:26.931 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002039045 2024-03-30T02:47:26.931 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.931 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-30T02:47:26.931 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 390, 2024-03-30T02:47:26.931 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.812356391, 2024-03-30T02:47:26.932 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.009775272 2024-03-30T02:47:26.932 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.932 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:26.932 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.932 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.932 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.932 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.932 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-30T02:47:26.932 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.933 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.933 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.933 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.933 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-30T02:47:26.933 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 112, 2024-03-30T02:47:26.934 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2.682520800, 2024-03-30T02:47:26.934 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.023951078 2024-03-30T02:47:26.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.934 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:26.934 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.934 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.934 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.935 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-30T02:47:26.935 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-03-30T02:47:26.935 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000328214, 2024-03-30T02:47:26.935 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000328214 2024-03-30T02:47:26.935 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.935 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-30T02:47:26.935 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.935 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.936 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.936 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:26.936 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.936 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.936 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.936 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:26.937 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.937 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.937 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.937 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.937 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-30T02:47:26.937 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10, 2024-03-30T02:47:26.937 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.004766944, 2024-03-30T02:47:26.937 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000476694 2024-03-30T02:47:26.938 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.938 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-30T02:47:26.938 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5164, 2024-03-30T02:47:26.938 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 9.679389865, 2024-03-30T02:47:26.938 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001874397 2024-03-30T02:47:26.938 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.938 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-30T02:47:26.939 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.939 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.939 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.939 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.939 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-30T02:47:26.939 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10, 2024-03-30T02:47:26.939 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 5.429509433, 2024-03-30T02:47:26.939 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.542950943 2024-03-30T02:47:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-30T02:47:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 654, 2024-03-30T02:47:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.401603068, 2024-03-30T02:47:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000614071 2024-03-30T02:47:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-30T02:47:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5713, 2024-03-30T02:47:26.940 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 19.721043302, 2024-03-30T02:47:26.941 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003451959 2024-03-30T02:47:26.941 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.941 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-30T02:47:26.941 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 30, 2024-03-30T02:47:26.941 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.048466601, 2024-03-30T02:47:26.941 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001615553 2024-03-30T02:47:26.941 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.942 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:26.942 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.942 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.942 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.942 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-30T02:47:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 110, 2024-03-30T02:47:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 4.279612758, 2024-03-30T02:47:26.943 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.038905570 2024-03-30T02:47:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4, 2024-03-30T02:47:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.054923017, 2024-03-30T02:47:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.013730754 2024-03-30T02:47:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.944 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-30T02:47:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 139, 2024-03-30T02:47:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.643694124, 2024-03-30T02:47:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.011825137 2024-03-30T02:47:26.945 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.946 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:26.946 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.946 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.946 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.946 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.946 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:26.946 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.946 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.947 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.947 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.947 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-30T02:47:26.947 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.947 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.947 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.947 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.947 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-30T02:47:26.948 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 9, 2024-03-30T02:47:26.948 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.022519576, 2024-03-30T02:47:26.948 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002502175 2024-03-30T02:47:26.948 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.948 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-30T02:47:26.948 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 62, 2024-03-30T02:47:26.948 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.062293121, 2024-03-30T02:47:26.948 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001004727 2024-03-30T02:47:26.949 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.949 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-30T02:47:26.949 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1152, 2024-03-30T02:47:26.949 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2.157409872, 2024-03-30T02:47:26.949 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001872751 2024-03-30T02:47:26.949 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.949 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:26.949 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:26.950 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:26.950 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.950 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.950 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.950 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:26.950 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.950 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-03-30T02:47:26.951 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 2, 2024-03-30T02:47:26.951 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 6, 2024-03-30T02:47:26.951 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 4, 2024-03-30T02:47:26.951 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-03-30T02:47:26.951 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-30T02:47:26.951 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 2392, 2024-03-30T02:47:26.951 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 1196, 2024-03-30T02:47:26.951 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 606, 2024-03-30T02:47:26.951 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:26.952 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.952 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-03-30T02:47:26.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:26.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:26.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:26.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:26.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:26.952 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:26.953 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:26.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:26.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:26.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:26.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:26.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:26.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:26.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:26.954 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:26.955 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:26.955 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:26.955 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:26.955 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:26.955 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:26.955 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:26.955 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:26.955 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:26.956 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:26.956 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-30T02:47:26.956 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:26.956 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:26.956 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:26.956 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:26.956 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 2850991, 2024-03-30T02:47:26.956 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 74810, 2024-03-30T02:47:26.958 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 88, 2024-03-30T02:47:26.958 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 1, 2024-03-30T02:47:26.959 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-30T02:47:26.959 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-30T02:47:26.959 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:26.959 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:26.959 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:26.960 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:26.960 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9656, 2024-03-30T02:47:26.960 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 203, 2024-03-30T02:47:26.960 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:26.960 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:26.960 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:26.961 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-30T02:47:26.961 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 336640590, 2024-03-30T02:47:26.961 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 4121523, 2024-03-30T02:47:26.961 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:26.961 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-30T02:47:26.961 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:26.961 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-30T02:47:26.961 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.962 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-03-30T02:47:26.962 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-30T02:47:26.962 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-30T02:47:26.962 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 759344, 2024-03-30T02:47:26.962 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 924876180, 2024-03-30T02:47:26.962 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 37, 2024-03-30T02:47:26.962 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 759344, 2024-03-30T02:47:26.962 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-30T02:47:26.962 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 759344, 2024-03-30T02:47:26.963 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 72253.361538027, 2024-03-30T02:47:26.963 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.095152344 2024-03-30T02:47:26.963 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.963 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-30T02:47:26.963 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-30T02:47:26.963 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 759344, 2024-03-30T02:47:26.963 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 912670 2024-03-30T02:47:26.963 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.964 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 759344, 2024-03-30T02:47:26.964 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 642464, 2024-03-30T02:47:26.964 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 116880, 2024-03-30T02:47:26.964 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-30T02:47:26.964 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-30T02:47:26.964 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 732, 2024-03-30T02:47:26.964 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 96982, 2024-03-30T02:47:26.965 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 95, 2024-03-30T02:47:26.965 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 10845, 2024-03-30T02:47:26.965 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 138, 2024-03-30T02:47:26.965 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-30T02:47:26.965 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-30T02:47:26.965 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-30T02:47:26.965 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 176, 2024-03-30T02:47:26.965 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 3260, 2024-03-30T02:47:26.966 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-30T02:47:26.966 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:26.966 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:26.966 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 642314, 2024-03-30T02:47:26.966 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 147001, 2024-03-30T02:47:26.966 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:26.966 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:26.966 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:26.967 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-30T02:47:26.967 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-30T02:47:26.967 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-30T02:47:26.967 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:26.967 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-30T02:47:26.967 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:26.967 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 70, 2024-03-30T02:47:26.967 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-30T02:47:26.967 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-30T02:47:26.968 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-30T02:47:26.968 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-30T02:47:26.968 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-30T02:47:26.968 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-30T02:47:26.968 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-30T02:47:26.968 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-30T02:47:26.968 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-30T02:47:26.969 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-30T02:47:26.969 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-30T02:47:26.969 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-30T02:47:26.969 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-30T02:47:26.969 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-30T02:47:26.969 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-30T02:47:26.969 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-30T02:47:26.970 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 96, 2024-03-30T02:47:26.970 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-30T02:47:26.970 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 18, 2024-03-30T02:47:26.970 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 12, 2024-03-30T02:47:26.970 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 12, 2024-03-30T02:47:26.970 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-30T02:47:26.970 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-30T02:47:26.970 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 10676, 2024-03-30T02:47:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 110, 2024-03-30T02:47:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 271 2024-03-30T02:47:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-03-30T02:47:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-30T02:47:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-30T02:47:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-30T02:47:26.971 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.972 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-03-30T02:47:26.972 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 100751 2024-03-30T02:47:26.972 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.972 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-03-30T02:47:26.972 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:26.972 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 10, 2024-03-30T02:47:26.973 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-30T02:47:26.973 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 9, 2024-03-30T02:47:26.973 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 1262, 2024-03-30T02:47:26.973 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:26.973 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.973 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:26.973 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:26.973 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-30T02:47:26.973 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:26.974 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 1003561, 2024-03-30T02:47:26.974 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 382983939, 2024-03-30T02:47:26.974 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:26.974 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 1003561, 2024-03-30T02:47:26.974 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:26.974 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:26.974 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1003561, 2024-03-30T02:47:26.975 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 382983939, 2024-03-30T02:47:26.975 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:26.975 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.975 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.975 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.975 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:26.975 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.975 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:26.976 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:26.976 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-30T02:47:26.976 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:26.976 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:26.976 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:26.976 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:26.976 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:26.976 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 759344, 2024-03-30T02:47:26.977 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 925012122, 2024-03-30T02:47:26.977 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 759344, 2024-03-30T02:47:26.977 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 925012122, 2024-03-30T02:47:26.977 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:26.977 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.977 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.977 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.977 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:26.978 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.978 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:26.978 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:26.978 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-30T02:47:26.978 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:26.978 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:26.978 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:26.979 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:26.979 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:26.979 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 759344, 2024-03-30T02:47:26.982 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 759344, 2024-03-30T02:47:26.982 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 759344, 2024-03-30T02:47:26.982 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 759344, 2024-03-30T02:47:26.982 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:26.982 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.983 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.983 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.983 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:26.983 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.983 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:26.983 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:26.983 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-30T02:47:26.983 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 1262, 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 136822, 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 1262, 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 81, 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 136822, 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:26.984 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.985 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.985 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.985 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:26.985 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:26.985 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x562b0781a0a0": { 2024-03-30T02:47:26.985 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:26.985 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-30T02:47:26.985 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:26.986 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 211876, 2024-03-30T02:47:26.986 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 796562610, 2024-03-30T02:47:26.986 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:26.986 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 211876, 2024-03-30T02:47:26.986 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:26.986 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:26.987 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 10764, 2024-03-30T02:47:26.987 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 796562610, 2024-03-30T02:47:26.987 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:26.987 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:26.987 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:26.987 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:26.987 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:26.987 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:26.988 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-30T02:47:27.228 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:27 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:27.229 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:27 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:27.229 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:27 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.229 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:27 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.230 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:27 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.230 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:27 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.230 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:27 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:47:27.230 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:27 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:47:27.230 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:27 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:27.267 INFO:tasks.check_counter:Daemon mds.b mds.exported=591 2024-03-30T02:47:27.267 INFO:tasks.check_counter:Daemon mds.b mds.imported=3 2024-03-30T02:47:27.267 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-30T02:47:27.267 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-03-30T02:47:27.268 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-30T02:47:27.268 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=217 2024-03-30T02:47:27.268 DEBUG:tasks.check_counter:Getting stats from e 2024-03-30T02:47:27.268 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.e perf dump 2024-03-30T02:47:27.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:27 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:27.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:27 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:27.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:27 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:27 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:27 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:27 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:27 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:47:27.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:27 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:47:27.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:27 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:27.440 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:27 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:27.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:27 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:27.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:27 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:27 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:27 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:27 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-30T02:47:27.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:27 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-30T02:47:27.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:27 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-30T02:47:27.441 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:27 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' 2024-03-30T02:47:27.504 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:47:28.481 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-30T02:47:28.482 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:28.482 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1867, 2024-03-30T02:47:28.482 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1869, 2024-03-30T02:47:28.482 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 12764784, 2024-03-30T02:47:28.482 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 580644, 2024-03-30T02:47:28.482 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-03-30T02:47:28.483 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-03-30T02:47:28.483 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.142950205, 2024-03-30T02:47:28.483 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.041729469, 2024-03-30T02:47:28.483 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.043550442, 2024-03-30T02:47:28.483 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.045580915, 2024-03-30T02:47:28.483 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:28.483 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1867, 2024-03-30T02:47:28.483 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.152636373, 2024-03-30T02:47:28.483 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000081754 2024-03-30T02:47:28.484 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.484 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:28.484 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.484 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.484 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.484 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.484 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T02:47:28.484 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T02:47:28.484 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.485 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:28.485 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 325, 2024-03-30T02:47:28.485 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 482, 2024-03-30T02:47:28.485 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 81850, 2024-03-30T02:47:28.485 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 525511, 2024-03-30T02:47:28.485 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-03-30T02:47:28.485 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-03-30T02:47:28.485 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.034428061, 2024-03-30T02:47:28.485 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.011411330, 2024-03-30T02:47:28.486 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.008230384, 2024-03-30T02:47:28.486 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.010750788, 2024-03-30T02:47:28.486 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:28.486 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 480, 2024-03-30T02:47:28.486 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.053914346, 2024-03-30T02:47:28.486 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000112321 2024-03-30T02:47:28.486 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.486 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 426240 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 255, 2024-03-30T02:47:28.487 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 229, 2024-03-30T02:47:28.488 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1935420, 2024-03-30T02:47:28.488 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 123175, 2024-03-30T02:47:28.488 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-03-30T02:47:28.488 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-03-30T02:47:28.488 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.045388509, 2024-03-30T02:47:28.488 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.011910630, 2024-03-30T02:47:28.488 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.040829853, 2024-03-30T02:47:28.488 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.005039286, 2024-03-30T02:47:28.488 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:28.489 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 228, 2024-03-30T02:47:28.489 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.016672108, 2024-03-30T02:47:28.489 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000073123 2024-03-30T02:47:28.489 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.489 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:28.489 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.489 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.489 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.489 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.490 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1935152, 2024-03-30T02:47:28.490 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 122864 2024-03-30T02:47:28.490 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.490 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-03-30T02:47:28.490 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-30T02:47:28.490 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-30T02:47:28.490 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.490 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-03-30T02:47:28.490 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-30T02:47:28.491 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-30T02:47:28.491 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3685, 2024-03-30T02:47:28.491 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.089133187, 2024-03-30T02:47:28.491 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000024188 2024-03-30T02:47:28.491 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:28.491 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.491 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:28.491 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-30T02:47:28.491 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-30T02:47:28.492 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.492 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.492 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.492 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:28.492 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.492 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-03-30T02:47:28.492 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2024-03-30T02:47:28.492 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2024-03-30T02:47:28.492 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-30T02:47:28.493 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.493 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.493 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.493 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.493 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-30T02:47:28.493 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-30T02:47:28.493 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-30T02:47:28.493 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:28.494 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-03-30T02:47:28.494 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-30T02:47:28.494 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-30T02:47:28.494 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 3591, 2024-03-30T02:47:28.494 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-30T02:47:28.494 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 3591, 2024-03-30T02:47:28.494 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-30T02:47:28.494 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 3591, 2024-03-30T02:47:28.494 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-30T02:47:28.495 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-03-30T02:47:28.495 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-03-30T02:47:28.495 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 113, 2024-03-30T02:47:28.495 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2024-03-30T02:47:28.495 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2024-03-30T02:47:28.495 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-30T02:47:28.495 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-30T02:47:28.495 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:28.495 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:28.496 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-30T02:47:28.496 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-30T02:47:28.496 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-30T02:47:28.496 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-30T02:47:28.496 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-30T02:47:28.496 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-30T02:47:28.496 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-30T02:47:28.496 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:28.496 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-30T02:47:28.497 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:28.497 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-30T02:47:28.497 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-30T02:47:28.497 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-30T02:47:28.497 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-30T02:47:28.497 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:28.497 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:28.497 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:28.498 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-30T02:47:28.498 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T02:47:28.498 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T02:47:28.498 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-30T02:47:28.498 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-30T02:47:28.498 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-30T02:47:28.498 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-30T02:47:28.498 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T02:47:28.498 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:28.499 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:28.499 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-30T02:47:28.499 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-30T02:47:28.499 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-30T02:47:28.499 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-03-30T02:47:28.499 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.499 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-03-30T02:47:28.499 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-30T02:47:28.499 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:28.500 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-30T02:47:28.500 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-30T02:47:28.500 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-30T02:47:28.500 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-30T02:47:28.500 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:28.500 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:28.500 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-30T02:47:28.500 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-30T02:47:28.500 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:28.501 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-30T02:47:28.501 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:28.501 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:28.501 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:28.501 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-30T02:47:28.501 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-30T02:47:28.501 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T02:47:28.501 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-30T02:47:28.502 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-30T02:47:28.502 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-30T02:47:28.502 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:28.502 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-30T02:47:28.502 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.502 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-03-30T02:47:28.502 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 0, 2024-03-30T02:47:28.502 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-30T02:47:28.502 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-30T02:47:28.503 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 4488, 2024-03-30T02:47:28.503 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-30T02:47:28.503 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-30T02:47:28.503 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-03-30T02:47:28.503 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-30T02:47:28.503 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-30T02:47:28.503 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 6, 2024-03-30T02:47:28.503 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-30T02:47:28.503 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-30T02:47:28.504 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-30T02:47:28.504 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 16422237, 2024-03-30T02:47:28.504 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 16421663, 2024-03-30T02:47:28.504 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-30T02:47:28.504 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.504 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.504 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.504 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.504 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 4488 2024-03-30T02:47:28.505 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.505 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-03-30T02:47:28.505 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 3516, 2024-03-30T02:47:28.505 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 3577, 2024-03-30T02:47:28.505 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 61, 2024-03-30T02:47:28.505 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 337, 2024-03-30T02:47:28.505 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 357, 2024-03-30T02:47:28.505 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 20, 2024-03-30T02:47:28.505 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 3591, 2024-03-30T02:47:28.506 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 3613, 2024-03-30T02:47:28.506 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 22, 2024-03-30T02:47:28.506 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-03-30T02:47:28.506 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2024-03-30T02:47:28.506 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-30T02:47:28.506 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 64728, 2024-03-30T02:47:28.506 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 207156 2024-03-30T02:47:28.506 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 0, 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.507 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.508 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.509 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.509 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-30T02:47:28.509 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.509 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.509 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.509 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.509 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-30T02:47:28.509 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.510 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.510 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.510 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:28.510 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.510 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.510 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.510 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.510 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-30T02:47:28.511 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.511 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.511 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.511 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.511 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-30T02:47:28.511 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.511 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.511 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.511 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.512 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:28.512 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.512 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.512 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.512 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.512 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:28.512 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.512 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-30T02:47:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-30T02:47:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.514 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.514 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.514 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.514 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-30T02:47:28.514 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.514 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.514 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.514 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.515 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-30T02:47:28.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.515 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.515 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-30T02:47:28.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.515 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.515 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.516 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-30T02:47:28.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.516 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.516 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.516 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-30T02:47:28.516 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.516 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.517 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-30T02:47:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.519 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.519 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.519 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.519 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:28.519 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.519 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.519 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.519 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.520 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-30T02:47:28.520 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.520 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.520 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.520 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.520 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:28.520 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.520 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.520 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.521 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.521 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:28.521 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.521 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.521 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.521 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.521 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-30T02:47:28.521 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.521 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.522 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.522 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-30T02:47:28.522 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.522 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.522 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.522 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-30T02:47:28.523 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.523 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.523 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.523 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-30T02:47:28.523 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.523 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.523 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.523 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.524 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:28.524 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:28.524 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:28.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.524 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.524 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.524 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:28.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.525 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-03-30T02:47:28.525 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-30T02:47:28.525 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2024-03-30T02:47:28.525 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2024-03-30T02:47:28.525 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-03-30T02:47:28.525 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-30T02:47:28.525 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2024-03-30T02:47:28.525 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 2552, 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:28.526 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:28.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:28.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:28.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:28.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:28.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:28.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:28.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:28.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:28.527 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:28.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:28.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:28.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:28.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:28.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:28.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:28.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:28.528 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-30T02:47:28.529 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:28.530 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:28.530 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:28.530 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:28.530 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 314804, 2024-03-30T02:47:28.530 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 3719, 2024-03-30T02:47:28.530 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-30T02:47:28.530 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-30T02:47:28.530 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-30T02:47:28.530 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-30T02:47:28.531 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:28.531 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:28.531 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:28.531 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:28.531 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9656, 2024-03-30T02:47:28.531 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 203, 2024-03-30T02:47:28.531 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:28.531 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:28.531 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:28.532 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-30T02:47:28.532 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 11086495, 2024-03-30T02:47:28.532 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 116317, 2024-03-30T02:47:28.532 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:28.532 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-30T02:47:28.532 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:28.532 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-30T02:47:28.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.532 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-03-30T02:47:28.533 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-30T02:47:28.533 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-30T02:47:28.533 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 2187, 2024-03-30T02:47:28.533 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 21, 2024-03-30T02:47:28.533 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 3, 2024-03-30T02:47:28.533 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 2187, 2024-03-30T02:47:28.533 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-30T02:47:28.533 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2187, 2024-03-30T02:47:28.534 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 23.888527312, 2024-03-30T02:47:28.534 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.010922966 2024-03-30T02:47:28.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.534 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-30T02:47:28.534 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-30T02:47:28.534 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2187, 2024-03-30T02:47:28.534 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2188 2024-03-30T02:47:28.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.534 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 2187, 2024-03-30T02:47:28.535 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 2187, 2024-03-30T02:47:28.535 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 0, 2024-03-30T02:47:28.535 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-30T02:47:28.535 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-30T02:47:28.535 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 1351, 2024-03-30T02:47:28.535 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-03-30T02:47:28.535 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 834, 2024-03-30T02:47:28.535 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 0, 2024-03-30T02:47:28.535 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 0, 2024-03-30T02:47:28.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-30T02:47:28.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-30T02:47:28.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-30T02:47:28.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-30T02:47:28.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 0, 2024-03-30T02:47:28.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-30T02:47:28.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:28.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:28.536 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1, 2024-03-30T02:47:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-03-30T02:47:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-30T02:47:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-30T02:47:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-30T02:47:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:28.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-30T02:47:28.538 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:28.538 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-30T02:47:28.538 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-30T02:47:28.538 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-30T02:47:28.538 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-30T02:47:28.538 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-30T02:47:28.538 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-30T02:47:28.538 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-30T02:47:28.539 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-30T02:47:28.539 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-30T02:47:28.539 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-30T02:47:28.539 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-30T02:47:28.539 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-30T02:47:28.539 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-30T02:47:28.539 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-30T02:47:28.539 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-30T02:47:28.539 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-30T02:47:28.540 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-30T02:47:28.540 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 96, 2024-03-30T02:47:28.540 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-30T02:47:28.540 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 13, 2024-03-30T02:47:28.540 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 5, 2024-03-30T02:47:28.540 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 5, 2024-03-30T02:47:28.540 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-30T02:47:28.540 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-30T02:47:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 0, 2024-03-30T02:47:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 2, 2024-03-30T02:47:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-03-30T02:47:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-03-30T02:47:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-03-30T02:47:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-30T02:47:28.541 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-30T02:47:28.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.542 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-03-30T02:47:28.542 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-30T02:47:28.542 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.542 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-03-30T02:47:28.542 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:28.542 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-30T02:47:28.542 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-30T02:47:28.543 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-30T02:47:28.543 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-30T02:47:28.543 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:28.543 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.543 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:28.543 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:28.543 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-30T02:47:28.543 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:28.543 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2443, 2024-03-30T02:47:28.544 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 14546294, 2024-03-30T02:47:28.544 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:28.544 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2443, 2024-03-30T02:47:28.544 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:28.544 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:28.544 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2443, 2024-03-30T02:47:28.544 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 14546294, 2024-03-30T02:47:28.544 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:28.544 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.545 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.545 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.545 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:28.545 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.545 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:28.545 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:28.545 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-30T02:47:28.545 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:28.545 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:28.546 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:28.546 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:28.546 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:28.546 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2187, 2024-03-30T02:47:28.546 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 12229426, 2024-03-30T02:47:28.546 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2187, 2024-03-30T02:47:28.546 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 12229426, 2024-03-30T02:47:28.546 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:28.547 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.547 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.547 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.547 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:28.547 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.547 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:28.547 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:28.547 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-30T02:47:28.547 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:28.548 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:28.548 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:28.548 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:28.548 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:28.548 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2187, 2024-03-30T02:47:28.548 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 2187, 2024-03-30T02:47:28.548 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2187, 2024-03-30T02:47:28.548 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2187, 2024-03-30T02:47:28.549 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:28.549 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.549 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.549 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.549 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:28.549 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.549 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:28.549 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:28.549 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-30T02:47:28.550 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:28.550 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:28.550 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:28.550 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:28.550 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:28.550 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:28.550 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:28.550 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-30T02:47:28.550 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-30T02:47:28.551 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:28.551 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.551 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.551 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.551 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:28.551 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:28.551 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55a0995405a0": { 2024-03-30T02:47:28.551 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:28.551 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-30T02:47:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-30T02:47:28.552 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-30T02:47:28.553 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:28.553 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:28.553 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:28.553 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:28.553 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:28.553 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:28.553 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-30T02:47:28.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:28 smithi082 ceph-mon[79176]: pgmap v323: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-03-30T02:47:28.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:28 smithi082 ceph-mon[79176]: Adjusting osd_memory_target on smithi067 to 221.5M 2024-03-30T02:47:28.649 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:28 smithi082 ceph-mon[79176]: Unable to set osd_memory_target on smithi067 to 232361472: error parsing value: Value '232361472' is below minimum 939524096 2024-03-30T02:47:28.845 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-30T02:47:28.845 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-30T02:47:28.845 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-30T02:47:28.845 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-30T02:47:28.845 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-30T02:47:28.845 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-03-30T02:47:28.845 DEBUG:tasks.check_counter:Getting stats from h 2024-03-30T02:47:28.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.h perf dump 2024-03-30T02:47:28.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:28 smithi067 ceph-mon[37808]: pgmap v323: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-03-30T02:47:28.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:28 smithi067 ceph-mon[37808]: Adjusting osd_memory_target on smithi067 to 221.5M 2024-03-30T02:47:28.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:28 smithi067 ceph-mon[37808]: Unable to set osd_memory_target on smithi067 to 232361472: error parsing value: Value '232361472' is below minimum 939524096 2024-03-30T02:47:28.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:28 smithi170 ceph-mon[78798]: pgmap v323: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-03-30T02:47:28.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:28 smithi170 ceph-mon[78798]: Adjusting osd_memory_target on smithi067 to 221.5M 2024-03-30T02:47:28.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:28 smithi170 ceph-mon[78798]: Unable to set osd_memory_target on smithi067 to 232361472: error parsing value: Value '232361472' is below minimum 939524096 2024-03-30T02:47:29.066 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:47:30.089 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-30T02:47:30.089 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:30.089 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 10618, 2024-03-30T02:47:30.090 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 12835, 2024-03-30T02:47:30.090 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 3346716, 2024-03-30T02:47:30.090 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 12849730, 2024-03-30T02:47:30.090 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 23, 2024-03-30T02:47:30.090 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-30T02:47:30.090 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.571684556, 2024-03-30T02:47:30.090 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.203612646, 2024-03-30T02:47:30.090 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.377812284, 2024-03-30T02:47:30.091 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.110000254, 2024-03-30T02:47:30.091 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:30.091 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 12834, 2024-03-30T02:47:30.091 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.842043299, 2024-03-30T02:47:30.091 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000299364 2024-03-30T02:47:30.091 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.091 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:30.091 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 9828, 2024-03-30T02:47:30.092 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.004070106, 2024-03-30T02:47:30.092 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000414 2024-03-30T02:47:30.092 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.092 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-30T02:47:30.092 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2024-03-30T02:47:30.092 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.092 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:30.092 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 9204, 2024-03-30T02:47:30.092 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 9327, 2024-03-30T02:47:30.093 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 2366803, 2024-03-30T02:47:30.093 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 6897174, 2024-03-30T02:47:30.093 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 19, 2024-03-30T02:47:30.093 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 7, 2024-03-30T02:47:30.093 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.507504447, 2024-03-30T02:47:30.093 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.167915783, 2024-03-30T02:47:30.093 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.122238467, 2024-03-30T02:47:30.093 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.180145673, 2024-03-30T02:47:30.094 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:30.094 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 9324, 2024-03-30T02:47:30.094 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.417760829, 2024-03-30T02:47:30.094 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000044804 2024-03-30T02:47:30.094 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.094 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:30.094 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 189, 2024-03-30T02:47:30.094 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000027710, 2024-03-30T02:47:30.094 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000146 2024-03-30T02:47:30.095 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.095 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 4784, 2024-03-30T02:47:30.095 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 434224 2024-03-30T02:47:30.095 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.095 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:30.095 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 17615, 2024-03-30T02:47:30.095 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 12548, 2024-03-30T02:47:30.095 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 9298621, 2024-03-30T02:47:30.095 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 4894128, 2024-03-30T02:47:30.096 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 23, 2024-03-30T02:47:30.096 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 6, 2024-03-30T02:47:30.096 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.890368722, 2024-03-30T02:47:30.096 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.331671973, 2024-03-30T02:47:30.096 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.574712831, 2024-03-30T02:47:30.096 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.122705977, 2024-03-30T02:47:30.096 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:30.096 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 12547, 2024-03-30T02:47:30.097 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.537230766, 2024-03-30T02:47:30.097 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000042817 2024-03-30T02:47:30.097 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.097 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:30.097 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 17630, 2024-03-30T02:47:30.098 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.002568166, 2024-03-30T02:47:30.098 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000145 2024-03-30T02:47:30.098 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.098 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1976448, 2024-03-30T02:47:30.098 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 429744 2024-03-30T02:47:30.098 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.098 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-03-30T02:47:30.099 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-30T02:47:30.099 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-30T02:47:30.099 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.099 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-03-30T02:47:30.099 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-30T02:47:30.099 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-30T02:47:30.099 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 14968, 2024-03-30T02:47:30.099 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.405909892, 2024-03-30T02:47:30.100 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000227546 2024-03-30T02:47:30.100 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:30.100 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.100 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:30.100 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-30T02:47:30.100 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-30T02:47:30.100 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 173, 2024-03-30T02:47:30.100 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.010573657, 2024-03-30T02:47:30.100 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000061119 2024-03-30T02:47:30.101 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:30.101 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.101 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-03-30T02:47:30.101 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 836, 2024-03-30T02:47:30.101 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 798, 2024-03-30T02:47:30.101 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-30T02:47:30.102 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 798, 2024-03-30T02:47:30.102 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.797297400, 2024-03-30T02:47:30.102 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000999119 2024-03-30T02:47:30.102 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.102 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-30T02:47:30.102 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 38, 2024-03-30T02:47:30.102 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-30T02:47:30.102 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:30.103 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 65, 2024-03-30T02:47:30.103 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 6, 2024-03-30T02:47:30.103 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-30T02:47:30.103 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 3816, 2024-03-30T02:47:30.103 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 322, 2024-03-30T02:47:30.103 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 3494, 2024-03-30T02:47:30.103 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-30T02:47:30.103 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 3356, 2024-03-30T02:47:30.104 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 1, 2024-03-30T02:47:30.104 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 2025, 2024-03-30T02:47:30.104 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 2025, 2024-03-30T02:47:30.104 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 155, 2024-03-30T02:47:30.104 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 1445, 2024-03-30T02:47:30.104 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 754, 2024-03-30T02:47:30.104 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 32, 2024-03-30T02:47:30.104 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 149, 2024-03-30T02:47:30.105 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:30.105 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:30.105 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-30T02:47:30.105 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-30T02:47:30.105 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-30T02:47:30.105 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-30T02:47:30.105 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-30T02:47:30.105 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 113, 2024-03-30T02:47:30.106 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 2029, 2024-03-30T02:47:30.106 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:30.106 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 19109, 2024-03-30T02:47:30.106 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:30.106 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 85384, 2024-03-30T02:47:30.106 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 1566089959, 2024-03-30T02:47:30.106 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 6, 2024-03-30T02:47:30.106 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 19109, 2024-03-30T02:47:30.107 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:30.107 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:30.107 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:30.107 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 366, 2024-03-30T02:47:30.107 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T02:47:30.107 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 366, 2024-03-30T02:47:30.107 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 18743, 2024-03-30T02:47:30.107 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 5, 2024-03-30T02:47:30.108 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 129, 2024-03-30T02:47:30.108 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 419, 2024-03-30T02:47:30.108 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 3, 2024-03-30T02:47:30.108 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:30.109 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:30.109 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 3830, 2024-03-30T02:47:30.109 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 3534, 2024-03-30T02:47:30.109 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 78, 2024-03-30T02:47:30.110 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 360 2024-03-30T02:47:30.110 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.110 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-03-30T02:47:30.110 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-30T02:47:30.111 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:30.111 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-30T02:47:30.111 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 121, 2024-03-30T02:47:30.111 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 10, 2024-03-30T02:47:30.112 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-30T02:47:30.112 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:30.113 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:30.113 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 59, 2024-03-30T02:47:30.113 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 59, 2024-03-30T02:47:30.113 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:30.114 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-30T02:47:30.114 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:30.114 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:30.115 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:30.115 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-30T02:47:30.115 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-30T02:47:30.116 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T02:47:30.116 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-30T02:47:30.116 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-30T02:47:30.117 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 6, 2024-03-30T02:47:30.117 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:30.117 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-30T02:47:30.117 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-03-30T02:47:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 4995, 2024-03-30T02:47:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-30T02:47:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-30T02:47:30.119 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 4995, 2024-03-30T02:47:30.119 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-30T02:47:30.119 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-30T02:47:30.120 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 6, 2024-03-30T02:47:30.120 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-30T02:47:30.120 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-30T02:47:30.121 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 6, 2024-03-30T02:47:30.121 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-30T02:47:30.121 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-30T02:47:30.121 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-30T02:47:30.122 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 15650191, 2024-03-30T02:47:30.122 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-03-30T02:47:30.122 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-30T02:47:30.122 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 333, 2024-03-30T02:47:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 8.786723982, 2024-03-30T02:47:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.026386558 2024-03-30T02:47:30.123 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-03-30T02:47:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-03-30T02:47:30.124 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 3709, 2024-03-30T02:47:30.125 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 3768, 2024-03-30T02:47:30.125 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 59, 2024-03-30T02:47:30.126 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 475, 2024-03-30T02:47:30.126 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 498, 2024-03-30T02:47:30.126 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 23, 2024-03-30T02:47:30.126 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 3816, 2024-03-30T02:47:30.127 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 3878, 2024-03-30T02:47:30.127 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 62, 2024-03-30T02:47:30.127 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 2025, 2024-03-30T02:47:30.128 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 3791, 2024-03-30T02:47:30.128 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 1766, 2024-03-30T02:47:30.128 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 74656, 2024-03-30T02:47:30.128 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 198964 2024-03-30T02:47:30.129 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.129 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-03-30T02:47:30.129 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 1060, 2024-03-30T02:47:30.129 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 4, 2024-03-30T02:47:30.130 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 836, 2024-03-30T02:47:30.130 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 16, 2024-03-30T02:47:30.130 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 70, 2024-03-30T02:47:30.131 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-30T02:47:30.131 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 187, 2024-03-30T02:47:30.131 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.287180661, 2024-03-30T02:47:30.132 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001535725 2024-03-30T02:47:30.132 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.132 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-30T02:47:30.132 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6, 2024-03-30T02:47:30.133 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.061959968, 2024-03-30T02:47:30.133 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.010326661 2024-03-30T02:47:30.133 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.133 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:30.134 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.134 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.134 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.134 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.135 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-30T02:47:30.135 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.135 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.135 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.135 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.136 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-30T02:47:30.136 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 288, 2024-03-30T02:47:30.136 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.104107392, 2024-03-30T02:47:30.136 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000361484 2024-03-30T02:47:30.136 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.136 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:30.136 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.137 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.137 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.137 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.137 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-30T02:47:30.137 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.137 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.138 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.138 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.138 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-30T02:47:30.138 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.138 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.138 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.139 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.139 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:30.139 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.139 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.139 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.139 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.139 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:30.139 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.139 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.140 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.140 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.140 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-30T02:47:30.140 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.140 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.140 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.140 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.140 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-30T02:47:30.140 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-30T02:47:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-30T02:47:30.142 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.142 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.142 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.142 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.142 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-30T02:47:30.142 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 63, 2024-03-30T02:47:30.142 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.026864899, 2024-03-30T02:47:30.142 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000426426 2024-03-30T02:47:30.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.143 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-30T02:47:30.143 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 192, 2024-03-30T02:47:30.143 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.210358342, 2024-03-30T02:47:30.143 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001095616 2024-03-30T02:47:30.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.143 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-30T02:47:30.144 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.144 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.144 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.144 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.144 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:30.144 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.144 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.145 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.145 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.145 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:30.145 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.145 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.145 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.146 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.146 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-30T02:47:30.146 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.146 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.146 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.146 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.146 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:30.146 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.147 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.147 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.147 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.147 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:30.147 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.147 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.147 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.148 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.148 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-30T02:47:30.148 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2024-03-30T02:47:30.148 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.020581234, 2024-03-30T02:47:30.148 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.006860411 2024-03-30T02:47:30.148 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.148 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:30.148 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.148 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.149 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.149 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.149 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:30.149 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.149 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.150 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.150 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.150 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-30T02:47:30.151 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.151 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.151 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.151 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.151 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-30T02:47:30.151 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.152 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.152 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.152 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.152 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-30T02:47:30.153 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.153 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.153 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.154 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.154 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-30T02:47:30.154 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 59, 2024-03-30T02:47:30.154 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.086244904, 2024-03-30T02:47:30.154 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001461778 2024-03-30T02:47:30.155 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.155 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:30.155 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:30.155 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:30.155 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.155 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.155 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.155 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:30.155 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.156 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-03-30T02:47:30.156 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-30T02:47:30.156 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-03-30T02:47:30.156 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 2, 2024-03-30T02:47:30.156 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-03-30T02:47:30.156 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-30T02:47:30.156 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 271, 2024-03-30T02:47:30.156 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 271, 2024-03-30T02:47:30.156 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 647, 2024-03-30T02:47:30.157 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:30.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.157 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-03-30T02:47:30.157 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:30.157 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:30.157 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:30.157 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:30.158 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:30.158 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:30.158 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:30.158 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:30.158 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:30.158 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:30.158 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:30.159 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:30.159 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:30.159 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:30.159 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:30.159 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:30.159 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:30.159 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:30.159 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:30.160 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:30.160 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:30.160 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:30.160 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:30.160 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:30.160 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:30.160 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:30.161 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:30.161 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:30.161 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:30.161 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:30.161 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:30.161 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-30T02:47:30.161 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:30.162 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:30.162 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:30.162 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:30.162 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 382969, 2024-03-30T02:47:30.162 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 2575, 2024-03-30T02:47:30.163 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 1056, 2024-03-30T02:47:30.163 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 12, 2024-03-30T02:47:30.163 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-30T02:47:30.163 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-30T02:47:30.163 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:30.164 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:30.164 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:30.164 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:30.164 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9656, 2024-03-30T02:47:30.165 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 203, 2024-03-30T02:47:30.165 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:30.165 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:30.165 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:30.165 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-30T02:47:30.166 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 12520170, 2024-03-30T02:47:30.166 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 141972, 2024-03-30T02:47:30.166 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:30.166 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-30T02:47:30.166 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:30.167 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-30T02:47:30.167 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.167 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-03-30T02:47:30.167 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-30T02:47:30.167 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-30T02:47:30.167 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 19815, 2024-03-30T02:47:30.167 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 12982457, 2024-03-30T02:47:30.167 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 24, 2024-03-30T02:47:30.168 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 19815, 2024-03-30T02:47:30.168 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-30T02:47:30.168 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 19815, 2024-03-30T02:47:30.168 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 28.496443032, 2024-03-30T02:47:30.168 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001438124 2024-03-30T02:47:30.168 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.168 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-30T02:47:30.169 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-30T02:47:30.169 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 19815, 2024-03-30T02:47:30.169 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 20005 2024-03-30T02:47:30.169 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.169 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 19815, 2024-03-30T02:47:30.169 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 19128, 2024-03-30T02:47:30.169 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 687, 2024-03-30T02:47:30.170 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-30T02:47:30.170 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-30T02:47:30.170 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-03-30T02:47:30.170 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 10, 2024-03-30T02:47:30.170 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 19, 2024-03-30T02:47:30.170 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 352, 2024-03-30T02:47:30.170 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 44, 2024-03-30T02:47:30.170 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-30T02:47:30.171 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-30T02:47:30.171 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-30T02:47:30.171 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 6, 2024-03-30T02:47:30.171 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 132, 2024-03-30T02:47:30.171 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-30T02:47:30.171 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:30.171 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:30.171 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 19109, 2024-03-30T02:47:30.172 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 20, 2024-03-30T02:47:30.172 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:30.172 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:30.172 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:30.172 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-30T02:47:30.172 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-30T02:47:30.172 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-30T02:47:30.172 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:30.172 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-30T02:47:30.173 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:30.173 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 3, 2024-03-30T02:47:30.173 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-30T02:47:30.173 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-30T02:47:30.173 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-30T02:47:30.173 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-30T02:47:30.173 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-30T02:47:30.173 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-30T02:47:30.174 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-30T02:47:30.174 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-30T02:47:30.174 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-30T02:47:30.174 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-30T02:47:30.174 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-30T02:47:30.174 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-30T02:47:30.174 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-30T02:47:30.174 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-30T02:47:30.175 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-30T02:47:30.175 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-30T02:47:30.175 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 96, 2024-03-30T02:47:30.175 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-30T02:47:30.175 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 13, 2024-03-30T02:47:30.175 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 12, 2024-03-30T02:47:30.176 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 12, 2024-03-30T02:47:30.176 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-30T02:47:30.176 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-30T02:47:30.176 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 254, 2024-03-30T02:47:30.176 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-03-30T02:47:30.176 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 56 2024-03-30T02:47:30.176 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.176 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-03-30T02:47:30.176 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-03-30T02:47:30.177 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:30.177 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-30T02:47:30.177 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-30T02:47:30.177 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.177 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-03-30T02:47:30.177 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 890 2024-03-30T02:47:30.177 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.177 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-03-30T02:47:30.178 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:30.178 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 10, 2024-03-30T02:47:30.178 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-30T02:47:30.178 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 5, 2024-03-30T02:47:30.178 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 59, 2024-03-30T02:47:30.178 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:30.178 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.178 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:30.179 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:30.179 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-30T02:47:30.179 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:30.179 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 37433, 2024-03-30T02:47:30.179 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 11618764, 2024-03-30T02:47:30.179 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:30.180 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 37433, 2024-03-30T02:47:30.180 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:30.180 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:30.180 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 37433, 2024-03-30T02:47:30.180 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 11618764, 2024-03-30T02:47:30.180 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:30.180 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.180 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.180 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.181 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:30.181 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.181 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:30.181 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:30.181 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-30T02:47:30.181 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:30.181 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:30.182 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:30.182 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:30.182 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:30.182 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 19815, 2024-03-30T02:47:30.182 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 12988888, 2024-03-30T02:47:30.182 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 19815, 2024-03-30T02:47:30.182 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 12988888, 2024-03-30T02:47:30.182 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:30.183 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.183 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.183 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.183 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:30.183 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.183 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:30.183 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:30.183 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-30T02:47:30.184 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:30.184 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:30.184 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:30.184 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:30.184 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:30.184 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 19815, 2024-03-30T02:47:30.185 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 19815, 2024-03-30T02:47:30.185 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 19815, 2024-03-30T02:47:30.185 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 19815, 2024-03-30T02:47:30.185 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:30.185 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.185 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.185 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.186 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:30.186 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.186 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:30.187 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:30.187 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-30T02:47:30.187 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:30.187 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 59, 2024-03-30T02:47:30.187 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 6431, 2024-03-30T02:47:30.188 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:30.188 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 59, 2024-03-30T02:47:30.188 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:30.188 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:30.189 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 19, 2024-03-30T02:47:30.189 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 6431, 2024-03-30T02:47:30.189 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:30.189 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.190 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.190 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.190 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:30.191 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:30.191 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55ca025e80a0": { 2024-03-30T02:47:30.191 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:30.191 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-30T02:47:30.191 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:30.192 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 4995, 2024-03-30T02:47:30.192 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 11455887, 2024-03-30T02:47:30.192 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:30.193 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 4995, 2024-03-30T02:47:30.193 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:30.193 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:30.193 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 333, 2024-03-30T02:47:30.194 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 11455887, 2024-03-30T02:47:30.194 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:30.194 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:30.194 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:30.195 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:30.195 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:30.195 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:30.196 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-30T02:47:30.453 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-30T02:47:30.453 INFO:tasks.check_counter:Daemon mds.h mds.imported=113 2024-03-30T02:47:30.453 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-30T02:47:30.454 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-30T02:47:30.454 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-30T02:47:30.454 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=6 2024-03-30T02:47:30.454 DEBUG:tasks.check_counter:Getting stats from k 2024-03-30T02:47:30.454 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.k perf dump 2024-03-30T02:47:30.499 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:30 smithi082 ceph-mon[79176]: pgmap v324: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-03-30T02:47:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.b/config 2024-03-30T02:47:30.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:30 smithi067 ceph-mon[37808]: pgmap v324: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-03-30T02:47:30.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:30 smithi170 ceph-mon[78798]: pgmap v324: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-03-30T02:47:31.698 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-30T02:47:31.698 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:31.698 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 575, 2024-03-30T02:47:31.698 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 577, 2024-03-30T02:47:31.698 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 16926281, 2024-03-30T02:47:31.698 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 178889, 2024-03-30T02:47:31.699 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 7, 2024-03-30T02:47:31.699 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2024-03-30T02:47:31.699 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.054688088, 2024-03-30T02:47:31.699 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.014977652, 2024-03-30T02:47:31.699 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.020574635, 2024-03-30T02:47:31.699 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.014271614, 2024-03-30T02:47:31.700 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:31.700 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 575, 2024-03-30T02:47:31.700 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.045553258, 2024-03-30T02:47:31.700 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000079223 2024-03-30T02:47:31.700 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.700 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:31.701 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.701 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.701 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.701 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.701 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T02:47:31.702 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T02:47:31.702 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.702 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:31.702 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 678, 2024-03-30T02:47:31.702 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 652, 2024-03-30T02:47:31.703 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 31417486, 2024-03-30T02:47:31.703 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 255295, 2024-03-30T02:47:31.703 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-03-30T02:47:31.703 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-03-30T02:47:31.703 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.096366877, 2024-03-30T02:47:31.704 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.019582679, 2024-03-30T02:47:31.704 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.069918524, 2024-03-30T02:47:31.704 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.017169110, 2024-03-30T02:47:31.704 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:31.704 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 651, 2024-03-30T02:47:31.705 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.052466494, 2024-03-30T02:47:31.705 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000080593 2024-03-30T02:47:31.705 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.705 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:31.705 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.705 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.706 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.706 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.706 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1935696, 2024-03-30T02:47:31.706 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 124048 2024-03-30T02:47:31.706 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.707 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:31.707 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1344, 2024-03-30T02:47:31.707 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1502, 2024-03-30T02:47:31.707 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 13923142, 2024-03-30T02:47:31.707 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 842534, 2024-03-30T02:47:31.708 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 8, 2024-03-30T02:47:31.708 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 6, 2024-03-30T02:47:31.708 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.126079834, 2024-03-30T02:47:31.708 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.034827610, 2024-03-30T02:47:31.708 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.038728515, 2024-03-30T02:47:31.709 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.040807656, 2024-03-30T02:47:31.709 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:31.709 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1500, 2024-03-30T02:47:31.709 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.112654797, 2024-03-30T02:47:31.709 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000075103 2024-03-30T02:47:31.709 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.710 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:31.710 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.710 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.710 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.710 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.711 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-30T02:47:31.711 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 426416 2024-03-30T02:47:31.711 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.711 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-03-30T02:47:31.711 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-03-30T02:47:31.712 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-03-30T02:47:31.712 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.712 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-03-30T02:47:31.712 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-30T02:47:31.712 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-30T02:47:31.712 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3893, 2024-03-30T02:47:31.713 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.106915402, 2024-03-30T02:47:31.713 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000027463 2024-03-30T02:47:31.713 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:31.713 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.713 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:31.713 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-03-30T02:47:31.713 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-03-30T02:47:31.713 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.713 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.714 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.714 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:31.714 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.714 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-03-30T02:47:31.714 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2024-03-30T02:47:31.714 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2024-03-30T02:47:31.714 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-03-30T02:47:31.715 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.715 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.715 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.715 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.715 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-03-30T02:47:31.715 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-03-30T02:47:31.715 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-03-30T02:47:31.715 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:31.716 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-03-30T02:47:31.716 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-03-30T02:47:31.716 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-03-30T02:47:31.716 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 12796, 2024-03-30T02:47:31.716 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-03-30T02:47:31.716 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 12796, 2024-03-30T02:47:31.716 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-03-30T02:47:31.716 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 12796, 2024-03-30T02:47:31.717 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-03-30T02:47:31.717 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-03-30T02:47:31.717 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-03-30T02:47:31.717 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 146, 2024-03-30T02:47:31.717 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2024-03-30T02:47:31.717 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2024-03-30T02:47:31.717 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-03-30T02:47:31.717 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-03-30T02:47:31.717 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:31.718 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:31.718 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-03-30T02:47:31.718 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-03-30T02:47:31.718 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-03-30T02:47:31.718 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-03-30T02:47:31.718 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-03-30T02:47:31.718 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-03-30T02:47:31.718 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-03-30T02:47:31.718 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-03-30T02:47:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2024-03-30T02:47:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2024-03-30T02:47:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-03-30T02:47:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-03-30T02:47:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:31.720 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-03-30T02:47:31.720 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T02:47:31.720 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T02:47:31.720 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-03-30T02:47:31.720 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-03-30T02:47:31.720 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-03-30T02:47:31.720 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-03-30T02:47:31.720 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T02:47:31.721 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:31.721 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:31.721 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-03-30T02:47:31.721 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-03-30T02:47:31.721 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-03-30T02:47:31.721 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-03-30T02:47:31.721 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.721 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-03-30T02:47:31.721 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-03-30T02:47:31.722 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:31.722 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-03-30T02:47:31.722 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-03-30T02:47:31.722 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-03-30T02:47:31.722 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-03-30T02:47:31.722 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:31.722 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:31.722 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-03-30T02:47:31.723 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-03-30T02:47:31.723 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:31.723 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-03-30T02:47:31.723 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:31.723 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:31.723 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:31.723 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-03-30T02:47:31.723 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-03-30T02:47:31.723 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T02:47:31.724 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-03-30T02:47:31.724 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-03-30T02:47:31.724 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-03-30T02:47:31.724 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:31.724 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-03-30T02:47:31.724 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.724 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-03-30T02:47:31.724 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 0, 2024-03-30T02:47:31.724 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-03-30T02:47:31.725 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-03-30T02:47:31.725 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 20328, 2024-03-30T02:47:31.725 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-03-30T02:47:31.725 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-03-30T02:47:31.725 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-03-30T02:47:31.725 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-03-30T02:47:31.725 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-03-30T02:47:31.725 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 30, 2024-03-30T02:47:31.725 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-03-30T02:47:31.726 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-03-30T02:47:31.726 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-03-30T02:47:31.726 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 63852700, 2024-03-30T02:47:31.726 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 63852126, 2024-03-30T02:47:31.726 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-03-30T02:47:31.726 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.726 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.726 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 20328 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 12751, 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 13240, 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 489, 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 831, 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 870, 2024-03-30T02:47:31.727 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 39, 2024-03-30T02:47:31.728 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 12796, 2024-03-30T02:47:31.728 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 13261, 2024-03-30T02:47:31.728 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 465, 2024-03-30T02:47:31.728 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-03-30T02:47:31.728 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2024-03-30T02:47:31.728 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-03-30T02:47:31.728 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 99780, 2024-03-30T02:47:31.728 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 207156 2024-03-30T02:47:31.729 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.729 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-03-30T02:47:31.729 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2024-03-30T02:47:31.729 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-03-30T02:47:31.729 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2024-03-30T02:47:31.729 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 0, 2024-03-30T02:47:31.729 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-03-30T02:47:31.729 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-03-30T02:47:31.729 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.730 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.730 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.730 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.730 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-03-30T02:47:31.730 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.730 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.730 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.730 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.730 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:31.731 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.731 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.731 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.731 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.731 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-03-30T02:47:31.731 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.731 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.731 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.731 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.732 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-03-30T02:47:31.732 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.732 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.732 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.732 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.732 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:31.732 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.732 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.732 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.733 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.733 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-03-30T02:47:31.733 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.733 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.733 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.733 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.733 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-03-30T02:47:31.734 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.734 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.734 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.734 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.734 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:31.734 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.734 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.735 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.735 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.735 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:31.735 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.735 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.735 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.735 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.735 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-03-30T02:47:31.735 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.736 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.736 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.736 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.736 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-03-30T02:47:31.736 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.736 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.736 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.736 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-03-30T02:47:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-03-30T02:47:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.737 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.738 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.738 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-03-30T02:47:31.738 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.738 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.738 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.738 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.738 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-03-30T02:47:31.739 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.739 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.739 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.739 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.739 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-03-30T02:47:31.739 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.739 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.740 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.740 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.740 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:31.740 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.740 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.740 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.740 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.740 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:31.741 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.741 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.741 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.741 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.741 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-03-30T02:47:31.741 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.741 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.742 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.742 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.742 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:31.742 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.742 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.742 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.742 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.742 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:31.742 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.743 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.743 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.743 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.743 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-03-30T02:47:31.743 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.743 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.743 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.743 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.743 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:31.744 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.744 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.744 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.744 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.744 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:31.744 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.744 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.744 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.744 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.745 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-03-30T02:47:31.745 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.745 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.745 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.745 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.745 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-03-30T02:47:31.745 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.745 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.745 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.746 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.746 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-03-30T02:47:31.746 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.746 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.746 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.746 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.746 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-03-30T02:47:31.746 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.747 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.747 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.747 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.747 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:31.747 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:31.747 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:31.747 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.747 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.747 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.748 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:31.748 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.748 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-03-30T02:47:31.748 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-03-30T02:47:31.748 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 4, 2024-03-30T02:47:31.748 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 3, 2024-03-30T02:47:31.748 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 3, 2024-03-30T02:47:31.748 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-03-30T02:47:31.748 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2024-03-30T02:47:31.749 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2024-03-30T02:47:31.749 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 6411, 2024-03-30T02:47:31.749 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:31.749 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.749 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-03-30T02:47:31.750 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:31.750 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:31.750 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:31.750 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:31.750 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:31.750 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:31.750 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:31.751 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:31.751 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:31.751 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:31.751 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:31.751 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:31.751 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:31.751 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:31.751 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:31.752 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:31.752 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:31.752 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:31.752 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:31.752 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:31.753 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:31.753 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:31.753 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:31.753 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:31.753 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:31.753 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:31.754 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:31.754 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:31.754 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:31.754 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:31.755 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:31.755 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-03-30T02:47:31.755 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:31.755 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:31.755 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:31.755 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:31.756 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 684404, 2024-03-30T02:47:31.756 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 13655, 2024-03-30T02:47:31.756 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-03-30T02:47:31.756 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-03-30T02:47:31.756 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-03-30T02:47:31.757 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-03-30T02:47:31.757 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:31.757 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:31.757 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:31.758 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:31.758 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9656, 2024-03-30T02:47:31.758 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 203, 2024-03-30T02:47:31.758 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:31.758 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:31.759 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:31.759 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-03-30T02:47:31.759 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 39707366, 2024-03-30T02:47:31.759 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 402622, 2024-03-30T02:47:31.759 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:31.760 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-03-30T02:47:31.760 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:31.760 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-03-30T02:47:31.760 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.760 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-03-30T02:47:31.761 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-03-30T02:47:31.761 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-03-30T02:47:31.761 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 2335, 2024-03-30T02:47:31.761 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 21, 2024-03-30T02:47:31.761 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 3, 2024-03-30T02:47:31.762 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 2335, 2024-03-30T02:47:31.762 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-03-30T02:47:31.762 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2335, 2024-03-30T02:47:31.762 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 13.270737543, 2024-03-30T02:47:31.762 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.005683399 2024-03-30T02:47:31.763 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.763 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-03-30T02:47:31.763 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-03-30T02:47:31.763 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2335, 2024-03-30T02:47:31.763 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2336 2024-03-30T02:47:31.764 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.764 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 2335, 2024-03-30T02:47:31.764 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 2335, 2024-03-30T02:47:31.764 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 0, 2024-03-30T02:47:31.764 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-03-30T02:47:31.765 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-03-30T02:47:31.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 1432, 2024-03-30T02:47:31.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-03-30T02:47:31.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 901, 2024-03-30T02:47:31.765 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 0, 2024-03-30T02:47:31.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 0, 2024-03-30T02:47:31.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-03-30T02:47:31.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-03-30T02:47:31.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-03-30T02:47:31.766 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-03-30T02:47:31.767 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 0, 2024-03-30T02:47:31.767 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-03-30T02:47:31.767 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:31.767 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:31.767 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1, 2024-03-30T02:47:31.767 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-03-30T02:47:31.767 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:31.767 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:31.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:31.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-03-30T02:47:31.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-03-30T02:47:31.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-03-30T02:47:31.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:31.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-03-30T02:47:31.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:31.768 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-03-30T02:47:31.768 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-03-30T02:47:31.769 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-03-30T02:47:31.769 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-03-30T02:47:31.769 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-03-30T02:47:31.769 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-03-30T02:47:31.769 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-03-30T02:47:31.769 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-03-30T02:47:31.769 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-03-30T02:47:31.769 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-03-30T02:47:31.769 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-03-30T02:47:31.770 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-03-30T02:47:31.770 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-03-30T02:47:31.770 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-03-30T02:47:31.770 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-03-30T02:47:31.770 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-03-30T02:47:31.770 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-03-30T02:47:31.770 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 96, 2024-03-30T02:47:31.770 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-03-30T02:47:31.770 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 13, 2024-03-30T02:47:31.771 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 9, 2024-03-30T02:47:31.771 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 9, 2024-03-30T02:47:31.771 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-03-30T02:47:31.771 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-03-30T02:47:31.771 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 0, 2024-03-30T02:47:31.771 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 2, 2024-03-30T02:47:31.771 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-03-30T02:47:31.771 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.771 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-03-30T02:47:31.772 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-03-30T02:47:31.772 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:31.772 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-03-30T02:47:31.772 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-03-30T02:47:31.772 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.772 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-03-30T02:47:31.772 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-03-30T02:47:31.772 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.773 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-03-30T02:47:31.773 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:31.773 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-03-30T02:47:31.773 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-03-30T02:47:31.773 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-03-30T02:47:31.773 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-03-30T02:47:31.773 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:31.773 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.773 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:31.774 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:31.774 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-30T02:47:31.774 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:31.774 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2593, 2024-03-30T02:47:31.774 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 62017545, 2024-03-30T02:47:31.774 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:31.774 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2593, 2024-03-30T02:47:31.774 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:31.774 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:31.775 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2593, 2024-03-30T02:47:31.775 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 62017545, 2024-03-30T02:47:31.775 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:31.775 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.775 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.775 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.775 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:31.775 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.775 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:31.776 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:31.776 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-03-30T02:47:31.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:31.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:31.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:31.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:31.776 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:31.776 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2335, 2024-03-30T02:47:31.776 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 59675009, 2024-03-30T02:47:31.777 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2335, 2024-03-30T02:47:31.777 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 59675009, 2024-03-30T02:47:31.777 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:31.777 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.777 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.777 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.777 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:31.777 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.777 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:31.778 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:31.778 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-03-30T02:47:31.778 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:31.778 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:31.778 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:31.778 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:31.778 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:31.778 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2335, 2024-03-30T02:47:31.778 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 2335, 2024-03-30T02:47:31.779 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2335, 2024-03-30T02:47:31.779 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2335, 2024-03-30T02:47:31.779 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:31.779 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.779 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.779 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.779 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:31.779 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:31.780 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x5620af0f8500": { 2024-03-30T02:47:31.781 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-03-30T02:47:31.782 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-03-30T02:47:31.782 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-03-30T02:47:31.782 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-03-30T02:47:31.782 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-03-30T02:47:31.782 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:31.782 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:31.782 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-03-30T02:47:31.782 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-03-30T02:47:31.783 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-03-30T02:47:31.783 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-03-30T02:47:31.783 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-03-30T02:47:31.783 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-03-30T02:47:31.783 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-03-30T02:47:31.783 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-03-30T02:47:31.783 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:31.783 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-30T02:47:31.783 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-30T02:47:32.061 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-30T02:47:32.061 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-30T02:47:32.061 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-03-30T02:47:32.062 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-30T02:47:32.062 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-30T02:47:32.062 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-03-30T02:47:32.062 DEBUG:tasks.check_counter:Getting stats from c 2024-03-30T02:47:32.062 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.c perf dump 2024-03-30T02:47:32.309 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:47:32.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:32 smithi170 ceph-mon[78798]: pgmap v325: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-30T02:47:32.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:32 smithi082 ceph-mon[79176]: pgmap v325: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-30T02:47:32.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:32 smithi067 ceph-mon[37808]: pgmap v325: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-03-30T02:47:33.330 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-30T02:47:33.330 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:33.331 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 19902, 2024-03-30T02:47:33.331 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 15304, 2024-03-30T02:47:33.331 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 10682332, 2024-03-30T02:47:33.331 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 14690514, 2024-03-30T02:47:33.331 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 10, 2024-03-30T02:47:33.331 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-03-30T02:47:33.331 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.968843396, 2024-03-30T02:47:33.331 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.330393291, 2024-03-30T02:47:33.332 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.591119983, 2024-03-30T02:47:33.332 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.159985853, 2024-03-30T02:47:33.332 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:33.332 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 15303, 2024-03-30T02:47:33.332 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.031719003, 2024-03-30T02:47:33.332 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000067419 2024-03-30T02:47:33.332 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.333 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:33.333 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 19461, 2024-03-30T02:47:33.333 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.003259141, 2024-03-30T02:47:33.333 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000167 2024-03-30T02:47:33.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.333 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-30T02:47:33.333 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448 2024-03-30T02:47:33.333 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.334 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:33.334 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 22381, 2024-03-30T02:47:33.334 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 26732, 2024-03-30T02:47:33.334 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 6640012, 2024-03-30T02:47:33.334 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 12265823, 2024-03-30T02:47:33.334 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 16, 2024-03-30T02:47:33.334 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-03-30T02:47:33.334 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.091725065, 2024-03-30T02:47:33.334 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.362416038, 2024-03-30T02:47:33.335 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.686489464, 2024-03-30T02:47:33.335 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.272892494, 2024-03-30T02:47:33.335 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:33.335 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 26729, 2024-03-30T02:47:33.335 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.881151179, 2024-03-30T02:47:33.335 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000145203 2024-03-30T02:47:33.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.336 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:33.336 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14375, 2024-03-30T02:47:33.336 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002683031, 2024-03-30T02:47:33.336 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000186 2024-03-30T02:47:33.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.336 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1232, 2024-03-30T02:47:33.336 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 429040 2024-03-30T02:47:33.336 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.337 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:33.337 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 15630, 2024-03-30T02:47:33.337 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 15746, 2024-03-30T02:47:33.337 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 5915702, 2024-03-30T02:47:33.337 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 15360677, 2024-03-30T02:47:33.337 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 37, 2024-03-30T02:47:33.337 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 9, 2024-03-30T02:47:33.337 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.936239864, 2024-03-30T02:47:33.337 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.331722719, 2024-03-30T02:47:33.338 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.263412019, 2024-03-30T02:47:33.338 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.297834140, 2024-03-30T02:47:33.338 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:33.338 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 15745, 2024-03-30T02:47:33.338 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.679117279, 2024-03-30T02:47:33.338 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000043132 2024-03-30T02:47:33.338 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.338 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:33.339 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 880, 2024-03-30T02:47:33.339 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000159719, 2024-03-30T02:47:33.339 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000181 2024-03-30T02:47:33.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.339 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1971440, 2024-03-30T02:47:33.339 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 426304 2024-03-30T02:47:33.339 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.339 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-03-30T02:47:33.339 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-03-30T02:47:33.340 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-03-30T02:47:33.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.340 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-03-30T02:47:33.340 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-30T02:47:33.340 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-30T02:47:33.340 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 27082, 2024-03-30T02:47:33.340 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 6.272665700, 2024-03-30T02:47:33.340 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000231617 2024-03-30T02:47:33.340 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:33.341 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.341 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:33.341 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-30T02:47:33.341 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-30T02:47:33.341 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 59, 2024-03-30T02:47:33.341 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002843746, 2024-03-30T02:47:33.341 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000048199 2024-03-30T02:47:33.341 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-03-30T02:47:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 647, 2024-03-30T02:47:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 568, 2024-03-30T02:47:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-03-30T02:47:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 568, 2024-03-30T02:47:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.710323666, 2024-03-30T02:47:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001250569 2024-03-30T02:47:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-03-30T02:47:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 79, 2024-03-30T02:47:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-03-30T02:47:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 47, 2024-03-30T02:47:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 4, 2024-03-30T02:47:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-03-30T02:47:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 6113, 2024-03-30T02:47:33.344 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 194, 2024-03-30T02:47:33.344 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 5919, 2024-03-30T02:47:33.344 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-03-30T02:47:33.344 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 5836, 2024-03-30T02:47:33.344 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-03-30T02:47:33.344 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 3994, 2024-03-30T02:47:33.344 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 3994, 2024-03-30T02:47:33.344 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 192, 2024-03-30T02:47:33.344 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 1093, 2024-03-30T02:47:33.345 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 569, 2024-03-30T02:47:33.345 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 58, 2024-03-30T02:47:33.345 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 300, 2024-03-30T02:47:33.345 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:33.345 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:33.345 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-03-30T02:47:33.345 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-03-30T02:47:33.345 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-03-30T02:47:33.346 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-03-30T02:47:33.346 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-03-30T02:47:33.346 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 222, 2024-03-30T02:47:33.346 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 3974, 2024-03-30T02:47:33.346 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:33.346 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 36212, 2024-03-30T02:47:33.346 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:33.346 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 85384, 2024-03-30T02:47:33.347 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 1566089959, 2024-03-30T02:47:33.347 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-03-30T02:47:33.347 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 36212, 2024-03-30T02:47:33.347 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:33.347 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:33.347 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:33.347 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 1984, 2024-03-30T02:47:33.347 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T02:47:33.347 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 1984, 2024-03-30T02:47:33.348 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 34228, 2024-03-30T02:47:33.348 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 22, 2024-03-30T02:47:33.348 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 172, 2024-03-30T02:47:33.348 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 408, 2024-03-30T02:47:33.348 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 2, 2024-03-30T02:47:33.348 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:33.348 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:33.348 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 6311, 2024-03-30T02:47:33.348 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 5998, 2024-03-30T02:47:33.349 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 62, 2024-03-30T02:47:33.349 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 371 2024-03-30T02:47:33.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.349 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-03-30T02:47:33.349 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-03-30T02:47:33.349 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:33.349 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-03-30T02:47:33.349 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 247, 2024-03-30T02:47:33.349 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 3, 2024-03-30T02:47:33.350 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-03-30T02:47:33.350 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:33.350 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:33.350 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 17, 2024-03-30T02:47:33.350 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 17, 2024-03-30T02:47:33.350 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:33.350 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-03-30T02:47:33.350 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:33.351 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:33.351 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:33.351 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-03-30T02:47:33.351 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-03-30T02:47:33.351 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T02:47:33.351 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-03-30T02:47:33.351 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-03-30T02:47:33.351 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 4, 2024-03-30T02:47:33.351 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:33.352 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-03-30T02:47:33.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.352 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-03-30T02:47:33.352 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 7279, 2024-03-30T02:47:33.352 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-03-30T02:47:33.352 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-03-30T02:47:33.352 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 7279, 2024-03-30T02:47:33.352 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-03-30T02:47:33.353 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-03-30T02:47:33.353 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 11, 2024-03-30T02:47:33.353 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-03-30T02:47:33.353 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-03-30T02:47:33.353 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 11, 2024-03-30T02:47:33.353 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-03-30T02:47:33.353 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-03-30T02:47:33.353 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-03-30T02:47:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 25814519, 2024-03-30T02:47:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-03-30T02:47:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-03-30T02:47:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 526, 2024-03-30T02:47:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 12.352190858, 2024-03-30T02:47:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.023483252 2024-03-30T02:47:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-03-30T02:47:33.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.355 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-03-30T02:47:33.355 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 6076, 2024-03-30T02:47:33.355 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 6093, 2024-03-30T02:47:33.355 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 17, 2024-03-30T02:47:33.355 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 642, 2024-03-30T02:47:33.355 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 664, 2024-03-30T02:47:33.355 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 22, 2024-03-30T02:47:33.355 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 6113, 2024-03-30T02:47:33.356 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 6130, 2024-03-30T02:47:33.356 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 17, 2024-03-30T02:47:33.356 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 3994, 2024-03-30T02:47:33.356 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 6484, 2024-03-30T02:47:33.356 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 2490, 2024-03-30T02:47:33.356 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 89060, 2024-03-30T02:47:33.356 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 198964 2024-03-30T02:47:33.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.357 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-03-30T02:47:33.357 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 805, 2024-03-30T02:47:33.357 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-03-30T02:47:33.357 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 647, 2024-03-30T02:47:33.357 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 12, 2024-03-30T02:47:33.357 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 20, 2024-03-30T02:47:33.357 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-03-30T02:47:33.357 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 55, 2024-03-30T02:47:33.357 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.099601158, 2024-03-30T02:47:33.358 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001810930 2024-03-30T02:47:33.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.358 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-03-30T02:47:33.358 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-03-30T02:47:33.358 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.011283543, 2024-03-30T02:47:33.358 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.011283543 2024-03-30T02:47:33.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.358 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:33.359 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.359 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.359 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.359 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-03-30T02:47:33.359 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.359 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.359 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.360 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-03-30T02:47:33.360 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 94, 2024-03-30T02:47:33.360 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.034520864, 2024-03-30T02:47:33.360 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000367243 2024-03-30T02:47:33.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.360 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:33.360 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.360 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.361 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.361 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-03-30T02:47:33.361 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.361 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.361 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.361 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-03-30T02:47:33.361 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.362 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.362 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.362 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:33.362 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.362 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.362 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.362 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:33.363 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.363 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.363 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.363 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-03-30T02:47:33.363 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.363 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.363 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.364 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-03-30T02:47:33.364 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.364 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.364 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.364 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-03-30T02:47:33.364 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.364 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.365 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.365 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-03-30T02:47:33.365 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.365 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.365 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.365 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-03-30T02:47:33.365 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 47, 2024-03-30T02:47:33.366 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.020523461, 2024-03-30T02:47:33.366 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000436669 2024-03-30T02:47:33.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.366 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-03-30T02:47:33.366 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 352, 2024-03-30T02:47:33.366 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.489077211, 2024-03-30T02:47:33.367 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001389423 2024-03-30T02:47:33.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.367 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-03-30T02:47:33.367 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.367 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.367 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.367 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:33.368 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.368 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.368 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.368 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:33.368 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.368 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.368 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.369 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-03-30T02:47:33.369 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.369 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.369 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.369 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:33.369 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.369 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.369 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.370 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:33.370 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.370 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.370 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.370 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-03-30T02:47:33.370 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2024-03-30T02:47:33.371 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.024517134, 2024-03-30T02:47:33.371 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.012258567 2024-03-30T02:47:33.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.371 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:33.371 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.371 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.371 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.371 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:33.372 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.372 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.372 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.372 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-03-30T02:47:33.372 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.372 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.372 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.373 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-03-30T02:47:33.373 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.373 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.373 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.373 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-03-30T02:47:33.373 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.373 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.374 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.374 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-03-30T02:47:33.374 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 17, 2024-03-30T02:47:33.374 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.030800295, 2024-03-30T02:47:33.374 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001811782 2024-03-30T02:47:33.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.375 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:33.375 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:33.375 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:33.375 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.375 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.375 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.375 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:33.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.376 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-03-30T02:47:33.376 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2024-03-30T02:47:33.376 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 2, 2024-03-30T02:47:33.376 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 1, 2024-03-30T02:47:33.376 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2024-03-30T02:47:33.376 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-03-30T02:47:33.376 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 113, 2024-03-30T02:47:33.377 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 113, 2024-03-30T02:47:33.377 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 2562, 2024-03-30T02:47:33.377 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:33.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.377 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-03-30T02:47:33.377 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:33.377 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:33.377 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:33.377 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:33.378 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:33.378 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:33.378 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:33.378 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:33.378 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:33.378 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:33.379 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:33.379 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:33.379 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:33.379 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:33.379 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:33.379 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:33.379 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:33.379 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:33.379 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:33.380 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:33.380 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:33.380 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:33.380 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:33.380 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:33.380 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:33.380 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:33.380 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:33.380 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:33.381 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:33.381 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:33.381 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:33.381 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-03-30T02:47:33.381 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:33.381 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:33.381 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:33.381 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:33.381 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 469773, 2024-03-30T02:47:33.382 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 4415, 2024-03-30T02:47:33.382 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 1232, 2024-03-30T02:47:33.382 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 14, 2024-03-30T02:47:33.382 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-03-30T02:47:33.382 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-03-30T02:47:33.382 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:33.382 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:33.383 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:33.383 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:33.383 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9656, 2024-03-30T02:47:33.383 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 203, 2024-03-30T02:47:33.383 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:33.383 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:33.383 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:33.383 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-03-30T02:47:33.383 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 20756772, 2024-03-30T02:47:33.384 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 232056, 2024-03-30T02:47:33.384 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:33.384 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-03-30T02:47:33.384 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:33.384 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-03-30T02:47:33.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.384 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-03-30T02:47:33.384 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-03-30T02:47:33.384 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-03-30T02:47:33.385 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 36964, 2024-03-30T02:47:33.385 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 22980196, 2024-03-30T02:47:33.385 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 33, 2024-03-30T02:47:33.385 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 36964, 2024-03-30T02:47:33.385 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-03-30T02:47:33.385 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 36964, 2024-03-30T02:47:33.385 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 58.947605652, 2024-03-30T02:47:33.385 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001594730 2024-03-30T02:47:33.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.386 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-03-30T02:47:33.386 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-03-30T02:47:33.386 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 36964, 2024-03-30T02:47:33.386 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 37119 2024-03-30T02:47:33.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.386 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 36964, 2024-03-30T02:47:33.386 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 36217, 2024-03-30T02:47:33.386 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 747, 2024-03-30T02:47:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-03-30T02:47:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-03-30T02:47:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-03-30T02:47:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-03-30T02:47:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 5, 2024-03-30T02:47:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 531, 2024-03-30T02:47:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 38, 2024-03-30T02:47:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-03-30T02:47:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-03-30T02:47:33.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-03-30T02:47:33.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 4, 2024-03-30T02:47:33.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 51, 2024-03-30T02:47:33.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-03-30T02:47:33.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:33.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:33.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 36212, 2024-03-30T02:47:33.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-03-30T02:47:33.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:33.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:33.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:33.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-03-30T02:47:33.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-03-30T02:47:33.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-03-30T02:47:33.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:33.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-03-30T02:47:33.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:33.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 2, 2024-03-30T02:47:33.390 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-03-30T02:47:33.390 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-03-30T02:47:33.390 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-03-30T02:47:33.390 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-03-30T02:47:33.390 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-03-30T02:47:33.390 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-03-30T02:47:33.390 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-03-30T02:47:33.390 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-03-30T02:47:33.391 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-03-30T02:47:33.391 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-03-30T02:47:33.391 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-03-30T02:47:33.391 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-03-30T02:47:33.391 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-03-30T02:47:33.391 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-03-30T02:47:33.391 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-03-30T02:47:33.391 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-03-30T02:47:33.391 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 96, 2024-03-30T02:47:33.392 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-03-30T02:47:33.392 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 13, 2024-03-30T02:47:33.392 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-03-30T02:47:33.392 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-03-30T02:47:33.392 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-03-30T02:47:33.392 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-03-30T02:47:33.392 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 199, 2024-03-30T02:47:33.392 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2024-03-30T02:47:33.392 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 47 2024-03-30T02:47:33.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.393 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-03-30T02:47:33.393 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-03-30T02:47:33.393 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:33.393 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-03-30T02:47:33.393 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-03-30T02:47:33.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.393 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-03-30T02:47:33.393 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 892 2024-03-30T02:47:33.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.394 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-03-30T02:47:33.394 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:33.394 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 10, 2024-03-30T02:47:33.394 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-03-30T02:47:33.394 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 5, 2024-03-30T02:47:33.394 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 17, 2024-03-30T02:47:33.394 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:33.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.395 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:33.395 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:33.395 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-30T02:47:33.395 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:33.395 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 57909, 2024-03-30T02:47:33.395 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 18002240, 2024-03-30T02:47:33.395 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:33.396 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 57909, 2024-03-30T02:47:33.396 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:33.396 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:33.396 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 57909, 2024-03-30T02:47:33.396 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 18002240, 2024-03-30T02:47:33.396 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:33.396 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.396 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.397 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.397 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:33.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.397 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:33.397 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:33.397 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-30T02:47:33.397 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:33.397 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:33.397 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:33.398 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:33.398 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:33.398 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 36964, 2024-03-30T02:47:33.398 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 22982049, 2024-03-30T02:47:33.398 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 36964, 2024-03-30T02:47:33.398 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 22982049, 2024-03-30T02:47:33.398 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:33.398 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.398 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.399 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.399 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:33.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.399 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:33.399 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:33.399 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-03-30T02:47:33.399 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:33.399 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:33.399 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:33.400 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:33.400 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:33.400 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 36964, 2024-03-30T02:47:33.400 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 36964, 2024-03-30T02:47:33.400 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 36964, 2024-03-30T02:47:33.400 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 36964, 2024-03-30T02:47:33.400 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:33.400 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.401 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.401 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.401 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:33.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.401 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:33.401 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:33.401 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-30T02:47:33.401 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:33.401 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 17, 2024-03-30T02:47:33.402 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1853, 2024-03-30T02:47:33.402 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:33.402 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 17, 2024-03-30T02:47:33.402 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:33.402 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:33.402 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 5, 2024-03-30T02:47:33.402 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1853, 2024-03-30T02:47:33.402 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:33.402 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.403 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.403 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.403 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:33.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:33.403 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5583195f20a0": { 2024-03-30T02:47:33.403 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:33.403 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-30T02:47:33.403 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:33.403 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 7279, 2024-03-30T02:47:33.404 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 21620215, 2024-03-30T02:47:33.404 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:33.404 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 7279, 2024-03-30T02:47:33.404 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:33.404 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:33.404 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 526, 2024-03-30T02:47:33.404 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 21620215, 2024-03-30T02:47:33.404 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:33.404 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:33.405 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:33.405 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:33.405 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:33.405 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:33.405 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-30T02:47:33.703 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-30T02:47:33.703 INFO:tasks.check_counter:Daemon mds.c mds.imported=222 2024-03-30T02:47:33.703 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-30T02:47:33.703 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-30T02:47:33.703 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-30T02:47:33.703 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=4 2024-03-30T02:47:33.703 DEBUG:tasks.check_counter:Getting stats from f 2024-03-30T02:47:33.704 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.f perf dump 2024-03-30T02:47:33.925 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:47:34.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:34 smithi170 ceph-mon[78798]: pgmap v326: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-03-30T02:47:34.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:34 smithi082 ceph-mon[79176]: pgmap v326: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-03-30T02:47:34.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:34 smithi067 ceph-mon[37808]: pgmap v326: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-03-30T02:47:34.924 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-30T02:47:34.924 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:34.924 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1639, 2024-03-30T02:47:34.924 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1641, 2024-03-30T02:47:34.924 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 289826188, 2024-03-30T02:47:34.925 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 509739, 2024-03-30T02:47:34.925 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 8, 2024-03-30T02:47:34.925 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-03-30T02:47:34.925 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.299277941, 2024-03-30T02:47:34.925 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.041712174, 2024-03-30T02:47:34.925 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.191588925, 2024-03-30T02:47:34.925 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.050053363, 2024-03-30T02:47:34.925 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:34.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1639, 2024-03-30T02:47:34.926 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.120911646, 2024-03-30T02:47:34.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000073771 2024-03-30T02:47:34.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.926 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:34.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.926 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.927 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-30T02:47:34.927 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-03-30T02:47:34.927 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.927 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:34.927 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 783, 2024-03-30T02:47:34.927 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 940, 2024-03-30T02:47:34.927 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 265167719, 2024-03-30T02:47:34.927 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 667953, 2024-03-30T02:47:34.928 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 9, 2024-03-30T02:47:34.928 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-03-30T02:47:34.928 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.190571818, 2024-03-30T02:47:34.928 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.023400717, 2024-03-30T02:47:34.928 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.139855694, 2024-03-30T02:47:34.928 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.019706735, 2024-03-30T02:47:34.928 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:34.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 938, 2024-03-30T02:47:34.928 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.092336982, 2024-03-30T02:47:34.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000098440 2024-03-30T02:47:34.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.929 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:34.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.929 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.929 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-03-30T02:47:34.929 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 426240 2024-03-30T02:47:34.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.930 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:34.930 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 994, 2024-03-30T02:47:34.930 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 968, 2024-03-30T02:47:34.930 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 250829909, 2024-03-30T02:47:34.930 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 353151, 2024-03-30T02:47:34.930 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-03-30T02:47:34.930 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-03-30T02:47:34.931 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.239521973, 2024-03-30T02:47:34.931 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.030792540, 2024-03-30T02:47:34.931 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.188592643, 2024-03-30T02:47:34.931 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.026159316, 2024-03-30T02:47:34.931 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:34.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 967, 2024-03-30T02:47:34.931 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.080033396, 2024-03-30T02:47:34.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000082764 2024-03-30T02:47:34.931 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.932 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:34.932 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.932 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.932 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.932 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1935584, 2024-03-30T02:47:34.932 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 123632 2024-03-30T02:47:34.932 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.932 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-03-30T02:47:34.933 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-03-30T02:47:34.933 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-03-30T02:47:34.933 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.933 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-03-30T02:47:34.933 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-30T02:47:34.933 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-30T02:47:34.933 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4840, 2024-03-30T02:47:34.933 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.847626515, 2024-03-30T02:47:34.933 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000175129 2024-03-30T02:47:34.934 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:34.934 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.934 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:34.934 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-30T02:47:34.934 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-30T02:47:34.934 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.934 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.934 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.934 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:34.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.935 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-03-30T02:47:34.935 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-03-30T02:47:34.935 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-03-30T02:47:34.935 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-03-30T02:47:34.935 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.935 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.935 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.936 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-03-30T02:47:34.936 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-03-30T02:47:34.936 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-03-30T02:47:34.936 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:34.936 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-03-30T02:47:34.936 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-03-30T02:47:34.936 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-03-30T02:47:34.936 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 99698, 2024-03-30T02:47:34.937 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 45180, 2024-03-30T02:47:34.937 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 54518, 2024-03-30T02:47:34.937 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-03-30T02:47:34.937 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 34640, 2024-03-30T02:47:34.937 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-03-30T02:47:34.937 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-03-30T02:47:34.937 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-03-30T02:47:34.937 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 469, 2024-03-30T02:47:34.937 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-03-30T02:47:34.938 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-03-30T02:47:34.938 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-03-30T02:47:34.938 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-03-30T02:47:34.938 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:34.938 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:34.938 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-03-30T02:47:34.938 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-03-30T02:47:34.938 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-03-30T02:47:34.938 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-03-30T02:47:34.939 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-03-30T02:47:34.939 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-03-30T02:47:34.939 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-03-30T02:47:34.939 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:34.939 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-03-30T02:47:34.939 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:34.939 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 85384, 2024-03-30T02:47:34.939 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 1566089959, 2024-03-30T02:47:34.939 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-03-30T02:47:34.940 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-03-30T02:47:34.940 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:34.940 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:34.940 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:34.940 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-03-30T02:47:34.940 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T02:47:34.940 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T02:47:34.940 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-03-30T02:47:34.940 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-03-30T02:47:34.941 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-03-30T02:47:34.941 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-03-30T02:47:34.941 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T02:47:34.941 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:34.941 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:34.941 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-03-30T02:47:34.941 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-03-30T02:47:34.941 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-03-30T02:47:34.941 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-03-30T02:47:34.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.942 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-03-30T02:47:34.942 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-03-30T02:47:34.942 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:34.942 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-03-30T02:47:34.942 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-03-30T02:47:34.942 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-03-30T02:47:34.942 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-03-30T02:47:34.943 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:34.943 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:34.943 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-03-30T02:47:34.943 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-03-30T02:47:34.943 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:34.943 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-03-30T02:47:34.943 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:34.943 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:34.943 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:34.944 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-03-30T02:47:34.944 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-03-30T02:47:34.944 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T02:47:34.944 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-03-30T02:47:34.944 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-03-30T02:47:34.944 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-03-30T02:47:34.944 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:34.944 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-03-30T02:47:34.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.945 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-03-30T02:47:34.945 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-03-30T02:47:34.945 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-03-30T02:47:34.945 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-03-30T02:47:34.945 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 211877, 2024-03-30T02:47:34.945 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-03-30T02:47:34.945 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-03-30T02:47:34.945 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-03-30T02:47:34.945 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-03-30T02:47:34.946 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-03-30T02:47:34.946 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 132, 2024-03-30T02:47:34.946 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-03-30T02:47:34.946 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-03-30T02:47:34.946 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 440411099, 2024-03-30T02:47:34.946 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 800792113, 2024-03-30T02:47:34.946 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 800756914, 2024-03-30T02:47:34.946 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-03-30T02:47:34.947 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.947 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.947 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.947 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 211877 2024-03-30T02:47:34.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.947 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-03-30T02:47:34.947 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 97275, 2024-03-30T02:47:34.947 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 110024, 2024-03-30T02:47:34.948 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 12749, 2024-03-30T02:47:34.948 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 6123, 2024-03-30T02:47:34.948 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 6997, 2024-03-30T02:47:34.948 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 874, 2024-03-30T02:47:34.948 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 99698, 2024-03-30T02:47:34.948 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 111261, 2024-03-30T02:47:34.948 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 11563, 2024-03-30T02:47:34.948 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-03-30T02:47:34.948 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-03-30T02:47:34.949 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-03-30T02:47:34.949 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 411620, 2024-03-30T02:47:34.949 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 207156 2024-03-30T02:47:34.949 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.949 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-03-30T02:47:34.949 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-03-30T02:47:34.949 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-03-30T02:47:34.949 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-03-30T02:47:34.949 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-03-30T02:47:34.950 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-03-30T02:47:34.950 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-03-30T02:47:34.950 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.950 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.950 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.950 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.950 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-03-30T02:47:34.950 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.950 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.951 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.951 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:34.951 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.951 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.951 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.951 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.951 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-03-30T02:47:34.951 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.952 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.952 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.952 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.953 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-03-30T02:47:34.953 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.954 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.954 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.954 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.954 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:34.954 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.954 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.954 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.954 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.955 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-03-30T02:47:34.955 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.955 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.955 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.955 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.955 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-03-30T02:47:34.955 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.955 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.955 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.956 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:34.956 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.956 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.956 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.956 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.956 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:34.956 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.956 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.957 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.957 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-03-30T02:47:34.957 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.957 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.957 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.957 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.957 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-03-30T02:47:34.957 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.958 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.958 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.958 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-03-30T02:47:34.958 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.958 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.958 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.958 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.959 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-03-30T02:47:34.959 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.959 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.959 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.959 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.959 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-03-30T02:47:34.959 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.959 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.959 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.960 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-03-30T02:47:34.960 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.960 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.960 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.960 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.960 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-03-30T02:47:34.960 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.960 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.961 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.961 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:34.961 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.961 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.961 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.961 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.961 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:34.961 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.962 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.962 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.962 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.962 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-03-30T02:47:34.962 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.962 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.962 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.962 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.963 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:34.963 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.963 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.963 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.963 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.963 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:34.963 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.963 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.963 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.964 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-03-30T02:47:34.964 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.964 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.964 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.964 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.964 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:34.964 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.964 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.965 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.965 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:34.965 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.965 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.965 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.965 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.965 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-03-30T02:47:34.966 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.966 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.966 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.966 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-03-30T02:47:34.966 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.966 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.966 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.966 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.967 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-03-30T02:47:34.967 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.967 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.967 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.967 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.967 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-03-30T02:47:34.967 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.967 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.968 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.968 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.968 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:34.968 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:34.968 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:34.968 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.968 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.968 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.968 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:34.969 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.969 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-03-30T02:47:34.969 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-03-30T02:47:34.969 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 6, 2024-03-30T02:47:34.969 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 4, 2024-03-30T02:47:34.969 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 3, 2024-03-30T02:47:34.969 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-03-30T02:47:34.969 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-03-30T02:47:34.969 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-03-30T02:47:34.970 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 6405, 2024-03-30T02:47:34.970 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:34.970 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.970 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-03-30T02:47:34.970 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:34.970 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:34.970 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:34.970 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:34.970 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:34.971 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:34.971 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:34.971 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:34.971 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:34.971 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:34.971 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:34.971 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:34.971 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:34.972 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:34.973 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:34.973 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:34.973 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:34.973 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:34.973 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:34.973 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:34.973 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:34.973 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:34.973 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:34.974 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-03-30T02:47:34.974 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:34.974 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:34.974 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:34.974 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:34.974 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 3844799, 2024-03-30T02:47:34.974 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 106140, 2024-03-30T02:47:34.974 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-03-30T02:47:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-03-30T02:47:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-03-30T02:47:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-03-30T02:47:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9656, 2024-03-30T02:47:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 203, 2024-03-30T02:47:34.976 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:34.976 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:34.976 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:34.976 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-03-30T02:47:34.976 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 306728065, 2024-03-30T02:47:34.976 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 3554857, 2024-03-30T02:47:34.976 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:34.976 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-03-30T02:47:34.976 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:34.977 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-03-30T02:47:34.977 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.977 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-03-30T02:47:34.977 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-03-30T02:47:34.977 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-03-30T02:47:34.977 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 3154, 2024-03-30T02:47:34.977 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 26, 2024-03-30T02:47:34.977 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 4, 2024-03-30T02:47:34.977 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 3154, 2024-03-30T02:47:34.978 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-03-30T02:47:34.978 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3154, 2024-03-30T02:47:34.978 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 29.931289978, 2024-03-30T02:47:34.978 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.009489946 2024-03-30T02:47:34.978 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.978 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-03-30T02:47:34.978 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-03-30T02:47:34.978 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3154, 2024-03-30T02:47:34.978 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3155 2024-03-30T02:47:34.979 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.979 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 3154, 2024-03-30T02:47:34.979 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 3154, 2024-03-30T02:47:34.979 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-03-30T02:47:34.979 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-03-30T02:47:34.979 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-03-30T02:47:34.979 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 1881, 2024-03-30T02:47:34.979 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-03-30T02:47:34.979 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 1270, 2024-03-30T02:47:34.980 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-03-30T02:47:34.980 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-03-30T02:47:34.980 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-03-30T02:47:34.980 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-03-30T02:47:34.980 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-03-30T02:47:34.980 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-03-30T02:47:34.980 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-03-30T02:47:34.980 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-03-30T02:47:34.981 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:34.981 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:34.981 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 2, 2024-03-30T02:47:34.981 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-03-30T02:47:34.981 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:34.981 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:34.981 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:34.981 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-03-30T02:47:34.981 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-03-30T02:47:34.982 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-03-30T02:47:34.982 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:34.982 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-03-30T02:47:34.982 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:34.982 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-03-30T02:47:34.982 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-03-30T02:47:34.982 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-03-30T02:47:34.982 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-03-30T02:47:34.982 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-03-30T02:47:34.983 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-03-30T02:47:34.983 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-03-30T02:47:34.983 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-03-30T02:47:34.983 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-03-30T02:47:34.983 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-03-30T02:47:34.983 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-03-30T02:47:34.983 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-03-30T02:47:34.983 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-03-30T02:47:34.984 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-03-30T02:47:34.984 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-03-30T02:47:34.984 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-03-30T02:47:34.984 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-03-30T02:47:34.984 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 96, 2024-03-30T02:47:34.984 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-03-30T02:47:34.984 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 13, 2024-03-30T02:47:34.984 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 11, 2024-03-30T02:47:34.984 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 11, 2024-03-30T02:47:34.985 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-03-30T02:47:34.985 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-03-30T02:47:34.985 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-03-30T02:47:34.985 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-03-30T02:47:34.985 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-03-30T02:47:34.985 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.985 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-03-30T02:47:34.985 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-03-30T02:47:34.985 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:34.986 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-03-30T02:47:34.986 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-03-30T02:47:34.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.986 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-03-30T02:47:34.986 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-03-30T02:47:34.986 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.986 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-03-30T02:47:34.986 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:34.986 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-03-30T02:47:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-03-30T02:47:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-03-30T02:47:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-03-30T02:47:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-30T02:47:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:34.988 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 3412, 2024-03-30T02:47:34.988 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 805500748, 2024-03-30T02:47:34.988 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:34.988 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 3412, 2024-03-30T02:47:34.988 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:34.988 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:34.988 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3412, 2024-03-30T02:47:34.988 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 805500748, 2024-03-30T02:47:34.989 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:34.989 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.989 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.989 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.989 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:34.989 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.989 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:34.989 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:34.989 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-30T02:47:34.990 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:34.990 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:34.990 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:34.990 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:34.990 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:34.990 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 3154, 2024-03-30T02:47:34.990 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 803022882, 2024-03-30T02:47:34.990 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3154, 2024-03-30T02:47:34.990 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 803022882, 2024-03-30T02:47:34.991 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:34.991 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.991 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.991 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.991 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:34.991 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.991 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:34.991 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:34.991 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-03-30T02:47:34.992 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:34.992 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:34.992 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:34.992 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:34.992 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:34.992 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 3154, 2024-03-30T02:47:34.992 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 3154, 2024-03-30T02:47:34.992 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3154, 2024-03-30T02:47:34.993 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 3154, 2024-03-30T02:47:34.993 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:34.993 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.993 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.993 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.993 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:34.993 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.993 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:34.993 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:34.994 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-30T02:47:34.994 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:34.994 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:34.994 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:34.994 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:34.994 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:34.994 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:34.994 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:34.994 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-03-30T02:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-03-30T02:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5651dd9b0640": { 2024-03-30T02:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:34.996 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-30T02:47:34.996 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:34.996 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:34.996 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:34.996 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:34.996 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:34.996 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:34.996 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:34.996 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-03-30T02:47:34.997 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-03-30T02:47:34.997 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:34.997 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:34.997 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:34.997 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:34.997 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:34.997 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:34.997 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-30T02:47:35.294 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-30T02:47:35.294 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-30T02:47:35.294 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-03-30T02:47:35.294 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-30T02:47:35.294 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-30T02:47:35.294 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-03-30T02:47:35.294 DEBUG:tasks.check_counter:Getting stats from i 2024-03-30T02:47:35.295 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.i perf dump 2024-03-30T02:47:35.517 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:47:36.521 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-30T02:47:36.521 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:36.521 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 36085, 2024-03-30T02:47:36.521 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 32335, 2024-03-30T02:47:36.522 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 16225124, 2024-03-30T02:47:36.522 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 22425609, 2024-03-30T02:47:36.522 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 15, 2024-03-30T02:47:36.522 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-03-30T02:47:36.522 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.772537180, 2024-03-30T02:47:36.523 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.576053075, 2024-03-30T02:47:36.523 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.789644741, 2024-03-30T02:47:36.523 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.466247968, 2024-03-30T02:47:36.523 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:36.523 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 32336, 2024-03-30T02:47:36.523 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.536721952, 2024-03-30T02:47:36.524 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000047523 2024-03-30T02:47:36.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.524 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:36.524 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 18145, 2024-03-30T02:47:36.524 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002861624, 2024-03-30T02:47:36.524 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000157 2024-03-30T02:47:36.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.525 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1979952, 2024-03-30T02:47:36.525 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 370704 2024-03-30T02:47:36.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.525 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:36.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 26724, 2024-03-30T02:47:36.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 27205, 2024-03-30T02:47:36.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 6830143, 2024-03-30T02:47:36.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 39699691, 2024-03-30T02:47:36.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 20, 2024-03-30T02:47:36.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-03-30T02:47:36.526 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.508252638, 2024-03-30T02:47:36.527 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.551432635, 2024-03-30T02:47:36.527 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.340848474, 2024-03-30T02:47:36.527 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.512237484, 2024-03-30T02:47:36.527 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:36.527 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 27202, 2024-03-30T02:47:36.527 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.053034758, 2024-03-30T02:47:36.527 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000038711 2024-03-30T02:47:36.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.528 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:36.528 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1153, 2024-03-30T02:47:36.528 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000230900, 2024-03-30T02:47:36.528 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000200 2024-03-30T02:47:36.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.528 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1232, 2024-03-30T02:47:36.528 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 430512 2024-03-30T02:47:36.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.528 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:36.529 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 35589, 2024-03-30T02:47:36.529 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 49839, 2024-03-30T02:47:36.529 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 11244055, 2024-03-30T02:47:36.529 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 32627668, 2024-03-30T02:47:36.529 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 31, 2024-03-30T02:47:36.529 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 8, 2024-03-30T02:47:36.529 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.854768454, 2024-03-30T02:47:36.530 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.727557116, 2024-03-30T02:47:36.530 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 2.283275222, 2024-03-30T02:47:36.530 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.284114781, 2024-03-30T02:47:36.530 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:36.530 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 49836, 2024-03-30T02:47:36.530 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8.533519391, 2024-03-30T02:47:36.530 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000171232 2024-03-30T02:47:36.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.530 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:36.531 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 47744, 2024-03-30T02:47:36.531 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.009230097, 2024-03-30T02:47:36.531 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000193 2024-03-30T02:47:36.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.531 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 6400, 2024-03-30T02:47:36.531 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 4672 2024-03-30T02:47:36.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.531 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-03-30T02:47:36.532 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-03-30T02:47:36.532 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-03-30T02:47:36.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.532 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-03-30T02:47:36.532 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-30T02:47:36.532 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-30T02:47:36.533 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 48627, 2024-03-30T02:47:36.533 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 18.219282673, 2024-03-30T02:47:36.533 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000374674 2024-03-30T02:47:36.533 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:36.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.534 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:36.534 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-30T02:47:36.534 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-30T02:47:36.534 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 92, 2024-03-30T02:47:36.534 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.003993000, 2024-03-30T02:47:36.534 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000043402 2024-03-30T02:47:36.534 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-03-30T02:47:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 5802, 2024-03-30T02:47:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 5728, 2024-03-30T02:47:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-03-30T02:47:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5728, 2024-03-30T02:47:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 6.950134947, 2024-03-30T02:47:36.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001213361 2024-03-30T02:47:36.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.536 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-03-30T02:47:36.536 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 74, 2024-03-30T02:47:36.536 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-03-30T02:47:36.536 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:36.536 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 182, 2024-03-30T02:47:36.536 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 19, 2024-03-30T02:47:36.536 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-03-30T02:47:36.537 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 12824, 2024-03-30T02:47:36.537 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 707, 2024-03-30T02:47:36.537 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 11560, 2024-03-30T02:47:36.537 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 557, 2024-03-30T02:47:36.537 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 11814, 2024-03-30T02:47:36.537 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 458, 2024-03-30T02:47:36.538 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 7876, 2024-03-30T02:47:36.538 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 7876, 2024-03-30T02:47:36.538 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 154, 2024-03-30T02:47:36.538 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 6276, 2024-03-30T02:47:36.538 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 5758, 2024-03-30T02:47:36.538 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 48, 2024-03-30T02:47:36.539 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 192, 2024-03-30T02:47:36.539 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:36.539 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:36.539 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-03-30T02:47:36.539 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-03-30T02:47:36.539 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-03-30T02:47:36.539 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 18, 2024-03-30T02:47:36.540 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 465, 2024-03-30T02:47:36.540 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 164, 2024-03-30T02:47:36.540 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 3649, 2024-03-30T02:47:36.540 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:36.540 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 63603, 2024-03-30T02:47:36.540 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:36.540 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 85384, 2024-03-30T02:47:36.540 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 1566089959, 2024-03-30T02:47:36.540 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-03-30T02:47:36.541 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 63603, 2024-03-30T02:47:36.541 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:36.541 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:36.541 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:36.541 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 3743, 2024-03-30T02:47:36.541 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T02:47:36.541 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 3743, 2024-03-30T02:47:36.541 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 59860, 2024-03-30T02:47:36.542 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 37, 2024-03-30T02:47:36.542 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 1206, 2024-03-30T02:47:36.542 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 5384, 2024-03-30T02:47:36.542 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 4, 2024-03-30T02:47:36.542 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:36.542 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:36.542 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 14556, 2024-03-30T02:47:36.542 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 13305, 2024-03-30T02:47:36.543 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 64, 2024-03-30T02:47:36.543 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 996 2024-03-30T02:47:36.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.543 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-03-30T02:47:36.543 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-03-30T02:47:36.543 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:36.543 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-03-30T02:47:36.543 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 187, 2024-03-30T02:47:36.544 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 26, 2024-03-30T02:47:36.544 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-03-30T02:47:36.544 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:36.544 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:36.544 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 24, 2024-03-30T02:47:36.544 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 24, 2024-03-30T02:47:36.545 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:36.545 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-03-30T02:47:36.545 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:36.545 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:36.545 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:36.545 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-03-30T02:47:36.545 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-03-30T02:47:36.545 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T02:47:36.545 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 18, 2024-03-30T02:47:36.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-03-30T02:47:36.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 20, 2024-03-30T02:47:36.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:36.546 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-03-30T02:47:36.546 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.546 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-03-30T02:47:36.546 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 20328, 2024-03-30T02:47:36.546 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-03-30T02:47:36.546 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-03-30T02:47:36.547 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 20328, 2024-03-30T02:47:36.547 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-03-30T02:47:36.547 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-03-30T02:47:36.547 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 30, 2024-03-30T02:47:36.547 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-03-30T02:47:36.547 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-03-30T02:47:36.547 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 30, 2024-03-30T02:47:36.547 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-03-30T02:47:36.547 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-03-30T02:47:36.548 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-03-30T02:47:36.548 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 63852700, 2024-03-30T02:47:36.548 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-03-30T02:47:36.548 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-03-30T02:47:36.548 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1071, 2024-03-30T02:47:36.548 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 28.243502012, 2024-03-30T02:47:36.548 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.026371150 2024-03-30T02:47:36.548 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.548 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-03-30T02:47:36.549 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.549 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-03-30T02:47:36.549 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 12767, 2024-03-30T02:47:36.549 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 13248, 2024-03-30T02:47:36.549 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 481, 2024-03-30T02:47:36.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 929, 2024-03-30T02:47:36.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 964, 2024-03-30T02:47:36.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 35, 2024-03-30T02:47:36.549 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 12824, 2024-03-30T02:47:36.550 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 13306, 2024-03-30T02:47:36.550 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 482, 2024-03-30T02:47:36.550 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 7876, 2024-03-30T02:47:36.550 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 10002, 2024-03-30T02:47:36.550 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 2126, 2024-03-30T02:47:36.550 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 127152, 2024-03-30T02:47:36.551 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 198964 2024-03-30T02:47:36.551 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.551 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-03-30T02:47:36.551 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 6359, 2024-03-30T02:47:36.552 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 3, 2024-03-30T02:47:36.552 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 5802, 2024-03-30T02:47:36.552 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 39, 2024-03-30T02:47:36.552 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 31, 2024-03-30T02:47:36.553 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-03-30T02:47:36.553 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4469, 2024-03-30T02:47:36.553 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.088685865, 2024-03-30T02:47:36.553 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000914899 2024-03-30T02:47:36.554 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.554 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-03-30T02:47:36.554 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2024-03-30T02:47:36.554 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000631621, 2024-03-30T02:47:36.555 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000315810 2024-03-30T02:47:36.555 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.555 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:36.555 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.555 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.556 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-03-30T02:47:36.556 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.557 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.557 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.557 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-03-30T02:47:36.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 207, 2024-03-30T02:47:36.558 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.070116931, 2024-03-30T02:47:36.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000338729 2024-03-30T02:47:36.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.559 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:36.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.559 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.560 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.560 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-03-30T02:47:36.560 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-03-30T02:47:36.560 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000102434, 2024-03-30T02:47:36.561 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000102434 2024-03-30T02:47:36.561 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.561 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-03-30T02:47:36.561 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.562 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.562 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.562 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.562 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:36.563 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.563 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.563 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.563 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.564 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:36.564 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.564 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.564 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.564 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.565 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-03-30T02:47:36.565 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.565 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.565 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.566 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.566 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-03-30T02:47:36.566 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 245, 2024-03-30T02:47:36.567 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.392212378, 2024-03-30T02:47:36.567 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001600866 2024-03-30T02:47:36.567 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.567 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-03-30T02:47:36.568 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.568 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.568 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.568 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.568 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-03-30T02:47:36.569 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.569 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.569 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.569 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.570 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-03-30T02:47:36.570 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 118, 2024-03-30T02:47:36.570 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.054137617, 2024-03-30T02:47:36.570 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000458793 2024-03-30T02:47:36.571 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.571 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-03-30T02:47:36.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 658, 2024-03-30T02:47:36.571 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.283299473, 2024-03-30T02:47:36.571 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003470059 2024-03-30T02:47:36.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.572 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-03-30T02:47:36.572 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.572 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.573 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.573 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:36.573 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.573 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.574 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:36.574 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.575 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.575 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.575 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.575 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-03-30T02:47:36.576 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.576 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.576 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.577 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:36.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.577 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.577 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.577 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.578 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:36.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.578 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.579 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-03-30T02:47:36.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4, 2024-03-30T02:47:36.579 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.016123391, 2024-03-30T02:47:36.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004030847 2024-03-30T02:47:36.580 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.580 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:36.580 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.581 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.581 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.581 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:36.581 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.582 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.582 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.582 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-03-30T02:47:36.583 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.583 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.583 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.584 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-03-30T02:47:36.584 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.584 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.584 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.584 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.584 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-03-30T02:47:36.585 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.585 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.585 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.585 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-03-30T02:47:36.585 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 24, 2024-03-30T02:47:36.585 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.044825237, 2024-03-30T02:47:36.585 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001867718 2024-03-30T02:47:36.585 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.586 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:36.586 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:36.586 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:36.586 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.586 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.586 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.586 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:36.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.586 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-03-30T02:47:36.587 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2024-03-30T02:47:36.587 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 4, 2024-03-30T02:47:36.587 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2024-03-30T02:47:36.587 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2024-03-30T02:47:36.587 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-03-30T02:47:36.587 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 385, 2024-03-30T02:47:36.587 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 385, 2024-03-30T02:47:36.587 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 629, 2024-03-30T02:47:36.587 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:36.588 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.588 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-03-30T02:47:36.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:36.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:36.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:36.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:36.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:36.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:36.588 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:36.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:36.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:36.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:36.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:36.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:36.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:36.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:36.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:36.589 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:36.590 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:36.590 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:36.590 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:36.590 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:36.590 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:36.590 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:36.590 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:36.590 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:36.591 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:36.591 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:36.591 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:36.591 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:36.591 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:36.591 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:36.591 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:36.591 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-03-30T02:47:36.592 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:36.592 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:36.592 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:36.592 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:36.592 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 583429, 2024-03-30T02:47:36.592 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 8259, 2024-03-30T02:47:36.592 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 880, 2024-03-30T02:47:36.592 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 10, 2024-03-30T02:47:36.592 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-03-30T02:47:36.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-03-30T02:47:36.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:36.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:36.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:36.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:36.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9656, 2024-03-30T02:47:36.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 203, 2024-03-30T02:47:36.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:36.593 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:36.594 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:36.594 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-03-30T02:47:36.594 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 42576232, 2024-03-30T02:47:36.594 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 418346, 2024-03-30T02:47:36.594 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:36.594 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-03-30T02:47:36.594 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:36.594 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-03-30T02:47:36.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.595 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-03-30T02:47:36.595 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-03-30T02:47:36.595 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-03-30T02:47:36.595 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 65096, 2024-03-30T02:47:36.595 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 63818108, 2024-03-30T02:47:36.595 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 27, 2024-03-30T02:47:36.595 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 65096, 2024-03-30T02:47:36.595 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-03-30T02:47:36.596 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 65096, 2024-03-30T02:47:36.596 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 72.388292613, 2024-03-30T02:47:36.596 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001112023 2024-03-30T02:47:36.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.596 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-03-30T02:47:36.596 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-03-30T02:47:36.596 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 65096, 2024-03-30T02:47:36.596 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 65411 2024-03-30T02:47:36.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.597 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 65096, 2024-03-30T02:47:36.597 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 63613, 2024-03-30T02:47:36.597 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 1483, 2024-03-30T02:47:36.597 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-03-30T02:47:36.597 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-03-30T02:47:36.597 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-03-30T02:47:36.597 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-03-30T02:47:36.597 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 10, 2024-03-30T02:47:36.597 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 1081, 2024-03-30T02:47:36.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 44, 2024-03-30T02:47:36.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-03-30T02:47:36.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-03-30T02:47:36.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-03-30T02:47:36.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 19, 2024-03-30T02:47:36.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 74, 2024-03-30T02:47:36.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-03-30T02:47:36.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:36.598 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 63603, 2024-03-30T02:47:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-03-30T02:47:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-03-30T02:47:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-03-30T02:47:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-03-30T02:47:36.599 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:36.600 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-03-30T02:47:36.600 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:36.600 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 4, 2024-03-30T02:47:36.600 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-03-30T02:47:36.600 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-03-30T02:47:36.600 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-03-30T02:47:36.600 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-03-30T02:47:36.600 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-03-30T02:47:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-03-30T02:47:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-03-30T02:47:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-03-30T02:47:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-03-30T02:47:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-03-30T02:47:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-03-30T02:47:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-03-30T02:47:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-03-30T02:47:36.601 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-03-30T02:47:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-03-30T02:47:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-03-30T02:47:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 96, 2024-03-30T02:47:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-03-30T02:47:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 13, 2024-03-30T02:47:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-03-30T02:47:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-03-30T02:47:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-03-30T02:47:36.602 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-03-30T02:47:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 477, 2024-03-30T02:47:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2024-03-30T02:47:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 69 2024-03-30T02:47:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-03-30T02:47:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-03-30T02:47:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:36.603 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-03-30T02:47:36.604 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-03-30T02:47:36.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.604 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-03-30T02:47:36.604 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 6149 2024-03-30T02:47:36.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.604 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-03-30T02:47:36.604 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:36.604 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 10, 2024-03-30T02:47:36.604 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-03-30T02:47:36.605 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 5, 2024-03-30T02:47:36.605 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 24, 2024-03-30T02:47:36.605 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:36.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.605 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:36.605 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:36.605 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-30T02:47:36.605 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:36.605 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 98394, 2024-03-30T02:47:36.606 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 25419645, 2024-03-30T02:47:36.606 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:36.606 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 98394, 2024-03-30T02:47:36.606 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:36.606 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:36.606 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 98394, 2024-03-30T02:47:36.606 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 25419645, 2024-03-30T02:47:36.606 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:36.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.607 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.607 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:36.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.607 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:36.607 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:36.607 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-30T02:47:36.607 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:36.607 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:36.608 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:36.608 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:36.608 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:36.608 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 65096, 2024-03-30T02:47:36.608 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 63820724, 2024-03-30T02:47:36.608 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 65096, 2024-03-30T02:47:36.608 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 63820724, 2024-03-30T02:47:36.608 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:36.608 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.609 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.609 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.609 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:36.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.609 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:36.609 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:36.609 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-03-30T02:47:36.609 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:36.609 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:36.610 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:36.610 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:36.610 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:36.610 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 65096, 2024-03-30T02:47:36.610 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 65096, 2024-03-30T02:47:36.610 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 65096, 2024-03-30T02:47:36.610 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 65096, 2024-03-30T02:47:36.610 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:36.611 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.611 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.611 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.611 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:36.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.611 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:36.611 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:36.611 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-30T02:47:36.611 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:36.612 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 24, 2024-03-30T02:47:36.612 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 2616, 2024-03-30T02:47:36.612 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:36.612 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 24, 2024-03-30T02:47:36.612 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:36.612 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:36.612 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 10, 2024-03-30T02:47:36.612 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2616, 2024-03-30T02:47:36.612 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:36.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.613 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.613 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:36.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:36.613 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5584c54ee0a0": { 2024-03-30T02:47:36.613 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:36.613 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-30T02:47:36.613 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:36.614 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 20328, 2024-03-30T02:47:36.614 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 59658396, 2024-03-30T02:47:36.614 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:36.614 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 20328, 2024-03-30T02:47:36.614 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:36.614 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:36.614 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1071, 2024-03-30T02:47:36.614 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 59658396, 2024-03-30T02:47:36.615 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:36.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:36.615 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:36.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:36.615 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:36.615 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:36.615 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-30T02:47:36.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:36 smithi170 ceph-mon[78798]: pgmap v327: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2024-03-30T02:47:36.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:36 smithi082 ceph-mon[79176]: pgmap v327: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2024-03-30T02:47:36.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:36 smithi067 ceph-mon[37808]: pgmap v327: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2024-03-30T02:47:36.918 INFO:tasks.check_counter:Daemon mds.i mds.exported=18 2024-03-30T02:47:36.919 INFO:tasks.check_counter:Daemon mds.i mds.imported=164 2024-03-30T02:47:36.919 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-03-30T02:47:36.919 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-30T02:47:36.919 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-30T02:47:36.919 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=19 2024-03-30T02:47:36.919 DEBUG:tasks.check_counter:Getting stats from l 2024-03-30T02:47:36.920 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:470100a4ccbcf7769052bf09a5413881ccf1b97f shell --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 -- ceph daemon mds.l perf dump 2024-03-30T02:47:37.139 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/mon.c/config 2024-03-30T02:47:38.179 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-03-30T02:47:38.179 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-03-30T02:47:38.179 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1090, 2024-03-30T02:47:38.179 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1090, 2024-03-30T02:47:38.180 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 4539643, 2024-03-30T02:47:38.180 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 338195, 2024-03-30T02:47:38.180 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 6, 2024-03-30T02:47:38.180 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 2, 2024-03-30T02:47:38.180 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.107933825, 2024-03-30T02:47:38.180 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.033117934, 2024-03-30T02:47:38.180 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.029412558, 2024-03-30T02:47:38.180 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.034226709, 2024-03-30T02:47:38.181 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:38.181 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1089, 2024-03-30T02:47:38.181 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.071729430, 2024-03-30T02:47:38.181 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000065867 2024-03-30T02:47:38.181 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.181 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:38.181 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.182 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.182 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.182 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-03-30T02:47:38.182 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448 2024-03-30T02:47:38.182 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.182 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-03-30T02:47:38.182 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 680, 2024-03-30T02:47:38.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 841, 2024-03-30T02:47:38.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 4368796, 2024-03-30T02:47:38.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 637070, 2024-03-30T02:47:38.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 7, 2024-03-30T02:47:38.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-03-30T02:47:38.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.071530245, 2024-03-30T02:47:38.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.025905351, 2024-03-30T02:47:38.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.017080163, 2024-03-30T02:47:38.184 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.021254660, 2024-03-30T02:47:38.184 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:38.184 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 838, 2024-03-30T02:47:38.184 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.092817155, 2024-03-30T02:47:38.184 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000110760 2024-03-30T02:47:38.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:38.185 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.185 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.185 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-30T02:47:38.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 426144 2024-03-30T02:47:38.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.186 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-03-30T02:47:38.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 756, 2024-03-30T02:47:38.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 730, 2024-03-30T02:47:38.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 5131745, 2024-03-30T02:47:38.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 280126, 2024-03-30T02:47:38.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 4, 2024-03-30T02:47:38.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-03-30T02:47:38.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.089144499, 2024-03-30T02:47:38.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.024214820, 2024-03-30T02:47:38.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.057079972, 2024-03-30T02:47:38.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.017531397, 2024-03-30T02:47:38.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-03-30T02:47:38.187 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 729, 2024-03-30T02:47:38.187 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.069845277, 2024-03-30T02:47:38.187 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000095809 2024-03-30T02:47:38.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.188 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-03-30T02:47:38.188 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.188 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.188 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.188 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1936240, 2024-03-30T02:47:38.189 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 125248 2024-03-30T02:47:38.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.189 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-03-30T02:47:38.189 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-03-30T02:47:38.189 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-03-30T02:47:38.189 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.189 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-03-30T02:47:38.189 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-30T02:47:38.190 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-30T02:47:38.190 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3825, 2024-03-30T02:47:38.190 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.105909087, 2024-03-30T02:47:38.190 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000027688 2024-03-30T02:47:38.190 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:38.190 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.190 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-03-30T02:47:38.190 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-03-30T02:47:38.190 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-03-30T02:47:38.191 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.191 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.191 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.191 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:38.191 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.191 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-03-30T02:47:38.192 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-03-30T02:47:38.192 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-03-30T02:47:38.192 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-03-30T02:47:38.192 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.192 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.192 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.192 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-03-30T02:47:38.193 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-03-30T02:47:38.193 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-03-30T02:47:38.193 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-03-30T02:47:38.193 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-03-30T02:47:38.193 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-03-30T02:47:38.193 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-03-30T02:47:38.193 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 3816, 2024-03-30T02:47:38.193 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 0, 2024-03-30T02:47:38.193 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 3816, 2024-03-30T02:47:38.194 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-03-30T02:47:38.194 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 3816, 2024-03-30T02:47:38.194 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-03-30T02:47:38.194 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-03-30T02:47:38.194 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-03-30T02:47:38.194 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 148, 2024-03-30T02:47:38.194 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-03-30T02:47:38.194 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-03-30T02:47:38.194 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-03-30T02:47:38.195 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-03-30T02:47:38.195 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-03-30T02:47:38.195 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-03-30T02:47:38.195 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-03-30T02:47:38.195 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-03-30T02:47:38.195 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-03-30T02:47:38.195 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-03-30T02:47:38.195 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-03-30T02:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-03-30T02:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-03-30T02:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-03-30T02:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-03-30T02:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-03-30T02:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 0, 2024-03-30T02:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 0, 2024-03-30T02:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2024-03-30T02:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-03-30T02:47:38.197 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-03-30T02:47:38.197 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-03-30T02:47:38.197 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-03-30T02:47:38.197 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-03-30T02:47:38.197 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-03-30T02:47:38.197 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-03-30T02:47:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-03-30T02:47:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-03-30T02:47:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-03-30T02:47:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-03-30T02:47:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-03-30T02:47:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-30T02:47:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-30T02:47:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-03-30T02:47:38.198 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-03-30T02:47:38.199 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-03-30T02:47:38.199 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-03-30T02:47:38.199 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.199 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-03-30T02:47:38.199 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-03-30T02:47:38.199 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-03-30T02:47:38.199 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-03-30T02:47:38.199 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-03-30T02:47:38.199 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-03-30T02:47:38.200 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-03-30T02:47:38.200 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-03-30T02:47:38.200 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-03-30T02:47:38.200 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-03-30T02:47:38.200 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-03-30T02:47:38.200 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-03-30T02:47:38.200 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-03-30T02:47:38.200 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-03-30T02:47:38.200 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-03-30T02:47:38.201 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-03-30T02:47:38.201 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-03-30T02:47:38.201 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-03-30T02:47:38.201 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-03-30T02:47:38.201 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-03-30T02:47:38.201 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-03-30T02:47:38.201 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-03-30T02:47:38.201 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-03-30T02:47:38.202 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-03-30T02:47:38.202 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.202 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-03-30T02:47:38.202 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-03-30T02:47:38.202 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-03-30T02:47:38.202 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-03-30T02:47:38.202 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 4995, 2024-03-30T02:47:38.202 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-03-30T02:47:38.203 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-03-30T02:47:38.203 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-03-30T02:47:38.203 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-03-30T02:47:38.203 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-03-30T02:47:38.203 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 6, 2024-03-30T02:47:38.203 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-03-30T02:47:38.203 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-03-30T02:47:38.203 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-03-30T02:47:38.203 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 15650191, 2024-03-30T02:47:38.204 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 15649617, 2024-03-30T02:47:38.204 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-03-30T02:47:38.204 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.204 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.204 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.204 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 4995 2024-03-30T02:47:38.204 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.204 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-03-30T02:47:38.205 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 3701, 2024-03-30T02:47:38.205 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 3760, 2024-03-30T02:47:38.205 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 59, 2024-03-30T02:47:38.205 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 366, 2024-03-30T02:47:38.205 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 386, 2024-03-30T02:47:38.205 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 20, 2024-03-30T02:47:38.205 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 3816, 2024-03-30T02:47:38.205 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 3816, 2024-03-30T02:47:38.205 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 0, 2024-03-30T02:47:38.206 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-03-30T02:47:38.206 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-03-30T02:47:38.206 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-03-30T02:47:38.206 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 63724, 2024-03-30T02:47:38.206 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 207156 2024-03-30T02:47:38.206 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.206 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-03-30T02:47:38.206 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-03-30T02:47:38.207 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-03-30T02:47:38.207 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-03-30T02:47:38.207 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-03-30T02:47:38.207 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-03-30T02:47:38.207 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-03-30T02:47:38.207 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.207 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.207 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.208 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-03-30T02:47:38.208 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.208 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.208 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.208 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.208 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-03-30T02:47:38.208 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.209 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.209 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.209 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-03-30T02:47:38.209 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.209 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.209 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.209 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.209 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-03-30T02:47:38.210 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.210 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.210 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.210 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.210 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-03-30T02:47:38.210 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.210 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.211 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.211 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-03-30T02:47:38.211 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.211 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.211 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.211 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.211 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-03-30T02:47:38.211 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.212 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.212 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.212 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-03-30T02:47:38.212 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.212 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.212 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.212 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.213 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-03-30T02:47:38.213 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.213 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.213 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.213 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.213 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-03-30T02:47:38.213 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.213 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.213 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.214 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-03-30T02:47:38.214 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.214 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.214 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.214 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.214 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-03-30T02:47:38.215 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.215 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.215 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.215 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-03-30T02:47:38.215 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.215 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.215 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.215 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.216 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-03-30T02:47:38.216 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.216 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.216 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.216 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.216 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-03-30T02:47:38.216 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.216 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.217 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.217 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-03-30T02:47:38.217 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.217 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.217 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.217 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.217 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-03-30T02:47:38.217 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.218 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.218 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.218 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-03-30T02:47:38.218 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.218 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.218 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.218 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.218 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-03-30T02:47:38.219 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.219 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.219 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.219 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-03-30T02:47:38.219 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.219 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.219 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.219 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.220 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-03-30T02:47:38.220 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.220 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.220 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.220 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.220 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-03-30T02:47:38.220 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.220 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.220 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.221 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-03-30T02:47:38.221 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.221 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.221 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.221 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.221 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-03-30T02:47:38.221 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.221 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.222 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.222 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.222 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-03-30T02:47:38.222 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.222 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.223 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.223 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.223 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-03-30T02:47:38.223 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.223 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.223 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.224 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-03-30T02:47:38.224 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.224 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.224 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.224 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.224 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-03-30T02:47:38.225 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.225 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.225 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.225 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.225 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-03-30T02:47:38.225 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-03-30T02:47:38.225 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-03-30T02:47:38.226 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.226 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.226 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.226 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:38.226 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.226 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-03-30T02:47:38.227 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2024-03-30T02:47:38.227 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 3, 2024-03-30T02:47:38.227 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 2, 2024-03-30T02:47:38.227 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-03-30T02:47:38.227 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-03-30T02:47:38.227 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-03-30T02:47:38.227 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-03-30T02:47:38.227 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 8330, 2024-03-30T02:47:38.228 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-03-30T02:47:38.228 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.228 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-03-30T02:47:38.228 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-03-30T02:47:38.228 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-03-30T02:47:38.228 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-03-30T02:47:38.228 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-03-30T02:47:38.228 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-03-30T02:47:38.229 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-03-30T02:47:38.229 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-30T02:47:38.229 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-03-30T02:47:38.229 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-30T02:47:38.229 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-03-30T02:47:38.229 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-03-30T02:47:38.229 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-03-30T02:47:38.229 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-30T02:47:38.230 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-03-30T02:47:38.230 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-03-30T02:47:38.230 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-03-30T02:47:38.230 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-03-30T02:47:38.230 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-03-30T02:47:38.230 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-30T02:47:38.230 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-03-30T02:47:38.231 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-30T02:47:38.231 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-03-30T02:47:38.231 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-03-30T02:47:38.231 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-03-30T02:47:38.231 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-03-30T02:47:38.231 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-03-30T02:47:38.231 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-30T02:47:38.231 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-03-30T02:47:38.231 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-03-30T02:47:38.232 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-03-30T02:47:38.232 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-03-30T02:47:38.232 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-03-30T02:47:38.232 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-03-30T02:47:38.232 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-03-30T02:47:38.232 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-03-30T02:47:38.232 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-03-30T02:47:38.233 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 327531, 2024-03-30T02:47:38.233 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 3859, 2024-03-30T02:47:38.233 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-03-30T02:47:38.233 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-03-30T02:47:38.233 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-03-30T02:47:38.233 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-03-30T02:47:38.233 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-03-30T02:47:38.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-03-30T02:47:38.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-03-30T02:47:38.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-03-30T02:47:38.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9656, 2024-03-30T02:47:38.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 203, 2024-03-30T02:47:38.234 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-03-30T02:47:38.235 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-03-30T02:47:38.235 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-03-30T02:47:38.235 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-03-30T02:47:38.235 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 11674181, 2024-03-30T02:47:38.235 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 138400, 2024-03-30T02:47:38.235 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-03-30T02:47:38.235 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-03-30T02:47:38.236 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-03-30T02:47:38.236 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-03-30T02:47:38.236 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.236 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-03-30T02:47:38.236 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-03-30T02:47:38.236 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-03-30T02:47:38.236 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 2262, 2024-03-30T02:47:38.237 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 21, 2024-03-30T02:47:38.237 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 3, 2024-03-30T02:47:38.237 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 2262, 2024-03-30T02:47:38.237 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-03-30T02:47:38.237 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2262, 2024-03-30T02:47:38.237 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 15.835624813, 2024-03-30T02:47:38.237 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.007000718 2024-03-30T02:47:38.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.238 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-03-30T02:47:38.238 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-03-30T02:47:38.238 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2262, 2024-03-30T02:47:38.238 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2263 2024-03-30T02:47:38.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.239 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 2262, 2024-03-30T02:47:38.239 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 2262, 2024-03-30T02:47:38.239 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-03-30T02:47:38.239 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-03-30T02:47:38.239 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-03-30T02:47:38.239 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 1368, 2024-03-30T02:47:38.239 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-03-30T02:47:38.240 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 892, 2024-03-30T02:47:38.240 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-03-30T02:47:38.240 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-03-30T02:47:38.240 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-03-30T02:47:38.240 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-03-30T02:47:38.240 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-03-30T02:47:38.240 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-03-30T02:47:38.241 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-03-30T02:47:38.241 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-03-30T02:47:38.241 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-03-30T02:47:38.241 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-03-30T02:47:38.241 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 1, 2024-03-30T02:47:38.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-03-30T02:47:38.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-03-30T02:47:38.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-03-30T02:47:38.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-03-30T02:47:38.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-03-30T02:47:38.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-03-30T02:47:38.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-03-30T02:47:38.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-03-30T02:47:38.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-03-30T02:47:38.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-03-30T02:47:38.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-03-30T02:47:38.243 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-03-30T02:47:38.243 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-03-30T02:47:38.243 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-03-30T02:47:38.243 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-03-30T02:47:38.243 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-03-30T02:47:38.244 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-03-30T02:47:38.244 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-03-30T02:47:38.244 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-03-30T02:47:38.244 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-03-30T02:47:38.244 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-03-30T02:47:38.244 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-03-30T02:47:38.244 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-03-30T02:47:38.244 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-03-30T02:47:38.245 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-03-30T02:47:38.245 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-03-30T02:47:38.245 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-03-30T02:47:38.245 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 96, 2024-03-30T02:47:38.245 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-03-30T02:47:38.245 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 13, 2024-03-30T02:47:38.245 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 6, 2024-03-30T02:47:38.246 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 6, 2024-03-30T02:47:38.246 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-03-30T02:47:38.246 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-03-30T02:47:38.246 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-03-30T02:47:38.246 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-03-30T02:47:38.246 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-03-30T02:47:38.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.246 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-03-30T02:47:38.247 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-03-30T02:47:38.247 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-03-30T02:47:38.247 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-03-30T02:47:38.247 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-03-30T02:47:38.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.247 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-03-30T02:47:38.247 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-03-30T02:47:38.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-30T02:47:38.248 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:38.249 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-30T02:47:38.249 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:38.249 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2522, 2024-03-30T02:47:38.249 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 13797106, 2024-03-30T02:47:38.249 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:38.249 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2522, 2024-03-30T02:47:38.249 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:38.249 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:38.249 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2522, 2024-03-30T02:47:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 13797106, 2024-03-30T02:47:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2262, 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 11466364, 2024-03-30T02:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2262, 2024-03-30T02:47:38.252 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 11466364, 2024-03-30T02:47:38.252 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:38.252 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.252 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.252 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:38.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.252 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-03-30T02:47:38.252 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2262, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 2262, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2262, 2024-03-30T02:47:38.253 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2262, 2024-03-30T02:47:38.254 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:38.254 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.254 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.254 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.254 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:38.254 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.254 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-03-30T02:47:38.254 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:38.254 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-30T02:47:38.255 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:38.255 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:38.255 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:38.255 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:38.255 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:38.255 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:38.255 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:38.255 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-03-30T02:47:38.256 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-03-30T02:47:38.256 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:38.256 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.256 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.256 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.256 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:38.256 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-03-30T02:47:38.256 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5587f21ca500": { 2024-03-30T02:47:38.256 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-03-30T02:47:38.257 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-03-30T02:47:38.257 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-03-30T02:47:38.257 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-03-30T02:47:38.257 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-03-30T02:47:38.257 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-03-30T02:47:38.257 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-03-30T02:47:38.257 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-03-30T02:47:38.257 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-03-30T02:47:38.257 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-03-30T02:47:38.258 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-03-30T02:47:38.258 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-03-30T02:47:38.258 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-03-30T02:47:38.258 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-03-30T02:47:38.258 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-03-30T02:47:38.258 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:38.258 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-03-30T02:47:38.258 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-03-30T02:47:38.551 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-30T02:47:38.552 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-30T02:47:38.552 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-03-30T02:47:38.552 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-03-30T02:47:38.552 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-03-30T02:47:38.552 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-03-30T02:47:38.552 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-30T02:47:38.563 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-30T02:47:38.563 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:47:38.563 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T02:47:38.613 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-03-30T02:47:38.613 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T02:47:38.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:47:38.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:38 smithi170 ceph-mon[78798]: pgmap v328: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-03-30T02:47:38.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:38 smithi082 ceph-mon[79176]: pgmap v328: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-03-30T02:47:38.760 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:38 smithi067 ceph-mon[37808]: pgmap v328: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-03-30T02:47:38.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.908+0000 7f3d66600700 1 -- 172.21.15.67:0/553350793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 msgr2=0x7f3d60101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:38.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.908+0000 7f3d66600700 1 --2- 172.21.15.67:0/553350793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 0x7f3d60101240 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3d500098f0 tx=0x7f3d50009c00 comp rx=0 tx=0).stop 2024-03-30T02:47:38.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.908+0000 7f3d66600700 1 -- 172.21.15.67:0/553350793 shutdown_connections 2024-03-30T02:47:38.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.908+0000 7f3d66600700 1 --2- 172.21.15.67:0/553350793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 0x7f3d60101240 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:38.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.908+0000 7f3d66600700 1 --2- 172.21.15.67:0/553350793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d60105f80 0x7f3d6010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:38.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.908+0000 7f3d66600700 1 --2- 172.21.15.67:0/553350793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d60101810 0x7f3d60105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:38.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.908+0000 7f3d66600700 1 -- 172.21.15.67:0/553350793 >> 172.21.15.67:0/553350793 conn(0x7f3d600fc700 msgr2=0x7f3d600feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:38.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.908+0000 7f3d66600700 1 -- 172.21.15.67:0/553350793 shutdown_connections 2024-03-30T02:47:38.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.908+0000 7f3d66600700 1 -- 172.21.15.67:0/553350793 wait complete. 2024-03-30T02:47:38.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.909+0000 7f3d66600700 1 Processor -- start 2024-03-30T02:47:38.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.909+0000 7f3d66600700 1 -- start start 2024-03-30T02:47:38.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d66600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 0x7f3d60197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:38.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d66600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d60105f80 0x7f3d60197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:38.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d66600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6019ba90 0x7f3d6019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:38.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d66600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6019c510 con 0x7f3d60100e60 2024-03-30T02:47:38.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d66600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6019c680 con 0x7f3d6019ba90 2024-03-30T02:47:38.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d66600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6019c840 con 0x7f3d60105f80 2024-03-30T02:47:38.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d5f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 0x7f3d60197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:38.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d5f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 0x7f3d60197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38082/0 (socket says 172.21.15.67:38082) 2024-03-30T02:47:38.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d5f400700 1 -- 172.21.15.67:0/1547114851 learned_addr learned my addr 172.21.15.67:0/1547114851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:38.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d5fe00700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6019ba90 0x7f3d6019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d5ea00700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d60105f80 0x7f3d60197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.910+0000 7f3d5f400700 1 -- 172.21.15.67:0/1547114851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d60105f80 msgr2=0x7f3d60197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d5f400700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d60105f80 0x7f3d60197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d5f400700 1 -- 172.21.15.67:0/1547114851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6019ba90 msgr2=0x7f3d6019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d5f400700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6019ba90 0x7f3d6019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d5f400700 1 -- 172.21.15.67:0/1547114851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0095f0 con 0x7f3d60100e60 2024-03-30T02:47:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d5fe00700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6019ba90 0x7f3d6019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:38.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d5f400700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 0x7f3d60197200 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3d50005950 tx=0x7f3d5000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:38.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d57e00700 1 -- 172.21.15.67:0/1547114851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5001c070 con 0x7f3d60100e60 2024-03-30T02:47:38.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d50009520 con 0x7f3d60100e60 2024-03-30T02:47:38.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d57e00700 1 -- 172.21.15.67:0/1547114851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5000b9c0 con 0x7f3d60100e60 2024-03-30T02:47:38.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d57e00700 1 -- 172.21.15.67:0/1547114851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500038e0 con 0x7f3d60100e60 2024-03-30T02:47:38.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.911+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d601a3fe0 con 0x7f3d60100e60 2024-03-30T02:47:38.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.912+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d601091f0 con 0x7f3d60100e60 2024-03-30T02:47:38.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.914+0000 7f3d57e00700 1 -- 172.21.15.67:0/1547114851 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f3d50020030 con 0x7f3d60100e60 2024-03-30T02:47:38.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.915+0000 7f3d57e00700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3d440a1290 0x7f3d440a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:38.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.915+0000 7f3d57e00700 1 -- 172.21.15.67:0/1547114851 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f3d500cde00 con 0x7f3d60100e60 2024-03-30T02:47:38.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.916+0000 7f3d5ea00700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3d440a1290 0x7f3d440a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:38.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.916+0000 7f3d5ea00700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3d440a1290 0x7f3d440a3750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d60198700 tx=0x7f3d4c009310 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:38.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:38.920+0000 7f3d57e00700 1 -- 172.21.15.67:0/1547114851 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f3d50096ee0 con 0x7f3d60100e60 2024-03-30T02:47:39.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.182+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d60102330 con 0x7f3d60100e60 2024-03-30T02:47:39.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.185+0000 7f3d57e00700 1 -- 172.21.15.67:0/1547114851 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+25099 (secure 0 0 0) 0x7f3d5009a500 con 0x7f3d60100e60 2024-03-30T02:47:39.185 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:47:39.185 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":96,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:47:16.442983+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/759150839":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/3018284010":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/3801648476":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/4226270132":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1220244664":"2024-03-31T02:37:02.362975+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/3801648476":"2024-03-31T02:37:02.362975+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2449045224":"2024-03-31T02:37:02.362975+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:47:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3d440a1290 msgr2=0x7f3d440a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3d440a1290 0x7f3d440a3750 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d60198700 tx=0x7f3d4c009310 comp rx=0 tx=0).stop 2024-03-30T02:47:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 msgr2=0x7f3d60197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 0x7f3d60197200 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3d50005950 tx=0x7f3d5000fa00 comp rx=0 tx=0).stop 2024-03-30T02:47:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 shutdown_connections 2024-03-30T02:47:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f3d440a1290 0x7f3d440a3750 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d60100e60 0x7f3d60197200 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6019ba90 0x7f3d6019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 --2- 172.21.15.67:0/1547114851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d60105f80 0x7f3d60197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 >> 172.21.15.67:0/1547114851 conn(0x7f3d600fc700 msgr2=0x7f3d600fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 shutdown_connections 2024-03-30T02:47:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.187+0000 7f3d66600700 1 -- 172.21.15.67:0/1547114851 wait complete. 2024-03-30T02:47:39.207 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T02:31:23.541523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T02:34:45.007923+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T02:34:46.178111+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T02:34:49.253894+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 95, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T02:47:39.207 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T02:47:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.411+0000 7f0ad0c00700 1 -- 172.21.15.67:0/3127567447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc106080 msgr2=0x7f0acc10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.411+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/3127567447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc106080 0x7f0acc10dc00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400b0a0 tx=0x7f0ac400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.411+0000 7f0ad0c00700 1 -- 172.21.15.67:0/3127567447 shutdown_connections 2024-03-30T02:47:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.411+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/3127567447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc106080 0x7f0acc10dc00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.411+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/3127567447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0acc100f60 0x7f0acc101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.411+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/3127567447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0acc101910 0x7f0acc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.411+0000 7f0ad0c00700 1 -- 172.21.15.67:0/3127567447 >> 172.21.15.67:0/3127567447 conn(0x7f0acc0fc7c0 msgr2=0x7f0acc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.411+0000 7f0ad0c00700 1 -- 172.21.15.67:0/3127567447 shutdown_connections 2024-03-30T02:47:39.413 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.411+0000 7f0ad0c00700 1 -- 172.21.15.67:0/3127567447 wait complete. 2024-03-30T02:47:39.414 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.412+0000 7f0ad0c00700 1 Processor -- start 2024-03-30T02:47:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ad0c00700 1 -- start start 2024-03-30T02:47:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ad0c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc100f60 0x7f0acc1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ad0c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0acc106080 0x7f0acc1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ad0c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0acc19bc30 0x7f0acc19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ad0c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc19c6b0 con 0x7f0acc100f60 2024-03-30T02:47:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ad0c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc19c820 con 0x7f0acc19bc30 2024-03-30T02:47:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ad0c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc19c9e0 con 0x7f0acc106080 2024-03-30T02:47:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ac9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc100f60 0x7f0acc1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:39.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ac9e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc100f60 0x7f0acc1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38112/0 (socket says 172.21.15.67:38112) 2024-03-30T02:47:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ac9e00700 1 -- 172.21.15.67:0/4125486360 learned_addr learned my addr 172.21.15.67:0/4125486360 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0aca800700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0acc19bc30 0x7f0acc19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.413+0000 7f0ac9400700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0acc106080 0x7f0acc1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ac9e00700 1 -- 172.21.15.67:0/4125486360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0acc106080 msgr2=0x7f0acc1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ac9e00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0acc106080 0x7f0acc1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ac9e00700 1 -- 172.21.15.67:0/4125486360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0acc19bc30 msgr2=0x7f0acc19c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ac9e00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0acc19bc30 0x7f0acc19c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ac9e00700 1 -- 172.21.15.67:0/4125486360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8009520 con 0x7f0acc100f60 2024-03-30T02:47:39.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ac9e00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc100f60 0x7f0acc1973a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00e9a0 tx=0x7f0abc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:39.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ac2a00700 1 -- 172.21.15.67:0/4125486360 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc009790 con 0x7f0acc100f60 2024-03-30T02:47:39.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ac2a00700 1 -- 172.21.15.67:0/4125486360 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0abc010430 con 0x7f0acc100f60 2024-03-30T02:47:39.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac4009d00 con 0x7f0acc100f60 2024-03-30T02:47:39.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ac2a00700 1 -- 172.21.15.67:0/4125486360 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc018680 con 0x7f0acc100f60 2024-03-30T02:47:39.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.414+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0acc1a4280 con 0x7f0acc100f60 2024-03-30T02:47:39.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.415+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0acc109350 con 0x7f0acc100f60 2024-03-30T02:47:39.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.417+0000 7f0ac2a00700 1 -- 172.21.15.67:0/4125486360 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0abc0105a0 con 0x7f0acc100f60 2024-03-30T02:47:39.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.419+0000 7f0ac2a00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0ab00a1310 0x7f0ab00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:39.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.419+0000 7f0ac2a00700 1 -- 172.21.15.67:0/4125486360 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f0abc014070 con 0x7f0acc100f60 2024-03-30T02:47:39.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.419+0000 7f0ac9400700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0ab00a1310 0x7f0ab00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:39.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.419+0000 7f0ac9400700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0ab00a1310 0x7f0ab00a37d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0acc1988a0 tx=0x7f0ab8009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:39.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.420+0000 7f0ac2a00700 1 -- 172.21.15.67:0/4125486360 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0abc0964e0 con 0x7f0acc100f60 2024-03-30T02:47:39.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.639+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0acc198650 con 0x7f0acc100f60 2024-03-30T02:47:39.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.640+0000 7f0ac2a00700 1 -- 172.21.15.67:0/4125486360 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0abc026070 con 0x7f0acc100f60 2024-03-30T02:47:39.640 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-03-30T02:47:39.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.642+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0ab00a1310 msgr2=0x7f0ab00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.642+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0ab00a1310 0x7f0ab00a37d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0acc1988a0 tx=0x7f0ab8009260 comp rx=0 tx=0).stop 2024-03-30T02:47:39.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.642+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc100f60 msgr2=0x7f0acc1973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.642+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc100f60 0x7f0acc1973a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0abc00e9a0 tx=0x7f0abc00ecb0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.643+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 shutdown_connections 2024-03-30T02:47:39.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.643+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0ab00a1310 0x7f0ab00a37d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.643+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0acc100f60 0x7f0acc1973a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.643+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0acc19bc30 0x7f0acc19c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.643+0000 7f0ad0c00700 1 --2- 172.21.15.67:0/4125486360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0acc106080 0x7f0acc1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.643+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 >> 172.21.15.67:0/4125486360 conn(0x7f0acc0fc7c0 msgr2=0x7f0acc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:39.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.643+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 shutdown_connections 2024-03-30T02:47:39.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.643+0000 7f0ad0c00700 1 -- 172.21.15.67:0/4125486360 wait complete. 2024-03-30T02:47:39.662 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T02:47:39.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:39 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1547114851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:47:39.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.865+0000 7fe86a800700 1 -- 172.21.15.67:0/2386660385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 msgr2=0x7fe864073560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.865+0000 7fe86a800700 1 --2- 172.21.15.67:0/2386660385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 0x7fe864073560 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe854009990 tx=0x7fe854009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.865+0000 7fe86a800700 1 -- 172.21.15.67:0/2386660385 shutdown_connections 2024-03-30T02:47:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.865+0000 7fe86a800700 1 --2- 172.21.15.67:0/2386660385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 0x7fe864073560 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.865+0000 7fe86a800700 1 --2- 172.21.15.67:0/2386660385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe86410d5a0 0x7fe86410f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.865+0000 7fe86a800700 1 --2- 172.21.15.67:0/2386660385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe864073aa0 0x7fe86410d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.865+0000 7fe86a800700 1 -- 172.21.15.67:0/2386660385 >> 172.21.15.67:0/2386660385 conn(0x7fe8640fc7f0 msgr2=0x7fe8640fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.865+0000 7fe86a800700 1 -- 172.21.15.67:0/2386660385 shutdown_connections 2024-03-30T02:47:39.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.865+0000 7fe86a800700 1 -- 172.21.15.67:0/2386660385 wait complete. 2024-03-30T02:47:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.866+0000 7fe86a800700 1 Processor -- start 2024-03-30T02:47:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.866+0000 7fe86a800700 1 -- start start 2024-03-30T02:47:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe86a800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 0x7fe8641993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe86a800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe864073aa0 0x7fe864199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe86a800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe86410d5a0 0x7fe86419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe86a800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86419e300 con 0x7fe864073180 2024-03-30T02:47:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe86a800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86419e470 con 0x7fe86410d5a0 2024-03-30T02:47:39.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe86a800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86419e630 con 0x7fe864073aa0 2024-03-30T02:47:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 0x7fe8641993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 0x7fe8641993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38130/0 (socket says 172.21.15.67:38130) 2024-03-30T02:47:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863400700 1 -- 172.21.15.67:0/771402948 learned_addr learned my addr 172.21.15.67:0/771402948 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe862a00700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe864073aa0 0x7fe864199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863e00700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe86410d5a0 0x7fe86419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863400700 1 -- 172.21.15.67:0/771402948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe864073aa0 msgr2=0x7fe864199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863400700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe864073aa0 0x7fe864199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863400700 1 -- 172.21.15.67:0/771402948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe86410d5a0 msgr2=0x7fe86419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:39.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863400700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe86410d5a0 0x7fe86419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863400700 1 -- 172.21.15.67:0/771402948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8540095f0 con 0x7fe864073180 2024-03-30T02:47:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe862a00700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe864073aa0 0x7fe864199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:47:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.867+0000 7fe863e00700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe86410d5a0 0x7fe86419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:47:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.868+0000 7fe863400700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 0x7fe8641993e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe854005910 tx=0x7fe854009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.868+0000 7fe85be00700 1 -- 172.21.15.67:0/771402948 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85401d070 con 0x7fe864073180 2024-03-30T02:47:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.868+0000 7fe85be00700 1 -- 172.21.15.67:0/771402948 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe854022470 con 0x7fe864073180 2024-03-30T02:47:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.868+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86419e8b0 con 0x7fe864073180 2024-03-30T02:47:39.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.868+0000 7fe85be00700 1 -- 172.21.15.67:0/771402948 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854017650 con 0x7fe864073180 2024-03-30T02:47:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.868+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8641a6150 con 0x7fe864073180 2024-03-30T02:47:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.869+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8641087b0 con 0x7fe864073180 2024-03-30T02:47:39.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.873+0000 7fe85be00700 1 -- 172.21.15.67:0/771402948 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fe8540225e0 con 0x7fe864073180 2024-03-30T02:47:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.874+0000 7fe85be00700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe8480a1290 0x7fe8480a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.874+0000 7fe85be00700 1 -- 172.21.15.67:0/771402948 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fe8540ce8d0 con 0x7fe864073180 2024-03-30T02:47:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.874+0000 7fe862a00700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe8480a1290 0x7fe8480a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:39.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.875+0000 7fe862a00700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe8480a1290 0x7fe8480a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe850005950 tx=0x7fe8500058e0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:39.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:39.876+0000 7fe85be00700 1 -- 172.21.15.67:0/771402948 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fe854097a10 con 0x7fe864073180 2024-03-30T02:47:39.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:39 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1547114851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:47:39.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:39 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1547114851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:47:40.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.093+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe864067af0 con 0x7fe864073180 2024-03-30T02:47:40.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.094+0000 7fe85be00700 1 -- 172.21.15.67:0/771402948 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe85409b030 con 0x7fe864073180 2024-03-30T02:47:40.094 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:47:40.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe8480a1290 msgr2=0x7fe8480a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe8480a1290 0x7fe8480a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe850005950 tx=0x7fe8500058e0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 msgr2=0x7fe8641993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 0x7fe8641993e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe854005910 tx=0x7fe854009570 comp rx=0 tx=0).stop 2024-03-30T02:47:40.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 shutdown_connections 2024-03-30T02:47:40.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fe8480a1290 0x7fe8480a3750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe864073180 0x7fe8641993e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe86410d5a0 0x7fe86419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 --2- 172.21.15.67:0/771402948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe864073aa0 0x7fe864199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 >> 172.21.15.67:0/771402948 conn(0x7fe8640fc7f0 msgr2=0x7fe8640fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:40.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 shutdown_connections 2024-03-30T02:47:40.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.096+0000 7fe86a800700 1 -- 172.21.15.67:0/771402948 wait complete. 2024-03-30T02:47:40.115 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T02:47:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.318+0000 7fee32200700 1 -- 172.21.15.67:0/3628231861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 msgr2=0x7fee2c10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.318+0000 7fee32200700 1 --2- 172.21.15.67:0/3628231861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 0x7fee2c10dc00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fee2400b0a0 tx=0x7fee2400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.318+0000 7fee32200700 1 -- 172.21.15.67:0/3628231861 shutdown_connections 2024-03-30T02:47:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.318+0000 7fee32200700 1 --2- 172.21.15.67:0/3628231861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 0x7fee2c10dc00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.318+0000 7fee32200700 1 --2- 172.21.15.67:0/3628231861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2c100f60 0x7fee2c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.318+0000 7fee32200700 1 --2- 172.21.15.67:0/3628231861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee2c101910 0x7fee2c105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.318+0000 7fee32200700 1 -- 172.21.15.67:0/3628231861 >> 172.21.15.67:0/3628231861 conn(0x7fee2c0fc7c0 msgr2=0x7fee2c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.318+0000 7fee32200700 1 -- 172.21.15.67:0/3628231861 shutdown_connections 2024-03-30T02:47:40.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.318+0000 7fee32200700 1 -- 172.21.15.67:0/3628231861 wait complete. 2024-03-30T02:47:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.319+0000 7fee32200700 1 Processor -- start 2024-03-30T02:47:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.319+0000 7fee32200700 1 -- start start 2024-03-30T02:47:40.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.319+0000 7fee32200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2c100f60 0x7fee2c0730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.319+0000 7fee32200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 0x7fee2c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.319+0000 7fee32200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee2c073b30 0x7fee2c1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee32200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c1a5b40 con 0x7fee2c106080 2024-03-30T02:47:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee32200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c1a5c80 con 0x7fee2c100f60 2024-03-30T02:47:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee32200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c1a5e40 con 0x7fee2c073b30 2024-03-30T02:47:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 0x7fee2c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2b400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2c100f60 0x7fee2c0730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:40.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2aa00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 0x7fee2c0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38156/0 (socket says 172.21.15.67:38156) 2024-03-30T02:47:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2aa00700 1 -- 172.21.15.67:0/610631476 learned_addr learned my addr 172.21.15.67:0/610631476 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2be00700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee2c073b30 0x7fee2c1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2aa00700 1 -- 172.21.15.67:0/610631476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee2c073b30 msgr2=0x7fee2c1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2aa00700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee2c073b30 0x7fee2c1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2aa00700 1 -- 172.21.15.67:0/610631476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2c100f60 msgr2=0x7fee2c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2aa00700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2c100f60 0x7fee2c0730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2aa00700 1 -- 172.21.15.67:0/610631476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c009520 con 0x7fee2c106080 2024-03-30T02:47:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2be00700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee2c073b30 0x7fee2c1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T02:47:40.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2b400700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2c100f60 0x7fee2c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.320+0000 7fee2aa00700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 0x7fee2c0735f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e970 tx=0x7fee1800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.321+0000 7fee23e00700 1 -- 172.21.15.67:0/610631476 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18009790 con 0x7fee2c106080 2024-03-30T02:47:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.321+0000 7fee23e00700 1 -- 172.21.15.67:0/610631476 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee1800eec0 con 0x7fee2c106080 2024-03-30T02:47:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.321+0000 7fee23e00700 1 -- 172.21.15.67:0/610631476 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18005590 con 0x7fee2c106080 2024-03-30T02:47:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.321+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee24009d00 con 0x7fee2c106080 2024-03-30T02:47:40.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.321+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c1a64a0 con 0x7fee2c106080 2024-03-30T02:47:40.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.322+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c109350 con 0x7fee2c106080 2024-03-30T02:47:40.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.324+0000 7fee23e00700 1 -- 172.21.15.67:0/610631476 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fee1800cbd0 con 0x7fee2c106080 2024-03-30T02:47:40.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.325+0000 7fee23e00700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fee100a1310 0x7fee100a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:40.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.325+0000 7fee23e00700 1 -- 172.21.15.67:0/610631476 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fee18014070 con 0x7fee2c106080 2024-03-30T02:47:40.325 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.325+0000 7fee2b400700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fee100a1310 0x7fee100a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:40.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.326+0000 7fee2b400700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fee100a1310 0x7fee100a37d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009fd0 tx=0x7fee1c009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:40.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.329+0000 7fee23e00700 1 -- 172.21.15.67:0/610631476 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fee180962c0 con 0x7fee2c106080 2024-03-30T02:47:40.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.547+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee2c074540 con 0x7fee2c106080 2024-03-30T02:47:40.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.548+0000 7fee23e00700 1 -- 172.21.15.67:0/610631476 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee18004ef0 con 0x7fee2c106080 2024-03-30T02:47:40.548 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-03-30T02:47:40.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.550+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fee100a1310 msgr2=0x7fee100a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.550+0000 7fee32200700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fee100a1310 0x7fee100a37d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009fd0 tx=0x7fee1c009260 comp rx=0 tx=0).stop 2024-03-30T02:47:40.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.550+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 msgr2=0x7fee2c0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.550+0000 7fee32200700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 0x7fee2c0735f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e970 tx=0x7fee1800ec80 comp rx=0 tx=0).stop 2024-03-30T02:47:40.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.551+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 shutdown_connections 2024-03-30T02:47:40.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.551+0000 7fee32200700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fee100a1310 0x7fee100a37d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.551+0000 7fee32200700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee2c073b30 0x7fee2c1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.551+0000 7fee32200700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee2c100f60 0x7fee2c0730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.551+0000 7fee32200700 1 --2- 172.21.15.67:0/610631476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fee2c106080 0x7fee2c0735f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.551+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 >> 172.21.15.67:0/610631476 conn(0x7fee2c0fc7c0 msgr2=0x7fee2c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:40.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.551+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 shutdown_connections 2024-03-30T02:47:40.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.551+0000 7fee32200700 1 -- 172.21.15.67:0/610631476 wait complete. 2024-03-30T02:47:40.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T02:47:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:40 smithi082 ceph-mon[79176]: pgmap v329: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-03-30T02:47:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:40 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/4125486360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:47:40.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:40 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/771402948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:47:40.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.771+0000 7f7254c00700 1 -- 172.21.15.67:0/3948435743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250100f60 msgr2=0x7f7250101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.771+0000 7f7254c00700 1 --2- 172.21.15.67:0/3948435743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250100f60 0x7f7250101340 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7240009990 tx=0x7f7240009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.771+0000 7f7254c00700 1 -- 172.21.15.67:0/3948435743 shutdown_connections 2024-03-30T02:47:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.771+0000 7f7254c00700 1 --2- 172.21.15.67:0/3948435743 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250100f60 0x7f7250101340 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.771+0000 7f7254c00700 1 --2- 172.21.15.67:0/3948435743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7250106080 0x7f725010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.771+0000 7f7254c00700 1 --2- 172.21.15.67:0/3948435743 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7250101910 0x7f7250105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.771+0000 7f7254c00700 1 -- 172.21.15.67:0/3948435743 >> 172.21.15.67:0/3948435743 conn(0x7f72500fc7c0 msgr2=0x7f72500febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.771+0000 7f7254c00700 1 -- 172.21.15.67:0/3948435743 shutdown_connections 2024-03-30T02:47:40.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.771+0000 7f7254c00700 1 -- 172.21.15.67:0/3948435743 wait complete. 2024-03-30T02:47:40.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.772+0000 7f7254c00700 1 Processor -- start 2024-03-30T02:47:40.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.772+0000 7f7254c00700 1 -- start start 2024-03-30T02:47:40.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.772+0000 7f7254c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7250100f60 0x7f72500730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:40.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.772+0000 7f7254c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250106080 0x7f72500735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:40.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.772+0000 7f7254c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7250073b30 0x7f72501a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:40.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.772+0000 7f7254c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501a59d0 con 0x7f7250106080 2024-03-30T02:47:40.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.772+0000 7f7254c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501a5b40 con 0x7f7250073b30 2024-03-30T02:47:40.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.772+0000 7f7254c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501a5d00 con 0x7f7250100f60 2024-03-30T02:47:40.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f7247e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250106080 0x7f72500735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:40.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7250100f60 0x7f72500730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:40.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f7247e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250106080 0x7f72500735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38180/0 (socket says 172.21.15.67:38180) 2024-03-30T02:47:40.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724de00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7250100f60 0x7f72500730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:39998/0 (socket says 172.21.15.67:39998) 2024-03-30T02:47:40.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f7247e00700 1 -- 172.21.15.67:0/1505326670 learned_addr learned my addr 172.21.15.67:0/1505326670 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:40.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724e800700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7250073b30 0x7f72501a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:40.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724e800700 1 -- 172.21.15.67:0/1505326670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7250100f60 msgr2=0x7f72500730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724e800700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7250100f60 0x7f72500730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724e800700 1 -- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250106080 msgr2=0x7f72500735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:40.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724e800700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250106080 0x7f72500735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:40.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724e800700 1 -- 172.21.15.67:0/1505326670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72400095f0 con 0x7f7250073b30 2024-03-30T02:47:40.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f7247e00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250106080 0x7f72500735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:40.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724de00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7250100f60 0x7f72500730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:40.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.773+0000 7f724e800700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7250073b30 0x7f72501a5400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f724800efc0 tx=0x7f724800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:40.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.774+0000 7f7246a00700 1 -- 172.21.15.67:0/1505326670 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248018070 con 0x7f7250073b30 2024-03-30T02:47:40.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.774+0000 7f7246a00700 1 -- 172.21.15.67:0/1505326670 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f724800f040 con 0x7f7250073b30 2024-03-30T02:47:40.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.774+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72501a5f80 con 0x7f7250073b30 2024-03-30T02:47:40.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.774+0000 7f7246a00700 1 -- 172.21.15.67:0/1505326670 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724800cd40 con 0x7f7250073b30 2024-03-30T02:47:40.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.774+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72501a6460 con 0x7f7250073b30 2024-03-30T02:47:40.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.775+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72501092f0 con 0x7f7250073b30 2024-03-30T02:47:40.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.777+0000 7f7246a00700 1 -- 172.21.15.67:0/1505326670 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f7248003a00 con 0x7f7250073b30 2024-03-30T02:47:40.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.778+0000 7f7246a00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f72340a1290 0x7f72340a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:40.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.778+0000 7f7246a00700 1 -- 172.21.15.67:0/1505326670 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f72480cd500 con 0x7f7250073b30 2024-03-30T02:47:40.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.778+0000 7f724de00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f72340a1290 0x7f72340a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:40.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.779+0000 7f724de00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f72340a1290 0x7f72340a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f724000b2e0 tx=0x7f7240005fd0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:40.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.780+0000 7f7246a00700 1 -- 172.21.15.67:0/1505326670 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f7248096640 con 0x7f7250073b30 2024-03-30T02:47:40.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:40 smithi067 ceph-mon[37808]: pgmap v329: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-03-30T02:47:40.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:40 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/4125486360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:47:40.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:40 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/771402948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:47:40.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:40 smithi170 ceph-mon[78798]: pgmap v329: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-03-30T02:47:40.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:40 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/4125486360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-30T02:47:40.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:40 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/771402948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-30T02:47:40.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.998+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7250074590 con 0x7f7250073b30 2024-03-30T02:47:40.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:40.999+0000 7f7246a00700 1 -- 172.21.15.67:0/1505326670 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7248099c60 con 0x7f7250073b30 2024-03-30T02:47:40.999 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-03-30T02:47:41.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f72340a1290 msgr2=0x7f72340a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f72340a1290 0x7f72340a3750 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f724000b2e0 tx=0x7f7240005fd0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7250073b30 msgr2=0x7f72501a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7250073b30 0x7f72501a5400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f724800efc0 tx=0x7f724800c3a0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 shutdown_connections 2024-03-30T02:47:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f72340a1290 0x7f72340a3750 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7250073b30 0x7f72501a5400 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.002 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7250100f60 0x7f72500730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 --2- 172.21.15.67:0/1505326670 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7250106080 0x7f72500735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.001+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 >> 172.21.15.67:0/1505326670 conn(0x7f72500fc7c0 msgr2=0x7f72500fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:41.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.002+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 shutdown_connections 2024-03-30T02:47:41.003 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.002+0000 7f7254c00700 1 -- 172.21.15.67:0/1505326670 wait complete. 2024-03-30T02:47:41.021 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T02:47:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.222+0000 7fb4d1200700 1 -- 172.21.15.67:0/2849405254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 msgr2=0x7fb4cc10dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.222+0000 7fb4d1200700 1 --2- 172.21.15.67:0/2849405254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 0x7fb4cc10dc00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400b0a0 tx=0x7fb4c400b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.222+0000 7fb4d1200700 1 -- 172.21.15.67:0/2849405254 shutdown_connections 2024-03-30T02:47:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.223+0000 7fb4d1200700 1 --2- 172.21.15.67:0/2849405254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc100f60 0x7fb4cc101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.223+0000 7fb4d1200700 1 --2- 172.21.15.67:0/2849405254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 0x7fb4cc10dc00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.223+0000 7fb4d1200700 1 --2- 172.21.15.67:0/2849405254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4cc101910 0x7fb4cc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.223+0000 7fb4d1200700 1 -- 172.21.15.67:0/2849405254 >> 172.21.15.67:0/2849405254 conn(0x7fb4cc0fc7c0 msgr2=0x7fb4cc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.223+0000 7fb4d1200700 1 -- 172.21.15.67:0/2849405254 shutdown_connections 2024-03-30T02:47:41.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.223+0000 7fb4d1200700 1 -- 172.21.15.67:0/2849405254 wait complete. 2024-03-30T02:47:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4d1200700 1 Processor -- start 2024-03-30T02:47:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4d1200700 1 -- start start 2024-03-30T02:47:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4d1200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4cc100f60 0x7fb4cc1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4d1200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 0x7fb4cc1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:41.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4d1200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc19bc30 0x7fb4cc19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4d1200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc19c6b0 con 0x7fb4cc19bc30 2024-03-30T02:47:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4d1200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc19c820 con 0x7fb4cc100f60 2024-03-30T02:47:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4d1200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc19c9e0 con 0x7fb4cc106080 2024-03-30T02:47:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4c9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4cc100f60 0x7fb4cc1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:41.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4c9400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 0x7fb4cc1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4c9400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 0x7fb4cc1978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:40008/0 (socket says 172.21.15.67:40008) 2024-03-30T02:47:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4ca800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc19bc30 0x7fb4cc19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4c9400700 1 -- 172.21.15.67:0/1602182849 learned_addr learned my addr 172.21.15.67:0/1602182849 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.224+0000 7fb4c9e00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4cc100f60 0x7fb4cc1973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48816/0 (socket says 172.21.15.67:48816) 2024-03-30T02:47:41.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4c9400700 1 -- 172.21.15.67:0/1602182849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4cc100f60 msgr2=0x7fb4cc1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4c9400700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4cc100f60 0x7fb4cc1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4c9400700 1 -- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc19bc30 msgr2=0x7fb4cc19c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4c9400700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc19bc30 0x7fb4cc19c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4c9400700 1 -- 172.21.15.67:0/1602182849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c4009d00 con 0x7fb4cc106080 2024-03-30T02:47:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4ca800700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc19bc30 0x7fb4cc19c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4c9400700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 0x7fb4cc1978e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800e970 tx=0x7fb4b800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4c2a00700 1 -- 172.21.15.67:0/1602182849 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8009790 con 0x7fb4cc106080 2024-03-30T02:47:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4c2a00700 1 -- 172.21.15.67:0/1602182849 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4b800c7e0 con 0x7fb4cc106080 2024-03-30T02:47:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.225+0000 7fb4c2a00700 1 -- 172.21.15.67:0/1602182849 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80054e0 con 0x7fb4cc106080 2024-03-30T02:47:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.226+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc1a3f40 con 0x7fb4cc106080 2024-03-30T02:47:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.226+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc1a4370 con 0x7fb4cc106080 2024-03-30T02:47:41.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.227+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc109350 con 0x7fb4cc106080 2024-03-30T02:47:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.229+0000 7fb4c2a00700 1 -- 172.21.15.67:0/1602182849 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fb4b800c950 con 0x7fb4cc106080 2024-03-30T02:47:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.230+0000 7fb4c2a00700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb4b00a1290 0x7fb4b00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.230+0000 7fb4c2a00700 1 -- 172.21.15.67:0/1602182849 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7fb4b8014070 con 0x7fb4cc106080 2024-03-30T02:47:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.230+0000 7fb4c9e00700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb4b00a1290 0x7fb4b00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.231+0000 7fb4c9e00700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb4b00a1290 0x7fb4b00a3750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009c60 tx=0x7fb4bc009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.232+0000 7fb4c2a00700 1 -- 172.21.15.67:0/1602182849 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fb4b80964b0 con 0x7fb4cc106080 2024-03-30T02:47:41.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.508+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb4cc102430 con 0x7fb4cc106080 2024-03-30T02:47:41.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.509+0000 7fb4c2a00700 1 -- 172.21.15.67:0/1602182849 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v96) v1 ==== 81+0+1002 (secure 0 0 0) 0x7fb4b8099ad0 con 0x7fb4cc106080 2024-03-30T02:47:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/759150839 2024-03-31T02:37:02.362975+0000 2024-03-30T02:47:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2693891742 2024-03-31T02:29:01.908985+0000 2024-03-30T02:47:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1250300809 2024-03-31T02:29:01.908985+0000 2024-03-30T02:47:41.509 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/4186481961 2024-03-31T02:29:01.908985+0000 2024-03-30T02:47:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3533289986 2024-03-31T02:29:26.929980+0000 2024-03-30T02:47:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3018284010 2024-03-31T02:37:02.362975+0000 2024-03-30T02:47:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/598232861 2024-03-31T02:29:26.929980+0000 2024-03-30T02:47:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2682757440 2024-03-31T02:29:26.929980+0000 2024-03-30T02:47:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/3801648476 2024-03-31T02:37:02.362975+0000 2024-03-30T02:47:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/4226270132 2024-03-31T02:37:02.362975+0000 2024-03-30T02:47:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3435746414 2024-03-31T02:29:01.908985+0000 2024-03-30T02:47:41.510 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1220244664 2024-03-31T02:37:02.362975+0000 2024-03-30T02:47:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2682757440 2024-03-31T02:29:26.929980+0000 2024-03-30T02:47:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2119993048 2024-03-31T02:29:26.929980+0000 2024-03-30T02:47:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/3801648476 2024-03-31T02:37:02.362975+0000 2024-03-30T02:47:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/4186481961 2024-03-31T02:29:01.908985+0000 2024-03-30T02:47:41.511 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2449045224 2024-03-31T02:37:02.362975+0000 2024-03-30T02:47:41.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb4b00a1290 msgr2=0x7fb4b00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:41.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb4b00a1290 0x7fb4b00a3750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009c60 tx=0x7fb4bc009260 comp rx=0 tx=0).stop 2024-03-30T02:47:41.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 msgr2=0x7fb4cc1978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:41.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 0x7fb4cc1978e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800e970 tx=0x7fb4b800ec80 comp rx=0 tx=0).stop 2024-03-30T02:47:41.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 shutdown_connections 2024-03-30T02:47:41.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fb4b00a1290 0x7fb4b00a3750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4cc19bc30 0x7fb4cc19c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4cc100f60 0x7fb4cc1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 --2- 172.21.15.67:0/1602182849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4cc106080 0x7fb4cc1978e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:41.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 >> 172.21.15.67:0/1602182849 conn(0x7fb4cc0fc7c0 msgr2=0x7fb4cc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:41.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 shutdown_connections 2024-03-30T02:47:41.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:41.512+0000 7fb4d1200700 1 -- 172.21.15.67:0/1602182849 wait complete. 2024-03-30T02:47:41.516 INFO:teuthology.orchestra.run.smithi067.stderr:listed 17 entries 2024-03-30T02:47:41.530 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-30T02:47:41.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:47:41.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-30T02:47:41.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:41 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/610631476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:47:41.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:41 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1505326670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:47:41.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:41 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/610631476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:47:41.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:41 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1505326670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:47:41.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:41 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/610631476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-30T02:47:41.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:41 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1505326670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-30T02:47:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:42 smithi082 ceph-mon[79176]: pgmap v330: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-03-30T02:47:42.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:42 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1602182849' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T02:47:42.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:42 smithi067 ceph-mon[37808]: pgmap v330: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-03-30T02:47:42.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:42 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1602182849' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T02:47:42.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:42 smithi170 ceph-mon[78798]: pgmap v330: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-03-30T02:47:42.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:42 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1602182849' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-30T02:47:43.623 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:47:43.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:47:43.624 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T02:47:43.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:47:43.669 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-03-30T02:47:43.754 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T02:47:43.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:47:43.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T02:47:43.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T02:47:43.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-03-30T02:47:43.964 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-30T02:47:43.975 INFO:tasks.cephadm:Teardown begin 2024-03-30T02:47:43.975 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T02:47:43.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:47:44.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.180+0000 7f1594800700 1 -- 172.21.15.67:0/590085273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f158c0731e0 msgr2=0x7f158c0735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.180+0000 7f1594800700 1 --2- 172.21.15.67:0/590085273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f158c0731e0 0x7f158c0735c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f15800098f0 tx=0x7f1580009c00 comp rx=0 tx=0).stop 2024-03-30T02:47:44.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.180+0000 7f1594800700 1 -- 172.21.15.67:0/590085273 shutdown_connections 2024-03-30T02:47:44.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.180+0000 7f1594800700 1 --2- 172.21.15.67:0/590085273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f158c10d430 0x7f158c10f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.180+0000 7f1594800700 1 --2- 172.21.15.67:0/590085273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f158c0731e0 0x7f158c0735c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.180+0000 7f1594800700 1 --2- 172.21.15.67:0/590085273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f158c073b00 0x7f158c10cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.180+0000 7f1594800700 1 -- 172.21.15.67:0/590085273 >> 172.21.15.67:0/590085273 conn(0x7f158c0fc700 msgr2=0x7f158c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:44.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.180+0000 7f1594800700 1 -- 172.21.15.67:0/590085273 shutdown_connections 2024-03-30T02:47:44.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.180+0000 7f1594800700 1 -- 172.21.15.67:0/590085273 wait complete. 2024-03-30T02:47:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.181+0000 7f1594800700 1 Processor -- start 2024-03-30T02:47:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.181+0000 7f1594800700 1 -- start start 2024-03-30T02:47:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.181+0000 7f1594800700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f158c0731e0 0x7f158c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.181+0000 7f1594800700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f158c073b00 0x7f158c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.181+0000 7f1594800700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f158c10d430 0x7f158c19dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:44.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.181+0000 7f1594800700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c19e300 con 0x7f158c0731e0 2024-03-30T02:47:44.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.181+0000 7f1594800700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c19e470 con 0x7f158c073b00 2024-03-30T02:47:44.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.181+0000 7f1594800700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c19e630 con 0x7f158c10d430 2024-03-30T02:47:44.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f158c0731e0 0x7f158c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:44.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f158c073b00 0x7f158c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:44.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f158c0731e0 0x7f158c1993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38228/0 (socket says 172.21.15.67:38228) 2024-03-30T02:47:44.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591e00700 1 -- 172.21.15.67:0/3207544530 learned_addr learned my addr 172.21.15.67:0/3207544530 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:44.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1592800700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f158c10d430 0x7f158c19dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:44.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591e00700 1 -- 172.21.15.67:0/3207544530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f158c10d430 msgr2=0x7f158c19dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591e00700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f158c10d430 0x7f158c19dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591e00700 1 -- 172.21.15.67:0/3207544530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f158c073b00 msgr2=0x7f158c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591e00700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f158c073b00 0x7f158c199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591e00700 1 -- 172.21.15.67:0/3207544530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0095f0 con 0x7f158c0731e0 2024-03-30T02:47:44.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591400700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f158c073b00 0x7f158c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:44.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.182+0000 7f1591e00700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f158c0731e0 0x7f158c1993e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1580005950 tx=0x7f158000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:44.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.183+0000 7f1586a00700 1 -- 172.21.15.67:0/3207544530 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158001c070 con 0x7f158c0731e0 2024-03-30T02:47:44.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.183+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1580009520 con 0x7f158c0731e0 2024-03-30T02:47:44.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.183+0000 7f1586a00700 1 -- 172.21.15.67:0/3207544530 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f158000baa0 con 0x7f158c0731e0 2024-03-30T02:47:44.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.183+0000 7f1586a00700 1 -- 172.21.15.67:0/3207544530 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800038e0 con 0x7f158c0731e0 2024-03-30T02:47:44.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.183+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c1a5f70 con 0x7f158c0731e0 2024-03-30T02:47:44.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.184+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c108640 con 0x7f158c0731e0 2024-03-30T02:47:44.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.186+0000 7f1586a00700 1 -- 172.21.15.67:0/3207544530 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f1580020030 con 0x7f158c0731e0 2024-03-30T02:47:44.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.188+0000 7f1586a00700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f15740a1290 0x7f15740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:44.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.188+0000 7f1586a00700 1 -- 172.21.15.67:0/3207544530 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f15800cde40 con 0x7f158c0731e0 2024-03-30T02:47:44.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.188+0000 7f1591400700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f15740a1290 0x7f15740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:44.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.189+0000 7f1591400700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f15740a1290 0x7f15740a3750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f158c19a930 tx=0x7f157c009310 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:44.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.192+0000 7f1586a00700 1 -- 172.21.15.67:0/3207544530 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f1580096f20 con 0x7f158c0731e0 2024-03-30T02:47:44.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.460+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f158c074890 con 0x7f158c0731e0 2024-03-30T02:47:44.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.461+0000 7f1586a00700 1 -- 172.21.15.67:0/3207544530 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f158009a540 con 0x7f158c0731e0 2024-03-30T02:47:44.461 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:47:44.461 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:45:07.400846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:47:44.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f15740a1290 msgr2=0x7f15740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f15740a1290 0x7f15740a3750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f158c19a930 tx=0x7f157c009310 comp rx=0 tx=0).stop 2024-03-30T02:47:44.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f158c0731e0 msgr2=0x7f158c1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f158c0731e0 0x7f158c1993e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1580005950 tx=0x7f158000fa00 comp rx=0 tx=0).stop 2024-03-30T02:47:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 shutdown_connections 2024-03-30T02:47:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f15740a1290 0x7f15740a3750 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f158c0731e0 0x7f158c1993e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f158c10d430 0x7f158c19dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 --2- 172.21.15.67:0/3207544530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f158c073b00 0x7f158c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 >> 172.21.15.67:0/3207544530 conn(0x7f158c0fc700 msgr2=0x7f158c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.463+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 shutdown_connections 2024-03-30T02:47:44.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.464+0000 7f1594800700 1 -- 172.21.15.67:0/3207544530 wait complete. 2024-03-30T02:47:44.465 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-03-30T02:47:44.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T02:47:44.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.685+0000 7f529d400700 1 -- 172.21.15.67:0/3578652908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5298100e60 msgr2=0x7f5298101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.685+0000 7f529d400700 1 --2- 172.21.15.67:0/3578652908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5298100e60 0x7f5298101240 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5288009990 tx=0x7f5288009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.685+0000 7f529d400700 1 -- 172.21.15.67:0/3578652908 shutdown_connections 2024-03-30T02:47:44.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.685+0000 7f529d400700 1 --2- 172.21.15.67:0/3578652908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5298100e60 0x7f5298101240 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.685+0000 7f529d400700 1 --2- 172.21.15.67:0/3578652908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5298105f80 0x7f529810db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.685+0000 7f529d400700 1 --2- 172.21.15.67:0/3578652908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298101810 0x7f5298105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.685+0000 7f529d400700 1 -- 172.21.15.67:0/3578652908 >> 172.21.15.67:0/3578652908 conn(0x7f52980fc700 msgr2=0x7f52980feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:44.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.685+0000 7f529d400700 1 -- 172.21.15.67:0/3578652908 shutdown_connections 2024-03-30T02:47:44.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.685+0000 7f529d400700 1 -- 172.21.15.67:0/3578652908 wait complete. 2024-03-30T02:47:44.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.686+0000 7f529d400700 1 Processor -- start 2024-03-30T02:47:44.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.686+0000 7f529d400700 1 -- start start 2024-03-30T02:47:44.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f529d400700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298100e60 0x7f5298199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:44.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f529d400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5298105f80 0x7f5298199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:44.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f529d400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f529819dc30 0x7f529819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:44.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f529d400700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529819e6b0 con 0x7f529819dc30 2024-03-30T02:47:44.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f529d400700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529819e820 con 0x7f5298105f80 2024-03-30T02:47:44.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f529d400700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529819e9e0 con 0x7f5298100e60 2024-03-30T02:47:44.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5297200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f529819dc30 0x7f529819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:44.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5297200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f529819dc30 0x7f529819e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38240/0 (socket says 172.21.15.67:38240) 2024-03-30T02:47:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5297200700 1 -- 172.21.15.67:0/2709853906 learned_addr learned my addr 172.21.15.67:0/2709853906 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5295e00700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5298105f80 0x7f5298199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5296800700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298100e60 0x7f5298199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5297200700 1 -- 172.21.15.67:0/2709853906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298100e60 msgr2=0x7f5298199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5297200700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298100e60 0x7f5298199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5297200700 1 -- 172.21.15.67:0/2709853906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5298105f80 msgr2=0x7f5298199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5297200700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5298105f80 0x7f5298199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.687+0000 7f5297200700 1 -- 172.21.15.67:0/2709853906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52880095f0 con 0x7f529819dc30 2024-03-30T02:47:44.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.688+0000 7f5297200700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f529819dc30 0x7f529819e0b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f529000ef30 tx=0x7f529000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:44.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.688+0000 7f528f400700 1 -- 172.21.15.67:0/2709853906 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290018070 con 0x7f529819dc30 2024-03-30T02:47:44.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.688+0000 7f528f400700 1 -- 172.21.15.67:0/2709853906 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f529000f040 con 0x7f529819dc30 2024-03-30T02:47:44.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.688+0000 7f528f400700 1 -- 172.21.15.67:0/2709853906 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529000cd00 con 0x7f529819dc30 2024-03-30T02:47:44.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.688+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52981a5ee0 con 0x7f529819dc30 2024-03-30T02:47:44.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.688+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52981a6370 con 0x7f529819dc30 2024-03-30T02:47:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.689+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5298102330 con 0x7f529819dc30 2024-03-30T02:47:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.691+0000 7f528f400700 1 -- 172.21.15.67:0/2709853906 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5290003a00 con 0x7f529819dc30 2024-03-30T02:47:44.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.692+0000 7f528f400700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f527c0a1290 0x7f527c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:44.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.693+0000 7f528f400700 1 -- 172.21.15.67:0/2709853906 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f52900ccb40 con 0x7f529819dc30 2024-03-30T02:47:44.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.693+0000 7f5296800700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f527c0a1290 0x7f527c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:44.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.693+0000 7f5296800700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f527c0a1290 0x7f527c0a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f528800b2e0 tx=0x7f5288005fd0 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:44.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.697+0000 7f528f400700 1 -- 172.21.15.67:0/2709853906 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5290095c20 con 0x7f529819dc30 2024-03-30T02:47:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:44 smithi082 ceph-mon[79176]: pgmap v331: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-03-30T02:47:44.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:44 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3207544530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:47:44.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:44 smithi067 ceph-mon[37808]: pgmap v331: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-03-30T02:47:44.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:44 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3207544530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:47:44.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:44 smithi170 ceph-mon[78798]: pgmap v331: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-03-30T02:47:44.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:44 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3207544530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:47:44.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.966+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5298101810 con 0x7f529819dc30 2024-03-30T02:47:44.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.967+0000 7f528f400700 1 -- 172.21.15.67:0/2709853906 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10909 (secure 0 0 0) 0x7f5290099240 con 0x7f529819dc30 2024-03-30T02:47:44.967 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:47:44.967 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14463,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/2327446464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":2327446464},{"type":"v1","addr":"172.21.15.67:6835","nonce":2327446464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14475,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/437498289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":437498289},{"type":"v1","addr":"172.21.15.67:6837","nonce":437498289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-30T02:34:46.713132+0000","modified":"2024-03-30T02:45:07.400846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24560,"mds_2":24541,"mds_3":24539,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14526":{"gid":14526,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/853027854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":853027854},{"type":"v1","addr":"172.21.15.67:6841","nonce":853027854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24466":{"gid":24466,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/3749225376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3749225376},{"type":"v1","addr":"172.21.15.170:6833","nonce":3749225376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24476":{"gid":24476,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6833/1365991767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1365991767},{"type":"v1","addr":"172.21.15.82:6833","nonce":1365991767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24512":{"gid":24512,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/3401372629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3401372629},{"type":"v1","addr":"172.21.15.82:6835","nonce":3401372629}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/1685909437","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1685909437},{"type":"v1","addr":"172.21.15.170:6835","nonce":1685909437}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6837/500476759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":500476759},{"type":"v1","addr":"172.21.15.170:6837","nonce":500476759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.82:6837/1183051781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1183051781},{"type":"v1","addr":"172.21.15.82:6837","nonce":1183051781}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24541":{"gid":24541,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6839/2570108618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2570108618},{"type":"v1","addr":"172.21.15.170:6839","nonce":2570108618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.67:6839/2974132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2974132199},{"type":"v1","addr":"172.21.15.67:6839","nonce":2974132199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6839/604601206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":604601206},{"type":"v1","addr":"172.21.15.82:6839","nonce":604601206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-30T02:47:44.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f527c0a1290 msgr2=0x7f527c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f527c0a1290 0x7f527c0a3750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f528800b2e0 tx=0x7f5288005fd0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f529819dc30 msgr2=0x7f529819e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:44.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f529819dc30 0x7f529819e0b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f529000ef30 tx=0x7f529000c3a0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 shutdown_connections 2024-03-30T02:47:44.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f527c0a1290 0x7f527c0a3750 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f529819dc30 0x7f529819e0b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298100e60 0x7f5298199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 --2- 172.21.15.67:0/2709853906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5298105f80 0x7f5298199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:44.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 >> 172.21.15.67:0/2709853906 conn(0x7f52980fc700 msgr2=0x7f52980fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:44.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 shutdown_connections 2024-03-30T02:47:44.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:44.969+0000 7f529d400700 1 -- 172.21.15.67:0/2709853906 wait complete. 2024-03-30T02:47:44.971 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 41 2024-03-30T02:47:44.989 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T02:47:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.190+0000 7f2af3c00700 1 -- 172.21.15.67:0/654913931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 msgr2=0x7f2aec101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.190+0000 7f2af3c00700 1 --2- 172.21.15.67:0/654913931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 0x7f2aec101240 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009990 tx=0x7f2ae0009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.190+0000 7f2af3c00700 1 -- 172.21.15.67:0/654913931 shutdown_connections 2024-03-30T02:47:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.190+0000 7f2af3c00700 1 --2- 172.21.15.67:0/654913931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 0x7f2aec101240 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.190+0000 7f2af3c00700 1 --2- 172.21.15.67:0/654913931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec105f80 0x7f2aec10db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.190+0000 7f2af3c00700 1 --2- 172.21.15.67:0/654913931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aec101810 0x7f2aec105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.190+0000 7f2af3c00700 1 -- 172.21.15.67:0/654913931 >> 172.21.15.67:0/654913931 conn(0x7f2aec0fc700 msgr2=0x7f2aec0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.190+0000 7f2af3c00700 1 -- 172.21.15.67:0/654913931 shutdown_connections 2024-03-30T02:47:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.190+0000 7f2af3c00700 1 -- 172.21.15.67:0/654913931 wait complete. 2024-03-30T02:47:45.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.191+0000 7f2af3c00700 1 Processor -- start 2024-03-30T02:47:45.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.191+0000 7f2af3c00700 1 -- start start 2024-03-30T02:47:45.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af3c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 0x7f2aec1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:45.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af3c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aec105f80 0x7f2aec199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:45.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af3c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec19dc80 0x7f2aec19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:45.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af3c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec19e700 con 0x7f2aec100e60 2024-03-30T02:47:45.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af3c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec19e870 con 0x7f2aec105f80 2024-03-30T02:47:45.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af3c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec19ea30 con 0x7f2aec19dc80 2024-03-30T02:47:45.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af1200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 0x7f2aec1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:45.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af1200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 0x7f2aec1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38254/0 (socket says 172.21.15.67:38254) 2024-03-30T02:47:45.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af1200700 1 -- 172.21.15.67:0/2498194145 learned_addr learned my addr 172.21.15.67:0/2498194145 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:45.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2aebe00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aec105f80 0x7f2aec199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:45.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.192+0000 7f2af1c00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec19dc80 0x7f2aec19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:45.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2af1200700 1 -- 172.21.15.67:0/2498194145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec19dc80 msgr2=0x7f2aec19e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2af1200700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec19dc80 0x7f2aec19e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2af1200700 1 -- 172.21.15.67:0/2498194145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aec105f80 msgr2=0x7f2aec199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2af1200700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aec105f80 0x7f2aec199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2af1200700 1 -- 172.21.15.67:0/2498194145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae00095f0 con 0x7f2aec100e60 2024-03-30T02:47:45.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2aebe00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aec105f80 0x7f2aec199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:45.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2af1200700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 0x7f2aec1993f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0000c00 tx=0x7f2ae0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:45.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2ae9600700 1 -- 172.21.15.67:0/2498194145 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae001d070 con 0x7f2aec100e60 2024-03-30T02:47:45.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2ae9600700 1 -- 172.21.15.67:0/2498194145 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae0022470 con 0x7f2aec100e60 2024-03-30T02:47:45.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2ae9600700 1 -- 172.21.15.67:0/2498194145 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0017650 con 0x7f2aec100e60 2024-03-30T02:47:45.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.193+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec1a5f30 con 0x7f2aec100e60 2024-03-30T02:47:45.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.194+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec1a62a0 con 0x7f2aec100e60 2024-03-30T02:47:45.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.194+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec102330 con 0x7f2aec100e60 2024-03-30T02:47:45.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.196+0000 7f2ae9600700 1 -- 172.21.15.67:0/2498194145 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f2ae00225e0 con 0x7f2aec100e60 2024-03-30T02:47:45.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.199+0000 7f2ae9600700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ad40a1310 0x7f2ad40a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:45.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.199+0000 7f2ae9600700 1 -- 172.21.15.67:0/2498194145 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f2ae00ce2c0 con 0x7f2aec100e60 2024-03-30T02:47:45.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.199+0000 7f2aebe00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ad40a1310 0x7f2ad40a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:45.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.199+0000 7f2aebe00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ad40a1310 0x7f2ad40a37d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2aec19a8f0 tx=0x7f2ad8009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:45.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.203+0000 7f2ae9600700 1 -- 172.21.15.67:0/2498194145 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f2ae00974c0 con 0x7f2aec100e60 2024-03-30T02:47:45.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.454+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2aec067a10 con 0x7f2aec100e60 2024-03-30T02:47:45.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.456+0000 7f2ae9600700 1 -- 172.21.15.67:0/2498194145 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+25099 (secure 0 0 0) 0x7f2ae009aae0 con 0x7f2aec100e60 2024-03-30T02:47:45.456 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-03-30T02:47:45.456 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":96,"fsid":"0ef0748e-ee3d-11ee-b647-cb9ed24678a4","created":"2024-03-30T02:28:23.927384+0000","modified":"2024-03-30T02:47:16.442983+0000","last_up_change":"2024-03-30T02:34:10.070158+0000","last_in_change":"2024-03-30T02:33:54.194247+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T02:31:23.541523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T02:34:45.007923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T02:34:46.178111+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T02:34:49.253894+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":95,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5709d485-60d6-4959-b2dd-a5aeb11363af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6803","nonce":1668225002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6805","nonce":1668225002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6809","nonce":1668225002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":1668225002},{"type":"v1","addr":"172.21.15.67:6807","nonce":1668225002}]},"public_addr":"172.21.15.67:6803/1668225002","cluster_addr":"172.21.15.67:6805/1668225002","heartbeat_back_addr":"172.21.15.67:6809/1668225002","heartbeat_front_addr":"172.21.15.67:6807/1668225002","state":["exists","up"]},{"osd":1,"uuid":"1deb576f-c68d-421e-b78a-cfd6e72aafc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6801","nonce":1982951943}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6803","nonce":1982951943}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6807","nonce":1982951943}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":1982951943},{"type":"v1","addr":"172.21.15.82:6805","nonce":1982951943}]},"public_addr":"172.21.15.82:6801/1982951943","cluster_addr":"172.21.15.82:6803/1982951943","heartbeat_back_addr":"172.21.15.82:6807/1982951943","heartbeat_front_addr":"172.21.15.82:6805/1982951943","state":["exists","up"]},{"osd":2,"uuid":"0bcf7865-4fd5-4a95-9cb1-294899b88e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6801","nonce":229013363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6803","nonce":229013363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6807","nonce":229013363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":229013363},{"type":"v1","addr":"172.21.15.170:6805","nonce":229013363}]},"public_addr":"172.21.15.170:6801/229013363","cluster_addr":"172.21.15.170:6803/229013363","heartbeat_back_addr":"172.21.15.170:6807/229013363","heartbeat_front_addr":"172.21.15.170:6805/229013363","state":["exists","up"]},{"osd":3,"uuid":"eea2b812-7830-4563-aa7e-b340ae8a35bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6811","nonce":2637526467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6813","nonce":2637526467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6817","nonce":2637526467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":2637526467},{"type":"v1","addr":"172.21.15.67:6815","nonce":2637526467}]},"public_addr":"172.21.15.67:6811/2637526467","cluster_addr":"172.21.15.67:6813/2637526467","heartbeat_back_addr":"172.21.15.67:6817/2637526467","heartbeat_front_addr":"172.21.15.67:6815/2637526467","state":["exists","up"]},{"osd":4,"uuid":"6a35b5bb-587d-489e-941d-5b9ec9c1c2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6809","nonce":1426586615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6811","nonce":1426586615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6815","nonce":1426586615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1426586615},{"type":"v1","addr":"172.21.15.82:6813","nonce":1426586615}]},"public_addr":"172.21.15.82:6809/1426586615","cluster_addr":"172.21.15.82:6811/1426586615","heartbeat_back_addr":"172.21.15.82:6815/1426586615","heartbeat_front_addr":"172.21.15.82:6813/1426586615","state":["exists","up"]},{"osd":5,"uuid":"787b667a-b309-4292-8a66-d7716b659d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6809","nonce":2583913458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6811","nonce":2583913458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6815","nonce":2583913458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":2583913458},{"type":"v1","addr":"172.21.15.170:6813","nonce":2583913458}]},"public_addr":"172.21.15.170:6809/2583913458","cluster_addr":"172.21.15.170:6811/2583913458","heartbeat_back_addr":"172.21.15.170:6815/2583913458","heartbeat_front_addr":"172.21.15.170:6813/2583913458","state":["exists","up"]},{"osd":6,"uuid":"986d28e3-52e9-462b-b728-d76c2457f15a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6819","nonce":3459663543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6821","nonce":3459663543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6825","nonce":3459663543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":3459663543},{"type":"v1","addr":"172.21.15.67:6823","nonce":3459663543}]},"public_addr":"172.21.15.67:6819/3459663543","cluster_addr":"172.21.15.67:6821/3459663543","heartbeat_back_addr":"172.21.15.67:6825/3459663543","heartbeat_front_addr":"172.21.15.67:6823/3459663543","state":["exists","up"]},{"osd":7,"uuid":"fa6e309c-75b6-4011-a6d1-bb0b1c438ef4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6817","nonce":318348288}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6819","nonce":318348288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6823","nonce":318348288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":318348288},{"type":"v1","addr":"172.21.15.82:6821","nonce":318348288}]},"public_addr":"172.21.15.82:6817/318348288","cluster_addr":"172.21.15.82:6819/318348288","heartbeat_back_addr":"172.21.15.82:6823/318348288","heartbeat_front_addr":"172.21.15.82:6821/318348288","state":["exists","up"]},{"osd":8,"uuid":"90ecfccd-eb50-4be0-927c-ae2c8a415c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6817","nonce":4041482230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6819","nonce":4041482230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6823","nonce":4041482230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":4041482230},{"type":"v1","addr":"172.21.15.170:6821","nonce":4041482230}]},"public_addr":"172.21.15.170:6817/4041482230","cluster_addr":"172.21.15.170:6819/4041482230","heartbeat_back_addr":"172.21.15.170:6823/4041482230","heartbeat_front_addr":"172.21.15.170:6821/4041482230","state":["exists","up"]},{"osd":9,"uuid":"1ec7fee7-63f1-4193-aa2c-769a1f5e54d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6827","nonce":2316326417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6829","nonce":2316326417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6833","nonce":2316326417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2316326417},{"type":"v1","addr":"172.21.15.67:6831","nonce":2316326417}]},"public_addr":"172.21.15.67:6827/2316326417","cluster_addr":"172.21.15.67:6829/2316326417","heartbeat_back_addr":"172.21.15.67:6833/2316326417","heartbeat_front_addr":"172.21.15.67:6831/2316326417","state":["exists","up"]},{"osd":10,"uuid":"2a7ade41-1c90-42e9-a53f-6f75074c9f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6825","nonce":3088174869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6827","nonce":3088174869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6831","nonce":3088174869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3088174869},{"type":"v1","addr":"172.21.15.82:6829","nonce":3088174869}]},"public_addr":"172.21.15.82:6825/3088174869","cluster_addr":"172.21.15.82:6827/3088174869","heartbeat_back_addr":"172.21.15.82:6831/3088174869","heartbeat_front_addr":"172.21.15.82:6829/3088174869","state":["exists","up"]},{"osd":11,"uuid":"a7b717af-28b4-4aff-98f9-b652333a9808","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6825","nonce":3333975805}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6827","nonce":3333975805}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6831","nonce":3333975805}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3333975805},{"type":"v1","addr":"172.21.15.170:6829","nonce":3333975805}]},"public_addr":"172.21.15.170:6825/3333975805","cluster_addr":"172.21.15.170:6827/3333975805","heartbeat_back_addr":"172.21.15.170:6831/3333975805","heartbeat_front_addr":"172.21.15.170:6829/3333975805","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:30:45.154639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:02.630864+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:19.928144+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:41.054615+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:31:58.449376+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:15.891242+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:37.123130+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:32:54.766393+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:12.457360+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:33.675080+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:33:51.566587+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T02:34:08.615109+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8},{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/759150839":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/2693891742":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1250300809":"2024-03-31T02:29:01.908985+0000","172.21.15.67:6800/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/3533289986":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/3018284010":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/598232861":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6800/3801648476":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/4226270132":"2024-03-31T02:37:02.362975+0000","172.21.15.67:0/3435746414":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/1220244664":"2024-03-31T02:37:02.362975+0000","172.21.15.67:6801/2682757440":"2024-03-31T02:29:26.929980+0000","172.21.15.67:0/2119993048":"2024-03-31T02:29:26.929980+0000","172.21.15.67:6801/3801648476":"2024-03-31T02:37:02.362975+0000","172.21.15.67:6801/4186481961":"2024-03-31T02:29:01.908985+0000","172.21.15.67:0/2449045224":"2024-03-31T02:37:02.362975+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T02:47:45.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.458+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ad40a1310 msgr2=0x7f2ad40a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.458+0000 7f2af3c00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ad40a1310 0x7f2ad40a37d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2aec19a8f0 tx=0x7f2ad8009260 comp rx=0 tx=0).stop 2024-03-30T02:47:45.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.458+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 msgr2=0x7f2aec1993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.458+0000 7f2af3c00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 0x7f2aec1993f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0000c00 tx=0x7f2ae0005740 comp rx=0 tx=0).stop 2024-03-30T02:47:45.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.459+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 shutdown_connections 2024-03-30T02:47:45.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.459+0000 7f2af3c00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f2ad40a1310 0x7f2ad40a37d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.459+0000 7f2af3c00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2aec100e60 0x7f2aec1993f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.459+0000 7f2af3c00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec19dc80 0x7f2aec19e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.459+0000 7f2af3c00700 1 --2- 172.21.15.67:0/2498194145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aec105f80 0x7f2aec199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.459+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 >> 172.21.15.67:0/2498194145 conn(0x7f2aec0fc700 msgr2=0x7f2aec0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:45.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.459+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 shutdown_connections 2024-03-30T02:47:45.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.459+0000 7f2af3c00700 1 -- 172.21.15.67:0/2498194145 wait complete. 2024-03-30T02:47:45.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T02:47:45.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.679+0000 7f0e9da00700 1 -- 172.21.15.67:0/1652172817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e98101910 msgr2=0x7f0e98105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.679+0000 7f0e9da00700 1 --2- 172.21.15.67:0/1652172817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e98101910 0x7f0e98105950 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009990 tx=0x7f0e84009ca0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.680+0000 7f0e9da00700 1 -- 172.21.15.67:0/1652172817 shutdown_connections 2024-03-30T02:47:45.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.680+0000 7f0e9da00700 1 --2- 172.21.15.67:0/1652172817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e98106080 0x7f0e9810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.680+0000 7f0e9da00700 1 --2- 172.21.15.67:0/1652172817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e98100f60 0x7f0e98101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.680+0000 7f0e9da00700 1 --2- 172.21.15.67:0/1652172817 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e98101910 0x7f0e98105950 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.680+0000 7f0e9da00700 1 -- 172.21.15.67:0/1652172817 >> 172.21.15.67:0/1652172817 conn(0x7f0e980fc7c0 msgr2=0x7f0e980febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.680+0000 7f0e9da00700 1 -- 172.21.15.67:0/1652172817 shutdown_connections 2024-03-30T02:47:45.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.680+0000 7f0e9da00700 1 -- 172.21.15.67:0/1652172817 wait complete. 2024-03-30T02:47:45.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e9da00700 1 Processor -- start 2024-03-30T02:47:45.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e9da00700 1 -- start start 2024-03-30T02:47:45.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e9da00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e98100f60 0x7f0e980730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e9da00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e98106080 0x7f0e980735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e9da00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e98073b30 0x7f0e981a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e9da00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e981a5b30 con 0x7f0e98100f60 2024-03-30T02:47:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e9da00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e981a5ca0 con 0x7f0e98106080 2024-03-30T02:47:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e9da00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e981a5e60 con 0x7f0e98073b30 2024-03-30T02:47:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e96a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e98100f60 0x7f0e980730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e96000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e98106080 0x7f0e980735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e96a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e98100f60 0x7f0e980730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38270/0 (socket says 172.21.15.67:38270) 2024-03-30T02:47:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e96a00700 1 -- 172.21.15.67:0/964355133 learned_addr learned my addr 172.21.15.67:0/964355133 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:45.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.681+0000 7f0e96000700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e98106080 0x7f0e980735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:48892/0 (socket says 172.21.15.67:48892) 2024-03-30T02:47:45.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e97400700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e98073b30 0x7f0e981a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:45.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e96000700 1 -- 172.21.15.67:0/964355133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e98073b30 msgr2=0x7f0e981a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e96000700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e98073b30 0x7f0e981a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e96000700 1 -- 172.21.15.67:0/964355133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e98100f60 msgr2=0x7f0e980730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e96000700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e98100f60 0x7f0e980730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e96000700 1 -- 172.21.15.67:0/964355133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e840095f0 con 0x7f0e98106080 2024-03-30T02:47:45.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e96000700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e98106080 0x7f0e980735f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0e840098e0 tx=0x7f0e84005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:45.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e8f400700 1 -- 172.21.15.67:0/964355133 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8401d070 con 0x7f0e98106080 2024-03-30T02:47:45.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e8f400700 1 -- 172.21.15.67:0/964355133 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e84022470 con 0x7f0e98106080 2024-03-30T02:47:45.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e981a60e0 con 0x7f0e98106080 2024-03-30T02:47:45.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.682+0000 7f0e8f400700 1 -- 172.21.15.67:0/964355133 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84017650 con 0x7f0e98106080 2024-03-30T02:47:45.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.683+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e981a6560 con 0x7f0e98106080 2024-03-30T02:47:45.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.683+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e98109350 con 0x7f0e98106080 2024-03-30T02:47:45.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.686+0000 7f0e8f400700 1 -- 172.21.15.67:0/964355133 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f0e840225e0 con 0x7f0e98106080 2024-03-30T02:47:45.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.687+0000 7f0e8f400700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0e7c0a1310 0x7f0e7c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:45.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.687+0000 7f0e8f400700 1 -- 172.21.15.67:0/964355133 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9454+0+0 (secure 0 0 0) 0x7f0e840ce9a0 con 0x7f0e98106080 2024-03-30T02:47:45.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.687+0000 7f0e96a00700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0e7c0a1310 0x7f0e7c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:45.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.688+0000 7f0e96a00700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0e7c0a1310 0x7f0e7c0a37d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009d20 tx=0x7f0e88009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:45.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.689+0000 7f0e8f400700 1 -- 172.21.15.67:0/964355133 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f0e84097a80 con 0x7f0e98106080 2024-03-30T02:47:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:45 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2709853906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:47:45.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:45 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2498194145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:47:45.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:45 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2709853906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:47:45.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:45 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2498194145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:47:45.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:45 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2709853906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-30T02:47:45.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:45 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2498194145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-30T02:47:45.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.955+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0e98067af0 con 0x7f0e98106080 2024-03-30T02:47:45.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.975+0000 7f0e8f400700 1 -- 172.21.15.67:0/964355133 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0e8409b0a0 con 0x7f0e98106080 2024-03-30T02:47:45.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0e7c0a1310 msgr2=0x7f0e7c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0e7c0a1310 0x7f0e7c0a37d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009d20 tx=0x7f0e88009260 comp rx=0 tx=0).stop 2024-03-30T02:47:45.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e98106080 msgr2=0x7f0e980735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:45.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e98106080 0x7f0e980735f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0e840098e0 tx=0x7f0e84005740 comp rx=0 tx=0).stop 2024-03-30T02:47:45.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 shutdown_connections 2024-03-30T02:47:45.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f0e7c0a1310 0x7f0e7c0a37d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0e98100f60 0x7f0e980730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e98073b30 0x7f0e981a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 --2- 172.21.15.67:0/964355133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e98106080 0x7f0e980735f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 >> 172.21.15.67:0/964355133 conn(0x7f0e980fc7c0 msgr2=0x7f0e980feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 shutdown_connections 2024-03-30T02:47:45.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:45.977+0000 7f0e9da00700 1 -- 172.21.15.67:0/964355133 wait complete. 2024-03-30T02:47:45.979 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T02:47:46.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T02:47:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.214+0000 7fef94600700 1 -- 172.21.15.67:0/2742634805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8c101910 msgr2=0x7fef8c105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.214+0000 7fef94600700 1 --2- 172.21.15.67:0/2742634805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8c101910 0x7fef8c105950 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fef780098f0 tx=0x7fef78009c00 comp rx=0 tx=0).stop 2024-03-30T02:47:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.215+0000 7fef94600700 1 -- 172.21.15.67:0/2742634805 shutdown_connections 2024-03-30T02:47:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.215+0000 7fef94600700 1 --2- 172.21.15.67:0/2742634805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c100f60 0x7fef8c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.215+0000 7fef94600700 1 --2- 172.21.15.67:0/2742634805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef8c106080 0x7fef8c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:46.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.215+0000 7fef94600700 1 --2- 172.21.15.67:0/2742634805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8c101910 0x7fef8c105950 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.215+0000 7fef94600700 1 -- 172.21.15.67:0/2742634805 >> 172.21.15.67:0/2742634805 conn(0x7fef8c0fc7c0 msgr2=0x7fef8c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.215+0000 7fef94600700 1 -- 172.21.15.67:0/2742634805 shutdown_connections 2024-03-30T02:47:46.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.215+0000 7fef94600700 1 -- 172.21.15.67:0/2742634805 wait complete. 2024-03-30T02:47:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.216+0000 7fef94600700 1 Processor -- start 2024-03-30T02:47:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.216+0000 7fef94600700 1 -- start start 2024-03-30T02:47:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef94600700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef8c100f60 0x7fef8c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:46.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef94600700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c106080 0x7fef8c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef94600700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8c19de60 0x7fef8c19e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef94600700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c19e8e0 con 0x7fef8c106080 2024-03-30T02:47:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef94600700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c19ea50 con 0x7fef8c100f60 2024-03-30T02:47:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef94600700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c19ec10 con 0x7fef8c19de60 2024-03-30T02:47:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef91c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef8c100f60 0x7fef8c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef91c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef8c100f60 0x7fef8c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.67:38906/0 (socket says 172.21.15.67:38906) 2024-03-30T02:47:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef91c00700 1 -- 172.21.15.67:0/1149868618 learned_addr learned my addr 172.21.15.67:0/1149868618 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:46.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef91200700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c106080 0x7fef8c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.217+0000 7fef92600700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8c19de60 0x7fef8c19e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:46.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef91c00700 1 -- 172.21.15.67:0/1149868618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8c19de60 msgr2=0x7fef8c19e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef91c00700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8c19de60 0x7fef8c19e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef91c00700 1 -- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c106080 msgr2=0x7fef8c199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef91c00700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c106080 0x7fef8c199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef91c00700 1 -- 172.21.15.67:0/1149868618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800095f0 con 0x7fef8c100f60 2024-03-30T02:47:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef91c00700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef8c100f60 0x7fef8c199580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef8000ec10 tx=0x7fef8000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef8a000700 1 -- 172.21.15.67:0/1149868618 <== mon.2 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8000cc90 con 0x7fef8c100f60 2024-03-30T02:47:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef8a000700 1 -- 172.21.15.67:0/1149868618 <== mon.2 v2:172.21.15.82:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef800044e0 con 0x7fef8c100f60 2024-03-30T02:47:46.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef78009520 con 0x7fef8c100f60 2024-03-30T02:47:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef8a000700 1 -- 172.21.15.67:0/1149868618 <== mon.2 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80010430 con 0x7fef8c100f60 2024-03-30T02:47:46.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.218+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c1a64c0 con 0x7fef8c100f60 2024-03-30T02:47:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.219+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c102430 con 0x7fef8c100f60 2024-03-30T02:47:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.221+0000 7fef8a000700 1 -- 172.21.15.67:0/1149868618 <== mon.2 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fef80003fb0 con 0x7fef8c100f60 2024-03-30T02:47:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.222+0000 7fef8a000700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fef740a1290 0x7fef740a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:46.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.223+0000 7fef8a000700 1 -- 172.21.15.67:0/1149868618 <== mon.2 v2:172.21.15.82:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9796+0+0 (secure 0 0 0) 0x7fef80014070 con 0x7fef8c100f60 2024-03-30T02:47:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.223+0000 7fef91200700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fef740a1290 0x7fef740a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.223+0000 7fef91200700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fef740a1290 0x7fef740a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef78005e90 tx=0x7fef78019040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:46.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.228+0000 7fef8a000700 1 -- 172.21.15.67:0/1149868618 <== mon.2 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fef80096660 con 0x7fef8c100f60 2024-03-30T02:47:46.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:45 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-a[37785]: 2024-03-30T02:47:45.957+0000 7f8566200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T02:47:46.413 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-a[87694]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.413 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-a[87694]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.414 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mds[87720]: -- 172.21.15.67:0/1170455123 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b8e81921a0 con 0x55b8e816b000 2024-03-30T02:47:46.414 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-a[87694]: starting mds.a at 2024-03-30T02:47:46.414 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-g[87313]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.415 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-g[87313]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.415 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mds[87355]: -- 172.21.15.67:0/1123680774 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55acc2f541a0 con 0x55acc2f2c800 2024-03-30T02:47:46.415 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-g[87313]: starting mds.g at 2024-03-30T02:47:46.439 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-c[116770]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.439 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-c[116770]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.439 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-c[116770]: starting mds.c at 2024-03-30T02:47:46.440 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-l[116080]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.440 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-l[116080]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.440 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-l[116080]: starting mds.l at 2024-03-30T02:47:46.440 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-f[116430]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.440 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-f[116430]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.441 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mds[116453]: -- 172.21.15.170:0/3711665586 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561d9f92c1a0 con 0x561d9f902800 2024-03-30T02:47:46.441 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-f[116430]: starting mds.f at 2024-03-30T02:47:46.441 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-i[117110]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.441 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-i[117110]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.441 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mds[117133]: -- 172.21.15.170:0/3194262615 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5614697721a0 con 0x561469748c00 2024-03-30T02:47:46.441 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-i[117110]: starting mds.i at 2024-03-30T02:47:46.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.494+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fef8c19a920 con 0x7fef8c100f60 2024-03-30T02:47:46.495 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.496 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.496 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mds[115865]: -- 172.21.15.82:0/1454725340 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56339d98a1a0 con 0x56339d960800 2024-03-30T02:47:46.496 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: starting mds.b at 2024-03-30T02:47:46.496 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-e[115157]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.496 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-e[115157]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.496 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mds[115180]: -- 172.21.15.82:0/716546361 <== mon.2 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560fcc58e1a0 con 0x560fcc565000 2024-03-30T02:47:46.497 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-e[115157]: starting mds.e at 2024-03-30T02:47:46.497 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-h[115499]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.497 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-h[115499]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.497 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-h[115499]: starting mds.h at 2024-03-30T02:47:46.497 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-h[115499]: 2024-03-30T02:47:46.030+0000 7fa93d2ce700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-03-30T02:47:46.498 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-k[114817]: ignoring --setuser ceph since I am not root 2024-03-30T02:47:46.498 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-k[114817]: ignoring --setgroup ceph since I am not root 2024-03-30T02:47:46.498 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-k[114817]: starting mds.k at 2024-03-30T02:47:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mon[79176]: pgmap v332: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-03-30T02:47:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/964355133' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-30T02:47:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-30T02:47:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mon[79176]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-30T02:47:46.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mon[79176]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T02:47:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mon[79176]: osdmap e97: 12 total, 12 up, 12 in 2024-03-30T02:47:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-30T02:47:46.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:46 smithi082 ceph-mon[79176]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-30T02:47:46.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mon[37808]: pgmap v332: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-03-30T02:47:46.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/964355133' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-30T02:47:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-30T02:47:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mon[37808]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-30T02:47:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mon[37808]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T02:47:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mon[37808]: osdmap e97: 12 total, 12 up, 12 in 2024-03-30T02:47:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-30T02:47:46.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:46 smithi067 ceph-mon[37808]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-30T02:47:46.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mon[78798]: pgmap v332: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-03-30T02:47:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/964355133' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-30T02:47:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-30T02:47:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mon[78798]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-30T02:47:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mon[78798]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T02:47:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mon[78798]: osdmap e97: 12 total, 12 up, 12 in 2024-03-30T02:47:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-30T02:47:46.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:46 smithi170 ceph-mon[78798]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-30T02:47:46.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.973+0000 7fef8a000700 1 -- 172.21.15.67:0/1149868618 <== mon.2 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fef80099c80 con 0x7fef8c100f60 2024-03-30T02:47:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fef740a1290 msgr2=0x7fef740a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fef740a1290 0x7fef740a3750 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef78005e90 tx=0x7fef78019040 comp rx=0 tx=0).stop 2024-03-30T02:47:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef8c100f60 msgr2=0x7fef8c199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:46.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef8c100f60 0x7fef8c199580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef8000ec10 tx=0x7fef8000ef20 comp rx=0 tx=0).stop 2024-03-30T02:47:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 shutdown_connections 2024-03-30T02:47:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fef740a1290 0x7fef740a3750 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef8c19de60 0x7fef8c19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef8c100f60 0x7fef8c199580 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 --2- 172.21.15.67:0/1149868618 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fef8c106080 0x7fef8c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:46.977 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.976+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 >> 172.21.15.67:0/1149868618 conn(0x7fef8c0fc7c0 msgr2=0x7fef8c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.977+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 shutdown_connections 2024-03-30T02:47:46.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:46.977+0000 7fef94600700 1 -- 172.21.15.67:0/1149868618 wait complete. 2024-03-30T02:47:46.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T02:47:47.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.203+0000 7f6476a00700 1 -- 172.21.15.67:0/3294652575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6470100f60 msgr2=0x7f6470101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:47.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.203+0000 7f6476a00700 1 --2- 172.21.15.67:0/3294652575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6470100f60 0x7f6470101340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f64600098f0 tx=0x7f6460009c00 comp rx=0 tx=0).stop 2024-03-30T02:47:47.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.203+0000 7f6476a00700 1 -- 172.21.15.67:0/3294652575 shutdown_connections 2024-03-30T02:47:47.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.203+0000 7f6476a00700 1 --2- 172.21.15.67:0/3294652575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6470106080 0x7f647010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:47.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.203+0000 7f6476a00700 1 --2- 172.21.15.67:0/3294652575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6470100f60 0x7f6470101340 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:47.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.203+0000 7f6476a00700 1 --2- 172.21.15.67:0/3294652575 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6470101910 0x7f6470105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:47.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.203+0000 7f6476a00700 1 -- 172.21.15.67:0/3294652575 >> 172.21.15.67:0/3294652575 conn(0x7f64700fc7c0 msgr2=0x7f64700febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:47.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.203+0000 7f6476a00700 1 -- 172.21.15.67:0/3294652575 shutdown_connections 2024-03-30T02:47:47.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.203+0000 7f6476a00700 1 -- 172.21.15.67:0/3294652575 wait complete. 2024-03-30T02:47:47.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.204+0000 7f6476a00700 1 Processor -- start 2024-03-30T02:47:47.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f6476a00700 1 -- start start 2024-03-30T02:47:47.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f6476a00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6470100f60 0x7f64700730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:47.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f6476a00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6470106080 0x7f64700735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:47.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f6476a00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6470073b30 0x7f64701a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:47.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f6476a00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64701a5a50 con 0x7f6470106080 2024-03-30T02:47:47.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f6476a00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64701a5bc0 con 0x7f6470073b30 2024-03-30T02:47:47.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f6476a00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64701a5d80 con 0x7f6470100f60 2024-03-30T02:47:47.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f646f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6470106080 0x7f64700735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:47.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f646f400700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6470106080 0x7f64700735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56508/0 (socket says 172.21.15.67:56508) 2024-03-30T02:47:47.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f646f400700 1 -- 172.21.15.67:0/2023703190 learned_addr learned my addr 172.21.15.67:0/2023703190 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:47.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f646fe00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6470100f60 0x7f64700730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:47.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.205+0000 7f6474a00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6470073b30 0x7f64701a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646f400700 1 -- 172.21.15.67:0/2023703190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6470100f60 msgr2=0x7f64700730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646f400700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6470100f60 0x7f64700730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646f400700 1 -- 172.21.15.67:0/2023703190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6470073b30 msgr2=0x7f64701a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646f400700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6470073b30 0x7f64701a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646f400700 1 -- 172.21.15.67:0/2023703190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0095f0 con 0x7f6470106080 2024-03-30T02:47:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f6474a00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6470073b30 0x7f64701a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646fe00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6470100f60 0x7f64700730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T02:47:47.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646f400700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6470106080 0x7f64700735f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f645c00ec70 tx=0x7f645c00ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:47.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646cc00700 1 -- 172.21.15.67:0/2023703190 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0099d0 con 0x7f6470106080 2024-03-30T02:47:47.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646cc00700 1 -- 172.21.15.67:0/2023703190 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f645c0044e0 con 0x7f6470106080 2024-03-30T02:47:47.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6460009520 con 0x7f6470106080 2024-03-30T02:47:47.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f646cc00700 1 -- 172.21.15.67:0/2023703190 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c010430 con 0x7f6470106080 2024-03-30T02:47:47.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.206+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64701a63e0 con 0x7f6470106080 2024-03-30T02:47:47.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.207+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64701092a0 con 0x7f6470106080 2024-03-30T02:47:47.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.209+0000 7f646cc00700 1 -- 172.21.15.67:0/2023703190 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f645c003f10 con 0x7f6470106080 2024-03-30T02:47:47.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.210+0000 7f646cc00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f64540a1290 0x7f64540a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:47.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.211+0000 7f646cc00700 1 -- 172.21.15.67:0/2023703190 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9796+0+0 (secure 0 0 0) 0x7f645c014070 con 0x7f6470106080 2024-03-30T02:47:47.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.211+0000 7f646fe00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f64540a1290 0x7f64540a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:47.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.211+0000 7f646fe00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f64540a1290 0x7f64540a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64600098c0 tx=0x7f6460019040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.216+0000 7f646cc00700 1 -- 172.21.15.67:0/2023703190 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f645c096610 con 0x7f6470106080 2024-03-30T02:47:47.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.434+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6470074590 con 0x7f6470106080 2024-03-30T02:47:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/1149868618' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:47:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:47:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-30T02:47:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-30T02:47:47.749 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: Cluster is now healthy 2024-03-30T02:47:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6832/2952064491,v1:172.21.15.170:6834/2952064491] up:boot 2024-03-30T02:47:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6833/1742644933,v1:172.21.15.170:6835/1742644933] up:boot 2024-03-30T02:47:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6836/3038754581,v1:172.21.15.82:6837/3038754581] up:boot 2024-03-30T02:47:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6838/3126982892,v1:172.21.15.170:6839/3126982892] up:boot 2024-03-30T02:47:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6832/208694284,v1:172.21.15.82:6834/208694284] up:boot 2024-03-30T02:47:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6833/284850690,v1:172.21.15.82:6835/284850690] up:boot 2024-03-30T02:47:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.170:6836/2366702820,v1:172.21.15.170:6837/2366702820] up:boot 2024-03-30T02:47:47.750 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.82:6838/1460130058,v1:172.21.15.82:6839/1460130058] up:boot 2024-03-30T02:47:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.67:6838/3283060865,v1:172.21.15.67:6840/3283060865] up:boot 2024-03-30T02:47:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: mds.? [v2:172.21.15.67:6839/1512171676,v1:172.21.15.67:6841/1512171676] up:boot 2024-03-30T02:47:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-30T02:47:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: fsmap 12 up:standby 2024-03-30T02:47:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T02:47:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T02:47:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T02:47:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T02:47:47.751 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T02:47:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T02:47:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T02:47:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T02:47:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T02:47:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T02:47:47.752 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:47 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2023703190' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:47.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/1149868618' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:47:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:47:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-30T02:47:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-30T02:47:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: Cluster is now healthy 2024-03-30T02:47:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6832/2952064491,v1:172.21.15.170:6834/2952064491] up:boot 2024-03-30T02:47:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6833/1742644933,v1:172.21.15.170:6835/1742644933] up:boot 2024-03-30T02:47:47.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6836/3038754581,v1:172.21.15.82:6837/3038754581] up:boot 2024-03-30T02:47:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6838/3126982892,v1:172.21.15.170:6839/3126982892] up:boot 2024-03-30T02:47:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6832/208694284,v1:172.21.15.82:6834/208694284] up:boot 2024-03-30T02:47:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6833/284850690,v1:172.21.15.82:6835/284850690] up:boot 2024-03-30T02:47:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.170:6836/2366702820,v1:172.21.15.170:6837/2366702820] up:boot 2024-03-30T02:47:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.82:6838/1460130058,v1:172.21.15.82:6839/1460130058] up:boot 2024-03-30T02:47:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.67:6838/3283060865,v1:172.21.15.67:6840/3283060865] up:boot 2024-03-30T02:47:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: mds.? [v2:172.21.15.67:6839/1512171676,v1:172.21.15.67:6841/1512171676] up:boot 2024-03-30T02:47:47.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-30T02:47:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: fsmap 12 up:standby 2024-03-30T02:47:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T02:47:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T02:47:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T02:47:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T02:47:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T02:47:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T02:47:47.915 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T02:47:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T02:47:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T02:47:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T02:47:47.916 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:47 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2023703190' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:47.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/1149868618' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:47:47.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-30T02:47:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-30T02:47:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-30T02:47:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: Cluster is now healthy 2024-03-30T02:47:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6832/2952064491,v1:172.21.15.170:6834/2952064491] up:boot 2024-03-30T02:47:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6833/1742644933,v1:172.21.15.170:6835/1742644933] up:boot 2024-03-30T02:47:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6836/3038754581,v1:172.21.15.82:6837/3038754581] up:boot 2024-03-30T02:47:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6838/3126982892,v1:172.21.15.170:6839/3126982892] up:boot 2024-03-30T02:47:47.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6832/208694284,v1:172.21.15.82:6834/208694284] up:boot 2024-03-30T02:47:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6833/284850690,v1:172.21.15.82:6835/284850690] up:boot 2024-03-30T02:47:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.170:6836/2366702820,v1:172.21.15.170:6837/2366702820] up:boot 2024-03-30T02:47:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.82:6838/1460130058,v1:172.21.15.82:6839/1460130058] up:boot 2024-03-30T02:47:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.67:6838/3283060865,v1:172.21.15.67:6840/3283060865] up:boot 2024-03-30T02:47:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: mds.? [v2:172.21.15.67:6839/1512171676,v1:172.21.15.67:6841/1512171676] up:boot 2024-03-30T02:47:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-30T02:47:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: fsmap 12 up:standby 2024-03-30T02:47:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-30T02:47:47.940 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-30T02:47:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-30T02:47:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-30T02:47:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-30T02:47:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-30T02:47:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-30T02:47:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-30T02:47:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-30T02:47:47.941 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='mgr.25634 172.21.15.67:0/792720386' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-30T02:47:47.942 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:47 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2023703190' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:47.975 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.976+0000 7f646cc00700 1 -- 172.21.15.67:0/2023703190 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7f645c099c30 con 0x7f6470106080 2024-03-30T02:47:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.979+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f64540a1290 msgr2=0x7f64540a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.979+0000 7f6476a00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f64540a1290 0x7f64540a3750 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f64600098c0 tx=0x7f6460019040 comp rx=0 tx=0).stop 2024-03-30T02:47:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.979+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6470106080 msgr2=0x7f64700735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.979+0000 7f6476a00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6470106080 0x7f64700735f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f645c00ec70 tx=0x7f645c00ef80 comp rx=0 tx=0).stop 2024-03-30T02:47:47.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.980+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 shutdown_connections 2024-03-30T02:47:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.980+0000 7f6476a00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f64540a1290 0x7f64540a3750 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.980+0000 7f6476a00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6470073b30 0x7f64701a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.980+0000 7f6476a00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6470100f60 0x7f64700730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.980+0000 7f6476a00700 1 --2- 172.21.15.67:0/2023703190 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6470106080 0x7f64700735f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.980+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 >> 172.21.15.67:0/2023703190 conn(0x7f64700fc7c0 msgr2=0x7f64700fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.980+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 shutdown_connections 2024-03-30T02:47:47.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:47.980+0000 7f6476a00700 1 -- 172.21.15.67:0/2023703190 wait complete. 2024-03-30T02:47:47.981 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-03-30T02:47:48.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T02:47:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.216+0000 7fc77e200700 1 -- 172.21.15.67:0/3016117126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc778106080 msgr2=0x7fc77810dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.216+0000 7fc77e200700 1 --2- 172.21.15.67:0/3016117126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc778106080 0x7fc77810dc00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc77000b0a0 tx=0x7fc77000b3b0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.217+0000 7fc77e200700 1 -- 172.21.15.67:0/3016117126 shutdown_connections 2024-03-30T02:47:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.217+0000 7fc77e200700 1 --2- 172.21.15.67:0/3016117126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc778106080 0x7fc77810dc00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.217+0000 7fc77e200700 1 --2- 172.21.15.67:0/3016117126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778100f60 0x7fc778101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.217+0000 7fc77e200700 1 --2- 172.21.15.67:0/3016117126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778101910 0x7fc778105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.217+0000 7fc77e200700 1 -- 172.21.15.67:0/3016117126 >> 172.21.15.67:0/3016117126 conn(0x7fc7780fc7c0 msgr2=0x7fc7780febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:48.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.217+0000 7fc77e200700 1 -- 172.21.15.67:0/3016117126 shutdown_connections 2024-03-30T02:47:48.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.217+0000 7fc77e200700 1 -- 172.21.15.67:0/3016117126 wait complete. 2024-03-30T02:47:48.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.218+0000 7fc77e200700 1 Processor -- start 2024-03-30T02:47:48.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.218+0000 7fc77e200700 1 -- start start 2024-03-30T02:47:48.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.218+0000 7fc77e200700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778100f60 0x7fc778199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:48.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.218+0000 7fc77e200700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778106080 0x7fc778199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:48.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.218+0000 7fc77e200700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc77819de20 0x7fc77819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:48.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.218+0000 7fc77e200700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77819e8a0 con 0x7fc77819de20 2024-03-30T02:47:48.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.218+0000 7fc77e200700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77819ea10 con 0x7fc778100f60 2024-03-30T02:47:48.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.218+0000 7fc77e200700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77819ebd0 con 0x7fc778106080 2024-03-30T02:47:48.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.218+0000 7fc777e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc77819de20 0x7fc77819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:48.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc777e00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc77819de20 0x7fc77819e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56538/0 (socket says 172.21.15.67:56538) 2024-03-30T02:47:48.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc777400700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778100f60 0x7fc778199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:48.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc777e00700 1 -- 172.21.15.67:0/457702655 learned_addr learned my addr 172.21.15.67:0/457702655 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:48.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc776a00700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778106080 0x7fc778199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:48.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc777e00700 1 -- 172.21.15.67:0/457702655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778106080 msgr2=0x7fc778199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:48.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc777e00700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778106080 0x7fc778199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc777e00700 1 -- 172.21.15.67:0/457702655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778100f60 msgr2=0x7fc778199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:48.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc777e00700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778100f60 0x7fc778199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc777e00700 1 -- 172.21.15.67:0/457702655 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc770009d00 con 0x7fc77819de20 2024-03-30T02:47:48.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc777e00700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc77819de20 0x7fc77819e2a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc77000b070 tx=0x7fc77000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:48.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc76fe00700 1 -- 172.21.15.67:0/457702655 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77000e040 con 0x7fc77819de20 2024-03-30T02:47:48.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc76fe00700 1 -- 172.21.15.67:0/457702655 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc77001d480 con 0x7fc77819de20 2024-03-30T02:47:48.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc76fe00700 1 -- 172.21.15.67:0/457702655 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700052a0 con 0x7fc77819de20 2024-03-30T02:47:48.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.219+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7781a60d0 con 0x7fc77819de20 2024-03-30T02:47:48.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.220+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7781a64a0 con 0x7fc77819de20 2024-03-30T02:47:48.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.221+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc778102430 con 0x7fc77819de20 2024-03-30T02:47:48.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.223+0000 7fc76fe00700 1 -- 172.21.15.67:0/457702655 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7fc77001d5f0 con 0x7fc77819de20 2024-03-30T02:47:48.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.225+0000 7fc76fe00700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc75c0a1310 0x7fc75c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:48.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.225+0000 7fc76fe00700 1 -- 172.21.15.67:0/457702655 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9263+0+0 (secure 0 0 0) 0x7fc7700cdfe0 con 0x7fc77819de20 2024-03-30T02:47:48.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.225+0000 7fc777400700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc75c0a1310 0x7fc75c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:48.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.226+0000 7fc777400700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc75c0a1310 0x7fc75c0a37d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc768009c60 tx=0x7fc768009260 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:48.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.227+0000 7fc76fe00700 1 -- 172.21.15.67:0/457702655 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7fc770097200 con 0x7fc77819de20 2024-03-30T02:47:48.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.446+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc778067af0 con 0x7fc77819de20 2024-03-30T02:47:48.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:48 smithi067 ceph-mon[37808]: pgmap v334: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-03-30T02:47:48.912 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:48 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/2023703190' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-30T02:47:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:48 smithi067 ceph-mon[37808]: osdmap e98: 12 total, 12 up, 12 in 2024-03-30T02:47:48.913 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:48 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/457702655' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:48.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:48 smithi170 ceph-mon[78798]: pgmap v334: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-03-30T02:47:48.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:48 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/2023703190' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-30T02:47:48.938 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:48 smithi170 ceph-mon[78798]: osdmap e98: 12 total, 12 up, 12 in 2024-03-30T02:47:48.939 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:48 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/457702655' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:48.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.981+0000 7fc76fe00700 1 -- 172.21.15.67:0/457702655 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc77009a820 con 0x7fc77819de20 2024-03-30T02:47:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc75c0a1310 msgr2=0x7fc75c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc75c0a1310 0x7fc75c0a37d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc768009c60 tx=0x7fc768009260 comp rx=0 tx=0).stop 2024-03-30T02:47:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc77819de20 msgr2=0x7fc77819e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc77819de20 0x7fc77819e2a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc77000b070 tx=0x7fc77000bda0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 shutdown_connections 2024-03-30T02:47:48.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7fc75c0a1310 0x7fc75c0a37d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc77819de20 0x7fc77819e2a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc778100f60 0x7fc778199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 --2- 172.21.15.67:0/457702655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc778106080 0x7fc778199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:48.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 >> 172.21.15.67:0/457702655 conn(0x7fc7780fc7c0 msgr2=0x7fc7780feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:48.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 shutdown_connections 2024-03-30T02:47:48.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:48.986+0000 7fc77e200700 1 -- 172.21.15.67:0/457702655 wait complete. 2024-03-30T02:47:48.987 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-03-30T02:47:48.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:48 smithi082 ceph-mon[79176]: pgmap v334: 129 pgs: 129 active+clean; 1.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-03-30T02:47:48.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:48 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/2023703190' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-30T02:47:48.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:48 smithi082 ceph-mon[79176]: osdmap e98: 12 total, 12 up, 12 in 2024-03-30T02:47:48.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:48 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/457702655' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:49.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-30T02:47:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.370+0000 7f5e41c00700 1 -- 172.21.15.67:0/2327001387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e3c11a590 msgr2=0x7f5e3c11c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.370+0000 7f5e41c00700 1 --2- 172.21.15.67:0/2327001387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e3c11a590 0x7f5e3c11c980 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5e3401c0d0 tx=0x7f5e3401c3e0 comp rx=0 tx=0).stop 2024-03-30T02:47:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.370+0000 7f5e41c00700 1 -- 172.21.15.67:0/2327001387 shutdown_connections 2024-03-30T02:47:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.370+0000 7f5e41c00700 1 --2- 172.21.15.67:0/2327001387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e3c10f540 0x7f5e3c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.370+0000 7f5e41c00700 1 --2- 172.21.15.67:0/2327001387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e3c11a590 0x7f5e3c11c980 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.370+0000 7f5e41c00700 1 --2- 172.21.15.67:0/2327001387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c1082f0 0x7f5e3c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.370+0000 7f5e41c00700 1 -- 172.21.15.67:0/2327001387 >> 172.21.15.67:0/2327001387 conn(0x7f5e3c06bdf0 msgr2=0x7f5e3c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.370+0000 7f5e41c00700 1 -- 172.21.15.67:0/2327001387 shutdown_connections 2024-03-30T02:47:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.371+0000 7f5e41c00700 1 -- 172.21.15.67:0/2327001387 wait complete. 2024-03-30T02:47:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.372+0000 7f5e41c00700 1 Processor -- start 2024-03-30T02:47:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.372+0000 7f5e41c00700 1 -- start start 2024-03-30T02:47:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.373+0000 7f5e41c00700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e3c1082f0 0x7f5e3c119a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.373+0000 7f5e41c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c10f540 0x7f5e3c119f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.373+0000 7f5e41c00700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e3c11a590 0x7f5e3c115bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.373+0000 7f5e41c00700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c116370 con 0x7f5e3c1082f0 2024-03-30T02:47:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.373+0000 7f5e41c00700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c1164e0 con 0x7f5e3c11a590 2024-03-30T02:47:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.373+0000 7f5e41c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c1166a0 con 0x7f5e3c10f540 2024-03-30T02:47:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.373+0000 7f5e3a000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c10f540 0x7f5e3c119f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.373+0000 7f5e3a000700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c10f540 0x7f5e3c119f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.67:41652/0 (socket says 172.21.15.67:41652) 2024-03-30T02:47:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.373+0000 7f5e3a000700 1 -- 172.21.15.67:0/3233269079 learned_addr learned my addr 172.21.15.67:0/3233269079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-03-30T02:47:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.374+0000 7f5e3b400700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e3c11a590 0x7f5e3c115bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.374+0000 7f5e3aa00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e3c1082f0 0x7f5e3c119a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.374+0000 7f5e3a000700 1 -- 172.21.15.67:0/3233269079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e3c11a590 msgr2=0x7f5e3c115bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.374+0000 7f5e3a000700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e3c11a590 0x7f5e3c115bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.374+0000 7f5e3a000700 1 -- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e3c1082f0 msgr2=0x7f5e3c119a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:49.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.374+0000 7f5e3a000700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e3c1082f0 0x7f5e3c119a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.374+0000 7f5e3a000700 1 -- 172.21.15.67:0/3233269079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3401ad10 con 0x7f5e3c10f540 2024-03-30T02:47:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.374+0000 7f5e3aa00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e3c1082f0 0x7f5e3c119a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T02:47:49.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.375+0000 7f5e3a000700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c10f540 0x7f5e3c119f60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800d980 tx=0x7f5e2800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.376+0000 7f5e33400700 1 -- 172.21.15.67:0/3233269079 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28004a30 con 0x7f5e3c10f540 2024-03-30T02:47:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.376+0000 7f5e33400700 1 -- 172.21.15.67:0/3233269079 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e280054e0 con 0x7f5e3c10f540 2024-03-30T02:47:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.376+0000 7f5e33400700 1 -- 172.21.15.67:0/3233269079 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28009c90 con 0x7f5e3c10f540 2024-03-30T02:47:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.376+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c1aa230 con 0x7f5e3c10f540 2024-03-30T02:47:49.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.376+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c1aa5a0 con 0x7f5e3c10f540 2024-03-30T02:47:49.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.377+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c06d270 con 0x7f5e3c10f540 2024-03-30T02:47:49.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.381+0000 7f5e33400700 1 -- 172.21.15.67:0/3233269079 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 28) v1 ==== 135996+0+0 (secure 0 0 0) 0x7f5e28004c20 con 0x7f5e3c10f540 2024-03-30T02:47:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.384+0000 7f5e33400700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e200a14a0 0x7f5e200a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T02:47:49.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.385+0000 7f5e3aa00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e200a14a0 0x7f5e200a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T02:47:49.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.385+0000 7f5e33400700 1 -- 172.21.15.67:0/3233269079 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8704+0+0 (secure 0 0 0) 0x7f5e280ccb90 con 0x7f5e3c10f540 2024-03-30T02:47:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.386+0000 7f5e33400700 1 -- 172.21.15.67:0/3233269079 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182738 (secure 0 0 0) 0x7f5e28095ef0 con 0x7f5e3c10f540 2024-03-30T02:47:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.386+0000 7f5e3aa00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e200a14a0 0x7f5e200a3960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c007900 tx=0x7f5e2c008040 comp rx=0 tx=0).ready entity=mgr.25634 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T02:47:49.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:49.699+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5e3c1123c0 con 0x7f5e3c10f540 2024-03-30T02:47:50.249 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:49 smithi082 ceph-mon[79176]: pgmap v336: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:47:50.249 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:49 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/457702655' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-30T02:47:50.249 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:49 smithi082 ceph-mon[79176]: osdmap e99: 12 total, 12 up, 12 in 2024-03-30T02:47:50.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:49 smithi082 ceph-mon[79176]: from='client.? 172.21.15.67:0/3233269079' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:50.250 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:49 smithi082 ceph-mon[79176]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:50.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:49 smithi067 ceph-mon[37808]: pgmap v336: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:47:50.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:49 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/457702655' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-30T02:47:50.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:49 smithi067 ceph-mon[37808]: osdmap e99: 12 total, 12 up, 12 in 2024-03-30T02:47:50.412 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:49 smithi067 ceph-mon[37808]: from='client.? 172.21.15.67:0/3233269079' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:50.413 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:49 smithi067 ceph-mon[37808]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:50.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:49 smithi170 ceph-mon[78798]: pgmap v336: 97 pgs: 97 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-03-30T02:47:50.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:49 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/457702655' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-30T02:47:50.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:49 smithi170 ceph-mon[78798]: osdmap e99: 12 total, 12 up, 12 in 2024-03-30T02:47:50.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:49 smithi170 ceph-mon[78798]: from='client.? 172.21.15.67:0/3233269079' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:50.439 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:49 smithi170 ceph-mon[78798]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-30T02:47:50.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.507+0000 7f5e33400700 1 -- 172.21.15.67:0/3233269079 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v100) v1 ==== 178+0+0 (secure 0 0 0) 0x7f5e28017070 con 0x7f5e3c10f540 2024-03-30T02:47:50.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.511+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e200a14a0 msgr2=0x7f5e200a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.511+0000 7f5e41c00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e200a14a0 0x7f5e200a3960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c007900 tx=0x7f5e2c008040 comp rx=0 tx=0).stop 2024-03-30T02:47:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c10f540 msgr2=0x7f5e3c119f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T02:47:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c10f540 0x7f5e3c119f60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800d980 tx=0x7f5e2800dc90 comp rx=0 tx=0).stop 2024-03-30T02:47:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 shutdown_connections 2024-03-30T02:47:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:6800/1710497985,v1:172.21.15.67:6801/1710497985] conn(0x7f5e200a14a0 0x7f5e200a3960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e3c1082f0 0x7f5e3c119a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e3c11a590 0x7f5e3c115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:50.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 --2- 172.21.15.67:0/3233269079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e3c10f540 0x7f5e3c119f60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T02:47:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 >> 172.21.15.67:0/3233269079 conn(0x7f5e3c06bdf0 msgr2=0x7f5e3c11a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T02:47:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 shutdown_connections 2024-03-30T02:47:50.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-03-30T02:47:50.512+0000 7f5e41c00700 1 -- 172.21.15.67:0/3233269079 wait complete. 2024-03-30T02:47:50.513 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' removed 2024-03-30T02:47:50.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T02:47:50.625 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T02:47:50.662 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T02:47:50.713 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-30T02:47:50.713 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-30T02:47:50.772 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-30T02:47:50.793 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-30T02:47:50.813 INFO:tasks.cephadm:Stopping all daemons... 2024-03-30T02:47:50.813 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-30T02:47:50.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.a 2024-03-30T02:47:51.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:50 smithi067 systemd[1]: Stopping Ceph mon.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:47:51.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:51 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-a[37785]: 2024-03-30T02:47:51.409+0000 7f856d000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:47:51.663 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:51 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-a[37785]: 2024-03-30T02:47:51.409+0000 7f856d000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-30T02:47:52.163 INFO:journalctl@ceph.mon.a.smithi067.stdout:Mar 30 02:47:51 smithi067 bash[131279]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-a 2024-03-30T02:47:52.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.a.service' 2024-03-30T02:47:52.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:47:52.420 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-30T02:47:52.420 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-30T02:47:52.421 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.b 2024-03-30T02:47:52.731 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:52 smithi082 systemd[1]: Stopping Ceph mon.b for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:47:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-b[79153]: 2024-03-30T02:47:52.731+0000 7f71470d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:47:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-b[79153]: 2024-03-30T02:47:52.731+0000 7f71470d9700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-30T02:47:52.999 INFO:journalctl@ceph.mon.b.smithi082.stdout:Mar 30 02:47:52 smithi082 bash[121793]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-b 2024-03-30T02:47:53.253 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.b.service' 2024-03-30T02:47:53.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:47:53.308 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-30T02:47:53.308 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-30T02:47:53.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.c 2024-03-30T02:47:53.688 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:53 smithi170 systemd[1]: Stopping Ceph mon.c for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:47:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-c[78766]: 2024-03-30T02:47:53.549+0000 7f7396b31700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:47:53.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-c[78766]: 2024-03-30T02:47:53.549+0000 7f7396b31700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-30T02:47:54.086 INFO:journalctl@ceph.mon.c.smithi170.stdout:Mar 30 02:47:53 smithi170 bash[123323]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mon-c 2024-03-30T02:47:54.094 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mon.c.service' 2024-03-30T02:47:54.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:47:54.204 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-30T02:47:54.204 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-30T02:47:54.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.x 2024-03-30T02:47:54.583 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:47:54 smithi067 systemd[1]: Stopping Ceph mgr.x for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:47:54.833 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:47:54 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x[38057]: 2024-03-30T02:47:54.582+0000 7f6c21a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:47:54.834 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Mar 30 02:47:54 smithi067 bash[131424]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-x 2024-03-30T02:47:55.215 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.x.service' 2024-03-30T02:47:55.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:47:55.306 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-30T02:47:55.306 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-30T02:47:55.306 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.y 2024-03-30T02:47:55.707 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:47:55 smithi082 systemd[1]: Stopping Ceph mgr.y for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:47:55.999 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:47:55 smithi082 bash[121934]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-y 2024-03-30T02:47:55.999 INFO:journalctl@ceph.mgr.y.smithi082.stdout:Mar 30 02:47:55 smithi082 systemd[1]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-30T02:47:56.013 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.y.service' 2024-03-30T02:47:56.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:47:56.050 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-30T02:47:56.050 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-30T02:47:56.050 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.z 2024-03-30T02:47:56.403 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:47:56 smithi170 systemd[1]: Stopping Ceph mgr.z for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:47:56.688 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:47:56 smithi170 bash[123464]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mgr-z 2024-03-30T02:47:56.689 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Mar 30 02:47:56 smithi170 systemd[1]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-30T02:47:56.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mgr.z.service' 2024-03-30T02:47:56.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:47:56.776 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-30T02:47:56.776 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-30T02:47:56.776 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.0 2024-03-30T02:47:57.139 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:47:56 smithi067 systemd[1]: Stopping Ceph osd.0 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:47:57.412 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:47:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-0[56175]: 2024-03-30T02:47:57.139+0000 7ff329000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:47:57.412 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:47:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-0[56175]: 2024-03-30T02:47:57.139+0000 7ff329000700 -1 osd.0 100 *** Got signal Terminated *** 2024-03-30T02:47:57.412 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:47:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-0[56175]: 2024-03-30T02:47:57.139+0000 7ff329000700 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:48:03.162 INFO:journalctl@ceph.osd.0.smithi067.stdout:Mar 30 02:48:02 smithi067 bash[131564]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-0 2024-03-30T02:48:04.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.0.service' 2024-03-30T02:48:04.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:48:04.560 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-30T02:48:04.560 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-30T02:48:04.560 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.1 2024-03-30T02:48:04.999 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:48:04 smithi082 systemd[1]: Stopping Ceph osd.1 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:48:04.999 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:48:04 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-1[84173]: 2024-03-30T02:48:04.787+0000 7f27dcb7c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:48:04.999 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:48:04 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-1[84173]: 2024-03-30T02:48:04.787+0000 7f27dcb7c700 -1 osd.1 100 *** Got signal Terminated *** 2024-03-30T02:48:04.999 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:48:04 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-1[84173]: 2024-03-30T02:48:04.787+0000 7f27dcb7c700 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:48:10.749 INFO:journalctl@ceph.osd.1.smithi082.stdout:Mar 30 02:48:10 smithi082 bash[122073]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-1 2024-03-30T02:48:11.648 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.1.service' 2024-03-30T02:48:11.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:48:11.690 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-30T02:48:11.691 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-30T02:48:11.691 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.2 2024-03-30T02:48:12.188 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:48:11 smithi170 systemd[1]: Stopping Ceph osd.2 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:48:12.189 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:48:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-2[84585]: 2024-03-30T02:48:11.929+0000 7fc37d1e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:48:12.189 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:48:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-2[84585]: 2024-03-30T02:48:11.929+0000 7fc37d1e2700 -1 osd.2 100 *** Got signal Terminated *** 2024-03-30T02:48:12.189 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:48:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-2[84585]: 2024-03-30T02:48:11.929+0000 7fc37d1e2700 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:48:17.938 INFO:journalctl@ceph.osd.2.smithi170.stdout:Mar 30 02:48:17 smithi170 bash[123608]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-2 2024-03-30T02:48:18.740 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.2.service' 2024-03-30T02:48:18.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:48:18.784 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-30T02:48:18.784 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-30T02:48:18.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.3 2024-03-30T02:48:19.154 INFO:journalctl@ceph.osd.3.smithi067.stdout:Mar 30 02:48:18 smithi067 systemd[1]: Stopping Ceph osd.3 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:48:19.412 INFO:journalctl@ceph.osd.3.smithi067.stdout:Mar 30 02:48:19 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-3[64175]: 2024-03-30T02:48:19.153+0000 7f88e3a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:48:19.412 INFO:journalctl@ceph.osd.3.smithi067.stdout:Mar 30 02:48:19 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-3[64175]: 2024-03-30T02:48:19.153+0000 7f88e3a00700 -1 osd.3 100 *** Got signal Terminated *** 2024-03-30T02:48:19.412 INFO:journalctl@ceph.osd.3.smithi067.stdout:Mar 30 02:48:19 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-3[64175]: 2024-03-30T02:48:19.153+0000 7f88e3a00700 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:48:23.912 INFO:journalctl@ceph.osd.3.smithi067.stdout:Mar 30 02:48:23 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-3[64175]: 2024-03-30T02:48:23.414+0000 7f88da200700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.214502+0000 front 2024-03-30T02:47:59.214462+0000 (oldest deadline 2024-03-30T02:48:23.314604+0000) 2024-03-30T02:48:23.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:23.537+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:24.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:24 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:24.562+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:24.999 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:24 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:24.675+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:25.162 INFO:journalctl@ceph.osd.3.smithi067.stdout:Mar 30 02:48:24 smithi067 bash[131838]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-3 2024-03-30T02:48:25.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:24 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:24.728+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:25.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:25 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:25.535+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:25.999 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:25 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:25.695+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:26.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:25 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:25.752+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:26.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.3.service' 2024-03-30T02:48:26.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:26 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:26.532+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:26.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:26 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:26.372+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:26.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:48:26.735 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-30T02:48:26.735 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-30T02:48:26.735 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.4 2024-03-30T02:48:26.912 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:26.730+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:26.999 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:26 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:26.743+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:26.999 INFO:journalctl@ceph.osd.4.smithi082.stdout:Mar 30 02:48:26 smithi082 systemd[1]: Stopping Ceph osd.4 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:48:26.999 INFO:journalctl@ceph.osd.4.smithi082.stdout:Mar 30 02:48:26 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-4[92189]: 2024-03-30T02:48:26.981+0000 7f68afb5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:48:26.999 INFO:journalctl@ceph.osd.4.smithi082.stdout:Mar 30 02:48:26 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-4[92189]: 2024-03-30T02:48:26.981+0000 7f68afb5f700 -1 osd.4 100 *** Got signal Terminated *** 2024-03-30T02:48:27.000 INFO:journalctl@ceph.osd.4.smithi082.stdout:Mar 30 02:48:26 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-4[92189]: 2024-03-30T02:48:26.981+0000 7f68afb5f700 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:48:27.688 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:27 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:27.346+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:27.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:27 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:27.507+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:28.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:27 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:27.701+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:28.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:27 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:27.793+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:28.688 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:28 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:28.358+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:28.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:28 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:28.521+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:29.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:28 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:28.716+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:29.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:28 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:28.759+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:29.688 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:29 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:29.376+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:29.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:29 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:29.565+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:30.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:29 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:29.747+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:30.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:29 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:29.756+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:30.688 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:30 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:30.380+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:30.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:30 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:30.563+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:31.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:30 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:30.739+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:31.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:30 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:30.739+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:31.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:31 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:31.001+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:31.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:30 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:30.772+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:31.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:31 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:31.542+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:31.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:31 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:31.390+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:31.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:31 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:31.390+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:32.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:31 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:31.733+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:32.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:31 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:31.733+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:32.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:32 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:32.043+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:32.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:31 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:31.814+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:32.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:32 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:32.552+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:32.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:32 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:32.408+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:32.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:32 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:32.408+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:32.999 INFO:journalctl@ceph.osd.4.smithi082.stdout:Mar 30 02:48:32 smithi082 bash[122339]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-4 2024-03-30T02:48:33.000 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:32 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:32.855+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:33.000 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:32 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:32.855+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:33.121 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:32 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:32.712+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:33.121 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:32 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:32.712+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:33.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:33 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:33.122+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:33.499 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:33 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:33.060+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:33.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:33 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:33.536+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:33.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:33 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:33.536+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:33.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:33 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:33.388+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:33.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:33 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:33.388+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:34.108 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:33 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:33.749+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:34.109 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:33 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:33.749+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:34.226 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:33 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:33.901+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:34.226 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:33 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:33.901+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:34.227 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:34 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:34.020+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:34.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:34 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:34.108+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:34.426 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.4.service' 2024-03-30T02:48:34.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:48:34.476 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-30T02:48:34.476 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-30T02:48:34.476 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.5 2024-03-30T02:48:34.689 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:34 smithi170 systemd[1]: Stopping Ceph osd.5 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:48:34.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:34 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:34.357+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:34.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:34 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:34.357+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:34.690 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:34 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:34.497+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:34.690 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:34 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:34.497+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:35.068 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:34 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:48:34.733+0000 7f20f2b63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:48:35.068 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:34 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:48:34.733+0000 7f20f2b63700 -1 osd.5 100 *** Got signal Terminated *** 2024-03-30T02:48:35.069 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:34 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:48:34.733+0000 7f20f2b63700 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:48:35.116 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:34 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:34.710+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:35.116 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:34 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:34.710+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:35.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:34 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:34.973+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:35.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:34 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:34.925+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:35.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:34 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:34.925+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:35.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:35 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:35.116+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:35.438 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:35 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:35.311+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:35.438 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:35 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:35.311+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:35.439 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:35 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:48:35.069+0000 7f20eb151700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.888780+0000 front 2024-03-30T02:48:08.888674+0000 (oldest deadline 2024-03-30T02:48:34.788889+0000) 2024-03-30T02:48:35.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:35 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:35.478+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:35.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:35 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:35.478+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:36.092 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:35 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:35.709+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:36.092 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:35 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:35.709+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:36.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:35 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:35.979+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:36.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:35 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:35.925+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:36.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:35 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:35.925+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:36.327 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:36 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:48:36.046+0000 7f20eb151700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.888780+0000 front 2024-03-30T02:48:08.888674+0000 (oldest deadline 2024-03-30T02:48:34.788889+0000) 2024-03-30T02:48:36.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:36 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:36.093+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:36.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:36 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:36.432+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:36.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:36 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:36.432+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:36.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:36 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:36.328+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:36.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:36 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:36.328+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:37.117 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:36 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:36.735+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:37.118 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:36 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:36.735+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:37.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:36 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:36.992+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:37.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:36 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:36.950+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:37.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:36 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:36.950+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:37.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:37 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:37.118+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:37.438 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:37 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:48:37.054+0000 7f20eb151700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.888780+0000 front 2024-03-30T02:48:08.888674+0000 (oldest deadline 2024-03-30T02:48:34.788889+0000) 2024-03-30T02:48:37.439 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:37 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:37.291+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:37.439 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:37 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:37.291+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:37.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:37 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:37.442+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:37.942 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:37 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:37.442+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:38.153 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:37 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:37.748+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:38.153 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:37 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:37.748+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:38.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:38 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:38.037+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:38.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:37 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:37.955+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:38.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:37 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:37.955+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:38.334 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:38 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:48:38.078+0000 7f20eb151700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.888780+0000 front 2024-03-30T02:48:08.888674+0000 (oldest deadline 2024-03-30T02:48:34.788889+0000) 2024-03-30T02:48:38.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:38 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:38.153+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:38.688 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:38 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:38.335+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:38.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:38 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:38.335+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:38.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:38 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:38.402+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:38.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:38 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:38.402+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:39.122 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:38 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:38.752+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:39.123 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:38 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:38.752+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:39.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:38 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:38.978+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:39.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:38 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:38.978+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:39.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:39 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:39.000+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:39.377 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:39 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5[93194]: 2024-03-30T02:48:39.123+0000 7f20eb151700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.888780+0000 front 2024-03-30T02:48:08.888674+0000 (oldest deadline 2024-03-30T02:48:34.788889+0000) 2024-03-30T02:48:39.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:39 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:39.123+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:39.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:39 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:39.123+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.988506+0000 front 2024-03-30T02:48:12.988513+0000 (oldest deadline 2024-03-30T02:48:38.288571+0000) 2024-03-30T02:48:39.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:39 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:39.429+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:39.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:39 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:39.429+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:39.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:39 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:39.377+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:39.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:39 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:39.377+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:39.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:39 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:39.377+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:40.131 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:39 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:39.739+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:40.131 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:39 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:39.739+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:40.131 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:39 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:39.739+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:40.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:40.132+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:40.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:40.132+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.988506+0000 front 2024-03-30T02:48:12.988513+0000 (oldest deadline 2024-03-30T02:48:38.288571+0000) 2024-03-30T02:48:40.499 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:40 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:40.012+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:40.499 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:40 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:40.008+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:40.499 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:40 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:40.008+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:40.688 INFO:journalctl@ceph.osd.5.smithi170.stdout:Mar 30 02:48:40 smithi170 bash[123873]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-5 2024-03-30T02:48:40.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:40 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:40.366+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:40.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:40 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:40.366+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:40.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:40 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:40.366+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:40.690 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:40 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:40.390+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:40.690 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:40 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:40.390+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:40.690 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:40 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:40.390+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:41.082 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:40.713+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:41.083 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:40.713+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:41.083 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:40 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:40.713+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:41.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:40 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:40.969+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:41.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:40 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:40.969+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:41.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:41 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:41.042+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:41.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:41 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:41.042+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:41.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:41 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:41.042+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:41.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:41.083+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:41.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:41.083+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.988506+0000 front 2024-03-30T02:48:12.988513+0000 (oldest deadline 2024-03-30T02:48:38.288571+0000) 2024-03-30T02:48:41.529 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:41 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:41.330+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:41.529 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:41 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:41.330+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:41.529 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:41 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:41.330+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:41.529 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:41 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:41.433+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:41.530 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:41 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:41.433+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:41.530 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:41 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:41.433+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:41.535 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.5.service' 2024-03-30T02:48:41.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:48:41.576 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-30T02:48:41.576 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-30T02:48:41.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.6 2024-03-30T02:48:41.912 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:41 smithi067 systemd[1]: Stopping Ceph osd.6 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:48:41.912 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:41.697+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:41.913 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:41.697+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:41.913 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:41.697+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:42.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:41 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:41.988+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:42.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:41 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:41.988+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:42.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:42 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:42.012+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:42.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:42 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:42.012+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:42.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:42 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:42.012+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:42.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:41.951+0000 7f6fc9000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:48:42.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:41.951+0000 7f6fc9000700 -1 osd.6 100 *** Got signal Terminated *** 2024-03-30T02:48:42.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:41 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:41.951+0000 7f6fc9000700 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:48:42.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:42.078+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:42.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:42.078+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.988506+0000 front 2024-03-30T02:48:12.988513+0000 (oldest deadline 2024-03-30T02:48:38.288571+0000) 2024-03-30T02:48:42.688 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:42 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:42.349+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:42.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:42 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:42.349+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:42.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:42 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:42.349+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:42.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:42 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:42.467+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:42.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:42 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:42.467+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:42.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:42 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:42.467+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:43.072 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:42.710+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:43.072 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:42.710+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:43.072 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:42 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:42.710+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:43.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:43 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:43.025+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:43.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:43 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:43.025+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:43.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:42 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:42.979+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:43.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:42 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:42.979+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:43.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:42 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:42.979+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:43.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:43 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:43.072+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:43.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:43 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:43.072+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.988506+0000 front 2024-03-30T02:48:12.988513+0000 (oldest deadline 2024-03-30T02:48:38.288571+0000) 2024-03-30T02:48:43.688 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:43 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:43.379+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:43.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:43 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:43.379+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:43.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:43 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:43.379+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:43.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:43 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:43.503+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:43.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:43 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:43.503+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:43.690 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:43 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:43.503+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:44.033 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:43 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:43.721+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:44.034 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:43 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:43.721+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:44.034 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:43 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:43.721+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:44.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:44 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:44.031+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:44.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:44 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:44.031+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.988506+0000 front 2024-03-30T02:48:12.988513+0000 (oldest deadline 2024-03-30T02:48:38.288571+0000) 2024-03-30T02:48:44.499 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:44 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:44.055+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:44.499 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:44 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:44.055+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:44.499 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:44 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:44.020+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:44.499 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:44 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:44.020+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:44.500 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:44 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:44.020+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:44.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:44 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:44.381+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:44.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:44 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:44.381+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:44.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:44 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:44.381+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:44.690 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:44 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:44.521+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:44.690 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:44 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:44.521+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:44.690 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:44 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:44.521+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:45.017 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:44 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:44.729+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:45.018 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:44 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:44.729+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:45.018 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:44 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:44.729+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:45.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:45 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:45.018+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:45.412 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:45 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:45.018+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.988506+0000 front 2024-03-30T02:48:12.988513+0000 (oldest deadline 2024-03-30T02:48:38.288571+0000) 2024-03-30T02:48:45.499 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:45 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:45.098+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:45.499 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:45 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:45.098+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:45.499 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:45 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:45.054+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:45.499 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:45 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:45.054+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:45.500 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:45 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:45.054+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:45.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:45 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:45.519+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:45.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:45 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:45.519+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:45.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:45 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:45.519+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:45.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:45 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:45.333+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:45.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:45 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:45.333+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:45.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:45 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:45.333+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:45.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:45 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:45.333+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:46.162 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:45 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:45.985+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:07.687825+0000 front 2024-03-30T02:48:07.687721+0000 (oldest deadline 2024-03-30T02:48:32.987761+0000) 2024-03-30T02:48:46.162 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:45 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6[72061]: 2024-03-30T02:48:45.985+0000 7f6fbf800700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.988506+0000 front 2024-03-30T02:48:12.988513+0000 (oldest deadline 2024-03-30T02:48:38.288571+0000) 2024-03-30T02:48:46.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:45 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:45.747+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:46.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:45 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:45.747+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:46.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:45 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:45.747+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:46.499 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:46.132+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:46.499 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:46.132+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:46.499 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:46.132+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:20.179213+0000 front 2024-03-30T02:48:20.179264+0000 (oldest deadline 2024-03-30T02:48:45.479283+0000) 2024-03-30T02:48:46.499 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:46.030+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:46.499 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:46.030+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:46.500 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:46.030+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:46.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:46.525+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:46.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:46.525+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:46.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:46.525+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:46.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:46.291+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:46.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:46.291+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:46.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:46.291+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:46.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:46 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:46.291+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:47.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:46.719+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:47.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:46.719+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:47.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:46.719+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:47.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:46 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:46.719+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:47.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:46.987+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:47.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:46.987+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:47.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:46.987+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:47.250 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:47 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:47.084+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:47.250 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:47 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:47.084+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:47.250 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:47 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:47.084+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:20.179213+0000 front 2024-03-30T02:48:20.179264+0000 (oldest deadline 2024-03-30T02:48:45.479283+0000) 2024-03-30T02:48:47.557 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:47 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:47.275+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:47.558 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:47 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:47.275+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:47.558 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:47 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:47.275+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:47.558 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:47 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:47.275+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:47.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:47 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:47.558+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:47.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:47 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:47.558+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:47.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:47 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:47.558+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:48.162 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:47 smithi067 bash[132094]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-6 2024-03-30T02:48:48.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:47 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:47.729+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:48.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:47 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:47.729+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:48.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:47 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:47.729+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:48.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:47 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:47.729+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:48.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:48 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:48.104+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:48.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:48 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:48.104+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:48.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:48 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:48.104+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:20.179213+0000 front 2024-03-30T02:48:20.179264+0000 (oldest deadline 2024-03-30T02:48:45.479283+0000) 2024-03-30T02:48:48.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:47 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:47.965+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:48.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:47 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:47.965+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:48.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:47 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:47.965+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:48.602 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:48 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:48.315+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:48.603 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:48 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:48.315+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:48.603 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:48 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:48.315+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:48.603 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:48 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:48.315+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:48.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:48 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:48.603+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:48.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:48 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:48.603+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:48.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:48 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:48.603+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:49.093 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:48 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:48.760+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:49.093 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:48 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:48.760+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:49.094 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:48 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:48.760+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:49.094 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:48 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:48.760+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:49.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:49.096+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:49.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:49.096+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:49.250 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:49.096+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:20.179213+0000 front 2024-03-30T02:48:20.179264+0000 (oldest deadline 2024-03-30T02:48:45.479283+0000) 2024-03-30T02:48:49.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:48 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:48.935+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:49.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:48 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:48.935+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:49.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:48 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:48.935+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:49.251 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:48 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:48.935+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:49.344 INFO:journalctl@ceph.osd.6.smithi067.stdout:Mar 30 02:48:49 smithi067 systemd[1]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.6.service: Succeeded. 2024-03-30T02:48:49.350 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.6.service' 2024-03-30T02:48:49.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:48:49.437 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-30T02:48:49.437 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-30T02:48:49.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.7 2024-03-30T02:48:49.608 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:49 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:49.320+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:49.608 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:49 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:49.320+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:49.608 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:49 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:49.321+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:49.608 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:49 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:49.321+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:49.749 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:49 smithi082 systemd[1]: Stopping Ceph osd.7 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:48:49.749 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:49.676+0000 7fa6adb78700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:48:49.750 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:49.676+0000 7fa6adb78700 -1 osd.7 100 *** Got signal Terminated *** 2024-03-30T02:48:49.750 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:49.676+0000 7fa6adb78700 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:48:49.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:49 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:49.608+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:49.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:49 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:49.608+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:49.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:49 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:49.608+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:50.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:49 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:49.764+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:50.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:49 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:49.764+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:50.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:49 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:49.764+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:50.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:49 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:49.764+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:50.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:50.082+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:50.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:50.082+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:50.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:50.082+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:20.179213+0000 front 2024-03-30T02:48:20.179264+0000 (oldest deadline 2024-03-30T02:48:45.479283+0000) 2024-03-30T02:48:50.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:49.895+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:50.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:49.895+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:50.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:49.895+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:50.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:49 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:49.895+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:50.611 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:50 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:50.304+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:50.612 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:50 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:50.304+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:50.612 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:50 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:50.304+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:50.612 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:50 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:50.304+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:50.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:50 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:50.612+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:50.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:50 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:50.612+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:50.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:50 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:50.612+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:50.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:50 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:50.612+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:51.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:50 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:50.759+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:51.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:50 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:50.759+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:51.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:50 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:50.759+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:51.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:50 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:50.759+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:51.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:51 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:51.099+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:51.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:51 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:51.099+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:51.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:51 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:51.100+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:20.179213+0000 front 2024-03-30T02:48:20.179264+0000 (oldest deadline 2024-03-30T02:48:45.479283+0000) 2024-03-30T02:48:51.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:50.907+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:51.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:50.907+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:51.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:50.907+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:51.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:50.907+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:51.581 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:51 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:51.310+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:51.582 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:51 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:51.310+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:51.582 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:51 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:51.310+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:51.582 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:51 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:51.310+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:51.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:51 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:51.582+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:51.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:51 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:51.582+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:51.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:51 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:51.582+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:51.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:51 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:51.582+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:52.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:51 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:51.775+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:52.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:51 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:51.775+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:52.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:51 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:51.775+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:52.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:51 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:51.775+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:52.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:52.110+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:52.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:52.110+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:52.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:52.110+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:20.179213+0000 front 2024-03-30T02:48:20.179264+0000 (oldest deadline 2024-03-30T02:48:45.479283+0000) 2024-03-30T02:48:52.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:51 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:51.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:52.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:51 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:51.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:52.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:51 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:51.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:52.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:51 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:51.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:52.570 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:52 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:52.295+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:52.570 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:52 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:52.295+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:52.570 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:52 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:52.295+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:52.570 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:52 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:52.295+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:52.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:52 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:52.571+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:52.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:52 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:52.571+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:52.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:52 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:52.571+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:52.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:52 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:52.571+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:53.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:52 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:52.779+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:53.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:52 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:52.779+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:53.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:52 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:52.779+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:53.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:52 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:52.779+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:53.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:53.112+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:53.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:53.112+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:53.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:53.112+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:20.179213+0000 front 2024-03-30T02:48:20.179264+0000 (oldest deadline 2024-03-30T02:48:45.479283+0000) 2024-03-30T02:48:53.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:52.890+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:53.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:52.890+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:53.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:52.890+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:53.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:52.890+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:53.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:52 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:52.890+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:48:53.591 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:53.306+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:53.592 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:53.306+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:53.592 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:53.306+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:53.592 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:53.306+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:53.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:53.592+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:53.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:53.592+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:53.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:53.592+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:53.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:53 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:53.592+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:54.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:53.777+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:54.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:53.777+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:54.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:53.777+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:54.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:53 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:53.777+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:54.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:54.104+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:06.176858+0000 front 2024-03-30T02:48:06.176882+0000 (oldest deadline 2024-03-30T02:48:30.877010+0000) 2024-03-30T02:48:54.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:54.104+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.078671+0000 front 2024-03-30T02:48:16.078823+0000 (oldest deadline 2024-03-30T02:48:40.178710+0000) 2024-03-30T02:48:54.249 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7[100958]: 2024-03-30T02:48:54.104+0000 7fa6aa16e700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:20.179213+0000 front 2024-03-30T02:48:20.179264+0000 (oldest deadline 2024-03-30T02:48:45.479283+0000) 2024-03-30T02:48:54.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:53.874+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:54.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:53.874+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:54.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:53.874+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:54.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:53.874+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:54.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:53 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:53.874+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:48:54.547 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:54.267+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:54.547 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:54.267+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:54.547 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:54.267+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:54.547 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:54.267+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:54.547 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:54.267+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:27.862525+0000 front 2024-03-30T02:48:27.862487+0000 (oldest deadline 2024-03-30T02:48:53.762388+0000) 2024-03-30T02:48:54.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:54.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:54.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:54.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:54.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:54.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:54.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:54 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:54.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:55.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:54 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:54.774+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:55.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:54 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:54.774+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:55.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:54 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:54.774+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:55.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:54 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:54.774+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:55.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:54.849+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:55.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:54.849+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:55.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:54.849+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:55.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:54.849+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:55.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:54.849+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:48:55.531 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:55.257+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:55.532 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:55.257+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:55.532 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:55.257+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:55.532 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:55.257+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:55.532 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:55.257+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:27.862525+0000 front 2024-03-30T02:48:27.862487+0000 (oldest deadline 2024-03-30T02:48:53.762388+0000) 2024-03-30T02:48:55.749 INFO:journalctl@ceph.osd.7.smithi082.stdout:Mar 30 02:48:55 smithi082 bash[122597]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-7 2024-03-30T02:48:55.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:55.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:55.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:55.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:55.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:55.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:55.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:55.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:55.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:55 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:55.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:48:56.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:55.737+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:56.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:55.737+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:56.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:55.737+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:56.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:55 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:55.737+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:56.239 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:55.802+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:56.240 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:55.802+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:56.240 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:55.802+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:56.240 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:55.802+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:56.240 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:55 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:55.802+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:48:56.578 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.7.service' 2024-03-30T02:48:56.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:48:56.619 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-30T02:48:56.620 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-30T02:48:56.620 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.8 2024-03-30T02:48:56.650 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:56.288+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:56.650 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:56.288+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:56.650 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:56.288+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:56.651 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:56.288+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:56.651 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:56.288+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:27.862525+0000 front 2024-03-30T02:48:27.862487+0000 (oldest deadline 2024-03-30T02:48:53.762388+0000) 2024-03-30T02:48:56.651 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:56.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:56.651 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:56.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:56.651 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:56.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:56.651 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:56.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:56.651 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:56.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:48:56.938 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:56 smithi170 systemd[1]: Stopping Ceph osd.8 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:48:56.939 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:56.847+0000 7f4bf87be700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:48:56.939 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:56.847+0000 7f4bf87be700 -1 osd.8 100 *** Got signal Terminated *** 2024-03-30T02:48:56.939 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:56 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:56.847+0000 7f4bf87be700 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:48:57.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:56 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:56.752+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:57.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:56 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:56.752+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:57.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:56 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:56.752+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:57.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:56 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:56.752+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:57.248 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:56 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:56.842+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:57.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:56 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:56.842+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:57.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:56 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:56.842+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:57.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:56 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:56.842+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:57.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:56 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:56.842+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:48:57.572 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:57.312+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:57.572 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:57.312+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:57.572 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:57.312+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:57.572 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:57.312+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:57.572 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:57.312+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:27.862525+0000 front 2024-03-30T02:48:27.862487+0000 (oldest deadline 2024-03-30T02:48:53.762388+0000) 2024-03-30T02:48:57.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:57.573+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:57.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:57.573+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:57.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:57.573+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:57.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:57.573+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:57.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:57 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:57.573+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:48:58.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:57.737+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:58.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:57.737+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:58.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:57.737+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:58.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:57 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:57.737+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:58.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:57 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:57.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:58.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:57 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:57.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:58.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:57 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:57.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:58.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:57 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:57.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:58.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:57 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:57.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:48:58.602 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:58.338+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:58.603 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:58.338+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:58.603 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:58.338+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:58.603 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:58.338+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:58.603 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:58.338+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:27.862525+0000 front 2024-03-30T02:48:27.862487+0000 (oldest deadline 2024-03-30T02:48:53.762388+0000) 2024-03-30T02:48:58.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:58.603+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:58.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:58.603+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:58.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:58.603+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:58.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:58.603+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:58.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:58.603+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:48:59.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:58.750+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:48:59.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:58.750+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:48:59.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:58.750+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:48:59.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:58 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:58.750+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:48:59.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:58 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:58.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:48:59.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:58 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:58.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:48:59.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:58 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:58.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:48:59.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:58 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:58.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:48:59.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:58 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:58.830+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:48:59.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:59.600+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:48:59.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:59.600+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:48:59.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:59.600+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:48:59.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:59.600+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:48:59.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:48:59.600+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:48:59.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:59.350+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:48:59.689 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:59.350+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:48:59.690 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:59.350+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:48:59.690 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:59.350+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:48:59.690 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:48:59 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:48:59.350+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:27.862525+0000 front 2024-03-30T02:48:27.862487+0000 (oldest deadline 2024-03-30T02:48:53.762388+0000) 2024-03-30T02:49:00.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:59 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:59.711+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:00.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:59 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:59.711+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:00.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:59 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:59.711+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:00.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:48:59 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:48:59.711+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:00.251 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:59.854+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:00.252 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:59.854+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:00.252 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:59.854+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:00.252 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:59.854+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:00.252 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:48:59 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:48:59.854+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:00.688 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:00.574+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:00.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:00.574+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:00.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:00.574+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:00.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:00.574+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:00.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:00.574+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:00.690 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:00.343+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:49:00.690 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:00.343+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:49:00.690 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:00.343+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:49:00.690 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:00.343+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:49:00.690 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:00 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:00.343+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:27.862525+0000 front 2024-03-30T02:48:27.862487+0000 (oldest deadline 2024-03-30T02:48:53.762388+0000) 2024-03-30T02:49:01.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:00.682+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:01.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:00.682+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:01.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:00.682+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:01.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:00 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:00.682+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:01.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:00 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:00.904+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:01.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:00 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:00.904+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:01.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:00 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:00.904+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:01.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:00 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:00.904+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:01.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:00 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:00.904+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:01.606 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:01.339+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:48:01.057984+0000 front 2024-03-30T02:48:01.058005+0000 (oldest deadline 2024-03-30T02:48:25.758056+0000) 2024-03-30T02:49:01.606 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:01.339+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:05.758685+0000 front 2024-03-30T02:48:05.758809+0000 (oldest deadline 2024-03-30T02:48:31.058538+0000) 2024-03-30T02:49:01.606 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:01.339+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.659812+0000 front 2024-03-30T02:48:12.659866+0000 (oldest deadline 2024-03-30T02:48:38.560130+0000) 2024-03-30T02:49:01.606 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:01.339+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:18.560695+0000 front 2024-03-30T02:48:18.560759+0000 (oldest deadline 2024-03-30T02:48:44.961242+0000) 2024-03-30T02:49:01.606 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8[101139]: 2024-03-30T02:49:01.339+0000 7f4bf4db4700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:27.862525+0000 front 2024-03-30T02:48:27.862487+0000 (oldest deadline 2024-03-30T02:48:53.762388+0000) 2024-03-30T02:49:01.913 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:01.662+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:01.913 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:01.662+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:01.914 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:01.662+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:01.914 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:01.662+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:01.914 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:01 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:01.662+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:37.516246+0000 front 2024-03-30T02:48:37.516096+0000 (oldest deadline 2024-03-30T02:49:01.615930+0000) 2024-03-30T02:49:01.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:01.606+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:01.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:01.606+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:01.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:01.606+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:01.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:01.606+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:01.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:01.606+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:01.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:01 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:01.606+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:02.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:01 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:01.865+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:02.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:01 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:01.865+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:02.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:01 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:01.865+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:02.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:01 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:01.865+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:02.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:01 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:01.865+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:02.688 INFO:journalctl@ceph.osd.8.smithi170.stdout:Mar 30 02:49:02 smithi170 bash[124130]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-8 2024-03-30T02:49:02.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:02 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:02.564+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:02.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:02 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:02.564+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:02.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:02 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:02.564+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:02.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:02 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:02.564+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:02.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:02 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:02.564+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:02.689 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:02 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:02.564+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:03.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:02.681+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:03.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:02.681+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:03.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:02.681+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:03.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:02.681+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:03.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:02 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:02.681+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:37.516246+0000 front 2024-03-30T02:48:37.516096+0000 (oldest deadline 2024-03-30T02:49:01.615930+0000) 2024-03-30T02:49:03.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:02 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:02.840+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:03.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:02 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:02.840+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:03.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:02 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:02.840+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:03.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:02 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:02.840+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:03.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:02 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:02.840+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:03.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:02 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:02.840+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:03.901 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:03 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:03.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:03.901 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:03 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:03.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:03.901 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:03 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:03.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:03.901 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:03 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:03.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:03.901 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:03 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:03.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:03.901 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:03 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:03.533+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:03.908 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.8.service' 2024-03-30T02:49:03.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:03.989 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-30T02:49:03.989 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-30T02:49:03.989 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.9 2024-03-30T02:49:04.058 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:03 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:03.685+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:04.058 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:03 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:03.685+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:04.058 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:03 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:03.685+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:04.058 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:03 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:03.685+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:04.058 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:03 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:03.685+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:37.516246+0000 front 2024-03-30T02:48:37.516096+0000 (oldest deadline 2024-03-30T02:49:01.615930+0000) 2024-03-30T02:49:04.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:03 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:03.859+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:04.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:03 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:03.859+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:04.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:03 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:03.859+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:04.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:03 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:03.859+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:04.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:03 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:03.859+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:04.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:03 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:03.859+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:04.406 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:04 smithi067 systemd[1]: Stopping Ceph osd.9 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:04.662 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:04 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:04.406+0000 7f7959200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:04.662 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:04 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:04.406+0000 7f7959200700 -1 osd.9 100 *** Got signal Terminated *** 2024-03-30T02:49:04.662 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:04 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:04.406+0000 7f7959200700 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:49:04.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:04 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:04.572+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:04.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:04 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:04.572+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:04.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:04 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:04.572+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:04.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:04 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:04.572+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:04.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:04 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:04.572+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:04.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:04 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:04.572+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:05.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:04 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:04.712+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:05.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:04 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:04.712+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:05.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:04 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:04.712+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:05.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:04 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:04.712+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:05.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:04 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:04.712+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:37.516246+0000 front 2024-03-30T02:48:37.516096+0000 (oldest deadline 2024-03-30T02:49:01.615930+0000) 2024-03-30T02:49:05.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:04 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:04.881+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:05.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:04 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:04.881+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:05.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:04 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:04.881+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:05.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:04 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:04.881+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:05.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:04 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:04.881+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:05.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:04 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:04.881+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:05.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:05 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:05.607+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:05.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:05 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:05.607+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:05.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:05 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:05.607+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:05.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:05 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:05.607+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:05.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:05 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:05.607+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:05.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:05 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:05.607+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:06.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:05 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:05.684+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:06.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:05 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:05.684+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:06.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:05 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:05.684+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:06.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:05 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:05.684+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:06.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:05 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:05.684+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:37.516246+0000 front 2024-03-30T02:48:37.516096+0000 (oldest deadline 2024-03-30T02:49:01.615930+0000) 2024-03-30T02:49:06.248 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:05 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:05.908+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:06.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:05 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:05.908+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:06.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:05 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:05.908+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:06.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:05 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:05.908+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:06.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:05 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:05.908+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:06.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:05 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:05.908+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:06.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:06 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:06.643+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:06.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:06 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:06.643+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:06.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:06 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:06.643+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:06.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:06 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:06.643+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:06.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:06 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:06.643+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:06.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:06 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:06.643+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:07.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:06.700+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:07.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:06.700+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:07.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:06.700+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:07.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:06.700+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:07.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:06 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:06.700+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:37.516246+0000 front 2024-03-30T02:48:37.516096+0000 (oldest deadline 2024-03-30T02:49:01.615930+0000) 2024-03-30T02:49:07.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:06 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:06.910+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:07.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:06 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:06.910+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:07.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:06 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:06.910+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:07.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:06 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:06.910+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:07.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:06 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:06.910+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:07.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:06 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:06.910+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:07.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:07 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:07.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:07.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:07 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:07.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:07.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:07 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:07.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:07.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:07 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:07.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:07.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:07 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:07.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:07.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:07 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:07.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:08.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:07 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:07.749+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:08.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:07 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:07.749+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:08.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:07 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:07.749+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:08.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:07 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:07.749+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:08.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:07 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:07.749+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:37.516246+0000 front 2024-03-30T02:48:37.516096+0000 (oldest deadline 2024-03-30T02:49:01.615930+0000) 2024-03-30T02:49:08.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:07 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:07.879+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:08.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:07 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:07.879+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:08.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:07 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:07.879+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:08.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:07 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:07.879+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:08.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:07 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:07.879+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:08.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:07 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:07.879+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:08.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:08 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:08.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:08.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:08 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:08.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:08.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:08 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:08.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:08.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:08 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:08.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:08.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:08 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:08.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:08.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:08 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:08.602+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:09.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:08 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:08.755+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.406389+0000 front 2024-03-30T02:47:59.406483+0000 (oldest deadline 2024-03-30T02:48:24.706628+0000) 2024-03-30T02:49:09.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:08 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:08.755+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.108550+0000 front 2024-03-30T02:48:08.108743+0000 (oldest deadline 2024-03-30T02:48:30.408591+0000) 2024-03-30T02:49:09.162 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:08 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:08.755+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:16.111701+0000 front 2024-03-30T02:48:16.111353+0000 (oldest deadline 2024-03-30T02:48:39.011499+0000) 2024-03-30T02:49:09.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:08 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:08.755+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:21.812970+0000 front 2024-03-30T02:48:21.813395+0000 (oldest deadline 2024-03-30T02:48:46.513030+0000) 2024-03-30T02:49:09.163 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:08 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9[80018]: 2024-03-30T02:49:08.755+0000 7f7954a00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:37.516246+0000 front 2024-03-30T02:48:37.516096+0000 (oldest deadline 2024-03-30T02:49:01.615930+0000) 2024-03-30T02:49:09.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:08 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:08.838+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:09.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:08 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:08.838+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:09.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:08 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:08.838+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:09.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:08 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:08.838+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:09.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:08 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:08.838+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:09.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:08 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:08.838+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:09.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:09 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:09.568+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:09.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:09 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:09.568+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:09.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:09 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:09.568+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:09.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:09 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:09.568+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:09.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:09 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:09.568+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:09.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:09 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:09.568+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:10.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:09 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:09.887+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:10.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:09 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:09.887+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:10.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:09 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:09.887+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:10.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:09 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:09.887+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:10.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:09 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:09.887+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:10.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:09 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:09.887+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:10.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:09 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:09.887+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:42.751856+0000 front 2024-03-30T02:48:42.751744+0000 (oldest deadline 2024-03-30T02:49:09.752210+0000) 2024-03-30T02:49:10.662 INFO:journalctl@ceph.osd.9.smithi067.stdout:Mar 30 02:49:10 smithi067 bash[132359]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-9 2024-03-30T02:49:10.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:10 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:10.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:10.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:10 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:10.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:10.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:10 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:10.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:10.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:10 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:10.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:10.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:10 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:10.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:10.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:10 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:10.548+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:11.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:10 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:10.869+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:11.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:10 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:10.869+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:11.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:10 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:10.869+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:11.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:10 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:10.869+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:11.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:10 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:10.869+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:11.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:10 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:10.869+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:11.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:10 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:10.869+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:42.751856+0000 front 2024-03-30T02:48:42.751744+0000 (oldest deadline 2024-03-30T02:49:09.752210+0000) 2024-03-30T02:49:11.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.9.service' 2024-03-30T02:49:11.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:11.890 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-30T02:49:11.890 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-30T02:49:11.890 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.10 2024-03-30T02:49:11.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:11.536+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:11.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:11.536+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:11.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:11.536+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:11.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:11.536+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:11.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:11.536+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:11.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:11.536+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:11.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:11 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:11.536+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:12.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:11 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:11.903+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:12.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:11 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:11.903+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:12.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:11 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:11.903+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:12.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:11 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:11.903+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:12.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:11 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:11.903+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:12.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:11 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:11.903+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:12.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:11 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:11.903+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:42.751856+0000 front 2024-03-30T02:48:42.751744+0000 (oldest deadline 2024-03-30T02:49:09.752210+0000) 2024-03-30T02:49:12.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:11 smithi082 systemd[1]: Stopping Ceph osd.10 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:12.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.130+0000 7f04e8032700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:12.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.130+0000 7f04e8032700 -1 osd.10 100 *** Got signal Terminated *** 2024-03-30T02:49:12.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.130+0000 7f04e8032700 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:49:12.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:12 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:12.537+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:12.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:12 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:12.537+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:12.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:12 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:12.537+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:12.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:12 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:12.537+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:12.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:12 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:12.537+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:12.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:12 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:12.537+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:12.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:12 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:12.537+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:13.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.940+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:13.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.940+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:13.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.940+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:13.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.940+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:13.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.940+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:13.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.940+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:13.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:12 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:12.940+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:42.751856+0000 front 2024-03-30T02:48:42.751744+0000 (oldest deadline 2024-03-30T02:49:09.752210+0000) 2024-03-30T02:49:13.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:13 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:13.532+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:13.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:13 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:13.532+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:13.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:13 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:13.532+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:13.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:13 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:13.532+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:13.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:13 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:13.532+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:13.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:13 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:13.532+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:13.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:13 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:13.532+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:14.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:13 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:13.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:14.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:13 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:13.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:14.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:13 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:13.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:14.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:13 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:13.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:14.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:13 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:13.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:14.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:13 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:13.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:14.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:13 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:13.915+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:42.751856+0000 front 2024-03-30T02:48:42.751744+0000 (oldest deadline 2024-03-30T02:49:09.752210+0000) 2024-03-30T02:49:14.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:14 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:14.509+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:14.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:14 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:14.509+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:14.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:14 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:14.509+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:14.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:14 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:14.509+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:14.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:14 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:14.509+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:14.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:14 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:14.509+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:14.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:14 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:14.509+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:15.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:14 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:14.919+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:15.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:14 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:14.919+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:15.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:14 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:14.919+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:15.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:14 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:14.919+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:15.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:14 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:14.919+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:15.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:14 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:14.919+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:15.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:14 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:14.919+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:42.751856+0000 front 2024-03-30T02:48:42.751744+0000 (oldest deadline 2024-03-30T02:49:09.752210+0000) 2024-03-30T02:49:15.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:15 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:15.464+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:15.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:15 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:15.464+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:15.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:15 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:15.464+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:15.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:15 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:15.464+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:15.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:15 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:15.464+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:15.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:15 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:15.464+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:15.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:15 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:15.464+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:16.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:15 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:15.941+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:16.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:15 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:15.941+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:16.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:15 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:15.941+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:16.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:15 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:15.941+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:16.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:15 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:15.941+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:16.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:15 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:15.941+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:16.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:15 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:15.941+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:42.751856+0000 front 2024-03-30T02:48:42.751744+0000 (oldest deadline 2024-03-30T02:49:09.752210+0000) 2024-03-30T02:49:16.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:16.505+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:16.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:16.505+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:16.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:16.505+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:16.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:16.505+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:16.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:16.505+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:16.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:16.505+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:16.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:16.505+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:16.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:16 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:16.505+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-03-30T02:48:50.994297+0000 front 2024-03-30T02:48:50.994207+0000 (oldest deadline 2024-03-30T02:49:16.294022+0000) 2024-03-30T02:49:17.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:16 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:16.924+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:59.744346+0000 front 2024-03-30T02:47:59.744359+0000 (oldest deadline 2024-03-30T02:48:23.844247+0000) 2024-03-30T02:49:17.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:16 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:16.924+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.445731+0000 front 2024-03-30T02:48:08.445953+0000 (oldest deadline 2024-03-30T02:48:32.545917+0000) 2024-03-30T02:49:17.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:16 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:16.924+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:12.546638+0000 front 2024-03-30T02:48:12.546550+0000 (oldest deadline 2024-03-30T02:48:40.147086+0000) 2024-03-30T02:49:17.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:16 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:16.924+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:23.449662+0000 front 2024-03-30T02:48:23.449416+0000 (oldest deadline 2024-03-30T02:48:48.749359+0000) 2024-03-30T02:49:17.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:16 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:16.924+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:31.650291+0000 front 2024-03-30T02:48:31.650266+0000 (oldest deadline 2024-03-30T02:48:52.750383+0000) 2024-03-30T02:49:17.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:16 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:16.924+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:38.051553+0000 front 2024-03-30T02:48:38.051498+0000 (oldest deadline 2024-03-30T02:49:02.751340+0000) 2024-03-30T02:49:17.250 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:16 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10[109585]: 2024-03-30T02:49:16.924+0000 7f04e0620700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:42.751856+0000 front 2024-03-30T02:48:42.751744+0000 (oldest deadline 2024-03-30T02:49:09.752210+0000) 2024-03-30T02:49:17.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:17 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:17.458+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:17.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:17 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:17.458+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:17.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:17 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:17.458+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:17.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:17 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:17.458+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:17.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:17 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:17.458+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:17.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:17 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:17.458+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:17.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:17 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:17.458+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:17.940 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:17 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:17.458+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-03-30T02:48:50.994297+0000 front 2024-03-30T02:48:50.994207+0000 (oldest deadline 2024-03-30T02:49:16.294022+0000) 2024-03-30T02:49:18.249 INFO:journalctl@ceph.osd.10.smithi082.stdout:Mar 30 02:49:17 smithi082 bash[122847]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-10 2024-03-30T02:49:18.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:18 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:18.451+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:18.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:18 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:18.451+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:18.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:18 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:18.451+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:18.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:18 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:18.451+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:18.940 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:18 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:18.451+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:18.940 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:18 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:18.451+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:18.940 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:18 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:18.451+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:18.940 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:18 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:18.451+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-03-30T02:48:50.994297+0000 front 2024-03-30T02:48:50.994207+0000 (oldest deadline 2024-03-30T02:49:16.294022+0000) 2024-03-30T02:49:18.982 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.10.service' 2024-03-30T02:49:19.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:19.024 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-30T02:49:19.024 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-30T02:49:19.024 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.11 2024-03-30T02:49:19.438 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 systemd[1]: Stopping Ceph osd.11 for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:19.439 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.252+0000 7f9e96070700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:19.439 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.252+0000 7f9e96070700 -1 osd.11 100 *** Got signal Terminated *** 2024-03-30T02:49:19.439 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.252+0000 7f9e96070700 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-30T02:49:19.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.455+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:19.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.455+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:19.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.455+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:19.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.455+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:19.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.455+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:19.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.455+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:19.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.455+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:19.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:19 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:19.455+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-03-30T02:48:50.994297+0000 front 2024-03-30T02:48:50.994207+0000 (oldest deadline 2024-03-30T02:49:16.294022+0000) 2024-03-30T02:49:20.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:20.462+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:20.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:20.462+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:20.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:20.462+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:20.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:20.462+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:20.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:20.462+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:20.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:20.462+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:20.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:20.462+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:20.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:20 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:20.462+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-03-30T02:48:50.994297+0000 front 2024-03-30T02:48:50.994207+0000 (oldest deadline 2024-03-30T02:49:16.294022+0000) 2024-03-30T02:49:21.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:21.506+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:21.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:21.506+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:21.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:21.506+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:21.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:21.506+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:21.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:21.506+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:21.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:21.506+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:21.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:21.506+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:21.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:21 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:21.506+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-03-30T02:48:50.994297+0000 front 2024-03-30T02:48:50.994207+0000 (oldest deadline 2024-03-30T02:49:16.294022+0000) 2024-03-30T02:49:22.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:22.547+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:22.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:22.547+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:22.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:22.547+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:22.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:22.547+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:22.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:22.547+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:22.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:22.547+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:22.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:22.547+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:22.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:22 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:22.547+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-03-30T02:48:50.994297+0000 front 2024-03-30T02:48:50.994207+0000 (oldest deadline 2024-03-30T02:49:16.294022+0000) 2024-03-30T02:49:23.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:23.507+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-03-30T02:47:58.887147+0000 front 2024-03-30T02:47:58.886994+0000 (oldest deadline 2024-03-30T02:48:22.986873+0000) 2024-03-30T02:49:23.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:23.507+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6804 osd.1 since back 2024-03-30T02:48:08.188618+0000 front 2024-03-30T02:48:08.188504+0000 (oldest deadline 2024-03-30T02:48:33.488543+0000) 2024-03-30T02:49:23.938 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:23.507+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-03-30T02:48:15.789465+0000 front 2024-03-30T02:48:15.789414+0000 (oldest deadline 2024-03-30T02:48:39.889601+0000) 2024-03-30T02:49:23.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:23.507+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-03-30T02:48:19.890039+0000 front 2024-03-30T02:48:19.890123+0000 (oldest deadline 2024-03-30T02:48:50.490772+0000) 2024-03-30T02:49:23.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:23.507+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6812 osd.4 since back 2024-03-30T02:48:30.491373+0000 front 2024-03-30T02:48:30.491424+0000 (oldest deadline 2024-03-30T02:48:54.591504+0000) 2024-03-30T02:49:23.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:23.507+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-03-30T02:48:36.292397+0000 front 2024-03-30T02:48:36.292479+0000 (oldest deadline 2024-03-30T02:49:00.992435+0000) 2024-03-30T02:49:23.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:23.507+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-03-30T02:48:46.293595+0000 front 2024-03-30T02:48:46.293541+0000 (oldest deadline 2024-03-30T02:49:10.993396+0000) 2024-03-30T02:49:23.939 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:23 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11[109923]: 2024-03-30T02:49:23.507+0000 7f9e92666700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.82:6820 osd.7 since back 2024-03-30T02:48:50.994297+0000 front 2024-03-30T02:48:50.994207+0000 (oldest deadline 2024-03-30T02:49:16.294022+0000) 2024-03-30T02:49:25.188 INFO:journalctl@ceph.osd.11.smithi170.stdout:Mar 30 02:49:24 smithi170 bash[124383]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-osd-11 2024-03-30T02:49:25.896 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@osd.11.service' 2024-03-30T02:49:25.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:25.935 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-30T02:49:25.935 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-30T02:49:25.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.a 2024-03-30T02:49:26.312 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:49:26 smithi067 systemd[1]: Stopping Ceph mds.a for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:26.662 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:49:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-a[87694]: 2024-03-30T02:49:26.313+0000 7ff858c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:26.663 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:49:26 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-a[87694]: 2024-03-30T02:49:26.313+0000 7ff858c00700 -1 mds.a *** got signal Terminated *** 2024-03-30T02:49:30.662 INFO:journalctl@ceph.mds.a.smithi067.stdout:Mar 30 02:49:30 smithi067 bash[132620]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-a 2024-03-30T02:49:30.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.a.service' 2024-03-30T02:49:30.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:30.805 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-30T02:49:30.806 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-30T02:49:30.806 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.d 2024-03-30T02:49:31.162 INFO:journalctl@ceph.mds.d.smithi067.stdout:Mar 30 02:49:30 smithi067 systemd[1]: Stopping Ceph mds.d for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:31.662 INFO:journalctl@ceph.mds.d.smithi067.stdout:Mar 30 02:49:31 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-d[86768]: 2024-03-30T02:49:31.229+0000 7f71fe000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:31.662 INFO:journalctl@ceph.mds.d.smithi067.stdout:Mar 30 02:49:31 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-d[86768]: 2024-03-30T02:49:31.229+0000 7f71fe000700 -1 mds.d *** got signal Terminated *** 2024-03-30T02:49:34.662 INFO:journalctl@ceph.mds.d.smithi067.stdout:Mar 30 02:49:34 smithi067 bash[132758]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-d 2024-03-30T02:49:34.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.d.service' 2024-03-30T02:49:34.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:34.894 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-30T02:49:34.894 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-30T02:49:34.894 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.g 2024-03-30T02:49:35.162 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:49:34 smithi067 systemd[1]: Stopping Ceph mds.g for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:35.662 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:49:35 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-g[87313]: 2024-03-30T02:49:35.244+0000 7fe73ba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:35.662 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:49:35 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-g[87313]: 2024-03-30T02:49:35.244+0000 7fe73ba00700 -1 mds.g *** got signal Terminated *** 2024-03-30T02:49:38.662 INFO:journalctl@ceph.mds.g.smithi067.stdout:Mar 30 02:49:38 smithi067 bash[132895]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-g 2024-03-30T02:49:38.709 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.g.service' 2024-03-30T02:49:38.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:38.789 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-30T02:49:38.789 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-30T02:49:38.790 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.j 2024-03-30T02:49:39.159 INFO:journalctl@ceph.mds.j.smithi067.stdout:Mar 30 02:49:38 smithi067 systemd[1]: Stopping Ceph mds.j for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:39.412 INFO:journalctl@ceph.mds.j.smithi067.stdout:Mar 30 02:49:39 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-j[86382]: 2024-03-30T02:49:39.160+0000 7fd08d200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:39.412 INFO:journalctl@ceph.mds.j.smithi067.stdout:Mar 30 02:49:39 smithi067 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-j[86382]: 2024-03-30T02:49:39.160+0000 7fd08d200700 -1 mds.j *** got signal Terminated *** 2024-03-30T02:49:43.662 INFO:journalctl@ceph.mds.j.smithi067.stdout:Mar 30 02:49:43 smithi067 bash[133035]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-j 2024-03-30T02:49:43.771 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.j.service' 2024-03-30T02:49:43.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:43.850 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-30T02:49:43.850 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-30T02:49:43.851 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.b 2024-03-30T02:49:44.249 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:49:43 smithi082 systemd[1]: Stopping Ceph mds.b for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:44.249 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:49:44 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: 2024-03-30T02:49:44.097+0000 7f8074077700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:44.249 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:49:44 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b[115842]: 2024-03-30T02:49:44.097+0000 7f8074077700 -1 mds.b *** got signal Terminated *** 2024-03-30T02:49:46.484 INFO:journalctl@ceph.mds.b.smithi082.stdout:Mar 30 02:49:46 smithi082 bash[123108]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-b 2024-03-30T02:49:46.489 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.b.service' 2024-03-30T02:49:46.525 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:46.525 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-30T02:49:46.526 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-30T02:49:46.526 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.e 2024-03-30T02:49:46.748 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:49:46 smithi082 systemd[1]: Stopping Ceph mds.e for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:47.248 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:49:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-e[115157]: 2024-03-30T02:49:46.787+0000 7fca0501b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:47.249 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:49:46 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-e[115157]: 2024-03-30T02:49:46.787+0000 7fca0501b700 -1 mds.e *** got signal Terminated *** 2024-03-30T02:49:50.499 INFO:journalctl@ceph.mds.e.smithi082.stdout:Mar 30 02:49:50 smithi082 bash[123248]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-e 2024-03-30T02:49:50.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.e.service' 2024-03-30T02:49:50.610 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:50.610 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-30T02:49:50.610 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-30T02:49:50.611 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.h 2024-03-30T02:49:50.869 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:49:50 smithi082 systemd[1]: Stopping Ceph mds.h for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:51.248 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:49:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-h[115499]: 2024-03-30T02:49:50.869+0000 7fa93b2ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:51.249 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:49:50 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-h[115499]: 2024-03-30T02:49:50.869+0000 7fa93b2ca700 -1 mds.h *** got signal Terminated *** 2024-03-30T02:49:54.499 INFO:journalctl@ceph.mds.h.smithi082.stdout:Mar 30 02:49:54 smithi082 bash[123387]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-h 2024-03-30T02:49:54.551 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.h.service' 2024-03-30T02:49:54.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:54.585 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-30T02:49:54.585 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-30T02:49:54.585 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.k 2024-03-30T02:49:54.844 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:49:54 smithi082 systemd[1]: Stopping Ceph mds.k for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:55.248 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:49:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-k[114817]: 2024-03-30T02:49:54.843+0000 7fc383e28700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:55.249 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:49:54 smithi082 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-k[114817]: 2024-03-30T02:49:54.843+0000 7fc383e28700 -1 mds.k *** got signal Terminated *** 2024-03-30T02:49:58.498 INFO:journalctl@ceph.mds.k.smithi082.stdout:Mar 30 02:49:58 smithi082 bash[123525]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-k 2024-03-30T02:49:58.555 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.k.service' 2024-03-30T02:49:58.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:49:58.588 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-30T02:49:58.588 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-30T02:49:58.589 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.c 2024-03-30T02:49:58.938 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:49:58 smithi170 systemd[1]: Stopping Ceph mds.c for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:49:58.938 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:49:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-c[116770]: 2024-03-30T02:49:58.836+0000 7ff504a81700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:49:58.939 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:49:58 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-c[116770]: 2024-03-30T02:49:58.836+0000 7ff504a81700 -1 mds.c *** got signal Terminated *** 2024-03-30T02:50:02.438 INFO:journalctl@ceph.mds.c.smithi170.stdout:Mar 30 02:50:02 smithi170 bash[124644]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-c 2024-03-30T02:50:02.476 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.c.service' 2024-03-30T02:50:02.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:50:02.514 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-30T02:50:02.514 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-30T02:50:02.514 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.f 2024-03-30T02:50:02.785 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:50:02 smithi170 systemd[1]: Stopping Ceph mds.f for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:50:03.188 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:50:02 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-f[116430]: 2024-03-30T02:50:02.786+0000 7f6fd92fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:50:03.188 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:50:02 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-f[116430]: 2024-03-30T02:50:02.786+0000 7f6fd92fa700 -1 mds.f *** got signal Terminated *** 2024-03-30T02:50:06.575 INFO:journalctl@ceph.mds.f.smithi170.stdout:Mar 30 02:50:06 smithi170 bash[124784]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-f 2024-03-30T02:50:06.581 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.f.service' 2024-03-30T02:50:06.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:50:06.616 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-30T02:50:06.616 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-30T02:50:06.616 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.i 2024-03-30T02:50:06.893 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:50:06 smithi170 systemd[1]: Stopping Ceph mds.i for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:50:07.188 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:50:06 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-i[117110]: 2024-03-30T02:50:06.893+0000 7fdb43c81700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:50:07.188 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:50:06 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-i[117110]: 2024-03-30T02:50:06.893+0000 7fdb43c81700 -1 mds.i *** got signal Terminated *** 2024-03-30T02:50:10.438 INFO:journalctl@ceph.mds.i.smithi170.stdout:Mar 30 02:50:10 smithi170 bash[124924]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-i 2024-03-30T02:50:10.581 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.i.service' 2024-03-30T02:50:10.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:50:10.616 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-30T02:50:10.616 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-30T02:50:10.616 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.l 2024-03-30T02:50:10.875 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:50:10 smithi170 systemd[1]: Stopping Ceph mds.l for 0ef0748e-ee3d-11ee-b647-cb9ed24678a4... 2024-03-30T02:50:11.188 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:50:10 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-l[116080]: 2024-03-30T02:50:10.876+0000 7fbb9a0c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-30T02:50:11.188 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:50:10 smithi170 ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-l[116080]: 2024-03-30T02:50:10.876+0000 7fbb9a0c6700 -1 mds.l *** got signal Terminated *** 2024-03-30T02:50:14.438 INFO:journalctl@ceph.mds.l.smithi170.stdout:Mar 30 02:50:14 smithi170 bash[125064]: ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4-mds-l 2024-03-30T02:50:14.583 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0ef0748e-ee3d-11ee-b647-cb9ed24678a4@mds.l.service' 2024-03-30T02:50:14.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-30T02:50:14.618 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-30T02:50:14.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-03-30T02:50:14.892 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:50:17.826 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-03-30T02:50:18.518 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:50:19.970 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-03-30T02:50:20.160 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:50:21.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T02:50:21.627 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T02:50:21.663 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-30T02:50:21.693 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-30T02:50:21.695 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/remote/smithi067/crash 2024-03-30T02:50:21.696 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/crash -- . 2024-03-30T02:50:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-03-30T02:50:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2024-03-30T02:50:21.768 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/remote/smithi082/crash 2024-03-30T02:50:21.769 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/crash -- . 2024-03-30T02:50:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-03-30T02:50:21.826 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2024-03-30T02:50:21.827 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/remote/smithi170/crash 2024-03-30T02:50:21.828 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/crash -- . 2024-03-30T02:50:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-03-30T02:50:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2024-03-30T02:50:21.860 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-30T02:50:21.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-30T02:50:21.939 INFO:teuthology.orchestra.run.smithi067.stderr:grep: /var/log/ceph/0ef0748e-ee3d-11ee-b647-cb9ed24678a4/ceph.log: No such file or directory 2024-03-30T02:50:21.945 INFO:tasks.cephadm:Compressing logs... 2024-03-30T02:50:21.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T02:50:21.949 DEBUG:teuthology.orchestra.run.smithi082:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T02:50:21.951 DEBUG:teuthology.orchestra.run.smithi170:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T02:50:21.978 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-30T02:50:22.007 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-30T02:50:22.010 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-30T02:55:22.240 INFO:tasks.cephadm:Archiving logs... 2024-03-30T02:55:22.241 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/remote/smithi067/log 2024-03-30T02:55:22.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T02:55:36.948 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/remote/smithi082/log 2024-03-30T02:55:36.953 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T02:56:13.890 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/remote/smithi170/log 2024-03-30T02:56:13.891 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T02:56:32.672 INFO:tasks.cephadm:Removing cluster... 2024-03-30T02:56:32.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 --force 2024-03-30T02:56:32.941 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:56:33.721 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 --force 2024-03-30T02:56:33.909 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:56:34.411 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 --force 2024-03-30T02:56:34.600 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 0ef0748e-ee3d-11ee-b647-cb9ed24678a4 2024-03-30T02:56:34.996 INFO:tasks.cephadm:Removing cephadm ... 2024-03-30T02:56:34.997 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-30T02:56:35.044 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-30T02:56:35.081 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-30T02:56:35.099 INFO:tasks.cephadm:Teardown complete 2024-03-30T02:56:35.100 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-30T02:56:35.331 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-30T02:56:35.331 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T02:56:35.336 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T02:56:35.339 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T02:56:35.482 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-30T02:56:35.502 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-30T02:56:35.502 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T02:56:35.507 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T02:56:35.510 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T02:56:35.526 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2024-03-30T02:56:35.527 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2024-03-30T02:56:35.530 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T02:56:35.531 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-03-30T02:56:35.531 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 32 +1543us[+1543us] +/- 72ms 2024-03-30T02:56:35.531 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -2651us[-2657us] +/- 74ms 2024-03-30T02:56:35.531 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +1390us[+1385us] +/- 79ms 2024-03-30T02:56:35.531 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:56:35.544 INFO:teuthology.orchestra.run.smithi067.stderr:bash: ntpq: command not found 2024-03-30T02:56:35.552 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T02:56:35.553 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-03-30T02:56:35.553 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 +1466us[+1483us] +/- 72ms 2024-03-30T02:56:35.553 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -2707us[-2689us] +/- 74ms 2024-03-30T02:56:35.553 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +1329us[+1346us] +/- 79ms 2024-03-30T02:56:35.553 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:56:35.555 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-30T02:56:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-03-30T02:56:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 +1458us[+1458us] +/- 72ms 2024-03-30T02:56:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -2706us[-2706us] +/- 74ms 2024-03-30T02:56:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +1312us[+1312us] +/- 79ms 2024-03-30T02:56:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-30T02:56:35.557 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-30T02:56:35.568 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-30T02:56:35.569 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-30T02:56:35.626 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-30T02:56:35.769 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-30T02:56:35.859 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-30T02:56:35.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T02:56:36.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:56:36.002 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T02:56:36.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:56:36.036 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-30T02:56:36.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:56:36.066 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-30T02:56:36.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-30T02:56:36.118 INFO:teuthology.task.internal:Duration was 2667.096647 seconds 2024-03-30T02:56:36.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-30T02:56:36.184 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-30T02:56:36.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T02:56:36.189 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T02:56:36.191 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T02:56:36.238 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T02:56:36.280 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T02:56:36.299 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-30T02:56:36.700 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-30T02:56:36.700 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-03-30T02:56:36.700 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T02:56:36.757 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-03-30T02:56:36.758 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T02:56:36.798 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-03-30T02:56:36.798 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T02:56:36.820 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-30T02:56:36.821 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T02:56:36.824 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T02:56:36.842 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T02:56:36.928 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-30T02:56:36.928 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T02:56:36.932 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T02:56:36.934 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T02:56:37.135 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-30T02:56:37.147 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-30T02:56:37.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T02:56:37.213 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T02:56:37.279 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T02:56:37.307 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-30T02:56:37.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T02:56:37.324 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T02:56:37.326 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T02:56:37.357 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-03-30T02:56:37.374 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-03-30T02:56:37.385 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-03-30T02:56:37.420 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T02:56:37.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:56:37.459 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T02:56:37.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:56:37.476 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T02:56:37.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T02:56:37.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-30T02:56:37.505 INFO:teuthology.task.internal:Transferring archived files... 2024-03-30T02:56:37.506 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/remote/smithi067 2024-03-30T02:56:37.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T02:56:37.590 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/remote/smithi082 2024-03-30T02:56:37.590 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T02:56:37.631 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363/remote/smithi170 2024-03-30T02:56:37.632 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T02:56:37.672 INFO:teuthology.task.internal:Removing archive directory... 2024-03-30T02:56:37.672 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T02:56:37.677 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T02:56:37.679 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T02:56:37.722 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-30T02:56:37.736 INFO:teuthology.task.internal:Not uploading archives. 2024-03-30T02:56:37.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-30T02:56:37.777 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-30T02:56:37.777 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T02:56:37.781 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T02:56:37.784 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T02:56:37.798 INFO:teuthology.orchestra.run.smithi082.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 02:56 /home/ubuntu/cephtest 2024-03-30T02:56:37.800 INFO:teuthology.orchestra.run.smithi170.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 02:56 /home/ubuntu/cephtest 2024-03-30T02:56:37.819 INFO:teuthology.orchestra.run.smithi067.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 02:56 /home/ubuntu/cephtest 2024-03-30T02:56:37.822 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-30T02:56:37.853 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-30T02:56:38.019 INFO:teuthology.nuke:Checking targets against current locks 2024-03-30T02:56:38.046 DEBUG:teuthology.nuke:shortname: smithi067 2024-03-30T02:56:38.046 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T02:56:38.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 02:00:40.399338', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBnV2lYsYPDWdzFnsqSHilP5gFUvEH6tYg2nrtts4HQdB1KD5f8DrYQZgF2G+KyE2hFQ0g4ZTS7B+RWAz4uajsI='} 2024-03-30T02:56:38.073 DEBUG:teuthology.nuke:shortname: smithi082 2024-03-30T02:56:38.074 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T02:56:38.097 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 02:00:40.401104', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAPsmWgWtA+qGMoofhF3cRgrQ8UIrqa4hvA1ltjlGnUPlNPmnDQ3qjKQEYuP1/Fq7A38uPb7hTepkjvXaoNCLpY='} 2024-03-30T02:56:38.102 DEBUG:teuthology.nuke:shortname: smithi170 2024-03-30T02:56:38.103 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T02:56:38.128 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smithi/7629363', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 02:00:40.402510', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH+u33hMnPJ9lV3T4yEUw1/EJlDzs4wChg+GzoMbYiZD04JbXQkOT/VZP0wc9iqvWfz1Xx5Ig68UionwRt9CbrQ='} 2024-03-30T02:56:38.136 INFO:teuthology.orchestra.console.smithi067:Power off 2024-03-30T02:56:38.136 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T02:56:38.160 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2024-03-30T02:56:38.160 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T02:56:38.180 INFO:teuthology.orchestra.console.smithi082:Power off 2024-03-30T02:56:38.181 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T02:56:38.193 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-03-30T02:56:38.204 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-03-30T02:56:38.204 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T02:56:38.220 INFO:teuthology.orchestra.console.smithi170:Power off 2024-03-30T02:56:38.220 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T02:56:38.229 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-03-30T02:56:38.243 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2024-03-30T02:56:38.244 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T02:56:38.268 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-30T02:56:42.195 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T02:56:42.231 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T02:56:42.269 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T02:56:42.306 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-03-30T02:56:42.342 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-03-30T02:56:42.380 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-03-30T02:56:46.309 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T02:56:46.343 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T02:56:46.381 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T02:56:46.420 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2024-03-30T02:56:46.456 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-03-30T02:56:46.494 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-03-30T02:56:46.522 INFO:teuthology.orchestra.console.smithi067:Power off completed 2024-03-30T02:56:46.557 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-03-30T02:56:46.595 INFO:teuthology.orchestra.console.smithi170:Power off completed 2024-03-30T02:56:46.702 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}} duration: 2667.0966472625732 failure_reason: 'Command failed (workunit test kernel_untar_build.sh) on smithi067 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=470100a4ccbcf7769052bf09a5413881ccf1b97f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=27bc5f1402394ff68594dc9c389b3116 status: fail success: false 2024-03-30T02:56:46.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T02:56:46.786 INFO:teuthology.run:FAIL